2024-09-16T23:28:35.214 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T23:28:35.238 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T23:28:35.326 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880 branch: squid description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7907880' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: e20bff7d16f5b09c58ead9548e8cf98cf87ee227 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 599 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 33/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: e20bff7d16f5b09c58ead9548e8cf98cf87ee227 targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDdlZ9W1gDL5e2c3Bqww6/d9IRhzOO2OpC6QN4o4q36uBt4jpqesM+C+G+MQmOye3VVu+5Fsig0H9JMHQ1xVuU4= smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6dA+Tt7EdOYmnA/Xjsn5HHayQsAd4hdkksPvOZ+jp7kmGkhKX+RtEZd6UhuGaVcbel2Ew/NZwZ8HI13rIvnM0= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_21:08:04 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-09-16T23:28:35.326 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa; will attempt to use it 2024-09-16T23:28:35.327 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks 2024-09-16T23:28:35.327 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T23:28:35.329 INFO:teuthology.task.internal:Checking packages... 2024-09-16T23:28:35.352 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-16T23:28:35.352 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T23:28:35.352 INFO:teuthology.packaging:ref: None 2024-09-16T23:28:35.352 INFO:teuthology.packaging:tag: None 2024-09-16T23:28:35.352 INFO:teuthology.packaging:branch: squid 2024-09-16T23:28:35.352 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:28:35.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T23:28:35.593 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-0 2024-09-16T23:28:35.594 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T23:28:35.601 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T23:28:35.601 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T23:28:35.608 INFO:teuthology.task.internal:Saving configuration 2024-09-16T23:28:35.639 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T23:28:35.646 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T23:28:35.669 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 23:23:53.508699', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDdlZ9W1gDL5e2c3Bqww6/d9IRhzOO2OpC6QN4o4q36uBt4jpqesM+C+G+MQmOye3VVu+5Fsig0H9JMHQ1xVuU4='} 2024-09-16T23:28:35.688 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 23:23:53.507460', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD6dA+Tt7EdOYmnA/Xjsn5HHayQsAd4hdkksPvOZ+jp7kmGkhKX+RtEZd6UhuGaVcbel2Ew/NZwZ8HI13rIvnM0='} 2024-09-16T23:28:35.688 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T23:28:35.694 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-09-16T23:28:35.694 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-09-16T23:28:35.694 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T23:28:35.778 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f373fd5fac0>, signals=[15]) 2024-09-16T23:28:35.779 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T23:28:35.785 INFO:teuthology.task.internal:Opening connections... 2024-09-16T23:28:35.785 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:28:35.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:28:35.864 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-09-16T23:28:35.865 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:28:35.938 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T23:28:35.944 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-09-16T23:28:35.962 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-09-16T23:28:35.963 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-09-16T23:28:36.021 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-09-16T23:28:36.021 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-09-16T23:28:36.021 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-09-16T23:28:36.021 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-09-16T23:28:36.021 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-09-16T23:28:36.021 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-09-16T23:28:36.021 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T23:28:36.022 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-09-16T23:28:36.022 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-09-16T23:28:36.022 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T23:28:36.022 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-09-16T23:28:36.022 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T23:28:36.022 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T23:28:36.022 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T23:28:36.022 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-09-16T23:28:36.044 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-09-16T23:28:36.062 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-09-16T23:28:36.062 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-09-16T23:28:36.121 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-09-16T23:28:36.121 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-09-16T23:28:36.121 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-09-16T23:28:36.121 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T23:28:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T23:28:36.123 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-09-16T23:28:36.143 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T23:28:36.152 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T23:28:36.158 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T23:28:36.158 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-09-16T23:28:36.160 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-09-16T23:28:36.177 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T23:28:36.183 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T23:28:36.183 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-09-16T23:28:36.217 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-09-16T23:28:36.248 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T23:28:36.295 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T23:28:36.309 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T23:28:36.309 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T23:28:36.310 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T23:28:36.310 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T23:28:36.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T23:28:36.327 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-16T23:28:36.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T23:28:36.328 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-16T23:28:36.385 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-503.el9.x86_64 2024-09-16T23:28:36.385 INFO:teuthology.task.kernel:Running kernel on smithi042: 5.14.0-503.el9.x86_64 2024-09-16T23:28:36.385 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2024-09-16T23:28:36.386 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-503.el9.x86_64 2024-09-16T23:28:36.386 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-503.el9.x86_64 2024-09-16T23:28:36.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-09-16T23:28:38.789 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-16T23:28:41.784 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.3 MB 00:03 2024-09-16T23:28:42.891 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 20 MB 00:02 2024-09-16T23:28:45.495 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 20 MB 00:02 2024-09-16T23:28:47.232 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-16T23:28:48.974 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-16T23:28:50.170 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-16T23:28:50.761 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-16T23:28:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-16T23:28:53.041 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-16T23:28:55.709 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-16T23:28:57.666 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T23:28:57.666 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T23:28:57.754 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:28:57.758 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:28:57.758 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-16T23:28:57.759 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:28:57.760 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 79 M 2024-09-16T23:28:57.760 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 126 M 2024-09-16T23:28:57.760 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:28:57.921 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:57.921 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:57.921 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:57.996 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:58.001 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:58.010 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:58.111 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 155 kB/s | 36 kB 00:00 2024-09-16T23:28:58.132 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:58.154 INFO:teuthology.orchestra.run.smithi043.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-16T23:28:58.572 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-16T23:29:00.537 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T23:29:00.537 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T23:29:00.632 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:29:00.643 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:29:00.643 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:29:00.643 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:29:00.643 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:29:00.643 INFO:teuthology.orchestra.run.smithi042.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-16T23:29:00.643 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:29:00.643 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-16T23:29:00.644 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-16T23:29:00.644 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-16T23:29:00.644 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:29:00.644 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:29:00.644 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:29:00.644 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-09-16T23:29:00.644 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:29:00.645 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 79 M 2024-09-16T23:29:00.645 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 126 M 2024-09-16T23:29:00.645 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:29:00.865 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 403 kB/s | 36 kB 00:00 2024-09-16T23:29:01.249 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-16T23:29:01.699 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 37 MB/s | 35 MB 00:00 2024-09-16T23:29:02.058 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-16T23:29:02.058 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:29:02.058 INFO:teuthology.orchestra.run.smithi042.stdout:Total 56 MB/s | 79 MB 00:01 2024-09-16T23:29:02.767 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:29:03.163 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:29:03.163 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:29:04.176 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:29:04.177 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:29:05.135 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-09-16T23:29:06.824 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:29:07.485 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-16T23:29:07.567 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T23:29:10.316 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T23:29:10.455 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T23:29:11.210 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 2.2 MB/s | 29 MB 00:13 2024-09-16T23:29:14.060 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 2.1 MB/s | 35 MB 00:16 2024-09-16T23:29:14.063 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:29:14.063 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.8 MB/s | 79 MB 00:16 2024-09-16T23:29:14.773 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:29:15.182 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:29:15.182 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:29:15.581 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T23:29:15.623 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:29:16.169 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:29:16.170 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:29:18.716 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:29:19.399 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-16T23:29:19.479 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T23:29:20.785 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:29:22.237 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T23:29:22.372 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T23:29:27.500 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T23:29:27.540 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:29:32.676 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:29:59.785 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:29:59.827 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:30:00.184 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:30:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-16T23:30:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T23:30:00.185 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:30:01.023 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:30:01.368 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T23:30:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T23:30:01.511 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T23:30:01.511 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-16T23:30:01.511 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T23:30:01.511 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T23:30:01.511 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-16T23:30:01.511 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T23:30:01.512 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T23:30:01.570 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T23:30:01.571 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T23:30:01.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 82.el9 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8274670 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T23:30:01.603 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:30:01.604 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T23:30:01.605 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T23:30:01.605 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 5.14.0-508.el9.x86_64 2024-09-16T23:30:01.606 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T23:30:02.282 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-09-16T23:30:03.546 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T23:30:03.575 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-09-16T23:30:03.581 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-09-16T23:30:03.596 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.pgbziiGad9 2024-09-16T23:30:03.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pgbziiGad9 2024-09-16T23:30:03.662 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.pgbziiGad9 2024-09-16T23:30:03.800 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.pgbziiGad9 is 6KB 2024-09-16T23:30:03.849 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.pgbziiGad9 2024-09-16T23:30:03.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T23:30:03.934 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T23:30:03.934 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T23:30:03.934 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T23:30:03.934 INFO:teuthology.orchestra.run.smithi042.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-16T23:30:03.936 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-16T23:30:04.092 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-09-16T23:30:11.649 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:30:11.690 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:30:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:30:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-16T23:30:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-16T23:30:12.057 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:30:12.946 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:30:13.277 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T23:30:13.406 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T23:30:13.407 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T23:30:13.407 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-16T23:30:13.407 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T23:30:13.407 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T23:30:13.407 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-16T23:30:13.407 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T23:30:13.407 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T23:30:13.465 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-16T23:30:13.466 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-16T23:30:13.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 82.el9 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8274670 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T23:30:13.505 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:30:13.506 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T23:30:13.507 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T23:30:13.507 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-508.el9.x86_64 2024-09-16T23:30:13.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T23:30:14.155 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-09-16T23:30:15.457 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T23:30:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-09-16T23:30:15.495 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-09-16T23:30:15.510 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.PN15kiWoP5 2024-09-16T23:30:15.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PN15kiWoP5 2024-09-16T23:30:15.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.PN15kiWoP5 2024-09-16T23:30:15.720 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.PN15kiWoP5 is 6KB 2024-09-16T23:30:15.769 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.PN15kiWoP5 2024-09-16T23:30:15.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T23:30:15.846 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T23:30:15.846 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T23:30:15.846 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T23:30:15.846 INFO:teuthology.orchestra.run.smithi043.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-16T23:30:15.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-16T23:30:15.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-09-16T23:30:34.106 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-16T23:30:34.106 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T23:30:34.107 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:30:46.009 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-16T23:30:46.009 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T23:30:46.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:30:52.605 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-09-16T23:31:01.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T23:31:01.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:31:04.381 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-09-16T23:31:13.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T23:31:13.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:32:01.664 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T23:32:13.405 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T23:32:13.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T23:32:13.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:32:13.982 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-16T23:32:14.369 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T23:32:14.369 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-508.el9.x86_64"... 2024-09-16T23:32:14.370 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-09-16T23:32:14.386 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-508.el9.x86_64 2024-09-16T23:32:14.386 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-16T23:32:14.386 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T23:32:14.387 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T23:32:15.387 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-16T23:32:15.387 DEBUG:teuthology.orchestra.run.smithi042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T23:32:15.470 INFO:teuthology.orchestra.run.smithi042.stdout:ttyS1 2024-09-16T23:32:15.494 DEBUG:teuthology.parallel:result is None 2024-09-16T23:32:25.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T23:32:25.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:32:25.703 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-16T23:32:26.048 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T23:32:26.048 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-508.el9.x86_64"... 2024-09-16T23:32:26.048 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-09-16T23:32:26.065 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-508.el9.x86_64 2024-09-16T23:32:26.065 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-16T23:32:26.065 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T23:32:26.065 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T23:32:27.065 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-16T23:32:27.066 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T23:32:27.140 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-09-16T23:32:27.159 DEBUG:teuthology.parallel:result is None 2024-09-16T23:32:27.159 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T23:32:27.165 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T23:32:27.165 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T23:32:27.168 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T23:32:27.186 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T23:32:27.202 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T23:32:27.239 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T23:32:27.239 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T23:32:27.242 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T23:32:27.271 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T23:32:27.308 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T23:32:27.308 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T23:32:27.313 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T23:32:27.349 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T23:32:27.353 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T23:32:27.360 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T23:32:27.364 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T23:32:27.366 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T23:32:27.373 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T23:32:27.373 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T23:32:27.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T23:32:27.435 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T23:32:27.443 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T23:32:27.444 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T23:32:27.475 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T23:32:27.492 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T23:32:27.577 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T23:32:27.627 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T23:32:27.713 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T23:32:27.756 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:32:27.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T23:32:27.824 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T23:32:27.870 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T23:32:27.925 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T23:32:28.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T23:32:28.055 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:32:28.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T23:32:28.125 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-09-16T23:32:28.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-09-16T23:32:28.188 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T23:32:28.217 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T23:32:28.670 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T23:32:28.677 INFO:teuthology.task.internal:Starting timer... 2024-09-16T23:32:28.677 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T23:32:28.778 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T23:32:28.874 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T23:32:28.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-09-16T23:32:28.938 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-09-16T23:32:28.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-09-16T23:32:28.982 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-09-16T23:32:28.984 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T23:32:28.984 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-09-16T23:32:29.011 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-09-16T23:32:29.012 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-09-16T23:32:29.050 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-09-16T23:32:29.051 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi043.front.sepia.ceph.com': 'permissive'} 2024-09-16T23:32:29.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T23:32:29.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T23:32:29.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T23:32:29.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T23:32:29.121 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T23:32:29.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-09-16T23:32:29.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-09-16T23:32:29.191 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T23:32:29.200 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T23:32:29.211 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T23:32:29.212 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi043.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T23:36:19.600 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi043.front.sepia.ceph.com')] 2024-09-16T23:36:19.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T23:36:19.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:36:19.687 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-09-16T23:36:19.764 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-09-16T23:36:19.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T23:36:19.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T23:36:19.842 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-09-16T23:36:19.921 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-09-16T23:36:19.921 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T23:36:19.931 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T23:36:19.932 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T23:36:19.932 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T23:36:19.935 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T23:36:19.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T23:36:19.963 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T23:36:19.979 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T23:36:19.992 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T23:36:20.005 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-09-16T23:36:20.007 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T23:36:20.020 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-09-16T23:36:20.033 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T23:36:20.033 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-09-16T23:36:20.044 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-09-16T23:36:20.048 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T23:36:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T23:36:20.073 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T23:36:20.107 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-09-16T23:36:20.110 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T23:36:20.110 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-09-16T23:36:20.111 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.111 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.111 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.111 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-16T23:36:20.135 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T23:36:20.136 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-16T23:36:20.136 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.136 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.136 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.136 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T23:36:20.137 INFO:teuthology.run_tasks:Running task install... 2024-09-16T23:36:20.145 DEBUG:teuthology.task.install:project ceph 2024-09-16T23:36:20.145 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-16T23:36:20.145 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-16T23:36:20.145 INFO:teuthology.task.install:Using flavor: default 2024-09-16T23:36:20.150 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T23:36:20.150 INFO:teuthology.task.install:extra packages: [] 2024-09-16T23:36:20.151 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-09-16T23:36:20.151 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-16T23:36:20.151 INFO:teuthology.packaging:ref: None 2024-09-16T23:36:20.151 INFO:teuthology.packaging:tag: v18.2.1 2024-09-16T23:36:20.151 INFO:teuthology.packaging:branch: None 2024-09-16T23:36:20.151 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:36:20.705 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-09-16T23:36:20.705 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-09-16T23:36:20.707 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-09-16T23:36:20.707 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-16T23:36:20.707 INFO:teuthology.packaging:ref: None 2024-09-16T23:36:20.707 INFO:teuthology.packaging:tag: v18.2.1 2024-09-16T23:36:20.707 INFO:teuthology.packaging:branch: None 2024-09-16T23:36:20.707 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:36:20.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-09-16T23:36:20.853 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-09-16T23:36:20.853 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-09-16T23:36:20.855 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-09-16T23:36:20.855 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-09-16T23:36:20.897 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T23:36:20.897 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:36:20.897 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T23:36:20.902 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T23:36:20.902 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:36:20.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T23:36:20.930 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T23:36:20.930 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-16T23:36:20.931 INFO:teuthology.packaging:ref: None 2024-09-16T23:36:20.931 INFO:teuthology.packaging:tag: v18.2.1 2024-09-16T23:36:20.931 INFO:teuthology.packaging:branch: None 2024-09-16T23:36:20.931 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:36:20.931 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T23:36:20.933 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T23:36:20.933 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-16T23:36:20.933 INFO:teuthology.packaging:ref: None 2024-09-16T23:36:20.933 INFO:teuthology.packaging:tag: v18.2.1 2024-09-16T23:36:20.933 INFO:teuthology.packaging:branch: None 2024-09-16T23:36:20.933 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:36:20.933 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T23:36:21.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T23:36:21.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T23:36:21.083 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T23:36:21.086 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T23:36:21.150 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-09-16T23:36:21.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-09-16T23:36:21.155 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-09-16T23:36:21.157 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-09-16T23:36:21.394 INFO:teuthology.orchestra.run.smithi043.stdout:45 files removed 2024-09-16T23:36:21.404 INFO:teuthology.orchestra.run.smithi042.stdout:45 files removed 2024-09-16T23:36:21.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-09-16T23:36:21.431 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw 2024-09-16T23:36:21.894 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 445 kB/s | 76 kB 00:00 2024-09-16T23:36:21.895 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 469 kB/s | 76 kB 00:00 2024-09-16T23:36:22.053 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 85 kB/s | 9.4 kB 00:00 2024-09-16T23:36:22.078 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 69 kB/s | 9.4 kB 00:00 2024-09-16T23:36:22.221 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 17 kB/s | 2.2 kB 00:00 2024-09-16T23:36:22.254 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 16 kB/s | 2.2 kB 00:00 2024-09-16T23:36:23.323 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.3 MB 00:01 2024-09-16T23:36:23.331 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.3 MB 00:01 2024-09-16T23:36:25.708 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-16T23:36:27.839 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 20 MB 00:03 2024-09-16T23:36:30.316 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.5 MB 00:00 2024-09-16T23:36:32.075 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-16T23:36:32.913 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T23:36:33.583 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-16T23:36:36.099 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-16T23:36:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-16T23:36:38.785 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-16T23:36:41.164 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:36:41.177 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T23:36:41.178 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T23:36:41.179 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T23:36:41.180 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout:Install 28 Packages 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 2 Packages 2024-09-16T23:36:41.181 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:36:41.182 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 53 M 2024-09-16T23:36:41.182 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:36:42.278 INFO:teuthology.orchestra.run.smithi042.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 6.0 MB/s | 5.2 MB 00:00 2024-09-16T23:36:42.378 INFO:teuthology.orchestra.run.smithi042.stdout:(2/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 7.9 MB/s | 7.7 MB 00:00 2024-09-16T23:36:42.395 INFO:teuthology.orchestra.run.smithi042.stdout:(3/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 206 kB/s | 24 kB 00:00 2024-09-16T23:36:42.437 INFO:teuthology.orchestra.run.smithi042.stdout:(4/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 11 MB/s | 658 kB 00:00 2024-09-16T23:36:42.487 INFO:teuthology.orchestra.run.smithi042.stdout:(5/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 5.0 MB/s | 474 kB 00:00 2024-09-16T23:36:42.688 INFO:teuthology.orchestra.run.smithi042.stdout:(6/30): ceph-common-18.2.1-0.el9.x86_64.rpm 14 MB/s | 18 MB 00:01 2024-09-16T23:36:42.713 INFO:teuthology.orchestra.run.smithi042.stdout:(7/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 199 kB/s | 45 kB 00:00 2024-09-16T23:36:42.747 INFO:teuthology.orchestra.run.smithi042.stdout:(8/30): python3-ceph-common-18.2.1-0.el9.x86_64 2.1 MB/s | 124 kB 00:00 2024-09-16T23:36:42.789 INFO:teuthology.orchestra.run.smithi042.stdout:(9/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 2.1 MB/s | 161 kB 00:00 2024-09-16T23:36:42.808 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-16T23:36:42.864 INFO:teuthology.orchestra.run.smithi042.stdout:(10/30): librgw2-18.2.1-0.el9.x86_64.rpm 10 MB/s | 4.5 MB 00:00 2024-09-16T23:36:42.889 INFO:teuthology.orchestra.run.smithi042.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 2.9 MB/s | 297 kB 00:00 2024-09-16T23:36:42.914 INFO:teuthology.orchestra.run.smithi042.stdout:(12/30): python3-rados-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 321 kB 00:00 2024-09-16T23:36:42.939 INFO:teuthology.orchestra.run.smithi042.stdout:(13/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 99 kB 00:00 2024-09-16T23:36:43.090 INFO:teuthology.orchestra.run.smithi042.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 301 kB/s | 45 kB 00:00 2024-09-16T23:36:43.124 INFO:teuthology.orchestra.run.smithi042.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 456 kB/s | 107 kB 00:00 2024-09-16T23:36:43.149 INFO:teuthology.orchestra.run.smithi042.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 684 kB/s | 160 kB 00:00 2024-09-16T23:36:43.224 INFO:teuthology.orchestra.run.smithi042.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 564 kB/s | 42 kB 00:00 2024-09-16T23:36:43.249 INFO:teuthology.orchestra.run.smithi042.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-16T23:36:43.308 INFO:teuthology.orchestra.run.smithi042.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-16T23:36:43.342 INFO:teuthology.orchestra.run.smithi042.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 945 kB/s | 111 kB 00:00 2024-09-16T23:36:43.367 INFO:teuthology.orchestra.run.smithi042.stdout:(21/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-16T23:36:43.392 INFO:teuthology.orchestra.run.smithi042.stdout:(22/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 541 kB/s | 26 kB 00:00 2024-09-16T23:36:43.417 INFO:teuthology.orchestra.run.smithi042.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-16T23:36:43.443 INFO:teuthology.orchestra.run.smithi042.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T23:36:43.476 INFO:teuthology.orchestra.run.smithi042.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-16T23:36:43.535 INFO:teuthology.orchestra.run.smithi042.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-16T23:36:43.602 INFO:teuthology.orchestra.run.smithi042.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-16T23:36:43.677 INFO:teuthology.orchestra.run.smithi042.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-16T23:36:43.753 INFO:teuthology.orchestra.run.smithi042.stdout:(29/30): librados2-18.2.1-0.el9.x86_64.rpm 15 MB/s | 3.3 MB 00:00 2024-09-16T23:36:43.803 INFO:teuthology.orchestra.run.smithi042.stdout:(30/30): librbd1-18.2.1-0.el9.x86_64.rpm 15 MB/s | 3.0 MB 00:00 2024-09-16T23:36:43.807 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:36:43.808 INFO:teuthology.orchestra.run.smithi042.stdout:Total 20 MB/s | 53 MB 00:02 2024-09-16T23:36:43.884 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:36:43.924 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:36:43.924 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:36:44.591 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:36:44.591 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:36:45.188 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:36:45.201 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:36:45.201 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T23:36:45.201 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:36:45.201 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-09-16T23:36:45.202 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T23:36:45.203 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-09-16T23:36:45.204 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:36:45.205 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:36:45.206 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:36:45.206 INFO:teuthology.orchestra.run.smithi043.stdout:Install 28 Packages 2024-09-16T23:36:45.206 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-09-16T23:36:45.206 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:36:45.207 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 53 M 2024-09-16T23:36:45.207 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:36:45.502 INFO:teuthology.orchestra.run.smithi043.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 39 MB/s | 5.2 MB 00:00 2024-09-16T23:36:45.527 INFO:teuthology.orchestra.run.smithi043.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 972 kB/s | 24 kB 00:00 2024-09-16T23:36:45.568 INFO:teuthology.orchestra.run.smithi043.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 16 MB/s | 658 kB 00:00 2024-09-16T23:36:45.602 INFO:teuthology.orchestra.run.smithi043.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-09-16T23:36:45.727 INFO:teuthology.orchestra.run.smithi043.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 21 MB/s | 7.7 MB 00:00 2024-09-16T23:36:45.744 INFO:teuthology.orchestra.run.smithi043.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 2.7 MB/s | 45 kB 00:00 2024-09-16T23:36:45.770 INFO:teuthology.orchestra.run.smithi043.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 4.9 MB/s | 124 kB 00:00 2024-09-16T23:36:45.795 INFO:teuthology.orchestra.run.smithi043.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 6.3 MB/s | 161 kB 00:00 2024-09-16T23:36:45.862 INFO:teuthology.orchestra.run.smithi043.stdout:(9/30): librgw2-18.2.1-0.el9.x86_64.rpm 17 MB/s | 4.5 MB 00:00 2024-09-16T23:36:45.887 INFO:teuthology.orchestra.run.smithi043.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 3.4 MB/s | 321 kB 00:00 2024-09-16T23:36:45.912 INFO:teuthology.orchestra.run.smithi043.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-09-16T23:36:45.937 INFO:teuthology.orchestra.run.smithi043.stdout:(12/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 3.9 MB/s | 297 kB 00:00 2024-09-16T23:36:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:36:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T23:36:46.163 INFO:teuthology.orchestra.run.smithi043.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 23 MB/s | 18 MB 00:00 2024-09-16T23:36:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T23:36:46.197 INFO:teuthology.orchestra.run.smithi043.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 376 kB/s | 107 kB 00:00 2024-09-16T23:36:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T23:36:46.222 INFO:teuthology.orchestra.run.smithi043.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 564 kB/s | 160 kB 00:00 2024-09-16T23:36:46.272 INFO:teuthology.orchestra.run.smithi043.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-16T23:36:46.339 INFO:teuthology.orchestra.run.smithi043.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-16T23:36:46.373 INFO:teuthology.orchestra.run.smithi043.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-16T23:36:46.406 INFO:teuthology.orchestra.run.smithi043.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 315 kB/s | 42 kB 00:00 2024-09-16T23:36:46.465 INFO:teuthology.orchestra.run.smithi043.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 879 kB/s | 111 kB 00:00 2024-09-16T23:36:46.524 INFO:teuthology.orchestra.run.smithi043.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 463 kB/s | 26 kB 00:00 2024-09-16T23:36:46.549 INFO:teuthology.orchestra.run.smithi043.stdout:(22/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.8 MB/s | 308 kB 00:00 2024-09-16T23:36:46.574 INFO:teuthology.orchestra.run.smithi043.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 964 kB/s | 49 kB 00:00 2024-09-16T23:36:46.599 INFO:teuthology.orchestra.run.smithi043.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T23:36:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T23:36:46.641 INFO:teuthology.orchestra.run.smithi043.stdout:(25/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-16T23:36:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-09-16T23:36:46.692 INFO:teuthology.orchestra.run.smithi043.stdout:(26/30): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-16T23:36:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-09-16T23:36:46.767 INFO:teuthology.orchestra.run.smithi043.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 44 MB/s | 3.3 MB 00:00 2024-09-16T23:36:46.809 INFO:teuthology.orchestra.run.smithi043.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-09-16T23:36:46.876 INFO:teuthology.orchestra.run.smithi043.stdout:(29/30): librbd1-18.2.1-0.el9.x86_64.rpm 28 MB/s | 3.0 MB 00:00 2024-09-16T23:36:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-09-16T23:36:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-09-16T23:36:46.943 INFO:teuthology.orchestra.run.smithi043.stdout:(30/30): libarrow-9.0.0-11.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-09-16T23:36:46.947 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:36:46.947 INFO:teuthology.orchestra.run.smithi043.stdout:Total 30 MB/s | 53 MB 00:01 2024-09-16T23:36:46.980 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-09-16T23:36:47.019 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:36:47.025 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T23:36:47.058 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:36:47.058 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:36:47.086 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-09-16T23:36:47.155 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-09-16T23:36:47.186 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-09-16T23:36:47.252 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-09-16T23:36:47.364 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T23:36:47.393 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T23:36:47.451 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T23:36:47.483 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T23:36:47.510 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T23:36:47.547 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T23:36:47.580 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T23:36:47.688 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:36:47.689 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:36:47.902 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T23:36:47.950 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-09-16T23:36:48.046 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-09-16T23:36:48.083 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-09-16T23:36:48.125 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T23:36:48.546 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T23:36:48.577 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-09-16T23:36:48.633 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-09-16T23:36:48.712 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-09-16T23:36:48.754 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-09-16T23:36:49.165 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:36:49.240 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T23:36:49.273 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T23:36:49.332 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T23:36:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T23:36:49.788 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-09-16T23:36:49.882 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-09-16T23:36:49.963 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-09-16T23:36:49.990 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-09-16T23:36:50.070 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-09-16T23:36:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T23:36:50.159 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-09-16T23:36:50.238 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-09-16T23:36:50.267 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-09-16T23:36:50.334 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-09-16T23:36:50.444 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T23:36:50.483 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T23:36:50.541 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T23:36:50.573 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T23:36:50.600 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T23:36:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T23:36:50.669 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T23:36:50.946 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-09-16T23:36:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T23:36:50.993 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-09-16T23:36:51.040 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-09-16T23:36:51.136 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-09-16T23:36:51.162 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-09-16T23:36:51.207 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T23:36:51.594 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-09-16T23:36:51.629 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T23:36:51.630 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-09-16T23:36:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-09-16T23:36:51.731 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-09-16T23:36:51.732 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-09-16T23:36:51.732 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T23:36:51.732 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T23:36:51.732 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:36:51.764 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-09-16T23:36:51.810 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-09-16T23:36:51.836 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-09-16T23:36:53.943 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-09-16T23:36:53.975 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-09-16T23:36:54.624 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-09-16T23:36:54.661 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-09-16T23:36:54.762 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-09-16T23:36:54.763 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T23:36:54.763 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T23:36:54.763 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:36:54.813 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-09-16T23:37:02.237 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:03.015 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-09-16T23:37:03.042 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-09-16T23:37:03.042 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:03.042 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T23:37:03.042 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T23:37:03.042 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T23:37:03.043 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:03.044 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T23:37:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T23:37:03.135 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T23:37:04.345 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T23:37:04.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-09-16T23:37:04.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-09-16T23:37:04.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T23:37:04.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T23:37:04.347 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-09-16T23:37:05.250 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-09-16T23:37:05.251 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:05.497 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.498 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T23:37:05.499 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T23:37:05.500 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T23:37:05.501 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:05.501 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:05.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-test 2024-09-16T23:37:06.028 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-09-16T23:37:06.055 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-09-16T23:37:06.055 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:06.055 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T23:37:06.055 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T23:37:06.055 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T23:37:06.055 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:06.057 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T23:37:06.147 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T23:37:06.148 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T23:37:06.447 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:28 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:06.592 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:06.592 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:06.592 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:37:06.592 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:06.592 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout:Install 4 Packages 2024-09-16T23:37:06.593 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:06.594 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 40 M 2024-09-16T23:37:06.594 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 166 M 2024-09-16T23:37:06.594 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-16T23:37:06.965 INFO:teuthology.orchestra.run.smithi042.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-16T23:37:06.990 INFO:teuthology.orchestra.run.smithi042.stdout:(3/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 592 kB/s | 165 kB 00:00 2024-09-16T23:37:07.262 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T23:37:07.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-09-16T23:37:07.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-09-16T23:37:07.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T23:37:07.263 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T23:37:07.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-09-16T23:37:07.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T23:37:07.265 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-09-16T23:37:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:(4/4): ceph-test-18.2.1-0.el9.x86_64.rpm 36 MB/s | 40 MB 00:01 2024-09-16T23:37:07.825 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:07.826 INFO:teuthology.orchestra.run.smithi042.stdout:Total 33 MB/s | 40 MB 00:01 2024-09-16T23:37:07.855 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:07.876 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:07.877 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:08.235 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:08.235 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.262 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T23:37:08.263 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:08.264 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:08.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-09-16T23:37:08.727 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:08.823 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T23:37:08.872 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T23:37:08.901 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-09-16T23:37:09.234 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:09.377 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:09.377 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-09-16T23:37:09.378 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:09.379 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 40 M 2024-09-16T23:37:09.379 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 166 M 2024-09-16T23:37:09.379 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:09.578 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): libcephsqlite-18.2.1-0.el9.x86_64.rpm 2.4 MB/s | 165 kB 00:00 2024-09-16T23:37:09.678 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-09-16T23:37:09.695 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 545 kB/s | 64 kB 00:00 2024-09-16T23:37:10.121 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): ceph-test-18.2.1-0.el9.x86_64.rpm 65 MB/s | 40 MB 00:00 2024-09-16T23:37:10.123 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:10.123 INFO:teuthology.orchestra.run.smithi043.stdout:Total 54 MB/s | 40 MB 00:00 2024-09-16T23:37:10.152 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:10.174 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:10.174 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:10.533 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:10.533 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:11.020 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:37:11.090 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T23:37:11.136 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T23:37:11.164 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-09-16T23:37:12.844 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-09-16T23:37:12.919 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-09-16T23:37:14.968 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-09-16T23:37:14.968 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-09-16T23:37:14.968 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-09-16T23:37:14.968 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T23:37:15.140 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 3/4 2024-09-16T23:37:15.223 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-09-16T23:37:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T23:37:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:15.277 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-6.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T23:37:15.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:15.277 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:15.594 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph 2024-09-16T23:37:16.196 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:38 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:16.344 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:16.346 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-16T23:37:16.346 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:16.346 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:16.346 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T23:37:16.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T23:37:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T23:37:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T23:37:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T23:37:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T23:37:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T23:37:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T23:37:16.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T23:37:16.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T23:37:16.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T23:37:16.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T23:37:16.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T23:37:16.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T23:37:16.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T23:37:16.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T23:37:16.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T23:37:16.351 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:16.351 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:16.351 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:16.351 INFO:teuthology.orchestra.run.smithi042.stdout:Install 33 Packages 2024-09-16T23:37:16.351 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:16.351 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 30 M 2024-09-16T23:37:16.351 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 110 M 2024-09-16T23:37:16.352 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:16.709 INFO:teuthology.orchestra.run.smithi042.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 73 kB/s | 6.4 kB 00:00 2024-09-16T23:37:16.784 INFO:teuthology.orchestra.run.smithi042.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 8.9 MB/s | 1.4 MB 00:00 2024-09-16T23:37:16.934 INFO:teuthology.orchestra.run.smithi042.stdout:(3/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-16T23:37:16.976 INFO:teuthology.orchestra.run.smithi042.stdout:(4/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 5.9 MB/s | 2.1 MB 00:00 2024-09-16T23:37:17.018 INFO:teuthology.orchestra.run.smithi042.stdout:(5/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 2.8 MB/s | 242 kB 00:00 2024-09-16T23:37:17.176 INFO:teuthology.orchestra.run.smithi042.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-16T23:37:17.331 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 4/4 2024-09-16T23:37:17.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/4 2024-09-16T23:37:17.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/4 2024-09-16T23:37:17.331 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T23:37:17.485 INFO:teuthology.orchestra.run.smithi042.stdout:(7/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 25 MB/s | 18 MB 00:00 2024-09-16T23:37:17.510 INFO:teuthology.orchestra.run.smithi042.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 319 kB/s | 106 kB 00:00 2024-09-16T23:37:17.560 INFO:teuthology.orchestra.run.smithi042.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-16T23:37:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T23:37:17.629 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:17.629 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:18.2.1-0.el9.x86_64 libcephsqlite-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-6.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T23:37:17.630 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:17.630 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:17.652 INFO:teuthology.orchestra.run.smithi042.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 1.9 MB/s | 1.2 MB 00:00 2024-09-16T23:37:17.678 INFO:teuthology.orchestra.run.smithi042.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-09-16T23:37:17.703 INFO:teuthology.orchestra.run.smithi042.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 649 kB/s | 125 kB 00:00 2024-09-16T23:37:17.720 INFO:teuthology.orchestra.run.smithi042.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 826 kB/s | 35 kB 00:00 2024-09-16T23:37:17.745 INFO:teuthology.orchestra.run.smithi042.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-16T23:37:17.762 INFO:teuthology.orchestra.run.smithi042.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 866 kB/s | 51 kB 00:00 2024-09-16T23:37:17.782 INFO:teuthology.orchestra.run.smithi042.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 669 kB/s | 42 kB 00:00 2024-09-16T23:37:17.833 INFO:teuthology.orchestra.run.smithi042.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 498 kB/s | 43 kB 00:00 2024-09-16T23:37:17.858 INFO:teuthology.orchestra.run.smithi042.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 463 kB/s | 11 kB 00:00 2024-09-16T23:37:17.875 INFO:teuthology.orchestra.run.smithi042.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-16T23:37:17.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-09-16T23:37:17.925 INFO:teuthology.orchestra.run.smithi042.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 264 kB/s | 18 kB 00:00 2024-09-16T23:37:17.950 INFO:teuthology.orchestra.run.smithi042.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 308 kB/s | 23 kB 00:00 2024-09-16T23:37:17.976 INFO:teuthology.orchestra.run.smithi042.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-16T23:37:18.001 INFO:teuthology.orchestra.run.smithi042.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-16T23:37:18.026 INFO:teuthology.orchestra.run.smithi042.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-16T23:37:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-16T23:37:18.077 INFO:teuthology.orchestra.run.smithi042.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-16T23:37:18.102 INFO:teuthology.orchestra.run.smithi042.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-16T23:37:18.127 INFO:teuthology.orchestra.run.smithi042.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T23:37:18.144 INFO:teuthology.orchestra.run.smithi042.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 853 kB/s | 36 kB 00:00 2024-09-16T23:37:18.169 INFO:teuthology.orchestra.run.smithi042.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-16T23:37:18.195 INFO:teuthology.orchestra.run.smithi042.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-16T23:37:18.211 INFO:teuthology.orchestra.run.smithi042.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 478 kB/s | 20 kB 00:00 2024-09-16T23:37:18.245 INFO:teuthology.orchestra.run.smithi042.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-16T23:37:18.249 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:18.249 INFO:teuthology.orchestra.run.smithi042.stdout:Total 16 MB/s | 30 MB 00:01 2024-09-16T23:37:18.368 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:18.409 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:18.409 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:18.531 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:36 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:18.673 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T23:37:18.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T23:37:18.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:18.676 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:18.677 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:18.677 INFO:teuthology.orchestra.run.smithi043.stdout:Install 33 Packages 2024-09-16T23:37:18.677 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:18.677 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 30 M 2024-09-16T23:37:18.677 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 110 M 2024-09-16T23:37:18.677 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:18.833 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:18.833 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:19.146 INFO:teuthology.orchestra.run.smithi043.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 126 kB/s | 6.4 kB 00:00 2024-09-16T23:37:19.196 INFO:teuthology.orchestra.run.smithi043.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-16T23:37:19.258 INFO:teuthology.orchestra.run.smithi043.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 13 MB/s | 2.1 MB 00:00 2024-09-16T23:37:19.280 INFO:teuthology.orchestra.run.smithi043.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 9.5 MB/s | 242 kB 00:00 2024-09-16T23:37:19.356 INFO:teuthology.orchestra.run.smithi043.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-16T23:37:19.590 INFO:teuthology.orchestra.run.smithi043.stdout:(6/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 45 MB/s | 18 MB 00:00 2024-09-16T23:37:19.623 INFO:teuthology.orchestra.run.smithi043.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 740 kB/s | 253 kB 00:00 2024-09-16T23:37:19.673 INFO:teuthology.orchestra.run.smithi043.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 3.9 MB/s | 1.2 MB 00:00 2024-09-16T23:37:19.698 INFO:teuthology.orchestra.run.smithi043.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-09-16T23:37:19.724 INFO:teuthology.orchestra.run.smithi043.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 795 kB/s | 106 kB 00:00 2024-09-16T23:37:19.749 INFO:teuthology.orchestra.run.smithi043.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-09-16T23:37:19.778 INFO:teuthology.orchestra.run.smithi043.stdout:(12/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.7 MB/s | 215 kB 00:00 2024-09-16T23:37:20.036 INFO:teuthology.orchestra.run.smithi043.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 121 kB/s | 35 kB 00:00 2024-09-16T23:37:20.132 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:20.145 INFO:teuthology.orchestra.run.smithi043.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 139 kB/s | 51 kB 00:00 2024-09-16T23:37:20.195 INFO:teuthology.orchestra.run.smithi043.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 870 kB/s | 43 kB 00:00 2024-09-16T23:37:20.207 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T23:37:20.220 INFO:teuthology.orchestra.run.smithi043.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.8 MB/s | 172 kB 00:00 2024-09-16T23:37:20.240 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T23:37:20.245 INFO:teuthology.orchestra.run.smithi043.stdout:(17/33): python3-toml-0.10.2-6.el9.noarch.rpm 199 kB/s | 42 kB 00:00 2024-09-16T23:37:20.279 INFO:teuthology.orchestra.run.smithi043.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 6.0 MB/s | 358 kB 00:00 2024-09-16T23:37:20.281 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T23:37:20.313 INFO:teuthology.orchestra.run.smithi043.stdout:(19/33): python3-mako-1.1.4-6.el9.noarch.rpm 292 kB/s | 172 kB 00:00 2024-09-16T23:37:20.338 INFO:teuthology.orchestra.run.smithi043.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 116 kB/s | 11 kB 00:00 2024-09-16T23:37:20.372 INFO:teuthology.orchestra.run.smithi043.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 196 kB/s | 18 kB 00:00 2024-09-16T23:37:20.377 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T23:37:20.397 INFO:teuthology.orchestra.run.smithi043.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 392 kB/s | 19 kB 00:00 2024-09-16T23:37:20.422 INFO:teuthology.orchestra.run.smithi043.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 410 kB/s | 20 kB 00:00 2024-09-16T23:37:20.456 INFO:teuthology.orchestra.run.smithi043.stdout:(24/33): python3-jaraco-collections-3.0.0-8.el9 162 kB/s | 23 kB 00:00 2024-09-16T23:37:20.463 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T23:37:20.481 INFO:teuthology.orchestra.run.smithi043.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 551 kB/s | 46 kB 00:00 2024-09-16T23:37:20.506 INFO:teuthology.orchestra.run.smithi043.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 938 kB/s | 79 kB 00:00 2024-09-16T23:37:20.509 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T23:37:20.532 INFO:teuthology.orchestra.run.smithi043.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-16T23:37:20.544 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T23:37:20.557 INFO:teuthology.orchestra.run.smithi043.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T23:37:20.573 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T23:37:20.582 INFO:teuthology.orchestra.run.smithi043.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-16T23:37:20.607 INFO:teuthology.orchestra.run.smithi043.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-16T23:37:20.608 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T23:37:20.633 INFO:teuthology.orchestra.run.smithi043.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T23:37:20.641 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T23:37:20.658 INFO:teuthology.orchestra.run.smithi043.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-16T23:37:20.682 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T23:37:20.691 INFO:teuthology.orchestra.run.smithi043.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-16T23:37:20.696 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:20.696 INFO:teuthology.orchestra.run.smithi043.stdout:Total 15 MB/s | 30 MB 00:02 2024-09-16T23:37:20.726 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T23:37:20.759 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T23:37:20.807 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T23:37:20.822 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:20.864 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:20.864 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:20.901 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T23:37:21.061 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T23:37:21.142 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T23:37:21.202 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T23:37:21.249 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T23:37:21.291 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:21.292 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:21.308 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T23:37:21.371 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T23:37:21.570 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T23:37:21.627 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T23:37:21.709 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T23:37:21.824 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T23:37:21.884 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T23:37:22.016 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T23:37:22.186 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-09-16T23:37:22.211 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-09-16T23:37:22.239 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-09-16T23:37:22.239 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:22.239 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T23:37:22.240 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T23:37:22.240 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T23:37:22.240 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:22.682 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:37:22.761 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T23:37:22.803 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T23:37:22.835 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T23:37:22.932 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T23:37:23.143 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T23:37:23.200 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T23:37:23.240 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T23:37:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T23:37:23.313 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T23:37:23.346 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T23:37:23.387 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T23:37:23.431 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T23:37:23.464 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T23:37:23.513 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T23:37:23.606 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T23:37:23.757 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T23:37:23.839 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T23:37:23.845 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-09-16T23:37:23.874 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-09-16T23:37:23.874 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:23.875 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T23:37:23.875 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T23:37:23.875 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T23:37:23.875 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:23.899 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T23:37:23.944 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T23:37:24.003 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T23:37:24.075 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T23:37:24.272 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T23:37:24.323 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T23:37:24.346 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-09-16T23:37:24.371 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-09-16T23:37:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:24.371 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T23:37:24.372 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T23:37:24.372 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T23:37:24.372 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:24.405 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T23:37:24.503 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T23:37:24.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T23:37:24.579 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-09-16T23:37:24.605 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-09-16T23:37:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T23:37:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T23:37:24.605 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T23:37:24.605 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:24.671 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-09-16T23:37:24.680 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T23:37:24.848 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-09-16T23:37:24.882 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-09-16T23:37:24.908 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-09-16T23:37:24.908 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:24.908 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T23:37:24.908 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T23:37:24.908 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T23:37:24.909 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T23:37:26.503 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T23:37:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T23:37:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T23:37:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T23:37:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T23:37:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T23:37:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T23:37:26.504 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T23:37:26.505 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T23:37:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T23:37:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T23:37:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T23:37:26.506 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T23:37:26.525 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-09-16T23:37:26.555 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-09-16T23:37:26.556 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:26.556 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T23:37:26.556 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T23:37:26.556 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T23:37:26.556 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:27.026 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-09-16T23:37:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-09-16T23:37:27.051 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:27.051 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T23:37:27.051 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T23:37:27.051 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T23:37:27.051 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:27.260 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-09-16T23:37:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-09-16T23:37:27.285 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:27.285 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T23:37:27.285 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T23:37:27.285 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T23:37:27.285 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:27.351 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-09-16T23:37:27.369 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T23:37:27.370 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T23:37:27.371 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:27.372 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:27.712 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-base 2024-09-16T23:37:28.323 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:28.416 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:37:28.461 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:28.462 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:37:28.463 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:28.527 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install cephadm 2024-09-16T23:37:29.029 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-09-16T23:37:29.029 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-09-16T23:37:29.029 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-09-16T23:37:29.029 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-09-16T23:37:29.029 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T23:37:29.030 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T23:37:29.031 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T23:37:29.127 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:51 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:29.263 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-16T23:37:29.264 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:29.265 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 221 k 2024-09-16T23:37:29.265 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 213 k 2024-09-16T23:37:29.265 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:29.339 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.9 MB/s | 221 kB 00:00 2024-09-16T23:37:29.340 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:29.340 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.9 MB/s | 221 kB 00:00 2024-09-16T23:37:29.340 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:29.346 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:29.346 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:29.348 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:29.349 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:29.421 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:29.869 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:29.953 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:29.989 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T23:37:29.990 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T23:37:29.991 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T23:37:29.992 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:29.992 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:30.299 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:30.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-09-16T23:37:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:30.541 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-09-16T23:37:30.541 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:30.542 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:30.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T23:37:30.944 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:31.036 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:37:31.079 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:31.080 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:37:31.080 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:31.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-09-16T23:37:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:31.521 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repo Size 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:31.522 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:31.523 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-16T23:37:31.523 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:31.523 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 142 k 2024-09-16T23:37:31.523 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 395 k 2024-09-16T23:37:31.523 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:31.603 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.7 MB/s | 142 kB 00:00 2024-09-16T23:37:31.604 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:31.604 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.7 MB/s | 142 kB 00:00 2024-09-16T23:37:31.604 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:31.612 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:31.612 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:31.672 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:31.672 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:31.748 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:49 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:31.822 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:31.884 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:31.884 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:31.884 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:37:31.884 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 221 k 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 213 k 2024-09-16T23:37:31.885 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:31.887 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:37:31.914 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:37:31.914 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:31.914 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T23:37:31.914 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:31.944 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-18.2.1-0.el9.noarch.rpm 3.7 MB/s | 221 kB 00:00 2024-09-16T23:37:31.945 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:31.945 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.6 MB/s | 221 kB 00:00 2024-09-16T23:37:31.945 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:31.950 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:31.951 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:31.953 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:31.954 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:32.024 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:37:32.458 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:32.541 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:37:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:32.841 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:32.841 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:32.841 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:32.945 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:33.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr 2024-09-16T23:37:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-09-16T23:37:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:33.208 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:33.208 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-09-16T23:37:33.209 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:33.209 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:33.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T23:37:33.659 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:33.750 INFO:teuthology.orchestra.run.smithi042.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:37:33.794 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:33.795 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:37:33.795 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:33.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T23:37:34.002 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:34.139 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:34.140 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:34.140 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 142 k 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 395 k 2024-09-16T23:37:34.141 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:34.191 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.8 MB/s | 142 kB 00:00 2024-09-16T23:37:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.7 MB/s | 142 kB 00:00 2024-09-16T23:37:34.192 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:34.200 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:34.200 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:34.259 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:34.260 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:34.415 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:37:34.462 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:34.492 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:37:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:37:34.518 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:37:34.518 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T23:37:34.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:34.600 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:34.600 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-16T23:37:34.601 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:34.602 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:34.602 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:34.602 INFO:teuthology.orchestra.run.smithi042.stdout:Install 7 Packages 2024-09-16T23:37:34.602 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:34.602 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 2.1 M 2024-09-16T23:37:34.602 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 8.5 M 2024-09-16T23:37:34.602 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:34.881 INFO:teuthology.orchestra.run.smithi042.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 265 kB/s | 23 kB 00:00 2024-09-16T23:37:34.932 INFO:teuthology.orchestra.run.smithi042.stdout:(2/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 13 MB/s | 1.8 MB 00:00 2024-09-16T23:37:34.957 INFO:teuthology.orchestra.run.smithi042.stdout:(3/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 90 kB/s | 15 kB 00:00 2024-09-16T23:37:34.990 INFO:teuthology.orchestra.run.smithi042.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2024-09-16T23:37:35.024 INFO:teuthology.orchestra.run.smithi042.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 455 kB/s | 41 kB 00:00 2024-09-16T23:37:35.049 INFO:teuthology.orchestra.run.smithi042.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 361 kB/s | 31 kB 00:00 2024-09-16T23:37:35.077 INFO:teuthology.orchestra.run.smithi042.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2024-09-16T23:37:35.078 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:35.079 INFO:teuthology.orchestra.run.smithi042.stdout:Total 4.3 MB/s | 2.1 MB 00:00 2024-09-16T23:37:35.079 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:35.087 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:35.087 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:35.174 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:35.175 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:35.475 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:37:35.475 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:35.475 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:35.475 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-09-16T23:37:35.476 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:35.476 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:35.487 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:35.542 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-16T23:37:35.577 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-16T23:37:35.624 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-16T23:37:35.655 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-16T23:37:35.684 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-09-16T23:37:35.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-09-16T23:37:35.933 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-09-16T23:37:35.962 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-09-16T23:37:36.305 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:36.396 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:37:36.440 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:36.441 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:37:36.441 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:36.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T23:37:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-09-16T23:37:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-09-16T23:37:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-09-16T23:37:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-09-16T23:37:36.522 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-16T23:37:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-16T23:37:36.523 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-16T23:37:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-16T23:37:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:36.802 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-09-16T23:37:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-09-16T23:37:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-09-16T23:37:36.802 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-16T23:37:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-16T23:37:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T23:37:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T23:37:36.803 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:36.803 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:37.004 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T23:37:37.113 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:37.250 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:37.251 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T23:37:37.251 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:37.251 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-09-16T23:37:37.251 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:37:37.251 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout:Install 7 Packages 2024-09-16T23:37:37.252 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:37.253 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.1 M 2024-09-16T23:37:37.253 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 8.5 M 2024-09-16T23:37:37.253 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:37.469 INFO:teuthology.orchestra.run.smithi043.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 377 kB/s | 23 kB 00:00 2024-09-16T23:37:37.495 INFO:teuthology.orchestra.run.smithi043.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 169 kB/s | 15 kB 00:00 2024-09-16T23:37:37.537 INFO:teuthology.orchestra.run.smithi043.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-16T23:37:37.562 INFO:teuthology.orchestra.run.smithi043.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-16T23:37:37.587 INFO:teuthology.orchestra.run.smithi043.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 445 kB/s | 41 kB 00:00 2024-09-16T23:37:37.612 INFO:teuthology.orchestra.run.smithi043.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T23:37:37.614 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:59 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:37.654 INFO:teuthology.orchestra.run.smithi043.stdout:(7/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 7.1 MB/s | 1.8 MB 00:00 2024-09-16T23:37:37.657 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:37.658 INFO:teuthology.orchestra.run.smithi043.stdout:Total 5.1 MB/s | 2.1 MB 00:00 2024-09-16T23:37:37.658 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:37.666 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:37.667 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:37.750 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:37.750 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:37.751 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T23:37:37.753 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout:Install 12 Packages 2024-09-16T23:37:37.754 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:37.756 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 38 M 2024-09-16T23:37:37.756 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 212 M 2024-09-16T23:37:37.756 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:38.049 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:37:38.131 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-16T23:37:38.165 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-16T23:37:38.222 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-16T23:37:38.260 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-16T23:37:38.264 INFO:teuthology.orchestra.run.smithi042.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-16T23:37:38.297 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-09-16T23:37:38.306 INFO:teuthology.orchestra.run.smithi042.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-16T23:37:38.373 INFO:teuthology.orchestra.run.smithi042.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-16T23:37:38.474 INFO:teuthology.orchestra.run.smithi042.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.1-0. 21 MB/s | 7.4 MB 00:00 2024-09-16T23:37:38.498 INFO:teuthology.orchestra.run.smithi042.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-09-16T23:37:38.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-09-16T23:37:38.583 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-09-16T23:37:38.641 INFO:teuthology.orchestra.run.smithi042.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.9 MB/s | 3.0 MB 00:00 2024-09-16T23:37:38.666 INFO:teuthology.orchestra.run.smithi042.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 200 kB/s | 38 kB 00:00 2024-09-16T23:37:38.716 INFO:teuthology.orchestra.run.smithi042.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 3.2 MB/s | 245 kB 00:00 2024-09-16T23:37:38.783 INFO:teuthology.orchestra.run.smithi042.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 17 MB/s | 4.9 MB 00:00 2024-09-16T23:37:38.809 INFO:teuthology.orchestra.run.smithi042.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-09-16T23:37:38.958 INFO:teuthology.orchestra.run.smithi042.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-16T23:37:39.115 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-09-16T23:37:39.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-09-16T23:37:39.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-09-16T23:37:39.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-09-16T23:37:39.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-16T23:37:39.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-16T23:37:39.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-16T23:37:39.217 INFO:teuthology.orchestra.run.smithi042.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-16T23:37:39.218 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:39.219 INFO:teuthology.orchestra.run.smithi042.stdout:Total 26 MB/s | 38 MB 00:01 2024-09-16T23:37:39.393 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-16T23:37:39.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:39.393 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:39.393 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-09-16T23:37:39.393 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-09-16T23:37:39.394 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-09-16T23:37:39.394 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-16T23:37:39.394 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-16T23:37:39.394 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T23:37:39.394 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T23:37:39.394 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:39.394 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:39.590 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:39.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T23:37:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:40.139 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:40.139 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:40.214 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:40.353 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:40.354 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:40.354 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T23:37:40.355 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:40.356 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 38 M 2024-09-16T23:37:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 212 M 2024-09-16T23:37:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:40.594 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 52 MB/s | 7.4 MB 00:00 2024-09-16T23:37:40.678 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-09-16T23:37:40.703 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 757 kB/s | 191 kB 00:00 2024-09-16T23:37:40.787 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 219 kB/s | 18 kB 00:00 2024-09-16T23:37:40.829 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 801 kB 00:00 2024-09-16T23:37:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 575 kB/s | 38 kB 00:00 2024-09-16T23:37:40.926 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:40.938 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-16T23:37:41.004 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T23:37:41.058 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T23:37:41.163 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T23:37:41.222 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-16T23:37:41.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T23:37:41.272 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-09-16T23:37:41.347 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 4.5 MB/s | 3.0 MB 00:00 2024-09-16T23:37:41.416 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T23:37:41.449 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T23:37:41.455 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.9 MB/s | 5.1 MB 00:00 2024-09-16T23:37:41.528 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T23:37:41.839 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-16T23:37:41.839 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:41.840 INFO:teuthology.orchestra.run.smithi043.stdout:Total 26 MB/s | 38 MB 00:01 2024-09-16T23:37:42.206 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:42.222 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T23:37:42.233 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:42.233 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:42.342 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T23:37:42.759 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:42.760 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:43.566 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:37:43.652 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T23:37:43.675 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T23:37:43.696 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T23:37:43.802 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T23:37:43.847 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T23:37:44.048 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T23:37:44.079 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T23:37:44.158 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T23:37:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T23:37:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-09-16T23:37:44.844 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T23:37:44.963 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T23:37:46.317 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T23:37:47.422 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T23:37:47.453 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-09-16T23:37:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-09-16T23:37:47.516 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T23:37:47.517 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T23:37:47.915 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T23:37:47.916 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T23:37:47.917 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T23:37:47.917 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T23:37:47.917 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:47.917 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:48.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-rook 2024-09-16T23:37:48.875 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:10 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-16T23:37:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:49.015 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T23:37:49.016 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout:Install 13 Packages 2024-09-16T23:37:49.017 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:49.018 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 2.2 M 2024-09-16T23:37:49.018 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 26 M 2024-09-16T23:37:49.018 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:49.523 INFO:teuthology.orchestra.run.smithi042.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 705 kB/s | 50 kB 00:00 2024-09-16T23:37:49.548 INFO:teuthology.orchestra.run.smithi042.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 266 kB/s | 26 kB 00:00 2024-09-16T23:37:49.574 INFO:teuthology.orchestra.run.smithi042.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 158 kB/s | 19 kB 00:00 2024-09-16T23:37:49.616 INFO:teuthology.orchestra.run.smithi042.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.4 MB/s | 222 kB 00:00 2024-09-16T23:37:49.657 INFO:teuthology.orchestra.run.smithi042.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-16T23:37:49.683 INFO:teuthology.orchestra.run.smithi042.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 801 kB/s | 54 kB 00:00 2024-09-16T23:37:49.725 INFO:teuthology.orchestra.run.smithi042.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.8 MB/s | 279 kB 00:00 2024-09-16T23:37:49.750 INFO:teuthology.orchestra.run.smithi042.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-16T23:37:49.783 INFO:teuthology.orchestra.run.smithi042.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-16T23:37:49.817 INFO:teuthology.orchestra.run.smithi042.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-16T23:37:49.859 INFO:teuthology.orchestra.run.smithi042.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-16T23:37:49.893 INFO:teuthology.orchestra.run.smithi042.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.3 MB/s | 220 kB 00:00 2024-09-16T23:37:49.927 INFO:teuthology.orchestra.run.smithi042.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.0 MB/s | 1.0 MB 00:00 2024-09-16T23:37:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-16T23:37:49.995 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:50.011 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:50.011 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:50.027 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-09-16T23:37:50.027 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T23:37:50.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T23:37:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T23:37:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:50.494 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-09-16T23:37:50.494 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:50.495 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:50.623 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:50.807 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T23:37:50.863 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T23:37:50.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-09-16T23:37:50.918 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T23:37:50.954 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T23:37:50.996 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T23:37:51.102 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T23:37:51.221 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T23:37:51.281 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T23:37:51.519 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:09 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:51.656 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:51.657 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:51.657 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T23:37:51.658 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-09-16T23:37:51.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:51.660 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.2 M 2024-09-16T23:37:51.660 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-09-16T23:37:51.660 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:51.952 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 1.1 MB/s | 50 kB 00:00 2024-09-16T23:37:52.044 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T23:37:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T23:37:52.128 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T23:37:52.185 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T23:37:52.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-09-16T23:37:52.436 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 48 kB/s | 26 kB 00:00 2024-09-16T23:37:52.461 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 34 kB/s | 19 kB 00:00 2024-09-16T23:37:52.695 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 299 kB/s | 222 kB 00:00 2024-09-16T23:37:53.049 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T23:37:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T23:37:53.203 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 207 kB/s | 159 kB 00:00 2024-09-16T23:37:53.245 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 356 kB/s | 279 kB 00:00 2024-09-16T23:37:53.270 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 484 kB/s | 32 kB 00:00 2024-09-16T23:37:53.304 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 89 kB/s | 54 kB 00:00 2024-09-16T23:37:53.329 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 169 kB/s | 14 kB 00:00 2024-09-16T23:37:53.363 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-16T23:37:53.388 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-16T23:37:53.391 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T23:37:53.391 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:53.391 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:53.391 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-09-16T23:37:53.391 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T23:37:53.391 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:53.392 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:53.413 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-16T23:37:53.455 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.9 MB/s | 1.0 MB 00:00 2024-09-16T23:37:53.460 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:53.460 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-09-16T23:37:53.533 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:53.550 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:53.551 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:53.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T23:37:53.649 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:53.649 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:54.179 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:37:54.222 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:16 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T23:37:54.360 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:54.361 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-09-16T23:37:54.361 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:54.361 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-09-16T23:37:54.361 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout:Install 6 Packages 2024-09-16T23:37:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:54.363 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 7.0 M 2024-09-16T23:37:54.363 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 33 M 2024-09-16T23:37:54.363 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:54.409 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T23:37:54.473 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T23:37:54.517 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T23:37:54.559 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T23:37:54.667 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T23:37:54.776 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T23:37:54.828 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T23:37:54.862 INFO:teuthology.orchestra.run.smithi042.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 1.9 MB/s | 132 kB 00:00 2024-09-16T23:37:54.912 INFO:teuthology.orchestra.run.smithi042.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-16T23:37:54.937 INFO:teuthology.orchestra.run.smithi042.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-16T23:37:54.962 INFO:teuthology.orchestra.run.smithi042.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-16T23:37:55.588 INFO:teuthology.orchestra.run.smithi042.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 313 kB/s | 248 kB 00:00 2024-09-16T23:37:55.611 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T23:37:55.665 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T23:37:55.699 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T23:37:55.747 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T23:37:55.776 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-09-16T23:37:55.955 INFO:teuthology.orchestra.run.smithi042.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-16T23:37:55.957 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:55.957 INFO:teuthology.orchestra.run.smithi042.stdout:Total 4.4 MB/s | 7.0 MB 00:01 2024-09-16T23:37:56.033 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:56.047 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:56.047 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:56.153 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:56.154 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:56.471 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:37:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-09-16T23:37:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-09-16T23:37:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T23:37:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T23:37:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T23:37:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T23:37:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T23:37:56.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T23:37:56.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T23:37:56.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T23:37:56.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T23:37:56.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T23:37:56.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T23:37:56.645 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T23:37:56.699 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T23:37:56.939 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T23:37:56.940 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T23:37:56.941 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T23:37:56.941 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T23:37:56.941 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T23:37:56.941 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T23:37:56.941 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:56.941 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:37:57.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T23:37:57.255 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T23:37:57.362 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T23:37:57.431 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T23:37:57.460 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-09-16T23:37:57.805 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:15 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:37:57.943 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:57.944 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:37:57.945 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:37:57.945 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-09-16T23:37:57.945 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:37:57.945 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 7.0 M 2024-09-16T23:37:57.945 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-09-16T23:37:57.945 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:37:58.216 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 2.4 MB/s | 132 kB 00:00 2024-09-16T23:37:58.275 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-16T23:37:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-09-16T23:37:58.283 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-09-16T23:37:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T23:37:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T23:37:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T23:37:58.284 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T23:37:58.300 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-16T23:37:58.325 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-16T23:37:58.556 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T23:37:58.556 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:58.557 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:37:58.799 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-fuse 2024-09-16T23:37:58.942 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 319 kB/s | 248 kB 00:00 2024-09-16T23:37:59.411 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:21 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:37:59.417 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-16T23:37:59.420 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:59.421 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.7 MB/s | 7.0 MB 00:01 2024-09-16T23:37:59.497 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:37:59.510 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:37:59.510 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:37:59.550 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:37:59.550 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:59.550 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:37:59.550 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:59.550 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:37:59.550 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout:Install 2 Packages 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:37:59.551 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 919 k 2024-09-16T23:37:59.552 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 2.7 M 2024-09-16T23:37:59.552 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:37:59.616 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:37:59.617 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:37:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 9.1 MB/s | 839 kB 00:00 2024-09-16T23:37:59.715 INFO:teuthology.orchestra.run.smithi042.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 691 kB/s | 80 kB 00:00 2024-09-16T23:37:59.715 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:37:59.715 INFO:teuthology.orchestra.run.smithi042.stdout:Total 5.5 MB/s | 919 kB 00:00 2024-09-16T23:37:59.727 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:37:59.750 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:37:59.750 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:37:59.834 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:37:59.835 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:37:59.946 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:38:00.021 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:38:00.132 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T23:38:00.241 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T23:38:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T23:38:00.241 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-09-16T23:38:00.758 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T23:38:00.850 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T23:38:00.915 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-09-16T23:38:00.915 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-09-16T23:38:00.917 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T23:38:00.947 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-09-16T23:38:01.192 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T23:38:01.193 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:01.193 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:38:01.193 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T23:38:01.193 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:01.193 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:01.496 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados-devel 2024-09-16T23:38:01.672 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-09-16T23:38:01.672 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-09-16T23:38:01.672 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T23:38:01.672 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T23:38:01.673 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T23:38:01.673 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T23:38:01.938 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T23:38:01.938 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:01.938 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:38:01.938 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-09-16T23:38:01.938 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T23:38:01.938 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T23:38:01.938 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T23:38:01.939 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T23:38:01.939 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T23:38:01.939 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:01.939 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:02.112 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:02.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-09-16T23:38:02.251 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:02.251 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:02.251 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:02.251 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:02.251 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:38:02.251 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-09-16T23:38:02.252 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:02.252 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:38:02.252 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:02.252 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-16T23:38:02.252 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:02.252 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 127 k 2024-09-16T23:38:02.252 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 456 k 2024-09-16T23:38:02.253 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:38:02.320 INFO:teuthology.orchestra.run.smithi042.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 127 kB 00:00 2024-09-16T23:38:02.320 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:02.320 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.8 MB/s | 127 kB 00:00 2024-09-16T23:38:02.321 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:38:02.326 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:38:02.326 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:38:02.392 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:38:02.393 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:38:02.560 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:38:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:02.777 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:20 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:02.915 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T23:38:02.916 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:02.917 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:38:02.917 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:02.917 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-09-16T23:38:02.917 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:02.917 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 919 k 2024-09-16T23:38:02.917 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.7 M 2024-09-16T23:38:02.917 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:38:03.011 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-16T23:38:03.086 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 546 kB/s | 80 kB 00:00 2024-09-16T23:38:03.086 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:03.086 INFO:teuthology.orchestra.run.smithi043.stdout:Total 5.3 MB/s | 919 kB 00:00 2024-09-16T23:38:03.087 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:03.098 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:38:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:38:03.121 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:38:03.205 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:38:03.205 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:38:03.353 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:03.353 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:03.353 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:38:03.353 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:03.353 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:03.354 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:03.397 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:38:03.537 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T23:38:03.613 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-09-16T23:38:03.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs2 2024-09-16T23:38:04.232 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:26 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:04.324 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-09-16T23:38:04.324 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-09-16T23:38:04.328 INFO:teuthology.orchestra.run.smithi042.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:04.373 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:04.374 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:38:04.374 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:04.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install libcephfs-devel 2024-09-16T23:38:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T23:38:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:04.587 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:38:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T23:38:04.587 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:04.587 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:04.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-09-16T23:38:05.050 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:05.189 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 31 k 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 139 k 2024-09-16T23:38:05.190 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:38:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 376 kB/s | 31 kB 00:00 2024-09-16T23:38:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:05.273 INFO:teuthology.orchestra.run.smithi042.stdout:Total 371 kB/s | 31 kB 00:00 2024-09-16T23:38:05.274 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:38:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:38:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:38:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:38:05.301 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:38:05.396 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:38:05.467 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:23 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:05.506 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:05.609 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:05.609 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:05.609 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T23:38:05.610 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:05.611 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 127 k 2024-09-16T23:38:05.611 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 456 k 2024-09-16T23:38:05.611 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:38:05.658 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 2.7 MB/s | 127 kB 00:00 2024-09-16T23:38:05.658 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:05.658 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.6 MB/s | 127 kB 00:00 2024-09-16T23:38:05.658 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:38:05.664 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:38:05.664 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:38:05.731 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:38:05.731 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:38:05.858 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:05.885 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:38:05.991 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:06.097 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:06.098 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:38:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:06.098 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:06.098 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:06.282 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librados2 2024-09-16T23:38:06.375 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:06.628 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:06.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:06.628 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:38:06.628 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:06.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:06.629 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:06.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-09-16T23:38:06.887 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:06.980 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:07.025 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:07.026 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:38:07.026 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:07.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install librbd1 2024-09-16T23:38:07.494 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:25 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:07.587 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:07.635 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:07.636 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:38:07.636 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:07.692 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:07.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-09-16T23:38:07.782 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:07.827 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:07.828 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:38:07.829 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:07.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rados 2024-09-16T23:38:08.307 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:26 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:08.446 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:08.447 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:08.448 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:08.448 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:38:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-09-16T23:38:08.448 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:08.448 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:38:08.449 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:08.449 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T23:38:08.449 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:08.449 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 31 k 2024-09-16T23:38:08.449 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 139 k 2024-09-16T23:38:08.450 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:38:08.503 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 562 kB/s | 31 kB 00:00 2024-09-16T23:38:08.503 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:08.503 INFO:teuthology.orchestra.run.smithi043.stdout:Total 549 kB/s | 31 kB 00:00 2024-09-16T23:38:08.504 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:38:08.504 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:08.507 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:38:08.507 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:38:08.525 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:38:08.525 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:38:08.597 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:08.622 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:38:08.642 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:38:08.643 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:08.701 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rgw 2024-09-16T23:38:08.728 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:09.062 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:09.306 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:38:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:09.306 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:09.306 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:09.309 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:09.402 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:09.448 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:09.449 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:38:09.449 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:09.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-09-16T23:38:09.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-cephfs 2024-09-16T23:38:10.105 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:10.109 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:10.197 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:10.202 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:10.242 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:10.243 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:38:10.243 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:10.247 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:10.249 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:38:10.249 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:10.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-09-16T23:38:10.318 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install python3-rbd 2024-09-16T23:38:10.916 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:10.925 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:11.008 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:11.018 INFO:teuthology.orchestra.run.smithi042.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:11.053 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:11.054 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:38:11.054 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:11.064 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:11.065 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-09-16T23:38:11.065 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:11.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-09-16T23:38:11.126 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-fuse 2024-09-16T23:38:11.728 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:11.732 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:11.821 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:11.867 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:11.868 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:38:11.869 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:11.869 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:11.870 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:11.870 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 86 k 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 226 k 2024-09-16T23:38:11.871 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:38:11.934 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-09-16T23:38:11.943 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-09-16T23:38:11.943 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:11.944 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-09-16T23:38:11.944 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:38:11.952 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:38:11.952 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:38:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:38:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:38:12.170 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:38:12.268 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:12.541 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:12.635 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:12.669 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:12.681 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:12.682 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:38:12.682 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:12.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-09-16T23:38:12.929 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:12.929 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:12.929 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:38:12.930 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:12.930 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:12.930 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:13.189 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-mirror 2024-09-16T23:38:13.353 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:13.445 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:13.492 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:13.493 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:38:13.493 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:13.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-09-16T23:38:13.803 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:13.941 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:13.942 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:13.942 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:13.942 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:38:13.942 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 3.0 M 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 12 M 2024-09-16T23:38:13.943 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:38:14.088 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-09-16T23:38:14.089 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:14.089 INFO:teuthology.orchestra.run.smithi042.stdout:Total 20 MB/s | 3.0 MB 00:00 2024-09-16T23:38:14.089 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:38:14.099 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:38:14.099 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:38:14.164 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:14.183 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:38:14.183 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:38:14.257 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-09-16T23:38:14.302 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:14.303 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-09-16T23:38:14.303 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:14.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-09-16T23:38:14.617 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:38:14.680 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:14.706 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:14.706 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:38:14.707 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T23:38:14.707 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T23:38:14.707 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T23:38:14.707 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:14.978 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:15.114 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:15.115 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:15.115 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:15.115 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:15.115 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:38:15.115 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 86 k 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 226 k 2024-09-16T23:38:15.116 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:38:15.173 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.5 MB/s | 86 kB 00:00 2024-09-16T23:38:15.173 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:15.173 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.5 MB/s | 86 kB 00:00 2024-09-16T23:38:15.173 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:38:15.180 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:38:15.180 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:38:15.247 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:38:15.247 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:38:15.403 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:38:15.497 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:15.751 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:15.751 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:15.751 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:38:15.751 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:15.751 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:15.751 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:15.887 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:16.019 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install rbd-nbd 2024-09-16T23:38:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:16.136 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:38:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:16.136 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:16.136 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:16.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-09-16T23:38:16.633 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 11:36:38 PM UTC. 2024-09-16T23:38:16.771 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-09-16T23:38:16.772 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:16.772 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:16.772 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:16.772 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-09-16T23:38:16.773 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-09-16T23:38:16.773 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:16.773 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-09-16T23:38:16.773 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2024-09-16T23:38:16.773 INFO:teuthology.orchestra.run.smithi042.stdout:Install 1 Package 2024-09-16T23:38:16.773 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:16.774 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 171 k 2024-09-16T23:38:16.774 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 494 k 2024-09-16T23:38:16.774 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-09-16T23:38:16.858 INFO:teuthology.orchestra.run.smithi042.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 1.9 MB/s | 171 kB 00:00 2024-09-16T23:38:16.859 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:16.859 INFO:teuthology.orchestra.run.smithi042.stdout:Total 1.9 MB/s | 171 kB 00:00 2024-09-16T23:38:16.859 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-09-16T23:38:16.867 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-09-16T23:38:16.867 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-09-16T23:38:16.934 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-09-16T23:38:16.934 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-09-16T23:38:17.011 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:17.101 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-09-16T23:38:17.150 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:17.150 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:17.150 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:17.150 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:17.150 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:38:17.150 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-09-16T23:38:17.150 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:17.151 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:38:17.151 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:17.151 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T23:38:17.151 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:17.151 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.0 M 2024-09-16T23:38:17.151 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 12 M 2024-09-16T23:38:17.151 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:38:17.200 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:17.242 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 33 MB/s | 3.0 MB 00:00 2024-09-16T23:38:17.243 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:17.243 INFO:teuthology.orchestra.run.smithi043.stdout:Total 32 MB/s | 3.0 MB 00:00 2024-09-16T23:38:17.243 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:38:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:38:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:38:17.337 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:38:17.337 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:38:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:17.776 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:38:17.834 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:17.861 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:17.861 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T23:38:17.861 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T23:38:17.861 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T23:38:17.861 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T23:38:17.861 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:17.882 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:17.883 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:17.883 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-09-16T23:38:17.883 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:17.883 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:17.883 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-09-16T23:38:18.105 DEBUG:teuthology.parallel:result is None 2024-09-16T23:38:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:19.057 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:19.057 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:38:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:19.058 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:19.058 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:19.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-09-16T23:38:19.951 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 11:36:42 PM UTC. 2024-09-16T23:38:20.087 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-09-16T23:38:20.088 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-09-16T23:38:20.089 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-09-16T23:38:20.089 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:20.089 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 171 k 2024-09-16T23:38:20.089 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 494 k 2024-09-16T23:38:20.089 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-09-16T23:38:20.137 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 3.5 MB/s | 171 kB 00:00 2024-09-16T23:38:20.138 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:20.138 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.4 MB/s | 171 kB 00:00 2024-09-16T23:38:20.138 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-09-16T23:38:20.146 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-09-16T23:38:20.147 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-09-16T23:38:20.213 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-09-16T23:38:20.214 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-09-16T23:38:20.387 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-09-16T23:38:20.495 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:20.916 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:21.168 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-09-16T23:38:21.169 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:21.169 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-09-16T23:38:21.169 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-09-16T23:38:21.169 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:38:21.169 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-09-16T23:38:21.419 DEBUG:teuthology.parallel:result is None 2024-09-16T23:38:21.420 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-16T23:38:21.420 INFO:teuthology.packaging:ref: None 2024-09-16T23:38:21.420 INFO:teuthology.packaging:tag: v18.2.1 2024-09-16T23:38:21.420 INFO:teuthology.packaging:branch: None 2024-09-16T23:38:21.420 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:38:21.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-09-16T23:38:21.567 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T23:38:21.596 INFO:teuthology.orchestra.run.smithi042.stdout:18.2.1-0.el9 2024-09-16T23:38:21.597 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-09-16T23:38:21.597 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-09-16T23:38:21.599 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-16T23:38:21.599 INFO:teuthology.packaging:ref: None 2024-09-16T23:38:21.599 INFO:teuthology.packaging:tag: v18.2.1 2024-09-16T23:38:21.599 INFO:teuthology.packaging:branch: None 2024-09-16T23:38:21.599 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:38:21.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-09-16T23:38:21.748 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T23:38:21.771 INFO:teuthology.orchestra.run.smithi043.stdout:18.2.1-0.el9 2024-09-16T23:38:21.771 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-09-16T23:38:21.771 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-09-16T23:38:21.774 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T23:38:21.775 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:38:21.775 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T23:38:21.807 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:38:21.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T23:38:21.840 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T23:38:21.840 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:38:21.840 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T23:38:21.876 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T23:38:21.951 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:38:21.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T23:38:21.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T23:38:22.046 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T23:38:22.046 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:38:22.046 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T23:38:22.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T23:38:22.147 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:38:22.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T23:38:22.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T23:38:22.243 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T23:38:22.244 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:38:22.244 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T23:38:22.273 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T23:38:22.342 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:38:22.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T23:38:22.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T23:38:22.436 INFO:teuthology.run_tasks:Running task print... 2024-09-16T23:38:22.444 INFO:teuthology.task.print:**** done install task... 2024-09-16T23:38:22.444 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T23:38:22.618 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-16T23:38:22.618 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-09-16T23:38:22.619 INFO:tasks.cephadm:Cluster fsid is c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:38:22.619 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T23:38:22.620 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-16T23:38:22.620 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi042': '172.21.15.42', 'mon.smithi043': '172.21.15.43'} 2024-09-16T23:38:22.620 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T23:38:22.620 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2024-09-16T23:38:22.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-09-16T23:38:22.680 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-09-16T23:38:22.680 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:38:22.823 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56092/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-330-gc2ddf972', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-330.gc2ddf972'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-16 16:11:17.578068', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T23:38:23.113 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref reef, sha1 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-09-16T23:38:23.115 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm 2024-09-16T23:38:23.115 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm 2024-09-16T23:38:23.115 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T23:38:23.620 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 16 23:38 /home/ubuntu/cephtest/cephadm 2024-09-16T23:38:23.620 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T23:38:24.107 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 217222 Sep 16 23:38 /home/ubuntu/cephtest/cephadm 2024-09-16T23:38:24.107 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T23:38:24.128 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T23:38:24.158 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-09-16T23:38:24.159 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-09-16T23:38:24.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-09-16T23:38:25.712 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-09-16T23:38:25.801 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-09-16T23:38:38.254 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:38:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-09-16T23:38:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-09-16T23:38:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "repo_digests": [ 2024-09-16T23:38:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-09-16T23:38:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-09-16T23:38:38.255 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-09-16T23:38:38.255 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:38:41.705 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T23:38:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-09-16T23:38:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-09-16T23:38:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-09-16T23:38:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-09-16T23:38:41.705 INFO:teuthology.orchestra.run.smithi043.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-09-16T23:38:41.706 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-09-16T23:38:41.706 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T23:38:41.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2024-09-16T23:38:41.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-09-16T23:38:41.795 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2024-09-16T23:38:41.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-09-16T23:38:41.876 INFO:tasks.cephadm:Writing seed config... 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T23:38:41.877 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T23:38:41.878 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T23:38:41.879 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T23:38:41.880 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T23:38:41.880 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T23:38:41.880 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:38:41.880 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T23:38:41.913 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c2e6f2e2-7484-11ef-bceb-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T23:38:41.913 DEBUG:teuthology.orchestra.run.smithi042:mon.smithi042> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service 2024-09-16T23:38:41.956 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T23:38:41.956 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.42 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T23:38:42.121 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-09-16T23:38:42.121 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', 'c2e6f2e2-7484-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.42', '--skip-admin-label'] 2024-09-16T23:38:42.145 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5.2.2 2024-09-16T23:38:42.145 INFO:teuthology.orchestra.run.smithi042.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T23:38:42.146 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying podman|docker is present... 2024-09-16T23:38:42.168 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5.2.2 2024-09-16T23:38:42.168 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying lvm2 is present... 2024-09-16T23:38:42.168 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying time synchronization is in place... 2024-09-16T23:38:42.176 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T23:38:42.176 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T23:38:42.182 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T23:38:42.182 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-09-16T23:38:42.189 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-09-16T23:38:42.196 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-09-16T23:38:42.196 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-09-16T23:38:42.196 INFO:teuthology.orchestra.run.smithi042.stdout:Repeating the final host check... 2024-09-16T23:38:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5.2.2 2024-09-16T23:38:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T23:38:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl is present 2024-09-16T23:38:42.218 INFO:teuthology.orchestra.run.smithi042.stdout:lvcreate is present 2024-09-16T23:38:42.225 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T23:38:42.225 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T23:38:42.231 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T23:38:42.231 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-09-16T23:38:42.239 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-09-16T23:38:42.245 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-09-16T23:38:42.245 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-09-16T23:38:42.245 INFO:teuthology.orchestra.run.smithi042.stdout:Host looks OK 2024-09-16T23:38:42.245 INFO:teuthology.orchestra.run.smithi042.stdout:Cluster fsid: c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:38:42.246 INFO:teuthology.orchestra.run.smithi042.stdout:Acquiring lock 140507649271120 on /run/cephadm/c2e6f2e2-7484-11ef-bceb-c7b262605968.lock 2024-09-16T23:38:42.246 INFO:teuthology.orchestra.run.smithi042.stdout:Lock 140507649271120 acquired on /run/cephadm/c2e6f2e2-7484-11ef-bceb-c7b262605968.lock 2024-09-16T23:38:42.246 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 3300 ... 2024-09-16T23:38:42.246 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 6789 ... 2024-09-16T23:38:42.246 INFO:teuthology.orchestra.run.smithi042.stdout:Base mon IP(s) is [172.21.15.42:3300, 172.21.15.42:6789], mon addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-09-16T23:38:42.250 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.42 metric 100 2024-09-16T23:38:42.250 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2024-09-16T23:38:42.252 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T23:38:42.252 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-16T23:38:42.252 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-09-16T23:38:42.255 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T23:38:42.255 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T23:38:42.255 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T23:38:42.255 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-16T23:38:42.255 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-09-16T23:38:42.255 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T23:38:42.256 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-09-16T23:38:42.256 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-09-16T23:38:42.256 INFO:teuthology.orchestra.run.smithi042.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T23:38:42.256 INFO:teuthology.orchestra.run.smithi042.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T23:38:42.257 INFO:teuthology.orchestra.run.smithi042.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-09-16T23:38:46.770 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-09-16T23:38:46.770 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-09-16T23:38:46.770 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T23:38:46.771 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-09-16T23:38:46.771 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-09-16T23:38:46.771 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-09-16T23:38:46.771 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T23:38:53.056 INFO:teuthology.orchestra.run.smithi042.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-09-16T23:38:53.056 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-09-16T23:38:53.056 INFO:teuthology.orchestra.run.smithi042.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T23:38:54.037 INFO:teuthology.orchestra.run.smithi042.stdout:stat: stdout 167 167 2024-09-16T23:38:54.037 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial keys... 2024-09-16T23:38:55.042 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCOwehmBOZsFhAAs1Lh5ANiPKzLC0ooRMasQg== 2024-09-16T23:38:56.023 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCPwehm4bi1FBAAz+OXpAEMyWbWfE41S4zrgA== 2024-09-16T23:38:56.979 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCQwehmUoUtFBAA0a92QxqnwwFi+UpunHz6sQ== 2024-09-16T23:38:56.980 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial monmap... 2024-09-16T23:38:57.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T23:38:57.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-16T23:38:57.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:38:57.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T23:38:57.870 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool for smithi042 [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T23:38:57.870 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = pacific 2024-09-16T23:38:57.870 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: set fsid to c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:38:57.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T23:38:57.871 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:38:57.871 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mon... 2024-09-16T23:38:59.744 INFO:teuthology.orchestra.run.smithi042.stdout:create mon.smithi042 on 2024-09-16T23:38:59.977 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T23:39:00.135 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T23:39:00.337 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968.target. 2024-09-16T23:39:00.337 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968.target. 2024-09-16T23:39:00.576 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042 2024-09-16T23:39:00.577 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service: Unit ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service not loaded. 2024-09-16T23:39:00.740 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968.target.wants/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service → /etc/systemd/system/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@.service. 2024-09-16T23:39:01.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:01 smithi042 podman[27530]: 2024-09-16 23:39:01.184464377 +0000 UTC m=+0.318760482 container start bc8477a47b3b921b023b79b4db8fe42b90e12d17fe5c89472957c743d49ed26b (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=HEAD, ceph=True, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-09-16T23:39:01.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:01 smithi042 ceph-mon[27548]: mkfs c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:39:01.275 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T23:39:01.275 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-09-16T23:39:01.282 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T23:39:01.282 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-09-16T23:39:01.282 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-09-16T23:39:01.282 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T23:39:01.283 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon to start... 2024-09-16T23:39:01.283 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon... 2024-09-16T23:39:01.514 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:01 smithi042 ceph-mon[27548]: mon.smithi042 is new leader, mons smithi042 in quorum (ranks 0) 2024-09-16T23:39:01.515 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:01 smithi042 bash[27530]: bc8477a47b3b921b023b79b4db8fe42b90e12d17fe5c89472957c743d49ed26b 2024-09-16T23:39:01.515 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:01 smithi042 systemd[1]: Started Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:39:02.393 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:02 smithi042 ceph-mon[27548]: mon.smithi042 is new leader, mons smithi042 in quorum (ranks 0) 2024-09-16T23:39:02.393 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:02 smithi042 ceph-mon[27548]: monmap e1: 1 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T23:39:02.393 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:02 smithi042 ceph-mon[27548]: fsmap 2024-09-16T23:39:02.393 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:02 smithi042 ceph-mon[27548]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T23:39:02.393 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:02 smithi042 ceph-mon[27548]: mgrmap e1: no daemons active 2024-09-16T23:39:02.394 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:02 smithi042 ceph-mon[27548]: from='client.? 172.21.15.42:0/798379356' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T23:39:02.401 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T23:39:02.401 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout id: c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout services: 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi042 (age 0.576535s) 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout data: 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T23:39:02.402 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.719+0000 7fa47189f700 1 Processor -- start 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.720+0000 7fa47189f700 1 -- start start 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.720+0000 7fa47189f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c105e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.720+0000 7fa47189f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1063d0 con 0x7fa46c1059e0 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.720+0000 7fa46affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c105e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.720+0000 7fa46affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c105e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45796/0 (socket says 172.21.15.42:45796) 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.720+0000 7fa46affd700 1 -- 172.21.15.42:0/4026624611 learned_addr learned my addr 172.21.15.42:0/4026624611 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.720+0000 7fa46affd700 1 -- 172.21.15.42:0/4026624611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c106700 con 0x7fa46c1059e0 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.721+0000 7fa46affd700 1 --2- 172.21.15.42:0/4026624611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c105e00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0098a0 tx=0x7fa45c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dd3f91937fa3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.721+0000 7fa469ffb700 1 -- 172.21.15.42:0/4026624611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa45c004180 con 0x7fa46c1059e0 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.721+0000 7fa469ffb700 1 -- 172.21.15.42:0/4026624611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c0042e0 con 0x7fa46c1059e0 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.721+0000 7fa469ffb700 1 -- 172.21.15.42:0/4026624611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa45c0036a0 con 0x7fa46c1059e0 2024-09-16T23:39:02.403 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.721+0000 7fa47189f700 1 -- 172.21.15.42:0/4026624611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 msgr2=0x7fa46c105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:02.404 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.721+0000 7fa47189f700 1 --2- 172.21.15.42:0/4026624611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c105e00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0098a0 tx=0x7fa45c009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:02.404 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.722+0000 7fa47189f700 1 -- 172.21.15.42:0/4026624611 shutdown_connections 2024-09-16T23:39:02.404 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.722+0000 7fa47189f700 1 --2- 172.21.15.42:0/4026624611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c105e00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:02.404 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.722+0000 7fa47189f700 1 -- 172.21.15.42:0/4026624611 >> 172.21.15.42:0/4026624611 conn(0x7fa46c100220 msgr2=0x7fa46c102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:02.404 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.722+0000 7fa47189f700 1 -- 172.21.15.42:0/4026624611 shutdown_connections 2024-09-16T23:39:02.404 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.722+0000 7fa47189f700 1 -- 172.21.15.42:0/4026624611 wait complete. 2024-09-16T23:39:02.404 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.722+0000 7fa47189f700 1 Processor -- start 2024-09-16T23:39:02.405 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa47189f700 1 -- start start 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa47189f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa47189f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c19a000 con 0x7fa46c1059e0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa46affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa46affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45806/0 (socket says 172.21.15.42:45806) 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa46affd700 1 -- 172.21.15.42:0/798379356 learned_addr learned my addr 172.21.15.42:0/798379356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa46affd700 1 -- 172.21.15.42:0/798379356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c009550 con 0x7fa46c1059e0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa46affd700 1 --2- 172.21.15.42:0/798379356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c199ac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa46c073ad0 tx=0x7fa45c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa47089d700 1 -- 172.21.15.42:0/798379356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa45c003ce0 con 0x7fa46c1059e0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa47089d700 1 -- 172.21.15.42:0/798379356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45c020070 con 0x7fa46c1059e0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c19a200 con 0x7fa46c1059e0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.723+0000 7fa47089d700 1 -- 172.21.15.42:0/798379356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa45c011630 con 0x7fa46c1059e0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.724+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c19a620 con 0x7fa46c1059e0 2024-09-16T23:39:02.406 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.724+0000 7fa47089d700 1 -- 172.21.15.42:0/798379356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa45c022030 con 0x7fa46c1059e0 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.724+0000 7fa47089d700 1 -- 172.21.15.42:0/798379356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa45c025d00 con 0x7fa46c1059e0 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.724+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46c1933a0 con 0x7fa46c1059e0 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.727+0000 7fa47089d700 1 -- 172.21.15.42:0/798379356 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fa45c025480 con 0x7fa46c1059e0 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.799+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa46c02cc70 con 0x7fa46c1059e0 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.799+0000 7fa47089d700 1 -- 172.21.15.42:0/798379356 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fa45c011c20 con 0x7fa46c1059e0 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.800+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 msgr2=0x7fa46c199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.800+0000 7fa47189f700 1 --2- 172.21.15.42:0/798379356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c199ac0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa46c073ad0 tx=0x7fa45c004750 comp rx=0 tx=0).stop 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.800+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 shutdown_connections 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.800+0000 7fa47189f700 1 --2- 172.21.15.42:0/798379356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa46c1059e0 0x7fa46c199ac0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.800+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 >> 172.21.15.42:0/798379356 conn(0x7fa46c100220 msgr2=0x7fa46c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.800+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 shutdown_connections 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:01.800+0000 7fa47189f700 1 -- 172.21.15.42:0/798379356 wait complete. 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:mon is available 2024-09-16T23:39:02.407 INFO:teuthology.orchestra.run.smithi042.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T23:39:03.477 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:03 smithi042 ceph-mon[27548]: from='client.? 172.21.15.42:0/3180707856' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T23:39:03.477 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:03 smithi042 ceph-mon[27548]: from='client.? 172.21.15.42:0/3180707856' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T23:39:03.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:03.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T23:39:03.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:39:03.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T23:39:03.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.847+0000 7fd780951700 1 Processor -- start 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.847+0000 7fd780951700 1 -- start start 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.847+0000 7fd780951700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.847+0000 7fd780951700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c07a8e0 con 0x7fd77c07beb0 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.847+0000 7fd77a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.847+0000 7fd77a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53366/0 (socket says 172.21.15.42:53366) 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.847+0000 7fd77a59c700 1 -- 172.21.15.42:0/3082026101 learned_addr learned my addr 172.21.15.42:0/3082026101 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd77a59c700 1 -- 172.21.15.42:0/3082026101 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c07aa20 con 0x7fd77c07beb0 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd77a59c700 1 --2- 172.21.15.42:0/3082026101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c07a310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0098a0 tx=0x7fd76c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57afd67c53cf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd77959a700 1 -- 172.21.15.42:0/3082026101 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76c004180 con 0x7fd77c07beb0 2024-09-16T23:39:03.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd77959a700 1 -- 172.21.15.42:0/3082026101 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c0042e0 con 0x7fd77c07beb0 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd77959a700 1 -- 172.21.15.42:0/3082026101 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76c0036a0 con 0x7fd77c07beb0 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd780951700 1 -- 172.21.15.42:0/3082026101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 msgr2=0x7fd77c07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd780951700 1 --2- 172.21.15.42:0/3082026101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c07a310 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0098a0 tx=0x7fd76c009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd780951700 1 -- 172.21.15.42:0/3082026101 shutdown_connections 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd780951700 1 --2- 172.21.15.42:0/3082026101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c07a310 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.848+0000 7fd780951700 1 -- 172.21.15.42:0/3082026101 >> 172.21.15.42:0/3082026101 conn(0x7fd77c101bd0 msgr2=0x7fd77c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.849+0000 7fd780951700 1 -- 172.21.15.42:0/3082026101 shutdown_connections 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.849+0000 7fd780951700 1 -- 172.21.15.42:0/3082026101 wait complete. 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.849+0000 7fd780951700 1 Processor -- start 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.849+0000 7fd780951700 1 -- start start 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.849+0000 7fd780951700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.849+0000 7fd780951700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c19c220 con 0x7fd77c07beb0 2024-09-16T23:39:03.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd77a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd77a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53376/0 (socket says 172.21.15.42:53376) 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd77a59c700 1 -- 172.21.15.42:0/3180707856 learned_addr learned my addr 172.21.15.42:0/3180707856 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd77a59c700 1 -- 172.21.15.42:0/3180707856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c009550 con 0x7fd77c07beb0 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd77a59c700 1 --2- 172.21.15.42:0/3180707856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c19bce0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd76c004000 tx=0x7fd76c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76c003ce0 con 0x7fd77c07beb0 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c020070 con 0x7fd77c07beb0 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c19c420 con 0x7fd77c07beb0 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76c011630 con 0x7fd77c07beb0 2024-09-16T23:39:03.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.850+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c19c840 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.851+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd76c022030 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.851+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd76c025d00 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.851+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c195b00 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.854+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fd76c025480 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.923+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd77c0623c0 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.927+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd76c011c20 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.927+0000 7fd76b7fe700 1 -- 172.21.15.42:0/3180707856 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fd77c0623c0 con 0x7fd77c07beb0 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.928+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 msgr2=0x7fd77c19bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.928+0000 7fd780951700 1 --2- 172.21.15.42:0/3180707856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c19bce0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd76c004000 tx=0x7fd76c004750 comp rx=0 tx=0).stop 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.928+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 shutdown_connections 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.928+0000 7fd780951700 1 --2- 172.21.15.42:0/3180707856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd77c07beb0 0x7fd77c19bce0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.928+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 >> 172.21.15.42:0/3180707856 conn(0x7fd77c101bd0 msgr2=0x7fd77c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:03.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.928+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 shutdown_connections 2024-09-16T23:39:03.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:02.928+0000 7fd780951700 1 -- 172.21.15.42:0/3180707856 wait complete. 2024-09-16T23:39:03.489 INFO:teuthology.orchestra.run.smithi042.stdout:Generating new minimal ceph.conf... 2024-09-16T23:39:04.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:04 smithi042 ceph-mon[27548]: from='client.? 172.21.15.42:0/3942927852' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:39:04.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a32e3b700 1 Processor -- start 2024-09-16T23:39:04.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a32e3b700 1 -- start start 2024-09-16T23:39:04.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a32e3b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:04.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a32e3b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c105ec0 con 0x7f5a2c1054d0 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a30bd7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a30bd7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1058f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53388/0 (socket says 172.21.15.42:53388) 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a30bd7700 1 -- 172.21.15.42:0/781342983 learned_addr learned my addr 172.21.15.42:0/781342983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a30bd7700 1 -- 172.21.15.42:0/781342983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c1066d0 con 0x7f5a2c1054d0 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.952+0000 7f5a30bd7700 1 --2- 172.21.15.42:0/781342983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1058f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0098a0 tx=0x7f5a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c49e9d4003d9b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/781342983 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a1c004180 con 0x7f5a2c1054d0 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/781342983 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a1c0042e0 con 0x7f5a2c1054d0 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a2b7fe700 1 -- 172.21.15.42:0/781342983 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a1c011a50 con 0x7f5a2c1054d0 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a32e3b700 1 -- 172.21.15.42:0/781342983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 msgr2=0x7f5a2c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a32e3b700 1 --2- 172.21.15.42:0/781342983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1058f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0098a0 tx=0x7f5a1c009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a32e3b700 1 -- 172.21.15.42:0/781342983 shutdown_connections 2024-09-16T23:39:04.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a32e3b700 1 --2- 172.21.15.42:0/781342983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1058f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a32e3b700 1 -- 172.21.15.42:0/781342983 >> 172.21.15.42:0/781342983 conn(0x7f5a2c100ad0 msgr2=0x7f5a2c102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a32e3b700 1 -- 172.21.15.42:0/781342983 shutdown_connections 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.953+0000 7f5a32e3b700 1 -- 172.21.15.42:0/781342983 wait complete. 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.954+0000 7f5a32e3b700 1 Processor -- start 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.954+0000 7f5a32e3b700 1 -- start start 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.954+0000 7f5a32e3b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.954+0000 7f5a32e3b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c1996f0 con 0x7f5a2c1054d0 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.954+0000 7f5a30bd7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.954+0000 7f5a30bd7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53392/0 (socket says 172.21.15.42:53392) 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.954+0000 7f5a30bd7700 1 -- 172.21.15.42:0/3942927852 learned_addr learned my addr 172.21.15.42:0/3942927852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a30bd7700 1 -- 172.21.15.42:0/3942927852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c009550 con 0x7f5a2c1054d0 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a30bd7700 1 --2- 172.21.15.42:0/3942927852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1991b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006af0 tx=0x7f5a1c003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:04.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a29ffb700 1 -- 172.21.15.42:0/3942927852 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a1c004140 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a29ffb700 1 -- 172.21.15.42:0/3942927852 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5a1c020070 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a29ffb700 1 -- 172.21.15.42:0/3942927852 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a1c01c770 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c1998f0 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c199d10 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a29ffb700 1 -- 172.21.15.42:0/3942927852 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5a1c022030 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.955+0000 7f5a29ffb700 1 -- 172.21.15.42:0/3942927852 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5a1c025770 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.956+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c0623c0 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:03.958+0000 7f5a29ffb700 1 -- 172.21.15.42:0/3942927852 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f5a1c01ce00 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.028+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5a2c079640 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.028+0000 7f5a29ffb700 1 -- 172.21.15.42:0/3942927852 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f5a1c01bb50 con 0x7f5a2c1054d0 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.029+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 msgr2=0x7f5a2c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.029+0000 7f5a32e3b700 1 --2- 172.21.15.42:0/3942927852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1991b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006af0 tx=0x7f5a1c003980 comp rx=0 tx=0).stop 2024-09-16T23:39:04.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.029+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 shutdown_connections 2024-09-16T23:39:04.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.029+0000 7f5a32e3b700 1 --2- 172.21.15.42:0/3942927852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5a2c1054d0 0x7f5a2c1991b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:04.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.029+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 >> 172.21.15.42:0/3942927852 conn(0x7f5a2c100ad0 msgr2=0x7f5a2c102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:04.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.029+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 shutdown_connections 2024-09-16T23:39:04.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:04.029+0000 7f5a32e3b700 1 -- 172.21.15.42:0/3942927852 wait complete. 2024-09-16T23:39:04.575 INFO:teuthology.orchestra.run.smithi042.stdout:Restarting the monitor... 2024-09-16T23:39:04.844 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:04 smithi042 systemd[1]: Stopping Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:39:05.243 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:04 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[27544]: 2024-09-16T23:39:04.846+0000 7f409c85e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi042 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:39:05.243 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:04 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[27544]: 2024-09-16T23:39:04.846+0000 7f409c85e700 -1 mon.smithi042@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T23:39:05.243 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 podman[27792]: 2024-09-16 23:39:05.00024898 +0000 UTC m=+0.260515051 container died bc8477a47b3b921b023b79b4db8fe42b90e12d17fe5c89472957c743d49ed26b (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, io.buildah.version=1.29.1) 2024-09-16T23:39:05.518 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 podman[27792]: 2024-09-16 23:39:05.345810215 +0000 UTC m=+0.606076277 container cleanup bc8477a47b3b921b023b79b4db8fe42b90e12d17fe5c89472957c743d49ed26b (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-09-16T23:39:05.519 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 bash[27792]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042 2024-09-16T23:39:05.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 podman[27804]: 2024-09-16 23:39:05.521920949 +0000 UTC m=+0.517190270 container remove bc8477a47b3b921b023b79b4db8fe42b90e12d17fe5c89472957c743d49ed26b (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, RELEASE=HEAD) 2024-09-16T23:39:05.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service: Deactivated successfully. 2024-09-16T23:39:05.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 systemd[1]: Stopped Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:39:05.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 systemd[1]: Starting Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:39:06.205 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 podman[27896]: 2024-09-16 23:39:05.8485426 +0000 UTC m=+0.016508827 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-09-16T23:39:06.205 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:05 smithi042 podman[27896]: 2024-09-16 23:39:05.947649098 +0000 UTC m=+0.115615344 container create 37006d2f61124c3b00092c5ff50edb0067221d792bab9f4099bdc71b932caa8a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T23:39:06.205 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 podman[27896]: 2024-09-16 23:39:06.142095869 +0000 UTC m=+0.310062100 container init 37006d2f61124c3b00092c5ff50edb0067221d792bab9f4099bdc71b932caa8a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T23:39:06.205 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 podman[27896]: 2024-09-16 23:39:06.14549878 +0000 UTC m=+0.313464991 container start 37006d2f61124c3b00092c5ff50edb0067221d792bab9f4099bdc71b932caa8a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-09-16T23:39:06.205 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T23:39:06.205 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: pidfile_write: ignore empty --pid-file 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: load: jerasure load: lrc 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: RocksDB version: 7.9.2 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Git sha 0 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Compile date 2023-12-11 22:07:34 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: DB SUMMARY 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: DB Session ID: 1ESMM8WFEIUMVDDO2HYZ 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: CURRENT file: CURRENT 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi042/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi042/store.db: 000009.log size: 98861 ; 2024-09-16T23:39:06.206 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.error_if_exists: 0 2024-09-16T23:39:06.207 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.create_if_missing: 0 2024-09-16T23:39:06.207 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.paranoid_checks: 1 2024-09-16T23:39:06.207 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T23:39:06.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T23:39:06.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T23:39:06.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.env: 0x557adcf04720 2024-09-16T23:39:06.208 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.info_log: 0x557adf9e12e0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.statistics: (nil) 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.use_fsync: 0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_log_file_size: 0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.allow_fallocate: 1 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.use_direct_reads: 0 2024-09-16T23:39:06.209 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.db_log_dir: 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.wal_dir: 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.write_buffer_manager: 0x557adec70320 2024-09-16T23:39:06.210 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.unordered_write: 0 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.row_cache: None 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.wal_filter: None 2024-09-16T23:39:06.211 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.two_write_queues: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.wal_compression: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.atomic_flush: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.log_readahead_size: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T23:39:06.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_background_jobs: 2 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_background_compactions: -1 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_subcompactions: 1 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T23:39:06.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_open_files: -1 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_background_flushes: -1 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Compression algorithms supported: 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kZSTD supported: 0 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kXpressCompression supported: 0 2024-09-16T23:39:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kZlibCompression supported: 1 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kSnappyCompression supported: 1 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kLZ4Compression supported: 1 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: kBZip2Compression supported: 0 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000010 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.merge_operator: 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_filter: None 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T23:39:06.215 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T23:39:06.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T23:39:06.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T23:39:06.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557adf9e13e0) 2024-09-16T23:39:06.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: cache_index_and_filter_blocks: 1 2024-09-16T23:39:06.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T23:39:06.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T23:39:06.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: pin_top_level_index_and_filter: 1 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: index_type: 0 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: data_block_index_type: 0 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: index_shortening: 1 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: checksum: 4 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: no_block_cache: 0 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache: 0x557adecf31f0 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache_name: BinnedLRUCache 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache_options: 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: capacity : 536870912 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: num_shard_bits : 4 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: strict_capacity_limit : 0 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: high_pri_pool_ratio: 0.000 2024-09-16T23:39:06.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache_compressed: (nil) 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: persistent_cache: (nil) 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_size: 4096 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_size_deviation: 10 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_restart_interval: 16 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: index_block_restart_interval: 1 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: metadata_block_size: 4096 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: partition_filters: 0 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: use_delta_encoding: 1 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: filter_policy: bloomfilter 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: whole_key_filtering: 1 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: verify_compression: 0 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: read_amp_bytes_per_bit: 0 2024-09-16T23:39:06.218 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: format_version: 5 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: enable_index_compression: 1 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_align: 0 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: max_auto_readahead_size: 262144 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: prepopulate_block_cache: 0 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: initial_auto_readahead_size: 8192 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression: NoCompression 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.num_levels: 7 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T23:39:06.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T23:39:06.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T23:39:06.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T23:39:06.222 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.table_properties_collectors: 2024-09-16T23:39:06.223 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.inplace_update_support: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.bloom_locality: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.max_successive_merges: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.ttl: 2592000 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T23:39:06.224 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.enable_blob_files: false 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.min_blob_size: 0 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T23:39:06.225 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: db6b78af-ad2b-4105-a874-aacbde5ad83e 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726529946175792, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726529946176909, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94107, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92113, "index_size": 276, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13701, "raw_average_key_size": 49, "raw_value_size": 84621, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726529946, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "db6b78af-ad2b-4105-a874-aacbde5ad83e", "db_session_id": "1ESMM8WFEIUMVDDO2HYZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726529946176995, "job": 1, "event": "recovery_finished"} 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi042/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557aded90000 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: DB pointer 0x557aded7c000 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** DB Stats ** 2024-09-16T23:39:06.226 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** Compaction Stats [default] ** 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: L0 2/0 93.77 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 125.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Sum 2/0 93.77 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 125.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 125.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:39:06.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** Compaction Stats [default] ** 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 125.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T23:39:06.228 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative compaction: 0.00 GB write, 27.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval compaction: 0.00 GB write, 27.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Block cache BinnedLRUCache@0x557adecf31f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: starting mon.smithi042 rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi042 fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???) e1 preinit fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???).mds e1 new map 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???).mds e1 print_map 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: e1 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:39:06.229 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: legacy client fscid: -1 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: No filesystems configured 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 bash[27896]: 37006d2f61124c3b00092c5ff50edb0067221d792bab9f4099bdc71b932caa8a 2024-09-16T23:39:06.230 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 systemd[1]: Started Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:39:06.230 INFO:teuthology.orchestra.run.smithi042.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T23:39:06.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mon.smithi042 is new leader, mons smithi042 in quorum (ranks 0) 2024-09-16T23:39:06.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: monmap e1: 1 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T23:39:06.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: fsmap 2024-09-16T23:39:06.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T23:39:06.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:06 smithi042 ceph-mon[27914]: mgrmap e1: no daemons active 2024-09-16T23:39:07.301 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.617+0000 7f578c77f700 1 Processor -- start 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.617+0000 7f578c77f700 1 -- start start 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.618+0000 7f578c77f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f57841069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.618+0000 7f578c77f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5784106fc0 con 0x7f57841065d0 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.618+0000 7f578a51b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f57841069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.618+0000 7f578a51b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f57841069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53396/0 (socket says 172.21.15.42:53396) 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.618+0000 7f578a51b700 1 -- 172.21.15.42:0/2659237025 learned_addr learned my addr 172.21.15.42:0/2659237025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.618+0000 7f578a51b700 1 -- 172.21.15.42:0/2659237025 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57841077d0 con 0x7f57841065d0 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.618+0000 7f578a51b700 1 --2- 172.21.15.42:0/2659237025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f57841069f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57800098a0 tx=0x7f5780009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893ed84554d82b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f5789519700 1 -- 172.21.15.42:0/2659237025 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5780004180 con 0x7f57841065d0 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f5789519700 1 -- 172.21.15.42:0/2659237025 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f57800042e0 con 0x7f57841065d0 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f5789519700 1 -- 172.21.15.42:0/2659237025 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5780011a50 con 0x7f57841065d0 2024-09-16T23:39:07.302 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f578c77f700 1 -- 172.21.15.42:0/2659237025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 msgr2=0x7f57841069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f578c77f700 1 --2- 172.21.15.42:0/2659237025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f57841069f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f57800098a0 tx=0x7f5780009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f578c77f700 1 -- 172.21.15.42:0/2659237025 shutdown_connections 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f578c77f700 1 --2- 172.21.15.42:0/2659237025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f57841069f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f578c77f700 1 -- 172.21.15.42:0/2659237025 >> 172.21.15.42:0/2659237025 conn(0x7f5784101bd0 msgr2=0x7f5784103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f578c77f700 1 -- 172.21.15.42:0/2659237025 shutdown_connections 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.619+0000 7f578c77f700 1 -- 172.21.15.42:0/2659237025 wait complete. 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.620+0000 7f578c77f700 1 Processor -- start 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.620+0000 7f578c77f700 1 -- start start 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.620+0000 7f578c77f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f578419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.620+0000 7f578c77f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578419a830 con 0x7f57841065d0 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.620+0000 7f578a51b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f578419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.620+0000 7f578a51b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f578419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53398/0 (socket says 172.21.15.42:53398) 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.620+0000 7f578a51b700 1 -- 172.21.15.42:0/1720715650 learned_addr learned my addr 172.21.15.42:0/1720715650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:07.303 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f578a51b700 1 -- 172.21.15.42:0/1720715650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780009550 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f578a51b700 1 --2- 172.21.15.42:0/1720715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f578419a2f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5780006af0 tx=0x7f5780011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f577b7fe700 1 -- 172.21.15.42:0/1720715650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5780003b90 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f577b7fe700 1 -- 172.21.15.42:0/1720715650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f578001a8a0 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578419aa30 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f577b7fe700 1 -- 172.21.15.42:0/1720715650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5780019880 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578419ae50 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f577b7fe700 1 -- 172.21.15.42:0/1720715650 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f578002d440 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.621+0000 7f577b7fe700 1 -- 172.21.15.42:0/1720715650 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5780024980 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.622+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57840623c0 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.624+0000 7f577b7fe700 1 -- 172.21.15.42:0/1720715650 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f578001aa10 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.693+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f578410a9f0 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.695+0000 7f577b7fe700 1 -- 172.21.15.42:0/1720715650 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f578001abb0 con 0x7f57841065d0 2024-09-16T23:39:07.304 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.696+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 msgr2=0x7f578419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:07.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.696+0000 7f578c77f700 1 --2- 172.21.15.42:0/1720715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f578419a2f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5780006af0 tx=0x7f5780011a90 comp rx=0 tx=0).stop 2024-09-16T23:39:07.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.696+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 shutdown_connections 2024-09-16T23:39:07.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.696+0000 7f578c77f700 1 --2- 172.21.15.42:0/1720715650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57841065d0 0x7f578419a2f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:07.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.696+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 >> 172.21.15.42:0/1720715650 conn(0x7f5784101bd0 msgr2=0x7f57841032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:07.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.696+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 shutdown_connections 2024-09-16T23:39:07.305 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:06.696+0000 7f578c77f700 1 -- 172.21.15.42:0/1720715650 wait complete. 2024-09-16T23:39:07.305 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T23:39:07.327 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T23:39:07.328 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mgr... 2024-09-16T23:39:07.328 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T23:39:07.329 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T23:39:07.329 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-16T23:39:07.593 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mgr.smithi042.zrkumz 2024-09-16T23:39:07.593 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mgr.smithi042.zrkumz.service: Unit ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mgr.smithi042.zrkumz.service not loaded. 2024-09-16T23:39:07.748 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968.target.wants/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mgr.smithi042.zrkumz.service → /etc/systemd/system/ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@.service. 2024-09-16T23:39:07.978 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:07 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1720715650' entity='client.admin' 2024-09-16T23:39:08.271 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T23:39:08.272 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-09-16T23:39:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T23:39:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-09-16T23:39:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-09-16T23:39:08.279 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T23:39:08.288 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T23:39:08.288 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-09-16T23:39:08.295 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T23:39:08.296 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-09-16T23:39:08.296 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-09-16T23:39:08.296 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-16T23:39:08.296 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr to start... 2024-09-16T23:39:08.296 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr... 2024-09-16T23:39:09.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:09 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2768984556' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T23:39:09.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:09.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:39:09.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "c2e6f2e2-7484-11ef-bceb-c7b262605968", 2024-09-16T23:39:09.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "smithi042" 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T23:39:09.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T23:39:09.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T23:39:09.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:09.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T23:39:01.225035+0000", 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.921+0000 7f81175c4700 1 Processor -- start 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.921+0000 7f81175c4700 1 -- start start 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.921+0000 7f81175c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f8110109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.921+0000 7f81175c4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8110074610 con 0x7f8110107090 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8115360700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f8110109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:09.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8115360700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f8110109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53412/0 (socket says 172.21.15.42:53412) 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8115360700 1 -- 172.21.15.42:0/1245120150 learned_addr learned my addr 172.21.15.42:0/1245120150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8115360700 1 -- 172.21.15.42:0/1245120150 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81101099c0 con 0x7f8110107090 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8115360700 1 --2- 172.21.15.42:0/1245120150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f8110109480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f810c0098a0 tx=0x7f810c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcae1f8824925d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8107fff700 1 -- 172.21.15.42:0/1245120150 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f810c004180 con 0x7f8110107090 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8107fff700 1 -- 172.21.15.42:0/1245120150 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f810c0042e0 con 0x7f8110107090 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.922+0000 7f8107fff700 1 -- 172.21.15.42:0/1245120150 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f810c011a50 con 0x7f8110107090 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.923+0000 7f81175c4700 1 -- 172.21.15.42:0/1245120150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 msgr2=0x7f8110109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.923+0000 7f81175c4700 1 --2- 172.21.15.42:0/1245120150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f8110109480 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f810c0098a0 tx=0x7f810c009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.923+0000 7f81175c4700 1 -- 172.21.15.42:0/1245120150 shutdown_connections 2024-09-16T23:39:09.970 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.923+0000 7f81175c4700 1 --2- 172.21.15.42:0/1245120150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f8110109480 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.923+0000 7f81175c4700 1 -- 172.21.15.42:0/1245120150 >> 172.21.15.42:0/1245120150 conn(0x7f8110100ac0 msgr2=0x7f8110102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.923+0000 7f81175c4700 1 -- 172.21.15.42:0/1245120150 shutdown_connections 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.923+0000 7f81175c4700 1 -- 172.21.15.42:0/1245120150 wait complete. 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f81175c4700 1 Processor -- start 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f81175c4700 1 -- start start 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f81175c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f81101069f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f81175c4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8110106f30 con 0x7f8110107090 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f8115360700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f81101069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f8115360700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f81101069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53424/0 (socket says 172.21.15.42:53424) 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f8115360700 1 -- 172.21.15.42:0/2768984556 learned_addr learned my addr 172.21.15.42:0/2768984556 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f8115360700 1 -- 172.21.15.42:0/2768984556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c009550 con 0x7f8110107090 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.924+0000 7f8115360700 1 --2- 172.21.15.42:0/2768984556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f81101069f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:09.971 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.925+0000 7f81067fc700 1 -- 172.21.15.42:0/2768984556 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f810c003b90 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.925+0000 7f81067fc700 1 -- 172.21.15.42:0/2768984556 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f810c0248b0 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.925+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8110105040 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.925+0000 7f81067fc700 1 -- 172.21.15.42:0/2768984556 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f810c01a8e0 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.925+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8110105460 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.925+0000 7f81067fc700 1 -- 172.21.15.42:0/2768984556 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f810c02d3f0 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.925+0000 7f81067fc700 1 -- 172.21.15.42:0/2768984556 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f810c019580 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.926+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811010a740 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:08.928+0000 7f81067fc700 1 -- 172.21.15.42:0/2768984556 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f810c024db0 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.001+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f811010a960 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.002+0000 7f81067fc700 1 -- 172.21.15.42:0/2768984556 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f810c024a20 con 0x7f8110107090 2024-09-16T23:39:09.972 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.003+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 msgr2=0x7f81101069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:09.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.003+0000 7f81175c4700 1 --2- 172.21.15.42:0/2768984556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f81101069f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:09.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.003+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 shutdown_connections 2024-09-16T23:39:09.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.003+0000 7f81175c4700 1 --2- 172.21.15.42:0/2768984556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8110107090 0x7f81101069f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:09.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.003+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 >> 172.21.15.42:0/2768984556 conn(0x7f8110100ac0 msgr2=0x7f8110101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:09.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.003+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 shutdown_connections 2024-09-16T23:39:09.973 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:09.003+0000 7f81175c4700 1 -- 172.21.15.42:0/2768984556 wait complete. 2024-09-16T23:39:09.973 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (1/15)... 2024-09-16T23:39:12.668 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:12 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2624181935' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T23:39:13.095 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:13.095 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:39:13.095 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "c2e6f2e2-7484-11ef-bceb-c7b262605968", 2024-09-16T23:39:13.095 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T23:39:13.095 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T23:39:13.095 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "smithi042" 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T23:39:13.096 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T23:39:13.097 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:13.097 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T23:39:13.097 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:13.097 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T23:39:13.097 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T23:39:13.097 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T23:39:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T23:39:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T23:39:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T23:39:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T23:39:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T23:39:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T23:39:13.105 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:13.106 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T23:39:01.225035+0000", 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.377+0000 7f48ad38a700 1 Processor -- start 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.377+0000 7f48ad38a700 1 -- start start 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.377+0000 7f48ad38a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a8109440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.377+0000 7f48ad38a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8109980 con 0x7f48a8107050 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.377+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a8109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.377+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a8109440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53432/0 (socket says 172.21.15.42:53432) 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.377+0000 7f48a6ffd700 1 -- 172.21.15.42:0/1731187815 learned_addr learned my addr 172.21.15.42:0/1731187815 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.378+0000 7f48a6ffd700 1 -- 172.21.15.42:0/1731187815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8109ac0 con 0x7f48a8107050 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.378+0000 7f48a6ffd700 1 --2- 172.21.15.42:0/1731187815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a8109440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48900098a0 tx=0x7f4890009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac742606c90b0f76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:13.107 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.378+0000 7f48a5ffb700 1 -- 172.21.15.42:0/1731187815 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4890004180 con 0x7f48a8107050 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.378+0000 7f48a5ffb700 1 -- 172.21.15.42:0/1731187815 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48900042e0 con 0x7f48a8107050 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.378+0000 7f48a5ffb700 1 -- 172.21.15.42:0/1731187815 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4890011a50 con 0x7f48a8107050 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 -- 172.21.15.42:0/1731187815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 msgr2=0x7f48a8109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 --2- 172.21.15.42:0/1731187815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a8109440 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48900098a0 tx=0x7f4890009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 -- 172.21.15.42:0/1731187815 shutdown_connections 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 --2- 172.21.15.42:0/1731187815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a8109440 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 -- 172.21.15.42:0/1731187815 >> 172.21.15.42:0/1731187815 conn(0x7f48a8100ae0 msgr2=0x7f48a8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 -- 172.21.15.42:0/1731187815 shutdown_connections 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 -- 172.21.15.42:0/1731187815 wait complete. 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.379+0000 7f48ad38a700 1 Processor -- start 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48ad38a700 1 -- start start 2024-09-16T23:39:13.108 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48ad38a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a81069b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48ad38a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8106ef0 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a81069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a81069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53436/0 (socket says 172.21.15.42:53436) 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48a6ffd700 1 -- 172.21.15.42:0/2624181935 learned_addr learned my addr 172.21.15.42:0/2624181935 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48a6ffd700 1 -- 172.21.15.42:0/2624181935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4890009550 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f48a6ffd700 1 --2- 172.21.15.42:0/2624181935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a81069b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f4890011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.380+0000 7f489ffff700 1 -- 172.21.15.42:0/2624181935 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4890003b90 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.381+0000 7f489ffff700 1 -- 172.21.15.42:0/2624181935 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48900248b0 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.381+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a8105000 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.381+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a8105420 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.381+0000 7f489ffff700 1 -- 172.21.15.42:0/2624181935 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f489001a8e0 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.381+0000 7f489ffff700 1 -- 172.21.15.42:0/2624181935 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f489002d3f0 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.381+0000 7f489ffff700 1 -- 172.21.15.42:0/2624181935 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4890019490 con 0x7f48a8107050 2024-09-16T23:39:13.109 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.382+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4888005320 con 0x7f48a8107050 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.384+0000 7f489ffff700 1 -- 172.21.15.42:0/2624181935 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f4890024440 con 0x7f48a8107050 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.457+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4888005cc0 con 0x7f48a8107050 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.458+0000 7f489ffff700 1 -- 172.21.15.42:0/2624181935 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f4890024690 con 0x7f48a8107050 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.459+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 msgr2=0x7f48a81069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.459+0000 7f48ad38a700 1 --2- 172.21.15.42:0/2624181935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a81069b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f4890011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.459+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 shutdown_connections 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.459+0000 7f48ad38a700 1 --2- 172.21.15.42:0/2624181935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48a8107050 0x7f48a81069b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.459+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 >> 172.21.15.42:0/2624181935 conn(0x7f48a8100ae0 msgr2=0x7f48a8102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.459+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 shutdown_connections 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:12.459+0000 7f48ad38a700 1 -- 172.21.15.42:0/2624181935 wait complete. 2024-09-16T23:39:13.110 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (2/15)... 2024-09-16T23:39:15.427 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: Activating manager daemon smithi042.zrkumz 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: mgrmap e2: smithi042.zrkumz(active, starting, since 0.00304723s) 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:39:15.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:39:15.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:15.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:15.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:15 smithi042 ceph-mon[27914]: from='mgr.14100 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:16.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:16.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "c2e6f2e2-7484-11ef-bceb-c7b262605968", 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T23:39:16.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "smithi042" 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T23:39:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T23:39:16.170 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T23:39:16.171 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T23:39:01.225035+0000", 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:39:16.172 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.530+0000 7ffad891c700 1 Processor -- start 2024-09-16T23:39:16.173 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.530+0000 7ffad891c700 1 -- start start 2024-09-16T23:39:16.173 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad891c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad4105940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:16.173 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad891c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4105f10 con 0x7ffad4105520 2024-09-16T23:39:16.173 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad4105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:16.173 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad4105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42808/0 (socket says 172.21.15.42:42808) 2024-09-16T23:39:16.173 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad259c700 1 -- 172.21.15.42:0/1644623049 learned_addr learned my addr 172.21.15.42:0/1644623049 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:16.173 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad259c700 1 -- 172.21.15.42:0/1644623049 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad4106720 con 0x7ffad4105520 2024-09-16T23:39:16.176 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad259c700 1 --2- 172.21.15.42:0/1644623049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad4105940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0098a0 tx=0x7ffabc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3890da2879be46a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:16.176 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad159a700 1 -- 172.21.15.42:0/1644623049 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffabc004180 con 0x7ffad4105520 2024-09-16T23:39:16.176 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad159a700 1 -- 172.21.15.42:0/1644623049 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffabc0042e0 con 0x7ffad4105520 2024-09-16T23:39:16.176 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.531+0000 7ffad159a700 1 -- 172.21.15.42:0/1644623049 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffabc011a50 con 0x7ffad4105520 2024-09-16T23:39:16.176 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.532+0000 7ffad891c700 1 -- 172.21.15.42:0/1644623049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 msgr2=0x7ffad4105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:16.176 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.532+0000 7ffad891c700 1 --2- 172.21.15.42:0/1644623049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad4105940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0098a0 tx=0x7ffabc009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.532+0000 7ffad891c700 1 -- 172.21.15.42:0/1644623049 shutdown_connections 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.532+0000 7ffad891c700 1 --2- 172.21.15.42:0/1644623049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad4105940 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.532+0000 7ffad891c700 1 -- 172.21.15.42:0/1644623049 >> 172.21.15.42:0/1644623049 conn(0x7ffad4100ac0 msgr2=0x7ffad4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.532+0000 7ffad891c700 1 -- 172.21.15.42:0/1644623049 shutdown_connections 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.532+0000 7ffad891c700 1 -- 172.21.15.42:0/1644623049 wait complete. 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad891c700 1 Processor -- start 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad891c700 1 -- start start 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad891c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad41906f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad891c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4190c30 con 0x7ffad4105520 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad41906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad41906f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42822/0 (socket says 172.21.15.42:42822) 2024-09-16T23:39:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad259c700 1 -- 172.21.15.42:0/155555652 learned_addr learned my addr 172.21.15.42:0/155555652 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.533+0000 7ffad259c700 1 -- 172.21.15.42:0/155555652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc009550 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.534+0000 7ffad259c700 1 --2- 172.21.15.42:0/155555652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad41906f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffabc006af0 tx=0x7ffabc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.534+0000 7ffacb7fe700 1 -- 172.21.15.42:0/155555652 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffabc003b90 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.534+0000 7ffacb7fe700 1 -- 172.21.15.42:0/155555652 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffabc01a8a0 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.534+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad418ed40 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.534+0000 7ffacb7fe700 1 -- 172.21.15.42:0/155555652 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffabc019880 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.534+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad418f160 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.535+0000 7ffacb7fe700 1 -- 172.21.15.42:0/155555652 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 2) v1 ==== 45039+0+0 (secure 0 0 0) 0x7ffabc019af0 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.535+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad40623c0 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.535+0000 7ffacb7fe700 1 -- 172.21.15.42:0/155555652 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffabc04c1a0 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.537+0000 7ffacb7fe700 1 -- 172.21.15.42:0/155555652 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ffabc019da0 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.611+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffad4109920 con 0x7ffad4105520 2024-09-16T23:39:16.178 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.611+0000 7ffacb7fe700 1 -- 172.21.15.42:0/155555652 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7ffabc0172b0 con 0x7ffad4105520 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.612+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 msgr2=0x7ffad41906f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.612+0000 7ffad891c700 1 --2- 172.21.15.42:0/155555652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad41906f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffabc006af0 tx=0x7ffabc011a90 comp rx=0 tx=0).stop 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.612+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 shutdown_connections 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.612+0000 7ffad891c700 1 --2- 172.21.15.42:0/155555652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffad4105520 0x7ffad41906f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.612+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 >> 172.21.15.42:0/155555652 conn(0x7ffad4100ac0 msgr2=0x7ffad4073fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.612+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 shutdown_connections 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:15.612+0000 7ffad891c700 1 -- 172.21.15.42:0/155555652 wait complete. 2024-09-16T23:39:16.179 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (3/15)... 2024-09-16T23:39:16.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:16 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/155555652' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T23:39:16.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:16 smithi042 ceph-mon[27914]: mgrmap e3: smithi042.zrkumz(active, since 1.00691s) 2024-09-16T23:39:18.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:18 smithi042 ceph-mon[27914]: mgrmap e4: smithi042.zrkumz(active, since 2s) 2024-09-16T23:39:19.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:19.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:39:19.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "c2e6f2e2-7484-11ef-bceb-c7b262605968", 2024-09-16T23:39:19.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T23:39:19.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T23:39:19.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "smithi042" 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T23:39:19.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T23:39:19.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:19.413 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T23:39:01.225035+0000", 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T23:39:19.414 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.569+0000 7f90c0ba2700 1 Processor -- start 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.569+0000 7f90c0ba2700 1 -- start start 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.569+0000 7f90c0ba2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc106820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.569+0000 7f90c0ba2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc106df0 con 0x7f90bc106400 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc106820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42824/0 (socket says 172.21.15.42:42824) 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90bb7fe700 1 -- 172.21.15.42:0/3170257139 learned_addr learned my addr 172.21.15.42:0/3170257139 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90bb7fe700 1 -- 172.21.15.42:0/3170257139 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc107600 con 0x7f90bc106400 2024-09-16T23:39:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90bb7fe700 1 --2- 172.21.15.42:0/3170257139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc106820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0098a0 tx=0x7f90ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b16c1f7f9d75db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90ba7fc700 1 -- 172.21.15.42:0/3170257139 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90ac004180 con 0x7f90bc106400 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90ba7fc700 1 -- 172.21.15.42:0/3170257139 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90ac0042e0 con 0x7f90bc106400 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.570+0000 7f90ba7fc700 1 -- 172.21.15.42:0/3170257139 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90ac011a50 con 0x7f90bc106400 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 -- 172.21.15.42:0/3170257139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 msgr2=0x7f90bc106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 --2- 172.21.15.42:0/3170257139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc106820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0098a0 tx=0x7f90ac009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 -- 172.21.15.42:0/3170257139 shutdown_connections 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 --2- 172.21.15.42:0/3170257139 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc106820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 -- 172.21.15.42:0/3170257139 >> 172.21.15.42:0/3170257139 conn(0x7f90bc101a00 msgr2=0x7f90bc103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 -- 172.21.15.42:0/3170257139 shutdown_connections 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 -- 172.21.15.42:0/3170257139 wait complete. 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.571+0000 7f90c0ba2700 1 Processor -- start 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90c0ba2700 1 -- start start 2024-09-16T23:39:19.416 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90c0ba2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90c0ba2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc19a660 con 0x7f90bc106400 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc19a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42834/0 (socket says 172.21.15.42:42834) 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90bb7fe700 1 -- 172.21.15.42:0/799929145 learned_addr learned my addr 172.21.15.42:0/799929145 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90bb7fe700 1 -- 172.21.15.42:0/799929145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac009550 con 0x7f90bc106400 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.572+0000 7f90bb7fe700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc19a120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90ac006af0 tx=0x7f90ac011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.573+0000 7f90b8ff9700 1 -- 172.21.15.42:0/799929145 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90ac003b90 con 0x7f90bc106400 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.573+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc19a860 con 0x7f90bc106400 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.573+0000 7f90b8ff9700 1 -- 172.21.15.42:0/799929145 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f90ac01a8a0 con 0x7f90bc106400 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.573+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc19ac80 con 0x7f90bc106400 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.573+0000 7f90b8ff9700 1 -- 172.21.15.42:0/799929145 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f90ac019880 con 0x7f90bc106400 2024-09-16T23:39:19.417 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.573+0000 7f90b8ff9700 1 -- 172.21.15.42:0/799929145 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f90ac019b10 con 0x7f90bc106400 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.574+0000 7f90b8ff9700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f90a80382e0 0x7f90a803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.574+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc0623c0 con 0x7f90bc106400 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.574+0000 7f90b8ff9700 1 -- 172.21.15.42:0/799929145 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f90ac04c610 con 0x7f90bc106400 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.574+0000 7f90baffd700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f90a80382e0 0x7f90a803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.574+0000 7f90baffd700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f90a80382e0 0x7f90a803a7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.579+0000 7f90b8ff9700 1 -- 172.21.15.42:0/799929145 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f90ac02c3a0 con 0x7f90bc106400 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.859+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f90bc19ddf0 con 0x7f90bc106400 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.859+0000 7f90b8ff9700 1 -- 172.21.15.42:0/799929145 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f90ac0172b0 con 0x7f90bc106400 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f90a80382e0 msgr2=0x7f90a803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:19.418 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f90a80382e0 0x7f90a803a7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:19.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 msgr2=0x7f90bc19a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:19.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc19a120 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90ac006af0 tx=0x7f90ac011a90 comp rx=0 tx=0).stop 2024-09-16T23:39:19.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 shutdown_connections 2024-09-16T23:39:19.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f90a80382e0 0x7f90a803a7a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:19.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 --2- 172.21.15.42:0/799929145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90bc106400 0x7f90bc19a120 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:19.419 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 >> 172.21.15.42:0/799929145 conn(0x7f90bc101a00 msgr2=0x7f90bc1030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:19.420 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 shutdown_connections 2024-09-16T23:39:19.420 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:18.862+0000 7f90c0ba2700 1 -- 172.21.15.42:0/799929145 wait complete. 2024-09-16T23:39:19.420 INFO:teuthology.orchestra.run.smithi042.stdout:mgr is available 2024-09-16T23:39:19.520 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:19 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/799929145' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T23:39:20.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:20 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3754455134' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T23:39:20.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:20.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T23:39:20.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:39:20.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T23:39:20.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T23:39:20.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.822+0000 7f775568d700 1 Processor -- start 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.822+0000 7f775568d700 1 -- start start 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.823+0000 7f775568d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f7750109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.823+0000 7f775568d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750079e80 con 0x7f7750107080 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.823+0000 7f774effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f7750109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.823+0000 7f774effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f7750109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42850/0 (socket says 172.21.15.42:42850) 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.823+0000 7f774effd700 1 -- 172.21.15.42:0/330453263 learned_addr learned my addr 172.21.15.42:0/330453263 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.823+0000 7f774effd700 1 -- 172.21.15.42:0/330453263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77501099b0 con 0x7f7750107080 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.823+0000 7f774effd700 1 --2- 172.21.15.42:0/330453263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f7750109470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f77400098a0 tx=0x7f7740009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd25d6ab0a033e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f774e7fc700 1 -- 172.21.15.42:0/330453263 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7740004180 con 0x7f7750107080 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f774e7fc700 1 -- 172.21.15.42:0/330453263 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77400042e0 con 0x7f7750107080 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f774e7fc700 1 -- 172.21.15.42:0/330453263 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7740011a50 con 0x7f7750107080 2024-09-16T23:39:20.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f775568d700 1 -- 172.21.15.42:0/330453263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 msgr2=0x7f7750109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f775568d700 1 --2- 172.21.15.42:0/330453263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f7750109470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f77400098a0 tx=0x7f7740009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f775568d700 1 -- 172.21.15.42:0/330453263 shutdown_connections 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f775568d700 1 --2- 172.21.15.42:0/330453263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f7750109470 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f775568d700 1 -- 172.21.15.42:0/330453263 >> 172.21.15.42:0/330453263 conn(0x7f7750100ad0 msgr2=0x7f7750102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f775568d700 1 -- 172.21.15.42:0/330453263 shutdown_connections 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.824+0000 7f775568d700 1 -- 172.21.15.42:0/330453263 wait complete. 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.825+0000 7f775568d700 1 Processor -- start 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.825+0000 7f775568d700 1 -- start start 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.825+0000 7f775568d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f77501069e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:20.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.825+0000 7f775568d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750106f20 con 0x7f7750107080 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.825+0000 7f774effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f77501069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.825+0000 7f774effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f77501069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42852/0 (socket says 172.21.15.42:42852) 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.825+0000 7f774effd700 1 -- 172.21.15.42:0/3754455134 learned_addr learned my addr 172.21.15.42:0/3754455134 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.826+0000 7f774effd700 1 -- 172.21.15.42:0/3754455134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7740009550 con 0x7f7750107080 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.826+0000 7f774effd700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f77501069e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7740000c00 tx=0x7f7740011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.826+0000 7f774cff9700 1 -- 172.21.15.42:0/3754455134 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7740003b90 con 0x7f7750107080 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.826+0000 7f774cff9700 1 -- 172.21.15.42:0/3754455134 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f77400248b0 con 0x7f7750107080 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.826+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750105030 con 0x7f7750107080 2024-09-16T23:39:20.839 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.826+0000 7f774cff9700 1 -- 172.21.15.42:0/3754455134 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f774001a8e0 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.826+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7750105450 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.827+0000 7f774cff9700 1 -- 172.21.15.42:0/3754455134 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f774001ab50 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.827+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775010a700 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.827+0000 7f774cff9700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f773c038320 0x7f773c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.828+0000 7f774cff9700 1 -- 172.21.15.42:0/3754455134 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f774004c200 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.828+0000 7f77467fc700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f773c038320 0x7f773c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.828+0000 7f77467fc700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f773c038320 0x7f773c03a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7738006fd0 tx=0x7f7738006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:19.832+0000 7f774cff9700 1 -- 172.21.15.42:0/3754455134 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f774001ae00 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.030+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f7750105f00 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.031+0000 7f774cff9700 1 -- 172.21.15.42:0/3754455134 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f7750105f00 con 0x7f7750107080 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f773c038320 msgr2=0x7f773c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f773c038320 0x7f773c03a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7738006fd0 tx=0x7f7738006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:20.840 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 msgr2=0x7f77501069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f77501069e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7740000c00 tx=0x7f7740011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 shutdown_connections 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f773c038320 0x7f773c03a7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 --2- 172.21.15.42:0/3754455134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7750107080 0x7f77501069e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.033+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 >> 172.21.15.42:0/3754455134 conn(0x7f7750100ad0 msgr2=0x7f77501021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.034+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 shutdown_connections 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:20.034+0000 7f775568d700 1 -- 172.21.15.42:0/3754455134 wait complete. 2024-09-16T23:39:20.841 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling cephadm module... 2024-09-16T23:39:22.448 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:22 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1332360000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.240+0000 7ff4b6335700 1 Processor -- start 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.240+0000 7ff4b6335700 1 -- start start 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.240+0000 7ff4b6335700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b0109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.240+0000 7ff4b6335700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b0074610 con 0x7ff4b0107090 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.240+0000 7ff4affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b0109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.241+0000 7ff4affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b0109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42860/0 (socket says 172.21.15.42:42860) 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.241+0000 7ff4affff700 1 -- 172.21.15.42:0/2867558989 learned_addr learned my addr 172.21.15.42:0/2867558989 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.241+0000 7ff4affff700 1 -- 172.21.15.42:0/2867558989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b01099c0 con 0x7ff4b0107090 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.241+0000 7ff4affff700 1 --2- 172.21.15.42:0/2867558989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b0109480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00098a0 tx=0x7ff4a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4124e75cb7107fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.241+0000 7ff4aeffd700 1 -- 172.21.15.42:0/2867558989 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a0004180 con 0x7ff4b0107090 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.241+0000 7ff4aeffd700 1 -- 172.21.15.42:0/2867558989 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4a00042e0 con 0x7ff4b0107090 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.241+0000 7ff4aeffd700 1 -- 172.21.15.42:0/2867558989 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a0011a50 con 0x7ff4b0107090 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 -- 172.21.15.42:0/2867558989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 msgr2=0x7ff4b0109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 --2- 172.21.15.42:0/2867558989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b0109480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00098a0 tx=0x7ff4a0009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 -- 172.21.15.42:0/2867558989 shutdown_connections 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 --2- 172.21.15.42:0/2867558989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b0109480 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 -- 172.21.15.42:0/2867558989 >> 172.21.15.42:0/2867558989 conn(0x7ff4b0100ac0 msgr2=0x7ff4b0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 -- 172.21.15.42:0/2867558989 shutdown_connections 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 -- 172.21.15.42:0/2867558989 wait complete. 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 Processor -- start 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.242+0000 7ff4b6335700 1 -- start start 2024-09-16T23:39:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4b6335700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b01069f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4b6335700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b0106f30 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b01069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b01069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42876/0 (socket says 172.21.15.42:42876) 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4affff700 1 -- 172.21.15.42:0/1332360000 learned_addr learned my addr 172.21.15.42:0/1332360000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4affff700 1 -- 172.21.15.42:0/1332360000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a0009550 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4affff700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b01069f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0000c00 tx=0x7ff4a0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.243+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a0003b90 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.244+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b0105040 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.244+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4a00248b0 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.244+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4a001a8e0 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.244+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b0105460 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.244+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7ff4a001aba0 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.245+0000 7ff4ad7fa700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff49c0382d0 0x7ff49c03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.245+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff4a004c1d0 con 0x7ff4b0107090 2024-09-16T23:39:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.245+0000 7ff4af7fe700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff49c0382d0 0x7ff49c03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.245+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b010a740 con 0x7ff4b0107090 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.245+0000 7ff4af7fe700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff49c0382d0 0x7ff49c03a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff498006fd0 tx=0x7ff498006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.250+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff4a00233a0 con 0x7ff4b0107090 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:21.517+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7ff4b010a960 con 0x7ff4b0107090 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.189+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff4a0018350 con 0x7ff4b0107090 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.190+0000 7ff4ad7fa700 1 -- 172.21.15.42:0/1332360000 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7ff4a0024db0 con 0x7ff4b0107090 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff49c0382d0 msgr2=0x7ff49c03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff49c0382d0 0x7ff49c03a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff498006fd0 tx=0x7ff498006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 msgr2=0x7ff4b01069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b01069f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0000c00 tx=0x7ff4a0011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 shutdown_connections 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff49c0382d0 0x7ff49c03a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 --2- 172.21.15.42:0/1332360000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4b0107090 0x7ff4b01069f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:22.845 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 >> 172.21.15.42:0/1332360000 conn(0x7ff4b0100ac0 msgr2=0x7ff4b0101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:22.845 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 shutdown_connections 2024-09-16T23:39:22.845 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:22.193+0000 7ff4b6335700 1 -- 172.21.15.42:0/1332360000 wait complete. 2024-09-16T23:39:23.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:23 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1332360000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T23:39:23.454 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:23 smithi042 ceph-mon[27914]: mgrmap e5: smithi042.zrkumz(active, since 7s) 2024-09-16T23:39:24.164 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:39:24.164 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T23:39:24.164 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "smithi042.zrkumz", 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.251+0000 7f499d399700 1 Processor -- start 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.251+0000 7f499d399700 1 -- start start 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.251+0000 7f499d399700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f4998109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.251+0000 7f499d399700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4998079ec0 con 0x7f4998107090 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f4998109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f4998109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56240/0 (socket says 172.21.15.42:56240) 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4996ffd700 1 -- 172.21.15.42:0/1918418541 learned_addr learned my addr 172.21.15.42:0/1918418541 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4996ffd700 1 -- 172.21.15.42:0/1918418541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49981099c0 con 0x7f4998107090 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4996ffd700 1 --2- 172.21.15.42:0/1918418541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f4998109480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49880098a0 tx=0x7f4988009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a08b99f1dbf7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4995ffb700 1 -- 172.21.15.42:0/1918418541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4988004180 con 0x7f4998107090 2024-09-16T23:39:24.165 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4995ffb700 1 -- 172.21.15.42:0/1918418541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49880042e0 con 0x7f4998107090 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.252+0000 7f4995ffb700 1 -- 172.21.15.42:0/1918418541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4988011a50 con 0x7f4998107090 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.253+0000 7f499d399700 1 -- 172.21.15.42:0/1918418541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 msgr2=0x7f4998109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.253+0000 7f499d399700 1 --2- 172.21.15.42:0/1918418541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f4998109480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49880098a0 tx=0x7f4988009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.253+0000 7f499d399700 1 -- 172.21.15.42:0/1918418541 shutdown_connections 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.253+0000 7f499d399700 1 --2- 172.21.15.42:0/1918418541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f4998109480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.253+0000 7f499d399700 1 -- 172.21.15.42:0/1918418541 >> 172.21.15.42:0/1918418541 conn(0x7f4998100ac0 msgr2=0x7f4998102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.253+0000 7f499d399700 1 -- 172.21.15.42:0/1918418541 shutdown_connections 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.253+0000 7f499d399700 1 -- 172.21.15.42:0/1918418541 wait complete. 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.254+0000 7f499d399700 1 Processor -- start 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.254+0000 7f499d399700 1 -- start start 2024-09-16T23:39:24.166 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.254+0000 7f499d399700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f499819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.254+0000 7f499d399700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499819a7d0 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.254+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f499819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.254+0000 7f4996ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f499819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56256/0 (socket says 172.21.15.42:56256) 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.254+0000 7f4996ffd700 1 -- 172.21.15.42:0/638545107 learned_addr learned my addr 172.21.15.42:0/638545107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.255+0000 7f4996ffd700 1 -- 172.21.15.42:0/638545107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988009550 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.255+0000 7f4996ffd700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f499819a290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4988000c00 tx=0x7f4988011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.255+0000 7f498ffff700 1 -- 172.21.15.42:0/638545107 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4988003b90 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.255+0000 7f498ffff700 1 -- 172.21.15.42:0/638545107 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f49880248b0 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.255+0000 7f498ffff700 1 -- 172.21.15.42:0/638545107 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f498801a8e0 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.255+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499819a9d0 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.255+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499819adf0 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.256+0000 7f498ffff700 1 -- 172.21.15.42:0/638545107 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f498801aba0 con 0x7f4998107090 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.256+0000 7f498ffff700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 0x7f498403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:24.167 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.256+0000 7f498ffff700 1 -- 172.21.15.42:0/638545107 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f498804c0c0 con 0x7f4998107090 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.256+0000 7f49967fc700 1 -- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 msgr2=0x7f498403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.256+0000 7f49967fc700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 0x7f498403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.256+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49980623c0 con 0x7f4998107090 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.261+0000 7f498ffff700 1 -- 172.21.15.42:0/638545107 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f49880237b0 con 0x7f4998107090 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.457+0000 7f49967fc700 1 -- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 msgr2=0x7f498403a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.457+0000 7f49967fc700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 0x7f498403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.520+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f499819d830 con 0x7f4998107090 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.520+0000 7f498ffff700 1 -- 172.21.15.42:0/638545107 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f49880172b0 con 0x7f4998107090 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 msgr2=0x7f498403a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 0x7f498403a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 msgr2=0x7f499819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f499819a290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4988000c00 tx=0x7f4988011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:24.168 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 shutdown_connections 2024-09-16T23:39:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4984038320 0x7f498403a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 --2- 172.21.15.42:0/638545107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4998107090 0x7f499819a290 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 >> 172.21.15.42:0/638545107 conn(0x7f4998100ac0 msgr2=0x7f4998101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.523+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 shutdown_connections 2024-09-16T23:39:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:23.524+0000 7f499d399700 1 -- 172.21.15.42:0/638545107 wait complete. 2024-09-16T23:39:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-09-16T23:39:24.169 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 5... 2024-09-16T23:39:24.466 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:24 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/638545107' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T23:39:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: Active manager daemon smithi042.zrkumz restarted 2024-09-16T23:39:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: Activating manager daemon smithi042.zrkumz 2024-09-16T23:39:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T23:39:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: mgrmap e6: smithi042.zrkumz(active, starting, since 0.00351663s) 2024-09-16T23:39:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:39:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:39:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:39:29.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:29 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:39:30.511 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:30 smithi042 ceph-mon[27914]: Found migration_current of "None". Setting to last migration. 2024-09-16T23:39:30.511 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:30 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:30.512 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:30 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:30.512 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:30 smithi042 ceph-mon[27914]: mgrmap e7: smithi042.zrkumz(active, since 1.00561s) 2024-09-16T23:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T23:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T23:39:30.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f99988b3700 1 Processor -- start 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f99988b3700 1 -- start start 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f99988b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f99988b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9994079ec0 con 0x7f9994104e80 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f999259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f999259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56264/0 (socket says 172.21.15.42:56264) 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f999259c700 1 -- 172.21.15.42:0/4193227619 learned_addr learned my addr 172.21.15.42:0/4193227619 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.573+0000 7f999259c700 1 -- 172.21.15.42:0/4193227619 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99941077f0 con 0x7f9994104e80 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.574+0000 7f999259c700 1 --2- 172.21.15.42:0/4193227619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941072b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f99840098a0 tx=0x7f9984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f5960944c809ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.574+0000 7f999159a700 1 -- 172.21.15.42:0/4193227619 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9984004180 con 0x7f9994104e80 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.574+0000 7f999159a700 1 -- 172.21.15.42:0/4193227619 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99840042e0 con 0x7f9994104e80 2024-09-16T23:39:30.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.574+0000 7f999159a700 1 -- 172.21.15.42:0/4193227619 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9984011a50 con 0x7f9994104e80 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.574+0000 7f99988b3700 1 -- 172.21.15.42:0/4193227619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 msgr2=0x7f99941072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.575+0000 7f99988b3700 1 --2- 172.21.15.42:0/4193227619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941072b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f99840098a0 tx=0x7f9984009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.575+0000 7f99988b3700 1 -- 172.21.15.42:0/4193227619 shutdown_connections 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.575+0000 7f99988b3700 1 --2- 172.21.15.42:0/4193227619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941072b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.575+0000 7f99988b3700 1 -- 172.21.15.42:0/4193227619 >> 172.21.15.42:0/4193227619 conn(0x7f9994100ac0 msgr2=0x7f9994102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.575+0000 7f99988b3700 1 -- 172.21.15.42:0/4193227619 shutdown_connections 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.575+0000 7f99988b3700 1 -- 172.21.15.42:0/4193227619 wait complete. 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f99988b3700 1 Processor -- start 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f99988b3700 1 -- start start 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f99988b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f99988b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941a0e00 con 0x7f9994104e80 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f999259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:30.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f999259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56270/0 (socket says 172.21.15.42:56270) 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f999259c700 1 -- 172.21.15.42:0/2757302680 learned_addr learned my addr 172.21.15.42:0/2757302680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f999259c700 1 -- 172.21.15.42:0/2757302680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984009550 con 0x7f9994104e80 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.576+0000 7f999259c700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941a08c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9984000c00 tx=0x7f9984011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.577+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9984003b90 con 0x7f9994104e80 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.577+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f99840248b0 con 0x7f9994104e80 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.577+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f998401a8e0 con 0x7f9994104e80 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.577+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99941a1000 con 0x7f9994104e80 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.577+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99941a1420 con 0x7f9994104e80 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.578+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f998401ab50 con 0x7f9994104e80 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.578+0000 7f99837fe700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.578+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f997c03af40 con 0x7f997c038370 2024-09-16T23:39:30.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.578+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f998404c2c0 con 0x7f9994104e80 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.578+0000 7f9991d9b700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 msgr2=0x7f997c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.578+0000 7f9991d9b700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.779+0000 7f9991d9b700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 msgr2=0x7f997c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:24.779+0000 7f9991d9b700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:25.180+0000 7f9991d9b700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 msgr2=0x7f997c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:25.180+0000 7f9991d9b700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:25.981+0000 7f9991d9b700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 msgr2=0x7f997c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:25.981+0000 7f9991d9b700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:27.582+0000 7f9991d9b700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 msgr2=0x7f997c03a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:27.582+0000 7f9991d9b700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:29.179+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7f998402d3f0 con 0x7f9994104e80 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:29.179+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 msgr2=0x7f997c03a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:29.179+0000 7f99837fe700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:30.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.180+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f998404df20 con 0x7f9994104e80 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.181+0000 7f99837fe700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.181+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f997c03af40 con 0x7f997c038370 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.181+0000 7f9991d9b700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.182+0000 7f9991d9b700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9988003a10 tx=0x7f99880092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.182+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f997c03af40 con 0x7f997c038370 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.188+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f99941a1c00 con 0x7f997c038370 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99837fe700 1 -- 172.21.15.42:0/2757302680 <== mgr.14120 v2:172.21.15.42:6800/2 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f99941a1c00 con 0x7f997c038370 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 msgr2=0x7f997c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9988003a10 tx=0x7f99880092b0 comp rx=0 tx=0).stop 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 msgr2=0x7f99941a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941a08c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9984000c00 tx=0x7f9984011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 shutdown_connections 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f997c038370 0x7f997c03a830 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:30.742 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 --2- 172.21.15.42:0/2757302680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9994104e80 0x7f99941a08c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:30.743 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 >> 172.21.15.42:0/2757302680 conn(0x7f9994100ac0 msgr2=0x7f9994102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:30.743 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 shutdown_connections 2024-09-16T23:39:30.743 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:30.189+0000 7f99988b3700 1 -- 172.21.15.42:0/2757302680 wait complete. 2024-09-16T23:39:30.743 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 5 is available 2024-09-16T23:39:30.743 INFO:teuthology.orchestra.run.smithi042.stdout:Setting orchestrator backend to cephadm... 2024-09-16T23:39:31.387 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:31 smithi042 ceph-mon[27914]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T23:39:31.387 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:31 smithi042 ceph-mon[27914]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T23:39:31.387 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:31 smithi042 ceph-mon[27914]: [16/Sep/2024:23:39:30] ENGINE Bus STARTING 2024-09-16T23:39:31.387 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:31 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:39:31.993 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.190+0000 7f0962222700 1 Processor -- start 2024-09-16T23:39:31.993 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f0962222700 1 -- start start 2024-09-16T23:39:31.993 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f0962222700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:31.993 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f0962222700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c106fc0 con 0x7f095c1065d0 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f095b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f095b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56360/0 (socket says 172.21.15.42:56360) 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f095b7fe700 1 -- 172.21.15.42:0/677022295 learned_addr learned my addr 172.21.15.42:0/677022295 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f095b7fe700 1 -- 172.21.15.42:0/677022295 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c1077d0 con 0x7f095c1065d0 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.191+0000 7f095b7fe700 1 --2- 172.21.15.42:0/677022295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c1069f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f094c0098a0 tx=0x7f094c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fcd02b439e3e8f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f095a7fc700 1 -- 172.21.15.42:0/677022295 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f094c004180 con 0x7f095c1065d0 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f095a7fc700 1 -- 172.21.15.42:0/677022295 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f094c0042e0 con 0x7f095c1065d0 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f095a7fc700 1 -- 172.21.15.42:0/677022295 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f094c011a50 con 0x7f095c1065d0 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f0962222700 1 -- 172.21.15.42:0/677022295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 msgr2=0x7f095c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f0962222700 1 --2- 172.21.15.42:0/677022295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c1069f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f094c0098a0 tx=0x7f094c009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f0962222700 1 -- 172.21.15.42:0/677022295 shutdown_connections 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f0962222700 1 --2- 172.21.15.42:0/677022295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c1069f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f0962222700 1 -- 172.21.15.42:0/677022295 >> 172.21.15.42:0/677022295 conn(0x7f095c101bd0 msgr2=0x7f095c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:31.994 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f0962222700 1 -- 172.21.15.42:0/677022295 shutdown_connections 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.192+0000 7f0962222700 1 -- 172.21.15.42:0/677022295 wait complete. 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.193+0000 7f0962222700 1 Processor -- start 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.193+0000 7f0962222700 1 -- start start 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.193+0000 7f0962222700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.193+0000 7f0962222700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c19ca30 con 0x7f095c1065d0 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.193+0000 7f095b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.193+0000 7f095b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56364/0 (socket says 172.21.15.42:56364) 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.193+0000 7f095b7fe700 1 -- 172.21.15.42:0/1221207866 learned_addr learned my addr 172.21.15.42:0/1221207866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.194+0000 7f095b7fe700 1 -- 172.21.15.42:0/1221207866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c009550 con 0x7f095c1065d0 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.194+0000 7f095b7fe700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c19c4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.194+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f094c003b90 con 0x7f095c1065d0 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.194+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f094c0248b0 con 0x7f095c1065d0 2024-09-16T23:39:31.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.194+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c19cc30 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.194+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f094c01a8e0 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.194+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c19d050 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.195+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f094c01aba0 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.195+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c0623c0 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.195+0000 7f0958ff9700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0944038200 0x7f094403a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.195+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f094c04c3d0 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.195+0000 7f095affd700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0944038200 0x7f094403a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.195+0000 7f095affd700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0944038200 0x7f094403a6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.200+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f094c018ca0 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.231+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f094c02dc90 con 0x7f095c1065d0 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.419+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f095c103ec0 con 0x7f0944038200 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.424+0000 7f0958ff9700 1 -- 172.21.15.42:0/1221207866 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f095c103ec0 con 0x7f0944038200 2024-09-16T23:39:31.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0944038200 msgr2=0x7f094403a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0944038200 0x7f094403a6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0950006fd0 tx=0x7f0950006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 msgr2=0x7f095c19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c19c4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f094c000c00 tx=0x7f094c011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 shutdown_connections 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0944038200 0x7f094403a6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 --2- 172.21.15.42:0/1221207866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f095c1065d0 0x7f095c19c4f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 >> 172.21.15.42:0/1221207866 conn(0x7f095c101bd0 msgr2=0x7f095c103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 shutdown_connections 2024-09-16T23:39:31.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:31.427+0000 7f0962222700 1 -- 172.21.15.42:0/1221207866 wait complete. 2024-09-16T23:39:32.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:32 smithi042 ceph-mon[27914]: [16/Sep/2024:23:39:30] ENGINE Serving on http://172.21.15.42:8765 2024-09-16T23:39:32.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:32 smithi042 ceph-mon[27914]: [16/Sep/2024:23:39:30] ENGINE Serving on https://172.21.15.42:7150 2024-09-16T23:39:32.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:32 smithi042 ceph-mon[27914]: [16/Sep/2024:23:39:30] ENGINE Bus STARTED 2024-09-16T23:39:32.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:32 smithi042 ceph-mon[27914]: mgrmap e8: smithi042.zrkumz(active, since 2s) 2024-09-16T23:39:32.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:32 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:32.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:32 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.394+0000 7f4f2cb09700 1 Processor -- start 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.394+0000 7f4f2cb09700 1 -- start start 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.394+0000 7f4f2cb09700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.394+0000 7f4f2cb09700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2807a8c0 con 0x7f4f2807be90 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.394+0000 7f4f2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.394+0000 7f4f2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56368/0 (socket says 172.21.15.42:56368) 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.394+0000 7f4f2659c700 1 -- 172.21.15.42:0/905871405 learned_addr learned my addr 172.21.15.42:0/905871405 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.395+0000 7f4f2659c700 1 -- 172.21.15.42:0/905871405 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2807aa00 con 0x7f4f2807be90 2024-09-16T23:39:33.151 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.395+0000 7f4f2659c700 1 --2- 172.21.15.42:0/905871405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2807a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4f180098a0 tx=0x7f4f18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f86a989c5af645a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.395+0000 7f4f2559a700 1 -- 172.21.15.42:0/905871405 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f18004180 con 0x7f4f2807be90 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.395+0000 7f4f2559a700 1 -- 172.21.15.42:0/905871405 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f180042e0 con 0x7f4f2807be90 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.395+0000 7f4f2559a700 1 -- 172.21.15.42:0/905871405 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f18011a50 con 0x7f4f2807be90 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.396+0000 7f4f2cb09700 1 -- 172.21.15.42:0/905871405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 msgr2=0x7f4f2807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.396+0000 7f4f2cb09700 1 --2- 172.21.15.42:0/905871405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2807a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4f180098a0 tx=0x7f4f18009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.396+0000 7f4f2cb09700 1 -- 172.21.15.42:0/905871405 shutdown_connections 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.396+0000 7f4f2cb09700 1 --2- 172.21.15.42:0/905871405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2807a2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.396+0000 7f4f2cb09700 1 -- 172.21.15.42:0/905871405 >> 172.21.15.42:0/905871405 conn(0x7f4f28101bd0 msgr2=0x7f4f28103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.396+0000 7f4f2cb09700 1 -- 172.21.15.42:0/905871405 shutdown_connections 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.396+0000 7f4f2cb09700 1 -- 172.21.15.42:0/905871405 wait complete. 2024-09-16T23:39:33.152 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2cb09700 1 Processor -- start 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2cb09700 1 -- start start 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2cb09700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2cb09700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2819c9d0 con 0x7f4f2807be90 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2819c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56372/0 (socket says 172.21.15.42:56372) 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2659c700 1 -- 172.21.15.42:0/2810327027 learned_addr learned my addr 172.21.15.42:0/2810327027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2659c700 1 -- 172.21.15.42:0/2810327027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18009550 con 0x7f4f2807be90 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.397+0000 7f4f2659c700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2819c490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f18000c00 tx=0x7f4f18011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.398+0000 7f4f177fe700 1 -- 172.21.15.42:0/2810327027 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f18003b90 con 0x7f4f2807be90 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.398+0000 7f4f177fe700 1 -- 172.21.15.42:0/2810327027 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4f180248b0 con 0x7f4f2807be90 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.398+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2819cbd0 con 0x7f4f2807be90 2024-09-16T23:39:33.153 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.398+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2819cff0 con 0x7f4f2807be90 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.398+0000 7f4f177fe700 1 -- 172.21.15.42:0/2810327027 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4f1801a8e0 con 0x7f4f2807be90 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.399+0000 7f4f177fe700 1 -- 172.21.15.42:0/2810327027 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f4f1801aba0 con 0x7f4f2807be90 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.399+0000 7f4f177fe700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4f10038320 0x7f4f1003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.399+0000 7f4f177fe700 1 -- 172.21.15.42:0/2810327027 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4f18024db0 con 0x7f4f2807be90 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.399+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f280623c0 con 0x7f4f2807be90 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.399+0000 7f4f25d9b700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4f10038320 0x7f4f1003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.399+0000 7f4f25d9b700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4f10038320 0x7f4f1003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c006fd0 tx=0x7f4f1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.404+0000 7f4f177fe700 1 -- 172.21.15.42:0/2810327027 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4f180237b0 con 0x7f4f2807be90 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.602+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f28103e90 con 0x7f4f10038320 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.603+0000 7f4f177fe700 1 -- 172.21.15.42:0/2810327027 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f4f28103e90 con 0x7f4f10038320 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4f10038320 msgr2=0x7f4f1003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:33.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4f10038320 0x7f4f1003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c006fd0 tx=0x7f4f1c006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 msgr2=0x7f4f2819c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2819c490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f18000c00 tx=0x7f4f18011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 shutdown_connections 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4f10038320 0x7f4f1003a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 --2- 172.21.15.42:0/2810327027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f2807be90 0x7f4f2819c490 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 >> 172.21.15.42:0/2810327027 conn(0x7f4f28101bd0 msgr2=0x7f4f28103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 shutdown_connections 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:32.605+0000 7f4f2cb09700 1 -- 172.21.15.42:0/2810327027 wait complete. 2024-09-16T23:39:33.155 INFO:teuthology.orchestra.run.smithi042.stdout:Generating ssh key... 2024-09-16T23:39:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:33 smithi042 ceph-mon[27914]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:34.507 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:34 smithi042 ceph-mon[27914]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:34.507 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:34 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:34.507 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:34 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:34.757 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.584+0000 7f19101f8700 1 Processor -- start 2024-09-16T23:39:34.757 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.584+0000 7f19101f8700 1 -- start start 2024-09-16T23:39:34.757 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.584+0000 7f19101f8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f1908109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:34.757 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.584+0000 7f19101f8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908079e80 con 0x7f1908107080 2024-09-16T23:39:34.757 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.584+0000 7f190df94700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f1908109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.584+0000 7f190df94700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f1908109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33044/0 (socket says 172.21.15.42:33044) 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.584+0000 7f190df94700 1 -- 172.21.15.42:0/564661006 learned_addr learned my addr 172.21.15.42:0/564661006 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.585+0000 7f190df94700 1 -- 172.21.15.42:0/564661006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19081099b0 con 0x7f1908107080 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.585+0000 7f190df94700 1 --2- 172.21.15.42:0/564661006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f1908109470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19040098a0 tx=0x7f1904009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8e13668b02706a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.585+0000 7f190cf92700 1 -- 172.21.15.42:0/564661006 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1904004180 con 0x7f1908107080 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.585+0000 7f190cf92700 1 -- 172.21.15.42:0/564661006 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19040042e0 con 0x7f1908107080 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.585+0000 7f190cf92700 1 -- 172.21.15.42:0/564661006 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1904011a50 con 0x7f1908107080 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 -- 172.21.15.42:0/564661006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 msgr2=0x7f1908109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 --2- 172.21.15.42:0/564661006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f1908109470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19040098a0 tx=0x7f1904009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 -- 172.21.15.42:0/564661006 shutdown_connections 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 --2- 172.21.15.42:0/564661006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f1908109470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:34.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 -- 172.21.15.42:0/564661006 >> 172.21.15.42:0/564661006 conn(0x7f1908100ad0 msgr2=0x7f1908102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 -- 172.21.15.42:0/564661006 shutdown_connections 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 -- 172.21.15.42:0/564661006 wait complete. 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.586+0000 7f19101f8700 1 Processor -- start 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f19101f8700 1 -- start start 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f19101f8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f190819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f19101f8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190819a800 con 0x7f1908107080 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f190df94700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f190819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f190df94700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f190819a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33060/0 (socket says 172.21.15.42:33060) 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f190df94700 1 -- 172.21.15.42:0/1270397596 learned_addr learned my addr 172.21.15.42:0/1270397596 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f190df94700 1 -- 172.21.15.42:0/1270397596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904009550 con 0x7f1908107080 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f190df94700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f190819a2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1904000c00 tx=0x7f1904011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f18feffd700 1 -- 172.21.15.42:0/1270397596 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1904003b90 con 0x7f1908107080 2024-09-16T23:39:34.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f18feffd700 1 -- 172.21.15.42:0/1270397596 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f19040248b0 con 0x7f1908107080 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.587+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190819aa00 con 0x7f1908107080 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.588+0000 7f18feffd700 1 -- 172.21.15.42:0/1270397596 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f190401a8e0 con 0x7f1908107080 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.588+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190819ae20 con 0x7f1908107080 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.588+0000 7f18feffd700 1 -- 172.21.15.42:0/1270397596 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f190401aba0 con 0x7f1908107080 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.589+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19080623c0 con 0x7f1908107080 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.589+0000 7f18feffd700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f18f4038320 0x7f18f403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.589+0000 7f18feffd700 1 -- 172.21.15.42:0/1270397596 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f190404c590 con 0x7f1908107080 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.589+0000 7f190d793700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f18f4038320 0x7f18f403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:34.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.589+0000 7f190d793700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f18f4038320 0x7f18f403a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f8006fd0 tx=0x7f18f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.593+0000 7f18feffd700 1 -- 172.21.15.42:0/1270397596 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1904024db0 con 0x7f1908107080 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:33.795+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1908102dd0 con 0x7f18f4038320 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.046+0000 7f18feffd700 1 -- 172.21.15.42:0/1270397596 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1908102dd0 con 0x7f18f4038320 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.048+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f18f4038320 msgr2=0x7f18f403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.048+0000 7f19101f8700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f18f4038320 0x7f18f403a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f18f8006fd0 tx=0x7f18f8006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.048+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 msgr2=0x7f190819a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.048+0000 7f19101f8700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f190819a2c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1904000c00 tx=0x7f1904011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.049+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 shutdown_connections 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.049+0000 7f19101f8700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f18f4038320 0x7f18f403a7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.049+0000 7f19101f8700 1 --2- 172.21.15.42:0/1270397596 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1908107080 0x7f190819a2c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:34.762 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.049+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 >> 172.21.15.42:0/1270397596 conn(0x7f1908100ad0 msgr2=0x7f19081021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:34.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.049+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 shutdown_connections 2024-09-16T23:39:34.763 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:34.049+0000 7f19101f8700 1 -- 172.21.15.42:0/1270397596 wait complete. 2024-09-16T23:39:35.381 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:35 smithi042 ceph-mon[27914]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:35.381 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:35 smithi042 ceph-mon[27914]: Generating ssh key... 2024-09-16T23:39:35.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8Lx5Nko6PWBLPOns0p2KXRniqJseOx2gkF24r0d0+0gVtYZrpUF84Zg7+HrTveO0D+VDlUW/kSWjnMcSoPK7MvvZU946VpEm5Bu98W6hIIQ0mNpFAEeVo492gaDggpOZx6TKD4HSCmpf+kW1VEyUkRDicglbcCHNTwXomsAe5sqJvE/51H31dA680ykFgWg0fEuABth5OIhjhJHze7SoQ15PpsfZVx+gmXGoOamLOylJohxGq+eV+MOegOZK/JCO5LjJFvLEpy7/FiXZhswRJTcJy+IolVPaaxJLVtJl+ypyqkDglm1IgHPR7OsfwSHJAa3syHH1J09NXEthxsAOXjdVfuRtXItGz7B3FZkEm8lCIqIZEyY0bRg5vui7WJ3V+WYbjvopbm5dUxLflzIw0c8y5rJfZvUhCxgozeBNUEqfzdhfOZQTe9akxZwVRDyPNjHXo1XpQOyR1VJ6hDQs+qiS+/EjMC+u5bCfTZEi0oUS4/nNZczkerdQCxC77Yi0= ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:39:35.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.174+0000 7f660aea1700 1 Processor -- start 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.174+0000 7f660aea1700 1 -- start start 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.174+0000 7f660aea1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.174+0000 7f660aea1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660407a8e0 con 0x7f660407beb0 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f6608c3d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f6608c3d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33062/0 (socket says 172.21.15.42:33062) 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f6608c3d700 1 -- 172.21.15.42:0/3672950435 learned_addr learned my addr 172.21.15.42:0/3672950435 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f6608c3d700 1 -- 172.21.15.42:0/3672950435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660407aa20 con 0x7f660407beb0 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f6608c3d700 1 --2- 172.21.15.42:0/3672950435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660407a310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65f40098a0 tx=0x7f65f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173b190ef4e844d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f66037fe700 1 -- 172.21.15.42:0/3672950435 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65f4004180 con 0x7f660407beb0 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f66037fe700 1 -- 172.21.15.42:0/3672950435 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f65f40042e0 con 0x7f660407beb0 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.175+0000 7f66037fe700 1 -- 172.21.15.42:0/3672950435 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65f4011a50 con 0x7f660407beb0 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.176+0000 7f660aea1700 1 -- 172.21.15.42:0/3672950435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 msgr2=0x7f660407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:35.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.176+0000 7f660aea1700 1 --2- 172.21.15.42:0/3672950435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660407a310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f65f40098a0 tx=0x7f65f4009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.176+0000 7f660aea1700 1 -- 172.21.15.42:0/3672950435 shutdown_connections 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.176+0000 7f660aea1700 1 --2- 172.21.15.42:0/3672950435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660407a310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.176+0000 7f660aea1700 1 -- 172.21.15.42:0/3672950435 >> 172.21.15.42:0/3672950435 conn(0x7f6604101bd0 msgr2=0x7f6604103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.176+0000 7f660aea1700 1 -- 172.21.15.42:0/3672950435 shutdown_connections 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.176+0000 7f660aea1700 1 -- 172.21.15.42:0/3672950435 wait complete. 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f660aea1700 1 Processor -- start 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f660aea1700 1 -- start start 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f660aea1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f660aea1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660419ca10 con 0x7f660407beb0 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f6608c3d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f6608c3d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33074/0 (socket says 172.21.15.42:33074) 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f6608c3d700 1 -- 172.21.15.42:0/3291886070 learned_addr learned my addr 172.21.15.42:0/3291886070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:35.966 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.177+0000 7f6608c3d700 1 -- 172.21.15.42:0/3291886070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f4009550 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.178+0000 7f6608c3d700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660419c4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006af0 tx=0x7f65f4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.178+0000 7f6601ffb700 1 -- 172.21.15.42:0/3291886070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65f4003b90 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.178+0000 7f6601ffb700 1 -- 172.21.15.42:0/3291886070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f65f401a8a0 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.178+0000 7f6601ffb700 1 -- 172.21.15.42:0/3291886070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65f4019880 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.178+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660419cc10 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.178+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660419d030 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.179+0000 7f6601ffb700 1 -- 172.21.15.42:0/3291886070 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f65f401aa10 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.179+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66040623c0 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.179+0000 7f6601ffb700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f65ec038330 0x7f65ec03a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.180+0000 7f6601ffb700 1 -- 172.21.15.42:0/3291886070 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f65f404c690 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.180+0000 7f6603fff700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f65ec038330 0x7f65ec03a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.180+0000 7f6603fff700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f65ec038330 0x7f65ec03a7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65f8006fd0 tx=0x7f65f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.185+0000 7f6601ffb700 1 -- 172.21.15.42:0/3291886070 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f65f401a430 con 0x7f660407beb0 2024-09-16T23:39:35.967 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.383+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6604103eb0 con 0x7f65ec038330 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.384+0000 7f6601ffb700 1 -- 172.21.15.42:0/3291886070 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f6604103eb0 con 0x7f65ec038330 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f65ec038330 msgr2=0x7f65ec03a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f65ec038330 0x7f65ec03a7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65f8006fd0 tx=0x7f65f8006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 msgr2=0x7f660419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660419c4d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f65f4006af0 tx=0x7f65f4011a90 comp rx=0 tx=0).stop 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 shutdown_connections 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f65ec038330 0x7f65ec03a7f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 --2- 172.21.15.42:0/3291886070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f660407beb0 0x7f660419c4d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 >> 172.21.15.42:0/3291886070 conn(0x7f6604101bd0 msgr2=0x7f6604103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 shutdown_connections 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:35.386+0000 7f660aea1700 1 -- 172.21.15.42:0/3291886070 wait complete. 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T23:39:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T23:39:35.972 INFO:teuthology.orchestra.run.smithi042.stdout:Adding host smithi042... 2024-09-16T23:39:37.494 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:37 smithi042 ceph-mon[27914]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:38.473 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:38 smithi042 ceph-mon[27914]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:38.473 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:38 smithi042 ceph-mon[27914]: Deploying cephadm binary to smithi042 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Added host 'smithi042' with addr '172.21.15.42' 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.390+0000 7f38d8669700 1 Processor -- start 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.390+0000 7f38d8669700 1 -- start start 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d8669700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d00791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d8669700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00797b0 con 0x7f38d007ad80 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d6405700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d00791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d6405700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d00791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33078/0 (socket says 172.21.15.42:33078) 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d6405700 1 -- 172.21.15.42:0/391985840 learned_addr learned my addr 172.21.15.42:0/391985840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d6405700 1 -- 172.21.15.42:0/391985840 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00798f0 con 0x7f38d007ad80 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d6405700 1 --2- 172.21.15.42:0/391985840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d00791e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0098a0 tx=0x7f38cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b05d3734ae49dda server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:39.614 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d5403700 1 -- 172.21.15.42:0/391985840 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38cc004180 con 0x7f38d007ad80 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d5403700 1 -- 172.21.15.42:0/391985840 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38cc0042e0 con 0x7f38d007ad80 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.391+0000 7f38d5403700 1 -- 172.21.15.42:0/391985840 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38cc011a50 con 0x7f38d007ad80 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.392+0000 7f38d8669700 1 -- 172.21.15.42:0/391985840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 msgr2=0x7f38d00791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.392+0000 7f38d8669700 1 --2- 172.21.15.42:0/391985840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d00791e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0098a0 tx=0x7f38cc009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.392+0000 7f38d8669700 1 -- 172.21.15.42:0/391985840 shutdown_connections 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.392+0000 7f38d8669700 1 --2- 172.21.15.42:0/391985840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d00791e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.392+0000 7f38d8669700 1 -- 172.21.15.42:0/391985840 >> 172.21.15.42:0/391985840 conn(0x7f38d0101bc0 msgr2=0x7f38d0104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.392+0000 7f38d8669700 1 -- 172.21.15.42:0/391985840 shutdown_connections 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.392+0000 7f38d8669700 1 -- 172.21.15.42:0/391985840 wait complete. 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d8669700 1 Processor -- start 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d8669700 1 -- start start 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d8669700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d019c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d8669700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d019c9f0 con 0x7f38d007ad80 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d6405700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d019c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:39.615 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d6405700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d019c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33086/0 (socket says 172.21.15.42:33086) 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d6405700 1 -- 172.21.15.42:0/3766291622 learned_addr learned my addr 172.21.15.42:0/3766291622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d6405700 1 -- 172.21.15.42:0/3766291622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009550 con 0x7f38d007ad80 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.393+0000 7f38d6405700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d019c4b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38cc000c00 tx=0x7f38cc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.394+0000 7f38c77fe700 1 -- 172.21.15.42:0/3766291622 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38cc003b90 con 0x7f38d007ad80 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.394+0000 7f38c77fe700 1 -- 172.21.15.42:0/3766291622 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f38cc0248b0 con 0x7f38d007ad80 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.394+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d019cbf0 con 0x7f38d007ad80 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.394+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d019d010 con 0x7f38d007ad80 2024-09-16T23:39:39.616 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.394+0000 7f38c77fe700 1 -- 172.21.15.42:0/3766291622 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38cc01a8e0 con 0x7f38d007ad80 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.395+0000 7f38c77fe700 1 -- 172.21.15.42:0/3766291622 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f38cc01ab50 con 0x7f38d007ad80 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.395+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d00623c0 con 0x7f38d007ad80 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.395+0000 7f38c77fe700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f38bc038370 0x7f38bc03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.395+0000 7f38c77fe700 1 -- 172.21.15.42:0/3766291622 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f38cc04c3b0 con 0x7f38d007ad80 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.395+0000 7f38d5c04700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f38bc038370 0x7f38bc03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.396+0000 7f38d5c04700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f38bc038370 0x7f38bc03a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38c0006fd0 tx=0x7f38c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.400+0000 7f38c77fe700 1 -- 172.21.15.42:0/3766291622 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f38cc01ae00 con 0x7f38d007ad80 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:36.602+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}) v1 -- 0x7f38d0103090 con 0x7f38bc038370 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.008+0000 7f38c77fe700 1 -- 172.21.15.42:0/3766291622 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f38d0103090 con 0x7f38bc038370 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f38bc038370 msgr2=0x7f38bc03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f38bc038370 0x7f38bc03a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38c0006fd0 tx=0x7f38c0006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 msgr2=0x7f38d019c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d019c4b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f38cc000c00 tx=0x7f38cc011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:39.617 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 shutdown_connections 2024-09-16T23:39:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f38bc038370 0x7f38bc03a830 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 --2- 172.21.15.42:0/3766291622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38d007ad80 0x7f38d019c4b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 >> 172.21.15.42:0/3766291622 conn(0x7f38d0101bc0 msgr2=0x7f38d0102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 shutdown_connections 2024-09-16T23:39:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:39.011+0000 7f38d8669700 1 -- 172.21.15.42:0/3766291622 wait complete. 2024-09-16T23:39:39.618 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying mon service with default placement... 2024-09-16T23:39:40.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:40 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:40.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:40 smithi042 ceph-mon[27914]: Added host smithi042 2024-09-16T23:39:40.227 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:40 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:39:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:41 smithi042 ceph-mon[27914]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:41 smithi042 ceph-mon[27914]: Saving service mon spec with placement count:5 2024-09-16T23:39:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:41 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:41 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.009+0000 7fd36385a700 1 Processor -- start 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.009+0000 7fd36385a700 1 -- start start 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.010+0000 7fd36385a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c104d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.010+0000 7fd36385a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c1052f0 con 0x7fd35c104900 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.010+0000 7fd3615f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c104d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.010+0000 7fd3615f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c104d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33102/0 (socket says 172.21.15.42:33102) 2024-09-16T23:39:41.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.010+0000 7fd3615f6700 1 -- 172.21.15.42:0/4075988559 learned_addr learned my addr 172.21.15.42:0/4075988559 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.010+0000 7fd3615f6700 1 -- 172.21.15.42:0/4075988559 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c107890 con 0x7fd35c104900 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.010+0000 7fd3615f6700 1 --2- 172.21.15.42:0/4075988559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c104d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd3580098a0 tx=0x7fd358009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c1d642a6afc2e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.011+0000 7fd353fff700 1 -- 172.21.15.42:0/4075988559 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd358004180 con 0x7fd35c104900 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.011+0000 7fd353fff700 1 -- 172.21.15.42:0/4075988559 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3580042e0 con 0x7fd35c104900 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.011+0000 7fd353fff700 1 -- 172.21.15.42:0/4075988559 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd358011a50 con 0x7fd35c104900 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.012+0000 7fd36385a700 1 -- 172.21.15.42:0/4075988559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 msgr2=0x7fd35c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.012+0000 7fd36385a700 1 --2- 172.21.15.42:0/4075988559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c104d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd3580098a0 tx=0x7fd358009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.012+0000 7fd36385a700 1 -- 172.21.15.42:0/4075988559 shutdown_connections 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.012+0000 7fd36385a700 1 --2- 172.21.15.42:0/4075988559 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c104d20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.012+0000 7fd36385a700 1 -- 172.21.15.42:0/4075988559 >> 172.21.15.42:0/4075988559 conn(0x7fd35c100230 msgr2=0x7fd35c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.012+0000 7fd36385a700 1 -- 172.21.15.42:0/4075988559 shutdown_connections 2024-09-16T23:39:41.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.012+0000 7fd36385a700 1 -- 172.21.15.42:0/4075988559 wait complete. 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd36385a700 1 Processor -- start 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd36385a700 1 -- start start 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd36385a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c1071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd36385a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c107730 con 0x7fd35c104900 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd3615f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c1071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd3615f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c1071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33108/0 (socket says 172.21.15.42:33108) 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd3615f6700 1 -- 172.21.15.42:0/3658746917 learned_addr learned my addr 172.21.15.42:0/3658746917 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd3615f6700 1 -- 172.21.15.42:0/3658746917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd358009550 con 0x7fd35c104900 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.013+0000 7fd3615f6700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c1071f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd358000c00 tx=0x7fd358011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.014+0000 7fd3527fc700 1 -- 172.21.15.42:0/3658746917 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd358003b90 con 0x7fd35c104900 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.014+0000 7fd3527fc700 1 -- 172.21.15.42:0/3658746917 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd3580248b0 con 0x7fd35c104900 2024-09-16T23:39:41.571 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.014+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c105840 con 0x7fd35c104900 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.014+0000 7fd3527fc700 1 -- 172.21.15.42:0/3658746917 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd35801a8e0 con 0x7fd35c104900 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.014+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c105c60 con 0x7fd35c104900 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.014+0000 7fd3527fc700 1 -- 172.21.15.42:0/3658746917 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd35801aba0 con 0x7fd35c104900 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.015+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c0623c0 con 0x7fd35c104900 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.015+0000 7fd3527fc700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd348038320 0x7fd34803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.015+0000 7fd3527fc700 1 -- 172.21.15.42:0/3658746917 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd35804c440 con 0x7fd35c104900 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.015+0000 7fd360df5700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd348038320 0x7fd34803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.015+0000 7fd360df5700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd348038320 0x7fd34803a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd34c006fd0 tx=0x7fd34c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.020+0000 7fd3527fc700 1 -- 172.21.15.42:0/3658746917 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd3580233a0 con 0x7fd35c104900 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.236+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fd35c1016f0 con 0x7fd348038320 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.240+0000 7fd3527fc700 1 -- 172.21.15.42:0/3658746917 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd35c1016f0 con 0x7fd348038320 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.242+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd348038320 msgr2=0x7fd34803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.242+0000 7fd36385a700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd348038320 0x7fd34803a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd34c006fd0 tx=0x7fd34c006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:41.572 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.242+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 msgr2=0x7fd35c1071f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.242+0000 7fd36385a700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c1071f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd358000c00 tx=0x7fd358011ab0 comp rx=0 tx=0).stop 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.243+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 shutdown_connections 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.243+0000 7fd36385a700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd348038320 0x7fd34803a7e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.243+0000 7fd36385a700 1 --2- 172.21.15.42:0/3658746917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd35c104900 0x7fd35c1071f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.243+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 >> 172.21.15.42:0/3658746917 conn(0x7fd35c100230 msgr2=0x7fd35c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.243+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 shutdown_connections 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:40.243+0000 7fd36385a700 1 -- 172.21.15.42:0/3658746917 wait complete. 2024-09-16T23:39:41.573 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying mgr service with default placement... 2024-09-16T23:39:42.836 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:42 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:42.836 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:42 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:43.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T23:39:43.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.069+0000 7f9c9db94700 1 Processor -- start 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c9db94700 1 -- start start 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c9db94700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c98106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c9db94700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98107010 con 0x7f9c98106620 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c98106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c98106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33118/0 (socket says 172.21.15.42:33118) 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c977fe700 1 -- 172.21.15.42:0/2229964536 learned_addr learned my addr 172.21.15.42:0/2229964536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c977fe700 1 -- 172.21.15.42:0/2229964536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c98107820 con 0x7f9c98106620 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.070+0000 7f9c977fe700 1 --2- 172.21.15.42:0/2229964536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c98106a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c880098a0 tx=0x7f9c88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f976729e38e80f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c967fc700 1 -- 172.21.15.42:0/2229964536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c88004180 con 0x7f9c98106620 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c967fc700 1 -- 172.21.15.42:0/2229964536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9c880042e0 con 0x7f9c98106620 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c967fc700 1 -- 172.21.15.42:0/2229964536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c88011af0 con 0x7f9c98106620 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c9db94700 1 -- 172.21.15.42:0/2229964536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 msgr2=0x7f9c98106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c9db94700 1 --2- 172.21.15.42:0/2229964536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c98106a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9c880098a0 tx=0x7f9c88009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:43.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c9db94700 1 -- 172.21.15.42:0/2229964536 shutdown_connections 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c9db94700 1 --2- 172.21.15.42:0/2229964536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c98106a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c9db94700 1 -- 172.21.15.42:0/2229964536 >> 172.21.15.42:0/2229964536 conn(0x7f9c98101bc0 msgr2=0x7f9c98104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.071+0000 7f9c9db94700 1 -- 172.21.15.42:0/2229964536 shutdown_connections 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.072+0000 7f9c9db94700 1 -- 172.21.15.42:0/2229964536 wait complete. 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.072+0000 7f9c9db94700 1 Processor -- start 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c9db94700 1 -- start start 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c9db94700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c9819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c9db94700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819a840 con 0x7f9c98106620 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c9819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c977fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c9819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33122/0 (socket says 172.21.15.42:33122) 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c977fe700 1 -- 172.21.15.42:0/422124697 learned_addr learned my addr 172.21.15.42:0/422124697 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c977fe700 1 -- 172.21.15.42:0/422124697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88009550 con 0x7f9c98106620 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.073+0000 7f9c977fe700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c9819a300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c88006ba0 tx=0x7f9c88011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:43.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.074+0000 7f9c94ff9700 1 -- 172.21.15.42:0/422124697 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c88003b00 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.074+0000 7f9c94ff9700 1 -- 172.21.15.42:0/422124697 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9c8801a8e0 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.074+0000 7f9c94ff9700 1 -- 172.21.15.42:0/422124697 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c88019990 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.074+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9819aa40 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.074+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9819ada0 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.075+0000 7f9c94ff9700 1 -- 172.21.15.42:0/422124697 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f9c88019c00 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.075+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c980623c0 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.075+0000 7f9c94ff9700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9c80038370 0x7f9c8003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.075+0000 7f9c94ff9700 1 -- 172.21.15.42:0/422124697 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9c8804c630 con 0x7f9c98106620 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.076+0000 7f9c96ffd700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9c80038370 0x7f9c8003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:43.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.076+0000 7f9c96ffd700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9c80038370 0x7f9c8003a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c006fd0 tx=0x7f9c8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.080+0000 7f9c94ff9700 1 -- 172.21.15.42:0/422124697 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9c8802d440 con 0x7f9c98106620 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.310+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c981030e0 con 0x7f9c80038370 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.314+0000 7f9c94ff9700 1 -- 172.21.15.42:0/422124697 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9c981030e0 con 0x7f9c80038370 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9c80038370 msgr2=0x7f9c8003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9c80038370 0x7f9c8003a830 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c006fd0 tx=0x7f9c8c006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 msgr2=0x7f9c9819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c9819a300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9c88006ba0 tx=0x7f9c88011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 shutdown_connections 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9c80038370 0x7f9c8003a830 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 --2- 172.21.15.42:0/422124697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c98106620 0x7f9c9819a300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 >> 172.21.15.42:0/422124697 conn(0x7f9c98101bc0 msgr2=0x7f9c981029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 shutdown_connections 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:42.316+0000 7f9c9db94700 1 -- 172.21.15.42:0/422124697 wait complete. 2024-09-16T23:39:43.122 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying crash service with default placement... 2024-09-16T23:39:44.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:43 smithi042 ceph-mon[27914]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:44.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:43 smithi042 ceph-mon[27914]: Saving service mgr spec with placement count:2 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.551+0000 7f09e8a89700 1 Processor -- start 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.551+0000 7f09e8a89700 1 -- start start 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e8a89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e41094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e8a89700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e40745b0 con 0x7f09e41070f0 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e41094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e41094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47790/0 (socket says 172.21.15.42:47790) 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e259c700 1 -- 172.21.15.42:0/3993486851 learned_addr learned my addr 172.21.15.42:0/3993486851 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e259c700 1 -- 172.21.15.42:0/3993486851 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4109a20 con 0x7f09e41070f0 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e259c700 1 --2- 172.21.15.42:0/3993486851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e41094e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09d40098a0 tx=0x7f09d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb2a8fa161e70ac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:44.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e159a700 1 -- 172.21.15.42:0/3993486851 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d4004180 con 0x7f09e41070f0 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.552+0000 7f09e159a700 1 -- 172.21.15.42:0/3993486851 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f09d40042e0 con 0x7f09e41070f0 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e159a700 1 -- 172.21.15.42:0/3993486851 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d4011af0 con 0x7f09e41070f0 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e8a89700 1 -- 172.21.15.42:0/3993486851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 msgr2=0x7f09e41094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e8a89700 1 --2- 172.21.15.42:0/3993486851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e41094e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f09d40098a0 tx=0x7f09d4009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e8a89700 1 -- 172.21.15.42:0/3993486851 shutdown_connections 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e8a89700 1 --2- 172.21.15.42:0/3993486851 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e41094e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e8a89700 1 -- 172.21.15.42:0/3993486851 >> 172.21.15.42:0/3993486851 conn(0x7f09e4100ac0 msgr2=0x7f09e4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e8a89700 1 -- 172.21.15.42:0/3993486851 shutdown_connections 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.553+0000 7f09e8a89700 1 -- 172.21.15.42:0/3993486851 wait complete. 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.554+0000 7f09e8a89700 1 Processor -- start 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.554+0000 7f09e8a89700 1 -- start start 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.554+0000 7f09e8a89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.554+0000 7f09e8a89700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e419a830 con 0x7f09e41070f0 2024-09-16T23:39:44.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.554+0000 7f09e259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.554+0000 7f09e259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e419a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47794/0 (socket says 172.21.15.42:47794) 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.554+0000 7f09e259c700 1 -- 172.21.15.42:0/949716903 learned_addr learned my addr 172.21.15.42:0/949716903 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.555+0000 7f09e259c700 1 -- 172.21.15.42:0/949716903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4009550 con 0x7f09e41070f0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.555+0000 7f09e259c700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e419a2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09d4011ad0 tx=0x7f09d4011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.555+0000 7f09d37fe700 1 -- 172.21.15.42:0/949716903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d4003ab0 con 0x7f09e41070f0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.555+0000 7f09d37fe700 1 -- 172.21.15.42:0/949716903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f09d4003c10 con 0x7f09e41070f0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.555+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e419aa30 con 0x7f09e41070f0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.555+0000 7f09d37fe700 1 -- 172.21.15.42:0/949716903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d40198e0 con 0x7f09e41070f0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.555+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e419ae50 con 0x7f09e41070f0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.556+0000 7f09d37fe700 1 -- 172.21.15.42:0/949716903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f09d4019b50 con 0x7f09e41070f0 2024-09-16T23:39:44.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.556+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e410a7b0 con 0x7f09e41070f0 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.556+0000 7f09d37fe700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f09cc038370 0x7f09cc03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.556+0000 7f09d37fe700 1 -- 172.21.15.42:0/949716903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f09d404c1a0 con 0x7f09e41070f0 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.556+0000 7f09e1d9b700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f09cc038370 0x7f09cc03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.557+0000 7f09e1d9b700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f09cc038370 0x7f09cc03a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09d8006fd0 tx=0x7f09d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.561+0000 7f09d37fe700 1 -- 172.21.15.42:0/949716903 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f09d4010da0 con 0x7f09e41070f0 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.774+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f09e419dcb0 con 0x7f09cc038370 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.779+0000 7f09d37fe700 1 -- 172.21.15.42:0/949716903 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f09e419dcb0 con 0x7f09cc038370 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f09cc038370 msgr2=0x7f09cc03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f09cc038370 0x7f09cc03a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09d8006fd0 tx=0x7f09d8006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 msgr2=0x7f09e419a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e419a2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f09d4011ad0 tx=0x7f09d4011bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 shutdown_connections 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f09cc038370 0x7f09cc03a830 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:44.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 --2- 172.21.15.42:0/949716903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09e41070f0 0x7f09e419a2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 >> 172.21.15.42:0/949716903 conn(0x7f09e4100ac0 msgr2=0x7f09e41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 shutdown_connections 2024-09-16T23:39:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:43.781+0000 7f09e8a89700 1 -- 172.21.15.42:0/949716903 wait complete. 2024-09-16T23:39:44.490 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying ceph-exporter service with default placement... 2024-09-16T23:39:45.152 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:44 smithi042 ceph-mon[27914]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:45.152 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:44 smithi042 ceph-mon[27914]: Saving service crash spec with placement * 2024-09-16T23:39:45.152 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:44 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:45.152 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:44 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:45.675 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.933+0000 7f300d539700 1 Processor -- start 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.933+0000 7f300d539700 1 -- start start 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.933+0000 7f300d539700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f3008106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.933+0000 7f300d539700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30081074a0 con 0x7f3008106ab0 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.933+0000 7f3006ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f3008106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.933+0000 7f3006ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f3008106ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47800/0 (socket says 172.21.15.42:47800) 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.933+0000 7f3006ffd700 1 -- 172.21.15.42:0/2912961044 learned_addr learned my addr 172.21.15.42:0/2912961044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.934+0000 7f3006ffd700 1 -- 172.21.15.42:0/2912961044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3008105710 con 0x7f3008106ab0 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.934+0000 7f3006ffd700 1 --2- 172.21.15.42:0/2912961044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f3008106ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80098a0 tx=0x7f2ff8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2dc0c25ade9269 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:45.676 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.934+0000 7f3005ffb700 1 -- 172.21.15.42:0/2912961044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ff8004180 con 0x7f3008106ab0 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.934+0000 7f3005ffb700 1 -- 172.21.15.42:0/2912961044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2ff80042e0 con 0x7f3008106ab0 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.934+0000 7f3005ffb700 1 -- 172.21.15.42:0/2912961044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ff8011af0 con 0x7f3008106ab0 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 -- 172.21.15.42:0/2912961044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 msgr2=0x7f3008106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 --2- 172.21.15.42:0/2912961044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f3008106ed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80098a0 tx=0x7f2ff8009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 -- 172.21.15.42:0/2912961044 shutdown_connections 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 --2- 172.21.15.42:0/2912961044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f3008106ed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 -- 172.21.15.42:0/2912961044 >> 172.21.15.42:0/2912961044 conn(0x7f3008100220 msgr2=0x7f3008102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 -- 172.21.15.42:0/2912961044 shutdown_connections 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 -- 172.21.15.42:0/2912961044 wait complete. 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.935+0000 7f300d539700 1 Processor -- start 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f300d539700 1 -- start start 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f300d539700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f30081906d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f300d539700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008190c10 con 0x7f3008106ab0 2024-09-16T23:39:45.677 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f3006ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f30081906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f3006ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f30081906d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47812/0 (socket says 172.21.15.42:47812) 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f3006ffd700 1 -- 172.21.15.42:0/4088729641 learned_addr learned my addr 172.21.15.42:0/4088729641 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f3006ffd700 1 -- 172.21.15.42:0/4088729641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8009550 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.936+0000 7f3006ffd700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f30081906d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006ba0 tx=0x7f2ff8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.937+0000 7f2feffff700 1 -- 172.21.15.42:0/4088729641 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ff8003b00 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.937+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300818ed20 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.937+0000 7f2feffff700 1 -- 172.21.15.42:0/4088729641 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2ff801a8e0 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.937+0000 7f2feffff700 1 -- 172.21.15.42:0/4088729641 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ff8019990 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.937+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300818f140 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.938+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30080623c0 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.938+0000 7f2feffff700 1 -- 172.21.15.42:0/4088729641 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f2ff8019c50 con 0x7f3008106ab0 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.938+0000 7f2feffff700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2ff00382d0 0x7f2ff003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:45.678 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.938+0000 7f2feffff700 1 -- 172.21.15.42:0/4088729641 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2ff804c5c0 con 0x7f3008106ab0 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.938+0000 7f30067fc700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2ff00382d0 0x7f2ff003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.939+0000 7f30067fc700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2ff00382d0 0x7f2ff003a790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:44.943+0000 7f2feffff700 1 -- 172.21.15.42:0/4088729641 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2ff8018530 con 0x7f3008106ab0 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.154+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f3008102410 con 0x7f2ff00382d0 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.161+0000 7f2feffff700 1 -- 172.21.15.42:0/4088729641 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f3008102410 con 0x7f2ff00382d0 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2ff00382d0 msgr2=0x7f2ff003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2ff00382d0 0x7f2ff003a790 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 msgr2=0x7f30081906d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f30081906d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006ba0 tx=0x7f2ff8011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 shutdown_connections 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2ff00382d0 0x7f2ff003a790 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 --2- 172.21.15.42:0/4088729641 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3008106ab0 0x7f30081906d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:45.679 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 >> 172.21.15.42:0/4088729641 conn(0x7f3008100220 msgr2=0x7f3008104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:45.680 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 shutdown_connections 2024-09-16T23:39:45.680 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:45.163+0000 7f300d539700 1 -- 172.21.15.42:0/4088729641 wait complete. 2024-09-16T23:39:45.680 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying prometheus service with default placement... 2024-09-16T23:39:46.363 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:46 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:46.364 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:46 smithi042 ceph-mon[27914]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:46.364 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:46 smithi042 ceph-mon[27914]: Saving service ceph-exporter spec with placement * 2024-09-16T23:39:46.364 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:46 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:47.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:47 smithi042 ceph-mon[27914]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:47 smithi042 ceph-mon[27914]: Saving service prometheus spec with placement count:1 2024-09-16T23:39:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:47 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-16T23:39:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.123+0000 7fc6aafdd700 1 Processor -- start 2024-09-16T23:39:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.123+0000 7fc6aafdd700 1 -- start start 2024-09-16T23:39:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.124+0000 7fc6aafdd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a4106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:47.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.124+0000 7fc6aafdd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a4107010 con 0x7fc6a4106620 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.124+0000 7fc6a8d79700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a4106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.124+0000 7fc6a8d79700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a4106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47816/0 (socket says 172.21.15.42:47816) 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.124+0000 7fc6a8d79700 1 -- 172.21.15.42:0/3597978019 learned_addr learned my addr 172.21.15.42:0/3597978019 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.124+0000 7fc6a8d79700 1 -- 172.21.15.42:0/3597978019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4107820 con 0x7fc6a4106620 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.124+0000 7fc6a8d79700 1 --2- 172.21.15.42:0/3597978019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a4106a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098a0 tx=0x7fc698009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9608bdbfd8e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6a37fe700 1 -- 172.21.15.42:0/3597978019 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc698004180 con 0x7fc6a4106620 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6a37fe700 1 -- 172.21.15.42:0/3597978019 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc6980042e0 con 0x7fc6a4106620 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6a37fe700 1 -- 172.21.15.42:0/3597978019 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc698011af0 con 0x7fc6a4106620 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6aafdd700 1 -- 172.21.15.42:0/3597978019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 msgr2=0x7fc6a4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6aafdd700 1 --2- 172.21.15.42:0/3597978019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a4106a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc6980098a0 tx=0x7fc698009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6aafdd700 1 -- 172.21.15.42:0/3597978019 shutdown_connections 2024-09-16T23:39:47.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6aafdd700 1 --2- 172.21.15.42:0/3597978019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a4106a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6aafdd700 1 -- 172.21.15.42:0/3597978019 >> 172.21.15.42:0/3597978019 conn(0x7fc6a4101bc0 msgr2=0x7fc6a4104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.125+0000 7fc6aafdd700 1 -- 172.21.15.42:0/3597978019 shutdown_connections 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.126+0000 7fc6aafdd700 1 -- 172.21.15.42:0/3597978019 wait complete. 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.126+0000 7fc6aafdd700 1 Processor -- start 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.126+0000 7fc6aafdd700 1 -- start start 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.126+0000 7fc6aafdd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.126+0000 7fc6aafdd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a419a840 con 0x7fc6a4106620 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.126+0000 7fc6a8d79700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.126+0000 7fc6a8d79700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47832/0 (socket says 172.21.15.42:47832) 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6a8d79700 1 -- 172.21.15.42:0/589959057 learned_addr learned my addr 172.21.15.42:0/589959057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6a8d79700 1 -- 172.21.15.42:0/589959057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc698009550 con 0x7fc6a4106620 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6a8d79700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a419a300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc698006ba0 tx=0x7fc698011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:47.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6a1ffb700 1 -- 172.21.15.42:0/589959057 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc698003b00 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6a1ffb700 1 -- 172.21.15.42:0/589959057 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc69801a8e0 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6a1ffb700 1 -- 172.21.15.42:0/589959057 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc698019990 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a419aa40 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.127+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a419ae60 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.128+0000 7fc6a1ffb700 1 -- 172.21.15.42:0/589959057 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc698019c00 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.128+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a40623c0 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.129+0000 7fc6a1ffb700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc694038320 0x7fc69403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.129+0000 7fc6a1ffb700 1 -- 172.21.15.42:0/589959057 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc69804c500 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.129+0000 7fc6a3fff700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc694038320 0x7fc69403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.129+0000 7fc6a3fff700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc694038320 0x7fc69403a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc690006fd0 tx=0x7fc690006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.137+0000 7fc6a1ffb700 1 -- 172.21.15.42:0/589959057 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc69801a430 con 0x7fc6a4106620 2024-09-16T23:39:47.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.366+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6a41030e0 con 0x7fc694038320 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.370+0000 7fc6a1ffb700 1 -- 172.21.15.42:0/589959057 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fc6a41030e0 con 0x7fc694038320 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.372+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc694038320 msgr2=0x7fc69403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.372+0000 7fc6aafdd700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc694038320 0x7fc69403a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc690006fd0 tx=0x7fc690006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.372+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 msgr2=0x7fc6a419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.372+0000 7fc6aafdd700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a419a300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc698006ba0 tx=0x7fc698011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.373+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 shutdown_connections 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.373+0000 7fc6aafdd700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc694038320 0x7fc69403a7e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.373+0000 7fc6aafdd700 1 --2- 172.21.15.42:0/589959057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6a4106620 0x7fc6a419a300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.373+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 >> 172.21.15.42:0/589959057 conn(0x7fc6a4101bc0 msgr2=0x7fc6a41029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.373+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 shutdown_connections 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:46.373+0000 7fc6aafdd700 1 -- 172.21.15.42:0/589959057 wait complete. 2024-09-16T23:39:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying grafana service with default placement... 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.198+0000 7ff4e53b0700 1 Processor -- start 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.198+0000 7ff4e53b0700 1 -- start start 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.198+0000 7ff4e53b0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.198+0000 7ff4e53b0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e007a8e0 con 0x7ff4e007beb0 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.198+0000 7ff4deffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.198+0000 7ff4deffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47842/0 (socket says 172.21.15.42:47842) 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.198+0000 7ff4deffd700 1 -- 172.21.15.42:0/3400439949 learned_addr learned my addr 172.21.15.42:0/3400439949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.199+0000 7ff4deffd700 1 -- 172.21.15.42:0/3400439949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e007aa20 con 0x7ff4e007beb0 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.199+0000 7ff4deffd700 1 --2- 172.21.15.42:0/3400439949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e007a310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00098a0 tx=0x7ff4d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf90ae97095629d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.199+0000 7ff4ddffb700 1 -- 172.21.15.42:0/3400439949 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4d0004180 con 0x7ff4e007beb0 2024-09-16T23:39:49.004 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.199+0000 7ff4ddffb700 1 -- 172.21.15.42:0/3400439949 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff4d00042e0 con 0x7ff4e007beb0 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.199+0000 7ff4ddffb700 1 -- 172.21.15.42:0/3400439949 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4d0011af0 con 0x7ff4e007beb0 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3400439949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 msgr2=0x7ff4e007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 --2- 172.21.15.42:0/3400439949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e007a310 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00098a0 tx=0x7ff4d0009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3400439949 shutdown_connections 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 --2- 172.21.15.42:0/3400439949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e007a310 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3400439949 >> 172.21.15.42:0/3400439949 conn(0x7ff4e0101bd0 msgr2=0x7ff4e0103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3400439949 shutdown_connections 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3400439949 wait complete. 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.200+0000 7ff4e53b0700 1 Processor -- start 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4e53b0700 1 -- start start 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4e53b0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4e53b0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e019ca50 con 0x7ff4e007beb0 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4deffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:49.005 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4deffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47856/0 (socket says 172.21.15.42:47856) 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4deffd700 1 -- 172.21.15.42:0/3045355534 learned_addr learned my addr 172.21.15.42:0/3045355534 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4deffd700 1 -- 172.21.15.42:0/3045355534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0009550 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4deffd700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e019c510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006ba0 tx=0x7ff4d0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4c7fff700 1 -- 172.21.15.42:0/3045355534 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4d0003b00 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e019cc50 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4c7fff700 1 -- 172.21.15.42:0/3045355534 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff4d001a8e0 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4c7fff700 1 -- 172.21.15.42:0/3045355534 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4d0019990 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.201+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e019d070 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.202+0000 7ff4c7fff700 1 -- 172.21.15.42:0/3045355534 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff4d0019c50 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.202+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e00623c0 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.203+0000 7ff4c7fff700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff4c80382d0 0x7ff4c803a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.203+0000 7ff4c7fff700 1 -- 172.21.15.42:0/3045355534 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff4d004c700 con 0x7ff4e007beb0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.203+0000 7ff4de7fc700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff4c80382d0 0x7ff4c803a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.203+0000 7ff4de7fc700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff4c80382d0 0x7ff4c803a790 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006fd0 tx=0x7ff4d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:49.006 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.208+0000 7ff4c7fff700 1 -- 172.21.15.42:0/3045355534 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff4d002b700 con 0x7ff4e007beb0 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.416+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4e0103eb0 con 0x7ff4c80382d0 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.421+0000 7ff4c7fff700 1 -- 172.21.15.42:0/3045355534 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff4e0103eb0 con 0x7ff4c80382d0 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff4c80382d0 msgr2=0x7ff4c803a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff4c80382d0 0x7ff4c803a790 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4006fd0 tx=0x7ff4d4006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 msgr2=0x7ff4e019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e019c510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0006ba0 tx=0x7ff4d0011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 shutdown_connections 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff4c80382d0 0x7ff4c803a790 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 --2- 172.21.15.42:0/3045355534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4e007beb0 0x7ff4e019c510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 >> 172.21.15.42:0/3045355534 conn(0x7ff4e0101bd0 msgr2=0x7ff4e0103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 shutdown_connections 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:48.423+0000 7ff4e53b0700 1 -- 172.21.15.42:0/3045355534 wait complete. 2024-09-16T23:39:49.007 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying node-exporter service with default placement... 2024-09-16T23:39:49.646 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:49 smithi042 ceph-mon[27914]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:49.647 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:49 smithi042 ceph-mon[27914]: Saving service grafana spec with placement count:1 2024-09-16T23:39:49.647 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:49 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:49.647 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:49 smithi042 ceph-mon[27914]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:39:50.195 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-16T23:39:50.195 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0c25f4700 1 Processor -- start 2024-09-16T23:39:50.195 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0c25f4700 1 -- start start 2024-09-16T23:39:50.195 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0c25f4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc1069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:50.195 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0c25f4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc106fc0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.195 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc1069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:50.195 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc1069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47866/0 (socket says 172.21.15.42:47866) 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0bbfff700 1 -- 172.21.15.42:0/332930182 learned_addr learned my addr 172.21.15.42:0/332930182 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0bbfff700 1 -- 172.21.15.42:0/332930182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc1077d0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.432+0000 7ff0bbfff700 1 --2- 172.21.15.42:0/332930182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc1069f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098a0 tx=0x7ff0ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1905f2c34885b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0bb7fe700 1 -- 172.21.15.42:0/332930182 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0ac004180 con 0x7ff0bc1065d0 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0bb7fe700 1 -- 172.21.15.42:0/332930182 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0ac0042e0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0bb7fe700 1 -- 172.21.15.42:0/332930182 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0ac011af0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0c25f4700 1 -- 172.21.15.42:0/332930182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 msgr2=0x7ff0bc1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0c25f4700 1 --2- 172.21.15.42:0/332930182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc1069f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac0098a0 tx=0x7ff0ac009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0c25f4700 1 -- 172.21.15.42:0/332930182 shutdown_connections 2024-09-16T23:39:50.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0c25f4700 1 --2- 172.21.15.42:0/332930182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc1069f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0c25f4700 1 -- 172.21.15.42:0/332930182 >> 172.21.15.42:0/332930182 conn(0x7ff0bc101bd0 msgr2=0x7ff0bc103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0c25f4700 1 -- 172.21.15.42:0/332930182 shutdown_connections 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.433+0000 7ff0c25f4700 1 -- 172.21.15.42:0/332930182 wait complete. 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0c25f4700 1 Processor -- start 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0c25f4700 1 -- start start 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0c25f4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0c25f4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc19a870 con 0x7ff0bc1065d0 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47872/0 (socket says 172.21.15.42:47872) 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0bbfff700 1 -- 172.21.15.42:0/4042654787 learned_addr learned my addr 172.21.15.42:0/4042654787 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.434+0000 7ff0bbfff700 1 -- 172.21.15.42:0/4042654787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac009550 con 0x7ff0bc1065d0 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.435+0000 7ff0bbfff700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc19a330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac000c00 tx=0x7ff0ac011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.435+0000 7ff0b9ffb700 1 -- 172.21.15.42:0/4042654787 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0ac003a70 con 0x7ff0bc1065d0 2024-09-16T23:39:50.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.435+0000 7ff0b9ffb700 1 -- 172.21.15.42:0/4042654787 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0ac003bd0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.435+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc19aa70 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.435+0000 7ff0b9ffb700 1 -- 172.21.15.42:0/4042654787 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0ac0238c0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.435+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc19ae90 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.436+0000 7ff0b9ffb700 1 -- 172.21.15.42:0/4042654787 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff0ac023b30 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.436+0000 7ff0b9ffb700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff0a8038370 0x7ff0a803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.436+0000 7ff0b9ffb700 1 -- 172.21.15.42:0/4042654787 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff0ac04c1b0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.436+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc0623c0 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.436+0000 7ff0b3fff700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff0a8038370 0x7ff0a803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.436+0000 7ff0b3fff700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff0a8038370 0x7ff0a803a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4006fd0 tx=0x7ff0a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.441+0000 7ff0b9ffb700 1 -- 172.21.15.42:0/4042654787 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff0ac023e30 con 0x7ff0bc1065d0 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.649+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0bc103ee0 con 0x7ff0a8038370 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.654+0000 7ff0b9ffb700 1 -- 172.21.15.42:0/4042654787 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7ff0bc103ee0 con 0x7ff0a8038370 2024-09-16T23:39:50.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff0a8038370 msgr2=0x7ff0a803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff0a8038370 0x7ff0a803a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4006fd0 tx=0x7ff0a4006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 msgr2=0x7ff0bc19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc19a330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac000c00 tx=0x7ff0ac011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 shutdown_connections 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff0a8038370 0x7ff0a803a830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 --2- 172.21.15.42:0/4042654787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff0bc1065d0 0x7ff0bc19a330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 >> 172.21.15.42:0/4042654787 conn(0x7ff0bc101bd0 msgr2=0x7ff0bc1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 shutdown_connections 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:49.656+0000 7ff0c25f4700 1 -- 172.21.15.42:0/4042654787 wait complete. 2024-09-16T23:39:50.199 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying alertmanager service with default placement... 2024-09-16T23:39:50.695 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:50 smithi042 ceph-mon[27914]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:50.695 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:50 smithi042 ceph-mon[27914]: Saving service node-exporter spec with placement * 2024-09-16T23:39:50.695 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:50 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:51.524 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.620+0000 7f551bd5c700 1 Processor -- start 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.620+0000 7f551bd5c700 1 -- start start 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f551bd5c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f55141094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f551bd5c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514079e80 con 0x7f55141070d0 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5519af8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f55141094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5519af8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f55141094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47882/0 (socket says 172.21.15.42:47882) 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5519af8700 1 -- 172.21.15.42:0/3916340967 learned_addr learned my addr 172.21.15.42:0/3916340967 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5519af8700 1 -- 172.21.15.42:0/3916340967 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514109a00 con 0x7f55141070d0 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5519af8700 1 --2- 172.21.15.42:0/3916340967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f55141094c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55100098a0 tx=0x7f5510009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c654e444c577525 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5518af6700 1 -- 172.21.15.42:0/3916340967 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5510004180 con 0x7f55141070d0 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5518af6700 1 -- 172.21.15.42:0/3916340967 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f55100042e0 con 0x7f55141070d0 2024-09-16T23:39:51.525 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.621+0000 7f5518af6700 1 -- 172.21.15.42:0/3916340967 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5510011af0 con 0x7f55141070d0 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.622+0000 7f551bd5c700 1 -- 172.21.15.42:0/3916340967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 msgr2=0x7f55141094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.622+0000 7f551bd5c700 1 --2- 172.21.15.42:0/3916340967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f55141094c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55100098a0 tx=0x7f5510009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.622+0000 7f551bd5c700 1 -- 172.21.15.42:0/3916340967 shutdown_connections 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.622+0000 7f551bd5c700 1 --2- 172.21.15.42:0/3916340967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f55141094c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.622+0000 7f551bd5c700 1 -- 172.21.15.42:0/3916340967 >> 172.21.15.42:0/3916340967 conn(0x7f5514100ac0 msgr2=0x7f5514102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.622+0000 7f551bd5c700 1 -- 172.21.15.42:0/3916340967 shutdown_connections 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.622+0000 7f551bd5c700 1 -- 172.21.15.42:0/3916340967 wait complete. 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f551bd5c700 1 Processor -- start 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f551bd5c700 1 -- start start 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f551bd5c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f551419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:51.526 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f551bd5c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551419a820 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f5519af8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f551419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f5519af8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f551419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47888/0 (socket says 172.21.15.42:47888) 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f5519af8700 1 -- 172.21.15.42:0/275692085 learned_addr learned my addr 172.21.15.42:0/275692085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.623+0000 7f5519af8700 1 -- 172.21.15.42:0/275692085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510009550 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.624+0000 7f5519af8700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f551419a2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5510006ba0 tx=0x7f5510011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.624+0000 7f5506ffd700 1 -- 172.21.15.42:0/275692085 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5510003b00 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.624+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551419aa20 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.624+0000 7f5506ffd700 1 -- 172.21.15.42:0/275692085 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f551001a8e0 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.624+0000 7f5506ffd700 1 -- 172.21.15.42:0/275692085 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55100199c0 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.624+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551419ae40 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.625+0000 7f5506ffd700 1 -- 172.21.15.42:0/275692085 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5510019c80 con 0x7f55141070d0 2024-09-16T23:39:51.527 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.625+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551410a790 con 0x7f55141070d0 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.625+0000 7f5506ffd700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f55000382d0 0x7f550003a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.625+0000 7f5506ffd700 1 -- 172.21.15.42:0/275692085 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f551004c5a0 con 0x7f55141070d0 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.625+0000 7f55192f7700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f55000382d0 0x7f550003a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.626+0000 7f55192f7700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f55000382d0 0x7f550003a790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5508006fd0 tx=0x7f5508006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.630+0000 7f5506ffd700 1 -- 172.21.15.42:0/275692085 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5510018b40 con 0x7f55141070d0 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.845+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f551419dca0 con 0x7f55000382d0 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.850+0000 7f5506ffd700 1 -- 172.21.15.42:0/275692085 <== mgr.14120 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f551419dca0 con 0x7f55000382d0 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f55000382d0 msgr2=0x7f550003a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f55000382d0 0x7f550003a790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5508006fd0 tx=0x7f5508006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 msgr2=0x7f551419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f551419a2e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5510006ba0 tx=0x7f5510011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 shutdown_connections 2024-09-16T23:39:51.528 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f55000382d0 0x7f550003a790 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:51.529 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 --2- 172.21.15.42:0/275692085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55141070d0 0x7f551419a2e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:51.529 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 >> 172.21.15.42:0/275692085 conn(0x7f5514100ac0 msgr2=0x7f5514101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:51.529 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 shutdown_connections 2024-09-16T23:39:51.529 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:50.852+0000 7f551bd5c700 1 -- 172.21.15.42:0/275692085 wait complete. 2024-09-16T23:39:52.134 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:51 smithi042 ceph-mon[27914]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:39:52.134 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:51 smithi042 ceph-mon[27914]: Saving service alertmanager spec with placement count:1 2024-09-16T23:39:52.134 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:51 smithi042 ceph-mon[27914]: from='mgr.14120 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:39:52.134 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:51 smithi042 ceph-mon[27914]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:39:52.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.931+0000 7fdaf8f5a700 1 Processor -- start 2024-09-16T23:39:52.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.931+0000 7fdaf8f5a700 1 -- start start 2024-09-16T23:39:52.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.931+0000 7fdaf8f5a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf4109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:52.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.931+0000 7fdaf8f5a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4079e80 con 0x7fdaf4107080 2024-09-16T23:39:52.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.931+0000 7fdaf259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf4109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:52.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.931+0000 7fdaf259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf4109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47900/0 (socket says 172.21.15.42:47900) 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.931+0000 7fdaf259c700 1 -- 172.21.15.42:0/3131353384 learned_addr learned my addr 172.21.15.42:0/3131353384 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.932+0000 7fdaf259c700 1 -- 172.21.15.42:0/3131353384 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf41099b0 con 0x7fdaf4107080 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.932+0000 7fdaf259c700 1 --2- 172.21.15.42:0/3131353384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf4109470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdae40098a0 tx=0x7fdae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a283158f0def1c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.932+0000 7fdaf159a700 1 -- 172.21.15.42:0/3131353384 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdae4004180 con 0x7fdaf4107080 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.932+0000 7fdaf159a700 1 -- 172.21.15.42:0/3131353384 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdae40042e0 con 0x7fdaf4107080 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.932+0000 7fdaf159a700 1 -- 172.21.15.42:0/3131353384 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdae4011af0 con 0x7fdaf4107080 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/3131353384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 msgr2=0x7fdaf4109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 --2- 172.21.15.42:0/3131353384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf4109470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdae40098a0 tx=0x7fdae4009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/3131353384 shutdown_connections 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 --2- 172.21.15.42:0/3131353384 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf4109470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/3131353384 >> 172.21.15.42:0/3131353384 conn(0x7fdaf4100ad0 msgr2=0x7fdaf4102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/3131353384 shutdown_connections 2024-09-16T23:39:52.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/3131353384 wait complete. 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 Processor -- start 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.933+0000 7fdaf8f5a700 1 -- start start 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf8f5a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf8f5a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf419a860 con 0x7fdaf4107080 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf419a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47914/0 (socket says 172.21.15.42:47914) 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf259c700 1 -- 172.21.15.42:0/97395035 learned_addr learned my addr 172.21.15.42:0/97395035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf259c700 1 -- 172.21.15.42:0/97395035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae4009550 con 0x7fdaf4107080 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf259c700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf419a320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdae4006ba0 tx=0x7fdae4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:52.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdae37fe700 1 -- 172.21.15.42:0/97395035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdae4003b00 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdae37fe700 1 -- 172.21.15.42:0/97395035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdae401a8e0 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdae37fe700 1 -- 172.21.15.42:0/97395035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdae40199c0 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf419aa60 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.934+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf419ae80 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.935+0000 7fdae37fe700 1 -- 172.21.15.42:0/97395035 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fdae4019c30 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.935+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf410a740 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.935+0000 7fdae37fe700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdadc038320 0x7fdadc03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.936+0000 7fdae37fe700 1 -- 172.21.15.42:0/97395035 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdae404c7c0 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.936+0000 7fdaf1d9b700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdadc038320 0x7fdadc03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.936+0000 7fdaf1d9b700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdadc038320 0x7fdadc03a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006fd0 tx=0x7fdae8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:51.940+0000 7fdae37fe700 1 -- 172.21.15.42:0/97395035 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdae402d440 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.137+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fdaf419dce0 con 0x7fdaf4107080 2024-09-16T23:39:52.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.140+0000 7fdae37fe700 1 -- 172.21.15.42:0/97395035 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fdae402b340 con 0x7fdaf4107080 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdadc038320 msgr2=0x7fdadc03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdadc038320 0x7fdadc03a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdae8006fd0 tx=0x7fdae8006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 msgr2=0x7fdaf419a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf419a320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdae4006ba0 tx=0x7fdae4011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 shutdown_connections 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdadc038320 0x7fdadc03a7e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 --2- 172.21.15.42:0/97395035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdaf4107080 0x7fdaf419a320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.142+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 >> 172.21.15.42:0/97395035 conn(0x7fdaf4100ad0 msgr2=0x7fdaf4102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.143+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 shutdown_connections 2024-09-16T23:39:52.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:52.143+0000 7fdaf8f5a700 1 -- 172.21.15.42:0/97395035 wait complete. 2024-09-16T23:39:53.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:53 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/97395035' entity='client.admin' 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.086+0000 7fa19ed09700 1 Processor -- start 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.086+0000 7fa19ed09700 1 -- start start 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.086+0000 7fa19ed09700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198105940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa19ed09700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198105f10 con 0x7fa198105520 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa19caa5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa19caa5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198105940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36868/0 (socket says 172.21.15.42:36868) 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa19caa5700 1 -- 172.21.15.42:0/209633136 learned_addr learned my addr 172.21.15.42:0/209633136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa19caa5700 1 -- 172.21.15.42:0/209633136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198106720 con 0x7fa198105520 2024-09-16T23:39:53.905 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa19caa5700 1 --2- 172.21.15.42:0/209633136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198105940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0098a0 tx=0x7fa18c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cca49cc366029a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa1977fe700 1 -- 172.21.15.42:0/209633136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18c004180 con 0x7fa198105520 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa1977fe700 1 -- 172.21.15.42:0/209633136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa18c0042e0 con 0x7fa198105520 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.087+0000 7fa1977fe700 1 -- 172.21.15.42:0/209633136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18c011af0 con 0x7fa198105520 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.088+0000 7fa19ed09700 1 -- 172.21.15.42:0/209633136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 msgr2=0x7fa198105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.088+0000 7fa19ed09700 1 --2- 172.21.15.42:0/209633136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198105940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0098a0 tx=0x7fa18c009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.088+0000 7fa19ed09700 1 -- 172.21.15.42:0/209633136 shutdown_connections 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.088+0000 7fa19ed09700 1 --2- 172.21.15.42:0/209633136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198105940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.088+0000 7fa19ed09700 1 -- 172.21.15.42:0/209633136 >> 172.21.15.42:0/209633136 conn(0x7fa198100ac0 msgr2=0x7fa198102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.088+0000 7fa19ed09700 1 -- 172.21.15.42:0/209633136 shutdown_connections 2024-09-16T23:39:53.906 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.088+0000 7fa19ed09700 1 -- 172.21.15.42:0/209633136 wait complete. 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19ed09700 1 Processor -- start 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19ed09700 1 -- start start 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19ed09700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198190710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19ed09700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198190c50 con 0x7fa198105520 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19caa5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198190710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19caa5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198190710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36880/0 (socket says 172.21.15.42:36880) 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19caa5700 1 -- 172.21.15.42:0/1533264673 learned_addr learned my addr 172.21.15.42:0/1533264673 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19caa5700 1 -- 172.21.15.42:0/1533264673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c009550 con 0x7fa198105520 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.089+0000 7fa19caa5700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198190710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa18c000c00 tx=0x7fa18c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.090+0000 7fa195ffb700 1 -- 172.21.15.42:0/1533264673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18c003a70 con 0x7fa198105520 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.090+0000 7fa195ffb700 1 -- 172.21.15.42:0/1533264673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa18c003bd0 con 0x7fa198105520 2024-09-16T23:39:53.907 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.090+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19818ed60 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.090+0000 7fa195ffb700 1 -- 172.21.15.42:0/1533264673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa18c0238c0 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.090+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19818f180 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.091+0000 7fa195ffb700 1 -- 172.21.15.42:0/1533264673 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa18c023b80 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.091+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1980623c0 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.091+0000 7fa195ffb700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa188038320 0x7fa18803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.091+0000 7fa195ffb700 1 -- 172.21.15.42:0/1533264673 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa18c04c220 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.091+0000 7fa197fff700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa188038320 0x7fa18803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.091+0000 7fa197fff700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa188038320 0x7fa18803a7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa184006fd0 tx=0x7fa184006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.096+0000 7fa195ffb700 1 -- 172.21.15.42:0/1533264673 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa18c010970 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.293+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa19818fa90 con 0x7fa198105520 2024-09-16T23:39:53.908 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.296+0000 7fa195ffb700 1 -- 172.21.15.42:0/1533264673 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa18c02c350 con 0x7fa198105520 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa188038320 msgr2=0x7fa18803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa188038320 0x7fa18803a7e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa184006fd0 tx=0x7fa184006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 msgr2=0x7fa198190710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198190710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa18c000c00 tx=0x7fa18c011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 shutdown_connections 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa188038320 0x7fa18803a7e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 --2- 172.21.15.42:0/1533264673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa198105520 0x7fa198190710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 >> 172.21.15.42:0/1533264673 conn(0x7fa198100ac0 msgr2=0x7fa198074040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 shutdown_connections 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:53.298+0000 7fa19ed09700 1 -- 172.21.15.42:0/1533264673 wait complete. 2024-09-16T23:39:53.909 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling the dashboard module... 2024-09-16T23:39:54.242 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:54 smithi042 ceph-mon[27914]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:39:54.242 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:54 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1533264673' entity='client.admin' 2024-09-16T23:39:55.415 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:55 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/277240986' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T23:39:55.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.349+0000 7fe2179dd700 1 Processor -- start 2024-09-16T23:39:55.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.349+0000 7fe2179dd700 1 -- start start 2024-09-16T23:39:55.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.349+0000 7fe2179dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe210109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:55.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.349+0000 7fe2179dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210079e80 con 0x7fe210107080 2024-09-16T23:39:55.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.349+0000 7fe215779700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe210109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:55.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.349+0000 7fe215779700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe210109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36894/0 (socket says 172.21.15.42:36894) 2024-09-16T23:39:55.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.349+0000 7fe215779700 1 -- 172.21.15.42:0/393612996 learned_addr learned my addr 172.21.15.42:0/393612996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:55.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.350+0000 7fe215779700 1 -- 172.21.15.42:0/393612996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2101099b0 con 0x7fe210107080 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.350+0000 7fe215779700 1 --2- 172.21.15.42:0/393612996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe210109470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0098a0 tx=0x7fe20c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0b5ae208ce04b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.350+0000 7fe207fff700 1 -- 172.21.15.42:0/393612996 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c004180 con 0x7fe210107080 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.350+0000 7fe207fff700 1 -- 172.21.15.42:0/393612996 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe20c0042e0 con 0x7fe210107080 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.350+0000 7fe207fff700 1 -- 172.21.15.42:0/393612996 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c011af0 con 0x7fe210107080 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 -- 172.21.15.42:0/393612996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 msgr2=0x7fe210109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 --2- 172.21.15.42:0/393612996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe210109470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0098a0 tx=0x7fe20c009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 -- 172.21.15.42:0/393612996 shutdown_connections 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 --2- 172.21.15.42:0/393612996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe210109470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 -- 172.21.15.42:0/393612996 >> 172.21.15.42:0/393612996 conn(0x7fe210100ad0 msgr2=0x7fe210102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 -- 172.21.15.42:0/393612996 shutdown_connections 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 -- 172.21.15.42:0/393612996 wait complete. 2024-09-16T23:39:55.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 Processor -- start 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.351+0000 7fe2179dd700 1 -- start start 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe2179dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe21019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe2179dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21019a840 con 0x7fe210107080 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe215779700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe21019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe215779700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe21019a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36902/0 (socket says 172.21.15.42:36902) 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe215779700 1 -- 172.21.15.42:0/277240986 learned_addr learned my addr 172.21.15.42:0/277240986 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe215779700 1 -- 172.21.15.42:0/277240986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c009550 con 0x7fe210107080 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe215779700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe21019a300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006ba0 tx=0x7fe20c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c003b00 con 0x7fe210107080 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe20c01a8e0 con 0x7fe210107080 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21019aa40 con 0x7fe210107080 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe20c0199c0 con 0x7fe210107080 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.352+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21019ae60 con 0x7fe210107080 2024-09-16T23:39:55.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.353+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fe20c019c80 con 0x7fe210107080 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.353+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21010a740 con 0x7fe210107080 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.354+0000 7fe2067fc700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1fc0382d0 0x7fe1fc03a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.354+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe20c04c5d0 con 0x7fe210107080 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.354+0000 7fe214f78700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1fc0382d0 0x7fe1fc03a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.354+0000 7fe214f78700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1fc0382d0 0x7fe1fc03a790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe200006fd0 tx=0x7fe200006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.358+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe20c01aa50 con 0x7fe210107080 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:54.614+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fe21010a900 con 0x7fe210107080 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.146+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fe20c00fda0 con 0x7fe210107080 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.147+0000 7fe2067fc700 1 -- 172.21.15.42:0/277240986 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fe20c01ace0 con 0x7fe210107080 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1fc0382d0 msgr2=0x7fe1fc03a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1fc0382d0 0x7fe1fc03a790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe200006fd0 tx=0x7fe200006e40 comp rx=0 tx=0).stop 2024-09-16T23:39:55.740 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 msgr2=0x7fe21019a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:55.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe21019a300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe20c006ba0 tx=0x7fe20c011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:55.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 shutdown_connections 2024-09-16T23:39:55.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1fc0382d0 0x7fe1fc03a790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:55.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 --2- 172.21.15.42:0/277240986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe210107080 0x7fe21019a300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:55.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 >> 172.21.15.42:0/277240986 conn(0x7fe210100ad0 msgr2=0x7fe2101021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:55.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 shutdown_connections 2024-09-16T23:39:55.741 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:55.150+0000 7fe2179dd700 1 -- 172.21.15.42:0/277240986 wait complete. 2024-09-16T23:39:56.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:56 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/277240986' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T23:39:56.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:56 smithi042 ceph-mon[27914]: mgrmap e9: smithi042.zrkumz(active, since 25s) 2024-09-16T23:39:57.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:39:57.068 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "smithi042.zrkumz", 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.239+0000 7f6a7dec5700 1 Processor -- start 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.239+0000 7f6a7dec5700 1 -- start start 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.239+0000 7f6a7dec5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a781094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:57.069 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.239+0000 7f6a7dec5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780745b0 con 0x7f6a781070f0 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a781094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a781094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36924/0 (socket says 172.21.15.42:36924) 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a777fe700 1 -- 172.21.15.42:0/4088820698 learned_addr learned my addr 172.21.15.42:0/4088820698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a777fe700 1 -- 172.21.15.42:0/4088820698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a78109a20 con 0x7f6a781070f0 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a777fe700 1 --2- 172.21.15.42:0/4088820698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a781094e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6a680098a0 tx=0x7f6a68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc68818890ba597 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a76ffd700 1 -- 172.21.15.42:0/4088820698 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a68004180 con 0x7f6a781070f0 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a76ffd700 1 -- 172.21.15.42:0/4088820698 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a680042e0 con 0x7f6a781070f0 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.240+0000 7f6a76ffd700 1 -- 172.21.15.42:0/4088820698 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a68011af0 con 0x7f6a781070f0 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.241+0000 7f6a7dec5700 1 -- 172.21.15.42:0/4088820698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 msgr2=0x7f6a781094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.241+0000 7f6a7dec5700 1 --2- 172.21.15.42:0/4088820698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a781094e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6a680098a0 tx=0x7f6a68009bb0 comp rx=0 tx=0).stop 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.241+0000 7f6a7dec5700 1 -- 172.21.15.42:0/4088820698 shutdown_connections 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.241+0000 7f6a7dec5700 1 --2- 172.21.15.42:0/4088820698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a781094e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.241+0000 7f6a7dec5700 1 -- 172.21.15.42:0/4088820698 >> 172.21.15.42:0/4088820698 conn(0x7f6a78100ac0 msgr2=0x7f6a78102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.241+0000 7f6a7dec5700 1 -- 172.21.15.42:0/4088820698 shutdown_connections 2024-09-16T23:39:57.070 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.241+0000 7f6a7dec5700 1 -- 172.21.15.42:0/4088820698 wait complete. 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.242+0000 7f6a7dec5700 1 Processor -- start 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.242+0000 7f6a7dec5700 1 -- start start 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.242+0000 7f6a7dec5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a7819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.242+0000 7f6a7dec5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7819a830 con 0x7f6a781070f0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.242+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a7819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.242+0000 7f6a777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a7819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36936/0 (socket says 172.21.15.42:36936) 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.242+0000 7f6a777fe700 1 -- 172.21.15.42:0/3003900157 learned_addr learned my addr 172.21.15.42:0/3003900157 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.243+0000 7f6a777fe700 1 -- 172.21.15.42:0/3003900157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a68009550 con 0x7f6a781070f0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.243+0000 7f6a777fe700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a7819a2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a68006ba0 tx=0x7f6a68011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.243+0000 7f6a757fa700 1 -- 172.21.15.42:0/3003900157 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a68003b00 con 0x7f6a781070f0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.243+0000 7f6a757fa700 1 -- 172.21.15.42:0/3003900157 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a6801a8e0 con 0x7f6a781070f0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.243+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7819aa30 con 0x7f6a781070f0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.243+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7819ae50 con 0x7f6a781070f0 2024-09-16T23:39:57.071 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.243+0000 7f6a757fa700 1 -- 172.21.15.42:0/3003900157 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a680199c0 con 0x7f6a781070f0 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.244+0000 7f6a757fa700 1 -- 172.21.15.42:0/3003900157 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f6a68019c30 con 0x7f6a781070f0 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.244+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a58005320 con 0x7f6a781070f0 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.244+0000 7f6a757fa700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 0x7f6a6403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.244+0000 7f6a757fa700 1 -- 172.21.15.42:0/3003900157 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6a6804d630 con 0x7f6a781070f0 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.244+0000 7f6a6ffff700 1 -- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 msgr2=0x7f6a6403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.244+0000 7f6a6ffff700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 0x7f6a6403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.249+0000 7f6a757fa700 1 -- 172.21.15.42:0/3003900157 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a6802b930 con 0x7f6a781070f0 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.445+0000 7f6a6ffff700 1 -- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 msgr2=0x7f6a6403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.445+0000 7f6a6ffff700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 0x7f6a6403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.507+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6a58006200 con 0x7f6a781070f0 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.507+0000 7f6a757fa700 1 -- 172.21.15.42:0/3003900157 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f6a680172b0 con 0x7f6a781070f0 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 msgr2=0x7f6a6403a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T23:39:57.072 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 0x7f6a6403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 msgr2=0x7f6a7819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a7819a2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6a68006ba0 tx=0x7f6a68011b50 comp rx=0 tx=0).stop 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 shutdown_connections 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6a64038370 0x7f6a6403a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 --2- 172.21.15.42:0/3003900157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a781070f0 0x7f6a7819a2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 >> 172.21.15.42:0/3003900157 conn(0x7f6a78100ac0 msgr2=0x7f6a781018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 shutdown_connections 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:56.509+0000 7f6a7dec5700 1 -- 172.21.15.42:0/3003900157 wait complete. 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-09-16T23:39:57.073 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 9... 2024-09-16T23:39:57.480 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:39:57 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3003900157' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T23:40:02.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: Active manager daemon smithi042.zrkumz restarted 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: Activating manager daemon smithi042.zrkumz 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: mgrmap e10: smithi042.zrkumz(active, starting, since 0.00340988s) 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:40:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:02 smithi042 ceph-mon[27914]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:40:03.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:03 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:40:03.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:03 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:40:03.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:03 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:40:03.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:03 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:40:03.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:03 smithi042 ceph-mon[27914]: mgrmap e11: smithi042.zrkumz(active, since 1.00553s) 2024-09-16T23:40:03.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-09-16T23:40:03.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T23:40:03.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T23:40:03.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-09-16T23:40:03.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.511+0000 7f47ff3a6700 1 Processor -- start 2024-09-16T23:40:03.871 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.511+0000 7f47ff3a6700 1 -- start start 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.511+0000 7f47ff3a6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f8105340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.511+0000 7f47ff3a6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8074570 con 0x7f47f8104f20 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.511+0000 7f47fd142700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f8105340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.512+0000 7f47fd142700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f8105340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36938/0 (socket says 172.21.15.42:36938) 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.512+0000 7f47fd142700 1 -- 172.21.15.42:0/3651975616 learned_addr learned my addr 172.21.15.42:0/3651975616 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.512+0000 7f47fd142700 1 -- 172.21.15.42:0/3651975616 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80746b0 con 0x7f47f8104f20 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.512+0000 7f47fd142700 1 --2- 172.21.15.42:0/3651975616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f8105340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f47f40098a0 tx=0x7f47f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec16723ebaf6c5de server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.512+0000 7f47effff700 1 -- 172.21.15.42:0/3651975616 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47f4004180 con 0x7f47f8104f20 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.512+0000 7f47effff700 1 -- 172.21.15.42:0/3651975616 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f47f40042e0 con 0x7f47f8104f20 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.512+0000 7f47effff700 1 -- 172.21.15.42:0/3651975616 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47f4011af0 con 0x7f47f8104f20 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.513+0000 7f47ff3a6700 1 -- 172.21.15.42:0/3651975616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 msgr2=0x7f47f8105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.513+0000 7f47ff3a6700 1 --2- 172.21.15.42:0/3651975616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f8105340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f47f40098a0 tx=0x7f47f4009bb0 comp rx=0 tx=0).stop 2024-09-16T23:40:03.872 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.513+0000 7f47ff3a6700 1 -- 172.21.15.42:0/3651975616 shutdown_connections 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.513+0000 7f47ff3a6700 1 --2- 172.21.15.42:0/3651975616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f8105340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.513+0000 7f47ff3a6700 1 -- 172.21.15.42:0/3651975616 >> 172.21.15.42:0/3651975616 conn(0x7f47f8100ac0 msgr2=0x7f47f8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.513+0000 7f47ff3a6700 1 -- 172.21.15.42:0/3651975616 shutdown_connections 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.513+0000 7f47ff3a6700 1 -- 172.21.15.42:0/3651975616 wait complete. 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47ff3a6700 1 Processor -- start 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47ff3a6700 1 -- start start 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47ff3a6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47ff3a6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f819ca30 con 0x7f47f8104f20 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47fd142700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47fd142700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36948/0 (socket says 172.21.15.42:36948) 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47fd142700 1 -- 172.21.15.42:0/4195460758 learned_addr learned my addr 172.21.15.42:0/4195460758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:03.873 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47fd142700 1 -- 172.21.15.42:0/4195460758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f4009550 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.514+0000 7f47fd142700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f819c4f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006ba0 tx=0x7f47f4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.515+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47f4003b00 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.515+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f819cc30 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.515+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f47f401a8e0 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.515+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f47f40199c0 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.515+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f819d050 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.516+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f47f4019c30 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.516+0000 7f47ee7fc700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.516+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47e403af40 con 0x7f47e4038370 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.516+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f47f404c740 con 0x7f47f8104f20 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.516+0000 7f47fc941700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 msgr2=0x7f47e403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.516+0000 7f47fc941700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.717+0000 7f47fc941700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 msgr2=0x7f47e403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:40:03.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:57.717+0000 7f47fc941700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:58.118+0000 7f47fc941700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 msgr2=0x7f47e403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:58.118+0000 7f47fc941700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:58.919+0000 7f47fc941700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 msgr2=0x7f47e403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:39:58.919+0000 7f47fc941700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:00.520+0000 7f47fc941700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 msgr2=0x7f47e403a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:00.520+0000 7f47fc941700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:02.177+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f47f402d620 con 0x7f47f8104f20 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:02.178+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 msgr2=0x7f47e403a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:02.178+0000 7f47ee7fc700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.179+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f47f402d9a0 con 0x7f47f8104f20 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.179+0000 7f47ee7fc700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.179+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47e403af40 con 0x7f47e4038370 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.179+0000 7f47fc941700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:03.875 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.180+0000 7f47fc941700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f47e8003a10 tx=0x7f47e8009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.180+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mgr.14164 v2:172.21.15.42:6800/2 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f47e403af40 con 0x7f47e4038370 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f47f819d830 con 0x7f47e4038370 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ee7fc700 1 -- 172.21.15.42:0/4195460758 <== mgr.14164 v2:172.21.15.42:6800/2 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f47f819d830 con 0x7f47e4038370 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 msgr2=0x7f47e403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f47e8003a10 tx=0x7f47e8009270 comp rx=0 tx=0).stop 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 msgr2=0x7f47f819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f819c4f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f47f4006ba0 tx=0x7f47f4011b50 comp rx=0 tx=0).stop 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 shutdown_connections 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f47e4038370 0x7f47e403a830 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 --2- 172.21.15.42:0/4195460758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47f8104f20 0x7f47f819c4f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.187+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 >> 172.21.15.42:0/4195460758 conn(0x7f47f8100ac0 msgr2=0x7f47f8190ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.188+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 shutdown_connections 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:03.188+0000 7f47ff3a6700 1 -- 172.21.15.42:0/4195460758 wait complete. 2024-09-16T23:40:03.876 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 9 is available 2024-09-16T23:40:03.877 INFO:teuthology.orchestra.run.smithi042.stdout:Generating a dashboard self-signed certificate... 2024-09-16T23:40:04.498 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:04 smithi042 ceph-mon[27914]: [16/Sep/2024:23:40:02] ENGINE Bus STARTING 2024-09-16T23:40:04.498 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:04 smithi042 ceph-mon[27914]: [16/Sep/2024:23:40:02] ENGINE Serving on https://172.21.15.42:7150 2024-09-16T23:40:04.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:04 smithi042 ceph-mon[27914]: [16/Sep/2024:23:40:03] ENGINE Serving on http://172.21.15.42:8765 2024-09-16T23:40:04.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:04 smithi042 ceph-mon[27914]: [16/Sep/2024:23:40:03] ENGINE Bus STARTED 2024-09-16T23:40:04.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:04 smithi042 ceph-mon[27914]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T23:40:04.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:04 smithi042 ceph-mon[27914]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T23:40:04.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:04 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:05.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T23:40:05.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.286+0000 7fe79f5df700 1 Processor -- start 2024-09-16T23:40:05.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.286+0000 7fe79f5df700 1 -- start start 2024-09-16T23:40:05.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.286+0000 7fe79f5df700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe798105300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:05.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.286+0000 7fe79f5df700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798079e60 con 0x7fe798104ee0 2024-09-16T23:40:05.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.286+0000 7fe79d37b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe798105300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:05.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.286+0000 7fe79d37b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe798105300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43728/0 (socket says 172.21.15.42:43728) 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.286+0000 7fe79d37b700 1 -- 172.21.15.42:0/3450289889 learned_addr learned my addr 172.21.15.42:0/3450289889 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.287+0000 7fe79d37b700 1 -- 172.21.15.42:0/3450289889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798105840 con 0x7fe798104ee0 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.287+0000 7fe79d37b700 1 --2- 172.21.15.42:0/3450289889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe798105300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7940098a0 tx=0x7fe794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaef8bea7df6e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.287+0000 7fe78ffff700 1 -- 172.21.15.42:0/3450289889 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe794004180 con 0x7fe798104ee0 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.287+0000 7fe78ffff700 1 -- 172.21.15.42:0/3450289889 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe7940042e0 con 0x7fe798104ee0 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.287+0000 7fe78ffff700 1 -- 172.21.15.42:0/3450289889 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe794011af0 con 0x7fe798104ee0 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 -- 172.21.15.42:0/3450289889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 msgr2=0x7fe798105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 --2- 172.21.15.42:0/3450289889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe798105300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7940098a0 tx=0x7fe794009bb0 comp rx=0 tx=0).stop 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 -- 172.21.15.42:0/3450289889 shutdown_connections 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 --2- 172.21.15.42:0/3450289889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe798105300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 -- 172.21.15.42:0/3450289889 >> 172.21.15.42:0/3450289889 conn(0x7fe798100b20 msgr2=0x7fe798102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 -- 172.21.15.42:0/3450289889 shutdown_connections 2024-09-16T23:40:05.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 -- 172.21.15.42:0/3450289889 wait complete. 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.288+0000 7fe79f5df700 1 Processor -- start 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79f5df700 1 -- start start 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79f5df700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe79819c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79f5df700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79819ca10 con 0x7fe798104ee0 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79d37b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe79819c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79d37b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe79819c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43730/0 (socket says 172.21.15.42:43730) 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79d37b700 1 -- 172.21.15.42:0/1573890318 learned_addr learned my addr 172.21.15.42:0/1573890318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79d37b700 1 -- 172.21.15.42:0/1573890318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794009550 con 0x7fe798104ee0 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79d37b700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe79819c4d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe794006ba0 tx=0x7fe794011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe78e7fc700 1 -- 172.21.15.42:0/1573890318 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe794003b00 con 0x7fe798104ee0 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79819cc10 con 0x7fe798104ee0 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe78e7fc700 1 -- 172.21.15.42:0/1573890318 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe79401a8e0 con 0x7fe798104ee0 2024-09-16T23:40:05.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.289+0000 7fe78e7fc700 1 -- 172.21.15.42:0/1573890318 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7940199c0 con 0x7fe798104ee0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.290+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79819d030 con 0x7fe798104ee0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.290+0000 7fe78e7fc700 1 -- 172.21.15.42:0/1573890318 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fe794019c80 con 0x7fe798104ee0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.290+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe798195df0 con 0x7fe798104ee0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.291+0000 7fe78e7fc700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe7840381f0 0x7fe78403a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.291+0000 7fe78e7fc700 1 -- 172.21.15.42:0/1573890318 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe79404c670 con 0x7fe798104ee0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.291+0000 7fe79cb7a700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe7840381f0 0x7fe78403a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.291+0000 7fe79cb7a700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe7840381f0 0x7fe78403a6b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe788006fd0 tx=0x7fe788006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.296+0000 7fe78e7fc700 1 -- 172.21.15.42:0/1573890318 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe794018b40 con 0x7fe798104ee0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.500+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7980611d0 con 0x7fe7840381f0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.581+0000 7fe78e7fc700 1 -- 172.21.15.42:0/1573890318 <== mgr.14164 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fe7980611d0 con 0x7fe7840381f0 2024-09-16T23:40:05.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe7840381f0 msgr2=0x7fe78403a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe7840381f0 0x7fe78403a6b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe788006fd0 tx=0x7fe788006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 msgr2=0x7fe79819c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe79819c4d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe794006ba0 tx=0x7fe794011b50 comp rx=0 tx=0).stop 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 shutdown_connections 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe7840381f0 0x7fe78403a6b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 --2- 172.21.15.42:0/1573890318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe798104ee0 0x7fe79819c4d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 >> 172.21.15.42:0/1573890318 conn(0x7fe798100b20 msgr2=0x7fe798190aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 shutdown_connections 2024-09-16T23:40:05.139 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:04.583+0000 7fe79f5df700 1 -- 172.21.15.42:0/1573890318 wait complete. 2024-09-16T23:40:05.140 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial admin user... 2024-09-16T23:40:05.827 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:05 smithi042 ceph-mon[27914]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:40:05.828 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:05 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:05.828 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:05 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:05.828 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:05 smithi042 ceph-mon[27914]: mgrmap e12: smithi042.zrkumz(active, since 2s) 2024-09-16T23:40:06.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:06 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:06.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:06 smithi042 ceph-mon[27914]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:40:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:06 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:06.920 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AJphNCXMZag0cAYU0zd4nONK0BWySx0WV7/KHOIeOADWFQR.ycKr2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726530006, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T23:40:06.920 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.599+0000 7f7643a1b700 1 Processor -- start 2024-09-16T23:40:06.920 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.599+0000 7f7643a1b700 1 -- start start 2024-09-16T23:40:06.920 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.599+0000 7f7643a1b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.599+0000 7f7643a1b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c07a8e0 con 0x7f763c07beb0 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.599+0000 7f76417b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.599+0000 7f76417b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43744/0 (socket says 172.21.15.42:43744) 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.599+0000 7f76417b7700 1 -- 172.21.15.42:0/1068924825 learned_addr learned my addr 172.21.15.42:0/1068924825 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.600+0000 7f76417b7700 1 -- 172.21.15.42:0/1068924825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c07aa20 con 0x7f763c07beb0 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.600+0000 7f76417b7700 1 --2- 172.21.15.42:0/1068924825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c07a310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76380098a0 tx=0x7f7638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cc62f80fbe983 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.600+0000 7f7633fff700 1 -- 172.21.15.42:0/1068924825 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7638004180 con 0x7f763c07beb0 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.600+0000 7f7633fff700 1 -- 172.21.15.42:0/1068924825 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f76380042e0 con 0x7f763c07beb0 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.600+0000 7f7633fff700 1 -- 172.21.15.42:0/1068924825 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7638011af0 con 0x7f763c07beb0 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 -- 172.21.15.42:0/1068924825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 msgr2=0x7f763c07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 --2- 172.21.15.42:0/1068924825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c07a310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f76380098a0 tx=0x7f7638009bb0 comp rx=0 tx=0).stop 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 -- 172.21.15.42:0/1068924825 shutdown_connections 2024-09-16T23:40:06.921 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 --2- 172.21.15.42:0/1068924825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c07a310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 -- 172.21.15.42:0/1068924825 >> 172.21.15.42:0/1068924825 conn(0x7f763c101bd0 msgr2=0x7f763c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 -- 172.21.15.42:0/1068924825 shutdown_connections 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 -- 172.21.15.42:0/1068924825 wait complete. 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 Processor -- start 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.601+0000 7f7643a1b700 1 -- start start 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f7643a1b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c19c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f7643a1b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c19ca70 con 0x7f763c07beb0 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76417b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c19c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76417b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c19c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43746/0 (socket says 172.21.15.42:43746) 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76417b7700 1 -- 172.21.15.42:0/4286262717 learned_addr learned my addr 172.21.15.42:0/4286262717 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76417b7700 1 -- 172.21.15.42:0/4286262717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638009550 con 0x7f763c07beb0 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76417b7700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c19c530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7638000c00 tx=0x7f7638011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76327fc700 1 -- 172.21.15.42:0/4286262717 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7638003a70 con 0x7f763c07beb0 2024-09-16T23:40:06.922 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76327fc700 1 -- 172.21.15.42:0/4286262717 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7638003bd0 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c19cc70 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f76327fc700 1 -- 172.21.15.42:0/4286262717 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f76380238c0 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.602+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c19d090 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.603+0000 7f76327fc700 1 -- 172.21.15.42:0/4286262717 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f763801a430 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.603+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c10c990 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.603+0000 7f76327fc700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f76280383c0 0x7f762803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.604+0000 7f76327fc700 1 -- 172.21.15.42:0/4286262717 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f763804c250 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.604+0000 7f7640fb6700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f76280383c0 0x7f762803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.604+0000 7f7640fb6700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f76280383c0 0x7f762803a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f762c006fd0 tx=0x7f762c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.608+0000 7f76327fc700 1 -- 172.21.15.42:0/4286262717 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7638010970 con 0x7f763c07beb0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:05.829+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f763c0621e0 con 0x7f76280383c0 2024-09-16T23:40:06.923 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.076+0000 7f76327fc700 1 -- 172.21.15.42:0/4286262717 <== mgr.14164 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f763c0621e0 con 0x7f76280383c0 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.078+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f76280383c0 msgr2=0x7f762803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.078+0000 7f7643a1b700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f76280383c0 0x7f762803a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f762c006fd0 tx=0x7f762c006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 msgr2=0x7f763c19c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c19c530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7638000c00 tx=0x7f7638011b50 comp rx=0 tx=0).stop 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 shutdown_connections 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f76280383c0 0x7f762803a880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 --2- 172.21.15.42:0/4286262717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f763c07beb0 0x7f763c19c530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 >> 172.21.15.42:0/4286262717 conn(0x7f763c101bd0 msgr2=0x7f763c1032c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:06.924 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 shutdown_connections 2024-09-16T23:40:06.925 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:06.079+0000 7f7643a1b700 1 -- 172.21.15.42:0/4286262717 wait complete. 2024-09-16T23:40:06.925 INFO:teuthology.orchestra.run.smithi042.stdout:Fetching dashboard port number... 2024-09-16T23:40:07.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:07 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/728604326' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T23:40:08.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T23:40:08.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.354+0000 7f6ce4338700 1 Processor -- start 2024-09-16T23:40:08.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.354+0000 7f6ce4338700 1 -- start start 2024-09-16T23:40:08.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.355+0000 7f6ce4338700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:08.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.355+0000 7f6ce4338700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc107000 con 0x7f6cdc106610 2024-09-16T23:40:08.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.355+0000 7f6ce20d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:08.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.355+0000 7f6ce20d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43758/0 (socket says 172.21.15.42:43758) 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.355+0000 7f6ce20d4700 1 -- 172.21.15.42:0/1235909712 learned_addr learned my addr 172.21.15.42:0/1235909712 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.355+0000 7f6ce20d4700 1 -- 172.21.15.42:0/1235909712 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc107810 con 0x7f6cdc106610 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.356+0000 7f6ce20d4700 1 --2- 172.21.15.42:0/1235909712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc106a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80098a0 tx=0x7f6cd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a96eac9a9c5fbca server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.356+0000 7f6ce10d2700 1 -- 172.21.15.42:0/1235909712 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cd8004180 con 0x7f6cdc106610 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.356+0000 7f6ce10d2700 1 -- 172.21.15.42:0/1235909712 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6cd80042e0 con 0x7f6cdc106610 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.356+0000 7f6ce10d2700 1 -- 172.21.15.42:0/1235909712 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cd8011af0 con 0x7f6cdc106610 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.356+0000 7f6ce4338700 1 -- 172.21.15.42:0/1235909712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 msgr2=0x7f6cdc106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.356+0000 7f6ce4338700 1 --2- 172.21.15.42:0/1235909712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc106a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80098a0 tx=0x7f6cd8009bb0 comp rx=0 tx=0).stop 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.357+0000 7f6ce4338700 1 -- 172.21.15.42:0/1235909712 shutdown_connections 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.357+0000 7f6ce4338700 1 --2- 172.21.15.42:0/1235909712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc106a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.357+0000 7f6ce4338700 1 -- 172.21.15.42:0/1235909712 >> 172.21.15.42:0/1235909712 conn(0x7f6cdc101bd0 msgr2=0x7f6cdc103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.357+0000 7f6ce4338700 1 -- 172.21.15.42:0/1235909712 shutdown_connections 2024-09-16T23:40:08.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.357+0000 7f6ce4338700 1 -- 172.21.15.42:0/1235909712 wait complete. 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.357+0000 7f6ce4338700 1 Processor -- start 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.357+0000 7f6ce4338700 1 -- start start 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce4338700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc07bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce4338700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc07c1a0 con 0x7f6cdc106610 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce20d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc07bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce20d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc07bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43772/0 (socket says 172.21.15.42:43772) 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce20d4700 1 -- 172.21.15.42:0/728604326 learned_addr learned my addr 172.21.15.42:0/728604326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce20d4700 1 -- 172.21.15.42:0/728604326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd8009550 con 0x7f6cdc106610 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce20d4700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc07bc60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006ba0 tx=0x7f6cd8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6cd37fe700 1 -- 172.21.15.42:0/728604326 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cd8003b00 con 0x7f6cdc106610 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc07a2b0 con 0x7f6cdc106610 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6cd37fe700 1 -- 172.21.15.42:0/728604326 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6cd801a8e0 con 0x7f6cdc106610 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6cd37fe700 1 -- 172.21.15.42:0/728604326 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6cd8019990 con 0x7f6cdc106610 2024-09-16T23:40:08.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.358+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc07a6d0 con 0x7f6cdc106610 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.359+0000 7f6cd37fe700 1 -- 172.21.15.42:0/728604326 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f6cd8019c00 con 0x7f6cdc106610 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.359+0000 7f6cd37fe700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6cc8038370 0x7f6cc803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.359+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc0623c0 con 0x7f6cdc106610 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.359+0000 7f6cd37fe700 1 -- 172.21.15.42:0/728604326 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f6cd804c770 con 0x7f6cdc106610 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.360+0000 7f6ce18d3700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6cc8038370 0x7f6cc803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.360+0000 7f6ce18d3700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6cc8038370 0x7f6cc803a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006fd0 tx=0x7f6ccc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.369+0000 7f6cd37fe700 1 -- 172.21.15.42:0/728604326 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6cd802b930 con 0x7f6cdc106610 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.584+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f6cdc07b0a0 con 0x7f6cdc106610 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.585+0000 7f6cd37fe700 1 -- 172.21.15.42:0/728604326 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6cd802b340 con 0x7f6cdc106610 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6cc8038370 msgr2=0x7f6cc803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6cc8038370 0x7f6cc803a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006fd0 tx=0x7f6ccc006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 msgr2=0x7f6cdc07bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:08.780 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc07bc60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8006ba0 tx=0x7f6cd8011b50 comp rx=0 tx=0).stop 2024-09-16T23:40:08.780 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 shutdown_connections 2024-09-16T23:40:08.780 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f6cc8038370 0x7f6cc803a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:08.780 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 --2- 172.21.15.42:0/728604326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cdc106610 0x7f6cdc07bc60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:08.780 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 >> 172.21.15.42:0/728604326 conn(0x7f6cdc101bd0 msgr2=0x7f6cdc103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:08.780 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 shutdown_connections 2024-09-16T23:40:08.780 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:07.587+0000 7f6ce4338700 1 -- 172.21.15.42:0/728604326 wait complete. 2024-09-16T23:40:08.781 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T23:40:08.781 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-09-16T23:40:08.789 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T23:40:08.789 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-09-16T23:40:08.789 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-09-16T23:40:08.789 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T23:40:08.791 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph Dashboard is now available at: 2024-09-16T23:40:08.791 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:08.791 INFO:teuthology.orchestra.run.smithi042.stdout: URL: https://smithi042.front.sepia.ceph.com:8443/ 2024-09-16T23:40:08.791 INFO:teuthology.orchestra.run.smithi042.stdout: User: admin 2024-09-16T23:40:08.791 INFO:teuthology.orchestra.run.smithi042.stdout: Password: 3gobwappqs 2024-09-16T23:40:08.791 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:08.792 INFO:teuthology.orchestra.run.smithi042.stdout:Saving cluster configuration to /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config directory 2024-09-16T23:40:08.792 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling autotune for osd_memory_target 2024-09-16T23:40:09.769 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:09 smithi042 ceph-mon[27914]: mgrmap e13: smithi042.zrkumz(active, since 6s) 2024-09-16T23:40:10.010 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.262+0000 7f2ef7c3f700 1 Processor -- start 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.262+0000 7f2ef7c3f700 1 -- start start 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef7c3f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef0105340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef7c3f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0074570 con 0x7f2ef0104f20 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef59db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef0105340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef59db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef0105340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43788/0 (socket says 172.21.15.42:43788) 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef59db700 1 -- 172.21.15.42:0/2139013095 learned_addr learned my addr 172.21.15.42:0/2139013095 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef59db700 1 -- 172.21.15.42:0/2139013095 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00746b0 con 0x7f2ef0104f20 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef59db700 1 --2- 172.21.15.42:0/2139013095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef0105340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0098a0 tx=0x7f2eec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6dbfe18cea9aea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef49d9700 1 -- 172.21.15.42:0/2139013095 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eec004180 con 0x7f2ef0104f20 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef49d9700 1 -- 172.21.15.42:0/2139013095 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2eec0042e0 con 0x7f2ef0104f20 2024-09-16T23:40:10.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.263+0000 7f2ef49d9700 1 -- 172.21.15.42:0/2139013095 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eec011af0 con 0x7f2ef0104f20 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.264+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/2139013095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 msgr2=0x7f2ef0105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.264+0000 7f2ef7c3f700 1 --2- 172.21.15.42:0/2139013095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef0105340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0098a0 tx=0x7f2eec009bb0 comp rx=0 tx=0).stop 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.264+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/2139013095 shutdown_connections 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.264+0000 7f2ef7c3f700 1 --2- 172.21.15.42:0/2139013095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef0105340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.264+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/2139013095 >> 172.21.15.42:0/2139013095 conn(0x7f2ef0100ac0 msgr2=0x7f2ef0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.264+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/2139013095 shutdown_connections 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.264+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/2139013095 wait complete. 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef7c3f700 1 Processor -- start 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef7c3f700 1 -- start start 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef7c3f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef7c3f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef019ca30 con 0x7f2ef0104f20 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef59db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:10.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef59db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43790/0 (socket says 172.21.15.42:43790) 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef59db700 1 -- 172.21.15.42:0/677873776 learned_addr learned my addr 172.21.15.42:0/677873776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.265+0000 7f2ef59db700 1 -- 172.21.15.42:0/677873776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec009550 con 0x7f2ef0104f20 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.266+0000 7f2ef59db700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef019c4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2eec006ba0 tx=0x7f2eec011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.266+0000 7f2ee6ffd700 1 -- 172.21.15.42:0/677873776 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eec003b00 con 0x7f2ef0104f20 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.266+0000 7f2ee6ffd700 1 -- 172.21.15.42:0/677873776 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2eec01a8e0 con 0x7f2ef0104f20 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.266+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef019cc30 con 0x7f2ef0104f20 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.266+0000 7f2ee6ffd700 1 -- 172.21.15.42:0/677873776 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2eec0199c0 con 0x7f2ef0104f20 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.266+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef019d050 con 0x7f2ef0104f20 2024-09-16T23:40:10.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.267+0000 7f2ee6ffd700 1 -- 172.21.15.42:0/677873776 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f2eec019c30 con 0x7f2ef0104f20 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.267+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef0195d60 con 0x7f2ef0104f20 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.267+0000 7f2ee6ffd700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.267+0000 7f2ee6ffd700 1 -- 172.21.15.42:0/677873776 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2eec04c6b0 con 0x7f2ef0104f20 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.267+0000 7f2ef51da700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.267+0000 7f2ef51da700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006fd0 tx=0x7f2ee0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.272+0000 7f2ee6ffd700 1 -- 172.21.15.42:0/677873776 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2eec01a430 con 0x7f2ef0104f20 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.473+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f2ef0062420 con 0x7f2ef0104f20 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.473+0000 7f2ee6ffd700 1 -- 172.21.15.42:0/677873776 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2eec018b40 con 0x7f2ef0104f20 2024-09-16T23:40:10.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 msgr2=0x7f2edc03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0006fd0 tx=0x7f2ee0006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 msgr2=0x7f2ef019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef019c4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2eec006ba0 tx=0x7f2eec011b50 comp rx=0 tx=0).stop 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 shutdown_connections 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 --2- 172.21.15.42:0/677873776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0104f20 0x7f2ef019c4f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 >> 172.21.15.42:0/677873776 conn(0x7f2ef0100ac0 msgr2=0x7f2ef0190ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 shutdown_connections 2024-09-16T23:40:10.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:09.476+0000 7f2ef7c3f700 1 -- 172.21.15.42:0/677873776 wait complete. 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f663a9c2700 1 Processor -- start 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f663a9c2700 1 -- start start 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f663a9c2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f6634105320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f663a9c2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6634074590 con 0x7f6634104f00 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f6633fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f6634105320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f6633fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f6634105320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43798/0 (socket says 172.21.15.42:43798) 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f6633fff700 1 -- 172.21.15.42:0/259427033 learned_addr learned my addr 172.21.15.42:0/259427033 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.434+0000 7f6633fff700 1 -- 172.21.15.42:0/259427033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634105860 con 0x7f6634104f00 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f6633fff700 1 --2- 172.21.15.42:0/259427033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f6634105320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66240098a0 tx=0x7f6624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1911b006cc30d264 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:11.394 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f6632ffd700 1 -- 172.21.15.42:0/259427033 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6624004180 con 0x7f6634104f00 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f6632ffd700 1 -- 172.21.15.42:0/259427033 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66240042e0 con 0x7f6634104f00 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f6632ffd700 1 -- 172.21.15.42:0/259427033 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6624011af0 con 0x7f6634104f00 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f663a9c2700 1 -- 172.21.15.42:0/259427033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 msgr2=0x7f6634105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f663a9c2700 1 --2- 172.21.15.42:0/259427033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f6634105320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f66240098a0 tx=0x7f6624009bb0 comp rx=0 tx=0).stop 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f663a9c2700 1 -- 172.21.15.42:0/259427033 shutdown_connections 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f663a9c2700 1 --2- 172.21.15.42:0/259427033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f6634105320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.435+0000 7f663a9c2700 1 -- 172.21.15.42:0/259427033 >> 172.21.15.42:0/259427033 conn(0x7f6634100ac0 msgr2=0x7f6634102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.436+0000 7f663a9c2700 1 -- 172.21.15.42:0/259427033 shutdown_connections 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.436+0000 7f663a9c2700 1 -- 172.21.15.42:0/259427033 wait complete. 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.436+0000 7f663a9c2700 1 Processor -- start 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.436+0000 7f663a9c2700 1 -- start start 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.436+0000 7f663a9c2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f663419c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.436+0000 7f663a9c2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663419ca00 con 0x7f6634104f00 2024-09-16T23:40:11.395 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f6633fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f663419c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f6633fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f663419c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43800/0 (socket says 172.21.15.42:43800) 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f6633fff700 1 -- 172.21.15.42:0/215738787 learned_addr learned my addr 172.21.15.42:0/215738787 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f6633fff700 1 -- 172.21.15.42:0/215738787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624009550 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f6633fff700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f663419c4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6624006ba0 tx=0x7f6624011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f66317fa700 1 -- 172.21.15.42:0/215738787 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6624003b00 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f66317fa700 1 -- 172.21.15.42:0/215738787 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f662401a8e0 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663419cc00 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f66317fa700 1 -- 172.21.15.42:0/215738787 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66240199c0 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.437+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663419d020 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.438+0000 7f66317fa700 1 -- 172.21.15.42:0/215738787 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f6624019c30 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.438+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6634195da0 con 0x7f6634104f00 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.438+0000 7f66317fa700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f661c038410 0x7f661c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:11.396 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.438+0000 7f66317fa700 1 -- 172.21.15.42:0/215738787 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f662404c720 con 0x7f6634104f00 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.439+0000 7f66337fe700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f661c038410 0x7f661c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.439+0000 7f66337fe700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f661c038410 0x7f661c03a8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6628006fd0 tx=0x7f6628006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.443+0000 7f66317fa700 1 -- 172.21.15.42:0/215738787 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f662402d440 con 0x7f6634104f00 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.749+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f663402cf70 con 0x7f6634104f00 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.752+0000 7f66317fa700 1 -- 172.21.15.42:0/215738787 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f66240172b0 con 0x7f6634104f00 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f661c038410 msgr2=0x7f661c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f661c038410 0x7f661c03a8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6628006fd0 tx=0x7f6628006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 msgr2=0x7f663419c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f663419c4c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6624006ba0 tx=0x7f6624011b50 comp rx=0 tx=0).stop 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 shutdown_connections 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f661c038410 0x7f661c03a8d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 --2- 172.21.15.42:0/215738787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6634104f00 0x7f663419c4c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 >> 172.21.15.42:0/215738787 conn(0x7f6634100ac0 msgr2=0x7f6634190ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:11.397 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 shutdown_connections 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-09-16T23:40:10.754+0000 7f663a9c2700 1 -- 172.21.15.42:0/215738787 wait complete. 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: ceph telemetry on 2024-09-16T23:40:11.398 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.399 INFO:teuthology.orchestra.run.smithi042.stdout:For more information see: 2024-09-16T23:40:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T23:40:11.399 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:11.399 INFO:teuthology.orchestra.run.smithi042.stdout:Bootstrap complete. 2024-09-16T23:40:11.432 INFO:tasks.cephadm:Fetching config... 2024-09-16T23:40:11.433 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:40:11.433 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T23:40:11.452 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T23:40:11.452 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:40:11.453 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T23:40:11.511 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T23:40:11.511 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:40:11.511 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/keyring of=/dev/stdout 2024-09-16T23:40:11.579 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T23:40:11.579 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:40:11.579 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T23:40:11.635 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T23:40:11.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8Lx5Nko6PWBLPOns0p2KXRniqJseOx2gkF24r0d0+0gVtYZrpUF84Zg7+HrTveO0D+VDlUW/kSWjnMcSoPK7MvvZU946VpEm5Bu98W6hIIQ0mNpFAEeVo492gaDggpOZx6TKD4HSCmpf+kW1VEyUkRDicglbcCHNTwXomsAe5sqJvE/51H31dA680ykFgWg0fEuABth5OIhjhJHze7SoQ15PpsfZVx+gmXGoOamLOylJohxGq+eV+MOegOZK/JCO5LjJFvLEpy7/FiXZhswRJTcJy+IolVPaaxJLVtJl+ypyqkDglm1IgHPR7OsfwSHJAa3syHH1J09NXEthxsAOXjdVfuRtXItGz7B3FZkEm8lCIqIZEyY0bRg5vui7WJ3V+WYbjvopbm5dUxLflzIw0c8y5rJfZvUhCxgozeBNUEqfzdhfOZQTe9akxZwVRDyPNjHXo1XpQOyR1VJ6hDQs+qiS+/EjMC+u5bCfTZEi0oUS4/nNZczkerdQCxC77Yi0= ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T23:40:11.712 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8Lx5Nko6PWBLPOns0p2KXRniqJseOx2gkF24r0d0+0gVtYZrpUF84Zg7+HrTveO0D+VDlUW/kSWjnMcSoPK7MvvZU946VpEm5Bu98W6hIIQ0mNpFAEeVo492gaDggpOZx6TKD4HSCmpf+kW1VEyUkRDicglbcCHNTwXomsAe5sqJvE/51H31dA680ykFgWg0fEuABth5OIhjhJHze7SoQ15PpsfZVx+gmXGoOamLOylJohxGq+eV+MOegOZK/JCO5LjJFvLEpy7/FiXZhswRJTcJy+IolVPaaxJLVtJl+ypyqkDglm1IgHPR7OsfwSHJAa3syHH1J09NXEthxsAOXjdVfuRtXItGz7B3FZkEm8lCIqIZEyY0bRg5vui7WJ3V+WYbjvopbm5dUxLflzIw0c8y5rJfZvUhCxgozeBNUEqfzdhfOZQTe9akxZwVRDyPNjHXo1XpQOyR1VJ6hDQs+qiS+/EjMC+u5bCfTZEi0oUS4/nNZczkerdQCxC77Yi0= ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:40:11.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8Lx5Nko6PWBLPOns0p2KXRniqJseOx2gkF24r0d0+0gVtYZrpUF84Zg7+HrTveO0D+VDlUW/kSWjnMcSoPK7MvvZU946VpEm5Bu98W6hIIQ0mNpFAEeVo492gaDggpOZx6TKD4HSCmpf+kW1VEyUkRDicglbcCHNTwXomsAe5sqJvE/51H31dA680ykFgWg0fEuABth5OIhjhJHze7SoQ15PpsfZVx+gmXGoOamLOylJohxGq+eV+MOegOZK/JCO5LjJFvLEpy7/FiXZhswRJTcJy+IolVPaaxJLVtJl+ypyqkDglm1IgHPR7OsfwSHJAa3syHH1J09NXEthxsAOXjdVfuRtXItGz7B3FZkEm8lCIqIZEyY0bRg5vui7WJ3V+WYbjvopbm5dUxLflzIw0c8y5rJfZvUhCxgozeBNUEqfzdhfOZQTe9akxZwVRDyPNjHXo1XpQOyR1VJ6hDQs+qiS+/EjMC+u5bCfTZEi0oUS4/nNZczkerdQCxC77Yi0= ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T23:40:11.774 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8Lx5Nko6PWBLPOns0p2KXRniqJseOx2gkF24r0d0+0gVtYZrpUF84Zg7+HrTveO0D+VDlUW/kSWjnMcSoPK7MvvZU946VpEm5Bu98W6hIIQ0mNpFAEeVo492gaDggpOZx6TKD4HSCmpf+kW1VEyUkRDicglbcCHNTwXomsAe5sqJvE/51H31dA680ykFgWg0fEuABth5OIhjhJHze7SoQ15PpsfZVx+gmXGoOamLOylJohxGq+eV+MOegOZK/JCO5LjJFvLEpy7/FiXZhswRJTcJy+IolVPaaxJLVtJl+ypyqkDglm1IgHPR7OsfwSHJAa3syHH1J09NXEthxsAOXjdVfuRtXItGz7B3FZkEm8lCIqIZEyY0bRg5vui7WJ3V+WYbjvopbm5dUxLflzIw0c8y5rJfZvUhCxgozeBNUEqfzdhfOZQTe9akxZwVRDyPNjHXo1XpQOyR1VJ6hDQs+qiS+/EjMC+u5bCfTZEi0oUS4/nNZczkerdQCxC77Yi0= ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:40:11.788 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T23:40:11.964 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:11 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/215738787' entity='client.admin' 2024-09-16T23:40:12.109 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:40:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.482+0000 7f32a09fc700 1 -- 172.21.15.42:0/961494997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 msgr2=0x7f329c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.482+0000 7f32a09fc700 1 --2- 172.21.15.42:0/961494997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c0ffba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f328c009910 tx=0x7f328c009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.482+0000 7f32a09fc700 1 -- 172.21.15.42:0/961494997 shutdown_connections 2024-09-16T23:40:14.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.482+0000 7f32a09fc700 1 --2- 172.21.15.42:0/961494997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c0ffba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.482+0000 7f32a09fc700 1 -- 172.21.15.42:0/961494997 >> 172.21.15.42:0/961494997 conn(0x7f329c0fad00 msgr2=0x7f329c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.482+0000 7f32a09fc700 1 -- 172.21.15.42:0/961494997 shutdown_connections 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.482+0000 7f32a09fc700 1 -- 172.21.15.42:0/961494997 wait complete. 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f32a09fc700 1 Processor -- start 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f32a09fc700 1 -- start start 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f32a09fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f32a09fc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329c198110 con 0x7f329c0ff780 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f329a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f329a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46690/0 (socket says 172.21.15.42:46690) 2024-09-16T23:40:14.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f329a59c700 1 -- 172.21.15.42:0/1886716029 learned_addr learned my addr 172.21.15.42:0/1886716029 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f329a59c700 1 -- 172.21.15.42:0/1886716029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0095f0 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.483+0000 7f329a59c700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c197bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f328c005b40 tx=0x7f328c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.484+0000 7f328b7fe700 1 -- 172.21.15.42:0/1886716029 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f328c01d070 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.484+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329c198310 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.484+0000 7f328b7fe700 1 -- 172.21.15.42:0/1886716029 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f328c00be10 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.484+0000 7f328b7fe700 1 -- 172.21.15.42:0/1886716029 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f328c00f460 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.484+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329c1987b0 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.485+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329c1917b0 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.485+0000 7f328b7fe700 1 -- 172.21.15.42:0/1886716029 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f328c0227e0 con 0x7f329c0ff780 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.485+0000 7f328b7fe700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32840383c0 0x7f328403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:14.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.485+0000 7f328b7fe700 1 -- 172.21.15.42:0/1886716029 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f328c04c480 con 0x7f329c0ff780 2024-09-16T23:40:14.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.485+0000 7f3299d9b700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32840383c0 0x7f328403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:14.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.486+0000 7f3299d9b700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32840383c0 0x7f328403a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3290006fd0 tx=0x7f3290006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:14.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.490+0000 7f328b7fe700 1 -- 172.21.15.42:0/1886716029 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f328c02bba0 con 0x7f329c0ff780 2024-09-16T23:40:14.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:14.564 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:14.565 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:40:14.565 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:14.565 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:40:14.565 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T23:40:14.565 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:40:14.565 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:14 smithi042 ceph-mon[27914]: Deploying daemon ceph-exporter.smithi042 on smithi042 2024-09-16T23:40:14.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.691+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f329c02cc70 con 0x7f329c0ff780 2024-09-16T23:40:14.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.694+0000 7f328b7fe700 1 -- 172.21.15.42:0/1886716029 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f328c027020 con 0x7f329c0ff780 2024-09-16T23:40:14.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32840383c0 msgr2=0x7f328403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32840383c0 0x7f328403a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3290006fd0 tx=0x7f3290006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 msgr2=0x7f329c197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c197bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f328c005b40 tx=0x7f328c004740 comp rx=0 tx=0).stop 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 shutdown_connections 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32840383c0 0x7f328403a880 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 --2- 172.21.15.42:0/1886716029 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f329c0ff780 0x7f329c197bd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 >> 172.21.15.42:0/1886716029 conn(0x7f329c0fad00 msgr2=0x7f329c069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 shutdown_connections 2024-09-16T23:40:14.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:14.696+0000 7f32a09fc700 1 -- 172.21.15.42:0/1886716029 wait complete. 2024-09-16T23:40:15.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:15 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1886716029' entity='client.admin' 2024-09-16T23:40:15.940 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T23:40:15.941 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T23:40:16.862 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:40:19.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.947+0000 7f82ba622700 1 -- 172.21.15.42:0/1917739734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 msgr2=0x7f82b40ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.947+0000 7f82ba622700 1 --2- 172.21.15.42:0/1917739734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b40ffba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009910 tx=0x7f82a4009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.947+0000 7f82ba622700 1 -- 172.21.15.42:0/1917739734 shutdown_connections 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.947+0000 7f82ba622700 1 --2- 172.21.15.42:0/1917739734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b40ffba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.947+0000 7f82ba622700 1 -- 172.21.15.42:0/1917739734 >> 172.21.15.42:0/1917739734 conn(0x7f82b40fad20 msgr2=0x7f82b40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.947+0000 7f82ba622700 1 -- 172.21.15.42:0/1917739734 shutdown_connections 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.947+0000 7f82ba622700 1 -- 172.21.15.42:0/1917739734 wait complete. 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82ba622700 1 Processor -- start 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82ba622700 1 -- start start 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82ba622700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b4068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82ba622700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b41986f0 con 0x7f82b40ff780 2024-09-16T23:40:19.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82b3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b4068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82b3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b4068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46712/0 (socket says 172.21.15.42:46712) 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82b3fff700 1 -- 172.21.15.42:0/2231738241 learned_addr learned my addr 172.21.15.42:0/2231738241 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.948+0000 7f82b3fff700 1 -- 172.21.15.42:0/2231738241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a40095f0 con 0x7f82b40ff780 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.949+0000 7f82b3fff700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b4068af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005b40 tx=0x7f82a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.949+0000 7f82b17fa700 1 -- 172.21.15.42:0/2231738241 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82a401d070 con 0x7f82b40ff780 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.949+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b4198830 con 0x7f82b40ff780 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.949+0000 7f82b17fa700 1 -- 172.21.15.42:0/2231738241 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f82a400be10 con 0x7f82b40ff780 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.949+0000 7f82b17fa700 1 -- 172.21.15.42:0/2231738241 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f82a400f460 con 0x7f82b40ff780 2024-09-16T23:40:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.949+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b4198cd0 con 0x7f82b40ff780 2024-09-16T23:40:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.950+0000 7f82b17fa700 1 -- 172.21.15.42:0/2231738241 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f82a400f6d0 con 0x7f82b40ff780 2024-09-16T23:40:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.950+0000 7f82b17fa700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f829c0383c0 0x7f829c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.950+0000 7f82b17fa700 1 -- 172.21.15.42:0/2231738241 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f82a404c450 con 0x7f82b40ff780 2024-09-16T23:40:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.950+0000 7f82b37fe700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f829c0383c0 0x7f829c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.950+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b418e3a0 con 0x7f82b40ff780 2024-09-16T23:40:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.951+0000 7f82b37fe700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f829c0383c0 0x7f829c03a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006fd0 tx=0x7f82a8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:18.955+0000 7f82b17fa700 1 -- 172.21.15.42:0/2231738241 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f82a4027070 con 0x7f82b40ff780 2024-09-16T23:40:19.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.168+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f82b4061260 con 0x7f829c0383c0 2024-09-16T23:40:19.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.172+0000 7f82b17fa700 1 -- 172.21.15.42:0/2231738241 <== mgr.14164 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f82b4061260 con 0x7f829c0383c0 2024-09-16T23:40:19.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f829c0383c0 msgr2=0x7f829c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:19.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f829c0383c0 0x7f829c03a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f82a8006fd0 tx=0x7f82a8006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:19.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 msgr2=0x7f82b4068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:19.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b4068af0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f82a4005b40 tx=0x7f82a4004740 comp rx=0 tx=0).stop 2024-09-16T23:40:19.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 shutdown_connections 2024-09-16T23:40:19.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f829c0383c0 0x7f829c03a880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:19.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 --2- 172.21.15.42:0/2231738241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b40ff780 0x7f82b4068af0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:19.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 >> 172.21.15.42:0/2231738241 conn(0x7f82b40fad20 msgr2=0x7f82b40fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:19.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 shutdown_connections 2024-09-16T23:40:19.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:19.174+0000 7f82ba622700 1 -- 172.21.15.42:0/2231738241 wait complete. 2024-09-16T23:40:19.762 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi043 2024-09-16T23:40:19.763 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:40:19.763 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/etc/ceph/ceph.conf 2024-09-16T23:40:19.783 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:40:19.784 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:40:19.843 INFO:tasks.cephadm:Adding host smithi043 to orchestrator... 2024-09-16T23:40:19.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch host add smithi043 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:40:20.307 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: Deploying daemon crash.smithi042 on smithi042 2024-09-16T23:40:20.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:40:20.308 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:20 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:20.541 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:40:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.858+0000 7fd464b66700 1 -- 172.21.15.42:0/2923171581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 msgr2=0x7fd460100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.858+0000 7fd464b66700 1 --2- 172.21.15.42:0/2923171581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd460100cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd450009910 tx=0x7fd450009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.859+0000 7fd464b66700 1 -- 172.21.15.42:0/2923171581 shutdown_connections 2024-09-16T23:40:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.859+0000 7fd464b66700 1 --2- 172.21.15.42:0/2923171581 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd460100cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.859+0000 7fd464b66700 1 -- 172.21.15.42:0/2923171581 >> 172.21.15.42:0/2923171581 conn(0x7fd4600fbe20 msgr2=0x7fd4600fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:22.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.859+0000 7fd464b66700 1 -- 172.21.15.42:0/2923171581 shutdown_connections 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.859+0000 7fd464b66700 1 -- 172.21.15.42:0/2923171581 wait complete. 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.859+0000 7fd464b66700 1 Processor -- start 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.859+0000 7fd464b66700 1 -- start start 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd464b66700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd4601959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd464b66700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd460195ee0 con 0x7fd4601008a0 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd45e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd4601959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd45e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd4601959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40344/0 (socket says 172.21.15.42:40344) 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd45e59c700 1 -- 172.21.15.42:0/4212852654 learned_addr learned my addr 172.21.15.42:0/4212852654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd45e59c700 1 -- 172.21.15.42:0/4212852654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4500095f0 con 0x7fd4601008a0 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd45e59c700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd4601959a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd450005b40 tx=0x7fd450004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd44f7fe700 1 -- 172.21.15.42:0/4212852654 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd45001d070 con 0x7fd4601008a0 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4601960e0 con 0x7fd4601008a0 2024-09-16T23:40:22.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd44f7fe700 1 -- 172.21.15.42:0/4212852654 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd45000be10 con 0x7fd4601008a0 2024-09-16T23:40:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd44f7fe700 1 -- 172.21.15.42:0/4212852654 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd45000f460 con 0x7fd4601008a0 2024-09-16T23:40:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.860+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd460196580 con 0x7fd4601008a0 2024-09-16T23:40:22.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.861+0000 7fd44f7fe700 1 -- 172.21.15.42:0/4212852654 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fd45000f6d0 con 0x7fd4601008a0 2024-09-16T23:40:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.861+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46018f4c0 con 0x7fd4601008a0 2024-09-16T23:40:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.861+0000 7fd44f7fe700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd4480383c0 0x7fd44803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.862+0000 7fd44f7fe700 1 -- 172.21.15.42:0/4212852654 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fd45004c3d0 con 0x7fd4601008a0 2024-09-16T23:40:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.862+0000 7fd45dd9b700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd4480383c0 0x7fd44803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:22.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.862+0000 7fd45dd9b700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd4480383c0 0x7fd44803a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd454006fd0 tx=0x7fd454006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:22.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:22.866+0000 7fd44f7fe700 1 -- 172.21.15.42:0/4212852654 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd45002a360 con 0x7fd4601008a0 2024-09-16T23:40:23.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:23.076+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4600611d0 con 0x7fd4480383c0 2024-09-16T23:40:23.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:23 smithi042 ceph-mon[27914]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:23.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:23 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:23.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:23 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:23.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:23 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:23.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:23 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:23.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:23 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:24.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:24 smithi042 ceph-mon[27914]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:40:24.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:24 smithi042 ceph-mon[27914]: Deploying daemon node-exporter.smithi042 on smithi042 2024-09-16T23:40:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:25 smithi042 ceph-mon[27914]: Deploying cephadm binary to smithi043 2024-09-16T23:40:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:25 smithi042 ceph-mon[27914]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:25.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.630+0000 7fd44f7fe700 1 -- 172.21.15.42:0/4212852654 <== mgr.14164 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fd4600611d0 con 0x7fd4480383c0 2024-09-16T23:40:25.630 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi043' with addr '172.21.15.43' 2024-09-16T23:40:25.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd4480383c0 msgr2=0x7fd44803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd4480383c0 0x7fd44803a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd454006fd0 tx=0x7fd454006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 msgr2=0x7fd4601959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd4601959a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd450005b40 tx=0x7fd450004740 comp rx=0 tx=0).stop 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 shutdown_connections 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fd4480383c0 0x7fd44803a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 --2- 172.21.15.42:0/4212852654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4601008a0 0x7fd4601959a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 >> 172.21.15.42:0/4212852654 conn(0x7fd4600fbe20 msgr2=0x7fd4600fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 shutdown_connections 2024-09-16T23:40:25.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:25.633+0000 7fd464b66700 1 -- 172.21.15.42:0/4212852654 wait complete. 2024-09-16T23:40:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:26 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:26 smithi042 ceph-mon[27914]: Added host smithi043 2024-09-16T23:40:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:26 smithi042 ceph-mon[27914]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:26.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T23:40:27.270 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:40:28.512 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:28 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:28.512 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:28 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:28.512 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:28 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:28.513 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:28 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:28.513 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:28 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:28.513 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:28 smithi042 ceph-mon[27914]: Deploying daemon alertmanager.smithi042 on smithi042 2024-09-16T23:40:28.513 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:28 smithi042 ceph-mon[27914]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.848+0000 7fb121738700 1 -- 172.21.15.42:0/2243735912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 msgr2=0x7fb11c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.848+0000 7fb121738700 1 --2- 172.21.15.42:0/2243735912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c0ffba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009910 tx=0x7fb10c009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.848+0000 7fb121738700 1 -- 172.21.15.42:0/2243735912 shutdown_connections 2024-09-16T23:40:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.848+0000 7fb121738700 1 --2- 172.21.15.42:0/2243735912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c0ffba0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.848+0000 7fb121738700 1 -- 172.21.15.42:0/2243735912 >> 172.21.15.42:0/2243735912 conn(0x7fb11c0fad00 msgr2=0x7fb11c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.848+0000 7fb121738700 1 -- 172.21.15.42:0/2243735912 shutdown_connections 2024-09-16T23:40:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.848+0000 7fb121738700 1 -- 172.21.15.42:0/2243735912 wait complete. 2024-09-16T23:40:28.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.849+0000 7fb121738700 1 Processor -- start 2024-09-16T23:40:28.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.849+0000 7fb121738700 1 -- start start 2024-09-16T23:40:28.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.850+0000 7fb121738700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c1948d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:28.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.850+0000 7fb121738700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c194e10 con 0x7fb11c0ff780 2024-09-16T23:40:28.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.850+0000 7fb11affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c1948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.850+0000 7fb11affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c1948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40360/0 (socket says 172.21.15.42:40360) 2024-09-16T23:40:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.850+0000 7fb11affd700 1 -- 172.21.15.42:0/3045323945 learned_addr learned my addr 172.21.15.42:0/3045323945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.851+0000 7fb11affd700 1 -- 172.21.15.42:0/3045323945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0095f0 con 0x7fb11c0ff780 2024-09-16T23:40:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.851+0000 7fb11affd700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c1948d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb10c005b40 tx=0x7fb10c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.851+0000 7fb103fff700 1 -- 172.21.15.42:0/3045323945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb10c01f070 con 0x7fb11c0ff780 2024-09-16T23:40:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.851+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c195010 con 0x7fb11c0ff780 2024-09-16T23:40:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.851+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c1954b0 con 0x7fb11c0ff780 2024-09-16T23:40:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.851+0000 7fb103fff700 1 -- 172.21.15.42:0/3045323945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb10c00be10 con 0x7fb11c0ff780 2024-09-16T23:40:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.851+0000 7fb103fff700 1 -- 172.21.15.42:0/3045323945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb10c024620 con 0x7fb11c0ff780 2024-09-16T23:40:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.852+0000 7fb103fff700 1 -- 172.21.15.42:0/3045323945 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fb10c00f460 con 0x7fb11c0ff780 2024-09-16T23:40:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.852+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c1055c0 con 0x7fb11c0ff780 2024-09-16T23:40:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.852+0000 7fb103fff700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb104038410 0x7fb10403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.852+0000 7fb103fff700 1 -- 172.21.15.42:0/3045323945 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fb10c04e7d0 con 0x7fb11c0ff780 2024-09-16T23:40:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.853+0000 7fb11a7fc700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb104038410 0x7fb10403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.853+0000 7fb11a7fc700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb104038410 0x7fb10403a8d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb110006fd0 tx=0x7fb110006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:28.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:28.857+0000 7fb103fff700 1 -- 172.21.15.42:0/3045323945 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb10c02c950 con 0x7fb11c0ff780 2024-09-16T23:40:29.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.070+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb11c068880 con 0x7fb104038410 2024-09-16T23:40:29.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.071+0000 7fb103fff700 1 -- 172.21.15.42:0/3045323945 <== mgr.14164 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fb11c068880 con 0x7fb104038410 2024-09-16T23:40:29.068 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:40:29.068 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}] 2024-09-16T23:40:29.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb104038410 msgr2=0x7fb10403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:29.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb104038410 0x7fb10403a8d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb110006fd0 tx=0x7fb110006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 msgr2=0x7fb11c1948d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c1948d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb10c005b40 tx=0x7fb10c004740 comp rx=0 tx=0).stop 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 shutdown_connections 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb104038410 0x7fb10403a8d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 --2- 172.21.15.42:0/3045323945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb11c0ff780 0x7fb11c1948d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 >> 172.21.15.42:0/3045323945 conn(0x7fb11c0fad00 msgr2=0x7fb11c0fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 shutdown_connections 2024-09-16T23:40:29.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:29.073+0000 7fb121738700 1 -- 172.21.15.42:0/3045323945 wait complete. 2024-09-16T23:40:29.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:29 smithi042 ceph-mon[27914]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T23:40:30.163 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T23:40:30.163 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T23:40:30.363 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:40:30.484 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:30 smithi042 ceph-mon[27914]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:33.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.361+0000 7facaf893700 1 -- 172.21.15.42:0/3379231828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 msgr2=0x7faca80ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.361+0000 7facaf893700 1 --2- 172.21.15.42:0/3379231828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca80ffba0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b0a0 tx=0x7fac9800b3b0 comp rx=0 tx=0).stop 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.361+0000 7facaf893700 1 -- 172.21.15.42:0/3379231828 shutdown_connections 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.361+0000 7facaf893700 1 --2- 172.21.15.42:0/3379231828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca80ffba0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.361+0000 7facaf893700 1 -- 172.21.15.42:0/3379231828 >> 172.21.15.42:0/3379231828 conn(0x7faca80fad20 msgr2=0x7faca80fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.361+0000 7facaf893700 1 -- 172.21.15.42:0/3379231828 shutdown_connections 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.361+0000 7facaf893700 1 -- 172.21.15.42:0/3379231828 wait complete. 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facaf893700 1 Processor -- start 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facaf893700 1 -- start start 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facaf893700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca81948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facaf893700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8194de0 con 0x7faca80ff780 2024-09-16T23:40:33.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facad62f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca81948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facad62f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca81948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56416/0 (socket says 172.21.15.42:56416) 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facad62f700 1 -- 172.21.15.42:0/961569382 learned_addr learned my addr 172.21.15.42:0/961569382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.362+0000 7facad62f700 1 -- 172.21.15.42:0/961569382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98009d00 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.363+0000 7facad62f700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca81948a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b830 tx=0x7fac9800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.363+0000 7faca67fc700 1 -- 172.21.15.42:0/961569382 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac9800e040 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.363+0000 7faca67fc700 1 -- 172.21.15.42:0/961569382 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fac98012af0 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.363+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca8194fe0 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.363+0000 7faca67fc700 1 -- 172.21.15.42:0/961569382 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac98019ae0 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.363+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca8195460 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.364+0000 7faca67fc700 1 -- 172.21.15.42:0/961569382 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fac98012c60 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.364+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca8062430 con 0x7faca80ff780 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.364+0000 7faca67fc700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fac90038410 0x7fac9003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:33.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.364+0000 7faca67fc700 1 -- 172.21.15.42:0/961569382 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fac9804def0 con 0x7faca80ff780 2024-09-16T23:40:33.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.364+0000 7facace2e700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fac90038410 0x7fac9003a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:33.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.364+0000 7facace2e700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fac90038410 0x7fac9003a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fac9c006fd0 tx=0x7fac9c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:33.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.369+0000 7faca67fc700 1 -- 172.21.15.42:0/961569382 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faca8062430 con 0x7faca80ff780 2024-09-16T23:40:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:33 smithi042 ceph-mon[27914]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:33 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:33.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:33.573+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7faca818dd80 con 0x7faca80ff780 2024-09-16T23:40:34.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.200+0000 7faca67fc700 1 -- 172.21.15.42:0/961569382 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fac9802a520 con 0x7faca80ff780 2024-09-16T23:40:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fac90038410 msgr2=0x7fac9003a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fac90038410 0x7fac9003a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fac9c006fd0 tx=0x7fac9c006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 msgr2=0x7faca81948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca81948a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fac9800b830 tx=0x7fac9800bd40 comp rx=0 tx=0).stop 2024-09-16T23:40:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 shutdown_connections 2024-09-16T23:40:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fac90038410 0x7fac9003a8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 --2- 172.21.15.42:0/961569382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faca80ff780 0x7faca81948a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:34.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 >> 172.21.15.42:0/961569382 conn(0x7faca80fad20 msgr2=0x7faca80fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:34.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 shutdown_connections 2024-09-16T23:40:34.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:40:34.202+0000 7facaf893700 1 -- 172.21.15.42:0/961569382 wait complete. 2024-09-16T23:40:34.200 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-09-16T23:40:34.492 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:34 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/961569382' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T23:40:34.770 INFO:tasks.cephadm:Adding mon.smithi042 on smithi042 2024-09-16T23:40:34.770 INFO:tasks.cephadm:Adding mon.smithi043 on smithi043 2024-09-16T23:40:34.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch apply mon '2;smithi042:172.21.15.42=smithi042;smithi043:172.21.15.43=smithi043' 2024-09-16T23:40:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:35.026 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:35.388 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:35 smithi042 ceph-mon[27914]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:35.388 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:35 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/961569382' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T23:40:35.388 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:35 smithi042 ceph-mon[27914]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T23:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.435+0000 7fefd1820700 1 -- 172.21.15.43:0/1404341198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 msgr2=0x7fefcc100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.435+0000 7fefd1820700 1 --2- 172.21.15.43:0/1404341198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc100cc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009910 tx=0x7fefbc009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.435+0000 7fefd1820700 1 -- 172.21.15.43:0/1404341198 shutdown_connections 2024-09-16T23:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.435+0000 7fefd1820700 1 --2- 172.21.15.43:0/1404341198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc100cc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.435+0000 7fefd1820700 1 -- 172.21.15.43:0/1404341198 >> 172.21.15.43:0/1404341198 conn(0x7fefcc0fbe20 msgr2=0x7fefcc0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:36.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.435+0000 7fefd1820700 1 -- 172.21.15.43:0/1404341198 shutdown_connections 2024-09-16T23:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.435+0000 7fefd1820700 1 -- 172.21.15.43:0/1404341198 wait complete. 2024-09-16T23:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.436+0000 7fefd1820700 1 Processor -- start 2024-09-16T23:40:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.436+0000 7fefd1820700 1 -- start start 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.436+0000 7fefd1820700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.436+0000 7fefd1820700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc075070 con 0x7fefcc1008a0 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.436+0000 7fefcaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.436+0000 7fefcaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:45508/0 (socket says 172.21.15.43:45508) 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.436+0000 7fefcaffd700 1 -- 172.21.15.43:0/2260160224 learned_addr learned my addr 172.21.15.43:0/2260160224 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.437+0000 7fefcaffd700 1 -- 172.21.15.43:0/2260160224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0095f0 con 0x7fefcc1008a0 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.437+0000 7fefcaffd700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc074b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005b40 tx=0x7fefbc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.437+0000 7fefd081e700 1 -- 172.21.15.43:0/2260160224 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefbc01d070 con 0x7fefcc1008a0 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.437+0000 7fefd081e700 1 -- 172.21.15.43:0/2260160224 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fefbc00be10 con 0x7fefcc1008a0 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.437+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc073180 con 0x7fefcc1008a0 2024-09-16T23:40:36.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.437+0000 7fefd081e700 1 -- 172.21.15.43:0/2260160224 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fefbc00f460 con 0x7fefcc1008a0 2024-09-16T23:40:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.437+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc073620 con 0x7fefcc1008a0 2024-09-16T23:40:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.438+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc0623c0 con 0x7fefcc1008a0 2024-09-16T23:40:36.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.438+0000 7fefd081e700 1 -- 172.21.15.43:0/2260160224 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fefbc0227e0 con 0x7fefcc1008a0 2024-09-16T23:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.438+0000 7fefd081e700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fefb40383c0 0x7fefb403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.438+0000 7fefd081e700 1 -- 172.21.15.43:0/2260160224 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fefbc04c440 con 0x7fefcc1008a0 2024-09-16T23:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.439+0000 7fefca7fc700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fefb40383c0 0x7fefb403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:36.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.439+0000 7fefca7fc700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fefb40383c0 0x7fefb403a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fefc0006fd0 tx=0x7fefc0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:36.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.444+0000 7fefd081e700 1 -- 172.21.15.43:0/2260160224 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fefbc02bba0 con 0x7fefcc1008a0 2024-09-16T23:40:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.653+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi042:172.21.15.42=smithi042;smithi043:172.21.15.43=smithi043", "target": ["mon-mgr", ""]}) v1 -- 0x7fefcc0739d0 con 0x7fefb40383c0 2024-09-16T23:40:36.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.659+0000 7fefd081e700 1 -- 172.21.15.43:0/2260160224 <== mgr.14164 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fefcc0739d0 con 0x7fefb40383c0 2024-09-16T23:40:36.656 INFO:teuthology.orchestra.run.smithi043.stdout:Scheduled mon update... 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fefb40383c0 msgr2=0x7fefb403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fefb40383c0 0x7fefb403a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fefc0006fd0 tx=0x7fefc0006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 msgr2=0x7fefcc074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc074b30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005b40 tx=0x7fefbc004740 comp rx=0 tx=0).stop 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 shutdown_connections 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fefb40383c0 0x7fefb403a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 --2- 172.21.15.43:0/2260160224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefcc1008a0 0x7fefcc074b30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 >> 172.21.15.43:0/2260160224 conn(0x7fefcc0fbe20 msgr2=0x7fefcc0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 shutdown_connections 2024-09-16T23:40:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:36.661+0000 7fefd1820700 1 -- 172.21.15.43:0/2260160224 wait complete. 2024-09-16T23:40:37.206 DEBUG:teuthology.orchestra.run.smithi043:mon.smithi043> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi043.service 2024-09-16T23:40:37.209 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:40:37.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:40:37.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:37 smithi042 ceph-mon[27914]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:37.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:37 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:37.458 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi042:172.21.15.42=smithi042;smithi043:172.21.15.43=smithi043", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:40:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: Saving service mon spec with placement smithi042:172.21.15.42=smithi042;smithi043:172.21.15.43=smithi043;count:2 2024-09-16T23:40:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T23:40:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:38 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.992+0000 7f84a3b40700 1 -- 172.21.15.43:0/2396004505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 msgr2=0x7f849c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.992+0000 7f84a3b40700 1 --2- 172.21.15.43:0/2396004505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c0ffba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8498009910 tx=0x7f8498009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.992+0000 7f84a3b40700 1 -- 172.21.15.43:0/2396004505 shutdown_connections 2024-09-16T23:40:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.992+0000 7f84a3b40700 1 --2- 172.21.15.43:0/2396004505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c0ffba0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.992+0000 7f84a3b40700 1 -- 172.21.15.43:0/2396004505 >> 172.21.15.43:0/2396004505 conn(0x7f849c0fad00 msgr2=0x7f849c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.992+0000 7f84a3b40700 1 -- 172.21.15.43:0/2396004505 shutdown_connections 2024-09-16T23:40:38.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.992+0000 7f84a3b40700 1 -- 172.21.15.43:0/2396004505 wait complete. 2024-09-16T23:40:38.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.993+0000 7f84a3b40700 1 Processor -- start 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.993+0000 7f84a3b40700 1 -- start start 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.993+0000 7f84a3b40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.993+0000 7f84a3b40700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c195ef0 con 0x7f849c0ff780 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f84a18dc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f84a18dc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c1959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:45530/0 (socket says 172.21.15.43:45530) 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f84a18dc700 1 -- 172.21.15.43:0/1389843416 learned_addr learned my addr 172.21.15.43:0/1389843416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f84a18dc700 1 -- 172.21.15.43:0/1389843416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980095f0 con 0x7f849c0ff780 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f84a18dc700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c1959b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8498005b40 tx=0x7f8498004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f8492ffd700 1 -- 172.21.15.43:0/1389843416 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f849801d070 con 0x7f849c0ff780 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c1960f0 con 0x7f849c0ff780 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f8492ffd700 1 -- 172.21.15.43:0/1389843416 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f849800be10 con 0x7f849c0ff780 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f8492ffd700 1 -- 172.21.15.43:0/1389843416 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f849800f460 con 0x7f849c0ff780 2024-09-16T23:40:38.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.994+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c196590 con 0x7f849c0ff780 2024-09-16T23:40:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.995+0000 7f8492ffd700 1 -- 172.21.15.43:0/1389843416 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f84980227e0 con 0x7f849c0ff780 2024-09-16T23:40:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.995+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c18f3b0 con 0x7f849c0ff780 2024-09-16T23:40:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.995+0000 7f8492ffd700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f84880383c0 0x7f848803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:38.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.995+0000 7f8492ffd700 1 -- 172.21.15.43:0/1389843416 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f849804c500 con 0x7f849c0ff780 2024-09-16T23:40:38.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.996+0000 7f84a10db700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f84880383c0 0x7f848803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:38.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:38.996+0000 7f84a10db700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f84880383c0 0x7f848803a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f848c006fd0 tx=0x7f848c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:38.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.000+0000 7f8492ffd700 1 -- 172.21.15.43:0/1389843416 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f849802bba0 con 0x7f849c0ff780 2024-09-16T23:40:39.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:39 smithi042 ceph-mon[27914]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T23:40:39.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:39 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T23:40:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:39 smithi042 ceph-mon[27914]: Deploying daemon grafana.smithi042 on smithi042 2024-09-16T23:40:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:39 smithi042 ceph-mon[27914]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:39.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.291+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f849c02cc70 con 0x7f849c0ff780 2024-09-16T23:40:39.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.291+0000 7f8492ffd700 1 -- 172.21.15.43:0/1389843416 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8498027020 con 0x7f849c0ff780 2024-09-16T23:40:39.289 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:40:39.289 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.293+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f84880383c0 msgr2=0x7f848803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.293+0000 7f84a3b40700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f84880383c0 0x7f848803a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f848c006fd0 tx=0x7f848c006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.293+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 msgr2=0x7f849c1959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.293+0000 7f84a3b40700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c1959b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8498005b40 tx=0x7f8498004740 comp rx=0 tx=0).stop 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.294+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 shutdown_connections 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.294+0000 7f84a3b40700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f84880383c0 0x7f848803a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.294+0000 7f84a3b40700 1 --2- 172.21.15.43:0/1389843416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c0ff780 0x7f849c1959b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:39.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.294+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 >> 172.21.15.43:0/1389843416 conn(0x7f849c0fad00 msgr2=0x7f849c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:39.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.294+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 shutdown_connections 2024-09-16T23:40:39.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:39.294+0000 7f84a3b40700 1 -- 172.21.15.43:0/1389843416 wait complete. 2024-09-16T23:40:39.292 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:40:40.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:40 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1389843416' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:40:40.899 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:40:40.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:40:41.108 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:41 smithi042 ceph-mon[27914]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.602+0000 7fa7bf241700 1 -- 172.21.15.43:0/1405163967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 msgr2=0x7fa7b8103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.602+0000 7fa7bf241700 1 --2- 172.21.15.43:0/1405163967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8103700 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa7a8009910 tx=0x7fa7a8009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.602+0000 7fa7bf241700 1 -- 172.21.15.43:0/1405163967 shutdown_connections 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.602+0000 7fa7bf241700 1 --2- 172.21.15.43:0/1405163967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8103700 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.602+0000 7fa7bf241700 1 -- 172.21.15.43:0/1405163967 >> 172.21.15.43:0/1405163967 conn(0x7fa7b80fad20 msgr2=0x7fa7b80fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.602+0000 7fa7bf241700 1 -- 172.21.15.43:0/1405163967 shutdown_connections 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.602+0000 7fa7bf241700 1 -- 172.21.15.43:0/1405163967 wait complete. 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.603+0000 7fa7bf241700 1 Processor -- start 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.603+0000 7fa7bf241700 1 -- start start 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.603+0000 7fa7bf241700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:42.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.603+0000 7fa7bf241700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8198070 con 0x7fa7b8101310 2024-09-16T23:40:42.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.603+0000 7fa7bcfdd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:42.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.603+0000 7fa7bcfdd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:45544/0 (socket says 172.21.15.43:45544) 2024-09-16T23:40:42.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.603+0000 7fa7bcfdd700 1 -- 172.21.15.43:0/2671167780 learned_addr learned my addr 172.21.15.43:0/2671167780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:40:42.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.604+0000 7fa7bcfdd700 1 -- 172.21.15.43:0/2671167780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80095f0 con 0x7fa7b8101310 2024-09-16T23:40:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.604+0000 7fa7bcfdd700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8197b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80092e0 tx=0x7fa7a8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.604+0000 7fa7b5ffb700 1 -- 172.21.15.43:0/2671167780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7a801d070 con 0x7fa7b8101310 2024-09-16T23:40:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.604+0000 7fa7b5ffb700 1 -- 172.21.15.43:0/2671167780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa7a800be10 con 0x7fa7b8101310 2024-09-16T23:40:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.604+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b8198270 con 0x7fa7b8101310 2024-09-16T23:40:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.604+0000 7fa7b5ffb700 1 -- 172.21.15.43:0/2671167780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7a800f460 con 0x7fa7b8101310 2024-09-16T23:40:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.604+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b8198710 con 0x7fa7b8101310 2024-09-16T23:40:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.605+0000 7fa7b5ffb700 1 -- 172.21.15.43:0/2671167780 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fa7a8022470 con 0x7fa7b8101310 2024-09-16T23:40:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.605+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b81918a0 con 0x7fa7b8101310 2024-09-16T23:40:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.605+0000 7fa7b5ffb700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7a4038410 0x7fa7a403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.605+0000 7fa7b5ffb700 1 -- 172.21.15.43:0/2671167780 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fa7a804c460 con 0x7fa7b8101310 2024-09-16T23:40:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.605+0000 7fa7b7fff700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7a4038410 0x7fa7a403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.606+0000 7fa7b7fff700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7a4038410 0x7fa7a403a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0006fd0 tx=0x7fa7a0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.614+0000 7fa7b5ffb700 1 -- 172.21.15.43:0/2671167780 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa7a800b960 con 0x7fa7b8101310 2024-09-16T23:40:42.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.895+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa7b802cc70 con 0x7fa7b8101310 2024-09-16T23:40:42.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.895+0000 7fa7b5ffb700 1 -- 172.21.15.43:0/2671167780 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa7a802b3b0 con 0x7fa7b8101310 2024-09-16T23:40:42.893 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:40:42.893 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7a4038410 msgr2=0x7fa7a403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7a4038410 0x7fa7a403a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0006fd0 tx=0x7fa7a0006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 msgr2=0x7fa7b8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8197b30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80092e0 tx=0x7fa7a8009e60 comp rx=0 tx=0).stop 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 shutdown_connections 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7a4038410 0x7fa7a403a8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 --2- 172.21.15.43:0/2671167780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7b8101310 0x7fa7b8197b30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 >> 172.21.15.43:0/2671167780 conn(0x7fa7b80fad20 msgr2=0x7fa7b80fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 shutdown_connections 2024-09-16T23:40:42.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:42.897+0000 7fa7bf241700 1 -- 172.21.15.43:0/2671167780 wait complete. 2024-09-16T23:40:42.896 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:40:43.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:43 smithi042 ceph-mon[27914]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:43.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:43 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:43.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:43 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/2671167780' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:40:44.494 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:40:44.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:40:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:44.743 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:45.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:45 smithi042 ceph-mon[27914]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.158+0000 7f331123b700 1 -- 172.21.15.43:0/1820030002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 msgr2=0x7f330c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.158+0000 7f331123b700 1 --2- 172.21.15.43:0/1820030002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c100cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009910 tx=0x7f32fc009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.159+0000 7f331123b700 1 -- 172.21.15.43:0/1820030002 shutdown_connections 2024-09-16T23:40:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.159+0000 7f331123b700 1 --2- 172.21.15.43:0/1820030002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c100cc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.159+0000 7f331123b700 1 -- 172.21.15.43:0/1820030002 >> 172.21.15.43:0/1820030002 conn(0x7f330c0fbe20 msgr2=0x7f330c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.159+0000 7f331123b700 1 -- 172.21.15.43:0/1820030002 shutdown_connections 2024-09-16T23:40:46.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.159+0000 7f331123b700 1 -- 172.21.15.43:0/1820030002 wait complete. 2024-09-16T23:40:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.159+0000 7f331123b700 1 Processor -- start 2024-09-16T23:40:46.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f331123b700 1 -- start start 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f331123b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f331123b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c075070 con 0x7f330c1008a0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f330ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f330ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:34740/0 (socket says 172.21.15.43:34740) 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f330ad9d700 1 -- 172.21.15.43:0/2765551691 learned_addr learned my addr 172.21.15.43:0/2765551691 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f330ad9d700 1 -- 172.21.15.43:0/2765551691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc0095f0 con 0x7f330c1008a0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.160+0000 7f330ad9d700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c074b30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005b40 tx=0x7f32fc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.161+0000 7f32f3fff700 1 -- 172.21.15.43:0/2765551691 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32fc01d070 con 0x7f330c1008a0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.161+0000 7f32f3fff700 1 -- 172.21.15.43:0/2765551691 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32fc00be10 con 0x7f330c1008a0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.161+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c073180 con 0x7f330c1008a0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.161+0000 7f32f3fff700 1 -- 172.21.15.43:0/2765551691 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32fc00f460 con 0x7f330c1008a0 2024-09-16T23:40:46.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.161+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c073620 con 0x7f330c1008a0 2024-09-16T23:40:46.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.161+0000 7f32f3fff700 1 -- 172.21.15.43:0/2765551691 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f32fc00f6d0 con 0x7f330c1008a0 2024-09-16T23:40:46.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.162+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c0623c0 con 0x7f330c1008a0 2024-09-16T23:40:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.162+0000 7f32f3fff700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32f40383c0 0x7f32f403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.162+0000 7f32f3fff700 1 -- 172.21.15.43:0/2765551691 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f32fc04c450 con 0x7f330c1008a0 2024-09-16T23:40:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.162+0000 7f330a59c700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32f40383c0 0x7f32f403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:46.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.162+0000 7f330a59c700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32f40383c0 0x7f32f403a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3300006fd0 tx=0x7f3300006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:46.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.167+0000 7f32f3fff700 1 -- 172.21.15.43:0/2765551691 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f32fc02a360 con 0x7f330c1008a0 2024-09-16T23:40:46.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.448+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f330c18eea0 con 0x7f330c1008a0 2024-09-16T23:40:46.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.449+0000 7f32f3fff700 1 -- 172.21.15.43:0/2765551691 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f32fc027030 con 0x7f330c1008a0 2024-09-16T23:40:46.446 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:40:46.446 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:40:46.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32f40383c0 msgr2=0x7f32f403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32f40383c0 0x7f32f403a880 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3300006fd0 tx=0x7f3300006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 msgr2=0x7f330c074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c074b30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32fc005b40 tx=0x7f32fc004740 comp rx=0 tx=0).stop 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 shutdown_connections 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f32f40383c0 0x7f32f403a880 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 --2- 172.21.15.43:0/2765551691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c1008a0 0x7f330c074b30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 >> 172.21.15.43:0/2765551691 conn(0x7f330c0fbe20 msgr2=0x7f330c0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 shutdown_connections 2024-09-16T23:40:46.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:46.451+0000 7f331123b700 1 -- 172.21.15.43:0/2765551691 wait complete. 2024-09-16T23:40:46.450 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:40:47.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:47 smithi042 ceph-mon[27914]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:47.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:47 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/2765551691' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:40:48.003 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:40:48.003 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:40:48.207 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:48.252 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:49.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:49 smithi042 ceph-mon[27914]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:49.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.703+0000 7fe86769a700 1 -- 172.21.15.43:0/3216513319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 msgr2=0x7fe860101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.703+0000 7fe86769a700 1 --2- 172.21.15.43:0/3216513319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe860101140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009910 tx=0x7fe85c009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.703+0000 7fe86769a700 1 -- 172.21.15.43:0/3216513319 shutdown_connections 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.703+0000 7fe86769a700 1 --2- 172.21.15.43:0/3216513319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe860101140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.703+0000 7fe86769a700 1 -- 172.21.15.43:0/3216513319 >> 172.21.15.43:0/3216513319 conn(0x7fe8600fa490 msgr2=0x7fe8600fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.704+0000 7fe86769a700 1 -- 172.21.15.43:0/3216513319 shutdown_connections 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.704+0000 7fe86769a700 1 -- 172.21.15.43:0/3216513319 wait complete. 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.704+0000 7fe86769a700 1 Processor -- start 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.704+0000 7fe86769a700 1 -- start start 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.705+0000 7fe86769a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe8600ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:49.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.705+0000 7fe86769a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8601986c0 con 0x7fe860100d20 2024-09-16T23:40:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.705+0000 7fe865436700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe8600ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.705+0000 7fe865436700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe8600ff4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:34758/0 (socket says 172.21.15.43:34758) 2024-09-16T23:40:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.705+0000 7fe865436700 1 -- 172.21.15.43:0/411366546 learned_addr learned my addr 172.21.15.43:0/411366546 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:40:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.705+0000 7fe865436700 1 -- 172.21.15.43:0/411366546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0095f0 con 0x7fe860100d20 2024-09-16T23:40:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.705+0000 7fe865436700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe8600ff4d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005950 tx=0x7fe85c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.706+0000 7fe8567fc700 1 -- 172.21.15.43:0/411366546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe85c01d070 con 0x7fe860100d20 2024-09-16T23:40:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.706+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe860198800 con 0x7fe860100d20 2024-09-16T23:40:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.706+0000 7fe8567fc700 1 -- 172.21.15.43:0/411366546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe85c00be10 con 0x7fe860100d20 2024-09-16T23:40:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.706+0000 7fe8567fc700 1 -- 172.21.15.43:0/411366546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe85c00f460 con 0x7fe860100d20 2024-09-16T23:40:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.706+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe860198ca0 con 0x7fe860100d20 2024-09-16T23:40:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.706+0000 7fe8567fc700 1 -- 172.21.15.43:0/411366546 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fe85c00f690 con 0x7fe860100d20 2024-09-16T23:40:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.707+0000 7fe8567fc700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe84c0383c0 0x7fe84c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.707+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86018e3b0 con 0x7fe860100d20 2024-09-16T23:40:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.707+0000 7fe8567fc700 1 -- 172.21.15.43:0/411366546 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe85c04c4a0 con 0x7fe860100d20 2024-09-16T23:40:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.707+0000 7fe864c35700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe84c0383c0 0x7fe84c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.707+0000 7fe864c35700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe84c0383c0 0x7fe84c03a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe850006fd0 tx=0x7fe850006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:49.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.712+0000 7fe8567fc700 1 -- 172.21.15.43:0/411366546 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe85c02c440 con 0x7fe860100d20 2024-09-16T23:40:49.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.992+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe86002cc70 con 0x7fe860100d20 2024-09-16T23:40:49.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.993+0000 7fe8567fc700 1 -- 172.21.15.43:0/411366546 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fe85c027020 con 0x7fe860100d20 2024-09-16T23:40:49.991 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:40:49.991 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe84c0383c0 msgr2=0x7fe84c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe84c0383c0 0x7fe84c03a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe850006fd0 tx=0x7fe850006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 msgr2=0x7fe8600ff4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe8600ff4d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005950 tx=0x7fe85c009e60 comp rx=0 tx=0).stop 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 shutdown_connections 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe84c0383c0 0x7fe84c03a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 --2- 172.21.15.43:0/411366546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe860100d20 0x7fe8600ff4d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 >> 172.21.15.43:0/411366546 conn(0x7fe8600fa490 msgr2=0x7fe8600fb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 shutdown_connections 2024-09-16T23:40:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:49.995+0000 7fe86769a700 1 -- 172.21.15.43:0/411366546 wait complete. 2024-09-16T23:40:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/411366546' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:50.240 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:50.241 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:50 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:51.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:51 smithi042 ceph-mon[27914]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:51.720 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:40:51.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:40:51.924 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:51.972 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:52.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:52 smithi042 ceph-mon[27914]: Deploying daemon prometheus.smithi042 on smithi042 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 -- 172.21.15.43:0/3942879999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 msgr2=0x7f2dfc0fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 --2- 172.21.15.43:0/3942879999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc0fef70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009910 tx=0x7f2dec009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 -- 172.21.15.43:0/3942879999 shutdown_connections 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 --2- 172.21.15.43:0/3942879999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc0fef70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 -- 172.21.15.43:0/3942879999 >> 172.21.15.43:0/3942879999 conn(0x7f2dfc0fa490 msgr2=0x7f2dfc0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 -- 172.21.15.43:0/3942879999 shutdown_connections 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 -- 172.21.15.43:0/3942879999 wait complete. 2024-09-16T23:40:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.332+0000 7f2e01621700 1 Processor -- start 2024-09-16T23:40:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.333+0000 7f2e01621700 1 -- start start 2024-09-16T23:40:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.333+0000 7f2e01621700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.333+0000 7f2e01621700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dfc195ef0 con 0x7f2dfc0feb50 2024-09-16T23:40:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.333+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.333+0000 7f2dfaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc1959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:34762/0 (socket says 172.21.15.43:34762) 2024-09-16T23:40:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.333+0000 7f2dfaffd700 1 -- 172.21.15.43:0/3766538848 learned_addr learned my addr 172.21.15.43:0/3766538848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:40:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.333+0000 7f2dfaffd700 1 -- 172.21.15.43:0/3766538848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec0095f0 con 0x7f2dfc0feb50 2024-09-16T23:40:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.334+0000 7f2dfaffd700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc1959b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2dec005b40 tx=0x7f2dec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.334+0000 7f2de3fff700 1 -- 172.21.15.43:0/3766538848 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dec01d070 con 0x7f2dfc0feb50 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.334+0000 7f2de3fff700 1 -- 172.21.15.43:0/3766538848 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2dec00be10 con 0x7f2dfc0feb50 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.334+0000 7f2de3fff700 1 -- 172.21.15.43:0/3766538848 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2dec00f460 con 0x7f2dfc0feb50 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.334+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dfc1960f0 con 0x7f2dfc0feb50 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.334+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dfc196590 con 0x7f2dfc0feb50 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.335+0000 7f2de3fff700 1 -- 172.21.15.43:0/3766538848 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f2dec00b960 con 0x7f2dfc0feb50 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.335+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dfc18f4c0 con 0x7f2dfc0feb50 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.335+0000 7f2de3fff700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2de40383c0 0x7f2de403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.335+0000 7f2de3fff700 1 -- 172.21.15.43:0/3766538848 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2dec04c390 con 0x7f2dfc0feb50 2024-09-16T23:40:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.335+0000 7f2dfa7fc700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2de40383c0 0x7f2de403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.336+0000 7f2dfa7fc700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2de40383c0 0x7f2de403a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:53.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.340+0000 7f2de3fff700 1 -- 172.21.15.43:0/3766538848 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2dec02bba0 con 0x7f2dfc0feb50 2024-09-16T23:40:53.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:53 smithi042 ceph-mon[27914]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:53.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:53 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:40:53.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.619+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2dfc02cc70 con 0x7f2dfc0feb50 2024-09-16T23:40:53.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.619+0000 7f2de3fff700 1 -- 172.21.15.43:0/3766538848 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2dec027030 con 0x7f2dfc0feb50 2024-09-16T23:40:53.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:40:53.617 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:40:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2de40383c0 msgr2=0x7f2de403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2de40383c0 0x7f2de403a880 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 msgr2=0x7f2dfc1959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc1959b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2dec005b40 tx=0x7f2dec004740 comp rx=0 tx=0).stop 2024-09-16T23:40:53.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 shutdown_connections 2024-09-16T23:40:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2de40383c0 0x7f2de403a880 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 --2- 172.21.15.43:0/3766538848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2dfc0feb50 0x7f2dfc1959b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 >> 172.21.15.43:0/3766538848 conn(0x7f2dfc0fa490 msgr2=0x7f2dfc0fb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 shutdown_connections 2024-09-16T23:40:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:53.622+0000 7f2e01621700 1 -- 172.21.15.43:0/3766538848 wait complete. 2024-09-16T23:40:53.620 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:40:54.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:54 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/3766538848' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:40:55.186 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:40:55.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:40:55.391 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:55 smithi042 ceph-mon[27914]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 -- 172.21.15.43:0/2614999402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 msgr2=0x7fc7180ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 --2- 172.21.15.43:0/2614999402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7180ffba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc708009910 tx=0x7fc708009c20 comp rx=0 tx=0).stop 2024-09-16T23:40:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 -- 172.21.15.43:0/2614999402 shutdown_connections 2024-09-16T23:40:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 --2- 172.21.15.43:0/2614999402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7180ffba0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 -- 172.21.15.43:0/2614999402 >> 172.21.15.43:0/2614999402 conn(0x7fc7180fad20 msgr2=0x7fc7180fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 -- 172.21.15.43:0/2614999402 shutdown_connections 2024-09-16T23:40:56.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 -- 172.21.15.43:0/2614999402 wait complete. 2024-09-16T23:40:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 Processor -- start 2024-09-16T23:40:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.922+0000 7fc71f216700 1 -- start start 2024-09-16T23:40:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71f216700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7181948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71f216700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc718194de0 con 0x7fc7180ff780 2024-09-16T23:40:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71cfb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7181948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71cfb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7181948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:56332/0 (socket says 172.21.15.43:56332) 2024-09-16T23:40:56.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71cfb2700 1 -- 172.21.15.43:0/3003882872 learned_addr learned my addr 172.21.15.43:0/3003882872 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:40:56.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71cfb2700 1 -- 172.21.15.43:0/3003882872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7080095f0 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71cfb2700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7181948a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc708005b40 tx=0x7fc708004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc715ffb700 1 -- 172.21.15.43:0/3003882872 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc70801d070 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc715ffb700 1 -- 172.21.15.43:0/3003882872 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc70800be10 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.923+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc718194fe0 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.924+0000 7fc715ffb700 1 -- 172.21.15.43:0/3003882872 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc70800f460 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.924+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc718195480 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.925+0000 7fc715ffb700 1 -- 172.21.15.43:0/3003882872 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7fc7080227e0 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.925+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71818e3a0 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.925+0000 7fc715ffb700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc7040383c0 0x7fc70403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.925+0000 7fc715ffb700 1 -- 172.21.15.43:0/3003882872 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fc70804c410 con 0x7fc7180ff780 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.925+0000 7fc717fff700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc7040383c0 0x7fc70403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:40:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.925+0000 7fc717fff700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc7040383c0 0x7fc70403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc700006fd0 tx=0x7fc700006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:40:56.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:56.930+0000 7fc715ffb700 1 -- 172.21.15.43:0/3003882872 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc70802bba0 con 0x7fc7180ff780 2024-09-16T23:40:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.211+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc71802cc70 con 0x7fc7180ff780 2024-09-16T23:40:57.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.211+0000 7fc715ffb700 1 -- 172.21.15.43:0/3003882872 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc708027020 con 0x7fc7180ff780 2024-09-16T23:40:57.209 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:40:57.209 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc7040383c0 msgr2=0x7fc70403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc7040383c0 0x7fc70403a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc700006fd0 tx=0x7fc700006e40 comp rx=0 tx=0).stop 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 msgr2=0x7fc7181948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7181948a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc708005b40 tx=0x7fc708004740 comp rx=0 tx=0).stop 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 shutdown_connections 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc7040383c0 0x7fc70403a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 --2- 172.21.15.43:0/3003882872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7180ff780 0x7fc7181948a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 >> 172.21.15.43:0/3003882872 conn(0x7fc7180fad20 msgr2=0x7fc7180fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 shutdown_connections 2024-09-16T23:40:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:40:57.213+0000 7fc71f216700 1 -- 172.21.15.43:0/3003882872 wait complete. 2024-09-16T23:40:57.212 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:40:57.470 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:57 smithi042 ceph-mon[27914]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:40:58.485 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:58 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/3003882872' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:40:58.819 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:40:58.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:40:59.023 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:59.067 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:40:59.468 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:40:59 smithi042 ceph-mon[27914]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.446+0000 7f852cfae700 1 -- 172.21.15.43:0/267322781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 msgr2=0x7f8528103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.446+0000 7f852cfae700 1 --2- 172.21.15.43:0/267322781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528103700 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8518009910 tx=0x7f8518009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:00.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.446+0000 7f852cfae700 1 -- 172.21.15.43:0/267322781 shutdown_connections 2024-09-16T23:41:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.446+0000 7f852cfae700 1 --2- 172.21.15.43:0/267322781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528103700 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.446+0000 7f852cfae700 1 -- 172.21.15.43:0/267322781 >> 172.21.15.43:0/267322781 conn(0x7f85280fad20 msgr2=0x7f85280fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.446+0000 7f852cfae700 1 -- 172.21.15.43:0/267322781 shutdown_connections 2024-09-16T23:41:00.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.447+0000 7f852cfae700 1 -- 172.21.15.43:0/267322781 wait complete. 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.447+0000 7f852cfae700 1 Processor -- start 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.447+0000 7f852cfae700 1 -- start start 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.447+0000 7f852cfae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.447+0000 7f852cfae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8528195ec0 con 0x7f8528101310 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f852659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f852659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:56348/0 (socket says 172.21.15.43:56348) 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f852659c700 1 -- 172.21.15.43:0/2721139682 learned_addr learned my addr 172.21.15.43:0/2721139682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f852659c700 1 -- 172.21.15.43:0/2721139682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85180095f0 con 0x7f8528101310 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f852659c700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528195980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8518005b40 tx=0x7f8518004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f851f7fe700 1 -- 172.21.15.43:0/2721139682 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f851801d070 con 0x7f8528101310 2024-09-16T23:41:00.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85281960c0 con 0x7f8528101310 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f851f7fe700 1 -- 172.21.15.43:0/2721139682 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f851800be10 con 0x7f8528101310 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f851f7fe700 1 -- 172.21.15.43:0/2721139682 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f851800f460 con 0x7f8528101310 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.448+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8528196560 con 0x7f8528101310 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.449+0000 7f851f7fe700 1 -- 172.21.15.43:0/2721139682 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45339+0+0 (secure 0 0 0) 0x7f851800b960 con 0x7f8528101310 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.449+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f852818f4a0 con 0x7f8528101310 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.449+0000 7f851f7fe700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f85140383c0 0x7f851403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.449+0000 7f851f7fe700 1 -- 172.21.15.43:0/2721139682 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f851804c450 con 0x7f8528101310 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.450+0000 7f8525d9b700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f85140383c0 0x7f851403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:00.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.450+0000 7f8525d9b700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f85140383c0 0x7f851403a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:00.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.454+0000 7f851f7fe700 1 -- 172.21.15.43:0/2721139682 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f851802a360 con 0x7f8528101310 2024-09-16T23:41:00.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.739+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f852802cc70 con 0x7f8528101310 2024-09-16T23:41:00.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.740+0000 7f851f7fe700 1 -- 172.21.15.43:0/2721139682 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8518027030 con 0x7f8528101310 2024-09-16T23:41:00.737 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:00.737 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:00.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f85140383c0 msgr2=0x7f851403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f85140383c0 0x7f851403a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 msgr2=0x7f8528195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528195980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8518005b40 tx=0x7f8518004740 comp rx=0 tx=0).stop 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 shutdown_connections 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f85140383c0 0x7f851403a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 --2- 172.21.15.43:0/2721139682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8528101310 0x7f8528195980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 >> 172.21.15.43:0/2721139682 conn(0x7f85280fad20 msgr2=0x7f85280fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 shutdown_connections 2024-09-16T23:41:00.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:00.742+0000 7f852cfae700 1 -- 172.21.15.43:0/2721139682 wait complete. 2024-09-16T23:41:00.741 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:01.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:01 smithi042 ceph-mon[27914]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:01.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:01 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/2721139682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:01.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:01 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:01.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:01 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:01.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:01 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:01.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:01 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T23:41:02.473 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:02.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:02.678 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:02.723 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:03.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:02 smithi042 ceph-mon[27914]: from='mgr.14164 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T23:41:03.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:02 smithi042 ceph-mon[27914]: mgrmap e14: smithi042.zrkumz(active, since 59s) 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.214+0000 7f2e7a42d700 1 -- 172.21.15.43:0/691255458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 msgr2=0x7f2e74100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.214+0000 7f2e7a42d700 1 --2- 172.21.15.43:0/691255458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74100cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009910 tx=0x7f2e64009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.214+0000 7f2e7a42d700 1 -- 172.21.15.43:0/691255458 shutdown_connections 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.214+0000 7f2e7a42d700 1 --2- 172.21.15.43:0/691255458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74100cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.214+0000 7f2e7a42d700 1 -- 172.21.15.43:0/691255458 >> 172.21.15.43:0/691255458 conn(0x7f2e740fbe20 msgr2=0x7f2e740fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.214+0000 7f2e7a42d700 1 -- 172.21.15.43:0/691255458 shutdown_connections 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.214+0000 7f2e7a42d700 1 -- 172.21.15.43:0/691255458 wait complete. 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.215+0000 7f2e7a42d700 1 Processor -- start 2024-09-16T23:41:04.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.215+0000 7f2e7a42d700 1 -- start start 2024-09-16T23:41:04.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.215+0000 7f2e7a42d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:04.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.215+0000 7f2e7a42d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e74075070 con 0x7f2e741008a0 2024-09-16T23:41:04.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.215+0000 7f2e73fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:04.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e73fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:56364/0 (socket says 172.21.15.43:56364) 2024-09-16T23:41:04.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e73fff700 1 -- 172.21.15.43:0/1091223090 learned_addr learned my addr 172.21.15.43:0/1091223090 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:04.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e73fff700 1 -- 172.21.15.43:0/1091223090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640095f0 con 0x7f2e741008a0 2024-09-16T23:41:04.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e73fff700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74074b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005950 tx=0x7f2e64009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:04.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e71ffb700 1 -- 172.21.15.43:0/1091223090 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e6401d070 con 0x7f2e741008a0 2024-09-16T23:41:04.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e71ffb700 1 -- 172.21.15.43:0/1091223090 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2e6400be10 con 0x7f2e741008a0 2024-09-16T23:41:04.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e74073180 con 0x7f2e741008a0 2024-09-16T23:41:04.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e71ffb700 1 -- 172.21.15.43:0/1091223090 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2e6400f460 con 0x7f2e741008a0 2024-09-16T23:41:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.216+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e74073620 con 0x7f2e741008a0 2024-09-16T23:41:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.217+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e740623c0 con 0x7f2e741008a0 2024-09-16T23:41:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.217+0000 7f2e71ffb700 1 -- 172.21.15.43:0/1091223090 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7f2e640227c0 con 0x7f2e741008a0 2024-09-16T23:41:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.217+0000 7f2e71ffb700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 0x7f2e6003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.218+0000 7f2e71ffb700 1 -- 172.21.15.43:0/1091223090 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2e6404c430 con 0x7f2e741008a0 2024-09-16T23:41:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.218+0000 7f2e6bfff700 1 -- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 msgr2=0x7f2e6003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:41:04.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.218+0000 7f2e6bfff700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 0x7f2e6003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T23:41:04.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.222+0000 7f2e71ffb700 1 -- 172.21.15.43:0/1091223090 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2e6402bba0 con 0x7f2e741008a0 2024-09-16T23:41:04.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.419+0000 7f2e6bfff700 1 -- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 msgr2=0x7f2e6003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:41:04.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.419+0000 7f2e6bfff700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 0x7f2e6003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T23:41:04.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.503+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2e7418eea0 con 0x7f2e741008a0 2024-09-16T23:41:04.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.504+0000 7f2e71ffb700 1 -- 172.21.15.43:0/1091223090 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2e64027030 con 0x7f2e741008a0 2024-09-16T23:41:04.501 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:04.501 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 msgr2=0x7f2e6003a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T23:41:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 0x7f2e6003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 msgr2=0x7f2e74074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74074b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005950 tx=0x7f2e64009e60 comp rx=0 tx=0).stop 2024-09-16T23:41:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 shutdown_connections 2024-09-16T23:41:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2e60038410 0x7f2e6003a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 --2- 172.21.15.43:0/1091223090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e741008a0 0x7f2e74074b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 >> 172.21.15.43:0/1091223090 conn(0x7f2e740fbe20 msgr2=0x7f2e740fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 shutdown_connections 2024-09-16T23:41:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:04.506+0000 7f2e7a42d700 1 -- 172.21.15.43:0/1091223090 wait complete. 2024-09-16T23:41:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:04.844 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:04 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1091223090' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:06.085 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:06.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:06.288 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:06.333 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.873+0000 7f2ef56dd700 1 -- 172.21.15.43:0/2478163508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 msgr2=0x7f2ef0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.873+0000 7f2ef56dd700 1 --2- 172.21.15.43:0/2478163508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0100ca0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009910 tx=0x7f2ee0009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.873+0000 7f2ef56dd700 1 -- 172.21.15.43:0/2478163508 shutdown_connections 2024-09-16T23:41:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.873+0000 7f2ef56dd700 1 --2- 172.21.15.43:0/2478163508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0100ca0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.873+0000 7f2ef56dd700 1 -- 172.21.15.43:0/2478163508 >> 172.21.15.43:0/2478163508 conn(0x7f2ef00fbe20 msgr2=0x7f2ef00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.873+0000 7f2ef56dd700 1 -- 172.21.15.43:0/2478163508 shutdown_connections 2024-09-16T23:41:07.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.873+0000 7f2ef56dd700 1 -- 172.21.15.43:0/2478163508 wait complete. 2024-09-16T23:41:07.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.874+0000 7f2ef56dd700 1 Processor -- start 2024-09-16T23:41:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.874+0000 7f2ef56dd700 1 -- start start 2024-09-16T23:41:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.874+0000 7f2ef56dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.874+0000 7f2ef56dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0075070 con 0x7f2ef0100880 2024-09-16T23:41:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.874+0000 7f2eeeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.874+0000 7f2eeeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:33886/0 (socket says 172.21.15.43:33886) 2024-09-16T23:41:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2eeeffd700 1 -- 172.21.15.43:0/865657449 learned_addr learned my addr 172.21.15.43:0/865657449 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:07.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2eeeffd700 1 -- 172.21.15.43:0/865657449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee00095f0 con 0x7f2ef0100880 2024-09-16T23:41:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2eeeffd700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0074b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0005b40 tx=0x7f2ee0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2ee7fff700 1 -- 172.21.15.43:0/865657449 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ee001d070 con 0x7f2ef0100880 2024-09-16T23:41:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef0073180 con 0x7f2ef0100880 2024-09-16T23:41:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2ee7fff700 1 -- 172.21.15.43:0/865657449 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2ee000be10 con 0x7f2ef0100880 2024-09-16T23:41:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2ee7fff700 1 -- 172.21.15.43:0/865657449 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2ee000f460 con 0x7f2ef0100880 2024-09-16T23:41:07.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.875+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef0073620 con 0x7f2ef0100880 2024-09-16T23:41:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.876+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef00623c0 con 0x7f2ef0100880 2024-09-16T23:41:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.876+0000 7f2ee7fff700 1 -- 172.21.15.43:0/865657449 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 14) v1 ==== 45353+0+0 (secure 0 0 0) 0x7f2ee000b960 con 0x7f2ef0100880 2024-09-16T23:41:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.876+0000 7f2ee7fff700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.876+0000 7f2ee7fff700 1 -- 172.21.15.43:0/865657449 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2ee004c3b0 con 0x7f2ef0100880 2024-09-16T23:41:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.876+0000 7f2eee7fc700 1 -- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 msgr2=0x7f2edc03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:41:07.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.877+0000 7f2eee7fc700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T23:41:07.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:07.881+0000 7f2ee7fff700 1 -- 172.21.15.43:0/865657449 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2ee002a360 con 0x7f2ef0100880 2024-09-16T23:41:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.078+0000 7f2eee7fc700 1 -- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 msgr2=0x7f2edc03a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2 2024-09-16T23:41:08.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.078+0000 7f2eee7fc700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T23:41:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.165+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2ef018ee80 con 0x7f2ef0100880 2024-09-16T23:41:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.165+0000 7f2ee7fff700 1 -- 172.21.15.43:0/865657449 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f2ee0027070 con 0x7f2ef0100880 2024-09-16T23:41:08.165 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:08.165 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 msgr2=0x7f2edc03a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T23:41:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 msgr2=0x7f2ef0074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0074b30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0005b40 tx=0x7f2ee0004740 comp rx=0 tx=0).stop 2024-09-16T23:41:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 shutdown_connections 2024-09-16T23:41:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2edc038410 0x7f2edc03a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 --2- 172.21.15.43:0/865657449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ef0100880 0x7f2ef0074b30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 >> 172.21.15.43:0/865657449 conn(0x7f2ef00fbe20 msgr2=0x7f2ef00fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 shutdown_connections 2024-09-16T23:41:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:08.167+0000 7f2ef56dd700 1 -- 172.21.15.43:0/865657449 wait complete. 2024-09-16T23:41:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:08.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:08 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/865657449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:09.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: Active manager daemon smithi042.zrkumz restarted 2024-09-16T23:41:09.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: Activating manager daemon smithi042.zrkumz 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: mgrmap e15: smithi042.zrkumz(active, starting, since 0.00317899s) 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:41:09.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:41:09.821 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:09.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:10.016 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:10.066 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:10 smithi042 ceph-mon[27914]: [16/Sep/2024:23:41:09] ENGINE Bus STARTING 2024-09-16T23:41:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:10 smithi042 ceph-mon[27914]: [16/Sep/2024:23:41:09] ENGINE Serving on https://172.21.15.42:7150 2024-09-16T23:41:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:10 smithi042 ceph-mon[27914]: [16/Sep/2024:23:41:09] ENGINE Serving on http://172.21.15.42:8765 2024-09-16T23:41:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:10 smithi042 ceph-mon[27914]: [16/Sep/2024:23:41:09] ENGINE Bus STARTED 2024-09-16T23:41:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:10 smithi042 ceph-mon[27914]: mgrmap e16: smithi042.zrkumz(active, since 1.00554s) 2024-09-16T23:41:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:11.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.572+0000 7fc5527a1700 1 -- 172.21.15.43:0/906962748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 msgr2=0x7fc54c0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:11.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.572+0000 7fc5527a1700 1 --2- 172.21.15.43:0/906962748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c0ffba0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009910 tx=0x7fc53c009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:11.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.572+0000 7fc5527a1700 1 -- 172.21.15.43:0/906962748 shutdown_connections 2024-09-16T23:41:11.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.572+0000 7fc5527a1700 1 --2- 172.21.15.43:0/906962748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c0ffba0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.572+0000 7fc5527a1700 1 -- 172.21.15.43:0/906962748 >> 172.21.15.43:0/906962748 conn(0x7fc54c0fad20 msgr2=0x7fc54c0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.572+0000 7fc5527a1700 1 -- 172.21.15.43:0/906962748 shutdown_connections 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.572+0000 7fc5527a1700 1 -- 172.21.15.43:0/906962748 wait complete. 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.573+0000 7fc5527a1700 1 Processor -- start 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.573+0000 7fc5527a1700 1 -- start start 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc5527a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c1948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc5527a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c194de0 con 0x7fc54c0ff780 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c1948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c1948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:33902/0 (socket says 172.21.15.43:33902) 2024-09-16T23:41:11.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc54bfff700 1 -- 172.21.15.43:0/4054021338 learned_addr learned my addr 172.21.15.43:0/4054021338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc54bfff700 1 -- 172.21.15.43:0/4054021338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c0095f0 con 0x7fc54c0ff780 2024-09-16T23:41:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc54bfff700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c1948a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc53c005b40 tx=0x7fc53c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc53c01d070 con 0x7fc54c0ff780 2024-09-16T23:41:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.574+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c194fe0 con 0x7fc54c0ff780 2024-09-16T23:41:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.575+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc53c00be10 con 0x7fc54c0ff780 2024-09-16T23:41:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.575+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc53c00f460 con 0x7fc54c0ff780 2024-09-16T23:41:11.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.575+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c195480 con 0x7fc54c0ff780 2024-09-16T23:41:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.575+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c1055c0 con 0x7fc54c0ff780 2024-09-16T23:41:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.575+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 45204+0+0 (secure 0 0 0) 0x7fc53c0227e0 con 0x7fc54c0ff780 2024-09-16T23:41:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.576+0000 7fc5497fa700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc53403a480 0x7fc53403c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.576+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7fc53c04c1c0 con 0x7fc54c0ff780 2024-09-16T23:41:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.576+0000 7fc54b7fe700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc53403a480 0x7fc53403c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:11.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.576+0000 7fc54b7fe700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc53403a480 0x7fc53403c940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:11.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.581+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc53c02bba0 con 0x7fc54c0ff780 2024-09-16T23:41:11.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.733+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fc53c00f6d0 con 0x7fc54c0ff780 2024-09-16T23:41:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.869+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc54c1057b0 con 0x7fc54c0ff780 2024-09-16T23:41:11.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.869+0000 7fc5497fa700 1 -- 172.21.15.43:0/4054021338 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc53c01f2d0 con 0x7fc54c0ff780 2024-09-16T23:41:11.867 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:11.867 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.871+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc53403a480 msgr2=0x7fc53403c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.871+0000 7fc5527a1700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc53403a480 0x7fc53403c940 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.871+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 msgr2=0x7fc54c1948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.871+0000 7fc5527a1700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c1948a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc53c005b40 tx=0x7fc53c004740 comp rx=0 tx=0).stop 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.872+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 shutdown_connections 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.872+0000 7fc5527a1700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc53403a480 0x7fc53403c940 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.872+0000 7fc5527a1700 1 --2- 172.21.15.43:0/4054021338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc54c0ff780 0x7fc54c1948a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.872+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 >> 172.21.15.43:0/4054021338 conn(0x7fc54c0fad20 msgr2=0x7fc54c0fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:11.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.872+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 shutdown_connections 2024-09-16T23:41:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:11.872+0000 7fc5527a1700 1 -- 172.21.15.43:0/4054021338 wait complete. 2024-09-16T23:41:11.870 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:13.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:13.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:12 smithi042 ceph-mon[27914]: mgrmap e17: smithi042.zrkumz(active, since 2s) 2024-09-16T23:41:13.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:12 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/4054021338' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:13.408 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:13.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:13.611 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:14.005 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:15.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.002+0000 7f8257578700 1 -- 172.21.15.43:0/1223220716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 msgr2=0x7f8250100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:16.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.002+0000 7f8257578700 1 --2- 172.21.15.43:0/1223220716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250100c20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f824c009910 tx=0x7f824c009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:16.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.002+0000 7f8257578700 1 -- 172.21.15.43:0/1223220716 shutdown_connections 2024-09-16T23:41:16.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.002+0000 7f8257578700 1 --2- 172.21.15.43:0/1223220716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250100c20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:16.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.002+0000 7f8257578700 1 -- 172.21.15.43:0/1223220716 >> 172.21.15.43:0/1223220716 conn(0x7f82500fbdc0 msgr2=0x7f82500fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:16.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.002+0000 7f8257578700 1 -- 172.21.15.43:0/1223220716 shutdown_connections 2024-09-16T23:41:16.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.002+0000 7f8257578700 1 -- 172.21.15.43:0/1223220716 wait complete. 2024-09-16T23:41:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.003+0000 7f8257578700 1 Processor -- start 2024-09-16T23:41:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.003+0000 7f8257578700 1 -- start start 2024-09-16T23:41:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.003+0000 7f8257578700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:16.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.003+0000 7f8257578700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8250195e70 con 0x7f8250100800 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.003+0000 7f8255314700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.003+0000 7f8255314700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:53668/0 (socket says 172.21.15.43:53668) 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.003+0000 7f8255314700 1 -- 172.21.15.43:0/1396604669 learned_addr learned my addr 172.21.15.43:0/1396604669 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.004+0000 7f8255314700 1 -- 172.21.15.43:0/1396604669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824c0095f0 con 0x7f8250100800 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.004+0000 7f8255314700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250195930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f824c005b40 tx=0x7f824c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.004+0000 7f82467fc700 1 -- 172.21.15.43:0/1396604669 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f824c01d070 con 0x7f8250100800 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.004+0000 7f82467fc700 1 -- 172.21.15.43:0/1396604669 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f824c00be10 con 0x7f8250100800 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.004+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8250196070 con 0x7f8250100800 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.004+0000 7f82467fc700 1 -- 172.21.15.43:0/1396604669 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f824c00f460 con 0x7f8250100800 2024-09-16T23:41:16.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.004+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8250196510 con 0x7f8250100800 2024-09-16T23:41:16.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.005+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f825018f420 con 0x7f8250100800 2024-09-16T23:41:16.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.005+0000 7f82467fc700 1 -- 172.21.15.43:0/1396604669 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f824c0227e0 con 0x7f8250100800 2024-09-16T23:41:16.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.006+0000 7f82467fc700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f823c0384c0 0x7f823c03a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:16.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.006+0000 7f82467fc700 1 -- 172.21.15.43:0/1396604669 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f824c04d010 con 0x7f8250100800 2024-09-16T23:41:16.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.006+0000 7f8254b13700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f823c0384c0 0x7f823c03a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:16.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.006+0000 7f8254b13700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f823c0384c0 0x7f823c03a980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8240006fd0 tx=0x7f8240006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:16.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.010+0000 7f82467fc700 1 -- 172.21.15.43:0/1396604669 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f824c022470 con 0x7f8250100800 2024-09-16T23:41:16.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.289+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f825002cc70 con 0x7f8250100800 2024-09-16T23:41:16.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.289+0000 7f82467fc700 1 -- 172.21.15.43:0/1396604669 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f824c027020 con 0x7f8250100800 2024-09-16T23:41:16.287 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:16.287 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:16.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f823c0384c0 msgr2=0x7f823c03a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:16.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f823c0384c0 0x7f823c03a980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8240006fd0 tx=0x7f8240006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:16.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 msgr2=0x7f8250195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:16.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250195930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f824c005b40 tx=0x7f824c004740 comp rx=0 tx=0).stop 2024-09-16T23:41:16.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 shutdown_connections 2024-09-16T23:41:16.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f823c0384c0 0x7f823c03a980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:16.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 --2- 172.21.15.43:0/1396604669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8250100800 0x7f8250195930 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:16.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.291+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 >> 172.21.15.43:0/1396604669 conn(0x7f82500fbdc0 msgr2=0x7f82500fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:16.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.292+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 shutdown_connections 2024-09-16T23:41:16.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:16.292+0000 7f8257578700 1 -- 172.21.15.43:0/1396604669 wait complete. 2024-09-16T23:41:16.290 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:41:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:16 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1396604669' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:17.859 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:17.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:18.066 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:18.113 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T23:41:19.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.633+0000 7fa7e60c1700 1 -- 172.21.15.43:0/3334616216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 msgr2=0x7fa7e0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:19.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.633+0000 7fa7e60c1700 1 --2- 172.21.15.43:0/3334616216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e0100ca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b0a0 tx=0x7fa7d000b3b0 comp rx=0 tx=0).stop 2024-09-16T23:41:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.633+0000 7fa7e60c1700 1 -- 172.21.15.43:0/3334616216 shutdown_connections 2024-09-16T23:41:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.633+0000 7fa7e60c1700 1 --2- 172.21.15.43:0/3334616216 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e0100ca0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.633+0000 7fa7e60c1700 1 -- 172.21.15.43:0/3334616216 >> 172.21.15.43:0/3334616216 conn(0x7fa7e00fbe00 msgr2=0x7fa7e00fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.633+0000 7fa7e60c1700 1 -- 172.21.15.43:0/3334616216 shutdown_connections 2024-09-16T23:41:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.633+0000 7fa7e60c1700 1 -- 172.21.15.43:0/3334616216 wait complete. 2024-09-16T23:41:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.634+0000 7fa7e60c1700 1 Processor -- start 2024-09-16T23:41:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.634+0000 7fa7e60c1700 1 -- start start 2024-09-16T23:41:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.634+0000 7fa7e60c1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e01959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.634+0000 7fa7e60c1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0195f10 con 0x7fa7e0100880 2024-09-16T23:41:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7df7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e01959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7df7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e01959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:53688/0 (socket says 172.21.15.43:53688) 2024-09-16T23:41:19.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7df7fe700 1 -- 172.21.15.43:0/44338133 learned_addr learned my addr 172.21.15.43:0/44338133 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7df7fe700 1 -- 172.21.15.43:0/44338133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d0009d00 con 0x7fa7e0100880 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7df7fe700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e01959d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b830 tx=0x7fa7d000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7dd7fa700 1 -- 172.21.15.43:0/44338133 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7d000e040 con 0x7fa7e0100880 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e0196110 con 0x7fa7e0100880 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7dd7fa700 1 -- 172.21.15.43:0/44338133 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa7d0012af0 con 0x7fa7e0100880 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7dd7fa700 1 -- 172.21.15.43:0/44338133 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7d001d9f0 con 0x7fa7e0100880 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.635+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e0196530 con 0x7fa7e0100880 2024-09-16T23:41:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.636+0000 7fa7dd7fa700 1 -- 172.21.15.43:0/44338133 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fa7d001dcb0 con 0x7fa7e0100880 2024-09-16T23:41:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.636+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e018f4a0 con 0x7fa7e0100880 2024-09-16T23:41:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.636+0000 7fa7dd7fa700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7cc0384b0 0x7fa7cc03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.637+0000 7fa7dd7fa700 1 -- 172.21.15.43:0/44338133 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7fa7d004cad0 con 0x7fa7e0100880 2024-09-16T23:41:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.637+0000 7fa7d6ffd700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7cc0384b0 0x7fa7cc03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.637+0000 7fa7d6ffd700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7cc0384b0 0x7fa7cc03a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8006fd0 tx=0x7fa7c8006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:19.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.646+0000 7fa7dd7fa700 1 -- 172.21.15.43:0/44338133 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa7d001c3e0 con 0x7fa7e0100880 2024-09-16T23:41:19.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.927+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa7e00623c0 con 0x7fa7e0100880 2024-09-16T23:41:19.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.927+0000 7fa7dd7fa700 1 -- 172.21.15.43:0/44338133 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fa7d0025360 con 0x7fa7e0100880 2024-09-16T23:41:19.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:19.925 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:19.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7cc0384b0 msgr2=0x7fa7cc03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:19.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7cc0384b0 0x7fa7cc03a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8006fd0 tx=0x7fa7c8006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:19.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 msgr2=0x7fa7e01959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:19.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e01959d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b830 tx=0x7fa7d000bd40 comp rx=0 tx=0).stop 2024-09-16T23:41:19.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 shutdown_connections 2024-09-16T23:41:19.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa7cc0384b0 0x7fa7cc03a970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:19.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 --2- 172.21.15.43:0/44338133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7e0100880 0x7fa7e01959d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:19.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 >> 172.21.15.43:0/44338133 conn(0x7fa7e00fbe00 msgr2=0x7fa7e00fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:19.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 shutdown_connections 2024-09-16T23:41:19.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:19.930+0000 7fa7e60c1700 1 -- 172.21.15.43:0/44338133 wait complete. 2024-09-16T23:41:19.928 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:20.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:20.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:20.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:20.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:20.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:41:20.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:20.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:41:21.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:41:21.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:41:21.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/44338133' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:21.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:21.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:21.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:21.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:21.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:21.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:21.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:21.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:41:21.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T23:41:21.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:21.607 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:21.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:22.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:21 smithi042 ceph-mon[27914]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:41:22.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:21 smithi042 ceph-mon[27914]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:41:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:21 smithi042 ceph-mon[27914]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:41:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:21 smithi042 ceph-mon[27914]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:41:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:21 smithi042 ceph-mon[27914]: Deploying daemon ceph-exporter.smithi043 on smithi043 2024-09-16T23:41:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.103+0000 7f9099f91700 1 -- 172.21.15.43:0/2499725339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 msgr2=0x7f9094105b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.103+0000 7f9099f91700 1 --2- 172.21.15.43:0/2499725339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094105b00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9084009910 tx=0x7f9084009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.103+0000 7f9099f91700 1 -- 172.21.15.43:0/2499725339 shutdown_connections 2024-09-16T23:41:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.103+0000 7f9099f91700 1 --2- 172.21.15.43:0/2499725339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094105b00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.103+0000 7f9099f91700 1 -- 172.21.15.43:0/2499725339 >> 172.21.15.43:0/2499725339 conn(0x7f90940fb230 msgr2=0x7f90940fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.103+0000 7f9099f91700 1 -- 172.21.15.43:0/2499725339 shutdown_connections 2024-09-16T23:41:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.103+0000 7f9099f91700 1 -- 172.21.15.43:0/2499725339 wait complete. 2024-09-16T23:41:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.104+0000 7f9099f91700 1 Processor -- start 2024-09-16T23:41:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.104+0000 7f9099f91700 1 -- start start 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.104+0000 7f9099f91700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.104+0000 7f9099f91700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90941970c0 con 0x7f9094105720 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:53716/0 (socket says 172.21.15.43:53716) 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90937fe700 1 -- 172.21.15.43:0/2484036488 learned_addr learned my addr 172.21.15.43:0/2484036488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90937fe700 1 -- 172.21.15.43:0/2484036488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840095f0 con 0x7f9094105720 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90937fe700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094196b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9084005b40 tx=0x7f9084004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90917fa700 1 -- 172.21.15.43:0/2484036488 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f908401d070 con 0x7f9094105720 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90941972c0 con 0x7f9094105720 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9094197740 con 0x7f9094105720 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90917fa700 1 -- 172.21.15.43:0/2484036488 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f90840044d0 con 0x7f9094105720 2024-09-16T23:41:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.105+0000 7f90917fa700 1 -- 172.21.15.43:0/2484036488 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9084003910 con 0x7f9094105720 2024-09-16T23:41:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.106+0000 7f90917fa700 1 -- 172.21.15.43:0/2484036488 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f9084003bc0 con 0x7f9094105720 2024-09-16T23:41:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.106+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90941039d0 con 0x7f9094105720 2024-09-16T23:41:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.106+0000 7f90917fa700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9080038500 0x7f908003a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.106+0000 7f90917fa700 1 -- 172.21.15.43:0/2484036488 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f908404c410 con 0x7f9094105720 2024-09-16T23:41:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.107+0000 7f908affd700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9080038500 0x7f908003a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.107+0000 7f908affd700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9080038500 0x7f908003a9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f907c006fd0 tx=0x7f907c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.112+0000 7f90917fa700 1 -- 172.21.15.43:0/2484036488 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f908402a540 con 0x7f9094105720 2024-09-16T23:41:24.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.407+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9094068f10 con 0x7f9094105720 2024-09-16T23:41:24.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.420+0000 7f90917fa700 1 -- 172.21.15.43:0/2484036488 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f908402a360 con 0x7f9094105720 2024-09-16T23:41:24.418 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:24.418 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:24.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9080038500 msgr2=0x7f908003a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:24.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9080038500 0x7f908003a9c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f907c006fd0 tx=0x7f907c006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:24.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 msgr2=0x7f9094196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:24.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094196b80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9084005b40 tx=0x7f9084004c80 comp rx=0 tx=0).stop 2024-09-16T23:41:24.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 shutdown_connections 2024-09-16T23:41:24.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9080038500 0x7f908003a9c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 --2- 172.21.15.43:0/2484036488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9094105720 0x7f9094196b80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 >> 172.21.15.43:0/2484036488 conn(0x7f90940fb230 msgr2=0x7f9094107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 shutdown_connections 2024-09-16T23:41:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:24.423+0000 7f9099f91700 1 -- 172.21.15.43:0/2484036488 wait complete. 2024-09-16T23:41:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:24.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:41:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/2484036488' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:41:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T23:41:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:25 smithi042 ceph-mon[27914]: Deploying daemon crash.smithi043 on smithi043 2024-09-16T23:41:26.281 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:26.282 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:26.482 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.043+0000 7f0f364b9700 1 -- 172.21.15.43:0/2656342823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 msgr2=0x7f0f30105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.043+0000 7f0f364b9700 1 --2- 172.21.15.43:0/2656342823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30105b60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009910 tx=0x7f0f20009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 -- 172.21.15.43:0/2656342823 shutdown_connections 2024-09-16T23:41:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 --2- 172.21.15.43:0/2656342823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30105b60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:28.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 -- 172.21.15.43:0/2656342823 >> 172.21.15.43:0/2656342823 conn(0x7f0f30075290 msgr2=0x7f0f300756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 -- 172.21.15.43:0/2656342823 shutdown_connections 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 -- 172.21.15.43:0/2656342823 wait complete. 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 Processor -- start 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 -- start start 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.044+0000 7f0f364b9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301970e0 con 0x7f0f30105780 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.045+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.045+0000 7f0f2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:45772/0 (socket says 172.21.15.43:45772) 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.045+0000 7f0f2ffff700 1 -- 172.21.15.43:0/117906754 learned_addr learned my addr 172.21.15.43:0/117906754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:28.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.045+0000 7f0f2ffff700 1 -- 172.21.15.43:0/117906754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f200095f0 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.045+0000 7f0f2ffff700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30196ba0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f20005b40 tx=0x7f0f20004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.045+0000 7f0f2dffb700 1 -- 172.21.15.43:0/117906754 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f2001d070 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.046+0000 7f0f2dffb700 1 -- 172.21.15.43:0/117906754 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0f200044d0 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.046+0000 7f0f2dffb700 1 -- 172.21.15.43:0/117906754 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0f20003930 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.046+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f301972e0 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.046+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f301976a0 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.046+0000 7f0f2dffb700 1 -- 172.21.15.43:0/117906754 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f0f20003bf0 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.047+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f301039c0 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.047+0000 7f0f2dffb700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0f1c0384b0 0x7f0f1c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.047+0000 7f0f2dffb700 1 -- 172.21.15.43:0/117906754 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f0f2004c3b0 con 0x7f0f30105780 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.047+0000 7f0f27fff700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0f1c0384b0 0x7f0f1c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:28.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.047+0000 7f0f27fff700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0f1c0384b0 0x7f0f1c03a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:28.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.052+0000 7f0f2dffb700 1 -- 172.21.15.43:0/117906754 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f2002a540 con 0x7f0f30105780 2024-09-16T23:41:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.339+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0f300623c0 con 0x7f0f30105780 2024-09-16T23:41:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.340+0000 7f0f2dffb700 1 -- 172.21.15.43:0/117906754 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f0f2002a360 con 0x7f0f30105780 2024-09-16T23:41:28.337 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:28.337 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0f1c0384b0 msgr2=0x7f0f1c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0f1c0384b0 0x7f0f1c03a970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006fd0 tx=0x7f0f18006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 msgr2=0x7f0f30196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30196ba0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f20005b40 tx=0x7f0f20004c80 comp rx=0 tx=0).stop 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 shutdown_connections 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0f1c0384b0 0x7f0f1c03a970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 --2- 172.21.15.43:0/117906754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f30105780 0x7f0f30196ba0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 >> 172.21.15.43:0/117906754 conn(0x7f0f30075290 msgr2=0x7f0f300fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 shutdown_connections 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:28.342+0000 7f0f364b9700 1 -- 172.21.15.43:0/117906754 wait complete. 2024-09-16T23:41:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:28 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/117906754' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:29.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:29.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:29.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:29 smithi042 ceph-mon[27914]: Deploying daemon node-exporter.smithi043 on smithi043 2024-09-16T23:41:29.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:30.012 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:30.012 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:30.214 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:30.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:30 smithi042 ceph-mon[27914]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:32 smithi042 ceph-mon[27914]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.326+0000 7f4ba531b700 1 -- 172.21.15.43:0/1681965677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 msgr2=0x7f4ba0105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.326+0000 7f4ba531b700 1 --2- 172.21.15.43:0/1681965677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba0105b60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009910 tx=0x7f4b90009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.326+0000 7f4ba531b700 1 -- 172.21.15.43:0/1681965677 shutdown_connections 2024-09-16T23:41:33.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.326+0000 7f4ba531b700 1 --2- 172.21.15.43:0/1681965677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba0105b60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.326+0000 7f4ba531b700 1 -- 172.21.15.43:0/1681965677 >> 172.21.15.43:0/1681965677 conn(0x7f4ba0075290 msgr2=0x7f4ba00756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.326+0000 7f4ba531b700 1 -- 172.21.15.43:0/1681965677 shutdown_connections 2024-09-16T23:41:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.326+0000 7f4ba531b700 1 -- 172.21.15.43:0/1681965677 wait complete. 2024-09-16T23:41:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.327+0000 7f4ba531b700 1 Processor -- start 2024-09-16T23:41:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.327+0000 7f4ba531b700 1 -- start start 2024-09-16T23:41:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.327+0000 7f4ba531b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba00fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:33.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.327+0000 7f4ba531b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba00feff0 con 0x7f4ba0105780 2024-09-16T23:41:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.327+0000 7f4b9effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba00fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:33.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.327+0000 7f4b9effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba00fd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:45788/0 (socket says 172.21.15.43:45788) 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.327+0000 7f4b9effd700 1 -- 172.21.15.43:0/1781781277 learned_addr learned my addr 172.21.15.43:0/1781781277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.328+0000 7f4b9effd700 1 -- 172.21.15.43:0/1781781277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900095f0 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.328+0000 7f4b9effd700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba00fd070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b900092e0 tx=0x7f4b90004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.328+0000 7f4b9cff9700 1 -- 172.21.15.43:0/1781781277 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b9001d070 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.328+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba00fd5b0 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.328+0000 7f4b9cff9700 1 -- 172.21.15.43:0/1781781277 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4b9000be10 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.328+0000 7f4b9cff9700 1 -- 172.21.15.43:0/1781781277 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4b90022e10 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.328+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba00fda30 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.329+0000 7f4b9cff9700 1 -- 172.21.15.43:0/1781781277 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f4b9000f460 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.329+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba00623c0 con 0x7f4ba0105780 2024-09-16T23:41:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.329+0000 7f4b9cff9700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4b8c038460 0x7f4b8c03a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.329+0000 7f4b9cff9700 1 -- 172.21.15.43:0/1781781277 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f4b9004d490 con 0x7f4ba0105780 2024-09-16T23:41:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.330+0000 7f4b97fff700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4b8c038460 0x7f4b8c03a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.330+0000 7f4b97fff700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4b8c038460 0x7f4b8c03a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4b88006fd0 tx=0x7f4b88006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:33.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.335+0000 7f4b9cff9700 1 -- 172.21.15.43:0/1781781277 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4b9000fe60 con 0x7f4ba0105780 2024-09-16T23:41:33.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.625+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ba00fe5a0 con 0x7f4ba0105780 2024-09-16T23:41:33.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.625+0000 7f4b9cff9700 1 -- 172.21.15.43:0/1781781277 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4b90027030 con 0x7f4ba0105780 2024-09-16T23:41:33.623 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:33.623 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.627+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4b8c038460 msgr2=0x7f4b8c03a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.627+0000 7f4ba531b700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4b8c038460 0x7f4b8c03a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4b88006fd0 tx=0x7f4b88006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.627+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 msgr2=0x7f4ba00fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.627+0000 7f4ba531b700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba00fd070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b900092e0 tx=0x7f4b90004c80 comp rx=0 tx=0).stop 2024-09-16T23:41:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.627+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 shutdown_connections 2024-09-16T23:41:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.627+0000 7f4ba531b700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4b8c038460 0x7f4b8c03a920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:33.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.628+0000 7f4ba531b700 1 --2- 172.21.15.43:0/1781781277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ba0105780 0x7f4ba00fd070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.628+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 >> 172.21.15.43:0/1781781277 conn(0x7f4ba0075290 msgr2=0x7f4ba0103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.628+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 shutdown_connections 2024-09-16T23:41:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:33.628+0000 7f4ba531b700 1 -- 172.21.15.43:0/1781781277 wait complete. 2024-09-16T23:41:33.626 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:34.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: Deploying daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1781781277' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:35.739 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:35.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:36.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:36 smithi042 ceph-mon[27914]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:37.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:37.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:37.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:41:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:37 smithi042 ceph-mon[27914]: Deploying daemon mon.smithi043 on smithi043 2024-09-16T23:41:38.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.755+0000 7fdb52b28700 1 -- 172.21.15.43:0/1211947799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 msgr2=0x7fdb4c101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:38.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.755+0000 7fdb52b28700 1 --2- 172.21.15.43:0/1211947799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c101da0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009910 tx=0x7fdb48009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:38.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.755+0000 7fdb52b28700 1 -- 172.21.15.43:0/1211947799 shutdown_connections 2024-09-16T23:41:38.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.755+0000 7fdb52b28700 1 --2- 172.21.15.43:0/1211947799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c101da0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:38.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.755+0000 7fdb52b28700 1 -- 172.21.15.43:0/1211947799 >> 172.21.15.43:0/1211947799 conn(0x7fdb4c0fb210 msgr2=0x7fdb4c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:38.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.755+0000 7fdb52b28700 1 -- 172.21.15.43:0/1211947799 shutdown_connections 2024-09-16T23:41:38.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.755+0000 7fdb52b28700 1 -- 172.21.15.43:0/1211947799 wait complete. 2024-09-16T23:41:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.756+0000 7fdb52b28700 1 Processor -- start 2024-09-16T23:41:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.756+0000 7fdb52b28700 1 -- start start 2024-09-16T23:41:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.756+0000 7fdb52b28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.756+0000 7fdb52b28700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c196380 con 0x7fdb4c1019c0 2024-09-16T23:41:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.756+0000 7fdb508c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:38.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb508c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:57910/0 (socket says 172.21.15.43:57910) 2024-09-16T23:41:38.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb508c4700 1 -- 172.21.15.43:0/1365253556 learned_addr learned my addr 172.21.15.43:0/1365253556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:38.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb508c4700 1 -- 172.21.15.43:0/1365253556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480095f0 con 0x7fdb4c1019c0 2024-09-16T23:41:38.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb508c4700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c195e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdb48005b40 tx=0x7fdb48004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb41ffb700 1 -- 172.21.15.43:0/1365253556 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb4801d070 con 0x7fdb4c1019c0 2024-09-16T23:41:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb41ffb700 1 -- 172.21.15.43:0/1365253556 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb480044d0 con 0x7fdb4c1019c0 2024-09-16T23:41:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb41ffb700 1 -- 172.21.15.43:0/1365253556 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb48003930 con 0x7fdb4c1019c0 2024-09-16T23:41:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.757+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c196580 con 0x7fdb4c1019c0 2024-09-16T23:41:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.758+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c196940 con 0x7fdb4c1019c0 2024-09-16T23:41:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.759+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c0623c0 con 0x7fdb4c1019c0 2024-09-16T23:41:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.759+0000 7fdb41ffb700 1 -- 172.21.15.43:0/1365253556 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7fdb4800b960 con 0x7fdb4c1019c0 2024-09-16T23:41:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.759+0000 7fdb41ffb700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb34038500 0x7fdb3403a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.759+0000 7fdb41ffb700 1 -- 172.21.15.43:0/1365253556 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7fdb4804c4e0 con 0x7fdb4c1019c0 2024-09-16T23:41:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.759+0000 7fdb43fff700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb34038500 0x7fdb3403a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:38.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.760+0000 7fdb43fff700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb34038500 0x7fdb3403a9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006fd0 tx=0x7fdb3c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:38.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:38.764+0000 7fdb41ffb700 1 -- 172.21.15.43:0/1365253556 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb48004330 con 0x7fdb4c1019c0 2024-09-16T23:41:38.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:38 smithi042 ceph-mon[27914]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:39.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.055+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdb4c107950 con 0x7fdb4c1019c0 2024-09-16T23:41:39.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.056+0000 7fdb41ffb700 1 -- 172.21.15.43:0/1365253556 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fdb4802a360 con 0x7fdb4c1019c0 2024-09-16T23:41:39.054 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:39.054 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":1,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:38:57.273836Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T23:41:39.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb34038500 msgr2=0x7fdb3403a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:39.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb34038500 0x7fdb3403a9c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006fd0 tx=0x7fdb3c006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:39.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 msgr2=0x7fdb4c195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c195e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdb48005b40 tx=0x7fdb48004c80 comp rx=0 tx=0).stop 2024-09-16T23:41:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 shutdown_connections 2024-09-16T23:41:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb34038500 0x7fdb3403a9c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 --2- 172.21.15.43:0/1365253556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb4c1019c0 0x7fdb4c195e40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 >> 172.21.15.43:0/1365253556 conn(0x7fdb4c0fb210 msgr2=0x7fdb4c105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:39.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 shutdown_connections 2024-09-16T23:41:39.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:39.058+0000 7fdb52b28700 1 -- 172.21.15.43:0/1365253556 wait complete. 2024-09-16T23:41:39.058 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 1 2024-09-16T23:41:39.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:39 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1365253556' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:39.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:41:39.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:40.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:40 smithi042 ceph-mon[27914]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:40.776 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T23:41:40.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T23:41:41.065 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:41:42.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:42 smithi042 ceph-mon[27914]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:43.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.151+0000 7f9a430d7700 1 -- 172.21.15.43:0/1274356800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 msgr2=0x7f9a3c100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.151+0000 7f9a430d7700 1 --2- 172.21.15.43:0/1274356800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c100790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009910 tx=0x7f9a30009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.151+0000 7f9a430d7700 1 -- 172.21.15.43:0/1274356800 shutdown_connections 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.151+0000 7f9a430d7700 1 --2- 172.21.15.43:0/1274356800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c100790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.151+0000 7f9a430d7700 1 -- 172.21.15.43:0/1274356800 >> 172.21.15.43:0/1274356800 conn(0x7f9a3c074ae0 msgr2=0x7f9a3c074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.151+0000 7f9a430d7700 1 -- 172.21.15.43:0/1274356800 shutdown_connections 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.151+0000 7f9a430d7700 1 -- 172.21.15.43:0/1274356800 wait complete. 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.152+0000 7f9a430d7700 1 Processor -- start 2024-09-16T23:41:43.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.152+0000 7f9a430d7700 1 -- start start 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.152+0000 7f9a430d7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c0fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.152+0000 7f9a430d7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c0fe070 con 0x7f9a3c1003b0 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.152+0000 7f9a40e73700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c0fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.152+0000 7f9a40e73700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c0fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.43:57916/0 (socket says 172.21.15.43:57916) 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.152+0000 7f9a40e73700 1 -- 172.21.15.43:0/3505900916 learned_addr learned my addr 172.21.15.43:0/3505900916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.153+0000 7f9a40e73700 1 -- 172.21.15.43:0/3505900916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a300095f0 con 0x7f9a3c1003b0 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.153+0000 7f9a40e73700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c0fc0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9a30005b40 tx=0x7f9a30004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.153+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a3001d070 con 0x7f9a3c1003b0 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.153+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9a300044d0 con 0x7f9a3c1003b0 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.153+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c0fc630 con 0x7f9a3c1003b0 2024-09-16T23:41:43.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.153+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a30003930 con 0x7f9a3c1003b0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.153+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c0fcab0 con 0x7f9a3c1003b0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.154+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 45396+0+0 (secure 0 0 0) 0x7f9a30003bf0 con 0x7f9a3c1003b0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.154+0000 7f9a39ffb700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9a2c0384b0 0x7f9a2c03a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.154+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f9a30004020 con 0x7f9a3c1003b0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.154+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c0623c0 con 0x7f9a3c1003b0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.155+0000 7f9a3bfff700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9a2c0384b0 0x7f9a2c03a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.155+0000 7f9a3bfff700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9a2c0384b0 0x7f9a2c03a970 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006fd0 tx=0x7f9a28006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.160+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9a3002a950 con 0x7f9a3c1003b0 2024-09-16T23:41:43.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.205+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a3002a950 con 0x7f9a3c1003b0 2024-09-16T23:41:43.325 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:43 smithi043 systemd[1]: Started Ceph mon.smithi043 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:41:43.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:43.449+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9a3c0fcde0 con 0x7f9a3c1003b0 2024-09-16T23:41:48.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.215+0000 7f9a39ffb700 1 -- 172.21.15.43:0/3505900916 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1018 (secure 0 0 0) 0x7f9a3002a360 con 0x7f9a3c1003b0 2024-09-16T23:41:48.213 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:41:48.213 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","modified":"2024-09-16T23:41:43.205311Z","created":"2024-09-16T23:38:57.273836Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi042","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi043","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T23:41:48.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9a2c0384b0 msgr2=0x7f9a2c03a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:48.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9a2c0384b0 0x7f9a2c03a970 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a28006fd0 tx=0x7f9a28006e40 comp rx=0 tx=0).stop 2024-09-16T23:41:48.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 msgr2=0x7f9a3c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:48.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c0fc0f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9a30005b40 tx=0x7f9a30004c80 comp rx=0 tx=0).stop 2024-09-16T23:41:48.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 shutdown_connections 2024-09-16T23:41:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9a2c0384b0 0x7f9a2c03a970 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 --2- 172.21.15.43:0/3505900916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a3c1003b0 0x7f9a3c0fc0f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 >> 172.21.15.43:0/3505900916 conn(0x7f9a3c074ae0 msgr2=0x7f9a3c103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 shutdown_connections 2024-09-16T23:41:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:41:48.218+0000 7f9a430d7700 1 -- 172.21.15.43:0/3505900916 wait complete. 2024-09-16T23:41:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 2 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: mon.smithi042 calling monitor election 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/3505900916' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: mon.smithi043 calling monitor election 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: mon.smithi042 is new leader, mons smithi042,smithi043 in quorum (ranks 0,1) 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi043=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: fsmap 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: mgrmap e17: smithi042.zrkumz(active, since 39s) 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:41:48.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: overall HEALTH_OK 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.576 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: mon.smithi042 calling monitor election 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/3505900916' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: mon.smithi043 calling monitor election 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:48.577 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: mon.smithi042 is new leader, mons smithi042,smithi043 in quorum (ranks 0,1) 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: monmap e2: 2 mons at {smithi042=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],smithi043=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: fsmap 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: mgrmap e17: smithi042.zrkumz(active, since 39s) 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: overall HEALTH_OK 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/4210663176' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.578 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.579 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:48.896 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T23:41:48.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T23:41:49.119 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:41:49.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:49 smithi042 ceph-mon[27914]: mgrmap e18: smithi042.zrkumz(active, since 39s), standbys: smithi043.xveyou 2024-09-16T23:41:49.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:41:49.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:41:49.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:49.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:49.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:49 smithi043 ceph-mon[35191]: mgrmap e18: smithi042.zrkumz(active, since 39s), standbys: smithi043.xveyou 2024-09-16T23:41:49.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:41:49.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:41:49.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:49.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:41:50.517 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:50 smithi042 ceph-mon[27914]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:50.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:50 smithi043 ceph-mon[35191]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:50.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.758+0000 7f9f671ce700 1 -- 172.21.15.42:0/695832433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 msgr2=0x7f9f60105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.758+0000 7f9f671ce700 1 --2- 172.21.15.42:0/695832433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f60105bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009910 tx=0x7f9f54009c20 comp rx=0 tx=0).stop 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.758+0000 7f9f671ce700 1 -- 172.21.15.42:0/695832433 shutdown_connections 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.758+0000 7f9f671ce700 1 --2- 172.21.15.42:0/695832433 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f60105bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.758+0000 7f9f671ce700 1 -- 172.21.15.42:0/695832433 >> 172.21.15.42:0/695832433 conn(0x7f9f60075df0 msgr2=0x7f9f60076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.758+0000 7f9f671ce700 1 -- 172.21.15.42:0/695832433 shutdown_connections 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.758+0000 7f9f671ce700 1 -- 172.21.15.42:0/695832433 wait complete. 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f671ce700 1 Processor -- start 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f671ce700 1 -- start start 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f671ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f600fff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f671ce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 0x7f9f601008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:50.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f671ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601a2060 con 0x7f9f60105810 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f671ce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f601a21a0 con 0x7f9f60100460 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f64f6a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f600fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f64f6a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f600fff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34752/0 (socket says 172.21.15.42:34752) 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f64f6a700 1 -- 172.21.15.42:0/2522575283 learned_addr learned my addr 172.21.15.42:0/2522575283 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f5ffff700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 0x7f9f601008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f5ffff700 1 -- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 msgr2=0x7f9f601008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f5ffff700 1 -- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 msgr2=0x7f9f601008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f5ffff700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 0x7f9f601008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f64f6a700 1 -- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 msgr2=0x7f9f601008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f5ffff700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 0x7f9f601008e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f64f6a700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 0x7f9f601008e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:50.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.759+0000 7f9f64f6a700 1 -- 172.21.15.42:0/2522575283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f540095f0 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.760+0000 7f9f64f6a700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f600fff20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f540092e0 tx=0x7f9f5400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.760+0000 7f9f5dffb700 1 -- 172.21.15.42:0/2522575283 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5401d070 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.760+0000 7f9f5dffb700 1 -- 172.21.15.42:0/2522575283 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f54005140 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.760+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f601a22e0 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.760+0000 7f9f5dffb700 1 -- 172.21.15.42:0/2522575283 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f54017440 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.760+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f601a2830 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.761+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f600623c0 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.761+0000 7f9f5dffb700 1 -- 172.21.15.42:0/2522575283 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f9f540175a0 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.762+0000 7f9f5dffb700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9f5006c5c0 0x7f9f5006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.762+0000 7f9f5dffb700 1 -- 172.21.15.42:0/2522575283 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f9f5408b900 con 0x7f9f60105810 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.762+0000 7f9f5ffff700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9f5006c5c0 0x7f9f5006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:41:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.763+0000 7f9f5ffff700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9f5006c5c0 0x7f9f5006ea80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009850 tx=0x7f9f4c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:41:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.767+0000 7f9f5dffb700 1 -- 172.21.15.42:0/2522575283 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9f54057270 con 0x7f9f60105810 2024-09-16T23:41:50.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.970+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9f60106300 con 0x7f9f60105810 2024-09-16T23:41:50.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.971+0000 7f9f5dffb700 1 -- 172.21.15.42:0/2522575283 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7f9f60106300 con 0x7f9f60105810 2024-09-16T23:41:50.968 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:41:50.968 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-09-16T23:41:50.969 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:41:50.969 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] 2024-09-16T23:41:50.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9f5006c5c0 msgr2=0x7f9f5006ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:50.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9f5006c5c0 0x7f9f5006ea80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009850 tx=0x7f9f4c008040 comp rx=0 tx=0).stop 2024-09-16T23:41:50.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 msgr2=0x7f9f600fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f600fff20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9f540092e0 tx=0x7f9f5400bda0 comp rx=0 tx=0).stop 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 shutdown_connections 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f9f5006c5c0 0x7f9f5006ea80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f60105810 0x7f9f600fff20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 --2- 172.21.15.42:0/2522575283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f60100460 0x7f9f601008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 >> 172.21.15.42:0/2522575283 conn(0x7f9f60075df0 msgr2=0x7f9f60102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 shutdown_connections 2024-09-16T23:41:50.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:41:50.973+0000 7f9f671ce700 1 -- 172.21.15.42:0/2522575283 wait complete. 2024-09-16T23:41:51.641 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T23:41:51.642 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:41:51.642 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T23:41:51.671 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:41:51.671 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:41:51.737 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:41:51.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T23:41:51.770 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:41:51.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:41:51.839 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T23:41:51.839 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:41:51.839 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T23:41:51.855 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T23:41:51.855 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-09-16T23:41:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2522575283' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:41:51.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:51.901 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.901 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.901 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:51.901 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:41:51.901 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2522575283' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:41:51.902 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:51.912 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 23:41:00.269500972 +0000 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 23:35:59.185473887 +0000 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 23:35:59.185473887 +0000 2024-09-16T23:41:51.913 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-16 23:35:59.185473887 +0000 2024-09-16T23:41:51.913 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T23:41:51.979 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T23:41:51.979 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T23:41:51.979 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000104302 s, 4.9 MB/s 2024-09-16T23:41:51.980 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T23:41:52.037 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-09-16T23:41:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T23:41:52.094 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:52.095 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T23:41:52.095 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:52.095 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:52.095 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 23:41:00.269500972 +0000 2024-09-16T23:41:52.095 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 23:35:59.502471402 +0000 2024-09-16T23:41:52.095 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 23:35:59.502471402 +0000 2024-09-16T23:41:52.095 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-16 23:35:59.502471402 +0000 2024-09-16T23:41:52.095 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T23:41:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T23:41:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T23:41:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.00010684 s, 4.8 MB/s 2024-09-16T23:41:52.161 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T23:41:52.217 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-09-16T23:41:52.274 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T23:41:52.274 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:52.274 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T23:41:52.274 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:52.274 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:52.274 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 23:41:00.269500972 +0000 2024-09-16T23:41:52.275 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 23:35:59.829468840 +0000 2024-09-16T23:41:52.275 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 23:35:59.829468840 +0000 2024-09-16T23:41:52.275 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-16 23:35:59.829468840 +0000 2024-09-16T23:41:52.275 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T23:41:52.338 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T23:41:52.338 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T23:41:52.338 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000104822 s, 4.9 MB/s 2024-09-16T23:41:52.340 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T23:41:52.397 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-09-16T23:41:52.452 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-09-16 23:41:00.270500964 +0000 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 23:36:00.151466316 +0000 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 23:36:00.151466316 +0000 2024-09-16T23:41:52.453 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-09-16 23:36:00.151466316 +0000 2024-09-16T23:41:52.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T23:41:52.517 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-09-16T23:41:52.517 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-09-16T23:41:52.517 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.00010798 s, 4.7 MB/s 2024-09-16T23:41:52.519 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T23:41:52.576 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:41:52.576 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T23:41:52.596 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T23:41:52.596 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-09-16T23:41:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T23:41:52.655 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:52.656 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T23:41:52.656 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:52.656 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:52.656 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 23:41:41.424247509 +0000 2024-09-16T23:41:52.656 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 23:35:59.838600062 +0000 2024-09-16T23:41:52.656 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 23:35:59.838600062 +0000 2024-09-16T23:41:52.656 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 23:35:59.838600062 +0000 2024-09-16T23:41:52.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T23:41:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T23:41:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T23:41:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000111438 s, 4.6 MB/s 2024-09-16T23:41:52.725 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T23:41:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:52 smithi042 ceph-mon[27914]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:52 smithi042 ceph-mon[27914]: Reconfiguring mon.smithi042 (unknown last config time)... 2024-09-16T23:41:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:52 smithi042 ceph-mon[27914]: Reconfiguring daemon mon.smithi042 on smithi042 2024-09-16T23:41:52.785 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-09-16T23:41:52.844 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T23:41:52.844 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:52.844 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T23:41:52.844 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:52.844 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:52.844 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 23:41:41.424247509 +0000 2024-09-16T23:41:52.844 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 23:36:00.173596199 +0000 2024-09-16T23:41:52.845 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 23:36:00.173596199 +0000 2024-09-16T23:41:52.845 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 23:36:00.173596199 +0000 2024-09-16T23:41:52.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T23:41:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:52 smithi043 ceph-mon[35191]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:52 smithi043 ceph-mon[35191]: Reconfiguring mon.smithi042 (unknown last config time)... 2024-09-16T23:41:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:52 smithi043 ceph-mon[35191]: Reconfiguring daemon mon.smithi042 on smithi042 2024-09-16T23:41:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T23:41:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T23:41:52.905 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000123822 s, 4.1 MB/s 2024-09-16T23:41:52.906 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T23:41:52.965 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-09-16T23:41:53.023 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T23:41:53.023 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:53.023 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T23:41:53.023 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:53.023 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:53.024 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 23:41:41.424247509 +0000 2024-09-16T23:41:53.024 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 23:36:00.500592429 +0000 2024-09-16T23:41:53.024 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 23:36:00.500592429 +0000 2024-09-16T23:41:53.024 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 23:36:00.500592429 +0000 2024-09-16T23:41:53.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T23:41:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T23:41:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T23:41:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000123318 s, 4.2 MB/s 2024-09-16T23:41:53.092 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T23:41:53.151 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-09-16T23:41:53.208 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-09-16 23:41:41.425247498 +0000 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 23:36:00.834588578 +0000 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 23:36:00.834588578 +0000 2024-09-16T23:41:53.209 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-09-16 23:36:00.834588578 +0000 2024-09-16T23:41:53.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T23:41:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-09-16T23:41:53.275 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-09-16T23:41:53.276 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000112635 s, 4.5 MB/s 2024-09-16T23:41:53.277 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T23:41:53.335 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2024-09-16T23:41:53.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T23:41:54.099 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:54.099 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:54.099 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:53 smithi042 ceph-mon[27914]: Reconfiguring mgr.smithi042.zrkumz (unknown last config time)... 2024-09-16T23:41:54.099 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:41:54.099 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:41:54.099 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:54.099 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:53 smithi042 ceph-mon[27914]: Reconfiguring daemon mgr.smithi042.zrkumz on smithi042 2024-09-16T23:41:54.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:54.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:54.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:53 smithi043 ceph-mon[35191]: Reconfiguring mgr.smithi042.zrkumz (unknown last config time)... 2024-09-16T23:41:54.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:41:54.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:41:54.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:54.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:53 smithi043 ceph-mon[35191]: Reconfiguring daemon mgr.smithi042.zrkumz on smithi042 2024-09-16T23:41:54.484 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:41:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:54 smithi042 ceph-mon[27914]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:41:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:41:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:54 smithi043 ceph-mon[35191]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:41:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:55.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:55.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:41:55.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:56.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:55 smithi043 ceph-mon[35191]: Reconfiguring ceph-exporter.smithi042 (monmap changed)... 2024-09-16T23:41:56.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:55 smithi043 ceph-mon[35191]: Reconfiguring daemon ceph-exporter.smithi042 on smithi042 2024-09-16T23:41:56.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:55 smithi042 ceph-mon[27914]: Reconfiguring ceph-exporter.smithi042 (monmap changed)... 2024-09-16T23:41:56.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:55 smithi042 ceph-mon[27914]: Reconfiguring daemon ceph-exporter.smithi042 on smithi042 2024-09-16T23:41:57.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:56 smithi043 ceph-mon[35191]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:57.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:57.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:41:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:57.164 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:56 smithi042 ceph-mon[27914]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:57.164 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:57.164 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:41:57.164 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:41:57.164 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:41:58.051 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:57 smithi042 ceph-mon[27914]: Reconfiguring crash.smithi042 (monmap changed)... 2024-09-16T23:41:58.051 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:57 smithi042 ceph-mon[27914]: Reconfiguring daemon crash.smithi042 on smithi042 2024-09-16T23:41:58.054 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:41:58.075 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2024-09-16T23:41:58.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:57 smithi043 ceph-mon[35191]: Reconfiguring crash.smithi042 (monmap changed)... 2024-09-16T23:41:58.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:57 smithi043 ceph-mon[35191]: Reconfiguring daemon crash.smithi042 on smithi042 2024-09-16T23:41:59.105 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:41:58 smithi042 ceph-mon[27914]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:59.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:41:58 smithi043 ceph-mon[35191]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:41:59.256 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:42:00.516 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:00 smithi042 ceph-mon[27914]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:00.516 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:00.516 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:00.516 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:00 smithi042 ceph-mon[27914]: Reconfiguring alertmanager.smithi042 (dependencies changed)... 2024-09-16T23:42:00.516 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:00 smithi042 ceph-mon[27914]: Reconfiguring daemon alertmanager.smithi042 on smithi042 2024-09-16T23:42:00.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:00 smithi043 ceph-mon[35191]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:00.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:00.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:00.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:00 smithi043 ceph-mon[35191]: Reconfiguring alertmanager.smithi042 (dependencies changed)... 2024-09-16T23:42:00.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:00 smithi043 ceph-mon[35191]: Reconfiguring daemon alertmanager.smithi042 on smithi042 2024-09-16T23:42:01.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 -- 172.21.15.42:0/925238341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 msgr2=0x7f91a8101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:01.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 --2- 172.21.15.42:0/925238341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a8101220 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9194009910 tx=0x7f9194009c20 comp rx=0 tx=0).stop 2024-09-16T23:42:01.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 -- 172.21.15.42:0/925238341 shutdown_connections 2024-09-16T23:42:01.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 --2- 172.21.15.42:0/925238341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a8101220 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:01.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 --2- 172.21.15.42:0/925238341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80ffbc0 0x7f91a80fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:01.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 -- 172.21.15.42:0/925238341 >> 172.21.15.42:0/925238341 conn(0x7f91a80fb160 msgr2=0x7f91a80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:42:01.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 -- 172.21.15.42:0/925238341 shutdown_connections 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 -- 172.21.15.42:0/925238341 wait complete. 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 Processor -- start 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.758+0000 7f91aeff3700 1 -- start start 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91aeff3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80ffbc0 0x7f91a8069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91aeff3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a819b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91aeff3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819bb90 con 0x7f91a8100dc0 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91aeff3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a819bcd0 con 0x7f91a80ffbc0 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a819b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a819b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49548/0 (socket says 172.21.15.42:49548) 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a7fff700 1 -- 172.21.15.42:0/6222767 learned_addr learned my addr 172.21.15.42:0/6222767 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91acd8f700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80ffbc0 0x7f91a8069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a7fff700 1 -- 172.21.15.42:0/6222767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80ffbc0 msgr2=0x7f91a8069970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a7fff700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80ffbc0 0x7f91a8069970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:01.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a7fff700 1 -- 172.21.15.42:0/6222767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91940095f0 con 0x7f91a8100dc0 2024-09-16T23:42:01.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91acd8f700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80ffbc0 0x7f91a8069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:42:01.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a7fff700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a819b5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9194005b40 tx=0x7f919400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:42:01.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a5ffb700 1 -- 172.21.15.42:0/6222767 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f919401d070 con 0x7f91a8100dc0 2024-09-16T23:42:01.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a5ffb700 1 -- 172.21.15.42:0/6222767 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9194022470 con 0x7f91a8100dc0 2024-09-16T23:42:01.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a819bf50 con 0x7f91a8100dc0 2024-09-16T23:42:01.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.759+0000 7f91a5ffb700 1 -- 172.21.15.42:0/6222767 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f919400f670 con 0x7f91a8100dc0 2024-09-16T23:42:01.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.760+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a819c410 con 0x7f91a8100dc0 2024-09-16T23:42:01.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.761+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a818f5d0 con 0x7f91a8100dc0 2024-09-16T23:42:01.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.761+0000 7f91a5ffb700 1 -- 172.21.15.42:0/6222767 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f91940225e0 con 0x7f91a8100dc0 2024-09-16T23:42:01.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.762+0000 7f91a5ffb700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f919806c470 0x7f919806e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:42:01.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.762+0000 7f91a5ffb700 1 -- 172.21.15.42:0/6222767 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f919408bc50 con 0x7f91a8100dc0 2024-09-16T23:42:01.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.762+0000 7f91acd8f700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f919806c470 0x7f919806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:42:01.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.762+0000 7f91acd8f700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f919806c470 0x7f919806e930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91a8100c20 tx=0x7f919c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:42:01.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.766+0000 7f91a5ffb700 1 -- 172.21.15.42:0/6222767 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9194091020 con 0x7f91a8100dc0 2024-09-16T23:42:01.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:01.974+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f91a802cfe0 con 0x7f919806c470 2024-09-16T23:42:02.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:02 smithi042 ceph-mon[27914]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:02.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:02 smithi042 ceph-mon[27914]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:42:02.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:02 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:42:02.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:02 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:42:02.497 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:02 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:02.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:02 smithi043 ceph-mon[35191]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:02.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:02 smithi043 ceph-mon[35191]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:42:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:02 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:42:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:02 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:42:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:02 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:04.432 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:04 smithi042 ceph-mon[27914]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:04.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:04 smithi043 ceph-mon[35191]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:06.357 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:06 smithi042 ceph-mon[27914]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:06.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:06 smithi043 ceph-mon[35191]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:07.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:07 smithi042 ceph-mon[27914]: Reconfiguring grafana.smithi042 (dependencies changed)... 2024-09-16T23:42:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:07 smithi042 ceph-mon[27914]: Reconfiguring daemon grafana.smithi042 on smithi042 2024-09-16T23:42:07.810 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:07.810 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:07.810 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:07 smithi043 ceph-mon[35191]: Reconfiguring grafana.smithi042 (dependencies changed)... 2024-09-16T23:42:07.810 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:07 smithi043 ceph-mon[35191]: Reconfiguring daemon grafana.smithi042 on smithi042 2024-09-16T23:42:08.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:08 smithi042 ceph-mon[27914]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:08.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:08 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/4187224784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c23a1b-b658-4424-9c3b-f324a6b0861b"}]: dispatch 2024-09-16T23:42:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:08 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/4187224784' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82c23a1b-b658-4424-9c3b-f324a6b0861b"}]': finished 2024-09-16T23:42:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:08 smithi042 ceph-mon[27914]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T23:42:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:08 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:08 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/51911910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:42:08.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:08 smithi043 ceph-mon[35191]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:08.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:08 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/4187224784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c23a1b-b658-4424-9c3b-f324a6b0861b"}]: dispatch 2024-09-16T23:42:08.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:08 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/4187224784' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82c23a1b-b658-4424-9c3b-f324a6b0861b"}]': finished 2024-09-16T23:42:08.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:08 smithi043 ceph-mon[35191]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T23:42:08.808 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:08 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:08.808 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:08 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/51911910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:42:09.770 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:09 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:09.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:10.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:10 smithi042 ceph-mon[27914]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:10.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:10 smithi043 ceph-mon[35191]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:12.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:12 smithi042 ceph-mon[27914]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:12.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:12 smithi043 ceph-mon[35191]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:14.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:14 smithi042 ceph-mon[27914]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:14.777 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:14 smithi043 ceph-mon[35191]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:16.576 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:16 smithi042 ceph-mon[27914]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:16.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:16 smithi043 ceph-mon[35191]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:17.797 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:17 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:17.798 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:17 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:17.798 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:17 smithi042 ceph-mon[27914]: Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-09-16T23:42:17.798 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:17 smithi042 ceph-mon[27914]: Reconfiguring daemon prometheus.smithi042 on smithi042 2024-09-16T23:42:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:17 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:18.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:17 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:18.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:17 smithi043 ceph-mon[35191]: Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-09-16T23:42:18.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:17 smithi043 ceph-mon[35191]: Reconfiguring daemon prometheus.smithi042 on smithi042 2024-09-16T23:42:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:18 smithi042 ceph-mon[27914]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:18 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T23:42:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:18 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:18 smithi043 ceph-mon[35191]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:18 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T23:42:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:18 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:20.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:19 smithi042 ceph-mon[27914]: Deploying daemon osd.0 on smithi042 2024-09-16T23:42:20.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:19 smithi043 ceph-mon[35191]: Deploying daemon osd.0 on smithi042 2024-09-16T23:42:20.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:20 smithi042 ceph-mon[27914]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:21.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:20 smithi043 ceph-mon[35191]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:23.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:22 smithi042 ceph-mon[27914]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:42:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:23.056 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:22 smithi043 ceph-mon[35191]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:23.056 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:23.056 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:23.056 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:42:23.056 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:23.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:23 smithi042 ceph-mon[27914]: Reconfiguring ceph-exporter.smithi043 (monmap changed)... 2024-09-16T23:42:23.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:23 smithi042 ceph-mon[27914]: Reconfiguring daemon ceph-exporter.smithi043 on smithi043 2024-09-16T23:42:24.029 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:23 smithi043 ceph-mon[35191]: Reconfiguring ceph-exporter.smithi043 (monmap changed)... 2024-09-16T23:42:24.029 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:23 smithi043 ceph-mon[35191]: Reconfiguring daemon ceph-exporter.smithi043 on smithi043 2024-09-16T23:42:24.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:24 smithi042 ceph-mon[27914]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:24.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:24.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:24.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:24.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:42:24.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:24.810 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:24 smithi043 ceph-mon[35191]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:24.810 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:24.811 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:24.811 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:24.811 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:42:24.811 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:25.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:25 smithi043 ceph-mon[35191]: Reconfiguring crash.smithi043 (monmap changed)... 2024-09-16T23:42:25.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:25 smithi043 ceph-mon[35191]: Reconfiguring daemon crash.smithi043 on smithi043 2024-09-16T23:42:25.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:25 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:25.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:25 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:25.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:25 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:42:25.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:25 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:42:25.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:25 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:25.940 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:25 smithi042 ceph-mon[27914]: Reconfiguring crash.smithi043 (monmap changed)... 2024-09-16T23:42:25.941 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:25 smithi042 ceph-mon[27914]: Reconfiguring daemon crash.smithi043 on smithi043 2024-09-16T23:42:25.941 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:25.941 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:25.941 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:42:25.941 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:42:25.941 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:26 smithi042 ceph-mon[27914]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:26 smithi042 ceph-mon[27914]: Reconfiguring mgr.smithi043.xveyou (monmap changed)... 2024-09-16T23:42:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:26 smithi042 ceph-mon[27914]: Reconfiguring daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:42:26.951 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:26 smithi043 ceph-mon[35191]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:26.951 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:26 smithi043 ceph-mon[35191]: Reconfiguring mgr.smithi043.xveyou (monmap changed)... 2024-09-16T23:42:26.951 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:26 smithi043 ceph-mon[35191]: Reconfiguring daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:42:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:42:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:42:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:28.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:28.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:42:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:42:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.154 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: Reconfiguring mon.smithi043 (monmap changed)... 2024-09-16T23:42:29.154 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: Reconfiguring daemon mon.smithi043 on smithi043 2024-09-16T23:42:29.154 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:29.154 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.154 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.154 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:42:29.154 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.155 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:42:29.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: Reconfiguring mon.smithi043 (monmap changed)... 2024-09-16T23:42:29.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: Reconfiguring daemon mon.smithi043 on smithi043 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:42:30.120 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:30 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:42:30.120 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:30 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T23:42:30.120 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:30 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:42:30.120 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:30 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T23:42:30.120 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:30 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:42:30.120 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:30 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T23:42:30.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:30 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:42:30.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:30 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi042.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T23:42:30.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:30 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:42:30.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:30 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi042.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T23:42:30.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:30 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:42:30.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:30 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi042.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T23:42:31.053 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:31 smithi042 ceph-mon[27914]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:31.053 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:31.053 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:31.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:31 smithi043 ceph-mon[35191]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:31.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:31.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:31.473 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2024-09-16T23:42:31.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.475+0000 7f91a5ffb700 1 -- 172.21.15.42:0/6222767 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f91a802cfe0 con 0x7f919806c470 2024-09-16T23:42:31.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f919806c470 msgr2=0x7f919806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f919806c470 0x7f919806e930 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f91a8100c20 tx=0x7f919c008040 comp rx=0 tx=0).stop 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 msgr2=0x7f91a819b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a819b5c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9194005b40 tx=0x7f919400be20 comp rx=0 tx=0).stop 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 shutdown_connections 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f919806c470 0x7f919806e930 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80ffbc0 0x7f91a8069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 --2- 172.21.15.42:0/6222767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91a8100dc0 0x7f91a819b5c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.479+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 >> 172.21.15.42:0/6222767 conn(0x7f91a80fb160 msgr2=0x7f91a81040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.480+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 shutdown_connections 2024-09-16T23:42:31.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:31.480+0000 7f91aeff3700 1 -- 172.21.15.42:0/6222767 wait complete. 2024-09-16T23:42:32.058 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T23:42:32.060 INFO:tasks.cephadm:Deploying osd.1 on smithi042 with /dev/vg_nvme/lv_3... 2024-09-16T23:42:32.060 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T23:42:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:32 smithi042 ceph-mon[27914]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:32 smithi042 ceph-mon[27914]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T23:42:32.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:32 smithi043 ceph-mon[35191]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:32.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:32.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:32.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:32 smithi043 ceph-mon[35191]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T23:42:33.102 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:42:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:33 smithi042 ceph-mon[27914]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T23:42:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:33 smithi042 ceph-mon[27914]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T23:42:33.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:33 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:33.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:33 smithi042 ceph-mon[27914]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:42:33.523 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:42:33 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[46711]: 2024-09-16T23:42:33.433+0000 7f220727a700 -1 osd.0 0 waiting for initial osdmap 2024-09-16T23:42:33.523 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:42:33 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[46711]: 2024-09-16T23:42:33.436+0000 7f220186c700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:42:33.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:33 smithi043 ceph-mon[35191]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T23:42:33.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:33 smithi043 ceph-mon[35191]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T23:42:33.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:33 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:33.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:33 smithi043 ceph-mon[35191]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:42:34.760 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:34 smithi042 ceph-mon[27914]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:34.760 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:34 smithi042 ceph-mon[27914]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-09-16T23:42:34.760 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:34 smithi042 ceph-mon[27914]: osdmap e8: 1 total, 0 up, 1 in 2024-09-16T23:42:34.760 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:34.760 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:34.802 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:34 smithi043 ceph-mon[35191]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:34.803 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:34 smithi043 ceph-mon[35191]: from='osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-09-16T23:42:34.803 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:34 smithi043 ceph-mon[35191]: osdmap e8: 1 total, 0 up, 1 in 2024-09-16T23:42:34.803 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:34.803 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:35.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:35 smithi042 ceph-mon[27914]: purged_snaps scrub starts 2024-09-16T23:42:35.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:35 smithi042 ceph-mon[27914]: purged_snaps scrub ok 2024-09-16T23:42:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:35 smithi042 ceph-mon[27914]: osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] boot 2024-09-16T23:42:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:35 smithi042 ceph-mon[27914]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T23:42:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:35 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:35.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:35 smithi043 ceph-mon[35191]: purged_snaps scrub starts 2024-09-16T23:42:35.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:35 smithi043 ceph-mon[35191]: purged_snaps scrub ok 2024-09-16T23:42:35.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:35 smithi043 ceph-mon[35191]: osd.0 [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] boot 2024-09-16T23:42:35.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:35 smithi043 ceph-mon[35191]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T23:42:35.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:35 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:42:36.753 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:36 smithi042 ceph-mon[27914]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:36.753 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:36 smithi042 ceph-mon[27914]: osdmap e10: 1 total, 1 up, 1 in 2024-09-16T23:42:36.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:36 smithi043 ceph-mon[35191]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T23:42:36.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:36 smithi043 ceph-mon[35191]: osdmap e10: 1 total, 1 up, 1 in 2024-09-16T23:42:37.048 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:42:37.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2024-09-16T23:42:37.282 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:42:38.396 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:38 smithi042 ceph-mon[27914]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:38.396 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:38 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:38.396 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:38 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:38.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:38 smithi043 ceph-mon[35191]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:38.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:38 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:38 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:38.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 -- 172.21.15.42:0/2566481772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4101740 msgr2=0x7f4ac4103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:38.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/2566481772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4101740 0x7f4ac4103b30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009910 tx=0x7f4ac0009c20 comp rx=0 tx=0).stop 2024-09-16T23:42:38.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 -- 172.21.15.42:0/2566481772 shutdown_connections 2024-09-16T23:42:38.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/2566481772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac4104070 0x7f4ac4106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:38.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/2566481772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4101740 0x7f4ac4103b30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:38.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 -- 172.21.15.42:0/2566481772 >> 172.21.15.42:0/2566481772 conn(0x7f4ac40fb150 msgr2=0x7f4ac40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:42:38.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 -- 172.21.15.42:0/2566481772 shutdown_connections 2024-09-16T23:42:38.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.910+0000 7f4acbdbd700 1 -- 172.21.15.42:0/2566481772 wait complete. 2024-09-16T23:42:38.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4acbdbd700 1 Processor -- start 2024-09-16T23:42:38.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4acbdbd700 1 -- start start 2024-09-16T23:42:38.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4acbdbd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac4101740 0x7f4ac41010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4acbdbd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4104070 0x7f4ac40ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4acbdbd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40ffc30 con 0x7f4ac4104070 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4acbdbd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac40ffd70 con 0x7f4ac4101740 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4ac9358700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4104070 0x7f4ac40ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4ac9358700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4104070 0x7f4ac40ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47014/0 (socket says 172.21.15.42:47014) 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4ac9b59700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac4101740 0x7f4ac41010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.911+0000 7f4ac9358700 1 -- 172.21.15.42:0/4188672347 learned_addr learned my addr 172.21.15.42:0/4188672347 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4ac9b59700 1 -- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4104070 msgr2=0x7f4ac40ff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4ac9b59700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4104070 0x7f4ac40ff6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4ac9b59700 1 -- 172.21.15.42:0/4188672347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac00095f0 con 0x7f4ac4101740 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4ac9358700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4104070 0x7f4ac40ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:42:38.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4ac9b59700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac4101740 0x7f4ac41010a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009910 tx=0x7f4ac000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:42:38.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4abaffd700 1 -- 172.21.15.42:0/4188672347 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac001d070 con 0x7f4ac4101740 2024-09-16T23:42:38.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac40ffff0 con 0x7f4ac4101740 2024-09-16T23:42:38.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4abaffd700 1 -- 172.21.15.42:0/4188672347 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ac000f460 con 0x7f4ac4101740 2024-09-16T23:42:38.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4abaffd700 1 -- 172.21.15.42:0/4188672347 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac0021620 con 0x7f4ac4101740 2024-09-16T23:42:38.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.912+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac41004e0 con 0x7f4ac4101740 2024-09-16T23:42:38.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.913+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac4190620 con 0x7f4ac4101740 2024-09-16T23:42:38.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.915+0000 7f4abaffd700 1 -- 172.21.15.42:0/4188672347 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f4ac000fa60 con 0x7f4ac4101740 2024-09-16T23:42:38.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.915+0000 7f4abaffd700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4ab006c470 0x7f4ab006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:42:38.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.915+0000 7f4abaffd700 1 -- 172.21.15.42:0/4188672347 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1810+0+0 (secure 0 0 0) 0x7f4ac008c1c0 con 0x7f4ac4101740 2024-09-16T23:42:38.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.916+0000 7f4ac9358700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4ab006c470 0x7f4ab006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:42:38.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.916+0000 7f4ac9358700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4ab006c470 0x7f4ab006e930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400f4d0 tx=0x7f4ab4005f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:42:38.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:38.920+0000 7f4abaffd700 1 -- 172.21.15.42:0/4188672347 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4ac005b330 con 0x7f4ac4101740 2024-09-16T23:42:39.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:39.136+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ac40611d0 con 0x7f4ab006c470 2024-09-16T23:42:39.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:39.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:42:39.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:42:39.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:39.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:39.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:42:39.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:42:39.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:40.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:40 smithi042 ceph-mon[27914]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:40.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:40 smithi042 ceph-mon[27914]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:42:40.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:40 smithi043 ceph-mon[35191]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:40.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:40 smithi043 ceph-mon[35191]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:42:42.434 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:42 smithi042 ceph-mon[27914]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:42.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:42 smithi043 ceph-mon[35191]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: Detected new or changed devices on smithi042 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi042 to 11844M 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.566 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:43 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:42:43.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: Detected new or changed devices on smithi042 2024-09-16T23:42:43.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi042 to 11844M 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:43 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:42:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:44 smithi042 ceph-mon[27914]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:44 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/164449174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "644cf390-4e6f-402f-92ff-dcd7330ee305"}]: dispatch 2024-09-16T23:42:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:44 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/164449174' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "644cf390-4e6f-402f-92ff-dcd7330ee305"}]': finished 2024-09-16T23:42:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:44 smithi042 ceph-mon[27914]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T23:42:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:44 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:42:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:44 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3799219111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:42:44.815 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:44 smithi043 ceph-mon[35191]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:44.816 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:44 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/164449174' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "644cf390-4e6f-402f-92ff-dcd7330ee305"}]: dispatch 2024-09-16T23:42:44.816 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:44 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/164449174' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "644cf390-4e6f-402f-92ff-dcd7330ee305"}]': finished 2024-09-16T23:42:44.816 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:44 smithi043 ceph-mon[35191]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T23:42:44.816 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:44 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:42:44.816 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:44 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3799219111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:42:46.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:45 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:46.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:45 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:46.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:45 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:46.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:45 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:47.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:46 smithi042 ceph-mon[27914]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:47.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:46 smithi043 ceph-mon[35191]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:48.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:48.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:48.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:48.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:42:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:48.161 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:48.161 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:48.161 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:48.161 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:42:48.161 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:48.971 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:48 smithi042 ceph-mon[27914]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:49.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:48 smithi043 ceph-mon[35191]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:50 smithi042 ceph-mon[27914]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:50 smithi043 ceph-mon[35191]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:52.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T23:42:52.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:51 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:52.247 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T23:42:52.247 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:51 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:42:53.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:52 smithi042 ceph-mon[27914]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:53.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:52 smithi042 ceph-mon[27914]: Deploying daemon osd.1 on smithi042 2024-09-16T23:42:53.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:52 smithi043 ceph-mon[35191]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:53.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:52 smithi043 ceph-mon[35191]: Deploying daemon osd.1 on smithi042 2024-09-16T23:42:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:54 smithi043 ceph-mon[35191]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:55.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:54 smithi042 ceph-mon[27914]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:55.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:42:56.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:56 smithi042 ceph-mon[27914]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:56.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:56.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:42:56.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:57.237 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:56 smithi043 ceph-mon[35191]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:57.238 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:57.238 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:42:57.238 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:58 smithi042 ceph-mon[27914]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:58 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:42:58 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:59.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:58 smithi043 ceph-mon[35191]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:42:59.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:58 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:59.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:42:58 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:42:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.893+0000 7f4abaffd700 1 -- 172.21.15.42:0/4188672347 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4ac40611d0 con 0x7f4ab006c470 2024-09-16T23:42:59.890 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 1 on host 'smithi042' 2024-09-16T23:42:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4ab006c470 msgr2=0x7f4ab006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4ab006c470 0x7f4ab006e930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400f4d0 tx=0x7f4ab4005f90 comp rx=0 tx=0).stop 2024-09-16T23:42:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac4101740 msgr2=0x7f4ac41010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:42:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac4101740 0x7f4ac41010a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009910 tx=0x7f4ac000bd70 comp rx=0 tx=0).stop 2024-09-16T23:42:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 shutdown_connections 2024-09-16T23:42:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4ab006c470 0x7f4ab006e930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac4101740 0x7f4ac41010a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 --2- 172.21.15.42:0/4188672347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ac4104070 0x7f4ac40ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:42:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 >> 172.21.15.42:0/4188672347 conn(0x7f4ac40fb150 msgr2=0x7f4ac40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:42:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 shutdown_connections 2024-09-16T23:42:59.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:42:59.895+0000 7f4acbdbd700 1 -- 172.21.15.42:0/4188672347 wait complete. 2024-09-16T23:43:00.549 DEBUG:teuthology.orchestra.run.smithi042:osd.1> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T23:43:00.552 INFO:tasks.cephadm:Deploying osd.2 on smithi042 with /dev/vg_nvme/lv_2... 2024-09-16T23:43:00.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T23:43:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:00 smithi043 ceph-mon[35191]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:43:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:00 smithi043 ceph-mon[35191]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T23:43:01.262 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:43:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:00 smithi042 ceph-mon[27914]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:43:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:00 smithi042 ceph-mon[27914]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T23:43:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:01 smithi042 ceph-mon[27914]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T23:43:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:01 smithi042 ceph-mon[27914]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T23:43:02.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:01 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:02.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:01 smithi042 ceph-mon[27914]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:43:02.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:01 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[54391]: 2024-09-16T23:43:01.902+0000 7fc70238d700 -1 osd.1 0 waiting for initial osdmap 2024-09-16T23:43:02.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:01 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[54391]: 2024-09-16T23:43:01.906+0000 7fc6fe983700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:43:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:01 smithi043 ceph-mon[35191]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T23:43:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:01 smithi043 ceph-mon[35191]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T23:43:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:01 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:01 smithi043 ceph-mon[35191]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:43:02.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:02 smithi042 ceph-mon[27914]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:43:02.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:02 smithi042 ceph-mon[27914]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-09-16T23:43:02.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:02 smithi042 ceph-mon[27914]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T23:43:02.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:02 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:02.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:02 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:03.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:02 smithi043 ceph-mon[35191]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T23:43:03.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:02 smithi043 ceph-mon[35191]: from='osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-09-16T23:43:03.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:02 smithi043 ceph-mon[35191]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T23:43:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:02 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:02 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:03 smithi042 ceph-mon[27914]: purged_snaps scrub starts 2024-09-16T23:43:04.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:03 smithi042 ceph-mon[27914]: purged_snaps scrub ok 2024-09-16T23:43:04.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:03 smithi042 ceph-mon[27914]: osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] boot 2024-09-16T23:43:04.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:03 smithi042 ceph-mon[27914]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T23:43:04.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:03 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:04.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:03 smithi043 ceph-mon[35191]: purged_snaps scrub starts 2024-09-16T23:43:04.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:03 smithi043 ceph-mon[35191]: purged_snaps scrub ok 2024-09-16T23:43:04.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:03 smithi043 ceph-mon[35191]: osd.1 [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] boot 2024-09-16T23:43:04.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:03 smithi043 ceph-mon[35191]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T23:43:04.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:03 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:43:04.738 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:43:04.757 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2024-09-16T23:43:05.021 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:04 smithi042 ceph-mon[27914]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:05.021 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:04 smithi042 ceph-mon[27914]: osdmap e15: 2 total, 2 up, 2 in 2024-09-16T23:43:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:04 smithi043 ceph-mon[35191]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:04 smithi043 ceph-mon[35191]: osdmap e15: 2 total, 2 up, 2 in 2024-09-16T23:43:05.461 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:43:06.847 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:06 smithi042 ceph-mon[27914]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:06.847 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:06 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:06.847 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:06 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:07.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:06 smithi043 ceph-mon[35191]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:07.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:06 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:07.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:06 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 -- 172.21.15.42:0/215130918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe4100e10 msgr2=0x7effe4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 --2- 172.21.15.42:0/215130918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe4100e10 0x7effe4101270 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7effd8009910 tx=0x7effd8009c20 comp rx=0 tx=0).stop 2024-09-16T23:43:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 -- 172.21.15.42:0/215130918 shutdown_connections 2024-09-16T23:43:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 --2- 172.21.15.42:0/215130918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe4100e10 0x7effe4101270 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 --2- 172.21.15.42:0/215130918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe40ffc10 0x7effe4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:08.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 -- 172.21.15.42:0/215130918 >> 172.21.15.42:0/215130918 conn(0x7effe40fb190 msgr2=0x7effe40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:43:08.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 -- 172.21.15.42:0/215130918 shutdown_connections 2024-09-16T23:43:08.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.010+0000 7effe89e4700 1 -- 172.21.15.42:0/215130918 wait complete. 2024-09-16T23:43:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.011+0000 7effe89e4700 1 Processor -- start 2024-09-16T23:43:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.011+0000 7effe89e4700 1 -- start start 2024-09-16T23:43:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe89e4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe40ffc10 0x7effe4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:43:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe89e4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe4100e10 0x7effe4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:43:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe89e4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe4199290 con 0x7effe40ffc10 2024-09-16T23:43:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe89e4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe41993d0 con 0x7effe4100e10 2024-09-16T23:43:08.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe40ffc10 0x7effe4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe40ffc10 0x7effe4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36164/0 (socket says 172.21.15.42:36164) 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe259c700 1 -- 172.21.15.42:0/162932471 learned_addr learned my addr 172.21.15.42:0/162932471 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe1d9b700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe4100e10 0x7effe4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe259c700 1 -- 172.21.15.42:0/162932471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe4100e10 msgr2=0x7effe4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe259c700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe4100e10 0x7effe4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe259c700 1 -- 172.21.15.42:0/162932471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80095f0 con 0x7effe40ffc10 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.012+0000 7effe259c700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe40ffc10 0x7effe4198730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effd400b480 tx=0x7effd400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.013+0000 7effd37fe700 1 -- 172.21.15.42:0/162932471 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effd4011800 con 0x7effe40ffc10 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.013+0000 7effd37fe700 1 -- 172.21.15.42:0/162932471 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7effd4011e10 con 0x7effe40ffc10 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.013+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe419de80 con 0x7effe40ffc10 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.013+0000 7effd37fe700 1 -- 172.21.15.42:0/162932471 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effd4010ea0 con 0x7effe40ffc10 2024-09-16T23:43:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.013+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe419e340 con 0x7effe40ffc10 2024-09-16T23:43:08.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.014+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe41087a0 con 0x7effe40ffc10 2024-09-16T23:43:08.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.014+0000 7effd37fe700 1 -- 172.21.15.42:0/162932471 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7effd4010380 con 0x7effe40ffc10 2024-09-16T23:43:08.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.015+0000 7effd37fe700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7effcc06c460 0x7effcc06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:43:08.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.015+0000 7effd37fe700 1 -- 172.21.15.42:0/162932471 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2242+0+0 (secure 0 0 0) 0x7effd408a5c0 con 0x7effe40ffc10 2024-09-16T23:43:08.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.015+0000 7effe1d9b700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7effcc06c460 0x7effcc06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:43:08.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.016+0000 7effe1d9b700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7effcc06c460 0x7effcc06e920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7effd8006010 tx=0x7effd800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:43:08.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.020+0000 7effd37fe700 1 -- 172.21.15.42:0/162932471 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7effd4059650 con 0x7effe40ffc10 2024-09-16T23:43:08.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:08.233+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7effe419ea40 con 0x7effcc06c460 2024-09-16T23:43:08.968 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:08 smithi042 ceph-mon[27914]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:08.968 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:08 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:43:08.969 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:08 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:43:08.969 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:08 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:09.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:08 smithi043 ceph-mon[35191]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:09.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:08 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:43:09.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:08 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:43:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:08 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:09 smithi042 ceph-mon[27914]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:43:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:10.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:09 smithi043 ceph-mon[35191]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:43:10.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:09 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:11.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:11.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:11.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:11.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:11.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:11.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:11.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:11.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:11 smithi043 ceph-mon[35191]: Detected new or changed devices on smithi042 2024-09-16T23:43:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:11 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi042 to 5922M 2024-09-16T23:43:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:11 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3794254498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27ed11a-999c-439e-b098-3381127c72e6"}]: dispatch 2024-09-16T23:43:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:11 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3794254498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27ed11a-999c-439e-b098-3381127c72e6"}]': finished 2024-09-16T23:43:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:11 smithi043 ceph-mon[35191]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T23:43:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:11 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:12.136 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:11 smithi042 ceph-mon[27914]: Detected new or changed devices on smithi042 2024-09-16T23:43:12.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:11 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi042 to 5922M 2024-09-16T23:43:12.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:11 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3794254498' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27ed11a-999c-439e-b098-3381127c72e6"}]: dispatch 2024-09-16T23:43:12.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:11 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3794254498' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27ed11a-999c-439e-b098-3381127c72e6"}]': finished 2024-09-16T23:43:12.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:11 smithi042 ceph-mon[27914]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T23:43:12.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:11 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:13.040 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:12 smithi042 ceph-mon[27914]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:13.040 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:12 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1754189702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:43:13.040 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:13.040 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:13.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:12 smithi043 ceph-mon[35191]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:12 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1754189702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:43:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:15.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:14 smithi043 ceph-mon[35191]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:15.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:14 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:15.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:14 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:15.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:14 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:15.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:14 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:15.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:14 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:15.238 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:14 smithi042 ceph-mon[27914]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:15.238 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:14 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:15.238 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:14 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:15.238 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:14 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:15.238 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:14 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:15.238 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:14 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:16.911 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:16 smithi042 ceph-mon[27914]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:17.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:16 smithi043 ceph-mon[35191]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:18 smithi043 ceph-mon[35191]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:18 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T23:43:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:18 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:19.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:18 smithi042 ceph-mon[27914]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:19.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:18 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T23:43:19.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:18 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:20.057 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:19 smithi042 ceph-mon[27914]: Deploying daemon osd.2 on smithi042 2024-09-16T23:43:20.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:19 smithi043 ceph-mon[35191]: Deploying daemon osd.2 on smithi042 2024-09-16T23:43:21.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:20 smithi042 ceph-mon[27914]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:21.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:20 smithi043 ceph-mon[35191]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:22 smithi042 ceph-mon[27914]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:23.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:22 smithi043 ceph-mon[35191]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:25.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:24 smithi043 ceph-mon[35191]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:25.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:24 smithi042 ceph-mon[27914]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:25.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:25.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:25.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:25.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:27.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:26 smithi043 ceph-mon[35191]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:27.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:26 smithi042 ceph-mon[27914]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:27.734 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 2 on host 'smithi042' 2024-09-16T23:43:27.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.737+0000 7effd37fe700 1 -- 172.21.15.42:0/162932471 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7effe419ea40 con 0x7effcc06c460 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7effcc06c460 msgr2=0x7effcc06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7effcc06c460 0x7effcc06e920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7effd8006010 tx=0x7effd800b540 comp rx=0 tx=0).stop 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe40ffc10 msgr2=0x7effe4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe40ffc10 0x7effe4198730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7effd400b480 tx=0x7effd400b790 comp rx=0 tx=0).stop 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 shutdown_connections 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7effcc06c460 0x7effcc06e920 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effe40ffc10 0x7effe4198730 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:27.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 --2- 172.21.15.42:0/162932471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7effe4100e10 0x7effe4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 >> 172.21.15.42:0/162932471 conn(0x7effe40fb190 msgr2=0x7effe4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:43:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 shutdown_connections 2024-09-16T23:43:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:43:27.739+0000 7effe89e4700 1 -- 172.21.15.42:0/162932471 wait complete. 2024-09-16T23:43:28.666 DEBUG:teuthology.orchestra.run.smithi042:osd.2> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T23:43:28.668 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_4... 2024-09-16T23:43:28.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T23:43:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:43:28.921 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:28 smithi042 ceph-mon[27914]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:28.921 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:28.921 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:28 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:28.921 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:28 smithi042 ceph-mon[27914]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T23:43:29.004 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:28 smithi043 ceph-mon[35191]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:29.005 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:29.005 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:28 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:29.005 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:28 smithi043 ceph-mon[35191]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T23:43:29.699 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:29 smithi042 ceph-mon[27914]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T23:43:29.699 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:29 smithi042 ceph-mon[27914]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T23:43:29.700 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:29.700 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:29 smithi042 ceph-mon[27914]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:43:29.700 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:29 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[61727]: 2024-09-16T23:43:29.697+0000 7f5081fc9700 -1 osd.2 0 waiting for initial osdmap 2024-09-16T23:43:29.700 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:29 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[61727]: 2024-09-16T23:43:29.701+0000 7f507c5bb700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:43:30.035 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:29 smithi043 ceph-mon[35191]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T23:43:30.035 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:29 smithi043 ceph-mon[35191]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T23:43:30.036 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:30.036 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:29 smithi043 ceph-mon[35191]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:43:30.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:30 smithi043 ceph-mon[35191]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:30.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:30 smithi043 ceph-mon[35191]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-09-16T23:43:30.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:30 smithi043 ceph-mon[35191]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T23:43:30.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:30 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:30.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:30 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:31.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:30 smithi042 ceph-mon[27914]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:31.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:30 smithi042 ceph-mon[27914]: from='osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-09-16T23:43:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:30 smithi042 ceph-mon[27914]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T23:43:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:30 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:30 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:31.489 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:43:31.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-09-16T23:43:31.719 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:43:31.793 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: purged_snaps scrub starts 2024-09-16T23:43:31.793 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: purged_snaps scrub ok 2024-09-16T23:43:31.793 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] boot 2024-09-16T23:43:31.793 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T23:43:31.793 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:31.794 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:31.794 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:31.794 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T23:43:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: purged_snaps scrub starts 2024-09-16T23:43:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: purged_snaps scrub ok 2024-09-16T23:43:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: osd.2 [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] boot 2024-09-16T23:43:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T23:43:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:43:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T23:43:32.875 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:32 smithi043 ceph-mon[35191]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:32.876 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T23:43:32.876 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:32 smithi043 ceph-mon[35191]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T23:43:32.876 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T23:43:32.914 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:32 smithi042 ceph-mon[27914]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T23:43:32.914 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T23:43:32.914 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:32 smithi042 ceph-mon[27914]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T23:43:32.915 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T23:43:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 -- 172.21.15.43:0/2587529707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770104130 msgr2=0x7f1770106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 --2- 172.21.15.43:0/2587529707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770104130 0x7f1770106520 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1764009910 tx=0x7f1764009c20 comp rx=0 tx=0).stop 2024-09-16T23:43:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 -- 172.21.15.43:0/2587529707 shutdown_connections 2024-09-16T23:43:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 --2- 172.21.15.43:0/2587529707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770104130 0x7f1770106520 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 --2- 172.21.15.43:0/2587529707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1770101800 0x7f1770103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 -- 172.21.15.43:0/2587529707 >> 172.21.15.43:0/2587529707 conn(0x7f17700fb190 msgr2=0x7f17700fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:43:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 -- 172.21.15.43:0/2587529707 shutdown_connections 2024-09-16T23:43:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.381+0000 7f1774edc700 1 -- 172.21.15.43:0/2587529707 wait complete. 2024-09-16T23:43:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f1774edc700 1 Processor -- start 2024-09-16T23:43:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f1774edc700 1 -- start start 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f1774edc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770101800 0x7f1770196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f1774edc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1770104130 0x7f17701969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f1774edc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1770196fc0 con 0x7f1770104130 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f1774edc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1770197100 con 0x7f1770101800 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f176e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770101800 0x7f1770196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f176e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770101800 0x7f1770196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49572/0 (socket says 172.21.15.43:49572) 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f176e59c700 1 -- 172.21.15.43:0/3708981379 learned_addr learned my addr 172.21.15.43:0/3708981379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.382+0000 7f176dd9b700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1770104130 0x7f17701969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f176e59c700 1 -- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1770104130 msgr2=0x7f17701969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f176e59c700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1770104130 0x7f17701969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f176e59c700 1 -- 172.21.15.43:0/3708981379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17640095f0 con 0x7f1770101800 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f176dd9b700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1770104130 0x7f17701969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:43:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f176e59c700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770101800 0x7f1770196460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f176000d8d0 tx=0x7f176000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:43:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1760009940 con 0x7f1770101800 2024-09-16T23:43:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1760010460 con 0x7f1770101800 2024-09-16T23:43:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177019bbb0 con 0x7f1770101800 2024-09-16T23:43:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f176000f5f0 con 0x7f1770101800 2024-09-16T23:43:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.383+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177019c100 con 0x7f1770101800 2024-09-16T23:43:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.384+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1770190610 con 0x7f1770101800 2024-09-16T23:43:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.384+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 18) v1 ==== 90256+0+0 (secure 0 0 0) 0x7f1760009aa0 con 0x7f1770101800 2024-09-16T23:43:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.385+0000 7f175f7fe700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f175806c460 0x7f175806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:43:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.385+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3060+0+0 (secure 0 0 0) 0x7f176008acb0 con 0x7f1770101800 2024-09-16T23:43:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.385+0000 7f176dd9b700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f175806c460 0x7f175806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:43:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.385+0000 7f176dd9b700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f175806c460 0x7f175806e920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1764005b40 tx=0x7f1764005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:43:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.390+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f17600599e0 con 0x7f1770101800 2024-09-16T23:43:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.594+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f17700611d0 con 0x7f175806c460 2024-09-16T23:43:33.707 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[64179]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T23:43:33.708 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[64179]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:33.708 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[64179]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:33.708 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[64179]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:33.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:33.717+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f1760056320 con 0x7f1770101800 2024-09-16T23:43:33.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:33 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T23:43:33.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:33 smithi043 ceph-mon[35191]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T23:43:33.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:33 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:43:33.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:33 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:43:33.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:33 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:34.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:33 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T23:43:34.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:33 smithi042 ceph-mon[27914]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T23:43:34.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:33 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:43:34.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:33 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:43:34.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:33 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:34.024 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[66126]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T23:43:34.024 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[66126]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:34.024 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[66126]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:34.024 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:43:33 smithi042 sudo[66126]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:34.487 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66132]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T23:43:34.487 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66132]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:34.487 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66132]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:34.488 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66132]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:34.772 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66235]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T23:43:34.772 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66235]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:34.773 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66235]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:34.773 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:43:34 smithi042 sudo[66235]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: pgmap v81: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: mgrmap e19: smithi042.zrkumz(active, since 2m), standbys: smithi043.xveyou 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:34.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:34.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:34.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:34.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:34.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:34.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:35.010 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: pgmap v81: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:35.010 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='client.24141 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:43:35.010 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: mgrmap e19: smithi042.zrkumz(active, since 2m), standbys: smithi043.xveyou 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:35.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:35.522 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66329]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T23:43:35.522 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:35.522 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66329]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:35.522 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66329]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:36.010 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66415]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T23:43:36.011 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66415]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:36.011 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66415]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:36.011 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:43:35 smithi042 sudo[66415]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:36.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66464]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T23:43:36.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:36.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66464]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:36.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66464]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:36.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:36 smithi043 ceph-mon[35191]: Detected new or changed devices on smithi042 2024-09-16T23:43:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:36 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi042 to 3948M 2024-09-16T23:43:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:36 smithi043 ceph-mon[35191]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T23:43:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66511]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T23:43:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66511]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 sudo[66511]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 ceph-mon[27914]: Detected new or changed devices on smithi042 2024-09-16T23:43:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi042 to 3948M 2024-09-16T23:43:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:36 smithi042 ceph-mon[27914]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T23:43:37.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39012]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T23:43:37.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39012]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:37.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39012]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:37.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39012]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:37.759 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39015]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T23:43:37.759 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39015]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39015]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 sudo[39015]: pam_unix(sudo:session): session closed for user root 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/1599192075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b84bdfd-14ef-4867-a5d0-b443a5dcec12"}]: dispatch 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b84bdfd-14ef-4867-a5d0-b443a5dcec12"}]: dispatch 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b84bdfd-14ef-4867-a5d0-b443a5dcec12"}]': finished 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T23:43:37.760 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:43:37.761 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:43:37.761 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T23:43:37.761 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:37.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: pgmap v83: 1 pgs: 1 active+clean; 449 KiB data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:37.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1599192075' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b84bdfd-14ef-4867-a5d0-b443a5dcec12"}]: dispatch 2024-09-16T23:43:37.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T23:43:37.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:43:37.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b84bdfd-14ef-4867-a5d0-b443a5dcec12"}]: dispatch 2024-09-16T23:43:37.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b84bdfd-14ef-4867-a5d0-b443a5dcec12"}]': finished 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T23:43:37.999 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:38.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:38 smithi043 ceph-mon[35191]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:38.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:38 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/2266725759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:43:38.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:38 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:38.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:38 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:38 smithi042 ceph-mon[27914]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:38 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/2266725759' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:43:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:38 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:38 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:39.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:39.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:39.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:39.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:43:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:41.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:41 smithi043 ceph-mon[35191]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-09-16T23:43:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:41 smithi042 ceph-mon[27914]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-09-16T23:43:43.179 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:43 smithi043 ceph-mon[35191]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-16T23:43:43.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:43 smithi042 ceph-mon[27914]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-09-16T23:43:45.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:45 smithi043 ceph-mon[35191]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 139 KiB/s wr, 9 op/s 2024-09-16T23:43:45.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:45 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T23:43:45.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:45 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:45.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:45 smithi042 ceph-mon[27914]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 139 KiB/s wr, 9 op/s 2024-09-16T23:43:45.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:45 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T23:43:45.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:45 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:43:46.355 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:46 smithi043 ceph-mon[35191]: Deploying daemon osd.3 on smithi043 2024-09-16T23:43:46.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:46 smithi042 ceph-mon[27914]: Deploying daemon osd.3 on smithi043 2024-09-16T23:43:47.298 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:47 smithi043 ceph-mon[35191]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-16T23:43:47.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:47 smithi042 ceph-mon[27914]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-16T23:43:49.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:49 smithi042 ceph-mon[27914]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 124 KiB/s wr, 8 op/s 2024-09-16T23:43:49.368 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:49 smithi043 ceph-mon[35191]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.6 KiB/s rd, 124 KiB/s wr, 8 op/s 2024-09-16T23:43:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:50 smithi042 ceph-mon[27914]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-16T23:43:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:50 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:50 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:50 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:50 smithi043 ceph-mon[35191]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-16T23:43:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:50 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:50 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:50 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:43:52.827 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:52 smithi043 ceph-mon[35191]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:53.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:52 smithi042 ceph-mon[27914]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:53.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.215+0000 7f175f7fe700 1 -- 172.21.15.43:0/3708981379 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f17700611d0 con 0x7f175806c460 2024-09-16T23:43:53.213 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-09-16T23:43:53.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.218+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f175806c460 msgr2=0x7f175806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.218+0000 7f1774edc700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f175806c460 0x7f175806e920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1764005b40 tx=0x7f1764005a90 comp rx=0 tx=0).stop 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770101800 msgr2=0x7f1770196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770101800 0x7f1770196460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f176000d8d0 tx=0x7f176000dc90 comp rx=0 tx=0).stop 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 shutdown_connections 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f175806c460 0x7f175806e920 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1770101800 0x7f1770196460 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 --2- 172.21.15.43:0/3708981379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1770104130 0x7f17701969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 >> 172.21.15.43:0/3708981379 conn(0x7f17700fb190 msgr2=0x7f17700fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 shutdown_connections 2024-09-16T23:43:53.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:43:53.219+0000 7f1774edc700 1 -- 172.21.15.43:0/3708981379 wait complete. 2024-09-16T23:43:53.845 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T23:43:53.848 INFO:tasks.cephadm:Deploying osd.4 on smithi043 with /dev/vg_nvme/lv_3... 2024-09-16T23:43:53.848 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T23:43:54.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:54 smithi043 ceph-mon[35191]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:54.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:54.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:54.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:54 smithi043 ceph-mon[35191]: from='osd.3 [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:43:54.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:54 smithi043 ceph-mon[35191]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:43:54.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:54.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:54 smithi042 ceph-mon[27914]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:54.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:54.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:54.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:54 smithi042 ceph-mon[27914]: from='osd.3 [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:43:54.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:54 smithi042 ceph-mon[27914]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:43:54.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:43:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:43:55.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-mon[35191]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T23:43:55.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-mon[35191]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T23:43:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-mon[35191]: from='osd.3 [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:43:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-mon[35191]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:43:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:55.378 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[41071]: 2024-09-16T23:43:55.196+0000 7f0d5e2f0700 -1 osd.3 0 waiting for initial osdmap 2024-09-16T23:43:55.378 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:43:55 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[41071]: 2024-09-16T23:43:55.202+0000 7f0d580e1700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:43:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:55 smithi042 ceph-mon[27914]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T23:43:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:55 smithi042 ceph-mon[27914]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T23:43:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:55 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:55 smithi042 ceph-mon[27914]: from='osd.3 [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:43:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:55 smithi042 ceph-mon[27914]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:43:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:55 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:55 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:43:56.364 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:56 smithi043 ceph-mon[35191]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:56.364 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:56 smithi043 ceph-mon[35191]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-16T23:43:56.364 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:56 smithi043 ceph-mon[35191]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T23:43:56.364 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:56.364 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:56 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:56.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:56 smithi042 ceph-mon[27914]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:56.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:56 smithi042 ceph-mon[27914]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-16T23:43:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:56 smithi042 ceph-mon[27914]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T23:43:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:56 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:57.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:57 smithi042 ceph-mon[27914]: purged_snaps scrub starts 2024-09-16T23:43:57.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:57 smithi042 ceph-mon[27914]: purged_snaps scrub ok 2024-09-16T23:43:57.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:57 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:57.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:57 smithi042 ceph-mon[27914]: osd.3 [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] boot 2024-09-16T23:43:57.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:57 smithi042 ceph-mon[27914]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T23:43:57.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:57 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:57.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:57 smithi042 ceph-mon[27914]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T23:43:57.560 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:57 smithi043 ceph-mon[35191]: purged_snaps scrub starts 2024-09-16T23:43:57.560 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:57 smithi043 ceph-mon[35191]: purged_snaps scrub ok 2024-09-16T23:43:57.560 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:57 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:57.560 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:57 smithi043 ceph-mon[35191]: osd.3 [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] boot 2024-09-16T23:43:57.561 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:57 smithi043 ceph-mon[35191]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T23:43:57.561 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:57 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:43:57.561 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:57 smithi043 ceph-mon[35191]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T23:43:57.564 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:43:57.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-09-16T23:43:58.031 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:43:58.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:58 smithi043 ceph-mon[35191]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:58.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:58 smithi043 ceph-mon[35191]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T23:43:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:58 smithi042 ceph-mon[27914]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-16T23:43:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:58 smithi042 ceph-mon[27914]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T23:43:59.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:43:59 smithi043 ceph-mon[35191]: osdmap e29: 4 total, 4 up, 4 in 2024-09-16T23:44:00.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:43:59 smithi042 ceph-mon[27914]: osdmap e29: 4 total, 4 up, 4 in 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 -- 172.21.15.43:0/3903348807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec0ffbc0 msgr2=0x7f67ec0fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 --2- 172.21.15.43:0/3903348807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec0ffbc0 0x7f67ec0fffe0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009910 tx=0x7f67dc009c20 comp rx=0 tx=0).stop 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 -- 172.21.15.43:0/3903348807 shutdown_connections 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 --2- 172.21.15.43:0/3903348807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67ec100dc0 0x7f67ec101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 --2- 172.21.15.43:0/3903348807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec0ffbc0 0x7f67ec0fffe0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 -- 172.21.15.43:0/3903348807 >> 172.21.15.43:0/3903348807 conn(0x7f67ec0fb160 msgr2=0x7f67ec0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 -- 172.21.15.43:0/3903348807 shutdown_connections 2024-09-16T23:44:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.707+0000 7f67f0d70700 1 -- 172.21.15.43:0/3903348807 wait complete. 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67f0d70700 1 Processor -- start 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67f0d70700 1 -- start start 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67f0d70700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67ec0ffbc0 0x7f67ec0699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67f0d70700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec100dc0 0x7f67ec19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67f0d70700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec19ba90 con 0x7f67ec0ffbc0 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67f0d70700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec19bbd0 con 0x7f67ec100dc0 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67e9d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec100dc0 0x7f67ec19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67ea59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67ec0ffbc0 0x7f67ec0699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:00.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67e9d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec100dc0 0x7f67ec19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56838/0 (socket says 172.21.15.43:56838) 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.708+0000 7f67e9d9b700 1 -- 172.21.15.43:0/2508839962 learned_addr learned my addr 172.21.15.43:0/2508839962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67e9d9b700 1 -- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67ec0ffbc0 msgr2=0x7f67ec0699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67e9d9b700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67ec0ffbc0 0x7f67ec0699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67e9d9b700 1 -- 172.21.15.43:0/2508839962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0095f0 con 0x7f67ec100dc0 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67e9d9b700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec100dc0 0x7f67ec19b500 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67e000c930 tx=0x7f67e000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67db7fe700 1 -- 172.21.15.43:0/2508839962 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e0007ad0 con 0x7f67ec100dc0 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67db7fe700 1 -- 172.21.15.43:0/2508839962 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f67e000ce80 con 0x7f67ec100dc0 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec19beb0 con 0x7f67ec100dc0 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67db7fe700 1 -- 172.21.15.43:0/2508839962 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e0018630 con 0x7f67ec100dc0 2024-09-16T23:44:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.709+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec19c3d0 con 0x7f67ec100dc0 2024-09-16T23:44:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.710+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec18f530 con 0x7f67ec100dc0 2024-09-16T23:44:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.711+0000 7f67db7fe700 1 -- 172.21.15.43:0/2508839962 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f67e0007c30 con 0x7f67ec100dc0 2024-09-16T23:44:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.711+0000 7f67db7fe700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67d406c4e0 0x7f67d406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.711+0000 7f67db7fe700 1 -- 172.21.15.43:0/2508839962 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3602+0+0 (secure 0 0 0) 0x7f67e008b030 con 0x7f67ec100dc0 2024-09-16T23:44:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.711+0000 7f67ea59c700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67d406c4e0 0x7f67d406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.712+0000 7f67ea59c700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67d406c4e0 0x7f67d406e9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006010 tx=0x7f67dc00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:00.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.716+0000 7f67db7fe700 1 -- 172.21.15.43:0/2508839962 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f67e00be0e0 con 0x7f67ec100dc0 2024-09-16T23:44:00.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-16T23:44:00.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: osdmap e30: 4 total, 4 up, 4 in 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:00 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:00.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:00.926+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f67ec0611d0 con 0x7f67d406c4e0 2024-09-16T23:44:01.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: osdmap e30: 4 total, 4 up, 4 in 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:00 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:01.708 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:01 smithi043 ceph-mon[35191]: Detected new or changed devices on smithi043 2024-09-16T23:44:01.708 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:01 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi043 to 14916M 2024-09-16T23:44:01.708 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:01 smithi043 ceph-mon[35191]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:44:01.709 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:01 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:44:01.709 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:01 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:44:01.709 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:01 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:01 smithi042 ceph-mon[27914]: Detected new or changed devices on smithi043 2024-09-16T23:44:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:01 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi043 to 14916M 2024-09-16T23:44:02.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:01 smithi042 ceph-mon[27914]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:44:02.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:01 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:44:02.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:01 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:44:02.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:01 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:02.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:02 smithi043 ceph-mon[35191]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-16T23:44:03.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:02 smithi042 ceph-mon[27914]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-16T23:44:03.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:03 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/448406413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "093b60b2-f2fd-4261-b038-4400c2a76180"}]: dispatch 2024-09-16T23:44:03.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:03 smithi043 ceph-mon[35191]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "093b60b2-f2fd-4261-b038-4400c2a76180"}]: dispatch 2024-09-16T23:44:03.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:03 smithi043 ceph-mon[35191]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "093b60b2-f2fd-4261-b038-4400c2a76180"}]': finished 2024-09-16T23:44:03.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:03 smithi043 ceph-mon[35191]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T23:44:03.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:03 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:03 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/448406413' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "093b60b2-f2fd-4261-b038-4400c2a76180"}]: dispatch 2024-09-16T23:44:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:03 smithi042 ceph-mon[27914]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "093b60b2-f2fd-4261-b038-4400c2a76180"}]: dispatch 2024-09-16T23:44:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:03 smithi042 ceph-mon[27914]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "093b60b2-f2fd-4261-b038-4400c2a76180"}]': finished 2024-09-16T23:44:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:03 smithi042 ceph-mon[27914]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T23:44:04.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:03 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:04.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:04 smithi043 ceph-mon[35191]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T23:44:04.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:04 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/1164578064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:44:05.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:04 smithi042 ceph-mon[27914]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T23:44:05.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:04 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1164578064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:44:06.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:06 smithi043 ceph-mon[35191]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:06 smithi042 ceph-mon[27914]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:08.682 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:08 smithi043 ceph-mon[35191]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:08 smithi042 ceph-mon[27914]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:09.863 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:09 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:10.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:10 smithi043 ceph-mon[35191]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:10.911 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:10 smithi042 ceph-mon[27914]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:11.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:11 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T23:44:11.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:11 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:11.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:11 smithi043 ceph-mon[35191]: Deploying daemon osd.4 on smithi043 2024-09-16T23:44:12.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:11 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T23:44:12.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:11 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:12.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:11 smithi042 ceph-mon[27914]: Deploying daemon osd.4 on smithi043 2024-09-16T23:44:12.833 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:12 smithi043 ceph-mon[35191]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:13.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:12 smithi042 ceph-mon[27914]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:14 smithi043 ceph-mon[35191]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:15.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:14 smithi042 ceph-mon[27914]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:16.848 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:16 smithi043 ceph-mon[35191]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:16.848 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:16 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:16.848 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:16 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:16.848 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:16 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:16 smithi042 ceph-mon[27914]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:18.764 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:18 smithi043 ceph-mon[35191]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:18 smithi042 ceph-mon[27914]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.202+0000 7f67db7fe700 1 -- 172.21.15.43:0/2508839962 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f67ec0611d0 con 0x7f67d406c4e0 2024-09-16T23:44:20.200 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 4 on host 'smithi043' 2024-09-16T23:44:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67d406c4e0 msgr2=0x7f67d406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67d406c4e0 0x7f67d406e9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006010 tx=0x7f67dc00b540 comp rx=0 tx=0).stop 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec100dc0 msgr2=0x7f67ec19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec100dc0 0x7f67ec19b500 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f67e000c930 tx=0x7f67e000cc40 comp rx=0 tx=0).stop 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 shutdown_connections 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67d406c4e0 0x7f67d406e9a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67ec0ffbc0 0x7f67ec0699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 --2- 172.21.15.43:0/2508839962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67ec100dc0 0x7f67ec19b500 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 >> 172.21.15.43:0/2508839962 conn(0x7f67ec0fb160 msgr2=0x7f67ec104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 shutdown_connections 2024-09-16T23:44:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:20.204+0000 7f67f0d70700 1 -- 172.21.15.43:0/2508839962 wait complete. 2024-09-16T23:44:20.809 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:20 smithi043 ceph-mon[35191]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:20.809 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:20 smithi043 ceph-mon[35191]: from='osd.4 [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:44:20.809 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:20 smithi043 ceph-mon[35191]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:44:20.809 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:20 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:20.809 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:20 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:20.831 DEBUG:teuthology.orchestra.run.smithi043:osd.4> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T23:44:20.834 INFO:tasks.cephadm:Deploying osd.5 on smithi043 with /dev/vg_nvme/lv_2... 2024-09-16T23:44:20.834 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T23:44:20.990 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:20 smithi042 ceph-mon[27914]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:20.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:20 smithi042 ceph-mon[27914]: from='osd.4 [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:44:20.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:20 smithi042 ceph-mon[27914]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:44:20.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:20.991 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:21.084 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:44:21.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-mon[35191]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T23:44:21.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-mon[35191]: osdmap e32: 5 total, 4 up, 5 in 2024-09-16T23:44:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-mon[35191]: from='osd.4 [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-mon[35191]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:21.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[47483]: 2024-09-16T23:44:21.653+0000 7fea0f3a2700 -1 osd.4 0 waiting for initial osdmap 2024-09-16T23:44:21.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:44:21 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[47483]: 2024-09-16T23:44:21.660+0000 7fea0618d700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:44:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:21 smithi042 ceph-mon[27914]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T23:44:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:21 smithi042 ceph-mon[27914]: osdmap e32: 5 total, 4 up, 5 in 2024-09-16T23:44:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:21 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:21 smithi042 ceph-mon[27914]: from='osd.4 [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:21 smithi042 ceph-mon[27914]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:21 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:21 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:22.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:22 smithi043 ceph-mon[35191]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:22.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:22 smithi043 ceph-mon[35191]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-16T23:44:22.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:22 smithi043 ceph-mon[35191]: osdmap e33: 5 total, 4 up, 5 in 2024-09-16T23:44:22.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:22.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:23.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:22 smithi042 ceph-mon[27914]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-16T23:44:23.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:22 smithi042 ceph-mon[27914]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-16T23:44:23.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:22 smithi042 ceph-mon[27914]: osdmap e33: 5 total, 4 up, 5 in 2024-09-16T23:44:23.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:23.941 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:23 smithi043 ceph-mon[35191]: purged_snaps scrub starts 2024-09-16T23:44:23.942 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:23 smithi043 ceph-mon[35191]: purged_snaps scrub ok 2024-09-16T23:44:23.942 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:23 smithi043 ceph-mon[35191]: osd.4 [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] boot 2024-09-16T23:44:23.942 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:23 smithi043 ceph-mon[35191]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T23:44:23.942 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:23 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:23 smithi042 ceph-mon[27914]: purged_snaps scrub starts 2024-09-16T23:44:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:23 smithi042 ceph-mon[27914]: purged_snaps scrub ok 2024-09-16T23:44:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:23 smithi042 ceph-mon[27914]: osd.4 [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] boot 2024-09-16T23:44:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:23 smithi042 ceph-mon[27914]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T23:44:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:23 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:44:24.945 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:24 smithi043 ceph-mon[35191]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:24.945 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:24 smithi043 ceph-mon[35191]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T23:44:24.946 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:24.949 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-09-16T23:44:24.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-09-16T23:44:25.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:24 smithi042 ceph-mon[27914]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:25.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:24 smithi042 ceph-mon[27914]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T23:44:25.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:44:26.874 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:26 smithi043 ceph-mon[35191]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:27.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:26 smithi042 ceph-mon[27914]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 -- 172.21.15.43:0/1198966721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 msgr2=0x7f4a5c0fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 --2- 172.21.15.43:0/1198966721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c0fffe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009910 tx=0x7f4a4c009c20 comp rx=0 tx=0).stop 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 -- 172.21.15.43:0/1198966721 shutdown_connections 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 --2- 172.21.15.43:0/1198966721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a5c100dc0 0x7f4a5c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 --2- 172.21.15.43:0/1198966721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c0fffe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 -- 172.21.15.43:0/1198966721 >> 172.21.15.43:0/1198966721 conn(0x7f4a5c0fb160 msgr2=0x7f4a5c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 -- 172.21.15.43:0/1198966721 shutdown_connections 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.869+0000 7f4a61314700 1 -- 172.21.15.43:0/1198966721 wait complete. 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a61314700 1 Processor -- start 2024-09-16T23:44:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a61314700 1 -- start start 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a61314700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c069710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a61314700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a5c100dc0 0x7f4a5c071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a61314700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c069d30 con 0x7f4a5c100dc0 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a61314700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c072230 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a5affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a5affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c069710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47456/0 (socket says 172.21.15.43:47456) 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.870+0000 7f4a5affd700 1 -- 172.21.15.43:0/1550224670 learned_addr learned my addr 172.21.15.43:0/1550224670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a5a7fc700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a5c100dc0 0x7f4a5c071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a5affd700 1 -- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a5c100dc0 msgr2=0x7f4a5c071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a5affd700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a5c100dc0 0x7f4a5c071cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a5affd700 1 -- 172.21.15.43:0/1550224670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0095f0 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a5affd700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c069710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00bda0 tx=0x7f4a4c00bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a53fff700 1 -- 172.21.15.43:0/1550224670 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4c01d070 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c0724b0 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a53fff700 1 -- 172.21.15.43:0/1550224670 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a4c022470 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a53fff700 1 -- 172.21.15.43:0/1550224670 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4c00f650 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.871+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c0729a0 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.873+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c06bf50 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.873+0000 7f4a53fff700 1 -- 172.21.15.43:0/1550224670 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f4a4c004af0 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.873+0000 7f4a53fff700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4a4806c4e0 0x7f4a4806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.873+0000 7f4a53fff700 1 -- 172.21.15.43:0/1550224670 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4034+0+0 (secure 0 0 0) 0x7f4a4c08c870 con 0x7f4a5c0ffbc0 2024-09-16T23:44:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.873+0000 7f4a5a7fc700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4a4806c4e0 0x7f4a4806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.874+0000 7f4a5a7fc700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4a4806c4e0 0x7f4a4806e9a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c069490 tx=0x7f4a4400a300 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:27.878+0000 7f4a53fff700 1 -- 172.21.15.43:0/1550224670 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a4c05b120 con 0x7f4a5c0ffbc0 2024-09-16T23:44:28.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:28.087+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a5c02cdb0 con 0x7f4a4806c4e0 2024-09-16T23:44:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: Detected new or changed devices on smithi043 2024-09-16T23:44:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi043 to 7458M 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:27 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: Detected new or changed devices on smithi043 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi043 to 7458M 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:28.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:27 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:29.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:29 smithi043 ceph-mon[35191]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:29.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:44:29.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:44:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:29 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:29.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:29 smithi042 ceph-mon[27914]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:29.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T23:44:29.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T23:44:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:29 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:30.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:30 smithi043 ceph-mon[35191]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:44:30.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:30 smithi042 ceph-mon[27914]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:44:31.357 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:31 smithi043 ceph-mon[35191]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:31.357 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:31 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/3298594' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e51d900-ce2d-4cb7-894b-aa8d04938cbd"}]: dispatch 2024-09-16T23:44:31.357 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:31 smithi043 ceph-mon[35191]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e51d900-ce2d-4cb7-894b-aa8d04938cbd"}]: dispatch 2024-09-16T23:44:31.357 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:31 smithi043 ceph-mon[35191]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e51d900-ce2d-4cb7-894b-aa8d04938cbd"}]': finished 2024-09-16T23:44:31.358 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:31 smithi043 ceph-mon[35191]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T23:44:31.358 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:31.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:31 smithi042 ceph-mon[27914]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:31.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:31 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/3298594' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e51d900-ce2d-4cb7-894b-aa8d04938cbd"}]: dispatch 2024-09-16T23:44:31.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:31 smithi042 ceph-mon[27914]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e51d900-ce2d-4cb7-894b-aa8d04938cbd"}]: dispatch 2024-09-16T23:44:31.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:31 smithi042 ceph-mon[27914]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e51d900-ce2d-4cb7-894b-aa8d04938cbd"}]': finished 2024-09-16T23:44:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:31 smithi042 ceph-mon[27914]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T23:44:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:32.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:32 smithi043 ceph-mon[35191]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:32.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:32 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/1532011212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:44:32.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:32 smithi042 ceph-mon[27914]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:32.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:32 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/1532011212' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T23:44:34.403 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:34 smithi043 ceph-mon[35191]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:34.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:34 smithi042 ceph-mon[27914]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:36.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:36 smithi042 ceph-mon[27914]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:36.529 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:36 smithi043 ceph-mon[35191]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:38.747 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:38 smithi043 ceph-mon[35191]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:38.748 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:38 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T23:44:38.748 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:38 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:38.748 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:38 smithi043 ceph-mon[35191]: Deploying daemon osd.5 on smithi043 2024-09-16T23:44:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:38 smithi042 ceph-mon[27914]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:38 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T23:44:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:38 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:38 smithi042 ceph-mon[27914]: Deploying daemon osd.5 on smithi043 2024-09-16T23:44:39.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:40.708 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:40 smithi043 ceph-mon[35191]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:41.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:40 smithi042 ceph-mon[27914]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:42.841 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:42 smithi043 ceph-mon[35191]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:43.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:42 smithi042 ceph-mon[27914]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:44 smithi043 ceph-mon[35191]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:44 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:44 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:44 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:44 smithi042 ceph-mon[27914]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:44 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:44 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:44 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:46.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:46 smithi042 ceph-mon[27914]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:46.860 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:46 smithi043 ceph-mon[35191]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.607+0000 7f4a53fff700 1 -- 172.21.15.43:0/1550224670 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a5c02cdb0 con 0x7f4a4806c4e0 2024-09-16T23:44:47.606 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 5 on host 'smithi043' 2024-09-16T23:44:47.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4a4806c4e0 msgr2=0x7f4a4806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4a4806c4e0 0x7f4a4806e9a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c069490 tx=0x7f4a4400a300 comp rx=0 tx=0).stop 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 msgr2=0x7f4a5c069710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c069710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00bda0 tx=0x7f4a4c00bdd0 comp rx=0 tx=0).stop 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 shutdown_connections 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f4a4806c4e0 0x7f4a4806e9a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a5c0ffbc0 0x7f4a5c069710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 --2- 172.21.15.43:0/1550224670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a5c100dc0 0x7f4a5c071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 >> 172.21.15.43:0/1550224670 conn(0x7f4a5c0fb160 msgr2=0x7f4a5c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 shutdown_connections 2024-09-16T23:44:47.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:44:47.610+0000 7f4a61314700 1 -- 172.21.15.43:0/1550224670 wait complete. 2024-09-16T23:44:47.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:47 smithi043 ceph-mon[35191]: from='osd.5 [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:44:47.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:47 smithi043 ceph-mon[35191]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:44:47.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:47.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:48.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:47 smithi042 ceph-mon[27914]: from='osd.5 [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:44:48.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:47 smithi042 ceph-mon[27914]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:44:48.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:48.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:48.211 DEBUG:teuthology.orchestra.run.smithi043:osd.5> sudo journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T23:44:48.214 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-09-16T23:44:48.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T23:44:48.438 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:44:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T23:44:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T23:44:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: from='osd.5 [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:48.878 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[54365]: 2024-09-16T23:44:48.613+0000 7fee09001700 -1 osd.5 0 waiting for initial osdmap 2024-09-16T23:44:48.878 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:44:48 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[54365]: 2024-09-16T23:44:48.620+0000 7fee03df4700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:44:48.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:48.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T23:44:48.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T23:44:48.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:48.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: from='osd.5 [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:48.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:44:48.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:48.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:48 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:49.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:49 smithi042 ceph-mon[27914]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-16T23:44:49.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:49 smithi042 ceph-mon[27914]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T23:44:49.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:49.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:49.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:49 smithi043 ceph-mon[35191]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-09-16T23:44:49.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:49 smithi043 ceph-mon[35191]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T23:44:49.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:49.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 -- 172.21.15.42:0/4032423735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 msgr2=0x7fb780102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 --2- 172.21.15.42:0/4032423735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780102310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb774009910 tx=0x7fb774009c20 comp rx=0 tx=0).stop 2024-09-16T23:44:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 -- 172.21.15.42:0/4032423735 shutdown_connections 2024-09-16T23:44:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 --2- 172.21.15.42:0/4032423735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780102310 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 --2- 172.21.15.42:0/4032423735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb780100cb0 0x7fb7801010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 -- 172.21.15.42:0/4032423735 >> 172.21.15.42:0/4032423735 conn(0x7fb7800fc230 msgr2=0x7fb7800fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:50.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 -- 172.21.15.42:0/4032423735 shutdown_connections 2024-09-16T23:44:50.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.017+0000 7fb7862c8700 1 -- 172.21.15.42:0/4032423735 wait complete. 2024-09-16T23:44:50.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb7862c8700 1 Processor -- start 2024-09-16T23:44:50.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb7862c8700 1 -- start start 2024-09-16T23:44:50.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb7862c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb780100cb0 0x7fb7801964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:50.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb7862c8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:50.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb7862c8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780197030 con 0x7fb780101eb0 2024-09-16T23:44:50.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb7862c8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb780197170 con 0x7fb780100cb0 2024-09-16T23:44:50.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb77f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:50.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb77f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45308/0 (socket says 172.21.15.42:45308) 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb77f7fe700 1 -- 172.21.15.42:0/1002298709 learned_addr learned my addr 172.21.15.42:0/1002298709 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.018+0000 7fb77ffff700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb780100cb0 0x7fb7801964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb77f7fe700 1 -- 172.21.15.42:0/1002298709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb780100cb0 msgr2=0x7fb7801964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb77f7fe700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb780100cb0 0x7fb7801964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb77f7fe700 1 -- 172.21.15.42:0/1002298709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740095f0 con 0x7fb780101eb0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb77f7fe700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780196a10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb774006010 tx=0x7fb77400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb77d7fa700 1 -- 172.21.15.42:0/1002298709 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77401d070 con 0x7fb780101eb0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb77d7fa700 1 -- 172.21.15.42:0/1002298709 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb77400f460 con 0x7fb780101eb0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78019bbc0 con 0x7fb780101eb0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb77d7fa700 1 -- 172.21.15.42:0/1002298709 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb774021640 con 0x7fb780101eb0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.019+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78019c0b0 con 0x7fb780101eb0 2024-09-16T23:44:50.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.020+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb780109830 con 0x7fb780101eb0 2024-09-16T23:44:50.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.021+0000 7fb77d7fa700 1 -- 172.21.15.42:0/1002298709 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb77400fab0 con 0x7fb780101eb0 2024-09-16T23:44:50.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.021+0000 7fb77d7fa700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb76806c4e0 0x7fb76806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:50.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.021+0000 7fb77d7fa700 1 -- 172.21.15.42:0/1002298709 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fb77408c680 con 0x7fb780101eb0 2024-09-16T23:44:50.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.021+0000 7fb77ffff700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb76806c4e0 0x7fb76806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:50.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.022+0000 7fb77ffff700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb76806c4e0 0x7fb76806e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb780101d10 tx=0x7fb77000b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.026+0000 7fb77d7fa700 1 -- 172.21.15.42:0/1002298709 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb77405acd0 con 0x7fb780101eb0 2024-09-16T23:44:50.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.226+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fb780109a20 con 0x7fb780101eb0 2024-09-16T23:44:50.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.226+0000 7fb77d7fa700 1 -- 172.21.15.42:0/1002298709 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fb77405a860 con 0x7fb780101eb0 2024-09-16T23:44:50.224 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.229+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb76806c4e0 msgr2=0x7fb76806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.229+0000 7fb7862c8700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb76806c4e0 0x7fb76806e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb780101d10 tx=0x7fb77000b410 comp rx=0 tx=0).stop 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.229+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 msgr2=0x7fb780196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.229+0000 7fb7862c8700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780196a10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb774006010 tx=0x7fb77400bd70 comp rx=0 tx=0).stop 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.230+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 shutdown_connections 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.230+0000 7fb7862c8700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb76806c4e0 0x7fb76806e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.230+0000 7fb7862c8700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb780100cb0 0x7fb7801964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:50.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.230+0000 7fb7862c8700 1 --2- 172.21.15.42:0/1002298709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb780101eb0 0x7fb780196a10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:50.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.230+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 >> 172.21.15.42:0/1002298709 conn(0x7fb7800fc230 msgr2=0x7fb7801050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:50.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.230+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 shutdown_connections 2024-09-16T23:44:50.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:50.230+0000 7fb7862c8700 1 -- 172.21.15.42:0/1002298709 wait complete. 2024-09-16T23:44:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:50 smithi042 ceph-mon[27914]: purged_snaps scrub starts 2024-09-16T23:44:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:50 smithi042 ceph-mon[27914]: purged_snaps scrub ok 2024-09-16T23:44:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:50 smithi042 ceph-mon[27914]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:50 smithi042 ceph-mon[27914]: osd.5 [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] boot 2024-09-16T23:44:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:50 smithi042 ceph-mon[27914]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T23:44:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:50 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:50 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1002298709' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T23:44:50.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:50 smithi043 ceph-mon[35191]: purged_snaps scrub starts 2024-09-16T23:44:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:50 smithi043 ceph-mon[35191]: purged_snaps scrub ok 2024-09-16T23:44:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:50 smithi043 ceph-mon[35191]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T23:44:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:50 smithi043 ceph-mon[35191]: osd.5 [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] boot 2024-09-16T23:44:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:50 smithi043 ceph-mon[35191]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T23:44:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:50 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:44:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:50 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1002298709' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T23:44:50.970 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1726530289,"num_in_osds":6,"osd_in_since":1726530270,"num_remapped_pgs":0} 2024-09-16T23:44:50.971 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T23:44:51.182 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:44:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:51 smithi043 ceph-mon[35191]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T23:44:51.887 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:51 smithi042 ceph-mon[27914]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T23:44:52.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 -- 172.21.15.42:0/785303223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 msgr2=0x7fce2c10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 --2- 172.21.15.42:0/785303223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c10abf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fce14009910 tx=0x7fce14009c20 comp rx=0 tx=0).stop 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 -- 172.21.15.42:0/785303223 shutdown_connections 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 --2- 172.21.15.42:0/785303223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c10abf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 --2- 172.21.15.42:0/785303223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c101de0 0x7fce2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 -- 172.21.15.42:0/785303223 >> 172.21.15.42:0/785303223 conn(0x7fce2c0fb610 msgr2=0x7fce2c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 -- 172.21.15.42:0/785303223 shutdown_connections 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.715+0000 7fce31638700 1 -- 172.21.15.42:0/785303223 wait complete. 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce31638700 1 Processor -- start 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce31638700 1 -- start start 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce31638700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c101de0 0x7fce2c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:52.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce31638700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce31638700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c19d2c0 con 0x7fce2c102700 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce31638700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c1967c0 con 0x7fce2c101de0 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce227fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce227fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58850/0 (socket says 172.21.15.42:58850) 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce227fc700 1 -- 172.21.15.42:0/2748048514 learned_addr learned my addr 172.21.15.42:0/2748048514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce2affd700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c101de0 0x7fce2c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce227fc700 1 -- 172.21.15.42:0/2748048514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c101de0 msgr2=0x7fce2c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.716+0000 7fce227fc700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c101de0 0x7fce2c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.717+0000 7fce227fc700 1 -- 172.21.15.42:0/2748048514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140095f0 con 0x7fce2c102700 2024-09-16T23:44:52.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.717+0000 7fce227fc700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c19cc30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce140092e0 tx=0x7fce1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:52.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.717+0000 7fce28ff9700 1 -- 172.21.15.42:0/2748048514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce1401d070 con 0x7fce2c102700 2024-09-16T23:44:52.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.717+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c196a40 con 0x7fce2c102700 2024-09-16T23:44:52.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.717+0000 7fce28ff9700 1 -- 172.21.15.42:0/2748048514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce1400f460 con 0x7fce2c102700 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.717+0000 7fce28ff9700 1 -- 172.21.15.42:0/2748048514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce14017630 con 0x7fce2c102700 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.717+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c196f30 con 0x7fce2c102700 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.718+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c1082f0 con 0x7fce2c102700 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.719+0000 7fce28ff9700 1 -- 172.21.15.42:0/2748048514 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fce1400f5d0 con 0x7fce2c102700 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.719+0000 7fce28ff9700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fce1806c4e0 0x7fce1806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.719+0000 7fce2affd700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fce1806c4e0 0x7fce1806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.719+0000 7fce28ff9700 1 -- 172.21.15.42:0/2748048514 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fce1408ca40 con 0x7fce2c102700 2024-09-16T23:44:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.720+0000 7fce2affd700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fce1806c4e0 0x7fce1806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce1c006fd0 tx=0x7fce1c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:52.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.724+0000 7fce28ff9700 1 -- 172.21.15.42:0/2748048514 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fce1405b090 con 0x7fce2c102700 2024-09-16T23:44:52.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.924+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce2c0679c0 con 0x7fce2c102700 2024-09-16T23:44:52.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.925+0000 7fce28ff9700 1 -- 172.21.15.42:0/2748048514 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11010 (secure 0 0 0) 0x7fce1405ac20 con 0x7fce2c102700 2024-09-16T23:44:52.923 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:44:52.923 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":40,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","created":"2024-09-16T23:39:01.224551+0000","modified":"2024-09-16T23:44:50.615829+0000","last_up_change":"2024-09-16T23:44:49.610877+0000","last_in_change":"2024-09-16T23:44:30.700582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T23:43:31.584431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"82c23a1b-b658-4424-9c3b-f324a6b0861b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6803","nonce":728536688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6805","nonce":728536688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6809","nonce":728536688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6807","nonce":728536688}]},"public_addr":"172.21.15.42:6803/728536688","cluster_addr":"172.21.15.42:6805/728536688","heartbeat_back_addr":"172.21.15.42:6809/728536688","heartbeat_front_addr":"172.21.15.42:6807/728536688","state":["exists","up"]},{"osd":1,"uuid":"644cf390-4e6f-402f-92ff-dcd7330ee305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6811","nonce":1956379932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6813","nonce":1956379932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6817","nonce":1956379932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6815","nonce":1956379932}]},"public_addr":"172.21.15.42:6811/1956379932","cluster_addr":"172.21.15.42:6813/1956379932","heartbeat_back_addr":"172.21.15.42:6817/1956379932","heartbeat_front_addr":"172.21.15.42:6815/1956379932","state":["exists","up"]},{"osd":2,"uuid":"a27ed11a-999c-439e-b098-3381127c72e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6819","nonce":3083836466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6821","nonce":3083836466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6825","nonce":3083836466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6823","nonce":3083836466}]},"public_addr":"172.21.15.42:6819/3083836466","cluster_addr":"172.21.15.42:6821/3083836466","heartbeat_back_addr":"172.21.15.42:6825/3083836466","heartbeat_front_addr":"172.21.15.42:6823/3083836466","state":["exists","up"]},{"osd":3,"uuid":"2b84bdfd-14ef-4867-a5d0-b443a5dcec12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6801","nonce":3558322430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6803","nonce":3558322430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6807","nonce":3558322430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6805","nonce":3558322430}]},"public_addr":"172.21.15.43:6801/3558322430","cluster_addr":"172.21.15.43:6803/3558322430","heartbeat_back_addr":"172.21.15.43:6807/3558322430","heartbeat_front_addr":"172.21.15.43:6805/3558322430","state":["exists","up"]},{"osd":4,"uuid":"093b60b2-f2fd-4261-b038-4400c2a76180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6809","nonce":3817520749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6811","nonce":3817520749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6815","nonce":3817520749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6813","nonce":3817520749}]},"public_addr":"172.21.15.43:6809/3817520749","cluster_addr":"172.21.15.43:6811/3817520749","heartbeat_back_addr":"172.21.15.43:6815/3817520749","heartbeat_front_addr":"172.21.15.43:6813/3817520749","state":["exists","up"]},{"osd":5,"uuid":"6e51d900-ce2d-4cb7-894b-aa8d04938cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6817","nonce":205646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6819","nonce":205646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6823","nonce":205646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6821","nonce":205646323}]},"public_addr":"172.21.15.43:6817/205646323","cluster_addr":"172.21.15.43:6819/205646323","heartbeat_back_addr":"172.21.15.43:6823/205646323","heartbeat_front_addr":"172.21.15.43:6821/205646323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:42:32.529981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:01.516391+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:28.822541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:54.549222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:44:21.118416+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:44:48.359630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/241787870":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/2316861404":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/4154448068":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/3709834403":"2024-09-17T23:40:02.177376+0000","172.21.15.42:0/1983510372":"2024-09-17T23:40:02.177376+0000","172.21.15.42:6801/2":"2024-09-17T23:39:29.178241+0000","172.21.15.42:6800/2":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/911608840":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/585424460":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/1732693245":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/3534532679":"2024-09-17T23:40:02.177376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T23:44:52.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fce1806c4e0 msgr2=0x7fce1806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fce1806c4e0 0x7fce1806e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fce1c006fd0 tx=0x7fce1c008040 comp rx=0 tx=0).stop 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 msgr2=0x7fce2c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c19cc30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce140092e0 tx=0x7fce1400be20 comp rx=0 tx=0).stop 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 shutdown_connections 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fce1806c4e0 0x7fce1806e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce2c101de0 0x7fce2c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 --2- 172.21.15.42:0/2748048514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce2c102700 0x7fce2c19cc30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 >> 172.21.15.42:0/2748048514 conn(0x7fce2c0fb610 msgr2=0x7fce2c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 shutdown_connections 2024-09-16T23:44:52.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:52.927+0000 7fce31638700 1 -- 172.21.15.42:0/2748048514 wait complete. 2024-09-16T23:44:53.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: Detected new or changed devices on smithi043 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi043 to 4972M 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:53.220 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:53.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:53.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:53.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.221 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:53 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2748048514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T23:44:53.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: Detected new or changed devices on smithi043 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi043 to 4972M 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:44:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:44:53.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:44:53.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:44:53.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:53 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2748048514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T23:44:53.628 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T23:43:31.584431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T23:44:53.628 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T23:44:53.838 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:44:55.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:55 smithi042 ceph-mon[27914]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:55.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:55 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:55.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:55 smithi043 ceph-mon[35191]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:55.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:55 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:44:55.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 -- 172.21.15.42:0/4181605926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8101e00 msgr2=0x7f11c81021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/4181605926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8101e00 0x7f11c81021e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f11b800b0a0 tx=0x7f11b800b3b0 comp rx=0 tx=0).stop 2024-09-16T23:44:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 -- 172.21.15.42:0/4181605926 shutdown_connections 2024-09-16T23:44:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/4181605926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8102720 0x7f11c810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/4181605926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8101e00 0x7f11c81021e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 -- 172.21.15.42:0/4181605926 >> 172.21.15.42:0/4181605926 conn(0x7f11c80fb650 msgr2=0x7f11c80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 -- 172.21.15.42:0/4181605926 shutdown_connections 2024-09-16T23:44:55.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.492+0000 7f11ccdd0700 1 -- 172.21.15.42:0/4181605926 wait complete. 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11ccdd0700 1 Processor -- start 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11ccdd0700 1 -- start start 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11ccdd0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8101e00 0x7f11c80ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11ccdd0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8102720 0x7f11c80fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11ccdd0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81004e0 con 0x7f11c8102720 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11ccdd0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8100620 con 0x7f11c8101e00 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11c5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8102720 0x7f11c80fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11c5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8102720 0x7f11c80fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58862/0 (socket says 172.21.15.42:58862) 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11c5d9b700 1 -- 172.21.15.42:0/3642629909 learned_addr learned my addr 172.21.15.42:0/3642629909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11c659c700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8101e00 0x7f11c80ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:55.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11c5d9b700 1 -- 172.21.15.42:0/3642629909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8101e00 msgr2=0x7f11c80ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11c5d9b700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8101e00 0x7f11c80ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.493+0000 7f11c5d9b700 1 -- 172.21.15.42:0/3642629909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8009d00 con 0x7f11c8102720 2024-09-16T23:44:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.494+0000 7f11c5d9b700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8102720 0x7f11c80fffa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00d6d0 tx=0x7f11bc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.494+0000 7f11b77fe700 1 -- 172.21.15.42:0/3642629909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc00f860 con 0x7f11c8102720 2024-09-16T23:44:55.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.494+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c81a2b20 con 0x7f11c8102720 2024-09-16T23:44:55.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.494+0000 7f11b77fe700 1 -- 172.21.15.42:0/3642629909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f11bc00fe70 con 0x7f11c8102720 2024-09-16T23:44:55.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.494+0000 7f11b77fe700 1 -- 172.21.15.42:0/3642629909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11bc00e5e0 con 0x7f11c8102720 2024-09-16T23:44:55.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.494+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c81a2f30 con 0x7f11c8102720 2024-09-16T23:44:55.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.495+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c8108310 con 0x7f11c8102720 2024-09-16T23:44:55.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.495+0000 7f11b77fe700 1 -- 172.21.15.42:0/3642629909 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f11bc010460 con 0x7f11c8102720 2024-09-16T23:44:55.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.496+0000 7f11b77fe700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f11b006c4e0 0x7f11b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:55.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.496+0000 7f11b77fe700 1 -- 172.21.15.42:0/3642629909 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f11bc08b000 con 0x7f11c8102720 2024-09-16T23:44:55.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.496+0000 7f11c659c700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f11b006c4e0 0x7f11b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:55.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.496+0000 7f11c659c700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f11b006c4e0 0x7f11b006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11b8015040 tx=0x7f11b8006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:55.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.500+0000 7f11b77fe700 1 -- 172.21.15.42:0/3642629909 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f11bc055bc0 con 0x7f11c8102720 2024-09-16T23:44:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.698+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11c8107cf0 con 0x7f11c8102720 2024-09-16T23:44:55.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.698+0000 7f11b77fe700 1 -- 172.21.15.42:0/3642629909 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11bc0591e0 con 0x7f11c8102720 2024-09-16T23:44:55.696 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f11b006c4e0 msgr2=0x7f11b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f11b006c4e0 0x7f11b006e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11b8015040 tx=0x7f11b8006000 comp rx=0 tx=0).stop 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8102720 msgr2=0x7f11c80fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8102720 0x7f11c80fffa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00d6d0 tx=0x7f11bc00da90 comp rx=0 tx=0).stop 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 shutdown_connections 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f11b006c4e0 0x7f11b006e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11c8101e00 0x7f11c80ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:55.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 --2- 172.21.15.42:0/3642629909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11c8102720 0x7f11c80fffa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 >> 172.21.15.42:0/3642629909 conn(0x7f11c80fb650 msgr2=0x7f11c8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 shutdown_connections 2024-09-16T23:44:55.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:55.701+0000 7f11ccdd0700 1 -- 172.21.15.42:0/3642629909 wait complete. 2024-09-16T23:44:56.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:56 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3642629909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T23:44:56.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:56 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3642629909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T23:44:56.423 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T23:44:56.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T23:44:56.630 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:44:57.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:57 smithi042 ceph-mon[27914]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:57.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:57 smithi043 ceph-mon[35191]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.165+0000 7fb2647c2700 1 -- 172.21.15.42:0/1188509642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c0ffbc0 msgr2=0x7fb25c0fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.165+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1188509642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c0ffbc0 0x7fb25c0fffe0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb258009910 tx=0x7fb258009c20 comp rx=0 tx=0).stop 2024-09-16T23:44:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.165+0000 7fb2647c2700 1 -- 172.21.15.42:0/1188509642 shutdown_connections 2024-09-16T23:44:58.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.165+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1188509642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25c100dc0 0x7fb25c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.165+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1188509642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c0ffbc0 0x7fb25c0fffe0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.165+0000 7fb2647c2700 1 -- 172.21.15.42:0/1188509642 >> 172.21.15.42:0/1188509642 conn(0x7fb25c0fb160 msgr2=0x7fb25c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.165+0000 7fb2647c2700 1 -- 172.21.15.42:0/1188509642 shutdown_connections 2024-09-16T23:44:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.166+0000 7fb2647c2700 1 -- 172.21.15.42:0/1188509642 wait complete. 2024-09-16T23:44:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.166+0000 7fb2647c2700 1 Processor -- start 2024-09-16T23:44:58.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.166+0000 7fb2647c2700 1 -- start start 2024-09-16T23:44:58.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2647c2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25c0ffbc0 0x7fb25c069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:58.165 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2647c2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c100dc0 0x7fb25c19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2647c2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c19bb00 con 0x7fb25c100dc0 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2647c2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c19bc70 con 0x7fb25c0ffbc0 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb261d5d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c100dc0 0x7fb25c19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb261d5d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c100dc0 0x7fb25c19b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58878/0 (socket says 172.21.15.42:58878) 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb261d5d700 1 -- 172.21.15.42:0/1160783155 learned_addr learned my addr 172.21.15.42:0/1160783155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb26255e700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25c0ffbc0 0x7fb25c069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb261d5d700 1 -- 172.21.15.42:0/1160783155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25c0ffbc0 msgr2=0x7fb25c069990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb261d5d700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25c0ffbc0 0x7fb25c069990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb261d5d700 1 -- 172.21.15.42:0/1160783155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580095f0 con 0x7fb25c100dc0 2024-09-16T23:44:58.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb261d5d700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c100dc0 0x7fb25c19b530 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00b4b0 tx=0x7fb24c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:58.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2537fe700 1 -- 172.21.15.42:0/1160783155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb24c011800 con 0x7fb25c100dc0 2024-09-16T23:44:58.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2537fe700 1 -- 172.21.15.42:0/1160783155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb24c011e10 con 0x7fb25c100dc0 2024-09-16T23:44:58.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c19bf50 con 0x7fb25c100dc0 2024-09-16T23:44:58.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2537fe700 1 -- 172.21.15.42:0/1160783155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb24c00f2e0 con 0x7fb25c100dc0 2024-09-16T23:44:58.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.167+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c19c470 con 0x7fb25c100dc0 2024-09-16T23:44:58.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.168+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c18f5b0 con 0x7fb25c100dc0 2024-09-16T23:44:58.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.169+0000 7fb2537fe700 1 -- 172.21.15.42:0/1160783155 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb24c010380 con 0x7fb25c100dc0 2024-09-16T23:44:58.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.169+0000 7fb2537fe700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb24806c4e0 0x7fb24806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:44:58.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.170+0000 7fb26255e700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb24806c4e0 0x7fb24806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:44:58.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.170+0000 7fb2537fe700 1 -- 172.21.15.42:0/1160783155 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fb24c08af00 con 0x7fb25c100dc0 2024-09-16T23:44:58.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.170+0000 7fb26255e700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb24806c4e0 0x7fb24806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb258006010 tx=0x7fb258005280 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:44:58.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.174+0000 7fb2537fe700 1 -- 172.21.15.42:0/1160783155 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb24c0595d0 con 0x7fb25c100dc0 2024-09-16T23:44:58.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.456+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fb25c068af0 con 0x7fb25c100dc0 2024-09-16T23:44:58.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.459+0000 7fb2537fe700 1 -- 172.21.15.42:0/1160783155 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fb24c0150f0 con 0x7fb25c100dc0 2024-09-16T23:44:58.457 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-09-16T23:44:58.457 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQD6wuhm+2hFGxAAY6k0LPAjGXsvSvijSu8jxA== 2024-09-16T23:44:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb24806c4e0 msgr2=0x7fb24806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb24806c4e0 0x7fb24806e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb258006010 tx=0x7fb258005280 comp rx=0 tx=0).stop 2024-09-16T23:44:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c100dc0 msgr2=0x7fb25c19b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:44:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c100dc0 0x7fb25c19b530 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00b4b0 tx=0x7fb24c00b7c0 comp rx=0 tx=0).stop 2024-09-16T23:44:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 shutdown_connections 2024-09-16T23:44:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb24806c4e0 0x7fb24806e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:58.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb25c0ffbc0 0x7fb25c069990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:58.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 --2- 172.21.15.42:0/1160783155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c100dc0 0x7fb25c19b530 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:44:58.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.461+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 >> 172.21.15.42:0/1160783155 conn(0x7fb25c0fb160 msgr2=0x7fb25c1040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:44:58.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.462+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 shutdown_connections 2024-09-16T23:44:58.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:44:58.462+0000 7fb2647c2700 1 -- 172.21.15.42:0/1160783155 wait complete. 2024-09-16T23:44:59.109 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:59 smithi042 ceph-mon[27914]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:59.109 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:59 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1160783155' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T23:44:59.109 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:44:59 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1160783155' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T23:44:59.133 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:44:59.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T23:44:59.133 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T23:44:59.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T23:44:59.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:59 smithi043 ceph-mon[35191]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:44:59.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:59 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1160783155' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T23:44:59.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:44:59 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1160783155' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T23:44:59.395 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 -- 172.21.15.43:0/1511178624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef40ffbc0 msgr2=0x7feef40fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 --2- 172.21.15.43:0/1511178624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef40ffbc0 0x7feef40fffe0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feee4009910 tx=0x7feee4009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 -- 172.21.15.43:0/1511178624 shutdown_connections 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 --2- 172.21.15.43:0/1511178624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feef4100dc0 0x7feef4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 --2- 172.21.15.43:0/1511178624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef40ffbc0 0x7feef40fffe0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 -- 172.21.15.43:0/1511178624 >> 172.21.15.43:0/1511178624 conn(0x7feef40fb160 msgr2=0x7feef40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 -- 172.21.15.43:0/1511178624 shutdown_connections 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.036+0000 7feef88bb700 1 -- 172.21.15.43:0/1511178624 wait complete. 2024-09-16T23:45:01.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef88bb700 1 Processor -- start 2024-09-16T23:45:01.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef88bb700 1 -- start start 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef88bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feef40ffbc0 0x7feef4195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef88bb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef4100dc0 0x7feef4195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef88bb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef4196260 con 0x7feef40ffbc0 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef88bb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef41963a0 con 0x7feef4100dc0 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef1d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef4100dc0 0x7feef4195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef1d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef4100dc0 0x7feef4195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52584/0 (socket says 172.21.15.43:52584) 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef1d9b700 1 -- 172.21.15.43:0/3801278905 learned_addr learned my addr 172.21.15.43:0/3801278905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef259c700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feef40ffbc0 0x7feef4195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:01.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef1d9b700 1 -- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feef40ffbc0 msgr2=0x7feef4195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef1d9b700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feef40ffbc0 0x7feef4195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.037+0000 7feef1d9b700 1 -- 172.21.15.43:0/3801278905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee40095f0 con 0x7feef4100dc0 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.038+0000 7feef1d9b700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef4100dc0 0x7feef4195c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feee8007b50 tx=0x7feee8007e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.038+0000 7feee37fe700 1 -- 172.21.15.43:0/3801278905 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee8011070 con 0x7feef4100dc0 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.038+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef4068e70 con 0x7feef4100dc0 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.038+0000 7feee37fe700 1 -- 172.21.15.43:0/3801278905 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feee800c2e0 con 0x7feef4100dc0 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.038+0000 7feee37fe700 1 -- 172.21.15.43:0/3801278905 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee800e6b0 con 0x7feef4100dc0 2024-09-16T23:45:01.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.038+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef40693c0 con 0x7feef4100dc0 2024-09-16T23:45:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.039+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef4068af0 con 0x7feef4100dc0 2024-09-16T23:45:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.039+0000 7feee37fe700 1 -- 172.21.15.43:0/3801278905 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7feee800e920 con 0x7feef4100dc0 2024-09-16T23:45:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.040+0000 7feee37fe700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7feedc06c4e0 0x7feedc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.040+0000 7feee37fe700 1 -- 172.21.15.43:0/3801278905 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7feee808ca20 con 0x7feef4100dc0 2024-09-16T23:45:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.040+0000 7feef259c700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7feedc06c4e0 0x7feedc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:01.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.040+0000 7feef259c700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7feedc06c4e0 0x7feedc06e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feee4006010 tx=0x7feee400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:01.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.045+0000 7feee37fe700 1 -- 172.21.15.43:0/3801278905 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feee805b120 con 0x7feef4100dc0 2024-09-16T23:45:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:01 smithi043 ceph-mon[35191]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:01 smithi042 ceph-mon[27914]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:01.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.324+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7feef418f410 con 0x7feef4100dc0 2024-09-16T23:45:01.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.329+0000 7feee37fe700 1 -- 172.21.15.43:0/3801278905 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7feee805acb0 con 0x7feef4100dc0 2024-09-16T23:45:01.327 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2024-09-16T23:45:01.327 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD9wuhms5l3ExAA0c7WJt4ozCsFc9RCtj7UUw== 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7feedc06c4e0 msgr2=0x7feedc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7feedc06c4e0 0x7feedc06e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feee4006010 tx=0x7feee400b540 comp rx=0 tx=0).stop 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef4100dc0 msgr2=0x7feef4195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef4100dc0 0x7feef4195c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7feee8007b50 tx=0x7feee8007e60 comp rx=0 tx=0).stop 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 shutdown_connections 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7feedc06c4e0 0x7feedc06e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feef40ffbc0 0x7feef4195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 --2- 172.21.15.43:0/3801278905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feef4100dc0 0x7feef4195c40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 >> 172.21.15.43:0/3801278905 conn(0x7feef40fb160 msgr2=0x7feef4104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 shutdown_connections 2024-09-16T23:45:01.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:45:01.332+0000 7feef88bb700 1 -- 172.21.15.43:0/3801278905 wait complete. 2024-09-16T23:45:01.982 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:45:01.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T23:45:01.982 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T23:45:02.022 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T23:45:02.022 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T23:45:02.022 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T23:45:02.240 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:02.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:02 smithi042 ceph-mon[27914]: from='client.? 172.21.15.43:0/3801278905' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T23:45:02.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:02 smithi042 ceph-mon[27914]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T23:45:02.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:02 smithi042 ceph-mon[27914]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T23:45:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:02 smithi043 ceph-mon[35191]: from='client.? 172.21.15.43:0/3801278905' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T23:45:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:02 smithi043 ceph-mon[35191]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T23:45:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:02 smithi043 ceph-mon[35191]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T23:45:03.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:03 smithi042 ceph-mon[27914]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:03 smithi043 ceph-mon[35191]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:03.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 -- 172.21.15.42:0/1128425156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 msgr2=0x7f0960105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 --2- 172.21.15.42:0/1128425156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f0960105fd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f095c009910 tx=0x7f095c009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 -- 172.21.15.42:0/1128425156 shutdown_connections 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 --2- 172.21.15.42:0/1128425156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09600ffb40 0x7f09600fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 --2- 172.21.15.42:0/1128425156 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f0960105fd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 -- 172.21.15.42:0/1128425156 >> 172.21.15.42:0/1128425156 conn(0x7f09600fb610 msgr2=0x7f09600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 -- 172.21.15.42:0/1128425156 shutdown_connections 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.915+0000 7f09685a9700 1 -- 172.21.15.42:0/1128425156 wait complete. 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f09685a9700 1 Processor -- start 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f09685a9700 1 -- start start 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f09685a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09600ffb40 0x7f096010aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f09685a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f096010afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:03.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f09685a9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096010b500 con 0x7f0960105bf0 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f09685a9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096010b640 con 0x7f09600ffb40 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f0965b44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f096010afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f0965b44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f096010afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56894/0 (socket says 172.21.15.42:56894) 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f0965b44700 1 -- 172.21.15.42:0/2755486475 learned_addr learned my addr 172.21.15.42:0/2755486475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.916+0000 7f0966345700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09600ffb40 0x7f096010aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f0965b44700 1 -- 172.21.15.42:0/2755486475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09600ffb40 msgr2=0x7f096010aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f0965b44700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09600ffb40 0x7f096010aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:03.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f0965b44700 1 -- 172.21.15.42:0/2755486475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0095f0 con 0x7f0960105bf0 2024-09-16T23:45:03.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f0965b44700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f096010afc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f095000c930 tx=0x7f095000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:03.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f09577fe700 1 -- 172.21.15.42:0/2755486475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0950007ad0 con 0x7f0960105bf0 2024-09-16T23:45:03.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f09577fe700 1 -- 172.21.15.42:0/2755486475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0950007c30 con 0x7f0960105bf0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09601a2a10 con 0x7f0960105bf0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f09577fe700 1 -- 172.21.15.42:0/2755486475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0950018630 con 0x7f0960105bf0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.917+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09601a2d70 con 0x7f0960105bf0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.918+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0960108020 con 0x7f0960105bf0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.919+0000 7f09577fe700 1 -- 172.21.15.42:0/2755486475 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f095000f450 con 0x7f0960105bf0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.919+0000 7f09577fe700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f094c06c4e0 0x7f094c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.919+0000 7f09577fe700 1 -- 172.21.15.42:0/2755486475 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f095008b540 con 0x7f0960105bf0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.919+0000 7f0966345700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f094c06c4e0 0x7f094c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:03.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.920+0000 7f0966345700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f094c06c4e0 0x7f094c06e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b5c0 tx=0x7f095c005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:03.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:03.924+0000 7f09577fe700 1 -- 172.21.15.42:0/2755486475 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0950056180 con 0x7f0960105bf0 2024-09-16T23:45:04.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.182+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0960107a00 con 0x7f0960105bf0 2024-09-16T23:45:04.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.186+0000 7f09577fe700 1 -- 172.21.15.42:0/2755486475 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173011 (secure 0 0 0) 0x7f09500597a0 con 0x7f0960105bf0 2024-09-16T23:45:04.186 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f094c06c4e0 msgr2=0x7f094c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f094c06c4e0 0x7f094c06e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f095c00b5c0 tx=0x7f095c005960 comp rx=0 tx=0).stop 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 msgr2=0x7f096010afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f096010afc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f095000c930 tx=0x7f095000ccf0 comp rx=0 tx=0).stop 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 shutdown_connections 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f094c06c4e0 0x7f094c06e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09600ffb40 0x7f096010aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:04.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 --2- 172.21.15.42:0/2755486475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0960105bf0 0x7f096010afc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 >> 172.21.15.42:0/2755486475 conn(0x7f09600fb610 msgr2=0x7f09600fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 shutdown_connections 2024-09-16T23:45:04.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:04.188+0000 7f09685a9700 1 -- 172.21.15.42:0/2755486475 wait complete. 2024-09-16T23:45:04.964 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi042.zrkumz","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":2},{"type":"v1","addr":"172.21.15.42:6801","nonce":2}]},"active_addr":"172.21.15.42:6801/2","active_change":"2024-09-16T23:41:09.064459+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14238,"name":"smithi043.xveyou","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.42:8443/","prometheus":"http://172.21.15.42:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":218133032}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1281900709}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1894696933}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3842592412}]}]} 2024-09-16T23:45:04.969 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T23:45:04.970 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T23:45:04.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T23:45:05.186 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:05 smithi042 ceph-mon[27914]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:05 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2755486475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T23:45:05.306 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:05 smithi043 ceph-mon[35191]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:05.306 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:05 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2755486475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T23:45:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 -- 172.21.15.42:0/4243106728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc101e60 msgr2=0x7fe1fc102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:06.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 --2- 172.21.15.42:0/4243106728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc101e60 0x7fe1fc102240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec00b0d0 tx=0x7fe1ec00b3e0 comp rx=0 tx=0).stop 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 -- 172.21.15.42:0/4243106728 shutdown_connections 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 --2- 172.21.15.42:0/4243106728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1fc102780 0x7fe1fc10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 --2- 172.21.15.42:0/4243106728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc101e60 0x7fe1fc102240 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 -- 172.21.15.42:0/4243106728 >> 172.21.15.42:0/4243106728 conn(0x7fe1fc0fb650 msgr2=0x7fe1fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 -- 172.21.15.42:0/4243106728 shutdown_connections 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.915+0000 7fe201ab7700 1 -- 172.21.15.42:0/4243106728 wait complete. 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe201ab7700 1 Processor -- start 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe201ab7700 1 -- start start 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe201ab7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1fc101e60 0x7fe1fc0ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe201ab7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc102780 0x7fe1fc100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe201ab7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc100540 con 0x7fe1fc102780 2024-09-16T23:45:06.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe201ab7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc100680 con 0x7fe1fc101e60 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe1faffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc102780 0x7fe1fc100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe1faffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc102780 0x7fe1fc100000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56910/0 (socket says 172.21.15.42:56910) 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.916+0000 7fe1faffd700 1 -- 172.21.15.42:0/2113711002 learned_addr learned my addr 172.21.15.42:0/2113711002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe1fb7fe700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1fc101e60 0x7fe1fc0ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe1faffd700 1 -- 172.21.15.42:0/2113711002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1fc101e60 msgr2=0x7fe1fc0ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe1faffd700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1fc101e60 0x7fe1fc0ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe1faffd700 1 -- 172.21.15.42:0/2113711002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec009d00 con 0x7fe1fc102780 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe1faffd700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc102780 0x7fe1fc100000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c960 tx=0x7fe1e400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe1f8ff9700 1 -- 172.21.15.42:0/2113711002 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e4007a70 con 0x7fe1fc102780 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc1a2b00 con 0x7fe1fc102780 2024-09-16T23:45:06.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.917+0000 7fe1f8ff9700 1 -- 172.21.15.42:0/2113711002 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe1e4007bd0 con 0x7fe1fc102780 2024-09-16T23:45:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.918+0000 7fe1f8ff9700 1 -- 172.21.15.42:0/2113711002 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e4018630 con 0x7fe1fc102780 2024-09-16T23:45:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.918+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc1a2e60 con 0x7fe1fc102780 2024-09-16T23:45:06.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.919+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc108370 con 0x7fe1fc102780 2024-09-16T23:45:06.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.919+0000 7fe1f8ff9700 1 -- 172.21.15.42:0/2113711002 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe1e400fa90 con 0x7fe1fc102780 2024-09-16T23:45:06.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.919+0000 7fe1f8ff9700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1e806c4e0 0x7fe1e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:06.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.920+0000 7fe1f8ff9700 1 -- 172.21.15.42:0/2113711002 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fe1e408b4a0 con 0x7fe1fc102780 2024-09-16T23:45:06.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.920+0000 7fe1fb7fe700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1e806c4e0 0x7fe1e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:06.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.920+0000 7fe1fb7fe700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1e806c4e0 0x7fe1e806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec000f80 tx=0x7fe1ec00b040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:06.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:06.924+0000 7fe1f8ff9700 1 -- 172.21.15.42:0/2113711002 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe1e4059b70 con 0x7fe1fc102780 2024-09-16T23:45:07.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.123+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1fc04f2e0 con 0x7fe1fc102780 2024-09-16T23:45:07.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.123+0000 7fe1f8ff9700 1 -- 172.21.15.42:0/2113711002 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11010 (secure 0 0 0) 0x7fe1e4059700 con 0x7fe1fc102780 2024-09-16T23:45:07.121 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:45:07.121 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":40,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","created":"2024-09-16T23:39:01.224551+0000","modified":"2024-09-16T23:44:50.615829+0000","last_up_change":"2024-09-16T23:44:49.610877+0000","last_in_change":"2024-09-16T23:44:30.700582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T23:43:31.584431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"82c23a1b-b658-4424-9c3b-f324a6b0861b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6803","nonce":728536688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6805","nonce":728536688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6809","nonce":728536688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6807","nonce":728536688}]},"public_addr":"172.21.15.42:6803/728536688","cluster_addr":"172.21.15.42:6805/728536688","heartbeat_back_addr":"172.21.15.42:6809/728536688","heartbeat_front_addr":"172.21.15.42:6807/728536688","state":["exists","up"]},{"osd":1,"uuid":"644cf390-4e6f-402f-92ff-dcd7330ee305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6811","nonce":1956379932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6813","nonce":1956379932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6817","nonce":1956379932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6815","nonce":1956379932}]},"public_addr":"172.21.15.42:6811/1956379932","cluster_addr":"172.21.15.42:6813/1956379932","heartbeat_back_addr":"172.21.15.42:6817/1956379932","heartbeat_front_addr":"172.21.15.42:6815/1956379932","state":["exists","up"]},{"osd":2,"uuid":"a27ed11a-999c-439e-b098-3381127c72e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6819","nonce":3083836466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6821","nonce":3083836466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6825","nonce":3083836466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6823","nonce":3083836466}]},"public_addr":"172.21.15.42:6819/3083836466","cluster_addr":"172.21.15.42:6821/3083836466","heartbeat_back_addr":"172.21.15.42:6825/3083836466","heartbeat_front_addr":"172.21.15.42:6823/3083836466","state":["exists","up"]},{"osd":3,"uuid":"2b84bdfd-14ef-4867-a5d0-b443a5dcec12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6801","nonce":3558322430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6803","nonce":3558322430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6807","nonce":3558322430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6805","nonce":3558322430}]},"public_addr":"172.21.15.43:6801/3558322430","cluster_addr":"172.21.15.43:6803/3558322430","heartbeat_back_addr":"172.21.15.43:6807/3558322430","heartbeat_front_addr":"172.21.15.43:6805/3558322430","state":["exists","up"]},{"osd":4,"uuid":"093b60b2-f2fd-4261-b038-4400c2a76180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6809","nonce":3817520749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6811","nonce":3817520749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6815","nonce":3817520749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6813","nonce":3817520749}]},"public_addr":"172.21.15.43:6809/3817520749","cluster_addr":"172.21.15.43:6811/3817520749","heartbeat_back_addr":"172.21.15.43:6815/3817520749","heartbeat_front_addr":"172.21.15.43:6813/3817520749","state":["exists","up"]},{"osd":5,"uuid":"6e51d900-ce2d-4cb7-894b-aa8d04938cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6817","nonce":205646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6819","nonce":205646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6823","nonce":205646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6821","nonce":205646323}]},"public_addr":"172.21.15.43:6817/205646323","cluster_addr":"172.21.15.43:6819/205646323","heartbeat_back_addr":"172.21.15.43:6823/205646323","heartbeat_front_addr":"172.21.15.43:6821/205646323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:42:32.529981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:01.516391+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:28.822541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:54.549222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:44:21.118416+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:44:48.359630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/241787870":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/2316861404":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/4154448068":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/3709834403":"2024-09-17T23:40:02.177376+0000","172.21.15.42:0/1983510372":"2024-09-17T23:40:02.177376+0000","172.21.15.42:6801/2":"2024-09-17T23:39:29.178241+0000","172.21.15.42:6800/2":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/911608840":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/585424460":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/1732693245":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/3534532679":"2024-09-17T23:40:02.177376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1e806c4e0 msgr2=0x7fe1e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1e806c4e0 0x7fe1e806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec000f80 tx=0x7fe1ec00b040 comp rx=0 tx=0).stop 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc102780 msgr2=0x7fe1fc100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc102780 0x7fe1fc100000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c960 tx=0x7fe1e400cd20 comp rx=0 tx=0).stop 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 shutdown_connections 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe1e806c4e0 0x7fe1e806e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1fc101e60 0x7fe1fc0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 --2- 172.21.15.42:0/2113711002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1fc102780 0x7fe1fc100000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:07.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 >> 172.21.15.42:0/2113711002 conn(0x7fe1fc0fb650 msgr2=0x7fe1fc1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 shutdown_connections 2024-09-16T23:45:07.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:07.126+0000 7fe201ab7700 1 -- 172.21.15.42:0/2113711002 wait complete. 2024-09-16T23:45:07.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:07 smithi042 ceph-mon[27914]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:07.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:07 smithi043 ceph-mon[35191]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:07.879 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T23:45:07.879 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T23:45:08.095 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:08.216 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:08 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2113711002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T23:45:08.289 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:08 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2113711002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T23:45:09.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:09 smithi043 ceph-mon[35191]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:09.388 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:09 smithi042 ceph-mon[27914]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 -- 172.21.15.42:0/785703704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 msgr2=0x7f0ec81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/785703704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec81070b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009910 tx=0x7f0eb8009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 -- 172.21.15.42:0/785703704 shutdown_connections 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/785703704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec8068580 0x7f0ec80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/785703704 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec81070b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 -- 172.21.15.42:0/785703704 >> 172.21.15.42:0/785703704 conn(0x7f0ec8075670 msgr2=0x7f0ec8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 -- 172.21.15.42:0/785703704 shutdown_connections 2024-09-16T23:45:09.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.752+0000 7f0ecda8d700 1 -- 172.21.15.42:0/785703704 wait complete. 2024-09-16T23:45:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ecda8d700 1 Processor -- start 2024-09-16T23:45:09.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ecda8d700 1 -- start start 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ecda8d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec8068580 0x7f0ec819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ecda8d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ecda8d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec819f410 con 0x7f0ec8106cd0 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ecda8d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec81988c0 con 0x7f0ec8068580 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ebeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ebeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56934/0 (socket says 172.21.15.42:56934) 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ebeffd700 1 -- 172.21.15.42:0/1406914671 learned_addr learned my addr 172.21.15.42:0/1406914671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.753+0000 7f0ec77fe700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec8068580 0x7f0ec819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:09.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ebeffd700 1 -- 172.21.15.42:0/1406914671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec8068580 msgr2=0x7f0ec819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ebeffd700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec8068580 0x7f0ec819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ebeffd700 1 -- 172.21.15.42:0/1406914671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80095f0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ebeffd700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec819ed80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000d8d0 tx=0x7f0eb000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ec57fa700 1 -- 172.21.15.42:0/1406914671 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb00098a0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec8198ba0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ec57fa700 1 -- 172.21.15.42:0/1406914671 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0eb0010460 con 0x7f0ec8106cd0 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ec57fa700 1 -- 172.21.15.42:0/1406914671 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eb000f5f0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.754+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec81990f0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.755+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec804f2e0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.756+0000 7f0ec57fa700 1 -- 172.21.15.42:0/1406914671 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0eb0009a00 con 0x7f0ec8106cd0 2024-09-16T23:45:09.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.756+0000 7f0ec57fa700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0eb406c4e0 0x7f0eb406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:09.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.756+0000 7f0ec57fa700 1 -- 172.21.15.42:0/1406914671 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f0eb008b210 con 0x7f0ec8106cd0 2024-09-16T23:45:09.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.756+0000 7f0ec77fe700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0eb406c4e0 0x7f0eb406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:09.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.756+0000 7f0ec77fe700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0eb406c4e0 0x7f0eb406e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800b5c0 tx=0x7f0eb801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:09.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.761+0000 7f0ec57fa700 1 -- 172.21.15.42:0/1406914671 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0eb00598e0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.961+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ec8199dd0 con 0x7f0ec8106cd0 2024-09-16T23:45:09.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.962+0000 7f0ec57fa700 1 -- 172.21.15.42:0/1406914671 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11010 (secure 0 0 0) 0x7f0eb0059470 con 0x7f0ec8106cd0 2024-09-16T23:45:09.960 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:45:09.960 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":40,"fsid":"c2e6f2e2-7484-11ef-bceb-c7b262605968","created":"2024-09-16T23:39:01.224551+0000","modified":"2024-09-16T23:44:50.615829+0000","last_up_change":"2024-09-16T23:44:49.610877+0000","last_in_change":"2024-09-16T23:44:30.700582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T23:43:31.584431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"82c23a1b-b658-4424-9c3b-f324a6b0861b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6803","nonce":728536688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6805","nonce":728536688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6809","nonce":728536688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":728536688},{"type":"v1","addr":"172.21.15.42:6807","nonce":728536688}]},"public_addr":"172.21.15.42:6803/728536688","cluster_addr":"172.21.15.42:6805/728536688","heartbeat_back_addr":"172.21.15.42:6809/728536688","heartbeat_front_addr":"172.21.15.42:6807/728536688","state":["exists","up"]},{"osd":1,"uuid":"644cf390-4e6f-402f-92ff-dcd7330ee305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6811","nonce":1956379932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6813","nonce":1956379932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6817","nonce":1956379932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1956379932},{"type":"v1","addr":"172.21.15.42:6815","nonce":1956379932}]},"public_addr":"172.21.15.42:6811/1956379932","cluster_addr":"172.21.15.42:6813/1956379932","heartbeat_back_addr":"172.21.15.42:6817/1956379932","heartbeat_front_addr":"172.21.15.42:6815/1956379932","state":["exists","up"]},{"osd":2,"uuid":"a27ed11a-999c-439e-b098-3381127c72e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6819","nonce":3083836466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6821","nonce":3083836466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6825","nonce":3083836466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":3083836466},{"type":"v1","addr":"172.21.15.42:6823","nonce":3083836466}]},"public_addr":"172.21.15.42:6819/3083836466","cluster_addr":"172.21.15.42:6821/3083836466","heartbeat_back_addr":"172.21.15.42:6825/3083836466","heartbeat_front_addr":"172.21.15.42:6823/3083836466","state":["exists","up"]},{"osd":3,"uuid":"2b84bdfd-14ef-4867-a5d0-b443a5dcec12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6801","nonce":3558322430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6803","nonce":3558322430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6807","nonce":3558322430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":3558322430},{"type":"v1","addr":"172.21.15.43:6805","nonce":3558322430}]},"public_addr":"172.21.15.43:6801/3558322430","cluster_addr":"172.21.15.43:6803/3558322430","heartbeat_back_addr":"172.21.15.43:6807/3558322430","heartbeat_front_addr":"172.21.15.43:6805/3558322430","state":["exists","up"]},{"osd":4,"uuid":"093b60b2-f2fd-4261-b038-4400c2a76180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6809","nonce":3817520749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6811","nonce":3817520749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6815","nonce":3817520749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":3817520749},{"type":"v1","addr":"172.21.15.43:6813","nonce":3817520749}]},"public_addr":"172.21.15.43:6809/3817520749","cluster_addr":"172.21.15.43:6811/3817520749","heartbeat_back_addr":"172.21.15.43:6815/3817520749","heartbeat_front_addr":"172.21.15.43:6813/3817520749","state":["exists","up"]},{"osd":5,"uuid":"6e51d900-ce2d-4cb7-894b-aa8d04938cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6817","nonce":205646323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6819","nonce":205646323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6823","nonce":205646323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":205646323},{"type":"v1","addr":"172.21.15.43:6821","nonce":205646323}]},"public_addr":"172.21.15.43:6817/205646323","cluster_addr":"172.21.15.43:6819/205646323","heartbeat_back_addr":"172.21.15.43:6823/205646323","heartbeat_front_addr":"172.21.15.43:6821/205646323","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:42:32.529981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:01.516391+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:28.822541+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:43:54.549222+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:44:21.118416+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T23:44:48.359630+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/241787870":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/2316861404":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/4154448068":"2024-09-17T23:41:09.064350+0000","172.21.15.42:0/3709834403":"2024-09-17T23:40:02.177376+0000","172.21.15.42:0/1983510372":"2024-09-17T23:40:02.177376+0000","172.21.15.42:6801/2":"2024-09-17T23:39:29.178241+0000","172.21.15.42:6800/2":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/911608840":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/585424460":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/1732693245":"2024-09-17T23:39:29.178241+0000","172.21.15.42:0/3534532679":"2024-09-17T23:40:02.177376+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0eb406c4e0 msgr2=0x7f0eb406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0eb406c4e0 0x7f0eb406e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800b5c0 tx=0x7f0eb801a040 comp rx=0 tx=0).stop 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 msgr2=0x7f0ec819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec819ed80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000d8d0 tx=0x7f0eb000dbe0 comp rx=0 tx=0).stop 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 shutdown_connections 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0eb406c4e0 0x7f0eb406e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec8068580 0x7f0ec819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 --2- 172.21.15.42:0/1406914671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0ec8106cd0 0x7f0ec819ed80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.964+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 >> 172.21.15.42:0/1406914671 conn(0x7f0ec8075670 msgr2=0x7f0ec80fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:09.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.965+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 shutdown_connections 2024-09-16T23:45:09.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:09.965+0000 7f0ecda8d700 1 -- 172.21.15.42:0/1406914671 wait complete. 2024-09-16T23:45:10.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:10.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:10 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1406914671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T23:45:10.345 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:10.345 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:10 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1406914671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T23:45:10.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T23:45:11.038 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T23:45:11.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T23:45:11.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T23:45:11.039 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T23:45:11.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T23:45:11.317 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:11.319 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:11.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:11 smithi043 ceph-mon[35191]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:11.404 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:11 smithi042 ceph-mon[27914]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:12.791 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:12.791 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:12.792 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:12.792 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:13.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:13 smithi042 ceph-mon[27914]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:13.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:13 smithi043 ceph-mon[35191]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:15.184 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:15 smithi042 ceph-mon[27914]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 -- 172.21.15.42:0/3450316208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 msgr2=0x7fb49c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 --2- 172.21.15.42:0/3450316208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c100080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009910 tx=0x7fb48c009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 -- 172.21.15.42:0/3450316208 shutdown_connections 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 --2- 172.21.15.42:0/3450316208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c100080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 --2- 172.21.15.42:0/3450316208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb49c105bd0 0x7fb49c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 -- 172.21.15.42:0/3450316208 >> 172.21.15.42:0/3450316208 conn(0x7fb49c075670 msgr2=0x7fb49c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 -- 172.21.15.42:0/3450316208 shutdown_connections 2024-09-16T23:45:15.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.301+0000 7fb4a36be700 1 -- 172.21.15.42:0/3450316208 wait complete. 2024-09-16T23:45:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a36be700 1 Processor -- start 2024-09-16T23:45:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a36be700 1 -- start start 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a36be700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c19c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a36be700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb49c105bd0 0x7fb49c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a36be700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c19d2d0 con 0x7fb49c0ffc20 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a36be700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c196790 con 0x7fb49c105bd0 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a145a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c19c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a145a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c19c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50066/0 (socket says 172.21.15.42:50066) 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.302+0000 7fb4a145a700 1 -- 172.21.15.42:0/1570557940 learned_addr learned my addr 172.21.15.42:0/1570557940 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a0c59700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb49c105bd0 0x7fb49c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a145a700 1 -- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb49c105bd0 msgr2=0x7fb49c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:15.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a145a700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb49c105bd0 0x7fb49c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a145a700 1 -- 172.21.15.42:0/1570557940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0095f0 con 0x7fb49c0ffc20 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a0c59700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb49c105bd0 0x7fb49c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a145a700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c19c6b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b4b0 tx=0x7fb49800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb498011800 con 0x7fb49c0ffc20 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb498011e10 con 0x7fb49c0ffc20 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c196a70 con 0x7fb49c0ffc20 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49800f2e0 con 0x7fb49c0ffc20 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.303+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c196fc0 con 0x7fb49c0ffc20 2024-09-16T23:45:15.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.304+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb49c10d020 con 0x7fb49c0ffc20 2024-09-16T23:45:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.305+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb498010380 con 0x7fb49c0ffc20 2024-09-16T23:45:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.305+0000 7fb4927fc700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb48806c4e0 0x7fb48806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.305+0000 7fb4a0c59700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb48806c4e0 0x7fb48806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.305+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fb49808afb0 con 0x7fb49c0ffc20 2024-09-16T23:45:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.306+0000 7fb4927fc700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] conn(0x7fb488072080 0x7fb4880744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.306+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 --> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb488074b50 con 0x7fb488072080 2024-09-16T23:45:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.306+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb49808d680 con 0x7fb49c0ffc20 2024-09-16T23:45:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.306+0000 7fb4a0c59700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb48806c4e0 0x7fb48806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb48c006010 tx=0x7fb48c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.306+0000 7fb4a1c5b700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] conn(0x7fb488072080 0x7fb4880744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.306+0000 7fb4a1c5b700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] conn(0x7fb488072080 0x7fb4880744a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.307+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== osd.5 v2:172.21.15.43:6816/205646323 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb488074b50 con 0x7fb488072080 2024-09-16T23:45:15.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.323+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 --> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb49c04f2e0 con 0x7fb488072080 2024-09-16T23:45:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.323+0000 7fb4927fc700 1 -- 172.21.15.42:0/1570557940 <== osd.5 v2:172.21.15.43:6816/205646323 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb49c04f2e0 con 0x7fb488072080 2024-09-16T23:45:15.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] conn(0x7fb488072080 msgr2=0x7fb4880744a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] conn(0x7fb488072080 0x7fb4880744a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb48806c4e0 msgr2=0x7fb48806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb48806c4e0 0x7fb48806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb48c006010 tx=0x7fb48c00b540 comp rx=0 tx=0).stop 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 msgr2=0x7fb49c19c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c19c6b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b4b0 tx=0x7fb49800b870 comp rx=0 tx=0).stop 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 shutdown_connections 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:6816/205646323,v1:172.21.15.43:6817/205646323] conn(0x7fb488072080 0x7fb4880744a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb48806c4e0 0x7fb48806e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb49c0ffc20 0x7fb49c19c6b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 --2- 172.21.15.42:0/1570557940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb49c105bd0 0x7fb49c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 >> 172.21.15.42:0/1570557940 conn(0x7fb49c075670 msgr2=0x7fb49c068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 shutdown_connections 2024-09-16T23:45:15.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:15.324+0000 7fb4a36be700 1 -- 172.21.15.42:0/1570557940 wait complete. 2024-09-16T23:45:15.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:15 smithi043 ceph-mon[35191]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:167503724551 2024-09-16T23:45:17.204 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T23:45:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:17 smithi043 ceph-mon[35191]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:17.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:17 smithi042 ceph-mon[27914]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:17.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 -- 172.21.15.42:0/3860033706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c100750 msgr2=0x7f3d1c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:17.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 --2- 172.21.15.42:0/3860033706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c100750 0x7f3d1c1047a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009910 tx=0x7f3d04009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:17.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 -- 172.21.15.42:0/3860033706 shutdown_connections 2024-09-16T23:45:17.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 --2- 172.21.15.42:0/3860033706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c100750 0x7f3d1c1047a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 --2- 172.21.15.42:0/3860033706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c0ffda0 0x7f3d1c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 -- 172.21.15.42:0/3860033706 >> 172.21.15.42:0/3860033706 conn(0x7f3d1c0fb650 msgr2=0x7f3d1c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 -- 172.21.15.42:0/3860033706 shutdown_connections 2024-09-16T23:45:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.874+0000 7f3d218fe700 1 -- 172.21.15.42:0/3860033706 wait complete. 2024-09-16T23:45:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d218fe700 1 Processor -- start 2024-09-16T23:45:17.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d218fe700 1 -- start start 2024-09-16T23:45:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d218fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c0ffda0 0x7f3d1c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d218fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c100750 0x7f3d1c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d218fe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c19f4e0 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d218fe700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1c198990 con 0x7f3d1c100750 2024-09-16T23:45:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d1affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c0ffda0 0x7f3d1c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d1affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c0ffda0 0x7f3d1c19e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50084/0 (socket says 172.21.15.42:50084) 2024-09-16T23:45:17.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.875+0000 7f3d1affd700 1 -- 172.21.15.42:0/2435475697 learned_addr learned my addr 172.21.15.42:0/2435475697 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d1a7fc700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c100750 0x7f3d1c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d1affd700 1 -- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c100750 msgr2=0x7f3d1c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d1affd700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c100750 0x7f3d1c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d1affd700 1 -- 172.21.15.42:0/2435475697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d040095f0 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d1a7fc700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c100750 0x7f3d1c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d1affd700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c0ffda0 0x7f3d1c19e910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00b4b0 tx=0x7f3d0c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0c011800 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d0c011e10 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c198c70 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0c00f2e0 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.876+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1c1991c0 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.877+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d1c10d040 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.878+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f3d0c010380 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.878+0000 7f3d208fc700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f3d0806c4e0 0x7f3d0806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.878+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f3d0c08af80 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.878+0000 7f3d1a7fc700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f3d0806c4e0 0x7f3d0806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.878+0000 7f3d208fc700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] conn(0x7f3d08072080 0x7f3d080744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.878+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 --> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d08074b50 con 0x7f3d08072080 2024-09-16T23:45:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.878+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d0c08d650 con 0x7f3d1c0ffda0 2024-09-16T23:45:17.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.879+0000 7f3d1a7fc700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f3d0806c4e0 0x7f3d0806e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400b5c0 tx=0x7f3d04005480 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.879+0000 7f3d1b7fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] conn(0x7f3d08072080 0x7f3d080744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.879+0000 7f3d1b7fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] conn(0x7f3d08072080 0x7f3d080744a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:17.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.880+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== osd.3 v2:172.21.15.43:6800/3558322430 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3d08074b50 con 0x7f3d08072080 2024-09-16T23:45:17.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.901+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 --> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d1c1996d0 con 0x7f3d08072080 2024-09-16T23:45:17.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d208fc700 1 -- 172.21.15.42:0/2435475697 <== osd.3 v2:172.21.15.43:6800/3558322430 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3d1c1996d0 con 0x7f3d08072080 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] conn(0x7f3d08072080 msgr2=0x7f3d080744a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] conn(0x7f3d08072080 0x7f3d080744a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f3d0806c4e0 msgr2=0x7f3d0806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f3d0806c4e0 0x7f3d0806e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400b5c0 tx=0x7f3d04005480 comp rx=0 tx=0).stop 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c0ffda0 msgr2=0x7f3d1c19e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c0ffda0 0x7f3d1c19e910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00b4b0 tx=0x7f3d0c00b870 comp rx=0 tx=0).stop 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 shutdown_connections 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f3d0806c4e0 0x7f3d0806e9a0 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400b5c0 tx=0x7f3d04005480 comp rx=0 tx=0).stop 2024-09-16T23:45:17.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:6800/3558322430,v1:172.21.15.43:6801/3558322430] conn(0x7f3d08072080 0x7f3d080744a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:17.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d1c0ffda0 0x7f3d1c19e910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:17.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 --2- 172.21.15.42:0/2435475697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d1c100750 0x7f3d1c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:17.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.902+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 >> 172.21.15.42:0/2435475697 conn(0x7f3d1c0fb650 msgr2=0x7f3d1c0696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:17.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.903+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 shutdown_connections 2024-09-16T23:45:17.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:17.903+0000 7f3d218fe700 1 -- 172.21.15.42:0/2435475697 wait complete. 2024-09-16T23:45:19.251 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:19 smithi042 ceph-mon[27914]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:19.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:19 smithi043 ceph-mon[35191]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:21.117 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:21 smithi042 ceph-mon[27914]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 -- 172.21.15.42:0/4063343986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa448105bd0 msgr2=0x7fa448105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/4063343986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa448105bd0 0x7fa448105fb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa438009910 tx=0x7fa438009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 -- 172.21.15.42:0/4063343986 shutdown_connections 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/4063343986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4480ffc20 0x7fa448100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/4063343986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa448105bd0 0x7fa448105fb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 -- 172.21.15.42:0/4063343986 >> 172.21.15.42:0/4063343986 conn(0x7fa448075670 msgr2=0x7fa448075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 -- 172.21.15.42:0/4063343986 shutdown_connections 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.132+0000 7fa44e0f2700 1 -- 172.21.15.42:0/4063343986 wait complete. 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa44e0f2700 1 Processor -- start 2024-09-16T23:45:21.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa44e0f2700 1 -- start start 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa44e0f2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4480ffc20 0x7fa44819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa44e0f2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa448105bd0 0x7fa44819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa44e0f2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa448195590 con 0x7fa4480ffc20 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa44e0f2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa448195700 con 0x7fa448105bd0 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa4477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4480ffc20 0x7fa44819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa4477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4480ffc20 0x7fa44819b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50100/0 (socket says 172.21.15.42:50100) 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.133+0000 7fa4477fe700 1 -- 172.21.15.42:0/1027077644 learned_addr learned my addr 172.21.15.42:0/1027077644 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa43effd700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa448105bd0 0x7fa44819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa4477fe700 1 -- 172.21.15.42:0/1027077644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa448105bd0 msgr2=0x7fa44819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa4477fe700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa448105bd0 0x7fa44819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa4477fe700 1 -- 172.21.15.42:0/1027077644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380095f0 con 0x7fa4480ffc20 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa4477fe700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4480ffc20 0x7fa44819b5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa438006010 tx=0x7fa43800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa43801d070 con 0x7fa4480ffc20 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa448195980 con 0x7fa4480ffc20 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa43800f460 con 0x7fa4480ffc20 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa438021640 con 0x7fa4480ffc20 2024-09-16T23:45:21.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.134+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa448195e70 con 0x7fa4480ffc20 2024-09-16T23:45:21.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.135+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa44810cfa0 con 0x7fa4480ffc20 2024-09-16T23:45:21.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.136+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa43800faf0 con 0x7fa4480ffc20 2024-09-16T23:45:21.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.136+0000 7fa4457fa700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa43406c4e0 0x7fa43406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.137+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fa43808d300 con 0x7fa4480ffc20 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.137+0000 7fa43effd700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa43406c4e0 0x7fa43406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.137+0000 7fa4457fa700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] conn(0x7fa434072080 0x7fa4340744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.137+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 --> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa434074b50 con 0x7fa434072080 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.137+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa43808d760 con 0x7fa4480ffc20 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.137+0000 7fa447fff700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] conn(0x7fa434072080 0x7fa4340744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.137+0000 7fa43effd700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa43406c4e0 0x7fa43406e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa448196e30 tx=0x7fa430005cf0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.138+0000 7fa447fff700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] conn(0x7fa434072080 0x7fa4340744a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:21.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.138+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== osd.2 v2:172.21.15.42:6818/3083836466 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa434074b50 con 0x7fa434072080 2024-09-16T23:45:21.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.154+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 --> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa44804f2e0 con 0x7fa434072080 2024-09-16T23:45:21.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.154+0000 7fa4457fa700 1 -- 172.21.15.42:0/1027077644 <== osd.2 v2:172.21.15.42:6818/3083836466 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa44804f2e0 con 0x7fa434072080 2024-09-16T23:45:21.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] conn(0x7fa434072080 msgr2=0x7fa4340744a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] conn(0x7fa434072080 0x7fa4340744a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa43406c4e0 msgr2=0x7fa43406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa43406c4e0 0x7fa43406e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa448196e30 tx=0x7fa430005cf0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4480ffc20 msgr2=0x7fa44819b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4480ffc20 0x7fa44819b5a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa438006010 tx=0x7fa43800be20 comp rx=0 tx=0).stop 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 shutdown_connections 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6818/3083836466,v1:172.21.15.42:6819/3083836466] conn(0x7fa434072080 0x7fa4340744a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa43406c4e0 0x7fa43406e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4480ffc20 0x7fa44819b5a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 --2- 172.21.15.42:0/1027077644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa448105bd0 0x7fa44819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 >> 172.21.15.42:0/1027077644 conn(0x7fa448075670 msgr2=0x7fa4480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 shutdown_connections 2024-09-16T23:45:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.155+0000 7fa44e0f2700 1 -- 172.21.15.42:0/1027077644 wait complete. 2024-09-16T23:45:21.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.340+0000 7f362cb25700 1 -- 172.21.15.42:0/2318542321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628068580 msgr2=0x7f36280689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.340+0000 7f362cb25700 1 --2- 172.21.15.42:0/2318542321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628068580 0x7f36280689e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f361c009910 tx=0x7f361c009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:21.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.341+0000 7f362cb25700 1 -- 172.21.15.42:0/2318542321 shutdown_connections 2024-09-16T23:45:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.341+0000 7f362cb25700 1 --2- 172.21.15.42:0/2318542321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628068580 0x7f36280689e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.341+0000 7f362cb25700 1 --2- 172.21.15.42:0/2318542321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3628106cd0 0x7f36281070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.341+0000 7f362cb25700 1 -- 172.21.15.42:0/2318542321 >> 172.21.15.42:0/2318542321 conn(0x7f3628075670 msgr2=0x7f3628075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.341+0000 7f362cb25700 1 -- 172.21.15.42:0/2318542321 shutdown_connections 2024-09-16T23:45:21.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.341+0000 7f362cb25700 1 -- 172.21.15.42:0/2318542321 wait complete. 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f362cb25700 1 Processor -- start 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f362cb25700 1 -- start start 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f362cb25700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3628068580 0x7f36280ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f362cb25700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628106cd0 0x7f3628100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f362cb25700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3628100770 con 0x7f3628106cd0 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f362cb25700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281008b0 con 0x7f3628068580 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f3625d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628106cd0 0x7f3628100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f3625d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628106cd0 0x7f3628100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50116/0 (socket says 172.21.15.42:50116) 2024-09-16T23:45:21.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f3625d9b700 1 -- 172.21.15.42:0/2036658945 learned_addr learned my addr 172.21.15.42:0/2036658945 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:21.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f362659c700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3628068580 0x7f36280ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f3625d9b700 1 -- 172.21.15.42:0/2036658945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3628068580 msgr2=0x7f36280ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f3625d9b700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3628068580 0x7f36280ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.342+0000 7f3625d9b700 1 -- 172.21.15.42:0/2036658945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0095f0 con 0x7f3628106cd0 2024-09-16T23:45:21.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.343+0000 7f3625d9b700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628106cd0 0x7f3628100230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f361c006010 tx=0x7f361c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:21.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.343+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c01d070 con 0x7f3628106cd0 2024-09-16T23:45:21.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.343+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f361c004fa0 con 0x7f3628106cd0 2024-09-16T23:45:21.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.343+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3628100b30 con 0x7f3628106cd0 2024-09-16T23:45:21.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.343+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361c00f690 con 0x7f3628106cd0 2024-09-16T23:45:21.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.343+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36281a2d60 con 0x7f3628106cd0 2024-09-16T23:45:21.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.344+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f362810e0a0 con 0x7f3628106cd0 2024-09-16T23:45:21.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.344+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f361c00f7f0 con 0x7f3628106cd0 2024-09-16T23:45:21.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.345+0000 7f36177fe700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f361006c4e0 0x7f361006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.345+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f361c08cb80 con 0x7f3628106cd0 2024-09-16T23:45:21.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.345+0000 7f362659c700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f361006c4e0 0x7f361006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.345+0000 7f36177fe700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] conn(0x7f3610072080 0x7f36100744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:21.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.345+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 --> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3610074b50 con 0x7f3610072080 2024-09-16T23:45:21.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.345+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f361c092260 con 0x7f3628106cd0 2024-09-16T23:45:21.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.345+0000 7f3626d9d700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] conn(0x7f3610072080 0x7f36100744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:21.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.346+0000 7f362659c700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f361006c4e0 0x7f361006e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3618005950 tx=0x7f361800a300 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:21.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.346+0000 7f3626d9d700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] conn(0x7f3610072080 0x7f36100744a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:21.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.346+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== osd.0 v2:172.21.15.42:6802/728536688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3610074b50 con 0x7f3610072080 2024-09-16T23:45:21.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.362+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 --> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f362804f2e0 con 0x7f3610072080 2024-09-16T23:45:21.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.362+0000 7f36177fe700 1 -- 172.21.15.42:0/2036658945 <== osd.0 v2:172.21.15.42:6802/728536688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f362804f2e0 con 0x7f3610072080 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] conn(0x7f3610072080 msgr2=0x7f36100744a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] conn(0x7f3610072080 0x7f36100744a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f361006c4e0 msgr2=0x7f361006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f361006c4e0 0x7f361006e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3618005950 tx=0x7f361800a300 comp rx=0 tx=0).stop 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628106cd0 msgr2=0x7f3628100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628106cd0 0x7f3628100230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f361c006010 tx=0x7f361c00bd70 comp rx=0 tx=0).stop 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 shutdown_connections 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6802/728536688,v1:172.21.15.42:6803/728536688] conn(0x7f3610072080 0x7f36100744a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f361006c4e0 0x7f361006e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3628068580 0x7f36280ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 --2- 172.21.15.42:0/2036658945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3628106cd0 0x7f3628100230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:21.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 >> 172.21.15.42:0/2036658945 conn(0x7f3628075670 msgr2=0x7f36280feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:21.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 shutdown_connections 2024-09-16T23:45:21.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:21.363+0000 7f362cb25700 1 -- 172.21.15.42:0/2036658945 wait complete. 2024-09-16T23:45:21.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:21 smithi043 ceph-mon[35191]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:22.110 INFO:teuthology.orchestra.run.smithi042.stdout:81604378648 2024-09-16T23:45:22.110 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T23:45:22.492 INFO:teuthology.orchestra.run.smithi042.stdout:111669149715 2024-09-16T23:45:22.493 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T23:45:23.100 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:23 smithi042 ceph-mon[27914]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:23.121 INFO:teuthology.orchestra.run.smithi042.stdout:38654705701 2024-09-16T23:45:23.121 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T23:45:23.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:23 smithi043 ceph-mon[35191]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 -- 172.21.15.42:0/1478863908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 msgr2=0x7f2554105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 --2- 172.21.15.42:0/1478863908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f2554105840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2548009910 tx=0x7f2548009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 -- 172.21.15.42:0/1478863908 shutdown_connections 2024-09-16T23:45:23.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 --2- 172.21.15.42:0/1478863908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f2554105840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 --2- 172.21.15.42:0/1478863908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25540ff5a0 0x7f25540ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 -- 172.21.15.42:0/1478863908 >> 172.21.15.42:0/1478863908 conn(0x7f2554074e30 msgr2=0x7f2554075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 -- 172.21.15.42:0/1478863908 shutdown_connections 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.679+0000 7f2559152700 1 -- 172.21.15.42:0/1478863908 wait complete. 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f2559152700 1 Processor -- start 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f2559152700 1 -- start start 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f2559152700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25540ff5a0 0x7f255419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f2559152700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f255419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f2559152700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255419d2c0 con 0x7f25540fff50 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f2559152700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2554196770 con 0x7f25540ff5a0 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f255259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f255419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f255259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f255419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55020/0 (socket says 172.21.15.42:55020) 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f255259c700 1 -- 172.21.15.42:0/2476723052 learned_addr learned my addr 172.21.15.42:0/2476723052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f2552d9d700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25540ff5a0 0x7f255419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.680+0000 7f255259c700 1 -- 172.21.15.42:0/2476723052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25540ff5a0 msgr2=0x7f255419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f255259c700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25540ff5a0 0x7f255419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f255259c700 1 -- 172.21.15.42:0/2476723052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25480095f0 con 0x7f25540fff50 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f2552d9d700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25540ff5a0 0x7f255419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f255259c700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f255419cc30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f25480092e0 tx=0x7f254800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254801d070 con 0x7f25540fff50 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25541969f0 con 0x7f25540fff50 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2548022470 con 0x7f25540fff50 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254800f690 con 0x7f25540fff50 2024-09-16T23:45:23.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.681+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2554196ee0 con 0x7f25540fff50 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.682+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f255410e130 con 0x7f25540fff50 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.682+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f25480225e0 con 0x7f25540fff50 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.683+0000 7f253bfff700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f253c06c4e0 0x7f253c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.683+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f254808d230 con 0x7f25540fff50 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.683+0000 7f2552d9d700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f253c06c4e0 0x7f253c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.683+0000 7f253bfff700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] conn(0x7f253c072080 0x7f253c0744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.683+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 --> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f253c074b50 con 0x7f253c072080 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.683+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f254808d690 con 0x7f25540fff50 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.683+0000 7f255359e700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] conn(0x7f253c072080 0x7f253c0744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.684+0000 7f2552d9d700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f253c06c4e0 0x7f253c06e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2544007900 tx=0x7f2544008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.684+0000 7f255359e700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] conn(0x7f253c072080 0x7f253c0744a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:23.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.684+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== osd.1 v2:172.21.15.42:6810/1956379932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f253c074b50 con 0x7f253c072080 2024-09-16T23:45:23.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 --> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f255404f2e0 con 0x7f253c072080 2024-09-16T23:45:23.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f253bfff700 1 -- 172.21.15.42:0/2476723052 <== osd.1 v2:172.21.15.42:6810/1956379932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f255404f2e0 con 0x7f253c072080 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] conn(0x7f253c072080 msgr2=0x7f253c0744a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] conn(0x7f253c072080 0x7f253c0744a0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f253c06c4e0 msgr2=0x7f253c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f253c06c4e0 0x7f253c06e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2544007900 tx=0x7f2544008040 comp rx=0 tx=0).stop 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 msgr2=0x7f255419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f255419cc30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f25480092e0 tx=0x7f254800be20 comp rx=0 tx=0).stop 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 shutdown_connections 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6810/1956379932,v1:172.21.15.42:6811/1956379932] conn(0x7f253c072080 0x7f253c0744a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f253c06c4e0 0x7f253c06e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25540ff5a0 0x7f255419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 --2- 172.21.15.42:0/2476723052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25540fff50 0x7f255419cc30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 >> 172.21.15.42:0/2476723052 conn(0x7f2554074e30 msgr2=0x7f25540fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 shutdown_connections 2024-09-16T23:45:23.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:23.700+0000 7f2559152700 1 -- 172.21.15.42:0/2476723052 wait complete. 2024-09-16T23:45:23.711 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 -- 172.21.15.42:0/3085903286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc0ffc20 msgr2=0x7f2fbc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/3085903286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc0ffc20 0x7f2fbc100080 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009910 tx=0x7f2fb0009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 -- 172.21.15.42:0/3085903286 shutdown_connections 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/3085903286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc0ffc20 0x7f2fbc100080 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/3085903286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fbc105bd0 0x7f2fbc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 -- 172.21.15.42:0/3085903286 >> 172.21.15.42:0/3085903286 conn(0x7f2fbc075670 msgr2=0x7f2fbc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 -- 172.21.15.42:0/3085903286 shutdown_connections 2024-09-16T23:45:24.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.132+0000 7f2fc08ab700 1 -- 172.21.15.42:0/3085903286 wait complete. 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.133+0000 7f2fc08ab700 1 Processor -- start 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.133+0000 7f2fc08ab700 1 -- start start 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.133+0000 7f2fc08ab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fbc0ffc20 0x7f2fbc197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.133+0000 7f2fc08ab700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc105bd0 0x7f2fbc197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.133+0000 7f2fc08ab700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc198380 con 0x7f2fbc105bd0 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.133+0000 7f2fc08ab700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc1984f0 con 0x7f2fbc0ffc20 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fb9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc105bd0 0x7f2fbc197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fb9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc105bd0 0x7f2fbc197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55034/0 (socket says 172.21.15.42:55034) 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fb9d9b700 1 -- 172.21.15.42:0/888898901 learned_addr learned my addr 172.21.15.42:0/888898901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fba59c700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fbc0ffc20 0x7f2fbc197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fb9d9b700 1 -- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fbc0ffc20 msgr2=0x7f2fbc197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:24.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fb9d9b700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fbc0ffc20 0x7f2fbc197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fb9d9b700 1 -- 172.21.15.42:0/888898901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00095f0 con 0x7f2fbc105bd0 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fb9d9b700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc105bd0 0x7f2fbc197ca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006010 tx=0x7f2fb000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fb001d070 con 0x7f2fbc105bd0 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2fb000f460 con 0x7f2fbc105bd0 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc1a07d0 con 0x7f2fbc105bd0 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fb0017630 con 0x7f2fbc105bd0 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.134+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc1a0c40 con 0x7f2fbc105bd0 2024-09-16T23:45:24.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.135+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fbc10cfa0 con 0x7f2fbc105bd0 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.136+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2fb000f5d0 con 0x7f2fbc105bd0 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.136+0000 7f2fab7fe700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2fa406c4e0 0x7f2fa406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.136+0000 7f2fba59c700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2fa406c4e0 0x7f2fa406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.136+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f2fb008cd70 con 0x7f2fbc105bd0 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.136+0000 7f2fab7fe700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] conn(0x7f2fa4072080 0x7f2fa40744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.136+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 --> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fa4074b50 con 0x7f2fa4072080 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.136+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fb008d1d0 con 0x7f2fbc105bd0 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.137+0000 7f2fba59c700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2fa406c4e0 0x7f2fa406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005950 tx=0x7f2fac00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.137+0000 7f2fbad9d700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] conn(0x7f2fa4072080 0x7f2fa40744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:24.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.137+0000 7f2fbad9d700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] conn(0x7f2fa4072080 0x7f2fa40744a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:24.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.138+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== osd.4 v2:172.21.15.43:6808/3817520749 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2fa4074b50 con 0x7f2fa4072080 2024-09-16T23:45:24.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.156+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 --> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2fbc04f2e0 con 0x7f2fa4072080 2024-09-16T23:45:24.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.156+0000 7f2fab7fe700 1 -- 172.21.15.42:0/888898901 <== osd.4 v2:172.21.15.43:6808/3817520749 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2fbc04f2e0 con 0x7f2fa4072080 2024-09-16T23:45:24.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] conn(0x7f2fa4072080 msgr2=0x7f2fa40744a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] conn(0x7f2fa4072080 0x7f2fa40744a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2fa406c4e0 msgr2=0x7f2fa406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2fa406c4e0 0x7f2fa406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005950 tx=0x7f2fac00b410 comp rx=0 tx=0).stop 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc105bd0 msgr2=0x7f2fbc197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc105bd0 0x7f2fbc197ca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006010 tx=0x7f2fb000bd70 comp rx=0 tx=0).stop 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 shutdown_connections 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:6808/3817520749,v1:172.21.15.43:6809/3817520749] conn(0x7f2fa4072080 0x7f2fa40744a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2fa406c4e0 0x7f2fa406e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fbc0ffc20 0x7f2fbc197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 --2- 172.21.15.42:0/888898901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fbc105bd0 0x7f2fbc197ca0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 >> 172.21.15.42:0/888898901 conn(0x7f2fbc075670 msgr2=0x7f2fbc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:24.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 shutdown_connections 2024-09-16T23:45:24.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:24.157+0000 7f2fc08ab700 1 -- 172.21.15.42:0/888898901 wait complete. 2024-09-16T23:45:24.409 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:24.410 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:25.252 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:25 smithi042 ceph-mon[27914]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:25.253 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:25.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:25 smithi043 ceph-mon[35191]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:25.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:25 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:26.303 INFO:teuthology.orchestra.run.smithi042.stdout:60129542175 2024-09-16T23:45:26.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T23:45:26.684 INFO:teuthology.orchestra.run.smithi042.stdout:146028888079 2024-09-16T23:45:26.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T23:45:27.232 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:27 smithi042 ceph-mon[27914]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:27.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:27 smithi043 ceph-mon[35191]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:29.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:29 smithi042 ceph-mon[27914]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 -- 172.21.15.42:0/4277337376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04101850 msgr2=0x7fbd041058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 --2- 172.21.15.42:0/4277337376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04101850 0x7fbd041058a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009910 tx=0x7fbcf4009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:29.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 -- 172.21.15.42:0/4277337376 shutdown_connections 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 --2- 172.21.15.42:0/4277337376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04101850 0x7fbd041058a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 --2- 172.21.15.42:0/4277337376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04100ea0 0x7fbd04101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 -- 172.21.15.42:0/4277337376 >> 172.21.15.42:0/4277337376 conn(0x7fbd040fc750 msgr2=0x7fbd040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 -- 172.21.15.42:0/4277337376 shutdown_connections 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.290+0000 7fbd0b900700 1 -- 172.21.15.42:0/4277337376 wait complete. 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0b900700 1 Processor -- start 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0b900700 1 -- start start 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0b900700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0b900700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0b900700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0419d2d0 con 0x7fbd04100ea0 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0b900700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd04196780 con 0x7fbd04101850 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0969c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0969c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55050/0 (socket says 172.21.15.42:55050) 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0969c700 1 -- 172.21.15.42:0/1852251930 learned_addr learned my addr 172.21.15.42:0/1852251930 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd08e9b700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0969c700 1 -- 172.21.15.42:0/1852251930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 msgr2=0x7fbd0419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0969c700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0969c700 1 -- 172.21.15.42:0/1852251930 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40095f0 con 0x7fbd04100ea0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.291+0000 7fbd0969c700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d8d0 tx=0x7fbd0000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.292+0000 7fbcfa7fc700 1 -- 172.21.15.42:0/1852251930 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd000098a0 con 0x7fbd04100ea0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.292+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd04196a60 con 0x7fbd04100ea0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.292+0000 7fbcfa7fc700 1 -- 172.21.15.42:0/1852251930 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbd00010460 con 0x7fbd04100ea0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.292+0000 7fbcfa7fc700 1 -- 172.21.15.42:0/1852251930 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0000f5f0 con 0x7fbd04100ea0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.292+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd04196fb0 con 0x7fbd04100ea0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.293+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd041092a0 con 0x7fbd04100ea0 2024-09-16T23:45:29.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.293+0000 7fbcfa7fc700 1 -- 172.21.15.42:0/1852251930 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbd00009a00 con 0x7fbd04100ea0 2024-09-16T23:45:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.294+0000 7fbcfa7fc700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbcf006c4e0 0x7fbcf006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.294+0000 7fbcfa7fc700 1 -- 172.21.15.42:0/1852251930 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fbd0008b330 con 0x7fbd04100ea0 2024-09-16T23:45:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.294+0000 7fbd08e9b700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbcf006c4e0 0x7fbcf006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.294+0000 7fbd08e9b700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbcf006c4e0 0x7fbcf006e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd04197bf0 tx=0x7fbcf401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:29.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.299+0000 7fbcfa7fc700 1 -- 172.21.15.42:0/1852251930 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbd00059980 con 0x7fbd04100ea0 2024-09-16T23:45:29.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:29 smithi043 ceph-mon[35191]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:29.497 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.502+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbd04197d30 con 0x7fbd04100ea0 2024-09-16T23:45:29.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.503+0000 7fbcfa7fc700 1 -- 172.21.15.42:0/1852251930 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd00059510 con 0x7fbd04100ea0 2024-09-16T23:45:29.500 INFO:teuthology.orchestra.run.smithi042.stdout:111669149716 2024-09-16T23:45:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbcf006c4e0 msgr2=0x7fbcf006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbcf006c4e0 0x7fbcf006e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd04197bf0 tx=0x7fbcf401a040 comp rx=0 tx=0).stop 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 msgr2=0x7fbd0419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000d8d0 tx=0x7fbd0000dbe0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 shutdown_connections 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbcf006c4e0 0x7fbcf006e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 --2- 172.21.15.42:0/1852251930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 >> 172.21.15.42:0/1852251930 conn(0x7fbd040fc750 msgr2=0x7fbd040fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 shutdown_connections 2024-09-16T23:45:29.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.505+0000 7fbd0b900700 1 -- 172.21.15.42:0/1852251930 wait complete. 2024-09-16T23:45:29.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 -- 172.21.15.42:0/1687691663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 msgr2=0x7f7d6c109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 --2- 172.21.15.42:0/1687691663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c109b10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009910 tx=0x7f7d68009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:29.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 -- 172.21.15.42:0/1687691663 shutdown_connections 2024-09-16T23:45:29.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 --2- 172.21.15.42:0/1687691663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c10a050 0x7f7d6c1127a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 --2- 172.21.15.42:0/1687691663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c109b10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 -- 172.21.15.42:0/1687691663 >> 172.21.15.42:0/1687691663 conn(0x7f7d6c06d160 msgr2=0x7f7d6c06d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:29.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 -- 172.21.15.42:0/1687691663 shutdown_connections 2024-09-16T23:45:29.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.508+0000 7f7d746a8700 1 -- 172.21.15.42:0/1687691663 wait complete. 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d746a8700 1 Processor -- start 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d746a8700 1 -- start start 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d746a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d746a8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c10a050 0x7f7d6c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d746a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c19fc70 con 0x7f7d6c109730 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d746a8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1a3a00 con 0x7f7d6c10a050 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d72444700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d72444700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c19f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55070/0 (socket says 172.21.15.42:55070) 2024-09-16T23:45:29.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d72444700 1 -- 172.21.15.42:0/3120093185 learned_addr learned my addr 172.21.15.42:0/3120093185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.509+0000 7f7d71c43700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c10a050 0x7f7d6c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.510+0000 7f7d72444700 1 -- 172.21.15.42:0/3120093185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c10a050 msgr2=0x7f7d6c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.510+0000 7f7d72444700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c10a050 0x7f7d6c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.510+0000 7f7d72444700 1 -- 172.21.15.42:0/3120093185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680095f0 con 0x7f7d6c109730 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.510+0000 7f7d71c43700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c10a050 0x7f7d6c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.510+0000 7f7d72444700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c19f050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d68006010 tx=0x7f7d6800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.510+0000 7f7d637fe700 1 -- 172.21.15.42:0/3120093185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d6801d070 con 0x7f7d6c109730 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.510+0000 7f7d637fe700 1 -- 172.21.15.42:0/3120093185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7d6800f460 con 0x7f7d6c109730 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.511+0000 7f7d637fe700 1 -- 172.21.15.42:0/3120093185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d68017630 con 0x7f7d6c109730 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.511+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c1a3ce0 con 0x7f7d6c109730 2024-09-16T23:45:29.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.511+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c1a4230 con 0x7f7d6c109730 2024-09-16T23:45:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.512+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c1110f0 con 0x7f7d6c109730 2024-09-16T23:45:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.512+0000 7f7d637fe700 1 -- 172.21.15.42:0/3120093185 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f7d68003a60 con 0x7f7d6c109730 2024-09-16T23:45:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.513+0000 7f7d637fe700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d5806c530 0x7f7d5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.513+0000 7f7d637fe700 1 -- 172.21.15.42:0/3120093185 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f7d6808c850 con 0x7f7d6c109730 2024-09-16T23:45:29.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.513+0000 7f7d71c43700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d5806c530 0x7f7d5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:29.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.513+0000 7f7d71c43700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d5806c530 0x7f7d5806e9f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c1a0670 tx=0x7f7d5c00b3c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:29.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.518+0000 7f7d637fe700 1 -- 172.21.15.42:0/3120093185 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7d68057370 con 0x7f7d6c109730 2024-09-16T23:45:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.721+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7d6c04f2e0 con 0x7f7d6c109730 2024-09-16T23:45:29.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.722+0000 7f7d637fe700 1 -- 172.21.15.42:0/3120093185 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d6805a990 con 0x7f7d6c109730 2024-09-16T23:45:29.719 INFO:teuthology.orchestra.run.smithi042.stdout:81604378650 2024-09-16T23:45:29.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d5806c530 msgr2=0x7f7d5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d5806c530 0x7f7d5806e9f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c1a0670 tx=0x7f7d5c00b3c0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 msgr2=0x7f7d6c19f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c19f050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d68006010 tx=0x7f7d6800bd70 comp rx=0 tx=0).stop 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 shutdown_connections 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d5806c530 0x7f7d5806e9f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d6c109730 0x7f7d6c19f050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 --2- 172.21.15.42:0/3120093185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d6c10a050 0x7f7d6c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 >> 172.21.15.42:0/3120093185 conn(0x7f7d6c06d160 msgr2=0x7f7d6c108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 shutdown_connections 2024-09-16T23:45:29.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:29.724+0000 7f7d746a8700 1 -- 172.21.15.42:0/3120093185 wait complete. 2024-09-16T23:45:30.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:30 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1852251930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T23:45:30.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:30 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3120093185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T23:45:30.412 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:30 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1852251930' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T23:45:30.412 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:30 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3120093185' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T23:45:30.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-09-16T23:45:30.439 DEBUG:teuthology.parallel:result is None 2024-09-16T23:45:31.304 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:31 smithi042 ceph-mon[27914]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:31.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:31 smithi043 ceph-mon[35191]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:31.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 -- 172.21.15.42:0/669134017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87800ffda0 msgr2=0x7f8780100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 --2- 172.21.15.42:0/669134017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87800ffda0 0x7f8780100180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f877000b0a0 tx=0x7f877000b3b0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 -- 172.21.15.42:0/669134017 shutdown_connections 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 --2- 172.21.15.42:0/669134017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8780100750 0x7f87801047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 --2- 172.21.15.42:0/669134017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87800ffda0 0x7f8780100180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 -- 172.21.15.42:0/669134017 >> 172.21.15.42:0/669134017 conn(0x7f87800fb650 msgr2=0x7f87800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 -- 172.21.15.42:0/669134017 shutdown_connections 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 -- 172.21.15.42:0/669134017 wait complete. 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.086+0000 7f87856c6700 1 Processor -- start 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f87856c6700 1 -- start start 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f87856c6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87800ffda0 0x7f8780101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f87856c6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8780100750 0x7f8780101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:31.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f87856c6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87801022c0 con 0x7f8780100750 2024-09-16T23:45:31.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f87856c6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8780102400 con 0x7f87800ffda0 2024-09-16T23:45:31.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f877e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8780100750 0x7f8780101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:31.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f877e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8780100750 0x7f8780101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55092/0 (socket says 172.21.15.42:55092) 2024-09-16T23:45:31.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f877e7fc700 1 -- 172.21.15.42:0/2090167124 learned_addr learned my addr 172.21.15.42:0/2090167124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:31.488 INFO:teuthology.orchestra.run.smithi042.stdout:167503724554 2024-09-16T23:45:31.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f877effd700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87800ffda0 0x7f8780101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:31.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f877e7fc700 1 -- 172.21.15.42:0/2090167124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87800ffda0 msgr2=0x7f8780101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f877e7fc700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87800ffda0 0x7f8780101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.087+0000 7f877e7fc700 1 -- 172.21.15.42:0/2090167124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8770009d00 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.088+0000 7f877e7fc700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8780100750 0x7f8780101d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f877400b480 tx=0x7f877400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.088+0000 7f8767fff700 1 -- 172.21.15.42:0/2090167124 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8774011800 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.088+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87801a2b40 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.088+0000 7f8767fff700 1 -- 172.21.15.42:0/2090167124 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8774011e10 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.088+0000 7f8767fff700 1 -- 172.21.15.42:0/2090167124 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8774010ea0 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.088+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87801a2f50 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.089+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878004f2e0 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.089+0000 7f8767fff700 1 -- 172.21.15.42:0/2090167124 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f8774010380 con 0x7f8780100750 2024-09-16T23:45:31.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.090+0000 7f8767fff700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f876806c4e0 0x7f876806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.090+0000 7f8767fff700 1 -- 172.21.15.42:0/2090167124 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f877408af80 con 0x7f8780100750 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.090+0000 7f877effd700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f876806c4e0 0x7f876806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.090+0000 7f877effd700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f876806c4e0 0x7f876806e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8770000f80 tx=0x7f87700060c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.094+0000 7f8767fff700 1 -- 172.21.15.42:0/2090167124 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8774055b40 con 0x7f8780100750 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.292+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f87801a3200 con 0x7f8780100750 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.293+0000 7f8767fff700 1 -- 172.21.15.42:0/2090167124 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8774059160 con 0x7f8780100750 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f876806c4e0 msgr2=0x7f876806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f876806c4e0 0x7f876806e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8770000f80 tx=0x7f87700060c0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8780100750 msgr2=0x7f8780101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8780100750 0x7f8780101d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f877400b480 tx=0x7f877400b790 comp rx=0 tx=0).stop 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 shutdown_connections 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f876806c4e0 0x7f876806e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87800ffda0 0x7f8780101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 --2- 172.21.15.42:0/2090167124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8780100750 0x7f8780101d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:31.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 >> 172.21.15.42:0/2090167124 conn(0x7f87800fb650 msgr2=0x7f8780104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:31.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 shutdown_connections 2024-09-16T23:45:31.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:31.295+0000 7f87856c6700 1 -- 172.21.15.42:0/2090167124 wait complete. 2024-09-16T23:45:32.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:32 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2090167124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T23:45:32.343 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:32.343 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:32.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:32 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2090167124' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T23:45:32.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149716 for osd.3 2024-09-16T23:45:32.733 DEBUG:teuthology.parallel:result is None 2024-09-16T23:45:33.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724551 got 167503724554 for osd.5 2024-09-16T23:45:33.164 DEBUG:teuthology.parallel:result is None 2024-09-16T23:45:33.348 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:33 smithi042 ceph-mon[27914]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:33.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:33 smithi043 ceph-mon[35191]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 -- 172.21.15.42:0/733800466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 msgr2=0x7efe6c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 --2- 172.21.15.42:0/733800466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c105fb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009910 tx=0x7efe5c009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 -- 172.21.15.42:0/733800466 shutdown_connections 2024-09-16T23:45:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 --2- 172.21.15.42:0/733800466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0ffc20 0x7efe6c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 --2- 172.21.15.42:0/733800466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c105fb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 -- 172.21.15.42:0/733800466 >> 172.21.15.42:0/733800466 conn(0x7efe6c075670 msgr2=0x7efe6c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 -- 172.21.15.42:0/733800466 shutdown_connections 2024-09-16T23:45:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.396+0000 7efe70a80700 1 -- 172.21.15.42:0/733800466 wait complete. 2024-09-16T23:45:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe70a80700 1 Processor -- start 2024-09-16T23:45:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe70a80700 1 -- start start 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe70a80700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0ffc20 0x7efe6c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe70a80700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe69d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe69d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36900/0 (socket says 172.21.15.42:36900) 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe69d9b700 1 -- 172.21.15.42:0/1681888160 learned_addr learned my addr 172.21.15.42:0/1681888160 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe70a80700 1 -- 172.21.15.42:0/1681888160 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c19f3f0 con 0x7efe6c105bd0 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.397+0000 7efe70a80700 1 -- 172.21.15.42:0/1681888160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c1988a0 con 0x7efe6c0ffc20 2024-09-16T23:45:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe6a59c700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0ffc20 0x7efe6c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe69d9b700 1 -- 172.21.15.42:0/1681888160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0ffc20 msgr2=0x7efe6c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe69d9b700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0ffc20 0x7efe6c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe69d9b700 1 -- 172.21.15.42:0/1681888160 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0095f0 con 0x7efe6c105bd0 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe69d9b700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c19ed60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e920 tx=0x7efe6000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe5b7fe700 1 -- 172.21.15.42:0/1681888160 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe60009790 con 0x7efe6c105bd0 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe70a80700 1 -- 172.21.15.42:0/1681888160 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c198b80 con 0x7efe6c105bd0 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe5b7fe700 1 -- 172.21.15.42:0/1681888160 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efe6000cba0 con 0x7efe6c105bd0 2024-09-16T23:45:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe5b7fe700 1 -- 172.21.15.42:0/1681888160 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe60018630 con 0x7efe6c105bd0 2024-09-16T23:45:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.398+0000 7efe70a80700 1 -- 172.21.15.42:0/1681888160 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c1990d0 con 0x7efe6c105bd0 2024-09-16T23:45:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.400+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c04f2e0 con 0x7efe6c105bd0 2024-09-16T23:45:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.401+0000 7efe5b7fe700 1 -- 172.21.15.42:0/1681888160 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7efe6000cd10 con 0x7efe6c105bd0 2024-09-16T23:45:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.402+0000 7efe5b7fe700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7efe5406c4e0 0x7efe5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.402+0000 7efe5b7fe700 1 -- 172.21.15.42:0/1681888160 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7efe60014070 con 0x7efe6c105bd0 2024-09-16T23:45:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.402+0000 7efe6a59c700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7efe5406c4e0 0x7efe5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.402+0000 7efe6a59c700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7efe5406c4e0 0x7efe5406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00b760 tx=0x7efe5c005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:33.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.406+0000 7efe5b7fe700 1 -- 172.21.15.42:0/1681888160 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efe6005a040 con 0x7efe6c105bd0 2024-09-16T23:45:33.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.605+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efe6c198d10 con 0x7efe6c105bd0 2024-09-16T23:45:33.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.605+0000 7efe5b7fe700 1 -- 172.21.15.42:0/1681888160 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe6c198d10 con 0x7efe6c105bd0 2024-09-16T23:45:33.603 INFO:teuthology.orchestra.run.smithi042.stdout:38654705703 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7efe5406c4e0 msgr2=0x7efe5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7efe5406c4e0 0x7efe5406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00b760 tx=0x7efe5c005940 comp rx=0 tx=0).stop 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 msgr2=0x7efe6c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c19ed60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e920 tx=0x7efe6000ece0 comp rx=0 tx=0).stop 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 shutdown_connections 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7efe5406c4e0 0x7efe5406e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe6c0ffc20 0x7efe6c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:33.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 --2- 172.21.15.42:0/1681888160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe6c105bd0 0x7efe6c19ed60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:33.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 >> 172.21.15.42:0/1681888160 conn(0x7efe6c075670 msgr2=0x7efe6c068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:33.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.607+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 shutdown_connections 2024-09-16T23:45:33.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:33.608+0000 7efe597fa700 1 -- 172.21.15.42:0/1681888160 wait complete. 2024-09-16T23:45:34.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:34 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1681888160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T23:45:34.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:34 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1681888160' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T23:45:35.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:35 smithi042 ceph-mon[27914]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:35.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:35 smithi043 ceph-mon[35191]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:36.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705701 got 38654705703 for osd.0 2024-09-16T23:45:36.506 DEBUG:teuthology.parallel:result is None 2024-09-16T23:45:36.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 -- 172.21.15.42:0/717661484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6804101060 msgr2=0x7f68041047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 --2- 172.21.15.42:0/717661484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6804101060 0x7f68041047b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009910 tx=0x7f67f8009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 -- 172.21.15.42:0/717661484 shutdown_connections 2024-09-16T23:45:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 --2- 172.21.15.42:0/717661484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6804101060 0x7f68041047b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 --2- 172.21.15.42:0/717661484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68041006b0 0x7f6804100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 -- 172.21.15.42:0/717661484 >> 172.21.15.42:0/717661484 conn(0x7f6804074e30 msgr2=0x7f6804075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 -- 172.21.15.42:0/717661484 shutdown_connections 2024-09-16T23:45:36.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.739+0000 7f6808b40700 1 -- 172.21.15.42:0/717661484 wait complete. 2024-09-16T23:45:36.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f6808b40700 1 Processor -- start 2024-09-16T23:45:36.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f6808b40700 1 -- start start 2024-09-16T23:45:36.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f6808b40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68041006b0 0x7f6804101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:36.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f6808b40700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6804101060 0x7f6804101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f6808b40700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68041022d0 con 0x7f68041006b0 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f6808b40700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6804102410 con 0x7f6804101060 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f680259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68041006b0 0x7f6804101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f680259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68041006b0 0x7f6804101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36908/0 (socket says 172.21.15.42:36908) 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f680259c700 1 -- 172.21.15.42:0/2815681243 learned_addr learned my addr 172.21.15.42:0/2815681243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f6801d9b700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6804101060 0x7f6804101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f680259c700 1 -- 172.21.15.42:0/2815681243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6804101060 msgr2=0x7f6804101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f680259c700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6804101060 0x7f6804101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:36.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.740+0000 7f680259c700 1 -- 172.21.15.42:0/2815681243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f80095f0 con 0x7f68041006b0 2024-09-16T23:45:36.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.741+0000 7f680259c700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68041006b0 0x7f6804101850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f67f400b4b0 tx=0x7f67f400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:36.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.741+0000 7f67f37fe700 1 -- 172.21.15.42:0/2815681243 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f4011800 con 0x7f68041006b0 2024-09-16T23:45:36.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.741+0000 7f67f37fe700 1 -- 172.21.15.42:0/2815681243 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f67f4011e10 con 0x7f68041006b0 2024-09-16T23:45:36.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.741+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68041a2ad0 con 0x7f68041006b0 2024-09-16T23:45:36.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.741+0000 7f67f37fe700 1 -- 172.21.15.42:0/2815681243 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f400f2e0 con 0x7f68041006b0 2024-09-16T23:45:36.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.741+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68041a2ee0 con 0x7f68041006b0 2024-09-16T23:45:36.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.742+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6804109150 con 0x7f68041006b0 2024-09-16T23:45:36.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.742+0000 7f67f37fe700 1 -- 172.21.15.42:0/2815681243 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f67f400f440 con 0x7f68041006b0 2024-09-16T23:45:36.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.743+0000 7f67f37fe700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67ec06c4e0 0x7f67ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:36.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.743+0000 7f67f37fe700 1 -- 172.21.15.42:0/2815681243 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f67f408af10 con 0x7f68041006b0 2024-09-16T23:45:36.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.743+0000 7f6801d9b700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67ec06c4e0 0x7f67ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:36.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.743+0000 7f6801d9b700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67ec06c4e0 0x7f67ec06e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6804103040 tx=0x7f67f801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:36.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.747+0000 7f67f37fe700 1 -- 172.21.15.42:0/2815681243 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f67f4055bc0 con 0x7f68041006b0 2024-09-16T23:45:36.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.954+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f680404f2e0 con 0x7f68041006b0 2024-09-16T23:45:36.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.954+0000 7f67f37fe700 1 -- 172.21.15.42:0/2815681243 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67f40591e0 con 0x7f68041006b0 2024-09-16T23:45:36.951 INFO:teuthology.orchestra.run.smithi042.stdout:146028888081 2024-09-16T23:45:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67ec06c4e0 msgr2=0x7f67ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67ec06c4e0 0x7f67ec06e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6804103040 tx=0x7f67f801a040 comp rx=0 tx=0).stop 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68041006b0 msgr2=0x7f6804101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68041006b0 0x7f6804101850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f67f400b4b0 tx=0x7f67f400b7c0 comp rx=0 tx=0).stop 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 shutdown_connections 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f67ec06c4e0 0x7f67ec06e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68041006b0 0x7f6804101850 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 --2- 172.21.15.42:0/2815681243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6804101060 0x7f6804101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 >> 172.21.15.42:0/2815681243 conn(0x7f6804074e30 msgr2=0x7f6804103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 shutdown_connections 2024-09-16T23:45:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:36.956+0000 7f6808b40700 1 -- 172.21.15.42:0/2815681243 wait complete. 2024-09-16T23:45:37.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:37 smithi042 ceph-mon[27914]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:37.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:37 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2815681243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T23:45:37.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:37 smithi043 ceph-mon[35191]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:37.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:37 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2815681243' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 -- 172.21.15.42:0/795341336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4100ea0 msgr2=0x7f0eb4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/795341336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4100ea0 0x7f0eb4101280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009910 tx=0x7f0ea4009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 -- 172.21.15.42:0/795341336 shutdown_connections 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/795341336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eb4101850 0x7f0eb41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/795341336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4100ea0 0x7f0eb4101280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 -- 172.21.15.42:0/795341336 >> 172.21.15.42:0/795341336 conn(0x7f0eb40fc750 msgr2=0x7f0eb40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 -- 172.21.15.42:0/795341336 shutdown_connections 2024-09-16T23:45:37.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.476+0000 7f0eb93d8700 1 -- 172.21.15.42:0/795341336 wait complete. 2024-09-16T23:45:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.477+0000 7f0eb93d8700 1 Processor -- start 2024-09-16T23:45:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.477+0000 7f0eb93d8700 1 -- start start 2024-09-16T23:45:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.477+0000 7f0eb93d8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eb4100ea0 0x7f0eb4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.477+0000 7f0eb93d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4101850 0x7f0eb40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.477+0000 7f0eb93d8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4073b10 con 0x7f0eb4101850 2024-09-16T23:45:37.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.477+0000 7f0eb93d8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4073c50 con 0x7f0eb4100ea0 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4101850 0x7f0eb40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4101850 0x7f0eb40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36922/0 (socket says 172.21.15.42:36922) 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb27fc700 1 -- 172.21.15.42:0/1380531944 learned_addr learned my addr 172.21.15.42:0/1380531944 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb2ffd700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eb4100ea0 0x7f0eb4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb27fc700 1 -- 172.21.15.42:0/1380531944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eb4100ea0 msgr2=0x7f0eb4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb27fc700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eb4100ea0 0x7f0eb4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb27fc700 1 -- 172.21.15.42:0/1380531944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40095f0 con 0x7f0eb4101850 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb27fc700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4101850 0x7f0eb40735d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d900 tx=0x7f0ea800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0e9bfff700 1 -- 172.21.15.42:0/1380531944 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea8009900 con 0x7f0eb4101850 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0e9bfff700 1 -- 172.21.15.42:0/1380531944 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ea8010460 con 0x7f0eb4101850 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb41a2af0 con 0x7f0eb4101850 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0e9bfff700 1 -- 172.21.15.42:0/1380531944 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ea800f5f0 con 0x7f0eb4101850 2024-09-16T23:45:37.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.478+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb41a2f00 con 0x7f0eb4101850 2024-09-16T23:45:37.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.479+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb4109240 con 0x7f0eb4101850 2024-09-16T23:45:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.480+0000 7f0e9bfff700 1 -- 172.21.15.42:0/1380531944 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0ea80105d0 con 0x7f0eb4101850 2024-09-16T23:45:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.480+0000 7f0e9bfff700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0e9c06c4e0 0x7f0e9c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.480+0000 7f0e9bfff700 1 -- 172.21.15.42:0/1380531944 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f0ea808b080 con 0x7f0eb4101850 2024-09-16T23:45:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.480+0000 7f0eb2ffd700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0e9c06c4e0 0x7f0e9c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:37.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.481+0000 7f0eb2ffd700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0e9c06c4e0 0x7f0e9c06e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4006010 tx=0x7f0ea401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:37.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.485+0000 7f0e9bfff700 1 -- 172.21.15.42:0/1380531944 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0ea8055ba0 con 0x7f0eb4101850 2024-09-16T23:45:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.682+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0eb4073e10 con 0x7f0eb4101850 2024-09-16T23:45:37.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.683+0000 7f0e9bfff700 1 -- 172.21.15.42:0/1380531944 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0eb4073e10 con 0x7f0eb4101850 2024-09-16T23:45:37.680 INFO:teuthology.orchestra.run.smithi042.stdout:60129542177 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0e9c06c4e0 msgr2=0x7f0e9c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0e9c06c4e0 0x7f0e9c06e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4006010 tx=0x7f0ea401a040 comp rx=0 tx=0).stop 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4101850 msgr2=0x7f0eb40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4101850 0x7f0eb40735d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d900 tx=0x7f0ea800dc10 comp rx=0 tx=0).stop 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 shutdown_connections 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0e9c06c4e0 0x7f0e9c06e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eb4100ea0 0x7f0eb4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 --2- 172.21.15.42:0/1380531944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0eb4101850 0x7f0eb40735d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 >> 172.21.15.42:0/1380531944 conn(0x7f0eb40fc750 msgr2=0x7f0eb40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 shutdown_connections 2024-09-16T23:45:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:37.685+0000 7f0eb93d8700 1 -- 172.21.15.42:0/1380531944 wait complete. 2024-09-16T23:45:37.950 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888081 for osd.4 2024-09-16T23:45:37.951 DEBUG:teuthology.parallel:result is None 2024-09-16T23:45:38.241 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:38 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1380531944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T23:45:38.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:38 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1380531944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T23:45:38.537 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542177 for osd.1 2024-09-16T23:45:38.537 DEBUG:teuthology.parallel:result is None 2024-09-16T23:45:38.537 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T23:45:38.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T23:45:38.757 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:39.195 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:39 smithi042 ceph-mon[27914]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:39.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:39 smithi043 ceph-mon[35191]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:40.202 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:40 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:40.202 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:40 smithi042 ceph-mon[27914]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:40.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:40 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:40.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:40 smithi043 ceph-mon[35191]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.449+0000 7f605ffa4700 1 -- 172.21.15.42:0/1330480957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 msgr2=0x7f6058101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:40.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.449+0000 7f605ffa4700 1 --2- 172.21.15.42:0/1330480957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058101280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6054009910 tx=0x7f6054009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 -- 172.21.15.42:0/1330480957 shutdown_connections 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 --2- 172.21.15.42:0/1330480957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058101850 0x7f60581058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 --2- 172.21.15.42:0/1330480957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058101280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 -- 172.21.15.42:0/1330480957 >> 172.21.15.42:0/1330480957 conn(0x7f60580fc750 msgr2=0x7f60580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 -- 172.21.15.42:0/1330480957 shutdown_connections 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 -- 172.21.15.42:0/1330480957 wait complete. 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 Processor -- start 2024-09-16T23:45:40.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.450+0000 7f605ffa4700 1 -- start start 2024-09-16T23:45:40.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605ffa4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:40.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605ffa4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058101850 0x7f60580735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:40.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605ffa4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058073b10 con 0x7f6058100ea0 2024-09-16T23:45:40.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605ffa4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058073c50 con 0x7f6058101850 2024-09-16T23:45:40.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605dd40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605dd40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36930/0 (socket says 172.21.15.42:36930) 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605dd40700 1 -- 172.21.15.42:0/881143773 learned_addr learned my addr 172.21.15.42:0/881143773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605dd40700 1 -- 172.21.15.42:0/881143773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058101850 msgr2=0x7f60580735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605dd40700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058101850 0x7f60580735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605dd40700 1 -- 172.21.15.42:0/881143773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60540095f0 con 0x7f6058100ea0 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605dd40700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058073090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6054006010 tx=0x7f605400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f604effd700 1 -- 172.21.15.42:0/881143773 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605401d070 con 0x7f6058100ea0 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f604effd700 1 -- 172.21.15.42:0/881143773 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f605400f460 con 0x7f6058100ea0 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6058073ed0 con 0x7f6058100ea0 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f604effd700 1 -- 172.21.15.42:0/881143773 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6054021640 con 0x7f6058100ea0 2024-09-16T23:45:40.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.451+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60581a2e40 con 0x7f6058100ea0 2024-09-16T23:45:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.452+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6058109240 con 0x7f6058100ea0 2024-09-16T23:45:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.453+0000 7f604effd700 1 -- 172.21.15.42:0/881143773 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f605400faf0 con 0x7f6058100ea0 2024-09-16T23:45:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.453+0000 7f604effd700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f604406c4e0 0x7f604406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.453+0000 7f604effd700 1 -- 172.21.15.42:0/881143773 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f605408c730 con 0x7f6058100ea0 2024-09-16T23:45:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.453+0000 7f605d53f700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f604406c4e0 0x7f604406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:40.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.454+0000 7f605d53f700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f604406c4e0 0x7f604406e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6058074880 tx=0x7f6048005c80 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:40.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.458+0000 7f604effd700 1 -- 172.21.15.42:0/881143773 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f60540572f0 con 0x7f6058100ea0 2024-09-16T23:45:40.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.650+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60580611d0 con 0x7f604406c4e0 2024-09-16T23:45:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.652+0000 7f604effd700 1 -- 172.21.15.42:0/881143773 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28028 (secure 0 0 0) 0x7f60580611d0 con 0x7f604406c4e0 2024-09-16T23:45:40.650 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:45:40.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f604406c4e0 msgr2=0x7f604406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:40.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f604406c4e0 0x7f604406e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6058074880 tx=0x7f6048005c80 comp rx=0 tx=0).stop 2024-09-16T23:45:40.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 msgr2=0x7f6058073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:40.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058073090 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6054006010 tx=0x7f605400be20 comp rx=0 tx=0).stop 2024-09-16T23:45:40.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 shutdown_connections 2024-09-16T23:45:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f604406c4e0 0x7f604406e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6058100ea0 0x7f6058073090 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 --2- 172.21.15.42:0/881143773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6058101850 0x7f60580735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 >> 172.21.15.42:0/881143773 conn(0x7f60580fc750 msgr2=0x7f60580fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 shutdown_connections 2024-09-16T23:45:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:40.654+0000 7f605ffa4700 1 -- 172.21.15.42:0/881143773 wait complete. 2024-09-16T23:45:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T23:45:41.273 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-16T23:45:39.130174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165540,"kb_used_data":3828,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305372,"statfs":{"total":575970213888,"available":575800700928,"internally_reserved":0,"allocated":3919872,"data_stored":2570982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.018310"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-16T23:44:49.614320+0000","last_change":"2024-09-16T23:44:22.653138+0000","last_active":"2024-09-16T23:44:49.614320+0000","last_peered":"2024-09-16T23:44:49.614320+0000","last_clean":"2024-09-16T23:44:49.614320+0000","last_became_active":"2024-09-16T23:44:22.652825+0000","last_became_peered":"2024-09-16T23:44:22.652825+0000","last_unstale":"2024-09-16T23:44:49.614320+0000","last_undegraded":"2024-09-16T23:44:49.614320+0000","last_fullsized":"2024-09-16T23:44:49.614320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T23:43:31.699965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T23:43:31.699965+0000","last_clean_scrub_stamp":"2024-09-16T23:43:31.699965+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:34.990910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27300,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717852,"statfs":{"total":95995035648,"available":95967080448,"internally_reserved":0,"allocated":356352,"data_stored":133313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27300,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717852,"statfs":{"total":95995035648,"available":95967080448,"internally_reserved":0,"allocated":356352,"data_stored":133313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.46400000000000002}]},{"osd":1,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.441}]},{"osd":2,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.51400000000000001}]},{"osd":3,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":723681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":5.8369999999999997,"5min":5.8369999999999997,"15min":5.8369999999999997},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.59699999999999998}]},{"osd":1,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":5.798,"5min":5.798,"15min":5.798},"last":0.55800000000000005}]},{"osd":2,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":19.591999999999999,"5min":19.591999999999999,"15min":19.591999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.433},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27300,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717852,"statfs":{"total":95995035648,"available":95967080448,"internally_reserved":0,"allocated":356352,"data_stored":133313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46800000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.5,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.48099999999999998}]},{"osd":1,"last update":"Mon Sep 16 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.379,"15min":0.38400000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.503},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48299999999999998,"5min":0.625,"15min":0.625},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":723681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 23:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.28000000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34100000000000003},"min":{"1min":0.28799999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Sep 16 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.437}]},{"osd":3,"last update":"Mon Sep 16 23:44:59 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":15.657999999999999,"5min":15.657999999999999,"15min":15.657999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":723681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:45:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.438,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.375},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35099999999999998},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.33000000000000002}]},{"osd":2,"last update":"Mon Sep 16 23:44:35 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.35399999999999998}]},{"osd":3,"last update":"Mon Sep 16 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.55300000000000005}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.68500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T23:45:41.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T23:45:41.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:41 smithi043 ceph-mon[35191]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T23:45:41.484 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:41 smithi042 ceph-mon[27914]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T23:45:42.854 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:42 smithi042 ceph-mon[27914]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:42.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:42 smithi043 ceph-mon[35191]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 -- 172.21.15.42:0/2144455741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60105bd0 msgr2=0x7fdb60105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 --2- 172.21.15.42:0/2144455741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60105bd0 0x7fdb60105fb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00b0a0 tx=0x7fdb5c00b3b0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 -- 172.21.15.42:0/2144455741 shutdown_connections 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 --2- 172.21.15.42:0/2144455741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb600ffc20 0x7fdb60100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 --2- 172.21.15.42:0/2144455741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb60105bd0 0x7fdb60105fb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 -- 172.21.15.42:0/2144455741 >> 172.21.15.42:0/2144455741 conn(0x7fdb60075670 msgr2=0x7fdb60075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 -- 172.21.15.42:0/2144455741 shutdown_connections 2024-09-16T23:45:43.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.081+0000 7fdb687ab700 1 -- 172.21.15.42:0/2144455741 wait complete. 2024-09-16T23:45:43.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb687ab700 1 Processor -- start 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb687ab700 1 -- start start 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb687ab700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb600ffc20 0x7fdb60101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb687ab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb60105bd0 0x7fdb60101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb687ab700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb601022c0 con 0x7fdb600ffc20 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb687ab700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60102400 con 0x7fdb60105bd0 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb66547700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb600ffc20 0x7fdb60101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb66547700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb600ffc20 0x7fdb60101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34630/0 (socket says 172.21.15.42:34630) 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.082+0000 7fdb66547700 1 -- 172.21.15.42:0/1478827774 learned_addr learned my addr 172.21.15.42:0/1478827774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb66547700 1 -- 172.21.15.42:0/1478827774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb60105bd0 msgr2=0x7fdb60101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb66547700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb60105bd0 0x7fdb60101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb66547700 1 -- 172.21.15.42:0/1478827774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c009d00 con 0x7fdb600ffc20 2024-09-16T23:45:43.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb66547700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb600ffc20 0x7fdb60101840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c000f80 tx=0x7fdb5c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb577fe700 1 -- 172.21.15.42:0/1478827774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5c00e040 con 0x7fdb600ffc20 2024-09-16T23:45:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb577fe700 1 -- 172.21.15.42:0/1478827774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb5c0044f0 con 0x7fdb600ffc20 2024-09-16T23:45:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb60102680 con 0x7fdb600ffc20 2024-09-16T23:45:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb577fe700 1 -- 172.21.15.42:0/1478827774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5c01d9f0 con 0x7fdb600ffc20 2024-09-16T23:45:43.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.083+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb601a2e80 con 0x7fdb600ffc20 2024-09-16T23:45:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.084+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6004f2e0 con 0x7fdb600ffc20 2024-09-16T23:45:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.085+0000 7fdb577fe700 1 -- 172.21.15.42:0/1478827774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fdb5c004660 con 0x7fdb600ffc20 2024-09-16T23:45:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.085+0000 7fdb577fe700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb4c06c4e0 0x7fdb4c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.085+0000 7fdb577fe700 1 -- 172.21.15.42:0/1478827774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fdb5c08ce60 con 0x7fdb600ffc20 2024-09-16T23:45:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.085+0000 7fdb65d46700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb4c06c4e0 0x7fdb4c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:43.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.085+0000 7fdb65d46700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb4c06c4e0 0x7fdb4c06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb60103030 tx=0x7fdb50006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:43.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.090+0000 7fdb577fe700 1 -- 172.21.15.42:0/1478827774 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb5c057a00 con 0x7fdb600ffc20 2024-09-16T23:45:43.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.282+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb60104990 con 0x7fdb4c06c4e0 2024-09-16T23:45:43.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.284+0000 7fdb577fe700 1 -- 172.21.15.42:0/1478827774 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28014 (secure 0 0 0) 0x7fdb60104990 con 0x7fdb4c06c4e0 2024-09-16T23:45:43.281 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb4c06c4e0 msgr2=0x7fdb4c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb4c06c4e0 0x7fdb4c06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb60103030 tx=0x7fdb50006d20 comp rx=0 tx=0).stop 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb600ffc20 msgr2=0x7fdb60101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb600ffc20 0x7fdb60101840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c000f80 tx=0x7fdb5c00cdb0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 shutdown_connections 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdb4c06c4e0 0x7fdb4c06e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb600ffc20 0x7fdb60101840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 --2- 172.21.15.42:0/1478827774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb60105bd0 0x7fdb60101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:43.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 >> 172.21.15.42:0/1478827774 conn(0x7fdb60075670 msgr2=0x7fdb600686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:43.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 shutdown_connections 2024-09-16T23:45:43.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:43.286+0000 7fdb687ab700 1 -- 172.21.15.42:0/1478827774 wait complete. 2024-09-16T23:45:43.284 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-09-16T23:45:43.872 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-16T23:45:43.130801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165540,"kb_used_data":3828,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305372,"statfs":{"total":575970213888,"available":575800700928,"internally_reserved":0,"allocated":3919872,"data_stored":2570982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002198"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-16T23:44:49.614320+0000","last_change":"2024-09-16T23:44:22.653138+0000","last_active":"2024-09-16T23:44:49.614320+0000","last_peered":"2024-09-16T23:44:49.614320+0000","last_clean":"2024-09-16T23:44:49.614320+0000","last_became_active":"2024-09-16T23:44:22.652825+0000","last_became_peered":"2024-09-16T23:44:22.652825+0000","last_unstale":"2024-09-16T23:44:49.614320+0000","last_undegraded":"2024-09-16T23:44:49.614320+0000","last_fullsized":"2024-09-16T23:44:49.614320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T23:43:31.699965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T23:43:31.699965+0000","last_clean_scrub_stamp":"2024-09-16T23:43:31.699965+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:34.990910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27300,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717852,"statfs":{"total":95995035648,"available":95967080448,"internally_reserved":0,"allocated":356352,"data_stored":133313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27300,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717852,"statfs":{"total":95995035648,"available":95967080448,"internally_reserved":0,"allocated":356352,"data_stored":133313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.57099999999999995}]},{"osd":1,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.45700000000000002}]},{"osd":3,"last update":"Mon Sep 16 23:45:23 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":723681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":5.8369999999999997,"5min":5.8369999999999997,"15min":5.8369999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56100000000000005}]},{"osd":1,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.503},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":5.798,"5min":5.798,"15min":5.798},"last":0.54300000000000004}]},{"osd":2,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":19.591999999999999,"5min":19.591999999999999,"15min":19.591999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27300,"kb_used_data":348,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717852,"statfs":{"total":95995035648,"available":95967080448,"internally_reserved":0,"allocated":356352,"data_stored":133313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.46800000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.434},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.376,"15min":0.38},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.5,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Sep 16 23:45:31 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.379,"15min":0.38400000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.48199999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.39000000000000001,"15min":0.39700000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48299999999999998,"5min":0.625,"15min":0.625},"last":0.40999999999999998}]},{"osd":3,"last update":"Mon Sep 16 23:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.78800000000000003}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.63400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":723681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 23:45:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.28000000000000003,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34100000000000003},"min":{"1min":0.28799999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.32800000000000001}]},{"osd":2,"last update":"Mon Sep 16 23:45:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.40600000000000003,"15min":0.40000000000000002},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.437}]},{"osd":3,"last update":"Mon Sep 16 23:44:59 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":15.657999999999999,"5min":15.657999999999999,"15min":15.657999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47699999999999998}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":723681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 23:45:05 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.438,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.35299999999999998,"15min":0.35099999999999998},"min":{"1min":0.29499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.30099999999999999}]},{"osd":2,"last update":"Mon Sep 16 23:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.36699999999999999,"15min":0.36799999999999999},"min":{"1min":0.27200000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.53700000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.376,"15min":0.377},"min":{"1min":0.26400000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.379}]},{"osd":3,"last update":"Mon Sep 16 23:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.436}]},{"osd":4,"last update":"Mon Sep 16 23:45:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.47999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T23:45:43.873 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T23:45:43.873 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T23:45:43.873 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T23:45:43.874 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T23:45:44.082 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:44.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:44 smithi043 ceph-mon[35191]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:44.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:44 smithi043 ceph-mon[35191]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T23:45:45.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:44 smithi042 ceph-mon[27914]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:45.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:44 smithi042 ceph-mon[27914]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T23:45:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/556143340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 msgr2=0x7fbd58105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/556143340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58105fb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009910 tx=0x7fbd48009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:45.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/556143340 shutdown_connections 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/556143340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd580ffc20 0x7fbd58100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/556143340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58105fb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/556143340 >> 172.21.15.42:0/556143340 conn(0x7fbd58075670 msgr2=0x7fbd58075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/556143340 shutdown_connections 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.688+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/556143340 wait complete. 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd5e5f2700 1 Processor -- start 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd5e5f2700 1 -- start start 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd5e5f2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd580ffc20 0x7fbd581976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd5e5f2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd5e5f2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58198260 con 0x7fbd58105bd0 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd5e5f2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd581983d0 con 0x7fbd580ffc20 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd4ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:45.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd4ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34644/0 (socket says 172.21.15.42:34644) 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.689+0000 7fbd4ffff700 1 -- 172.21.15.42:0/158248027 learned_addr learned my addr 172.21.15.42:0/158248027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd4ffff700 1 -- 172.21.15.42:0/158248027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd580ffc20 msgr2=0x7fbd581976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd57fff700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd580ffc20 0x7fbd581976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd4ffff700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd580ffc20 0x7fbd581976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd4ffff700 1 -- 172.21.15.42:0/158248027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480095f0 con 0x7fbd58105bd0 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd4ffff700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58197c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c930 tx=0x7fbd4000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd55ffb700 1 -- 172.21.15.42:0/158248027 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd40007ad0 con 0x7fbd58105bd0 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd55ffb700 1 -- 172.21.15.42:0/158248027 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbd40007c30 con 0x7fbd58105bd0 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd55ffb700 1 -- 172.21.15.42:0/158248027 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd40018630 con 0x7fbd58105bd0 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd581a0740 con 0x7fbd58105bd0 2024-09-16T23:45:45.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.690+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd581a0c00 con 0x7fbd58105bd0 2024-09-16T23:45:45.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.691+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd581080d0 con 0x7fbd58105bd0 2024-09-16T23:45:45.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.691+0000 7fbd55ffb700 1 -- 172.21.15.42:0/158248027 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbd4000f450 con 0x7fbd58105bd0 2024-09-16T23:45:45.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.692+0000 7fbd55ffb700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbd4406c4e0 0x7fbd4406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:45.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.692+0000 7fbd55ffb700 1 -- 172.21.15.42:0/158248027 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fbd4008b540 con 0x7fbd58105bd0 2024-09-16T23:45:45.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.692+0000 7fbd57fff700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbd4406c4e0 0x7fbd4406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:45.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.692+0000 7fbd57fff700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbd4406c4e0 0x7fbd4406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:45.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.696+0000 7fbd55ffb700 1 -- 172.21.15.42:0/158248027 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbd40056180 con 0x7fbd58105bd0 2024-09-16T23:45:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.940+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbd5804f2e0 con 0x7fbd58105bd0 2024-09-16T23:45:45.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.941+0000 7fbd55ffb700 1 -- 172.21.15.42:0/158248027 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbd400597a0 con 0x7fbd58105bd0 2024-09-16T23:45:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:45:45.938 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T23:45:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbd4406c4e0 msgr2=0x7fbd4406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbd4406c4e0 0x7fbd4406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48005fd0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 msgr2=0x7fbd58197c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58197c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000c930 tx=0x7fbd4000ccf0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 shutdown_connections 2024-09-16T23:45:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbd4406c4e0 0x7fbd4406e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd580ffc20 0x7fbd581976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 --2- 172.21.15.42:0/158248027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd58105bd0 0x7fbd58197c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:45.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 >> 172.21.15.42:0/158248027 conn(0x7fbd58075670 msgr2=0x7fbd580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:45.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 shutdown_connections 2024-09-16T23:45:45.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:45.943+0000 7fbd5e5f2700 1 -- 172.21.15.42:0/158248027 wait complete. 2024-09-16T23:45:46.558 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T23:45:46.558 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T23:45:46.558 INFO:teuthology.run_tasks:Running task print... 2024-09-16T23:45:46.567 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-09-16T23:45:46.567 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T23:45:46.575 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:45:46.576 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-09-16T23:45:46.797 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:46.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:46 smithi043 ceph-mon[35191]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:46.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:46 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/158248027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T23:45:46.918 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:46 smithi042 ceph-mon[27914]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:46.918 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:46 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/158248027' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 -- 172.21.15.42:0/1138197954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41140ff320 msgr2=0x7f41140ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 --2- 172.21.15.42:0/1138197954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41140ff320 0x7f41140ff740 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f410400b0a0 tx=0x7f410400b3b0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 -- 172.21.15.42:0/1138197954 shutdown_connections 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 --2- 172.21.15.42:0/1138197954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4114100520 0x7f4114100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 --2- 172.21.15.42:0/1138197954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41140ff320 0x7f41140ff740 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 -- 172.21.15.42:0/1138197954 >> 172.21.15.42:0/1138197954 conn(0x7f41140fa8c0 msgr2=0x7f41140fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 -- 172.21.15.42:0/1138197954 shutdown_connections 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.571+0000 7f4119b97700 1 -- 172.21.15.42:0/1138197954 wait complete. 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4119b97700 1 Processor -- start 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4119b97700 1 -- start start 2024-09-16T23:45:48.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4119b97700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41140ff320 0x7f41141953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4119b97700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4114100520 0x7f4114195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4119b97700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114195f20 con 0x7f4114100520 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4119b97700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4114196060 con 0x7f41140ff320 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4112ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4114100520 0x7f4114195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4112ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4114100520 0x7f4114195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34658/0 (socket says 172.21.15.42:34658) 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f4112ffd700 1 -- 172.21.15.42:0/3145334159 learned_addr learned my addr 172.21.15.42:0/3145334159 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.572+0000 7f41137fe700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41140ff320 0x7f41141953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4112ffd700 1 -- 172.21.15.42:0/3145334159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41140ff320 msgr2=0x7f41141953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:48.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4112ffd700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41140ff320 0x7f41141953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4112ffd700 1 -- 172.21.15.42:0/3145334159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4104009d00 con 0x7f4114100520 2024-09-16T23:45:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4112ffd700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4114100520 0x7f4114195900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f410800d8d0 tx=0x7f410800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4110ff9700 1 -- 172.21.15.42:0/3145334159 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41080098a0 con 0x7f4114100520 2024-09-16T23:45:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411419ab10 con 0x7f4114100520 2024-09-16T23:45:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411419b060 con 0x7f4114100520 2024-09-16T23:45:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4110ff9700 1 -- 172.21.15.42:0/3145334159 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4108010460 con 0x7f4114100520 2024-09-16T23:45:48.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.573+0000 7f4110ff9700 1 -- 172.21.15.42:0/3145334159 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410800f5d0 con 0x7f4114100520 2024-09-16T23:45:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.574+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4100005320 con 0x7f4114100520 2024-09-16T23:45:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.575+0000 7f4110ff9700 1 -- 172.21.15.42:0/3145334159 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f410800f730 con 0x7f4114100520 2024-09-16T23:45:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.575+0000 7f4110ff9700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f40fc06c4e0 0x7f40fc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.576+0000 7f4110ff9700 1 -- 172.21.15.42:0/3145334159 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f410808b2c0 con 0x7f4114100520 2024-09-16T23:45:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.576+0000 7f41137fe700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f40fc06c4e0 0x7f40fc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:48.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.576+0000 7f41137fe700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f40fc06c4e0 0x7f40fc06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f410400b830 tx=0x7f41040060c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:48.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.580+0000 7f4110ff9700 1 -- 172.21.15.42:0/3145334159 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4108059910 con 0x7f4114100520 2024-09-16T23:45:48.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.778+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f41000059f0 con 0x7f4114100520 2024-09-16T23:45:48.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.782+0000 7f4110ff9700 1 -- 172.21.15.42:0/3145334159 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f41080594a0 con 0x7f4114100520 2024-09-16T23:45:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f40fc06c4e0 msgr2=0x7f40fc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f40fc06c4e0 0x7f40fc06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f410400b830 tx=0x7f41040060c0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4114100520 msgr2=0x7f4114195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:48.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4114100520 0x7f4114195900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f410800d8d0 tx=0x7f410800dbe0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 shutdown_connections 2024-09-16T23:45:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f40fc06c4e0 0x7f40fc06e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41140ff320 0x7f41141953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 --2- 172.21.15.42:0/3145334159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4114100520 0x7f4114195900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 >> 172.21.15.42:0/3145334159 conn(0x7f41140fa8c0 msgr2=0x7f41140fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 shutdown_connections 2024-09-16T23:45:48.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:48.785+0000 7f4119b97700 1 -- 172.21.15.42:0/3145334159 wait complete. 2024-09-16T23:45:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:48 smithi043 ceph-mon[35191]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:48.896 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:48 smithi042 ceph-mon[27914]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:49.513 INFO:teuthology.run_tasks:Running task print... 2024-09-16T23:45:49.523 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-09-16T23:45:49.523 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T23:45:49.531 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:45:49.531 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T23:45:49.744 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:50.111 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:49 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3145334159' entity='client.admin' 2024-09-16T23:45:50.111 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:45:50.111 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:45:50.111 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:45:50.111 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:49 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:45:50.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:49 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3145334159' entity='client.admin' 2024-09-16T23:45:50.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:45:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:45:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:45:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:49 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:45:51.038 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:50 smithi042 ceph-mon[27914]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:50 smithi043 ceph-mon[35191]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:51.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 -- 172.21.15.42:0/1441653040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 msgr2=0x7fbc30101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:51.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 --2- 172.21.15.42:0/1441653040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30101220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009910 tx=0x7fbc24009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 -- 172.21.15.42:0/1441653040 shutdown_connections 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 --2- 172.21.15.42:0/1441653040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30101220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 --2- 172.21.15.42:0/1441653040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc300ffbc0 0x7fbc300fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 -- 172.21.15.42:0/1441653040 >> 172.21.15.42:0/1441653040 conn(0x7fbc300fb160 msgr2=0x7fbc300fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 -- 172.21.15.42:0/1441653040 shutdown_connections 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 -- 172.21.15.42:0/1441653040 wait complete. 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.432+0000 7fbc35930700 1 Processor -- start 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc35930700 1 -- start start 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc35930700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc300ffbc0 0x7fbc301967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc35930700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc35930700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30197330 con 0x7fbc30100dc0 2024-09-16T23:45:51.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc35930700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30197470 con 0x7fbc300ffbc0 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34678/0 (socket says 172.21.15.42:34678) 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2e7fc700 1 -- 172.21.15.42:0/3110778876 learned_addr learned my addr 172.21.15.42:0/3110778876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2effd700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc300ffbc0 0x7fbc301967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2e7fc700 1 -- 172.21.15.42:0/3110778876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc300ffbc0 msgr2=0x7fbc301967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2e7fc700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc300ffbc0 0x7fbc301967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2e7fc700 1 -- 172.21.15.42:0/3110778876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240095f0 con 0x7fbc30100dc0 2024-09-16T23:45:51.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.433+0000 7fbc2e7fc700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30196d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbc240092e0 tx=0x7fbc2400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.434+0000 7fbc3492e700 1 -- 172.21.15.42:0/3110778876 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2401d070 con 0x7fbc30100dc0 2024-09-16T23:45:51.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.434+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3019bec0 con 0x7fbc30100dc0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.434+0000 7fbc3492e700 1 -- 172.21.15.42:0/3110778876 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc2400f460 con 0x7fbc30100dc0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.434+0000 7fbc3492e700 1 -- 172.21.15.42:0/3110778876 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc24021620 con 0x7fbc30100dc0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.434+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3019c380 con 0x7fbc30100dc0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.435+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc30068af0 con 0x7fbc30100dc0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.435+0000 7fbc3492e700 1 -- 172.21.15.42:0/3110778876 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fbc24003a60 con 0x7fbc30100dc0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.436+0000 7fbc3492e700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbc1806c4e0 0x7fbc1806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.436+0000 7fbc3492e700 1 -- 172.21.15.42:0/3110778876 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fbc2408c6a0 con 0x7fbc30100dc0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.436+0000 7fbc2effd700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbc1806c4e0 0x7fbc1806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:51.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.436+0000 7fbc2effd700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbc1806c4e0 0x7fbc1806e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc30100c20 tx=0x7fbc2000b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:51.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.440+0000 7fbc3492e700 1 -- 172.21.15.42:0/3110778876 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fbc2405acf0 con 0x7fbc30100dc0 2024-09-16T23:45:51.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.655+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc30069560 con 0x7fbc1806c4e0 2024-09-16T23:45:51.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.656+0000 7fbc3492e700 1 -- 172.21.15.42:0/3110778876 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbc30069560 con 0x7fbc1806c4e0 2024-09-16T23:45:51.653 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2024-09-16T23:45:51.653 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2024-09-16T23:45:51.654 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2024-09-16T23:45:51.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbc1806c4e0 msgr2=0x7fbc1806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbc1806c4e0 0x7fbc1806e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc30100c20 tx=0x7fbc2000b410 comp rx=0 tx=0).stop 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 msgr2=0x7fbc30196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30196d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbc240092e0 tx=0x7fbc2400bd70 comp rx=0 tx=0).stop 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 shutdown_connections 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fbc1806c4e0 0x7fbc1806e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc300ffbc0 0x7fbc301967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 --2- 172.21.15.42:0/3110778876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbc30100dc0 0x7fbc30196d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 >> 172.21.15.42:0/3110778876 conn(0x7fbc300fb160 msgr2=0x7fbc30068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 shutdown_connections 2024-09-16T23:45:51.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:51.658+0000 7fbc35930700 1 -- 172.21.15.42:0/3110778876 wait complete. 2024-09-16T23:45:52.290 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T23:45:52.502 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:52.960 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:52 smithi042 ceph-mon[27914]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:52.960 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:52 smithi042 ceph-mon[27914]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:45:53.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:52 smithi043 ceph-mon[35191]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:52 smithi043 ceph-mon[35191]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 -- 172.21.15.42:0/3010859719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8100cb0 msgr2=0x7f5ed81010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 --2- 172.21.15.42:0/3010859719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8100cb0 0x7f5ed81010d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800b0a0 tx=0x7f5ec800b3b0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 -- 172.21.15.42:0/3010859719 shutdown_connections 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 --2- 172.21.15.42:0/3010859719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ed8101eb0 0x7f5ed8102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 --2- 172.21.15.42:0/3010859719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8100cb0 0x7f5ed81010d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 -- 172.21.15.42:0/3010859719 >> 172.21.15.42:0/3010859719 conn(0x7f5ed80fc230 msgr2=0x7f5ed80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 -- 172.21.15.42:0/3010859719 shutdown_connections 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.130+0000 7f5edd939700 1 -- 172.21.15.42:0/3010859719 wait complete. 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5edd939700 1 Processor -- start 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5edd939700 1 -- start start 2024-09-16T23:45:54.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5edd939700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ed8100cb0 0x7f5ed81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5edd939700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8101eb0 0x7f5ed8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5edd939700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed8199340 con 0x7f5ed8101eb0 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5edd939700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed8199480 con 0x7f5ed8100cb0 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5ec7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8101eb0 0x7f5ed8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5ec7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8101eb0 0x7f5ed8198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59304/0 (socket says 172.21.15.42:59304) 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5ec7fff700 1 -- 172.21.15.42:0/3752477849 learned_addr learned my addr 172.21.15.42:0/3752477849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5ed6ffd700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ed8100cb0 0x7f5ed81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5ec7fff700 1 -- 172.21.15.42:0/3752477849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ed8100cb0 msgr2=0x7f5ed81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.131+0000 7f5ec7fff700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ed8100cb0 0x7f5ed81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5ec7fff700 1 -- 172.21.15.42:0/3752477849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec8009d00 con 0x7f5ed8101eb0 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5ed6ffd700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ed8100cb0 0x7f5ed81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:45:54.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5ec7fff700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8101eb0 0x7f5ed8198d20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00b4b0 tx=0x7f5ecc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:54.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5edc937700 1 -- 172.21.15.42:0/3752477849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ecc011800 con 0x7f5ed8101eb0 2024-09-16T23:45:54.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5edc937700 1 -- 172.21.15.42:0/3752477849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ecc011e10 con 0x7f5ed8101eb0 2024-09-16T23:45:54.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed819df30 con 0x7f5ed8101eb0 2024-09-16T23:45:54.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5edc937700 1 -- 172.21.15.42:0/3752477849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ecc00f2e0 con 0x7f5ed8101eb0 2024-09-16T23:45:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.132+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed819e450 con 0x7f5ed8101eb0 2024-09-16T23:45:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.133+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed8105f90 con 0x7f5ed8101eb0 2024-09-16T23:45:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.133+0000 7f5edc937700 1 -- 172.21.15.42:0/3752477849 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5ecc010380 con 0x7f5ed8101eb0 2024-09-16T23:45:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.134+0000 7f5edc937700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5ec006c4e0 0x7f5ec006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.134+0000 7f5edc937700 1 -- 172.21.15.42:0/3752477849 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f5ecc08afb0 con 0x7f5ed8101eb0 2024-09-16T23:45:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.134+0000 7f5ed6ffd700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5ec006c4e0 0x7f5ec006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.134+0000 7f5ed6ffd700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5ec006c4e0 0x7f5ec006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8000f80 tx=0x7f5ec800b7b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.139+0000 7f5edc937700 1 -- 172.21.15.42:0/3752477849 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5ecc059680 con 0x7f5ed8101eb0 2024-09-16T23:45:54.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.347+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ed80611d0 con 0x7f5ec006c4e0 2024-09-16T23:45:54.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.356+0000 7f5edc937700 1 -- 172.21.15.42:0/3752477849 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7f5ed80611d0 con 0x7f5ec006c4e0 2024-09-16T23:45:54.353 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:45:54.353 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (3m) 2m ago 5m 26.8M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:45:54.353 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (5m) 2m ago 5m 7033k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (4m) 66s ago 4m 6933k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (5m) 2m ago 5m 6660k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (4m) 66s ago 4m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (3m) 2m ago 5m 77.3M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:9283,8765,8443 running (6m) 2m ago 6m 481M - 18.2.1 5be31c24972a cd8abd4f468e 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (4m) 66s ago 4m 441M - 18.2.1 5be31c24972a b8099e88d36b 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (6m) 2m ago 6m 46.0M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (4m) 66s ago 4m 43.2M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:45:54.354 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 7c84e9bf6c1c 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (4m) 66s ago 4m 17.0M - 1.5.0 0da6a335fe13 82d41c3cc1ee 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (3m) 2m ago 3m 40.7M 3948M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (2m) 2m ago 2m 40.5M 3948M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (2m) 2m ago 2m 40.6M 3948M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (2m) 66s ago 2m 41.2M 4972M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (98s) 66s ago 98s 38.4M 4972M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (71s) 66s ago 70s 11.5M 4972M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:45:54.355 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (3m) 2m ago 4m 38.2M - 2.43.0 a07b618ecd1d 8ebaffdf2443 2024-09-16T23:45:54.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5ec006c4e0 msgr2=0x7f5ec006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:54.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5ec006c4e0 0x7f5ec006e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8000f80 tx=0x7f5ec800b7b0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8101eb0 msgr2=0x7f5ed8198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8101eb0 0x7f5ed8198d20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00b4b0 tx=0x7f5ecc00b870 comp rx=0 tx=0).stop 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 shutdown_connections 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5ec006c4e0 0x7f5ec006e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ed8100cb0 0x7f5ed81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 --2- 172.21.15.42:0/3752477849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ed8101eb0 0x7f5ed8198d20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 >> 172.21.15.42:0/3752477849 conn(0x7f5ed80fc230 msgr2=0x7f5ed81030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 shutdown_connections 2024-09-16T23:45:54.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:54.358+0000 7f5edd939700 1 -- 172.21.15.42:0/3752477849 wait complete. 2024-09-16T23:45:55.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:54 smithi042 ceph-mon[27914]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:55.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:55.037 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T23:45:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:54 smithi043 ceph-mon[35191]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:55.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:45:55.247 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:56.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:55 smithi042 ceph-mon[27914]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:45:56.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:55 smithi043 ceph-mon[35191]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:45:56.967 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:56 smithi042 ceph-mon[27914]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 -- 172.21.15.42:0/1750069310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 msgr2=0x7fae28101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 --2- 172.21.15.42:0/1750069310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28101220 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009910 tx=0x7fae1c009c20 comp rx=0 tx=0).stop 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 -- 172.21.15.42:0/1750069310 shutdown_connections 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 --2- 172.21.15.42:0/1750069310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28101220 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 --2- 172.21.15.42:0/1750069310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae280ffbc0 0x7fae280fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 -- 172.21.15.42:0/1750069310 >> 172.21.15.42:0/1750069310 conn(0x7fae280fb160 msgr2=0x7fae280fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 -- 172.21.15.42:0/1750069310 shutdown_connections 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.960+0000 7fae2cbca700 1 -- 172.21.15.42:0/1750069310 wait complete. 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.961+0000 7fae2cbca700 1 Processor -- start 2024-09-16T23:45:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.961+0000 7fae2cbca700 1 -- start start 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.961+0000 7fae2cbca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae280ffbc0 0x7fae281967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.961+0000 7fae2cbca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.961+0000 7fae2cbca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae281972a0 con 0x7fae28100dc0 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.961+0000 7fae2cbca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae281973e0 con 0x7fae280ffbc0 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59314/0 (socket says 172.21.15.42:59314) 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae25d9b700 1 -- 172.21.15.42:0/574913536 learned_addr learned my addr 172.21.15.42:0/574913536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae25d9b700 1 -- 172.21.15.42:0/574913536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae280ffbc0 msgr2=0x7fae281967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae25d9b700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae280ffbc0 0x7fae281967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae25d9b700 1 -- 172.21.15.42:0/574913536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0095f0 con 0x7fae28100dc0 2024-09-16T23:45:56.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae25d9b700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28196d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae1c006010 tx=0x7fae1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae177fe700 1 -- 172.21.15.42:0/574913536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae1c01d070 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae177fe700 1 -- 172.21.15.42:0/574913536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fae1c00f460 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2819be40 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae177fe700 1 -- 172.21.15.42:0/574913536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae1c021640 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.962+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2819c300 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.963+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae28068af0 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.964+0000 7fae177fe700 1 -- 172.21.15.42:0/574913536 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fae1c00fab0 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.964+0000 7fae177fe700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fae1006c4e0 0x7fae1006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.964+0000 7fae2659c700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fae1006c4e0 0x7fae1006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.964+0000 7fae177fe700 1 -- 172.21.15.42:0/574913536 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fae1c08c7e0 con 0x7fae28100dc0 2024-09-16T23:45:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.965+0000 7fae2659c700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fae1006c4e0 0x7fae1006e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae28100c20 tx=0x7fae18005d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:56.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:56.969+0000 7fae177fe700 1 -- 172.21.15.42:0/574913536 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fae1c05aeb0 con 0x7fae28100dc0 2024-09-16T23:45:57.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:56 smithi043 ceph-mon[35191]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:57.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.179+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fae28069560 con 0x7fae1006c4e0 2024-09-16T23:45:57.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.184+0000 7fae177fe700 1 -- 172.21.15.42:0/574913536 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7fae28069560 con 0x7fae1006c4e0 2024-09-16T23:45:57.181 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T23:45:57.181 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-09-16T23:45:57.181 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter 2/2 2m ago 6m * 2024-09-16T23:45:57.181 INFO:teuthology.orchestra.run.smithi042.stdout:crash 2/2 2m ago 6m * 2024-09-16T23:45:57.181 INFO:teuthology.orchestra.run.smithi042.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-09-16T23:45:57.182 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 2/2 2m ago 6m count:2 2024-09-16T23:45:57.182 INFO:teuthology.orchestra.run.smithi042.stdout:mon 2/2 2m ago 5m smithi042:172.21.15.42=smithi042;smithi043:172.21.15.43=smithi043;count:2 2024-09-16T23:45:57.182 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-09-16T23:45:57.182 INFO:teuthology.orchestra.run.smithi042.stdout:osd 6 2m ago - 2024-09-16T23:45:57.182 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-09-16T23:45:57.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fae1006c4e0 msgr2=0x7fae1006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fae1006c4e0 0x7fae1006e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fae28100c20 tx=0x7fae18005d20 comp rx=0 tx=0).stop 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 msgr2=0x7fae28196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28196d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fae1c006010 tx=0x7fae1c00be20 comp rx=0 tx=0).stop 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 shutdown_connections 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fae1006c4e0 0x7fae1006e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae280ffbc0 0x7fae281967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 --2- 172.21.15.42:0/574913536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae28100dc0 0x7fae28196d10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 >> 172.21.15.42:0/574913536 conn(0x7fae280fb160 msgr2=0x7fae28068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 shutdown_connections 2024-09-16T23:45:57.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:57.186+0000 7fae2cbca700 1 -- 172.21.15.42:0/574913536 wait complete. 2024-09-16T23:45:57.879 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T23:45:58.090 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:45:59.101 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:58 smithi042 ceph-mon[27914]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:59.101 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:45:58 smithi042 ceph-mon[27914]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:45:59.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:58 smithi043 ceph-mon[35191]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:45:59.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:45:58 smithi043 ceph-mon[35191]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.690+0000 7ff85f059700 1 -- 172.21.15.42:0/1376446712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 msgr2=0x7ff8581010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.690+0000 7ff85f059700 1 --2- 172.21.15.42:0/1376446712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff8581010c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff85400b0a0 tx=0x7ff85400b3b0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.690+0000 7ff85f059700 1 -- 172.21.15.42:0/1376446712 shutdown_connections 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.690+0000 7ff85f059700 1 --2- 172.21.15.42:0/1376446712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff858101ea0 0x7ff858102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.690+0000 7ff85f059700 1 --2- 172.21.15.42:0/1376446712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff8581010c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.690+0000 7ff85f059700 1 -- 172.21.15.42:0/1376446712 >> 172.21.15.42:0/1376446712 conn(0x7ff8580fc260 msgr2=0x7ff8580fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.690+0000 7ff85f059700 1 -- 172.21.15.42:0/1376446712 shutdown_connections 2024-09-16T23:45:59.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85f059700 1 -- 172.21.15.42:0/1376446712 wait complete. 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85f059700 1 Processor -- start 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85f059700 1 -- start start 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85f059700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff858074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85f059700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff858101ea0 0x7ff858073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85f059700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8580736c0 con 0x7ff858100ca0 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85f059700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858073800 con 0x7ff858101ea0 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.691+0000 7ff85cdf5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff858074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:59.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85cdf5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff858074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59330/0 (socket says 172.21.15.42:59330) 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85cdf5700 1 -- 172.21.15.42:0/1179932294 learned_addr learned my addr 172.21.15.42:0/1179932294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff84ffff700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff858101ea0 0x7ff858073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85cdf5700 1 -- 172.21.15.42:0/1179932294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff858101ea0 msgr2=0x7ff858073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85cdf5700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff858101ea0 0x7ff858073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85cdf5700 1 -- 172.21.15.42:0/1179932294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff854009d00 con 0x7ff858100ca0 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85cdf5700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff858074b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8540077b0 tx=0x7ff854003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff84dffb700 1 -- 172.21.15.42:0/1179932294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff85400e040 con 0x7ff858100ca0 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff84dffb700 1 -- 172.21.15.42:0/1179932294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff85400ce60 con 0x7ff858100ca0 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff858073a80 con 0x7ff858100ca0 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff84dffb700 1 -- 172.21.15.42:0/1179932294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff85401db10 con 0x7ff858100ca0 2024-09-16T23:45:59.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.692+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff858073f70 con 0x7ff858100ca0 2024-09-16T23:45:59.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.693+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff858109800 con 0x7ff858100ca0 2024-09-16T23:45:59.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.694+0000 7ff84dffb700 1 -- 172.21.15.42:0/1179932294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7ff85401dc90 con 0x7ff858100ca0 2024-09-16T23:45:59.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.694+0000 7ff84dffb700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff84006c4e0 0x7ff84006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:45:59.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.694+0000 7ff84dffb700 1 -- 172.21.15.42:0/1179932294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7ff85408ce20 con 0x7ff858100ca0 2024-09-16T23:45:59.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.695+0000 7ff84ffff700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff84006c4e0 0x7ff84006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:45:59.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.695+0000 7ff84ffff700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff84006c4e0 0x7ff84006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff84800ba60 tx=0x7ff848005d50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:45:59.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.699+0000 7ff84dffb700 1 -- 172.21.15.42:0/1179932294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff85405b4f0 con 0x7ff858100ca0 2024-09-16T23:45:59.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.905+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff85819e950 con 0x7ff84006c4e0 2024-09-16T23:45:59.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.907+0000 7ff84dffb700 1 -- 172.21.15.42:0/1179932294 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7ff85819e950 con 0x7ff84006c4e0 2024-09-16T23:45:59.904 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2024-09-16T23:45:59.904 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2024-09-16T23:45:59.904 INFO:teuthology.orchestra.run.smithi042.stdout:smithi043 172.21.15.43 2024-09-16T23:45:59.904 INFO:teuthology.orchestra.run.smithi042.stdout:2 hosts in cluster 2024-09-16T23:45:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff84006c4e0 msgr2=0x7ff84006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff84006c4e0 0x7ff84006e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff84800ba60 tx=0x7ff848005d50 comp rx=0 tx=0).stop 2024-09-16T23:45:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 msgr2=0x7ff858074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:45:59.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff858074b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff8540077b0 tx=0x7ff854003ce0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 shutdown_connections 2024-09-16T23:45:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff84006c4e0 0x7ff84006e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff858100ca0 0x7ff858074b30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 --2- 172.21.15.42:0/1179932294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff858101ea0 0x7ff858073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:45:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 >> 172.21.15.42:0/1179932294 conn(0x7ff8580fc260 msgr2=0x7ff8581050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:45:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 shutdown_connections 2024-09-16T23:45:59.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:45:59.909+0000 7ff85f059700 1 -- 172.21.15.42:0/1179932294 wait complete. 2024-09-16T23:46:00.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T23:46:00.961 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:01.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:00 smithi042 ceph-mon[27914]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:01.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:00 smithi042 ceph-mon[27914]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:46:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:00 smithi043 ceph-mon[35191]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:00 smithi043 ceph-mon[35191]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 -- 172.21.15.42:0/3536792100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 msgr2=0x7f611c0ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 --2- 172.21.15.42:0/3536792100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c0ff380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6118009910 tx=0x7f6118009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 -- 172.21.15.42:0/3536792100 shutdown_connections 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 --2- 172.21.15.42:0/3536792100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c0ffcd0 0x7f611c100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 --2- 172.21.15.42:0/3536792100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c0ff380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 -- 172.21.15.42:0/3536792100 >> 172.21.15.42:0/3536792100 conn(0x7f611c0fa8c0 msgr2=0x7f611c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 -- 172.21.15.42:0/3536792100 shutdown_connections 2024-09-16T23:46:02.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 -- 172.21.15.42:0/3536792100 wait complete. 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 Processor -- start 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.606+0000 7f6123bf2700 1 -- start start 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6123bf2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6123bf2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c0ffcd0 0x7f611c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6123bf2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c199300 con 0x7f611c0fef60 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6123bf2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c199440 con 0x7f611c0ffcd0 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612198e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612198e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37424/0 (socket says 172.21.15.42:37424) 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612198e700 1 -- 172.21.15.42:0/2030853028 learned_addr learned my addr 172.21.15.42:0/2030853028 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612118d700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c0ffcd0 0x7f611c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612198e700 1 -- 172.21.15.42:0/2030853028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c0ffcd0 msgr2=0x7f611c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612198e700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c0ffcd0 0x7f611c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:02.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612198e700 1 -- 172.21.15.42:0/2030853028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180095f0 con 0x7f611c0fef60 2024-09-16T23:46:02.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f612198e700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c1987a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61180092e0 tx=0x7f611800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:02.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6112ffd700 1 -- 172.21.15.42:0/2030853028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f611801d070 con 0x7f611c0fef60 2024-09-16T23:46:02.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6112ffd700 1 -- 172.21.15.42:0/2030853028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f611800f460 con 0x7f611c0fef60 2024-09-16T23:46:02.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c19de90 con 0x7f611c0fef60 2024-09-16T23:46:02.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6112ffd700 1 -- 172.21.15.42:0/2030853028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6118021640 con 0x7f611c0fef60 2024-09-16T23:46:02.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.607+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c19e300 con 0x7f611c0fef60 2024-09-16T23:46:02.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.608+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c10ba90 con 0x7f611c0fef60 2024-09-16T23:46:02.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.609+0000 7f6112ffd700 1 -- 172.21.15.42:0/2030853028 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f611800fa90 con 0x7f611c0fef60 2024-09-16T23:46:02.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.609+0000 7f6112ffd700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f610806c4e0 0x7f610806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:02.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.610+0000 7f6112ffd700 1 -- 172.21.15.42:0/2030853028 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f611808c4e0 con 0x7f611c0fef60 2024-09-16T23:46:02.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.610+0000 7f612118d700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f610806c4e0 0x7f610806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:02.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.610+0000 7f612118d700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f610806c4e0 0x7f610806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f610c006fd0 tx=0x7f610c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:02.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.614+0000 7f6112ffd700 1 -- 172.21.15.42:0/2030853028 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f611805ac60 con 0x7f611c0fef60 2024-09-16T23:46:02.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.822+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f611c109a80 con 0x7f610806c4e0 2024-09-16T23:46:02.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.823+0000 7f6112ffd700 1 -- 172.21.15.42:0/2030853028 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f611c109a80 con 0x7f610806c4e0 2024-09-16T23:46:02.822 INFO:teuthology.orchestra.run.smithi042.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T23:46:02.822 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130021480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-16T23:46:02.822 INFO:teuthology.orchestra.run.smithi042.stdout:smithi043 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300GH480DGN 447G No 70s ago Has a FileSystem, LVM detected 2024-09-16T23:46:02.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f610806c4e0 msgr2=0x7f610806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:02.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f610806c4e0 0x7f610806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f610c006fd0 tx=0x7f610c008040 comp rx=0 tx=0).stop 2024-09-16T23:46:02.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 msgr2=0x7f611c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:02.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c1987a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f61180092e0 tx=0x7f611800be20 comp rx=0 tx=0).stop 2024-09-16T23:46:02.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 shutdown_connections 2024-09-16T23:46:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f610806c4e0 0x7f610806e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f611c0fef60 0x7f611c1987a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 --2- 172.21.15.42:0/2030853028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f611c0ffcd0 0x7f611c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 >> 172.21.15.42:0/2030853028 conn(0x7f611c0fa8c0 msgr2=0x7f611c108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 shutdown_connections 2024-09-16T23:46:02.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:02.826+0000 7f6123bf2700 1 -- 172.21.15.42:0/2030853028 wait complete. 2024-09-16T23:46:03.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:02 smithi042 ceph-mon[27914]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:03.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:02 smithi043 ceph-mon[35191]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:03.492 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T23:46:03.501 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:46:03.501 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-09-16T23:46:03.715 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:04.078 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:03 smithi042 ceph-mon[27914]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:46:04.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:03 smithi043 ceph-mon[35191]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:46:04.809 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:04 smithi042 ceph-mon[27914]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:04 smithi043 ceph-mon[35191]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:05.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 -- 172.21.15.42:0/1741114375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 msgr2=0x7fa4900ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:05.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 --2- 172.21.15.42:0/1741114375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa4900ffeb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009910 tx=0x7fa48c009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:05.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 -- 172.21.15.42:0/1741114375 shutdown_connections 2024-09-16T23:46:05.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 --2- 172.21.15.42:0/1741114375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490100c90 0x7fa4901010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:05.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 --2- 172.21.15.42:0/1741114375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa4900ffeb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 -- 172.21.15.42:0/1741114375 >> 172.21.15.42:0/1741114375 conn(0x7fa4900fb050 msgr2=0x7fa4900fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 -- 172.21.15.42:0/1741114375 shutdown_connections 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.325+0000 7fa497db1700 1 -- 172.21.15.42:0/1741114375 wait complete. 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa497db1700 1 Processor -- start 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa497db1700 1 -- start start 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa497db1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa490196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa497db1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490100c90 0x7fa490196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa497db1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490196e70 con 0x7fa4900ffa90 2024-09-16T23:46:05.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa497db1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490196fb0 con 0x7fa490100c90 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa495b4d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa490196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa495b4d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa490196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37430/0 (socket says 172.21.15.42:37430) 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa495b4d700 1 -- 172.21.15.42:0/2074896714 learned_addr learned my addr 172.21.15.42:0/2074896714 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa49534c700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490100c90 0x7fa490196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa495b4d700 1 -- 172.21.15.42:0/2074896714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490100c90 msgr2=0x7fa490196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa495b4d700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490100c90 0x7fa490196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa495b4d700 1 -- 172.21.15.42:0/2074896714 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0095f0 con 0x7fa4900ffa90 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.326+0000 7fa495b4d700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa490196310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000c00 tx=0x7fa48c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.327+0000 7fa486ffd700 1 -- 172.21.15.42:0/2074896714 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa48c01d070 con 0x7fa4900ffa90 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.327+0000 7fa486ffd700 1 -- 172.21.15.42:0/2074896714 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa48c005dc0 con 0x7fa4900ffa90 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.327+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49019ba00 con 0x7fa4900ffa90 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.327+0000 7fa486ffd700 1 -- 172.21.15.42:0/2074896714 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa48c022470 con 0x7fa4900ffa90 2024-09-16T23:46:05.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.327+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49019bef0 con 0x7fa4900ffa90 2024-09-16T23:46:05.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.328+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4901085f0 con 0x7fa4900ffa90 2024-09-16T23:46:05.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.328+0000 7fa486ffd700 1 -- 172.21.15.42:0/2074896714 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa48c003810 con 0x7fa4900ffa90 2024-09-16T23:46:05.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.329+0000 7fa486ffd700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa47c06c4e0 0x7fa47c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:05.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.329+0000 7fa486ffd700 1 -- 172.21.15.42:0/2074896714 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fa48c08c9c0 con 0x7fa4900ffa90 2024-09-16T23:46:05.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.329+0000 7fa49534c700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa47c06c4e0 0x7fa47c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:05.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.329+0000 7fa49534c700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa47c06c4e0 0x7fa47c06e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa480009c60 tx=0x7fa480009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:05.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.333+0000 7fa486ffd700 1 -- 172.21.15.42:0/2074896714 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa48c05b010 con 0x7fa4900ffa90 2024-09-16T23:46:05.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:05.562+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa490069520 con 0x7fa47c06c4e0 2024-09-16T23:46:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:06 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-16T23:46:06.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:06 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-16T23:46:07.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.075+0000 7fa486ffd700 1 -- 172.21.15.42:0/2074896714 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa490069520 con 0x7fa47c06c4e0 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa47c06c4e0 msgr2=0x7fa47c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa47c06c4e0 0x7fa47c06e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa480009c60 tx=0x7fa480009260 comp rx=0 tx=0).stop 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 msgr2=0x7fa490196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa490196310 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa48c000c00 tx=0x7fa48c00f690 comp rx=0 tx=0).stop 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 shutdown_connections 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa47c06c4e0 0x7fa47c06e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4900ffa90 0x7fa490196310 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 --2- 172.21.15.42:0/2074896714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa490100c90 0x7fa490196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.078+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 >> 172.21.15.42:0/2074896714 conn(0x7fa4900fb050 msgr2=0x7fa490068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.079+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 shutdown_connections 2024-09-16T23:46:07.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:07.079+0000 7fa497db1700 1 -- 172.21.15.42:0/2074896714 wait complete. 2024-09-16T23:46:07.090 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[27910]: 2024-09-16T23:46:07.053+0000 7ff77c793700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T23:46:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:46:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-16T23:46:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T23:46:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-16T23:46:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-16T23:46:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: fsmap cephfs:0 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: Saving service mds.cephfs spec with placement count:4 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:07.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:07.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:07.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:07 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:07.830 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T23:46:07.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:07.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:46:07.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-16T23:46:07.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T23:46:07.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-16T23:46:07.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: fsmap cephfs:0 2024-09-16T23:46:07.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: Saving service mds.cephfs spec with placement count:4 2024-09-16T23:46:07.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:07.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:46:07.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:07.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:46:07.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:07.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:07.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:07.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:07 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:09.003 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:09.243 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:09 smithi042 ceph-mon[27914]: Deploying daemon mds.cephfs.smithi042.ctckas on smithi042 2024-09-16T23:46:09.243 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:09 smithi042 ceph-mon[27914]: pgmap v180: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:09.243 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:09 smithi042 ceph-mon[27914]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T23:46:09.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:09 smithi043 ceph-mon[35191]: Deploying daemon mds.cephfs.smithi042.ctckas on smithi042 2024-09-16T23:46:09.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:09 smithi043 ceph-mon[35191]: pgmap v180: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:09.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:09 smithi043 ceph-mon[35191]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T23:46:10.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:10 smithi042 ceph-mon[27914]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T23:46:10.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:10.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:10 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:10.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:10 smithi043 ceph-mon[35191]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T23:46:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:10 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:11.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3733113284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 msgr2=0x7f34a80731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:11.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3733113284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a80731a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3498009910 tx=0x7f3498009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:11.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3733113284 shutdown_connections 2024-09-16T23:46:11.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3733113284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34a8073770 0x7f34a8073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3733113284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a80731a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3733113284 >> 172.21.15.42:0/3733113284 conn(0x7f34a80fc230 msgr2=0x7f34a80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3733113284 shutdown_connections 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.020+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3733113284 wait complete. 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34ac9a9700 1 Processor -- start 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34ac9a9700 1 -- start start 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34ac9a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34a8073770 0x7f34a81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34ac9a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34ac9a9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8199200 con 0x7f34a8074d40 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34ac9a9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a8199340 con 0x7f34a8073770 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:11.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a8198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37452/0 (socket says 172.21.15.42:37452) 2024-09-16T23:46:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a5d9b700 1 -- 172.21.15.42:0/3559869531 learned_addr learned my addr 172.21.15.42:0/3559869531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a659c700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34a8073770 0x7f34a81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a5d9b700 1 -- 172.21.15.42:0/3559869531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34a8073770 msgr2=0x7f34a81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a5d9b700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34a8073770 0x7f34a81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a5d9b700 1 -- 172.21.15.42:0/3559869531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34980095f0 con 0x7f34a8074d40 2024-09-16T23:46:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.021+0000 7f34a5d9b700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a8198be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f349c00c8a0 tx=0x7f349c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.022+0000 7f34977fe700 1 -- 172.21.15.42:0/3559869531 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f349c0078e0 con 0x7f34a8074d40 2024-09-16T23:46:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.022+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a819ddf0 con 0x7f34a8074d40 2024-09-16T23:46:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.022+0000 7f34977fe700 1 -- 172.21.15.42:0/3559869531 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f349c00f450 con 0x7f34a8074d40 2024-09-16T23:46:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.022+0000 7f34977fe700 1 -- 172.21.15.42:0/3559869531 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f349c00e5e0 con 0x7f34a8074d40 2024-09-16T23:46:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.022+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a819e310 con 0x7f34a8074d40 2024-09-16T23:46:11.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.023+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a810ba00 con 0x7f34a8074d40 2024-09-16T23:46:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.023+0000 7f34977fe700 1 -- 172.21.15.42:0/3559869531 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f349c00f5c0 con 0x7f34a8074d40 2024-09-16T23:46:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.024+0000 7f34977fe700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f349006c4e0 0x7f349006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.024+0000 7f34977fe700 1 -- 172.21.15.42:0/3559869531 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f349c08b700 con 0x7f34a8074d40 2024-09-16T23:46:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.024+0000 7f34a659c700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f349006c4e0 0x7f349006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:11.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.024+0000 7f34a659c700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f349006c4e0 0x7f349006e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3498006010 tx=0x7f34980058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:11.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.028+0000 7f34977fe700 1 -- 172.21.15.42:0/3559869531 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f349c059a20 con 0x7f34a8074d40 2024-09-16T23:46:11.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:11 smithi043 ceph-mon[35191]: pgmap v183: 65 pgs: 20 active+clean, 19 creating+peering, 26 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:11.175 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:11 smithi042 ceph-mon[27914]: pgmap v183: 65 pgs: 20 active+clean, 19 creating+peering, 26 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:11.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.277+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f34a810bbf0 con 0x7f34a8074d40 2024-09-16T23:46:11.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.278+0000 7f34977fe700 1 -- 172.21.15.42:0/3559869531 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f349c0595b0 con 0x7f34a8074d40 2024-09-16T23:46:11.276 INFO:teuthology.orchestra.run.smithi042.stdout:e2 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 2 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:46:07.055473+0000 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:46:11.277 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 1 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:in 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:46:11.278 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:11.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:11.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f349006c4e0 msgr2=0x7f349006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:11.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f349006c4e0 0x7f349006e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3498006010 tx=0x7f34980058e0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 msgr2=0x7f34a8198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:11.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a8198be0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f349c00c8a0 tx=0x7f349c00cbb0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 shutdown_connections 2024-09-16T23:46:11.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f349006c4e0 0x7f349006e9a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34a8073770 0x7f34a81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 --2- 172.21.15.42:0/3559869531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34a8074d40 0x7f34a8198be0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 >> 172.21.15.42:0/3559869531 conn(0x7f34a80fc230 msgr2=0x7f34a8106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 shutdown_connections 2024-09-16T23:46:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:11.280+0000 7f34ac9a9700 1 -- 172.21.15.42:0/3559869531 wait complete. 2024-09-16T23:46:11.281 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-09-16T23:46:12.071 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T23:46:12.080 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:46:12.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-09-16T23:46:12.276 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3559869531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:46:12.276 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: daemon mds.cephfs.smithi042.ctckas assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: Cluster is now healthy 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6826/2136666874,v1:172.21.15.42:6827/2136666874] up:boot 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: fsmap cephfs:1 {0=cephfs.smithi042.ctckas=up:creating} 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:46:12.277 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:12 smithi043 ceph-mon[35191]: daemon mds.cephfs.smithi042.ctckas is now active in filesystem cephfs as rank 0 2024-09-16T23:46:12.289 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:12.410 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3559869531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: daemon mds.cephfs.smithi042.ctckas assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: Cluster is now healthy 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6826/2136666874,v1:172.21.15.42:6827/2136666874] up:boot 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: fsmap cephfs:1 {0=cephfs.smithi042.ctckas=up:creating} 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:46:12.411 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:12 smithi042 ceph-mon[27914]: daemon mds.cephfs.smithi042.ctckas is now active in filesystem cephfs as rank 0 2024-09-16T23:46:13.296 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:13 smithi043 ceph-mon[35191]: pgmap v184: 65 pgs: 42 active+clean, 19 creating+peering, 4 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:13.297 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:13 smithi043 ceph-mon[35191]: Deploying daemon mds.cephfs.smithi043.pzcdzg on smithi043 2024-09-16T23:46:13.297 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:13 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6826/2136666874,v1:172.21.15.42:6827/2136666874] up:active 2024-09-16T23:46:13.297 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:13 smithi043 ceph-mon[35191]: fsmap cephfs:1 {0=cephfs.smithi042.ctckas=up:active} 2024-09-16T23:46:13.373 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:13 smithi042 ceph-mon[27914]: pgmap v184: 65 pgs: 42 active+clean, 19 creating+peering, 4 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:13.373 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:13 smithi042 ceph-mon[27914]: Deploying daemon mds.cephfs.smithi043.pzcdzg on smithi043 2024-09-16T23:46:13.374 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:13 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6826/2136666874,v1:172.21.15.42:6827/2136666874] up:active 2024-09-16T23:46:13.374 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:13 smithi042 ceph-mon[27914]: fsmap cephfs:1 {0=cephfs.smithi042.ctckas=up:active} 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 -- 172.21.15.42:0/860985569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c100dc0 msgr2=0x7f707c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 --2- 172.21.15.42:0/860985569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c100dc0 0x7f707c101220 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f706c009910 tx=0x7f706c009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 -- 172.21.15.42:0/860985569 shutdown_connections 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 --2- 172.21.15.42:0/860985569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c100dc0 0x7f707c101220 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 --2- 172.21.15.42:0/860985569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f707c0ffbc0 0x7f707c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 -- 172.21.15.42:0/860985569 >> 172.21.15.42:0/860985569 conn(0x7f707c0fb160 msgr2=0x7f707c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 -- 172.21.15.42:0/860985569 shutdown_connections 2024-09-16T23:46:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.875+0000 7f7083306700 1 -- 172.21.15.42:0/860985569 wait complete. 2024-09-16T23:46:13.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f7083306700 1 Processor -- start 2024-09-16T23:46:13.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f7083306700 1 -- start start 2024-09-16T23:46:13.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f7083306700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0ffbc0 0x7f707c1956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:13.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f7083306700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f707c100dc0 0x7f707c195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:13.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f7083306700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c196220 con 0x7f707c0ffbc0 2024-09-16T23:46:13.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f7083306700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c196360 con 0x7f707c100dc0 2024-09-16T23:46:13.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f70810a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0ffbc0 0x7f707c1956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f70810a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0ffbc0 0x7f707c1956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53878/0 (socket says 172.21.15.42:53878) 2024-09-16T23:46:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f70810a2700 1 -- 172.21.15.42:0/3251104908 learned_addr learned my addr 172.21.15.42:0/3251104908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f70808a1700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f707c100dc0 0x7f707c195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f70810a2700 1 -- 172.21.15.42:0/3251104908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f707c100dc0 msgr2=0x7f707c195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f70810a2700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f707c100dc0 0x7f707c195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.876+0000 7f70810a2700 1 -- 172.21.15.42:0/3251104908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0095f0 con 0x7f707c0ffbc0 2024-09-16T23:46:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.877+0000 7f70810a2700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0ffbc0 0x7f707c1956c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f707800d900 tx=0x7f707800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.877+0000 7f70727fc700 1 -- 172.21.15.42:0/3251104908 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707800b920 con 0x7f707c0ffbc0 2024-09-16T23:46:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.877+0000 7f70727fc700 1 -- 172.21.15.42:0/3251104908 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7078010460 con 0x7f707c0ffbc0 2024-09-16T23:46:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.877+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c19ae10 con 0x7f707c0ffbc0 2024-09-16T23:46:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.877+0000 7f70727fc700 1 -- 172.21.15.42:0/3251104908 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707800f590 con 0x7f707c0ffbc0 2024-09-16T23:46:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.877+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c19b2d0 con 0x7f707c0ffbc0 2024-09-16T23:46:13.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.878+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c18f530 con 0x7f707c0ffbc0 2024-09-16T23:46:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.878+0000 7f70727fc700 1 -- 172.21.15.42:0/3251104908 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f70780105d0 con 0x7f707c0ffbc0 2024-09-16T23:46:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.879+0000 7f70727fc700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f706806c4e0 0x7f706806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.879+0000 7f70727fc700 1 -- 172.21.15.42:0/3251104908 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f707808b660 con 0x7f707c0ffbc0 2024-09-16T23:46:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.879+0000 7f70808a1700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f706806c4e0 0x7f706806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:13.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.879+0000 7f70808a1700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f706806c4e0 0x7f706806e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f706c006010 tx=0x7f706c00b580 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:13.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:13.883+0000 7f70727fc700 1 -- 172.21.15.42:0/3251104908 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7078059980 con 0x7f707c0ffbc0 2024-09-16T23:46:14.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:14.126+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f707c02cda0 con 0x7f707c0ffbc0 2024-09-16T23:46:15.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.075+0000 7f70727fc700 1 -- 172.21.15.42:0/3251104908 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7078059510 con 0x7f707c0ffbc0 2024-09-16T23:46:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f706806c4e0 msgr2=0x7f706806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f706806c4e0 0x7f706806e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f706c006010 tx=0x7f706c00b580 comp rx=0 tx=0).stop 2024-09-16T23:46:15.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0ffbc0 msgr2=0x7f707c1956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0ffbc0 0x7f707c1956c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f707800d900 tx=0x7f707800dcc0 comp rx=0 tx=0).stop 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 shutdown_connections 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f706806c4e0 0x7f706806e9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f707c0ffbc0 0x7f707c1956c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 --2- 172.21.15.42:0/3251104908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f707c100dc0 0x7f707c195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 >> 172.21.15.42:0/3251104908 conn(0x7f707c0fb160 msgr2=0x7f707c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 shutdown_connections 2024-09-16T23:46:15.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:15.078+0000 7f7083306700 1 -- 172.21.15.42:0/3251104908 wait complete. 2024-09-16T23:46:15.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: pgmap v185: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1011 B/s wr, 6 op/s 2024-09-16T23:46:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3251104908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-16T23:46:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:15 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:15.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: pgmap v185: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1011 B/s wr, 6 op/s 2024-09-16T23:46:15.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3251104908' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-16T23:46:15.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:15.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:15.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:15.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:15.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:15.445 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:15 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: Deploying daemon mds.cephfs.smithi042.anvvuy on smithi042 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3251104908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] up:boot 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: daemon mds.cephfs.smithi043.pzcdzg assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: fsmap cephfs:1 {0=cephfs.smithi042.ctckas=up:active} 1 up:standby 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:creating} 2024-09-16T23:46:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:16 smithi043 ceph-mon[35191]: daemon mds.cephfs.smithi043.pzcdzg is now active in filesystem cephfs as rank 1 2024-09-16T23:46:16.186 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T23:46:16.197 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:46:16.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-09-16T23:46:16.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: Deploying daemon mds.cephfs.smithi042.anvvuy on smithi042 2024-09-16T23:46:16.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T23:46:16.313 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3251104908' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] up:boot 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: daemon mds.cephfs.smithi043.pzcdzg assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: fsmap cephfs:1 {0=cephfs.smithi042.ctckas=up:active} 1 up:standby 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:creating} 2024-09-16T23:46:16.314 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:16 smithi042 ceph-mon[27914]: daemon mds.cephfs.smithi043.pzcdzg is now active in filesystem cephfs as rank 1 2024-09-16T23:46:16.763 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:17.136 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:17 smithi042 ceph-mon[27914]: pgmap v186: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 4 op/s 2024-09-16T23:46:17.136 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:17 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] up:active 2024-09-16T23:46:17.136 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:17 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2024-09-16T23:46:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:17 smithi043 ceph-mon[35191]: pgmap v186: 65 pgs: 65 active+clean; 578 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 4 op/s 2024-09-16T23:46:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:17 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] up:active 2024-09-16T23:46:17.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:17 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2024-09-16T23:46:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/2530548443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 msgr2=0x7fdd381010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/2530548443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381010d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009910 tx=0x7fdd34009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:18.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/2530548443 shutdown_connections 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/2530548443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38101eb0 0x7fdd38102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/2530548443 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381010d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/2530548443 >> 172.21.15.42:0/2530548443 conn(0x7fdd380fc230 msgr2=0x7fdd380fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/2530548443 shutdown_connections 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/2530548443 wait complete. 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.557+0000 7fdd3ffe8700 1 Processor -- start 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3ffe8700 1 -- start start 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3ffe8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3ffe8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38101eb0 0x7fdd38198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3ffe8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38199250 con 0x7fdd38100cb0 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3ffe8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd38199390 con 0x7fdd38101eb0 2024-09-16T23:46:18.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3dd84700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3dd84700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53896/0 (socket says 172.21.15.42:53896) 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3dd84700 1 -- 172.21.15.42:0/625983188 learned_addr learned my addr 172.21.15.42:0/625983188 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3d583700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38101eb0 0x7fdd38198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3dd84700 1 -- 172.21.15.42:0/625983188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38101eb0 msgr2=0x7fdd38198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3dd84700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38101eb0 0x7fdd38198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3dd84700 1 -- 172.21.15.42:0/625983188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd340095f0 con 0x7fdd38100cb0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.558+0000 7fdd3dd84700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381986f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400bd70 tx=0x7fdd3400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.559+0000 7fdd2effd700 1 -- 172.21.15.42:0/625983188 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd3401d070 con 0x7fdd38100cb0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.559+0000 7fdd2effd700 1 -- 172.21.15.42:0/625983188 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdd3400f460 con 0x7fdd38100cb0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.559+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3819dde0 con 0x7fdd38100cb0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.559+0000 7fdd2effd700 1 -- 172.21.15.42:0/625983188 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd34021640 con 0x7fdd38100cb0 2024-09-16T23:46:18.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.559+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3819e270 con 0x7fdd38100cb0 2024-09-16T23:46:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.560+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd38105f90 con 0x7fdd38100cb0 2024-09-16T23:46:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.560+0000 7fdd2effd700 1 -- 172.21.15.42:0/625983188 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fdd3400fab0 con 0x7fdd38100cb0 2024-09-16T23:46:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.561+0000 7fdd2effd700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdd2406c4e0 0x7fdd2406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:18.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.561+0000 7fdd2effd700 1 -- 172.21.15.42:0/625983188 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdd3408c980 con 0x7fdd38100cb0 2024-09-16T23:46:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.561+0000 7fdd3d583700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdd2406c4e0 0x7fdd2406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:18.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.561+0000 7fdd3d583700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdd2406c4e0 0x7fdd2406e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdd28004630 tx=0x7fdd2800b3c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:18.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.565+0000 7fdd2effd700 1 -- 172.21.15.42:0/625983188 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdd3405aca0 con 0x7fdd38100cb0 2024-09-16T23:46:18.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:18.815+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdd3802cc70 con 0x7fdd38100cb0 2024-09-16T23:46:19.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.087+0000 7fdd2effd700 1 -- 172.21.15.42:0/625983188 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd3405a830 con 0x7fdd38100cb0 2024-09-16T23:46:19.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdd2406c4e0 msgr2=0x7fdd2406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdd2406c4e0 0x7fdd2406e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdd28004630 tx=0x7fdd2800b3c0 comp rx=0 tx=0).stop 2024-09-16T23:46:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 msgr2=0x7fdd381986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381986f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400bd70 tx=0x7fdd3400bda0 comp rx=0 tx=0).stop 2024-09-16T23:46:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 shutdown_connections 2024-09-16T23:46:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fdd2406c4e0 0x7fdd2406e9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:19.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd38100cb0 0x7fdd381986f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 --2- 172.21.15.42:0/625983188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd38101eb0 0x7fdd38198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 >> 172.21.15.42:0/625983188 conn(0x7fdd380fc230 msgr2=0x7fdd381030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 shutdown_connections 2024-09-16T23:46:19.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:19.090+0000 7fdd3ffe8700 1 -- 172.21.15.42:0/625983188 wait complete. 2024-09-16T23:46:19.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-16T23:46:19.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/625983188' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-09-16T23:46:19.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:19.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:19.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:19.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:19.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:19.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:19 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 6 op/s 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/625983188' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T23:46:19.383 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:19 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:19.734 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T23:46:19.744 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:46:19.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-09-16T23:46:19.960 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:20.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: Deploying daemon mds.cephfs.smithi043.riwwhn on smithi043 2024-09-16T23:46:20.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T23:46:20.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: Cluster is now healthy 2024-09-16T23:46:20.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/625983188' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-09-16T23:46:20.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:boot 2024-09-16T23:46:20.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 1 up:standby 2024-09-16T23:46:20.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:46:20.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:20 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: Deploying daemon mds.cephfs.smithi043.riwwhn on smithi043 2024-09-16T23:46:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T23:46:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: Cluster is now healthy 2024-09-16T23:46:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/625983188' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-09-16T23:46:20.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:boot 2024-09-16T23:46:20.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 1 up:standby 2024-09-16T23:46:20.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:46:20.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:20 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:21.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:21 smithi042 ceph-mon[27914]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-16T23:46:21.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:21 smithi043 ceph-mon[35191]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-16T23:46:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 -- 172.21.15.42:0/287980648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef70100db0 msgr2=0x7fef70101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 --2- 172.21.15.42:0/287980648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef70100db0 0x7fef70101210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fef60009910 tx=0x7fef60009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 -- 172.21.15.42:0/287980648 shutdown_connections 2024-09-16T23:46:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 --2- 172.21.15.42:0/287980648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef70100db0 0x7fef70101210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 --2- 172.21.15.42:0/287980648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef700ffbb0 0x7fef700fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 -- 172.21.15.42:0/287980648 >> 172.21.15.42:0/287980648 conn(0x7fef700fb150 msgr2=0x7fef700fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 -- 172.21.15.42:0/287980648 shutdown_connections 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.575+0000 7fef77f77700 1 -- 172.21.15.42:0/287980648 wait complete. 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef77f77700 1 Processor -- start 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef77f77700 1 -- start start 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef77f77700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef700ffbb0 0x7fef70069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef77f77700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef70100db0 0x7fef7019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef77f77700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7019b7c0 con 0x7fef700ffbb0 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef77f77700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7019b930 con 0x7fef70100db0 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef75d13700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef700ffbb0 0x7fef70069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef75d13700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef700ffbb0 0x7fef70069990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53938/0 (socket says 172.21.15.42:53938) 2024-09-16T23:46:21.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.576+0000 7fef75d13700 1 -- 172.21.15.42:0/3879995732 learned_addr learned my addr 172.21.15.42:0/3879995732 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef75512700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef70100db0 0x7fef7019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef75d13700 1 -- 172.21.15.42:0/3879995732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef70100db0 msgr2=0x7fef7019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef75d13700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef70100db0 0x7fef7019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef75d13700 1 -- 172.21.15.42:0/3879995732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef600095f0 con 0x7fef700ffbb0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef75512700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef70100db0 0x7fef7019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef75d13700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef700ffbb0 0x7fef70069990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00d900 tx=0x7fef6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef66ffd700 1 -- 172.21.15.42:0/3879995732 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6c0041f0 con 0x7fef700ffbb0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef66ffd700 1 -- 172.21.15.42:0/3879995732 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef6c004350 con 0x7fef700ffbb0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7019bc10 con 0x7fef700ffbb0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef66ffd700 1 -- 172.21.15.42:0/3879995732 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6c003da0 con 0x7fef700ffbb0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.577+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7019c160 con 0x7fef700ffbb0 2024-09-16T23:46:21.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.578+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef70108710 con 0x7fef700ffbb0 2024-09-16T23:46:21.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.579+0000 7fef66ffd700 1 -- 172.21.15.42:0/3879995732 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fef6c009730 con 0x7fef700ffbb0 2024-09-16T23:46:21.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.579+0000 7fef66ffd700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fef5c06c4e0 0x7fef5c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:21.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.579+0000 7fef66ffd700 1 -- 172.21.15.42:0/3879995732 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fef6c08b8e0 con 0x7fef700ffbb0 2024-09-16T23:46:21.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.579+0000 7fef75512700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fef5c06c4e0 0x7fef5c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:21.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.580+0000 7fef75512700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fef5c06c4e0 0x7fef5c06e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef60006010 tx=0x7fef6000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:21.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.584+0000 7fef66ffd700 1 -- 172.21.15.42:0/3879995732 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef6c059c00 con 0x7fef700ffbb0 2024-09-16T23:46:21.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:21.833+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef70108900 con 0x7fef700ffbb0 2024-09-16T23:46:22.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.634+0000 7fef66ffd700 1 -- 172.21.15.42:0/3879995732 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v9) v1 ==== 168+0+0 (secure 0 0 0) 0x7fef6c059790 con 0x7fef700ffbb0 2024-09-16T23:46:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fef5c06c4e0 msgr2=0x7fef5c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fef5c06c4e0 0x7fef5c06e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fef60006010 tx=0x7fef6000b540 comp rx=0 tx=0).stop 2024-09-16T23:46:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef700ffbb0 msgr2=0x7fef70069990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef700ffbb0 0x7fef70069990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00d900 tx=0x7fef6c00dcc0 comp rx=0 tx=0).stop 2024-09-16T23:46:22.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 shutdown_connections 2024-09-16T23:46:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fef5c06c4e0 0x7fef5c06e9a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef700ffbb0 0x7fef70069990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 --2- 172.21.15.42:0/3879995732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef70100db0 0x7fef7019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 >> 172.21.15.42:0/3879995732 conn(0x7fef700fb150 msgr2=0x7fef70103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 shutdown_connections 2024-09-16T23:46:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:22.637+0000 7fef77f77700 1 -- 172.21.15.42:0/3879995732 wait complete. 2024-09-16T23:46:22.635 INFO:teuthology.orchestra.run.smithi042.stderr:inline data enabled 2024-09-16T23:46:22.897 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-16T23:46:22.897 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3879995732' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T23:46:22.897 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:22 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3879995732' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:22.912 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:22 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:46:23.398 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T23:46:23.409 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:46:23.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T23:46:23.859 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:23.927 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:23 smithi043 ceph-mon[35191]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-16T23:46:23.927 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:23 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] up:boot 2024-09-16T23:46:23.927 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:23 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3879995732' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T23:46:23.927 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:23 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:23.927 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:23 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:46:23.982 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:23 smithi042 ceph-mon[27914]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-16T23:46:23.982 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:23 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] up:boot 2024-09-16T23:46:23.982 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:23 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3879995732' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T23:46:23.982 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:23 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:23.982 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:23 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:standby 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] up:active 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: Dropping low affinity active daemon mds.cephfs.smithi042.ctckas in favor of higher affinity standby. 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: Replacing daemon mds.cephfs.smithi042.ctckas as rank 0 with standby daemon mds.cephfs.smithi042.anvvuy 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:replay,1=cephfs.smithi043.pzcdzg=up:active} 1 up:standby 2024-09-16T23:46:24.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:24.994 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:24.994 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:24.994 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:24.994 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:standby 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] up:active 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: Dropping low affinity active daemon mds.cephfs.smithi042.ctckas in favor of higher affinity standby. 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: Replacing daemon mds.cephfs.smithi042.ctckas as rank 0 with standby daemon mds.cephfs.smithi042.anvvuy 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T23:46:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:replay,1=cephfs.smithi043.pzcdzg=up:active} 1 up:standby 2024-09-16T23:46:25.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:25.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:25.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:25.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:25.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 -- 172.21.15.42:0/1741538274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa064100dc0 msgr2=0x7fa064101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 --2- 172.21.15.42:0/1741538274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa064100dc0 0x7fa064101220 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa054009910 tx=0x7fa054009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 -- 172.21.15.42:0/1741538274 shutdown_connections 2024-09-16T23:46:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 --2- 172.21.15.42:0/1741538274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa064100dc0 0x7fa064101220 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 --2- 172.21.15.42:0/1741538274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0640ffbc0 0x7fa0640fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 -- 172.21.15.42:0/1741538274 >> 172.21.15.42:0/1741538274 conn(0x7fa0640fb160 msgr2=0x7fa0640fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:25.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 -- 172.21.15.42:0/1741538274 shutdown_connections 2024-09-16T23:46:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.591+0000 7fa06be32700 1 -- 172.21.15.42:0/1741538274 wait complete. 2024-09-16T23:46:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa06be32700 1 Processor -- start 2024-09-16T23:46:25.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa06be32700 1 -- start start 2024-09-16T23:46:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa06be32700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0640ffbc0 0x7fa064069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa06be32700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa064100dc0 0x7fa06419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa06be32700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06419baf0 con 0x7fa0640ffbc0 2024-09-16T23:46:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa06be32700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06419bc30 con 0x7fa064100dc0 2024-09-16T23:46:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa069bce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0640ffbc0 0x7fa064069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa069bce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0640ffbc0 0x7fa064069970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33258/0 (socket says 172.21.15.42:33258) 2024-09-16T23:46:25.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa069bce700 1 -- 172.21.15.42:0/204072903 learned_addr learned my addr 172.21.15.42:0/204072903 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.592+0000 7fa0693cd700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa064100dc0 0x7fa06419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.593+0000 7fa069bce700 1 -- 172.21.15.42:0/204072903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa064100dc0 msgr2=0x7fa06419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.593+0000 7fa069bce700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa064100dc0 0x7fa06419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.593+0000 7fa069bce700 1 -- 172.21.15.42:0/204072903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540095f0 con 0x7fa0640ffbc0 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.593+0000 7fa069bce700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0640ffbc0 0x7fa064069970 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d900 tx=0x7fa06000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.593+0000 7fa05affd700 1 -- 172.21.15.42:0/204072903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0600041f0 con 0x7fa0640ffbc0 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.593+0000 7fa05affd700 1 -- 172.21.15.42:0/204072903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa060004350 con 0x7fa0640ffbc0 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.593+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06419bf10 con 0x7fa0640ffbc0 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.594+0000 7fa05affd700 1 -- 172.21.15.42:0/204072903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa060003da0 con 0x7fa0640ffbc0 2024-09-16T23:46:25.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.594+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06419c430 con 0x7fa0640ffbc0 2024-09-16T23:46:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.595+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06418f530 con 0x7fa0640ffbc0 2024-09-16T23:46:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.595+0000 7fa05affd700 1 -- 172.21.15.42:0/204072903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fa060009730 con 0x7fa0640ffbc0 2024-09-16T23:46:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.595+0000 7fa05affd700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa05006c4e0 0x7fa05006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.595+0000 7fa05affd700 1 -- 172.21.15.42:0/204072903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fa06008b920 con 0x7fa0640ffbc0 2024-09-16T23:46:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.595+0000 7fa0693cd700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa05006c4e0 0x7fa05006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:25.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.596+0000 7fa0693cd700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa05006c4e0 0x7fa05006e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa064069ef0 tx=0x7fa054005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:25.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.601+0000 7fa05affd700 1 -- 172.21.15.42:0/204072903 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa060059be0 con 0x7fa0640ffbc0 2024-09-16T23:46:25.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.846+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa06402d0c0 con 0x7fa0640ffbc0 2024-09-16T23:46:25.844 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:25 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:resolve 2024-09-16T23:46:25.844 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:25 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] up:boot 2024-09-16T23:46:25.844 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:25 smithi042 ceph-mon[27914]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:resolve,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:25.844 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:25 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:46:25.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.846+0000 7fa05affd700 1 -- 172.21.15.42:0/204072903 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1822 (secure 0 0 0) 0x7fa060059770 con 0x7fa0640ffbc0 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:e13 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 13 2024-09-16T23:46:25.845 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:46:25.647803+0000 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:46:25.846 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:46:25.847 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:reconnect seq 4 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:46:25.848 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:46:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:25.848 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:46:25.848 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:25.848 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:46:25.848 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:46:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa05006c4e0 msgr2=0x7fa05006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa05006c4e0 0x7fa05006e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa064069ef0 tx=0x7fa054005a90 comp rx=0 tx=0).stop 2024-09-16T23:46:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0640ffbc0 msgr2=0x7fa064069970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0640ffbc0 0x7fa064069970 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa06000d900 tx=0x7fa06000dcc0 comp rx=0 tx=0).stop 2024-09-16T23:46:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 shutdown_connections 2024-09-16T23:46:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fa05006c4e0 0x7fa05006e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:25.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0640ffbc0 0x7fa064069970 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 --2- 172.21.15.42:0/204072903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa064100dc0 0x7fa06419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 >> 172.21.15.42:0/204072903 conn(0x7fa0640fb160 msgr2=0x7fa064104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 shutdown_connections 2024-09-16T23:46:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:25.849+0000 7fa06be32700 1 -- 172.21.15.42:0/204072903 wait complete. 2024-09-16T23:46:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-09-16T23:46:25.963 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:25 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:resolve 2024-09-16T23:46:25.963 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:25 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] up:boot 2024-09-16T23:46:25.963 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:25 smithi043 ceph-mon[35191]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:resolve,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:25.964 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:25 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:46:26.821 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:26 smithi042 ceph-mon[27914]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s wr, 4 op/s 2024-09-16T23:46:26.821 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:26 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:reconnect 2024-09-16T23:46:26.821 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:26 smithi042 ceph-mon[27914]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:reconnect,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:26.821 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:26 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/204072903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:46:26.844 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-09-16T23:46:27.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:26 smithi043 ceph-mon[35191]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s wr, 4 op/s 2024-09-16T23:46:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:26 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:reconnect 2024-09-16T23:46:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:26 smithi043 ceph-mon[35191]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:reconnect,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:26 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/204072903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:46:27.626 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:27.978 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:27 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:rejoin 2024-09-16T23:46:27.979 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:27 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] up:standby 2024-09-16T23:46:27.979 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:27 smithi042 ceph-mon[27914]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:rejoin,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:27.979 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:27 smithi042 ceph-mon[27914]: daemon mds.cephfs.smithi042.anvvuy is now active in filesystem cephfs as rank 0 2024-09-16T23:46:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:27 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:rejoin 2024-09-16T23:46:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:27 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] up:standby 2024-09-16T23:46:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:27 smithi043 ceph-mon[35191]: fsmap cephfs:2/2 {0=cephfs.smithi042.anvvuy=up:rejoin,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:27 smithi043 ceph-mon[35191]: daemon mds.cephfs.smithi042.anvvuy is now active in filesystem cephfs as rank 0 2024-09-16T23:46:28.880 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:28 smithi042 ceph-mon[27914]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 204 B/s rd, 511 B/s wr, 2 op/s 2024-09-16T23:46:28.880 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:28 smithi042 ceph-mon[27914]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T23:46:28.880 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:28 smithi042 ceph-mon[27914]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:active 2024-09-16T23:46:28.880 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:28 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:29.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:28 smithi043 ceph-mon[35191]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 204 B/s rd, 511 B/s wr, 2 op/s 2024-09-16T23:46:29.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:28 smithi043 ceph-mon[35191]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T23:46:29.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:28 smithi043 ceph-mon[35191]: mds.? [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] up:active 2024-09-16T23:46:29.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:28 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:46:29.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 -- 172.21.15.42:0/2852069190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 msgr2=0x7fe4981010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:29.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/2852069190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe4981010d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe488009910 tx=0x7fe488009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:29.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 -- 172.21.15.42:0/2852069190 shutdown_connections 2024-09-16T23:46:29.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/2852069190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe498101eb0 0x7fe498102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/2852069190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe4981010d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 -- 172.21.15.42:0/2852069190 >> 172.21.15.42:0/2852069190 conn(0x7fe4980fc230 msgr2=0x7fe4980fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 -- 172.21.15.42:0/2852069190 shutdown_connections 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 -- 172.21.15.42:0/2852069190 wait complete. 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 Processor -- start 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.916+0000 7fe49e3f7700 1 -- start start 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe49e3f7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe498074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe49e3f7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe498101eb0 0x7fe498073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe49e3f7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4980736c0 con 0x7fe498100cb0 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe49e3f7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe498073800 con 0x7fe498101eb0 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe497fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe498074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:29.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe497fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe498074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33272/0 (socket says 172.21.15.42:33272) 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe497fff700 1 -- 172.21.15.42:0/1452609949 learned_addr learned my addr 172.21.15.42:0/1452609949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe4977fe700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe498101eb0 0x7fe498073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe497fff700 1 -- 172.21.15.42:0/1452609949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe498101eb0 msgr2=0x7fe498073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe497fff700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe498101eb0 0x7fe498073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe497fff700 1 -- 172.21.15.42:0/1452609949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880095f0 con 0x7fe498100cb0 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.917+0000 7fe497fff700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe498074b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe48800bda0 tx=0x7fe48800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.918+0000 7fe4957fa700 1 -- 172.21.15.42:0/1452609949 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48801d070 con 0x7fe498100cb0 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.918+0000 7fe4957fa700 1 -- 172.21.15.42:0/1452609949 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe488004c50 con 0x7fe498100cb0 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.918+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe498073a80 con 0x7fe498100cb0 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.918+0000 7fe4957fa700 1 -- 172.21.15.42:0/1452609949 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48800f690 con 0x7fe498100cb0 2024-09-16T23:46:29.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.918+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe498073f70 con 0x7fe498100cb0 2024-09-16T23:46:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.919+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe498109810 con 0x7fe498100cb0 2024-09-16T23:46:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.919+0000 7fe4957fa700 1 -- 172.21.15.42:0/1452609949 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fe48800f810 con 0x7fe498100cb0 2024-09-16T23:46:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.920+0000 7fe4957fa700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe48006c4e0 0x7fe48006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:29.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.920+0000 7fe4957fa700 1 -- 172.21.15.42:0/1452609949 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe48808cec0 con 0x7fe498100cb0 2024-09-16T23:46:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.920+0000 7fe4977fe700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe48006c4e0 0x7fe48006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:29.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.920+0000 7fe4977fe700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe48006c4e0 0x7fe48006e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe48c005fd0 tx=0x7fe48c00a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:29.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:29.924+0000 7fe4957fa700 1 -- 172.21.15.42:0/1452609949 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe48805b2a0 con 0x7fe498100cb0 2024-09-16T23:46:30.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.171+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe49819e940 con 0x7fe498100cb0 2024-09-16T23:46:30.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.172+0000 7fe4957fa700 1 -- 172.21.15.42:0/1452609949 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4754 (secure 0 0 0) 0x7fe48805ae30 con 0x7fe498100cb0 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.173+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe48006c4e0 msgr2=0x7fe48006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.173+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe48006c4e0 0x7fe48006e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe48c005fd0 tx=0x7fe48c00a560 comp rx=0 tx=0).stop 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 msgr2=0x7fe498074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe498074b30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe48800bda0 tx=0x7fe48800be80 comp rx=0 tx=0).stop 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 shutdown_connections 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe48006c4e0 0x7fe48006e9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe498100cb0 0x7fe498074b30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 --2- 172.21.15.42:0/1452609949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe498101eb0 0x7fe498073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:30.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 >> 172.21.15.42:0/1452609949 conn(0x7fe4980fc230 msgr2=0x7fe4981050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:30.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 shutdown_connections 2024-09-16T23:46:30.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:30.174+0000 7fe49e3f7700 1 -- 172.21.15.42:0/1452609949 wait complete. 2024-09-16T23:46:30.172 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 15 2024-09-16T23:46:30.182 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:46:31.014 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:30 smithi043 ceph-mon[35191]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T23:46:31.014 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:30 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1452609949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T23:46:31.021 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:30 smithi042 ceph-mon[27914]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-16T23:46:31.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:30 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1452609949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T23:46:31.046 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-09-16T23:46:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:32.585 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:32 smithi042 ceph-mon[27914]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T23:46:32.585 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:32.585 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:32.585 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:32.585 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:46:32.585 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:32.585 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:46:32.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:32 smithi043 ceph-mon[35191]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T23:46:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:46:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 -- 172.21.15.42:0/3432598619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d240ffba0 msgr2=0x7f7d240fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/3432598619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d240ffba0 0x7f7d240fffc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009910 tx=0x7f7d20009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 -- 172.21.15.42:0/3432598619 shutdown_connections 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/3432598619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d24100da0 0x7f7d24101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/3432598619 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d240ffba0 0x7f7d240fffc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 -- 172.21.15.42:0/3432598619 >> 172.21.15.42:0/3432598619 conn(0x7f7d240fb120 msgr2=0x7f7d240fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 -- 172.21.15.42:0/3432598619 shutdown_connections 2024-09-16T23:46:32.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 -- 172.21.15.42:0/3432598619 wait complete. 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 Processor -- start 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.825+0000 7f7d2c48d700 1 -- start start 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d2c48d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d240ffba0 0x7f7d241964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d2c48d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d24100da0 0x7f7d24196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d2c48d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24197040 con 0x7f7d24100da0 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d2c48d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24197180 con 0x7f7d240ffba0 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d29a28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d24100da0 0x7f7d24196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d29a28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d24100da0 0x7f7d24196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54562/0 (socket says 172.21.15.42:54562) 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d29a28700 1 -- 172.21.15.42:0/854588200 learned_addr learned my addr 172.21.15.42:0/854588200 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d2a229700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d240ffba0 0x7f7d241964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d29a28700 1 -- 172.21.15.42:0/854588200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d240ffba0 msgr2=0x7f7d241964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:32.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d29a28700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d240ffba0 0x7f7d241964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d29a28700 1 -- 172.21.15.42:0/854588200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d200095f0 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d29a28700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d24100da0 0x7f7d24196a20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400e920 tx=0x7f7d1400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d1b7fe700 1 -- 172.21.15.42:0/854588200 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d1400c9d0 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d1b7fe700 1 -- 172.21.15.42:0/854588200 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7d14010430 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2419bc30 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d1b7fe700 1 -- 172.21.15.42:0/854588200 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d14018650 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.826+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2419c180 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.827+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24109800 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.828+0000 7f7d1b7fe700 1 -- 172.21.15.42:0/854588200 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f7d140105a0 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.828+0000 7f7d1b7fe700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d1006c4e0 0x7f7d1006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.828+0000 7f7d1b7fe700 1 -- 172.21.15.42:0/854588200 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7d14014070 con 0x7f7d24100da0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.829+0000 7f7d2a229700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d1006c4e0 0x7f7d1006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:32.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.829+0000 7f7d2a229700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d1006c4e0 0x7f7d1006e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000b6f0 tx=0x7f7d20005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:32.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:32.833+0000 7f7d1b7fe700 1 -- 172.21.15.42:0/854588200 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7d140520e0 con 0x7f7d24100da0 2024-09-16T23:46:33.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.116+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f7d241099f0 con 0x7f7d24100da0 2024-09-16T23:46:33.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.116+0000 7f7d1b7fe700 1 -- 172.21.15.42:0/854588200 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+83 (secure 0 0 0) 0x7f7d14059d60 con 0x7f7d24100da0 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.118+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d1006c4e0 msgr2=0x7f7d1006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.118+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d1006c4e0 0x7f7d1006e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000b6f0 tx=0x7f7d20005940 comp rx=0 tx=0).stop 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.118+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d24100da0 msgr2=0x7f7d24196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.118+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d24100da0 0x7f7d24196a20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7d1400e920 tx=0x7f7d1400ec30 comp rx=0 tx=0).stop 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.119+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 shutdown_connections 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.119+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f7d1006c4e0 0x7f7d1006e9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.119+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d240ffba0 0x7f7d241964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.119+0000 7f7d2c48d700 1 --2- 172.21.15.42:0/854588200 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7d24100da0 0x7f7d24196a20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:33.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.119+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 >> 172.21.15.42:0/854588200 conn(0x7f7d240fb120 msgr2=0x7f7d24103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:33.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.119+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 shutdown_connections 2024-09-16T23:46:33.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:33.119+0000 7f7d2c48d700 1 -- 172.21.15.42:0/854588200 wait complete. 2024-09-16T23:46:33.126 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:46:33.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:33 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/854588200' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-16T23:46:33.748 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:33 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/854588200' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-16T23:46:34.087 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-09-16T23:46:34.103 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T23:46:34.963 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:35.088 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:34 smithi042 ceph-mon[27914]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T23:46:35.089 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:35.089 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:34 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:35.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:34 smithi043 ceph-mon[35191]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T23:46:35.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:35.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:34 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:36.917 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:36 smithi042 ceph-mon[27914]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T23:46:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 -- 172.21.15.42:0/240871254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 msgr2=0x7f56700689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:37.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 --2- 172.21.15.42:0/240871254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f56700689e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5660009910 tx=0x7f5660009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 -- 172.21.15.42:0/240871254 shutdown_connections 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 --2- 172.21.15.42:0/240871254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f56700689e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 --2- 172.21.15.42:0/240871254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5670106cd0 0x7f56701070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 -- 172.21.15.42:0/240871254 >> 172.21.15.42:0/240871254 conn(0x7f5670075670 msgr2=0x7f5670075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 -- 172.21.15.42:0/240871254 shutdown_connections 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.128+0000 7f56779dd700 1 -- 172.21.15.42:0/240871254 wait complete. 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f56779dd700 1 Processor -- start 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f56779dd700 1 -- start start 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f56779dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f567019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f56779dd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5670106cd0 0x7f567019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f56779dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567019d200 con 0x7f5670068580 2024-09-16T23:46:37.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f56779dd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56701966b0 con 0x7f5670106cd0 2024-09-16T23:46:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:36 smithi043 ceph-mon[35191]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.1 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T23:46:37.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f5675779700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f567019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f5675779700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f567019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54580/0 (socket says 172.21.15.42:54580) 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.129+0000 7f5675779700 1 -- 172.21.15.42:0/2093755849 learned_addr learned my addr 172.21.15.42:0/2093755849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f5674f78700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5670106cd0 0x7f567019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f5675779700 1 -- 172.21.15.42:0/2093755849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5670106cd0 msgr2=0x7f567019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f5675779700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5670106cd0 0x7f567019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f5675779700 1 -- 172.21.15.42:0/2093755849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600095f0 con 0x7f5670068580 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f5674f78700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5670106cd0 0x7f567019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f5675779700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f567019c630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f566c00d8d0 tx=0x7f566c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f56667fc700 1 -- 172.21.15.42:0/2093755849 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f566c009940 con 0x7f5670068580 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f56667fc700 1 -- 172.21.15.42:0/2093755849 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f566c010460 con 0x7f5670068580 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5670196990 con 0x7f5670068580 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f56667fc700 1 -- 172.21.15.42:0/2093755849 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f566c00f5f0 con 0x7f5670068580 2024-09-16T23:46:37.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.130+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5670196ee0 con 0x7f5670068580 2024-09-16T23:46:37.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.131+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56701091d0 con 0x7f5670068580 2024-09-16T23:46:37.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.132+0000 7f56667fc700 1 -- 172.21.15.42:0/2093755849 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f566c0105d0 con 0x7f5670068580 2024-09-16T23:46:37.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.132+0000 7f56667fc700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f565c06c4e0 0x7f565c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:37.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.132+0000 7f56667fc700 1 -- 172.21.15.42:0/2093755849 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f566c08b6e0 con 0x7f5670068580 2024-09-16T23:46:37.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.132+0000 7f5674f78700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f565c06c4e0 0x7f565c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:37.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.133+0000 7f5674f78700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f565c06c4e0 0x7f565c06e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f566000b5c0 tx=0x7f5660005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:37.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.137+0000 7f56667fc700 1 -- 172.21.15.42:0/2093755849 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f566c059a20 con 0x7f5670068580 2024-09-16T23:46:37.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.383+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5670197cd0 con 0x7f5670068580 2024-09-16T23:46:37.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.384+0000 7f56667fc700 1 -- 172.21.15.42:0/2093755849 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4754 (secure 0 0 0) 0x7f566c08b020 con 0x7f5670068580 2024-09-16T23:46:37.381 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:46:37.382 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3253599157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3253599157},{"type":"v1","addr":"172.21.15.42:6827","nonce":3253599157}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:46:27.657253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi042.anvvuy","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.42:6829/1702969563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1702969563},{"type":"v1","addr":"172.21.15.42:6829","nonce":1702969563}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi043.pzcdzg","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6825/3963651257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3963651257},{"type":"v1","addr":"172.21.15.43:6825","nonce":3963651257}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-16T23:46:37.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f565c06c4e0 msgr2=0x7f565c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:37.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f565c06c4e0 0x7f565c06e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f566000b5c0 tx=0x7f5660005960 comp rx=0 tx=0).stop 2024-09-16T23:46:37.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 msgr2=0x7f567019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:37.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f567019c630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f566c00d8d0 tx=0x7f566c00dc90 comp rx=0 tx=0).stop 2024-09-16T23:46:37.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 shutdown_connections 2024-09-16T23:46:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f565c06c4e0 0x7f565c06e9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5670068580 0x7f567019c630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 --2- 172.21.15.42:0/2093755849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5670106cd0 0x7f567019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 >> 172.21.15.42:0/2093755849 conn(0x7f5670075670 msgr2=0x7f56700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 shutdown_connections 2024-09-16T23:46:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:37.386+0000 7f56779dd700 1 -- 172.21.15.42:0/2093755849 wait complete. 2024-09-16T23:46:37.384 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 15 2024-09-16T23:46:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:37 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2093755849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T23:46:38.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:37 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2093755849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T23:46:38.466 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 2, 'flags': 18} 2024-09-16T23:46:38.467 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-16T23:46:38.929 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-16T23:46:38.929 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-09-16T23:46:38.930 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-16T23:46:38.930 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T23:46:38.930 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:38.930 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-16T23:46:38.930 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T23:46:38.930 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:38.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:38.931 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-09-16T23:46:38.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:38.950 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-09-16T23:46:39.001 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:38 smithi042 ceph-mon[27914]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T23:46:39.022 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-09-16T23:46:39.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T23:46:39.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:39.024 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-16T23:46:39.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:39.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-09-16T23:46:39.113 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:38 smithi043 ceph-mon[35191]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T23:46:39.121 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-09-16T23:46:39.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T23:46:39.123 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-16T23:46:39.123 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-16T23:46:39.123 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T23:46:39.344 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:40.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:40 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:40.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:40 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:40.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:40 smithi043 ceph-mon[35191]: pgmap v199: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T23:46:40.506 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:40 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:40.506 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:40 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:40.506 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:40 smithi042 ceph-mon[27914]: pgmap v199: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T23:46:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 -- 172.21.15.42:0/2454977061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 msgr2=0x7f488c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 --2- 172.21.15.42:0/2454977061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c100080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4880009910 tx=0x7f4880009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 -- 172.21.15.42:0/2454977061 shutdown_connections 2024-09-16T23:46:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 --2- 172.21.15.42:0/2454977061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c100080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 --2- 172.21.15.42:0/2454977061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c105bd0 0x7f488c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 -- 172.21.15.42:0/2454977061 >> 172.21.15.42:0/2454977061 conn(0x7f488c075670 msgr2=0x7f488c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:41.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 -- 172.21.15.42:0/2454977061 shutdown_connections 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.001+0000 7f4890a7f700 1 -- 172.21.15.42:0/2454977061 wait complete. 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f4890a7f700 1 Processor -- start 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f4890a7f700 1 -- start start 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f4890a7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f4890a7f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c105bd0 0x7f488c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f4890a7f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c19d260 con 0x7f488c0ffc20 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f4890a7f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c196710 con 0x7f488c105bd0 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f488a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f488a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54610/0 (socket says 172.21.15.42:54610) 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f488a59c700 1 -- 172.21.15.42:0/3508305317 learned_addr learned my addr 172.21.15.42:0/3508305317 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f4889d9b700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c105bd0 0x7f488c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f488a59c700 1 -- 172.21.15.42:0/3508305317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c105bd0 msgr2=0x7f488c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f488a59c700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c105bd0 0x7f488c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:41.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f488a59c700 1 -- 172.21.15.42:0/3508305317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800095f0 con 0x7f488c0ffc20 2024-09-16T23:46:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f488a59c700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c19c690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f487c00b4b0 tx=0x7f487c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.002+0000 7f487b7fe700 1 -- 172.21.15.42:0/3508305317 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c011800 con 0x7f488c0ffc20 2024-09-16T23:46:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.003+0000 7f487b7fe700 1 -- 172.21.15.42:0/3508305317 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f487c011e10 con 0x7f488c0ffc20 2024-09-16T23:46:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.003+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c1969f0 con 0x7f488c0ffc20 2024-09-16T23:46:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.003+0000 7f487b7fe700 1 -- 172.21.15.42:0/3508305317 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c00f2e0 con 0x7f488c0ffc20 2024-09-16T23:46:41.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.003+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c196f40 con 0x7f488c0ffc20 2024-09-16T23:46:41.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.004+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c04f2e0 con 0x7f488c0ffc20 2024-09-16T23:46:41.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.004+0000 7f487b7fe700 1 -- 172.21.15.42:0/3508305317 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f487c010380 con 0x7f488c0ffc20 2024-09-16T23:46:41.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.005+0000 7f487b7fe700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f487406c4e0 0x7f487406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:41.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.005+0000 7f487b7fe700 1 -- 172.21.15.42:0/3508305317 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f487c08b290 con 0x7f488c0ffc20 2024-09-16T23:46:41.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.005+0000 7f4889d9b700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f487406c4e0 0x7f487406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:41.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.005+0000 7f4889d9b700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f487406c4e0 0x7f487406e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f488c197b80 tx=0x7f488001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:41.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.009+0000 7f487b7fe700 1 -- 172.21.15.42:0/3508305317 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f487c0595d0 con 0x7f488c0ffc20 2024-09-16T23:46:41.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.252+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f488c197210 con 0x7f488c0ffc20 2024-09-16T23:46:41.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.252+0000 7f487b7fe700 1 -- 172.21.15.42:0/3508305317 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f487c059160 con 0x7f488c0ffc20 2024-09-16T23:46:41.250 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-16T23:46:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f487406c4e0 msgr2=0x7f487406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f487406c4e0 0x7f487406e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f488c197b80 tx=0x7f488001a040 comp rx=0 tx=0).stop 2024-09-16T23:46:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 msgr2=0x7f488c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:41.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c19c690 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f487c00b4b0 tx=0x7f487c00b7c0 comp rx=0 tx=0).stop 2024-09-16T23:46:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 shutdown_connections 2024-09-16T23:46:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f487406c4e0 0x7f487406e9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f488c0ffc20 0x7f488c19c690 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 --2- 172.21.15.42:0/3508305317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f488c105bd0 0x7f488c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 >> 172.21.15.42:0/3508305317 conn(0x7f488c075670 msgr2=0x7f488c068620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 shutdown_connections 2024-09-16T23:46:41.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:41.254+0000 7f4890a7f700 1 -- 172.21.15.42:0/3508305317 wait complete. 2024-09-16T23:46:41.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:41 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:41.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:41 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:41.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:41 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:41.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:41 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:46:41.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:41 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:41 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:41 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:41 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:46:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:41 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:46:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:41 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:46:41.891 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T23:46:41.891 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T23:46:41.891 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-09-16T23:46:41.892 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T23:46:41.892 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:41.892 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T23:46:41.892 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T23:46:41.892 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T23:46:41.892 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T23:46:41.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:41.892 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-09-16T23:46:41.911 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:41.912 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:41.913 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-09-16T23:46:41.914 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T23:46:41.914 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:41.914 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-09-16T23:46:41.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-09-16T23:46:41.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-09-16T23:46:41.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-09-16T23:46:41.914 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T23:46:41.914 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-09-16T23:46:42.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:42.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:42.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:42.125 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T23:46:42.151 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-09-16T23:46:42.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:42.153 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-09-16T23:46:42.221 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-09-16T23:46:42.222 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T23:46:42.222 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T23:46:42.224 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T23:46:42.224 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T23:46:42.225 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:42.225 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-09-16T23:46:42.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T23:46:42.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T23:46:42.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T23:46:42.225 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-09-16T23:46:42.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:42.425 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:42 smithi042 ceph-mon[27914]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T23:46:42.425 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:42 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3508305317' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T23:46:42.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:42.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:42.527 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-09-16T23:46:42.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:42.544 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-09-16T23:46:42.599 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:42.599 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-09-16T23:46:42.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T23:46:42.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T23:46:42.599 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-09-16T23:46:42.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:42 smithi043 ceph-mon[35191]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.3 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T23:46:42.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:42 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3508305317' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T23:46:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:42.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:42.719 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T23:46:42.719 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:42.720 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-09-16T23:46:42.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T23:46:42.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T23:46:42.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T23:46:42.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T23:46:42.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T23:46:42.720 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-09-16T23:46:42.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:42.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:42.850 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:42.850 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-09-16T23:46:42.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-09-16T23:46:42.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T23:46:42.851 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-09-16T23:46:42.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:42.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:42.930 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-16T23:46:42.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T23:46:42.931 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:42.947 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T23:46:42.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T23:46:42.947 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:43.003 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-09-16T23:46:43.069 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T23:46:43.125 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:43.126 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ec80e1bffec911e67358830bbfeff5db30e0e36c715d8eab8f61657166b65fa0/merged 2024-09-16T23:46:43.127 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/dc46265ca7a42d36f5c5ee6824f5633a80760195f9ac2f8b467ee966601d3f98/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/7d2a6fcb8940f69df3cac8e5cba75b34f0aab836ea9ec7b97d2e52a79582c952/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3d5ccfd41be38c7c553de442eb02df6d452dced270b04f5edb48b87d4be7571b/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ff555614038032dc9ed9620d3fee34c320926c91c09bb2808066c8d6b6ace769/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/405d82980773e733762569f9c8b6146a984aa8e201be14ab450fef6beba9e28a/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/5d4b6763af23702c6cc5a942496d28340d42f7877051bf3becb06b0153618f65/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/dfcfdad057e748ed1a2283ba4f5aa74343f64a9da5e121145089bd3bfb67bca3/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3229a4f31a7f24b5998e5149ba8141b264eb6ee9faccf2f13b469c03e326ee91/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/f182680d765452bee8d04ca84bc811deff0a0c77ddbb048e28f8e2799d49a310/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/831d849ac159ed3e3f937987e93620509a37819ce123991ec85b6f83f715089c/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/96a4e70881192c8c4fd9605027fc72dbf21af2eb5d4a9df195d648786fabcfcf/merged 2024-09-16T23:46:43.128 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4934cf4a30a49b8183b7c399ee0c379f7718b8e60a2d046b2618d7c6aca678c3/merged 2024-09-16T23:46:43.129 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-09-16T23:46:43.129 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T23:46:43.129 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T23:46:43.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:43.129 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-09-16T23:46:43.182 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T23:46:43.182 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-16T23:46:43.225 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2024-09-16T23:46:43.253 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T23:46:43.309 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:43.310 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ec80e1bffec911e67358830bbfeff5db30e0e36c715d8eab8f61657166b65fa0/merged 2024-09-16T23:46:43.311 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/dc46265ca7a42d36f5c5ee6824f5633a80760195f9ac2f8b467ee966601d3f98/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/7d2a6fcb8940f69df3cac8e5cba75b34f0aab836ea9ec7b97d2e52a79582c952/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3d5ccfd41be38c7c553de442eb02df6d452dced270b04f5edb48b87d4be7571b/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/ff555614038032dc9ed9620d3fee34c320926c91c09bb2808066c8d6b6ace769/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/405d82980773e733762569f9c8b6146a984aa8e201be14ab450fef6beba9e28a/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/5d4b6763af23702c6cc5a942496d28340d42f7877051bf3becb06b0153618f65/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/dfcfdad057e748ed1a2283ba4f5aa74343f64a9da5e121145089bd3bfb67bca3/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3229a4f31a7f24b5998e5149ba8141b264eb6ee9faccf2f13b469c03e326ee91/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/f182680d765452bee8d04ca84bc811deff0a0c77ddbb048e28f8e2799d49a310/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/831d849ac159ed3e3f937987e93620509a37819ce123991ec85b6f83f715089c/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/96a4e70881192c8c4fd9605027fc72dbf21af2eb5d4a9df195d648786fabcfcf/merged 2024-09-16T23:46:43.312 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4934cf4a30a49b8183b7c399ee0c379f7718b8e60a2d046b2618d7c6aca678c3/merged 2024-09-16T23:46:43.313 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2024-09-16T23:46:43.313 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T23:46:43.313 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T23:46:43.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:43.313 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2024-09-16T23:46:43.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:2024-09-16T23:46:43.353+0000 7fdaba63b480 -1 init, newargv = 0x56193a332c90 newargc=15 2024-09-16T23:46:43.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[75722]: starting ceph client 2024-09-16T23:46:43.359 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[75722]: starting fuse 2024-09-16T23:46:43.367 INFO:teuthology.orchestra.run.smithi042.stdout:132 2024-09-16T23:46:43.367 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [132] 2024-09-16T23:46:43.367 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> import re 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> import os 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2024-09-16T23:46:43.368 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> return f 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.0")) 2024-09-16T23:46:43.369 DEBUG:teuthology.orchestra.run.smithi042:> ' 2024-09-16T23:46:43.487 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.0.75722.asok 2024-09-16T23:46:43.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:43.498 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.75722.asok 2024-09-16T23:46:43.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:43.498 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.75722.asok status 2024-09-16T23:46:43.634 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "0", 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "75722", 2024-09-16T23:46:43.635 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 14514, 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 14514 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T23:46:43.636 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2919541394 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2919541394 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.14514 192.168.0.1:0/2919541394", 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/2919541394", 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 15, 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 46, 2024-09-16T23:46:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2024-09-16T23:46:43.638 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2024-09-16T23:46:43.638 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2024-09-16T23:46:43.638 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:46:43.643 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-16T23:46:43.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T23:46:43.850 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:44.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:44 smithi042 ceph-mon[27914]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T23:46:44.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:44 smithi043 ceph-mon[35191]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 -- 172.21.15.42:0/1341191872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0100ea0 msgr2=0x7f85a0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 --2- 172.21.15.42:0/1341191872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0100ea0 0x7f85a0101280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8590009910 tx=0x7f8590009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 -- 172.21.15.42:0/1341191872 shutdown_connections 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 --2- 172.21.15.42:0/1341191872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0101850 0x7f85a01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 --2- 172.21.15.42:0/1341191872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0100ea0 0x7f85a0101280 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 -- 172.21.15.42:0/1341191872 >> 172.21.15.42:0/1341191872 conn(0x7f85a00fc750 msgr2=0x7f85a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 -- 172.21.15.42:0/1341191872 shutdown_connections 2024-09-16T23:46:45.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.477+0000 7f85a7115700 1 -- 172.21.15.42:0/1341191872 wait complete. 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.478+0000 7f85a7115700 1 Processor -- start 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.478+0000 7f85a7115700 1 -- start start 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.478+0000 7f85a7115700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0100ea0 0x7f85a019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.478+0000 7f85a7115700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0101850 0x7f85a019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.478+0000 7f85a7115700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a019d2c0 con 0x7f85a0101850 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.478+0000 7f85a7115700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0196770 con 0x7f85a0100ea0 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.478+0000 7f859ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0101850 0x7f85a019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0101850 0x7f85a019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40700/0 (socket says 172.21.15.42:40700) 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859ffff700 1 -- 172.21.15.42:0/2269265554 learned_addr learned my addr 172.21.15.42:0/2269265554 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:45.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f85a4eb1700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0100ea0 0x7f85a019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859ffff700 1 -- 172.21.15.42:0/2269265554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0100ea0 msgr2=0x7f85a019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859ffff700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0100ea0 0x7f85a019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859ffff700 1 -- 172.21.15.42:0/2269265554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900095f0 con 0x7f85a0101850 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859ffff700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0101850 0x7f85a019cc30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f859400d900 tx=0x7f859400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859dffb700 1 -- 172.21.15.42:0/2269265554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8594009900 con 0x7f85a0101850 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a0196a50 con 0x7f85a0101850 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859dffb700 1 -- 172.21.15.42:0/2269265554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8594010460 con 0x7f85a0101850 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f859dffb700 1 -- 172.21.15.42:0/2269265554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f859400f5f0 con 0x7f85a0101850 2024-09-16T23:46:45.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.479+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a0196fa0 con 0x7f85a0101850 2024-09-16T23:46:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.480+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a0109240 con 0x7f85a0101850 2024-09-16T23:46:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.481+0000 7f859dffb700 1 -- 172.21.15.42:0/2269265554 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f85940105d0 con 0x7f85a0101850 2024-09-16T23:46:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.481+0000 7f859dffb700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f858806c4e0 0x7f858806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.481+0000 7f859dffb700 1 -- 172.21.15.42:0/2269265554 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f859408b620 con 0x7f85a0101850 2024-09-16T23:46:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.481+0000 7f85a4eb1700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f858806c4e0 0x7f858806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:45.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.482+0000 7f85a4eb1700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f858806c4e0 0x7f858806e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f859000b5c0 tx=0x7f8590005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:45.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.486+0000 7f859dffb700 1 -- 172.21.15.42:0/2269265554 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f85940598e0 con 0x7f85a0101850 2024-09-16T23:46:45.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.728+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f85a004f2e0 con 0x7f85a0101850 2024-09-16T23:46:45.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.729+0000 7f859dffb700 1 -- 172.21.15.42:0/2269265554 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f8594059470 con 0x7f85a0101850 2024-09-16T23:46:45.726 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-16T23:46:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f858806c4e0 msgr2=0x7f858806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f858806c4e0 0x7f858806e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f859000b5c0 tx=0x7f8590005960 comp rx=0 tx=0).stop 2024-09-16T23:46:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0101850 msgr2=0x7f85a019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:45.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0101850 0x7f85a019cc30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f859400d900 tx=0x7f859400dcc0 comp rx=0 tx=0).stop 2024-09-16T23:46:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 shutdown_connections 2024-09-16T23:46:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f858806c4e0 0x7f858806e9a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a0100ea0 0x7f85a019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 --2- 172.21.15.42:0/2269265554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85a0101850 0x7f85a019cc30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 >> 172.21.15.42:0/2269265554 conn(0x7f85a00fc750 msgr2=0x7f85a00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 shutdown_connections 2024-09-16T23:46:45.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:45.731+0000 7f85a7115700 1 -- 172.21.15.42:0/2269265554 wait complete. 2024-09-16T23:46:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:46 smithi042 ceph-mon[27914]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:46.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:46 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2269265554' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T23:46:46.347 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T23:46:46.348 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T23:46:46.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:46.348 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T23:46:46.368 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42301sec preferred_lft 42301sec 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-09-16T23:46:46.369 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-09-16T23:46:46.370 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T23:46:46.370 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:46.370 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-16T23:46:46.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-09-16T23:46:46.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-09-16T23:46:46.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-09-16T23:46:46.370 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T23:46:46.370 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-16T23:46:46.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:46.474 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:46 smithi043 ceph-mon[35191]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:46:46.474 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:46 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2269265554' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T23:46:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:46.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:46.594 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T23:46:46.622 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-09-16T23:46:46.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:46.622 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-09-16T23:46:46.693 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-09-16T23:46:46.693 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T23:46:46.693 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T23:46:46.693 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T23:46:46.693 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T23:46:46.694 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:46.694 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-16T23:46:46.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T23:46:46.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T23:46:46.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T23:46:46.694 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-16T23:46:46.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:46.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:46.981 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-09-16T23:46:46.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:46.997 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-09-16T23:46:47.053 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:47.054 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-16T23:46:47.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T23:46:47.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-16T23:46:47.054 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-16T23:46:47.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:47.173 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-16T23:46:47.173 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:47.173 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-16T23:46:47.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-16T23:46:47.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T23:46:47.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-16T23:46:47.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-16T23:46:47.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-16T23:46:47.174 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-16T23:46:47.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:47.300 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T23:46:47.300 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-09-16T23:46:47.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-09-16T23:46:47.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T23:46:47.300 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-09-16T23:46:47.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T23:46:47.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:47.378 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-16T23:46:47.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T23:46:47.379 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:47.394 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T23:46:47.394 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T23:46:47.394 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:47.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-09-16T23:46:47.516 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T23:46:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-09-16T23:46:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-09-16T23:46:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-09-16T23:46:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-09-16T23:46:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-09-16T23:46:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-09-16T23:46:47.576 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-09-16T23:46:47.576 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-09-16T23:46:47.576 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-09-16T23:46:47.576 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T23:46:47.576 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T23:46:47.576 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T23:46:47.576 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T23:46:47.577 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-09-16T23:46:47.577 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-09-16T23:46:47.577 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-09-16T23:46:47.577 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:47.577 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-09-16T23:46:47.577 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ecb50e18ab6a0a82f007db11968033a57ae2296f1581bfe0c494aba01d053cea/merged 2024-09-16T23:46:47.578 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/e04c862ebe718dc7a02fa070a153b2ef0eb5bef0bae64c018547b2f152b512cc/merged 2024-09-16T23:46:47.578 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6b1bef319084be364ddf889b513caea791e5f2bd8b947e41774d969b2795507d/merged 2024-09-16T23:46:47.578 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d1a3194be720e926a0e22cef05061fbc8a2bd0721026ec86bfb3174dcfeb3065/merged 2024-09-16T23:46:47.578 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/c5d12889fd6b6b1bda9c7ffeef12553f3851b31ba2ef00b3701484bd9810b418/merged 2024-09-16T23:46:47.578 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ba7145fd93621ee9492a24daafdb660c879af985ea0512cc25d045c6820be1a2/merged 2024-09-16T23:46:47.578 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/a01ded829a390f11504e9381a2f28aebd0b4626e76037ebce9ee7edbcaf66c7b/merged 2024-09-16T23:46:47.579 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ef0175f489121043c8f95cab16f5a93c8d857e724d7d275ca412ba89d1e7da59/merged 2024-09-16T23:46:47.579 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/299e7ee90bbbfe8d0865671a6ee56b1a114ca780d435671f8fdf58479f763559/merged 2024-09-16T23:46:47.579 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/550fe2cb6ca0e7616e013978435a8961b8b6690db716235d47fafdcabe01029a/merged 2024-09-16T23:46:47.579 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-09-16T23:46:47.579 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T23:46:47.579 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T23:46:47.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:47.580 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-09-16T23:46:47.630 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T23:46:47.630 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-16T23:46:47.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-09-16T23:46:47.698 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-09-16T23:46:47.755 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T23:46:47.756 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/0 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ecb50e18ab6a0a82f007db11968033a57ae2296f1581bfe0c494aba01d053cea/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/e04c862ebe718dc7a02fa070a153b2ef0eb5bef0bae64c018547b2f152b512cc/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/6b1bef319084be364ddf889b513caea791e5f2bd8b947e41774d969b2795507d/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/d1a3194be720e926a0e22cef05061fbc8a2bd0721026ec86bfb3174dcfeb3065/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/c5d12889fd6b6b1bda9c7ffeef12553f3851b31ba2ef00b3701484bd9810b418/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ba7145fd93621ee9492a24daafdb660c879af985ea0512cc25d045c6820be1a2/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/a01ded829a390f11504e9381a2f28aebd0b4626e76037ebce9ee7edbcaf66c7b/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/ef0175f489121043c8f95cab16f5a93c8d857e724d7d275ca412ba89d1e7da59/merged 2024-09-16T23:46:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/299e7ee90bbbfe8d0865671a6ee56b1a114ca780d435671f8fdf58479f763559/merged 2024-09-16T23:46:47.758 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/containers/storage/overlay/550fe2cb6ca0e7616e013978435a8961b8b6690db716235d47fafdcabe01029a/merged 2024-09-16T23:46:47.758 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-09-16T23:46:47.758 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T23:46:47.758 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T23:46:47.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:47.758 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-09-16T23:46:47.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi043.stderr:2024-09-16T23:46:47.778+0000 7f53e976e480 -1 init, newargv = 0x5617c2381d30 newargc=15 2024-09-16T23:46:47.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi043.stderr:ceph-fuse[60871]: starting ceph client 2024-09-16T23:46:47.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi043.stderr:ceph-fuse[60871]: starting fuse 2024-09-16T23:46:47.793 INFO:teuthology.orchestra.run.smithi043.stdout:129 2024-09-16T23:46:47.794 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [129] 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-09-16T23:46:47.794 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-09-16T23:46:47.795 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-09-16T23:46:47.796 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T23:46:47.796 DEBUG:teuthology.orchestra.run.smithi043:> 2024-09-16T23:46:47.796 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.1")) 2024-09-16T23:46:47.796 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-09-16T23:46:47.914 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.1.60871.asok 2024-09-16T23:46:47.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-09-16T23:46:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T23:46:47.924 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.60871.asok 2024-09-16T23:46:47.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:47.925 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.60871.asok status 2024-09-16T23:46:48.060 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "1", 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "60871", 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-09-16T23:46:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 24305, 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 24305 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2491590655 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-09-16T23:46:48.062 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2491590655 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.24305 192.168.0.1:0/2491590655", 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2491590655", 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 15, 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 46, 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-09-16T23:46:48.063 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-09-16T23:46:48.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:48.069 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2024-09-16T23:46:48.069 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:48.091 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2024-09-16T23:46:48.091 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:48.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:48.091 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:48.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:48.167 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-09-16T23:46:48.167 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:48.184 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-09-16T23:46:48.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:48.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T23:46:48.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:48.914 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:48 smithi043 ceph-mon[35191]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T23:46:48.915 INFO:teuthology.run_tasks:Running task print... 2024-09-16T23:46:48.934 INFO:teuthology.task.print:**** done client 2024-09-16T23:46:48.934 INFO:teuthology.run_tasks:Running task parallel... 2024-09-16T23:46:48.945 INFO:teuthology.task.parallel:starting parallel... 2024-09-16T23:46:48.946 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-16T23:46:48.946 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T23:46:48.947 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:46:48.947 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-16T23:46:48.948 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-16T23:46:48.948 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-09-16T23:46:48.952 INFO:tasks.workunit:Pulling workunits from ref e20bff7d16f5b09c58ead9548e8cf98cf87ee227 2024-09-16T23:46:48.953 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T23:46:48.953 INFO:tasks.workunit:timeout=3h 2024-09-16T23:46:48.953 INFO:tasks.workunit:cleanup=True 2024-09-16T23:46:48.953 DEBUG:teuthology.orchestra.run.smithi042:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:48.974 INFO:teuthology.orchestra.run.smithi042.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 84h/132d Inode: 1 Links: 2 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-09-16 23:46:11.622443826 +0000 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-09-16 23:46:48.166524895 +0000 2024-09-16T23:46:48.975 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2024-09-16T23:46:48.975 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T23:46:48.975 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T23:46:48.981 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:48 smithi042 ceph-mon[27914]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T23:46:49.017 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:49.038 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-16T23:46:49.038 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 81h/129d Inode: 1 Links: 3 2024-09-16T23:46:49.038 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T23:46:49.038 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-16T23:46:49.038 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-16T23:46:49.039 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-09-16 23:46:49.013763962 +0000 2024-09-16T23:46:49.039 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-09-16 23:46:49.013763962 +0000 2024-09-16T23:46:49.039 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-09-16T23:46:49.040 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-16T23:46:49.040 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-16T23:46:49.115 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e20bff7d16f5b09c58ead9548e8cf98cf87ee227 2024-09-16T23:46:49.116 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout e20bff7d16f5b09c58ead9548e8cf98cf87ee227 2024-09-16T23:46:49.133 INFO:tasks.workunit.client.0.smithi042.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T23:46:49.174 INFO:tasks.workunit.client.1.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-16T23:46:49.178 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:50.960 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:50 smithi042 ceph-mon[27914]: pgmap v204: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-16T23:46:50.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.990+0000 7f2760c6e700 1 -- 172.21.15.42:0/141365881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c101eb0 msgr2=0x7f275c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:50.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.990+0000 7f2760c6e700 1 --2- 172.21.15.42:0/141365881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c101eb0 0x7f275c102310 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2750009910 tx=0x7f2750009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:50.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.990+0000 7f2760c6e700 1 -- 172.21.15.42:0/141365881 shutdown_connections 2024-09-16T23:46:50.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.990+0000 7f2760c6e700 1 --2- 172.21.15.42:0/141365881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c101eb0 0x7f275c102310 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:50.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.990+0000 7f2760c6e700 1 --2- 172.21.15.42:0/141365881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c100cb0 0x7f275c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:50.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.990+0000 7f2760c6e700 1 -- 172.21.15.42:0/141365881 >> 172.21.15.42:0/141365881 conn(0x7f275c0fc230 msgr2=0x7f275c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:50.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.991+0000 7f2760c6e700 1 -- 172.21.15.42:0/141365881 shutdown_connections 2024-09-16T23:46:50.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.991+0000 7f2760c6e700 1 -- 172.21.15.42:0/141365881 wait complete. 2024-09-16T23:46:50.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.991+0000 7f2760c6e700 1 Processor -- start 2024-09-16T23:46:50.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.991+0000 7f2760c6e700 1 -- start start 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.991+0000 7f2760c6e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c100cb0 0x7f275c196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f2760c6e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c101eb0 0x7f275c196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f2760c6e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c1970b0 con 0x7f275c100cb0 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f2760c6e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c1971f0 con 0x7f275c101eb0 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f275a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c100cb0 0x7f275c196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f275a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c100cb0 0x7f275c196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40718/0 (socket says 172.21.15.42:40718) 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f275a59c700 1 -- 172.21.15.42:0/3278687409 learned_addr learned my addr 172.21.15.42:0/3278687409 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f2759d9b700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c101eb0 0x7f275c196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f275a59c700 1 -- 172.21.15.42:0/3278687409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c101eb0 msgr2=0x7f275c196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f275a59c700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c101eb0 0x7f275c196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f275a59c700 1 -- 172.21.15.42:0/3278687409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500095f0 con 0x7f275c100cb0 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f275a59c700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c100cb0 0x7f275c196550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f274c00b480 tx=0x7f274c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:50.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f274b7fe700 1 -- 172.21.15.42:0/3278687409 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f274c0118c0 con 0x7f275c100cb0 2024-09-16T23:46:50.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f274b7fe700 1 -- 172.21.15.42:0/3278687409 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f274c010380 con 0x7f275c100cb0 2024-09-16T23:46:50.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c19bca0 con 0x7f275c100cb0 2024-09-16T23:46:50.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.992+0000 7f274b7fe700 1 -- 172.21.15.42:0/3278687409 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f274c00f4f0 con 0x7f275c100cb0 2024-09-16T23:46:50.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.993+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c19c1a0 con 0x7f275c100cb0 2024-09-16T23:46:50.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.994+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c1098b0 con 0x7f275c100cb0 2024-09-16T23:46:50.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.994+0000 7f274b7fe700 1 -- 172.21.15.42:0/3278687409 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f274c0104f0 con 0x7f275c100cb0 2024-09-16T23:46:50.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.995+0000 7f274b7fe700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f274406c4e0 0x7f274406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:50.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.995+0000 7f274b7fe700 1 -- 172.21.15.42:0/3278687409 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f274c08b410 con 0x7f275c100cb0 2024-09-16T23:46:50.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.995+0000 7f2759d9b700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f274406c4e0 0x7f274406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:50.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.995+0000 7f2759d9b700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f274406c4e0 0x7f274406e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f275000b5c0 tx=0x7f2750005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:50.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:50.999+0000 7f274b7fe700 1 -- 172.21.15.42:0/3278687409 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f274c0596b0 con 0x7f275c100cb0 2024-09-16T23:46:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:50 smithi043 ceph-mon[35191]: pgmap v204: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-16T23:46:51.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.197+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f275c19c840 con 0x7f275c100cb0 2024-09-16T23:46:51.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.197+0000 7f274b7fe700 1 -- 172.21.15.42:0/3278687409 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f274c059240 con 0x7f275c100cb0 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f274406c4e0 msgr2=0x7f274406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f274406c4e0 0x7f274406e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f275000b5c0 tx=0x7f2750005960 comp rx=0 tx=0).stop 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c100cb0 msgr2=0x7f275c196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c100cb0 0x7f275c196550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f274c00b480 tx=0x7f274c00b840 comp rx=0 tx=0).stop 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 shutdown_connections 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f274406c4e0 0x7f274406e9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f275c100cb0 0x7f275c196550 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 --2- 172.21.15.42:0/3278687409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f275c101eb0 0x7f275c196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 >> 172.21.15.42:0/3278687409 conn(0x7f275c0fc230 msgr2=0x7f275c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 shutdown_connections 2024-09-16T23:46:51.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:51.199+0000 7f2760c6e700 1 -- 172.21.15.42:0/3278687409 wait complete. 2024-09-16T23:46:51.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-16T23:46:52.146 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:52 smithi042 ceph-mon[27914]: pgmap v205: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-16T23:46:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:52 smithi043 ceph-mon[35191]: pgmap v205: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-16T23:46:53.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 -- 172.21.15.42:0/1047650946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 msgr2=0x7f5d701002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:53.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 --2- 172.21.15.42:0/1047650946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d701002f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009910 tx=0x7f5d60009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:53.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 -- 172.21.15.42:0/1047650946 shutdown_connections 2024-09-16T23:46:53.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 --2- 172.21.15.42:0/1047650946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d701002f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:53.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 --2- 172.21.15.42:0/1047650946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d701011c0 0x7f5d701015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:53.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 -- 172.21.15.42:0/1047650946 >> 172.21.15.42:0/1047650946 conn(0x7f5d700fa8c0 msgr2=0x7f5d700fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:53.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 -- 172.21.15.42:0/1047650946 shutdown_connections 2024-09-16T23:46:53.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.912+0000 7f5d779cd700 1 -- 172.21.15.42:0/1047650946 wait complete. 2024-09-16T23:46:53.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d779cd700 1 Processor -- start 2024-09-16T23:46:53.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d779cd700 1 -- start start 2024-09-16T23:46:53.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d779cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d70196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:53.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d779cd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d701011c0 0x7f5d70196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:53.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d779cd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d701970a0 con 0x7f5d700ffe70 2024-09-16T23:46:53.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d779cd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d701971e0 con 0x7f5d701011c0 2024-09-16T23:46:53.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d75769700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d70196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:53.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d75769700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d70196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52386/0 (socket says 172.21.15.42:52386) 2024-09-16T23:46:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.913+0000 7f5d75769700 1 -- 172.21.15.42:0/3913219322 learned_addr learned my addr 172.21.15.42:0/3913219322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d74f68700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d701011c0 0x7f5d70196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d75769700 1 -- 172.21.15.42:0/3913219322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d701011c0 msgr2=0x7f5d70196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d75769700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d701011c0 0x7f5d70196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d75769700 1 -- 172.21.15.42:0/3913219322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600095f0 con 0x7f5d700ffe70 2024-09-16T23:46:53.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d75769700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d70196540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c8f0 tx=0x7f5d6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d667fc700 1 -- 172.21.15.42:0/3913219322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d6c007a70 con 0x7f5d700ffe70 2024-09-16T23:46:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d667fc700 1 -- 172.21.15.42:0/3913219322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5d6c007bd0 con 0x7f5d700ffe70 2024-09-16T23:46:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7019bc90 con 0x7f5d700ffe70 2024-09-16T23:46:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.914+0000 7f5d667fc700 1 -- 172.21.15.42:0/3913219322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d6c018630 con 0x7f5d700ffe70 2024-09-16T23:46:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.915+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7019c190 con 0x7f5d700ffe70 2024-09-16T23:46:53.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.916+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d70108740 con 0x7f5d700ffe70 2024-09-16T23:46:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.916+0000 7f5d667fc700 1 -- 172.21.15.42:0/3913219322 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5d6c00f450 con 0x7f5d700ffe70 2024-09-16T23:46:53.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.916+0000 7f5d667fc700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5d5c06c4e0 0x7f5d5c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:53.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.917+0000 7f5d667fc700 1 -- 172.21.15.42:0/3913219322 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f5d6c08b780 con 0x7f5d700ffe70 2024-09-16T23:46:53.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.917+0000 7f5d74f68700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5d5c06c4e0 0x7f5d5c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:53.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.917+0000 7f5d74f68700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5d5c06c4e0 0x7f5d5c06e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d60005090 tx=0x7f5d6001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:53.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:53.921+0000 7f5d667fc700 1 -- 172.21.15.42:0/3913219322 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5d6c059a40 con 0x7f5d700ffe70 2024-09-16T23:46:54.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.118+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f5d70108930 con 0x7f5d700ffe70 2024-09-16T23:46:54.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.118+0000 7f5d667fc700 1 -- 172.21.15.42:0/3913219322 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f5d6c0595d0 con 0x7f5d700ffe70 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5d5c06c4e0 msgr2=0x7f5d5c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5d5c06c4e0 0x7f5d5c06e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d60005090 tx=0x7f5d6001a040 comp rx=0 tx=0).stop 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 msgr2=0x7f5d70196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d70196540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00c8f0 tx=0x7f5d6c00ccb0 comp rx=0 tx=0).stop 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 shutdown_connections 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f5d5c06c4e0 0x7f5d5c06e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d700ffe70 0x7f5d70196540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 --2- 172.21.15.42:0/3913219322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d701011c0 0x7f5d70196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 >> 172.21.15.42:0/3913219322 conn(0x7f5d700fa8c0 msgr2=0x7f5d700fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:54.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 shutdown_connections 2024-09-16T23:46:54.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:54.120+0000 7f5d779cd700 1 -- 172.21.15.42:0/3913219322 wait complete. 2024-09-16T23:46:54.865 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:54 smithi042 ceph-mon[27914]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:46:54.865 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:54.891 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-16T23:46:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:54 smithi043 ceph-mon[35191]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:46:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:46:56.876 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:56 smithi042 ceph-mon[27914]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:46:56.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.878+0000 7f152398f700 1 -- 172.21.15.42:0/2486394451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 msgr2=0x7f151c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.878+0000 7f152398f700 1 --2- 172.21.15.42:0/2486394451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c0731e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1518009910 tx=0x7f1518009c20 comp rx=0 tx=0).stop 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.878+0000 7f152398f700 1 -- 172.21.15.42:0/2486394451 shutdown_connections 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.878+0000 7f152398f700 1 --2- 172.21.15.42:0/2486394451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c0737b0 0x7f151c073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.878+0000 7f152398f700 1 --2- 172.21.15.42:0/2486394451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c0731e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.878+0000 7f152398f700 1 -- 172.21.15.42:0/2486394451 >> 172.21.15.42:0/2486394451 conn(0x7f151c0fc230 msgr2=0x7f151c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.878+0000 7f152398f700 1 -- 172.21.15.42:0/2486394451 shutdown_connections 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.879+0000 7f152398f700 1 -- 172.21.15.42:0/2486394451 wait complete. 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.879+0000 7f152398f700 1 Processor -- start 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.879+0000 7f152398f700 1 -- start start 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.879+0000 7f152398f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c0737b0 0x7f151c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.879+0000 7f152398f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.879+0000 7f152398f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c1991e0 con 0x7f151c074d80 2024-09-16T23:46:56.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.879+0000 7f152398f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c199320 con 0x7f151c0737b0 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f1520f2a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f1520f2a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52406/0 (socket says 172.21.15.42:52406) 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f1520f2a700 1 -- 172.21.15.42:0/2018933593 learned_addr learned my addr 172.21.15.42:0/2018933593 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f152172b700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c0737b0 0x7f151c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f1520f2a700 1 -- 172.21.15.42:0/2018933593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c0737b0 msgr2=0x7f151c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f1520f2a700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c0737b0 0x7f151c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f1520f2a700 1 -- 172.21.15.42:0/2018933593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180095f0 con 0x7f151c074d80 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f1520f2a700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c198bc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d8d0 tx=0x7f150c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f15127fc700 1 -- 172.21.15.42:0/2018933593 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150c0098a0 con 0x7f151c074d80 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f15127fc700 1 -- 172.21.15.42:0/2018933593 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f150c010460 con 0x7f151c074d80 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c19ddd0 con 0x7f151c074d80 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f15127fc700 1 -- 172.21.15.42:0/2018933593 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150c00f5f0 con 0x7f151c074d80 2024-09-16T23:46:56.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.880+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c19e2f0 con 0x7f151c074d80 2024-09-16T23:46:56.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:56 smithi043 ceph-mon[35191]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:46:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.882+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1500005320 con 0x7f151c074d80 2024-09-16T23:46:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.882+0000 7f15127fc700 1 -- 172.21.15.42:0/2018933593 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f150c009a00 con 0x7f151c074d80 2024-09-16T23:46:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.883+0000 7f15127fc700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f150806c4e0 0x7f150806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:46:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.883+0000 7f15127fc700 1 -- 172.21.15.42:0/2018933593 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f150c08b5e0 con 0x7f151c074d80 2024-09-16T23:46:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.883+0000 7f152172b700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f150806c4e0 0x7f150806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:46:56.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.883+0000 7f152172b700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f150806c4e0 0x7f150806e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1518006010 tx=0x7f151800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:46:56.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:56.887+0000 7f15127fc700 1 -- 172.21.15.42:0/2018933593 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f150c059880 con 0x7f151c074d80 2024-09-16T23:46:57.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.084+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f1500005f70 con 0x7f151c074d80 2024-09-16T23:46:57.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.084+0000 7f15127fc700 1 -- 172.21.15.42:0/2018933593 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f150c059410 con 0x7f151c074d80 2024-09-16T23:46:57.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f150806c4e0 msgr2=0x7f150806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:57.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f150806c4e0 0x7f150806e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1518006010 tx=0x7f151800b540 comp rx=0 tx=0).stop 2024-09-16T23:46:57.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 msgr2=0x7f151c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:46:57.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c198bc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f150c00d8d0 tx=0x7f150c00dbe0 comp rx=0 tx=0).stop 2024-09-16T23:46:57.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 shutdown_connections 2024-09-16T23:46:57.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f150806c4e0 0x7f150806e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:57.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f151c0737b0 0x7f151c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:57.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 --2- 172.21.15.42:0/2018933593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f151c074d80 0x7f151c198bc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:46:57.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 >> 172.21.15.42:0/2018933593 conn(0x7f151c0fc230 msgr2=0x7f151c106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:46:57.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 shutdown_connections 2024-09-16T23:46:57.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:46:57.086+0000 7f152398f700 1 -- 172.21.15.42:0/2018933593 wait complete. 2024-09-16T23:46:57.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-09-16T23:46:58.053 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:46:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:46:58 smithi042 ceph-mon[27914]: pgmap v208: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:46:59.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:46:58 smithi043 ceph-mon[35191]: pgmap v208: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:47:00.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 -- 172.21.15.42:0/4108030616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c101800 msgr2=0x7fb33c103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:00.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 --2- 172.21.15.42:0/4108030616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c101800 0x7fb33c103bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b0a0 tx=0x7fb32c00b3b0 comp rx=0 tx=0).stop 2024-09-16T23:47:00.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 -- 172.21.15.42:0/4108030616 shutdown_connections 2024-09-16T23:47:00.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 --2- 172.21.15.42:0/4108030616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c104130 0x7fb33c106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:00.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 --2- 172.21.15.42:0/4108030616 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c101800 0x7fb33c103bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:00.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 -- 172.21.15.42:0/4108030616 >> 172.21.15.42:0/4108030616 conn(0x7fb33c0fb190 msgr2=0x7fb33c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:00.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 -- 172.21.15.42:0/4108030616 shutdown_connections 2024-09-16T23:47:00.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.025+0000 7fb341fae700 1 -- 172.21.15.42:0/4108030616 wait complete. 2024-09-16T23:47:00.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb341fae700 1 Processor -- start 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb341fae700 1 -- start start 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb341fae700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c101800 0x7fb33c101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb341fae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c104130 0x7fb33c0ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb341fae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0ffcf0 con 0x7fb33c104130 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb341fae700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0ffe30 con 0x7fb33c101800 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb33affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c104130 0x7fb33c0ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb33affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c104130 0x7fb33c0ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52424/0 (socket says 172.21.15.42:52424) 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.026+0000 7fb33affd700 1 -- 172.21.15.42:0/214867910 learned_addr learned my addr 172.21.15.42:0/214867910 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb33b7fe700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c101800 0x7fb33c101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb33affd700 1 -- 172.21.15.42:0/214867910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c101800 msgr2=0x7fb33c101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:00.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb33affd700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c101800 0x7fb33c101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb33affd700 1 -- 172.21.15.42:0/214867910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009d00 con 0x7fb33c104130 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb33b7fe700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c101800 0x7fb33c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb33affd700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c104130 0x7fb33c0ff7b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb33000b4b0 tx=0x7fb33000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb338ff9700 1 -- 172.21.15.42:0/214867910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb330011800 con 0x7fb33c104130 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c100110 con 0x7fb33c104130 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb338ff9700 1 -- 172.21.15.42:0/214867910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb330011e10 con 0x7fb33c104130 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb338ff9700 1 -- 172.21.15.42:0/214867910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33000f2e0 con 0x7fb33c104130 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.027+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c19e5e0 con 0x7fb33c104130 2024-09-16T23:47:00.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.028+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c190690 con 0x7fb33c104130 2024-09-16T23:47:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.029+0000 7fb338ff9700 1 -- 172.21.15.42:0/214867910 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb330010380 con 0x7fb33c104130 2024-09-16T23:47:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.029+0000 7fb338ff9700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb32406c4e0 0x7fb32406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.029+0000 7fb338ff9700 1 -- 172.21.15.42:0/214867910 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb33008b420 con 0x7fb33c104130 2024-09-16T23:47:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.029+0000 7fb33b7fe700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb32406c4e0 0x7fb32406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:00.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.030+0000 7fb33b7fe700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb32406c4e0 0x7fb32406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b830 tx=0x7fb32c006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:00.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.034+0000 7fb338ff9700 1 -- 172.21.15.42:0/214867910 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb3300596e0 con 0x7fb33c104130 2024-09-16T23:47:00.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:00.254+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fb33c061280 con 0x7fb32406c4e0 2024-09-16T23:47:01.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:00 smithi042 ceph-mon[27914]: pgmap v209: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:47:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:00 smithi043 ceph-mon[35191]: pgmap v209: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:47:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:01 smithi042 ceph-mon[27914]: from='client.14536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:02.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:01 smithi043 ceph-mon[35191]: from='client.14536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:03.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:02 smithi042 ceph-mon[27914]: pgmap v210: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-09-16T23:47:03.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:02 smithi043 ceph-mon[35191]: pgmap v210: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-09-16T23:47:05.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:04 smithi042 ceph-mon[27914]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-16T23:47:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:04 smithi043 ceph-mon[35191]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-09-16T23:47:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:06 smithi042 ceph-mon[27914]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T23:47:07.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:06 smithi043 ceph-mon[35191]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T23:47:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:08 smithi042 ceph-mon[27914]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T23:47:09.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:08 smithi043 ceph-mon[35191]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T23:47:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:10.028 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:09 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:10 smithi042 ceph-mon[27914]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T23:47:11.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:10 smithi043 ceph-mon[35191]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T23:47:12.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:12 smithi042 ceph-mon[27914]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:47:13.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:12 smithi043 ceph-mon[35191]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:47:15.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:14 smithi042 ceph-mon[27914]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:47:15.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:14 smithi043 ceph-mon[35191]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:47:16.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:16 smithi043 ceph-mon[35191]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:16 smithi042 ceph-mon[27914]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:18 smithi042 ceph-mon[27914]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:18 smithi043 ceph-mon[35191]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:21.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:20 smithi042 ceph-mon[27914]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:21.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:20 smithi043 ceph-mon[35191]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:22 smithi042 ceph-mon[27914]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:23.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:22 smithi043 ceph-mon[35191]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:25.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:24 smithi042 ceph-mon[27914]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:25.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:25.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:24 smithi043 ceph-mon[35191]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:26.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:26 smithi043 ceph-mon[35191]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:27.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:26 smithi042 ceph-mon[27914]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:29.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:28 smithi042 ceph-mon[27914]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:29.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:28 smithi043 ceph-mon[35191]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:31.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:30 smithi042 ceph-mon[27914]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:31.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:30 smithi043 ceph-mon[35191]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:33.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:32 smithi042 ceph-mon[27914]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:33.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:32 smithi043 ceph-mon[35191]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:34.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:34 smithi042 ceph-mon[27914]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:35.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:34 smithi043 ceph-mon[35191]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:35.969 INFO:teuthology.orchestra.run.smithi042.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:47:35.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.972+0000 7fb338ff9700 1 -- 172.21.15.42:0/214867910 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fb33c061280 con 0x7fb32406c4e0 2024-09-16T23:47:35.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.974+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb32406c4e0 msgr2=0x7fb32406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:35.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.974+0000 7fb341fae700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb32406c4e0 0x7fb32406e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00b830 tx=0x7fb32c006000 comp rx=0 tx=0).stop 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c104130 msgr2=0x7fb33c0ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c104130 0x7fb33c0ff7b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb33000b4b0 tx=0x7fb33000b870 comp rx=0 tx=0).stop 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 shutdown_connections 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb32406c4e0 0x7fb32406e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb33c101800 0x7fb33c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 --2- 172.21.15.42:0/214867910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb33c104130 0x7fb33c0ff7b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 >> 172.21.15.42:0/214867910 conn(0x7fb33c0fb190 msgr2=0x7fb33c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 shutdown_connections 2024-09-16T23:47:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:35.975+0000 7fb341fae700 1 -- 172.21.15.42:0/214867910 wait complete. 2024-09-16T23:47:36.772 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-09-16T23:47:37.077 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:47:37.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:36 smithi043 ceph-mon[35191]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:37.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:36 smithi043 ceph-mon[35191]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:47:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:36 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:36 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:47:37.242 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:36 smithi042 ceph-mon[27914]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:37.242 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:36 smithi042 ceph-mon[27914]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:47:37.242 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:36 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:37.242 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:36 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:47:38.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:47:38.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:47:38.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:37 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:38.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:47:38.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:47:38.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:37 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:39.195 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:38 smithi042 ceph-mon[27914]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:39.195 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:38 smithi042 ceph-mon[27914]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:47:39.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:38 smithi043 ceph-mon[35191]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:38 smithi043 ceph-mon[35191]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:47:40.263 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:40.278 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:41.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:40 smithi042 ceph-mon[27914]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:41.295 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:40 smithi043 ceph-mon[35191]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.918+0000 7f5930d69700 1 -- 172.21.15.42:0/3830386716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 msgr2=0x7f592c0ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.918+0000 7f5930d69700 1 --2- 172.21.15.42:0/3830386716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 0x7f592c0ff380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f591c009910 tx=0x7f591c009c20 comp rx=0 tx=0).stop 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.919+0000 7f5930d69700 1 -- 172.21.15.42:0/3830386716 shutdown_connections 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.919+0000 7f5930d69700 1 --2- 172.21.15.42:0/3830386716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c101f40 0x7f592c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.919+0000 7f5930d69700 1 --2- 172.21.15.42:0/3830386716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 0x7f592c0ff380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.919+0000 7f5930d69700 1 -- 172.21.15.42:0/3830386716 >> 172.21.15.42:0/3830386716 conn(0x7f592c0fa8c0 msgr2=0x7f592c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.919+0000 7f5930d69700 1 -- 172.21.15.42:0/3830386716 shutdown_connections 2024-09-16T23:47:41.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.919+0000 7f5930d69700 1 -- 172.21.15.42:0/3830386716 wait complete. 2024-09-16T23:47:41.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.920+0000 7f5930d69700 1 Processor -- start 2024-09-16T23:47:41.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.921+0000 7f5930d69700 1 -- start start 2024-09-16T23:47:41.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.921+0000 7f5930d69700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 0x7f592c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:41.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.921+0000 7f5930d69700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c101f40 0x7f592c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:41.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.921+0000 7f5930d69700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c197030 con 0x7f592c0fef60 2024-09-16T23:47:41.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.921+0000 7f5930d69700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c197170 con 0x7f592c101f40 2024-09-16T23:47:41.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f592a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 0x7f592c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:41.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f5929d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c101f40 0x7f592c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:41.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f592a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 0x7f592c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54498/0 (socket says 172.21.15.42:54498) 2024-09-16T23:47:41.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f592a59c700 1 -- 172.21.15.42:0/1961074847 learned_addr learned my addr 172.21.15.42:0/1961074847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:47:41.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f5929d9b700 1 -- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 msgr2=0x7f592c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:41.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f5929d9b700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 0x7f592c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:41.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f5929d9b700 1 -- 172.21.15.42:0/1961074847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0095f0 con 0x7f592c101f40 2024-09-16T23:47:41.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.922+0000 7f5929d9b700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c101f40 0x7f592c196a10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f592000e970 tx=0x7f592000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:41.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.923+0000 7f591b7fe700 1 -- 172.21.15.42:0/1961074847 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f592000c9d0 con 0x7f592c101f40 2024-09-16T23:47:41.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.923+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c19bc20 con 0x7f592c101f40 2024-09-16T23:47:41.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.923+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c19c170 con 0x7f592c101f40 2024-09-16T23:47:41.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.923+0000 7f591b7fe700 1 -- 172.21.15.42:0/1961074847 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5920010430 con 0x7f592c101f40 2024-09-16T23:47:41.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.923+0000 7f591b7fe700 1 -- 172.21.15.42:0/1961074847 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5920018630 con 0x7f592c101f40 2024-09-16T23:47:41.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.925+0000 7f591b7fe700 1 -- 172.21.15.42:0/1961074847 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f5920018790 con 0x7f592c101f40 2024-09-16T23:47:41.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.925+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c005320 con 0x7f592c101f40 2024-09-16T23:47:41.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.926+0000 7f591b7fe700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f591406c4e0 0x7f591406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:41.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.926+0000 7f591b7fe700 1 -- 172.21.15.42:0/1961074847 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f5920014070 con 0x7f592c101f40 2024-09-16T23:47:41.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.926+0000 7f592a59c700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f591406c4e0 0x7f591406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:41.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.926+0000 7f592a59c700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f591406c4e0 0x7f591406e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b6f0 tx=0x7f591c005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:41.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:41.934+0000 7f591b7fe700 1 -- 172.21.15.42:0/1961074847 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f59200520e0 con 0x7f592c101f40 2024-09-16T23:47:42.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.169+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f590c000bf0 con 0x7f591406c4e0 2024-09-16T23:47:42.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.170+0000 7f591b7fe700 1 -- 172.21.15.42:0/1961074847 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f590c000bf0 con 0x7f591406c4e0 2024-09-16T23:47:42.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f591406c4e0 msgr2=0x7f591406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f591406c4e0 0x7f591406e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f591c00b6f0 tx=0x7f591c005a90 comp rx=0 tx=0).stop 2024-09-16T23:47:42.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c101f40 msgr2=0x7f592c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c101f40 0x7f592c196a10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f592000e970 tx=0x7f592000ec80 comp rx=0 tx=0).stop 2024-09-16T23:47:42.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 shutdown_connections 2024-09-16T23:47:42.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f591406c4e0 0x7f591406e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f592c0fef60 0x7f592c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 --2- 172.21.15.42:0/1961074847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f592c101f40 0x7f592c196a10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 >> 172.21.15.42:0/1961074847 conn(0x7f592c0fa8c0 msgr2=0x7f592c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:42.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 shutdown_connections 2024-09-16T23:47:42.171 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.173+0000 7f5930d69700 1 -- 172.21.15.42:0/1961074847 wait complete. 2024-09-16T23:47:42.182 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 -- 172.21.15.42:0/1979795370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 msgr2=0x7f0afc0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 --2- 172.21.15.42:0/1979795370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc0ff9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009910 tx=0x7f0af8009c20 comp rx=0 tx=0).stop 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 -- 172.21.15.42:0/1979795370 shutdown_connections 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 --2- 172.21.15.42:0/1979795370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0afc0fff00 0x7f0afc100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 --2- 172.21.15.42:0/1979795370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc0ff9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 -- 172.21.15.42:0/1979795370 >> 172.21.15.42:0/1979795370 conn(0x7f0afc0fb160 msgr2=0x7f0afc0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 -- 172.21.15.42:0/1979795370 shutdown_connections 2024-09-16T23:47:42.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.286+0000 7f0b04237700 1 -- 172.21.15.42:0/1979795370 wait complete. 2024-09-16T23:47:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.287+0000 7f0b04237700 1 Processor -- start 2024-09-16T23:47:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.287+0000 7f0b04237700 1 -- start start 2024-09-16T23:47:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.287+0000 7f0b04237700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc105120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:42.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.287+0000 7f0b04237700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0afc0fff00 0x7f0afc105660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.287+0000 7f0b04237700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc101c10 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b04237700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc101d80 con 0x7f0afc0fff00 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b01fd3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc105120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b01fd3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc105120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54504/0 (socket says 172.21.15.42:54504) 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b01fd3700 1 -- 172.21.15.42:0/2447804437 learned_addr learned my addr 172.21.15.42:0/2447804437 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b017d2700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0afc0fff00 0x7f0afc105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b01fd3700 1 -- 172.21.15.42:0/2447804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0afc0fff00 msgr2=0x7f0afc105660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b01fd3700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0afc0fff00 0x7f0afc105660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b01fd3700 1 -- 172.21.15.42:0/2447804437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80095f0 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b017d2700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0afc0fff00 0x7f0afc105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:47:42.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0b01fd3700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc105120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0af8006010 tx=0x7f0af800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:42.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.288+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2447804437 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af801d070 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.289+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2447804437 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0af800f460 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.289+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2447804437 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0af8017630 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.289+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc102000 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.289+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc1024c0 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.291+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc192830 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.291+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2447804437 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0af800f5d0 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.291+0000 7f0af2ffd700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0ae806c4e0 0x7f0ae806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:42.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.292+0000 7f0b017d2700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0ae806c4e0 0x7f0ae806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:42.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.292+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2447804437 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0af805eb30 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.292+0000 7f0b017d2700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0ae806c4e0 0x7f0ae806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0aec005950 tx=0x7f0aec0058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:42.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.297+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2447804437 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0af80577d0 con 0x7f0afc0ff5a0 2024-09-16T23:47:42.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: Upgrade: Target is version 19.1.1-330-gc2ddf972 (unknown) 2024-09-16T23:47:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3'] 2024-09-16T23:47:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:47:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: Upgrade: Need to upgrade myself (mgr.smithi042.zrkumz) 2024-09-16T23:47:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:42 smithi042 ceph-mon[27914]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 on smithi043 2024-09-16T23:47:42.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.544+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0afc02cdd0 con 0x7f0ae806c4e0 2024-09-16T23:47:42.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.546+0000 7f0af2ffd700 1 -- 172.21.15.42:0/2447804437 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0afc02cdd0 con 0x7f0ae806c4e0 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.548+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0ae806c4e0 msgr2=0x7f0ae806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.548+0000 7f0b04237700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0ae806c4e0 0x7f0ae806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0aec005950 tx=0x7f0aec0058e0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.548+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 msgr2=0x7f0afc105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.548+0000 7f0b04237700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc105120 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0af8006010 tx=0x7f0af800be20 comp rx=0 tx=0).stop 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.549+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 shutdown_connections 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.549+0000 7f0b04237700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0ae806c4e0 0x7f0ae806e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.549+0000 7f0b04237700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0afc0ff5a0 0x7f0afc105120 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.549+0000 7f0b04237700 1 --2- 172.21.15.42:0/2447804437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0afc0fff00 0x7f0afc105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.549+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 >> 172.21.15.42:0/2447804437 conn(0x7f0afc0fb160 msgr2=0x7f0afc107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:42.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.549+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 shutdown_connections 2024-09-16T23:47:42.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.549+0000 7f0b04237700 1 -- 172.21.15.42:0/2447804437 wait complete. 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: Upgrade: Target is version 19.1.1-330-gc2ddf972 (unknown) 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3'] 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: Upgrade: Need to upgrade myself (mgr.smithi042.zrkumz) 2024-09-16T23:47:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:42 smithi043 ceph-mon[35191]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 on smithi043 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 -- 172.21.15.42:0/1982205016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104070 msgr2=0x7f2d70106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 --2- 172.21.15.42:0/1982205016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104070 0x7f2d70106460 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009910 tx=0x7f2d60009c20 comp rx=0 tx=0).stop 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 -- 172.21.15.42:0/1982205016 shutdown_connections 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 --2- 172.21.15.42:0/1982205016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70104070 0x7f2d70106460 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 --2- 172.21.15.42:0/1982205016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70101740 0x7f2d70103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 -- 172.21.15.42:0/1982205016 >> 172.21.15.42:0/1982205016 conn(0x7f2d700fb150 msgr2=0x7f2d700fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 -- 172.21.15.42:0/1982205016 shutdown_connections 2024-09-16T23:47:42.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.665+0000 7f2d7839d700 1 -- 172.21.15.42:0/1982205016 wait complete. 2024-09-16T23:47:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.666+0000 7f2d7839d700 1 Processor -- start 2024-09-16T23:47:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.666+0000 7f2d7839d700 1 -- start start 2024-09-16T23:47:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.666+0000 7f2d7839d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70101740 0x7f2d70196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.666+0000 7f2d7839d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70104070 0x7f2d701969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:42.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d7839d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d70196fc0 con 0x7f2d70101740 2024-09-16T23:47:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d7839d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d70197100 con 0x7f2d70104070 2024-09-16T23:47:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d76139700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70101740 0x7f2d70196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d76139700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70101740 0x7f2d70196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36184/0 (socket says 172.21.15.42:36184) 2024-09-16T23:47:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d76139700 1 -- 172.21.15.42:0/3086387479 learned_addr learned my addr 172.21.15.42:0/3086387479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:47:42.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d75938700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70104070 0x7f2d701969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d76139700 1 -- 172.21.15.42:0/3086387479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70104070 msgr2=0x7f2d701969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d76139700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70104070 0x7f2d701969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d76139700 1 -- 172.21.15.42:0/3086387479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d600095f0 con 0x7f2d70101740 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.667+0000 7f2d76139700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70101740 0x7f2d70196460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00d8d0 tx=0x7f2d6c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.668+0000 7f2d677fe700 1 -- 172.21.15.42:0/3086387479 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d6c0098a0 con 0x7f2d70101740 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.668+0000 7f2d677fe700 1 -- 172.21.15.42:0/3086387479 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c010460 con 0x7f2d70101740 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.668+0000 7f2d677fe700 1 -- 172.21.15.42:0/3086387479 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d6c0098a0 con 0x7f2d70101740 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.668+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7019bbb0 con 0x7f2d70101740 2024-09-16T23:47:42.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.668+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7019c0b0 con 0x7f2d70101740 2024-09-16T23:47:42.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.669+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d70190620 con 0x7f2d70101740 2024-09-16T23:47:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.671+0000 7f2d677fe700 1 -- 172.21.15.42:0/3086387479 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f2d6c0105d0 con 0x7f2d70101740 2024-09-16T23:47:42.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.671+0000 7f2d677fe700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2d5c06c550 0x7f2d5c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:42.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.671+0000 7f2d677fe700 1 -- 172.21.15.42:0/3086387479 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f2d6c08b310 con 0x7f2d70101740 2024-09-16T23:47:42.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.672+0000 7f2d75938700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2d5c06c550 0x7f2d5c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:42.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.673+0000 7f2d75938700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2d5c06c550 0x7f2d5c06ea10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d6000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:42.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.676+0000 7f2d677fe700 1 -- 172.21.15.42:0/3086387479 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2d6c059570 con 0x7f2d70101740 2024-09-16T23:47:42.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.905+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d700611d0 con 0x7f2d5c06c550 2024-09-16T23:47:42.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.927+0000 7f2d677fe700 1 -- 172.21.15.42:0/3086387479 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f2d700611d0 con 0x7f2d5c06c550 2024-09-16T23:47:42.925 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:47:42.925 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (5m) 62s ago 7m 25.5M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:47:42.925 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (7m) 62s ago 7m 7453k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:47:42.926 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (6m) 69s ago 6m 7386k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:47:42.926 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (7m) 62s ago 7m 6660k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:47:42.926 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (6m) 69s ago 6m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:47:42.926 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (5m) 62s ago 6m 83.1M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:47:42.926 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (84s) 62s ago 83s 15.9M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:47:42.927 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (92s) 62s ago 91s 11.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:47:42.927 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (89s) 69s ago 88s 16.3M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:47:42.927 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (81s) 69s ago 81s 13.0M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:47:42.927 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:9283,8765,8443 running (8m) 62s ago 8m 488M - 18.2.1 5be31c24972a cd8abd4f468e 2024-09-16T23:47:42.927 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (6m) 69s ago 6m 442M - 18.2.1 5be31c24972a b8099e88d36b 2024-09-16T23:47:42.927 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (8m) 62s ago 8m 45.4M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:47:42.928 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (6m) 69s ago 5m 38.5M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:47:42.928 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (7m) 62s ago 7m 16.2M - 1.5.0 0da6a335fe13 7c84e9bf6c1c 2024-09-16T23:47:42.928 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (6m) 69s ago 6m 16.4M - 1.5.0 0da6a335fe13 82d41c3cc1ee 2024-09-16T23:47:42.928 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (5m) 62s ago 5m 48.3M 3948M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:47:42.928 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (4m) 62s ago 4m 47.2M 3948M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:47:42.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (4m) 62s ago 4m 46.4M 3948M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:47:42.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (3m) 69s ago 3m 47.2M 4972M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:47:42.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (3m) 69s ago 3m 43.9M 4972M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:47:42.929 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (2m) 69s ago 2m 44.5M 4972M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:47:42.929 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (5m) 62s ago 6m 56.4M - 2.43.0 a07b618ecd1d 8ebaffdf2443 2024-09-16T23:47:42.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.929+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2d5c06c550 msgr2=0x7f2d5c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.929+0000 7f2d7839d700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2d5c06c550 0x7f2d5c06ea10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2d60006010 tx=0x7f2d6000b540 comp rx=0 tx=0).stop 2024-09-16T23:47:42.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70101740 msgr2=0x7f2d70196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:42.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70101740 0x7f2d70196460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00d8d0 tx=0x7f2d6c00dbe0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 shutdown_connections 2024-09-16T23:47:42.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f2d5c06c550 0x7f2d5c06ea10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d70101740 0x7f2d70196460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 --2- 172.21.15.42:0/3086387479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d70104070 0x7f2d701969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:42.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 >> 172.21.15.42:0/3086387479 conn(0x7f2d700fb150 msgr2=0x7f2d700fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:42.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 shutdown_connections 2024-09-16T23:47:42.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:42.930+0000 7f2d7839d700 1 -- 172.21.15.42:0/3086387479 wait complete. 2024-09-16T23:47:43.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2094040734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b40ffbb0 msgr2=0x7f70b40fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:43.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2094040734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b40ffbb0 0x7f70b40fffd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f70a4009910 tx=0x7f70a4009c20 comp rx=0 tx=0).stop 2024-09-16T23:47:43.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2094040734 shutdown_connections 2024-09-16T23:47:43.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2094040734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b4100db0 0x7f70b4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2094040734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b40ffbb0 0x7f70b40fffd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2094040734 >> 172.21.15.42:0/2094040734 conn(0x7f70b40fb150 msgr2=0x7f70b40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:43.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2094040734 shutdown_connections 2024-09-16T23:47:43.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.103+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2094040734 wait complete. 2024-09-16T23:47:43.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.105+0000 7f70ba7dc700 1 Processor -- start 2024-09-16T23:47:43.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.105+0000 7f70ba7dc700 1 -- start start 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.105+0000 7f70ba7dc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b40ffbb0 0x7f70b4194330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.105+0000 7f70ba7dc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4100db0 0x7f70b4194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.105+0000 7f70ba7dc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4194e90 con 0x7f70b40ffbb0 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.105+0000 7f70ba7dc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4194fd0 con 0x7f70b4100db0 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.106+0000 7f70b3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b40ffbb0 0x7f70b4194330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.106+0000 7f70b3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b40ffbb0 0x7f70b4194330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36210/0 (socket says 172.21.15.42:36210) 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.106+0000 7f70b3fff700 1 -- 172.21.15.42:0/2034828057 learned_addr learned my addr 172.21.15.42:0/2034828057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.106+0000 7f70b3fff700 1 -- 172.21.15.42:0/2034828057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4100db0 msgr2=0x7f70b4194870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:43.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.106+0000 7f70b3fff700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4100db0 0x7f70b4194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.106+0000 7f70b3fff700 1 -- 172.21.15.42:0/2034828057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a40095f0 con 0x7f70b40ffbb0 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.106+0000 7f70b3fff700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b40ffbb0 0x7f70b4194330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.107+0000 7f70b17fa700 1 -- 172.21.15.42:0/2034828057 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a401d070 con 0x7f70b40ffbb0 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.107+0000 7f70b17fa700 1 -- 172.21.15.42:0/2034828057 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70a4005dc0 con 0x7f70b40ffbb0 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.107+0000 7f70b17fa700 1 -- 172.21.15.42:0/2034828057 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a40226a0 con 0x7f70b40ffbb0 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.107+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b419de30 con 0x7f70b40ffbb0 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.107+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b419e2d0 con 0x7f70b40ffbb0 2024-09-16T23:47:43.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.108+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4107600 con 0x7f70b40ffbb0 2024-09-16T23:47:43.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.110+0000 7f70b17fa700 1 -- 172.21.15.42:0/2034828057 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f70a4003910 con 0x7f70b40ffbb0 2024-09-16T23:47:43.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.111+0000 7f70b17fa700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f709c06c530 0x7f709c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:43.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.111+0000 7f70b17fa700 1 -- 172.21.15.42:0/2034828057 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f70a408cfe0 con 0x7f70b40ffbb0 2024-09-16T23:47:43.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.111+0000 7f70b37fe700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f709c06c530 0x7f709c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:43.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.113+0000 7f70b37fe700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f709c06c530 0x7f709c06e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b3c0 tx=0x7f70a800d040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:43.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.115+0000 7f70b17fa700 1 -- 172.21.15.42:0/2034828057 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f70a405b3c0 con 0x7f70b40ffbb0 2024-09-16T23:47:43.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.549+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f70b419e9d0 con 0x7f70b40ffbb0 2024-09-16T23:47:43.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.551+0000 7f70b17fa700 1 -- 172.21.15.42:0/2034828057 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f70a405af50 con 0x7f70b40ffbb0 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:47:43.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:47:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:47:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-09-16T23:47:43.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:47:43.550 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:47:43.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.554+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f709c06c530 msgr2=0x7f709c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:43.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.554+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f709c06c530 0x7f709c06e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f70a800b3c0 tx=0x7f70a800d040 comp rx=0 tx=0).stop 2024-09-16T23:47:43.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.554+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b40ffbb0 msgr2=0x7f70b4194330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:43.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.554+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b40ffbb0 0x7f70b4194330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f70a4000c00 tx=0x7f70a400f740 comp rx=0 tx=0).stop 2024-09-16T23:47:43.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.555+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 shutdown_connections 2024-09-16T23:47:43.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.555+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f709c06c530 0x7f709c06e9f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.555+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70b40ffbb0 0x7f70b4194330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.555+0000 7f70ba7dc700 1 --2- 172.21.15.42:0/2034828057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b4100db0 0x7f70b4194870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.555+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 >> 172.21.15.42:0/2034828057 conn(0x7f70b40fb150 msgr2=0x7f70b4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:43.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.555+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 shutdown_connections 2024-09-16T23:47:43.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.555+0000 7f70ba7dc700 1 -- 172.21.15.42:0/2034828057 wait complete. 2024-09-16T23:47:43.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:43 smithi043 ceph-mon[35191]: from='client.24315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:43.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:43 smithi043 ceph-mon[35191]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:43.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:43 smithi043 ceph-mon[35191]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:43.688 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:43 smithi042 ceph-mon[27914]: from='client.24315 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:43.688 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:43 smithi042 ceph-mon[27914]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:43.688 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:43 smithi042 ceph-mon[27914]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.718+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2690816086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 msgr2=0x7f0e0c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.718+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2690816086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 0x7f0e0c101220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009870 tx=0x7f0e00009b80 comp rx=0 tx=0).stop 2024-09-16T23:47:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.719+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2690816086 shutdown_connections 2024-09-16T23:47:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.719+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2690816086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 0x7f0e0c101220 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.719+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2690816086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e0c0ffbc0 0x7f0e0c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.719+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2690816086 >> 172.21.15.42:0/2690816086 conn(0x7f0e0c0fb160 msgr2=0x7f0e0c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.720+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2690816086 shutdown_connections 2024-09-16T23:47:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.720+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2690816086 wait complete. 2024-09-16T23:47:43.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.721+0000 7f0e10cd0700 1 Processor -- start 2024-09-16T23:47:43.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.721+0000 7f0e10cd0700 1 -- start start 2024-09-16T23:47:43.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.721+0000 7f0e10cd0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e0c0ffbc0 0x7f0e0c1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:43.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.721+0000 7f0e10cd0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 0x7f0e0c196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e0a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e0c0ffbc0 0x7f0e0c1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e10cd0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c197120 con 0x7f0e0c100dc0 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e10cd0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c197260 con 0x7f0e0c0ffbc0 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e0a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e0c0ffbc0 0x7f0e0c1965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:54626/0 (socket says 172.21.15.42:54626) 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e0a59c700 1 -- 172.21.15.42:0/2353591193 learned_addr learned my addr 172.21.15.42:0/2353591193 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e09d9b700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 0x7f0e0c196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e09d9b700 1 -- 172.21.15.42:0/2353591193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e0c0ffbc0 msgr2=0x7f0e0c1965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:43.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e09d9b700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e0c0ffbc0 0x7f0e0c1965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.722+0000 7f0e09d9b700 1 -- 172.21.15.42:0/2353591193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0095f0 con 0x7f0e0c100dc0 2024-09-16T23:47:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.723+0000 7f0e09d9b700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 0x7f0e0c196b00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009870 tx=0x7f0e000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.723+0000 7f0dfb7fe700 1 -- 172.21.15.42:0/2353591193 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0001c070 con 0x7f0e0c100dc0 2024-09-16T23:47:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.723+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e00009520 con 0x7f0e0c100dc0 2024-09-16T23:47:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.723+0000 7f0dfb7fe700 1 -- 172.21.15.42:0/2353591193 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e0000f460 con 0x7f0e0c100dc0 2024-09-16T23:47:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.723+0000 7f0dfb7fe700 1 -- 172.21.15.42:0/2353591193 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e00022630 con 0x7f0e0c100dc0 2024-09-16T23:47:43.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.723+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c106360 con 0x7f0e0c100dc0 2024-09-16T23:47:43.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.725+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c068af0 con 0x7f0e0c100dc0 2024-09-16T23:47:43.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.725+0000 7f0dfb7fe700 1 -- 172.21.15.42:0/2353591193 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f0e00020070 con 0x7f0e0c100dc0 2024-09-16T23:47:43.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.726+0000 7f0dfb7fe700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0df406c4e0 0x7f0df406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:47:43.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.726+0000 7f0dfb7fe700 1 -- 172.21.15.42:0/2353591193 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0e0008cda0 con 0x7f0e0c100dc0 2024-09-16T23:47:43.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.726+0000 7f0e0a59c700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0df406c4e0 0x7f0df406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:47:43.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.727+0000 7f0e0a59c700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0df406c4e0 0x7f0df406e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005950 tx=0x7f0dfc012040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:47:43.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:43.733+0000 7f0dfb7fe700 1 -- 172.21.15.42:0/2353591193 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0e0005afc0 con 0x7f0e0c100dc0 2024-09-16T23:47:44.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.158+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e0c105710 con 0x7f0df406c4e0 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.160+0000 7f0dfb7fe700 1 -- 172.21.15.42:0/2353591193 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0e0c105710 con 0x7f0df406c4e0 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "0/2 daemons upgraded", 2024-09-16T23:47:44.157 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 image on host smithi043", 2024-09-16T23:47:44.158 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:47:44.158 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:47:44.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.162+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0df406c4e0 msgr2=0x7f0df406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:44.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.162+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0df406c4e0 0x7f0df406e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc005950 tx=0x7f0dfc012040 comp rx=0 tx=0).stop 2024-09-16T23:47:44.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 msgr2=0x7f0e0c196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:47:44.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 0x7f0e0c196b00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009870 tx=0x7f0e000047c0 comp rx=0 tx=0).stop 2024-09-16T23:47:44.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 shutdown_connections 2024-09-16T23:47:44.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f0df406c4e0 0x7f0df406e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:44.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e0c0ffbc0 0x7f0e0c1965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:44.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 --2- 172.21.15.42:0/2353591193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e0c100dc0 0x7f0e0c196b00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:47:44.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 >> 172.21.15.42:0/2353591193 conn(0x7f0e0c0fb160 msgr2=0x7f0e0c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:47:44.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 shutdown_connections 2024-09-16T23:47:44.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:47:44.163+0000 7f0e10cd0700 1 -- 172.21.15.42:0/2353591193 wait complete. 2024-09-16T23:47:44.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:44 smithi042 ceph-mon[27914]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:44.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:44 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2034828057' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:47:44.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:44 smithi043 ceph-mon[35191]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:44.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:44 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2034828057' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:47:45.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:45 smithi042 ceph-mon[27914]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:45.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:45 smithi043 ceph-mon[35191]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:47:46.657 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:46 smithi042 ceph-mon[27914]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:46.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:46 smithi043 ceph-mon[35191]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:49.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:48 smithi042 ceph-mon[27914]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:49.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:48 smithi043 ceph-mon[35191]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:49.750 INFO:tasks.workunit.client.1.smithi043.stderr:Updating files: 85% (11065/12893) Updating files: 86% (11088/12893) Updating files: 87% (11217/12893) Updating files: 88% (11346/12893) Updating files: 89% (11475/12893) Updating files: 90% (11604/12893) Updating files: 91% (11733/12893) Updating files: 92% (11862/12893) Updating files: 93% (11991/12893) Updating files: 94% (12120/12893) Updating files: 95% (12249/12893) Updating files: 96% (12378/12893) Updating files: 97% (12507/12893) Updating files: 98% (12636/12893) Updating files: 99% (12765/12893) Updating files: 100% (12893/12893) Updating files: 100% (12893/12893), done. 2024-09-16T23:47:50.717 INFO:tasks.workunit.client.1.smithi043.stderr:Note: switching to 'e20bff7d16f5b09c58ead9548e8cf98cf87ee227'. 2024-09-16T23:47:50.718 INFO:tasks.workunit.client.1.smithi043.stderr: 2024-09-16T23:47:50.718 INFO:tasks.workunit.client.1.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T23:47:50.718 INFO:tasks.workunit.client.1.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T23:47:50.718 INFO:tasks.workunit.client.1.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T23:47:50.718 INFO:tasks.workunit.client.1.smithi043.stderr: 2024-09-16T23:47:50.718 INFO:tasks.workunit.client.1.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T23:47:50.718 INFO:tasks.workunit.client.1.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T23:47:50.719 INFO:tasks.workunit.client.1.smithi043.stderr: 2024-09-16T23:47:50.719 INFO:tasks.workunit.client.1.smithi043.stderr: git switch -c 2024-09-16T23:47:50.719 INFO:tasks.workunit.client.1.smithi043.stderr: 2024-09-16T23:47:50.719 INFO:tasks.workunit.client.1.smithi043.stderr:Or undo this operation with: 2024-09-16T23:47:50.719 INFO:tasks.workunit.client.1.smithi043.stderr: 2024-09-16T23:47:50.719 INFO:tasks.workunit.client.1.smithi043.stderr: git switch - 2024-09-16T23:47:50.720 INFO:tasks.workunit.client.1.smithi043.stderr: 2024-09-16T23:47:50.720 INFO:tasks.workunit.client.1.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T23:47:50.720 INFO:tasks.workunit.client.1.smithi043.stderr: 2024-09-16T23:47:50.720 INFO:tasks.workunit.client.1.smithi043.stderr:HEAD is now at e20bff7d16f Merge pull request #59722 from aaSharma14/wip-67990-squid 2024-09-16T23:47:50.723 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-16T23:47:50.792 INFO:tasks.workunit.client.1.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T23:47:50.795 INFO:tasks.workunit.client.1.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-16T23:47:50.795 INFO:tasks.workunit.client.1.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T23:47:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:50 smithi042 ceph-mon[27914]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:50 smithi043 ceph-mon[35191]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:51.483 INFO:tasks.workunit.client.1.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T23:47:51.571 INFO:tasks.workunit.client.1.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T23:47:51.617 INFO:tasks.workunit.client.1.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-16T23:47:51.619 INFO:tasks.workunit.client.1.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-16T23:47:51.619 INFO:tasks.workunit.client.1.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T23:47:51.664 INFO:tasks.workunit.client.1.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-16T23:47:51.668 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-09-16T23:47:51.668 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-16T23:47:51.726 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-09-16T23:47:51.727 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-16T23:47:51.728 DEBUG:teuthology.orchestra.run.smithi043:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e20bff7d16f5b09c58ead9548e8cf98cf87ee227 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-09-16T23:47:51.797 INFO:tasks.workunit.client.1.smithi043.stderr:+ mkdir -p fsstress 2024-09-16T23:47:51.798 INFO:tasks.workunit.client.1.smithi043.stderr:+ pushd fsstress 2024-09-16T23:47:51.800 INFO:tasks.workunit.client.1.smithi043.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T23:47:51.800 INFO:tasks.workunit.client.1.smithi043.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-16T23:47:52.387 INFO:tasks.workunit.client.1.smithi043.stderr:+ tar xzf ltp-full.tgz 2024-09-16T23:47:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:52 smithi043 ceph-mon[35191]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:53.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:52 smithi042 ceph-mon[27914]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T23:47:54.187 INFO:tasks.workunit.client.0.smithi042.stderr:Updating files: 87% (11296/12893) Updating files: 88% (11346/12893) Updating files: 89% (11475/12893) Updating files: 90% (11604/12893) Updating files: 91% (11733/12893) Updating files: 92% (11862/12893) Updating files: 93% (11991/12893) Updating files: 94% (12120/12893) Updating files: 95% (12249/12893) Updating files: 96% (12378/12893) Updating files: 97% (12507/12893) Updating files: 98% (12636/12893) Updating files: 99% (12765/12893) Updating files: 100% (12893/12893) Updating files: 100% (12893/12893), done. 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr:Note: switching to 'e20bff7d16f5b09c58ead9548e8cf98cf87ee227'. 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T23:47:54.672 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr: git switch -c 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr:Or undo this operation with: 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr: git switch - 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-09-16T23:47:54.673 INFO:tasks.workunit.client.0.smithi042.stderr:HEAD is now at e20bff7d16f Merge pull request #59722 from aaSharma14/wip-67990-squid 2024-09-16T23:47:54.678 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T23:47:54.701 INFO:tasks.workunit.client.0.smithi042.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T23:47:54.704 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T23:47:54.704 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T23:47:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:54 smithi042 ceph-mon[27914]: pgmap v236: 65 pgs: 65 active+clean; 7.0 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 547 KiB/s wr, 1 op/s 2024-09-16T23:47:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:54 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:54 smithi043 ceph-mon[35191]: pgmap v236: 65 pgs: 65 active+clean; 7.0 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 547 KiB/s wr, 1 op/s 2024-09-16T23:47:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:54 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:47:55.425 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T23:47:55.494 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T23:47:55.536 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T23:47:55.538 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T23:47:55.538 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T23:47:55.579 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T23:47:55.582 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-16T23:47:55.582 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T23:47:55.640 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-16T23:47:55.642 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-16T23:47:55.642 DEBUG:teuthology.orchestra.run.smithi042:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e20bff7d16f5b09c58ead9548e8cf98cf87ee227 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-16T23:47:55.706 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir -p fsstress 2024-09-16T23:47:55.708 INFO:tasks.workunit.client.0.smithi042.stderr:+ pushd fsstress 2024-09-16T23:47:55.709 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T23:47:55.709 INFO:tasks.workunit.client.0.smithi042.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-16T23:47:56.591 INFO:tasks.workunit.client.0.smithi042.stderr:+ tar xzf ltp-full.tgz 2024-09-16T23:47:57.011 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:56 smithi043 ceph-mon[35191]: pgmap v237: 65 pgs: 65 active+clean; 7.0 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 547 KiB/s wr, 1 op/s 2024-09-16T23:47:57.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:56 smithi042 ceph-mon[27914]: pgmap v237: 65 pgs: 65 active+clean; 7.0 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 547 KiB/s wr, 1 op/s 2024-09-16T23:47:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:47:58 smithi042 ceph-mon[27914]: pgmap v238: 65 pgs: 65 active+clean; 12 MiB data, 189 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1007 KiB/s wr, 23 op/s 2024-09-16T23:47:59.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:47:58 smithi043 ceph-mon[35191]: pgmap v238: 65 pgs: 65 active+clean; 12 MiB data, 189 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1007 KiB/s wr, 23 op/s 2024-09-16T23:48:01.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:00 smithi042 ceph-mon[27914]: pgmap v239: 65 pgs: 65 active+clean; 20 MiB data, 220 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 54 op/s 2024-09-16T23:48:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:00 smithi043 ceph-mon[35191]: pgmap v239: 65 pgs: 65 active+clean; 20 MiB data, 220 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1.6 MiB/s wr, 54 op/s 2024-09-16T23:48:03.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:02 smithi042 ceph-mon[27914]: pgmap v240: 65 pgs: 65 active+clean; 29 MiB data, 249 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 85 op/s 2024-09-16T23:48:03.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:02 smithi043 ceph-mon[35191]: pgmap v240: 65 pgs: 65 active+clean; 29 MiB data, 249 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.4 MiB/s wr, 85 op/s 2024-09-16T23:48:05.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:04 smithi042 ceph-mon[27914]: pgmap v241: 65 pgs: 65 active+clean; 41 MiB data, 296 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 130 op/s 2024-09-16T23:48:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:04 smithi043 ceph-mon[35191]: pgmap v241: 65 pgs: 65 active+clean; 41 MiB data, 296 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.4 MiB/s wr, 130 op/s 2024-09-16T23:48:06.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:05 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:48:06.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:05 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:48:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:06 smithi042 ceph-mon[27914]: pgmap v242: 65 pgs: 65 active+clean; 41 MiB data, 296 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 128 op/s 2024-09-16T23:48:07.122 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:06 smithi043 ceph-mon[35191]: pgmap v242: 65 pgs: 65 active+clean; 41 MiB data, 296 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 2.8 MiB/s wr, 128 op/s 2024-09-16T23:48:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:08 smithi042 ceph-mon[27914]: pgmap v243: 65 pgs: 65 active+clean; 54 MiB data, 336 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 163 op/s 2024-09-16T23:48:09.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:08 smithi043 ceph-mon[35191]: pgmap v243: 65 pgs: 65 active+clean; 54 MiB data, 336 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 163 op/s 2024-09-16T23:48:09.929 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:09 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:09 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:10 smithi042 ceph-mon[27914]: pgmap v244: 65 pgs: 65 active+clean; 68 MiB data, 373 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 171 op/s 2024-09-16T23:48:11.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:10 smithi043 ceph-mon[35191]: pgmap v244: 65 pgs: 65 active+clean; 68 MiB data, 373 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 171 op/s 2024-09-16T23:48:11.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:11 smithi043 ceph-mon[35191]: Upgrade: Updating mgr.smithi043.xveyou 2024-09-16T23:48:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:11 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:11 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:48:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:11 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:48:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:11 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:48:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:11 smithi043 ceph-mon[35191]: Deploying daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:48:12.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:11 smithi042 ceph-mon[27914]: Upgrade: Updating mgr.smithi043.xveyou 2024-09-16T23:48:12.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:11 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:12.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:11 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:48:12.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:11 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:48:12.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:11 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:48:12.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:11 smithi042 ceph-mon[27914]: Deploying daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:48:13.034 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:12 smithi043 ceph-mon[35191]: pgmap v245: 65 pgs: 65 active+clean; 72 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 166 op/s 2024-09-16T23:48:13.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:12 smithi042 ceph-mon[27914]: pgmap v245: 65 pgs: 65 active+clean; 72 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 166 op/s 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 -- 172.21.15.42:0/2863877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98681040c0 msgr2=0x7f98681064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2863877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98681040c0 0x7f98681064b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f985c009870 tx=0x7f985c009b80 comp rx=0 tx=0).stop 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 -- 172.21.15.42:0/2863877392 shutdown_connections 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2863877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98681040c0 0x7f98681064b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2863877392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9868101790 0x7f9868103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 -- 172.21.15.42:0/2863877392 >> 172.21.15.42:0/2863877392 conn(0x7f98680fb120 msgr2=0x7f98680fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 -- 172.21.15.42:0/2863877392 shutdown_connections 2024-09-16T23:48:14.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.316+0000 7f986cfcc700 1 -- 172.21.15.42:0/2863877392 wait complete. 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.317+0000 7f986cfcc700 1 Processor -- start 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.317+0000 7f986cfcc700 1 -- start start 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986cfcc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9868101790 0x7f9868196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986cfcc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98681040c0 0x7f98681969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986cfcc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868196fd0 con 0x7f98681040c0 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986cfcc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868197110 con 0x7f9868101790 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9868101790 0x7f9868196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9868101790 0x7f9868196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:57614/0 (socket says 172.21.15.42:57614) 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986659c700 1 -- 172.21.15.42:0/2523618943 learned_addr learned my addr 172.21.15.42:0/2523618943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:14.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f9865d9b700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98681040c0 0x7f98681969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986659c700 1 -- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98681040c0 msgr2=0x7f98681969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986659c700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98681040c0 0x7f98681969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.318+0000 7f986659c700 1 -- 172.21.15.42:0/2523618943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98580095f0 con 0x7f9868101790 2024-09-16T23:48:14.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.319+0000 7f986659c700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9868101790 0x7f9868196470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f985800eb90 tx=0x7f985800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.319+0000 7f98577fe700 1 -- 172.21.15.42:0/2523618943 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f985800ccc0 con 0x7f9868101790 2024-09-16T23:48:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.319+0000 7f98577fe700 1 -- 172.21.15.42:0/2523618943 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f985800ce20 con 0x7f9868101790 2024-09-16T23:48:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.319+0000 7f98577fe700 1 -- 172.21.15.42:0/2523618943 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9858010430 con 0x7f9868101790 2024-09-16T23:48:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.319+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c009520 con 0x7f9868101790 2024-09-16T23:48:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.319+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986819bf80 con 0x7f9868101790 2024-09-16T23:48:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.320+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9868190610 con 0x7f9868101790 2024-09-16T23:48:14.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.321+0000 7f98577fe700 1 -- 172.21.15.42:0/2523618943 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f9858004000 con 0x7f9868101790 2024-09-16T23:48:14.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.322+0000 7f98577fe700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f985006c530 0x7f985006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:14.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.322+0000 7f98577fe700 1 -- 172.21.15.42:0/2523618943 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9858014070 con 0x7f9868101790 2024-09-16T23:48:14.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.322+0000 7f9865d9b700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f985006c530 0x7f985006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:14.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.323+0000 7f9865d9b700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f985006c530 0x7f985006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f985c00b6f0 tx=0x7f985c019040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:14.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.326+0000 7f98577fe700 1 -- 172.21.15.42:0/2523618943 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f985805a070 con 0x7f9868101790 2024-09-16T23:48:14.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.662+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f986819baa0 con 0x7f985006c530 2024-09-16T23:48:14.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.663+0000 7f98577fe700 1 -- 172.21.15.42:0/2523618943 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f986819baa0 con 0x7f985006c530 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f985006c530 msgr2=0x7f985006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f985006c530 0x7f985006e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f985c00b6f0 tx=0x7f985c019040 comp rx=0 tx=0).stop 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9868101790 msgr2=0x7f9868196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9868101790 0x7f9868196470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f985800eb90 tx=0x7f985800eea0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 shutdown_connections 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f985006c530 0x7f985006e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9868101790 0x7f9868196470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 --2- 172.21.15.42:0/2523618943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98681040c0 0x7f98681969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.666+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 >> 172.21.15.42:0/2523618943 conn(0x7f98680fb120 msgr2=0x7f98680fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:14.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.667+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 shutdown_connections 2024-09-16T23:48:14.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.667+0000 7f986cfcc700 1 -- 172.21.15.42:0/2523618943 wait complete. 2024-09-16T23:48:14.675 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:48:14.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.786+0000 7fb081a4f700 1 -- 172.21.15.42:0/696933213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 msgr2=0x7fb07c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:14.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.786+0000 7fb081a4f700 1 --2- 172.21.15.42:0/696933213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c100030 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009910 tx=0x7fb06c009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:14.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.786+0000 7fb081a4f700 1 -- 172.21.15.42:0/696933213 shutdown_connections 2024-09-16T23:48:14.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.786+0000 7fb081a4f700 1 --2- 172.21.15.42:0/696933213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb07c100e10 0x7fb07c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.786+0000 7fb081a4f700 1 --2- 172.21.15.42:0/696933213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c100030 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.786+0000 7fb081a4f700 1 -- 172.21.15.42:0/696933213 >> 172.21.15.42:0/696933213 conn(0x7fb07c0fb190 msgr2=0x7fb07c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:14.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.786+0000 7fb081a4f700 1 -- 172.21.15.42:0/696933213 shutdown_connections 2024-09-16T23:48:14.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.787+0000 7fb081a4f700 1 -- 172.21.15.42:0/696933213 wait complete. 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.787+0000 7fb081a4f700 1 Processor -- start 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb081a4f700 1 -- start start 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb081a4f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb081a4f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb07c100e10 0x7fb07c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb081a4f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c19b7e0 con 0x7fb07c0ffc10 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb081a4f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c19b950 con 0x7fb07c100e10 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb07affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb07affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c069100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37818/0 (socket says 172.21.15.42:37818) 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb07affd700 1 -- 172.21.15.42:0/2532437508 learned_addr learned my addr 172.21.15.42:0/2532437508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:14.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb07a7fc700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb07c100e10 0x7fb07c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:14.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb07affd700 1 -- 172.21.15.42:0/2532437508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb07c100e10 msgr2=0x7fb07c19b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:14.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb07affd700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb07c100e10 0x7fb07c19b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:14.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.788+0000 7fb07affd700 1 -- 172.21.15.42:0/2532437508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c0095f0 con 0x7fb07c0ffc10 2024-09-16T23:48:14.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.789+0000 7fb07affd700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c069100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00bda0 tx=0x7fb06c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:14.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.789+0000 7fb080a4d700 1 -- 172.21.15.42:0/2532437508 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06c01d070 con 0x7fb07c0ffc10 2024-09-16T23:48:14.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.789+0000 7fb080a4d700 1 -- 172.21.15.42:0/2532437508 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb06c00f460 con 0x7fb07c0ffc10 2024-09-16T23:48:14.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.789+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c19bbd0 con 0x7fb07c0ffc10 2024-09-16T23:48:14.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.789+0000 7fb080a4d700 1 -- 172.21.15.42:0/2532437508 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06c021620 con 0x7fb07c0ffc10 2024-09-16T23:48:14.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.789+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c19c0c0 con 0x7fb07c0ffc10 2024-09-16T23:48:14.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.790+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c108700 con 0x7fb07c0ffc10 2024-09-16T23:48:14.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.791+0000 7fb080a4d700 1 -- 172.21.15.42:0/2532437508 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fb06c00f5d0 con 0x7fb07c0ffc10 2024-09-16T23:48:14.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.792+0000 7fb080a4d700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb06406c4e0 0x7fb06406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:14.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.792+0000 7fb080a4d700 1 -- 172.21.15.42:0/2532437508 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fb06c08ca00 con 0x7fb07c0ffc10 2024-09-16T23:48:14.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.792+0000 7fb07a7fc700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb06406c4e0 0x7fb06406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:14.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.793+0000 7fb07a7fc700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb06406c4e0 0x7fb06406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb07c069660 tx=0x7fb07000b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:14.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:14.798+0000 7fb080a4d700 1 -- 172.21.15.42:0/2532437508 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb06c05acc0 con 0x7fb07c0ffc10 2024-09-16T23:48:15.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:14 smithi042 ceph-mon[27914]: pgmap v246: 65 pgs: 65 active+clean; 88 MiB data, 449 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 171 op/s 2024-09-16T23:48:15.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.082+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb07c105760 con 0x7fb06406c4e0 2024-09-16T23:48:15.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.083+0000 7fb080a4d700 1 -- 172.21.15.42:0/2532437508 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fb07c105760 con 0x7fb06406c4e0 2024-09-16T23:48:15.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb06406c4e0 msgr2=0x7fb06406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb06406c4e0 0x7fb06406e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb07c069660 tx=0x7fb07000b410 comp rx=0 tx=0).stop 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 msgr2=0x7fb07c069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c069100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00bda0 tx=0x7fb06c00bdd0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 shutdown_connections 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fb06406c4e0 0x7fb06406e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb07c0ffc10 0x7fb07c069100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 --2- 172.21.15.42:0/2532437508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb07c100e10 0x7fb07c19b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 >> 172.21.15.42:0/2532437508 conn(0x7fb07c0fb190 msgr2=0x7fb07c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 shutdown_connections 2024-09-16T23:48:15.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.086+0000 7fb081a4f700 1 -- 172.21.15.42:0/2532437508 wait complete. 2024-09-16T23:48:15.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:14 smithi043 ceph-mon[35191]: pgmap v246: 65 pgs: 65 active+clean; 88 MiB data, 449 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 171 op/s 2024-09-16T23:48:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.218+0000 7f936d343700 1 -- 172.21.15.42:0/17437907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93680ffc10 msgr2=0x7f9368100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.218+0000 7f936d343700 1 --2- 172.21.15.42:0/17437907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93680ffc10 0x7f9368100030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9358009910 tx=0x7f9358009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:15.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.219+0000 7f936d343700 1 -- 172.21.15.42:0/17437907 shutdown_connections 2024-09-16T23:48:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.219+0000 7f936d343700 1 --2- 172.21.15.42:0/17437907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9368100e10 0x7f9368101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.219+0000 7f936d343700 1 --2- 172.21.15.42:0/17437907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93680ffc10 0x7f9368100030 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.219+0000 7f936d343700 1 -- 172.21.15.42:0/17437907 >> 172.21.15.42:0/17437907 conn(0x7f93680fb190 msgr2=0x7f93680fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.219+0000 7f936d343700 1 -- 172.21.15.42:0/17437907 shutdown_connections 2024-09-16T23:48:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.219+0000 7f936d343700 1 -- 172.21.15.42:0/17437907 wait complete. 2024-09-16T23:48:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f936d343700 1 Processor -- start 2024-09-16T23:48:15.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f936d343700 1 -- start start 2024-09-16T23:48:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f936d343700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93680ffc10 0x7f9368069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f936d343700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9368100e10 0x7f936819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f936d343700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9368197080 con 0x7f9368100e10 2024-09-16T23:48:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f936d343700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936819db50 con 0x7f93680ffc10 2024-09-16T23:48:15.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f93667fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9368100e10 0x7f936819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f9366ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93680ffc10 0x7f9368069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f93667fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9368100e10 0x7f936819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37836/0 (socket says 172.21.15.42:37836) 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.220+0000 7f93667fc700 1 -- 172.21.15.42:0/3921157785 learned_addr learned my addr 172.21.15.42:0/3921157785 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f9366ffd700 1 -- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9368100e10 msgr2=0x7f936819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f9366ffd700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9368100e10 0x7f936819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f9366ffd700 1 -- 172.21.15.42:0/3921157785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580095f0 con 0x7f93680ffc10 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f9366ffd700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93680ffc10 0x7f9368069100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9358000c00 tx=0x7f935800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f934ffff700 1 -- 172.21.15.42:0/3921157785 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935801d070 con 0x7f93680ffc10 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f934ffff700 1 -- 172.21.15.42:0/3921157785 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9358005dc0 con 0x7f93680ffc10 2024-09-16T23:48:15.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936819ddd0 con 0x7f93680ffc10 2024-09-16T23:48:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.221+0000 7f934ffff700 1 -- 172.21.15.42:0/3921157785 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9358017400 con 0x7f93680ffc10 2024-09-16T23:48:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.222+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936819e260 con 0x7f93680ffc10 2024-09-16T23:48:15.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.223+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9368108700 con 0x7f93680ffc10 2024-09-16T23:48:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.224+0000 7f934ffff700 1 -- 172.21.15.42:0/3921157785 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f935800fa40 con 0x7f93680ffc10 2024-09-16T23:48:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.224+0000 7f934ffff700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f935006c4e0 0x7f935006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.224+0000 7f934ffff700 1 -- 172.21.15.42:0/3921157785 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f935808c600 con 0x7f93680ffc10 2024-09-16T23:48:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.224+0000 7f93667fc700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f935006c4e0 0x7f935006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:15.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.225+0000 7f93667fc700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f935006c4e0 0x7f935006e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9368069660 tx=0x7f935c009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:15.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.229+0000 7f934ffff700 1 -- 172.21.15.42:0/3921157785 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f935805a8c0 con 0x7f93680ffc10 2024-09-16T23:48:15.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.515+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9368105760 con 0x7f935006c4e0 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.529+0000 7f934ffff700 1 -- 172.21.15.42:0/3921157785 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f9368105760 con 0x7f935006c4e0 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (6m) 95s ago 7m 25.5M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (7m) 95s ago 7m 7453k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (6m) 101s ago 6m 7386k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (7m) 95s ago 7m 6660k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (6m) 101s ago 6m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:48:15.527 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (5m) 95s ago 7m 83.1M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (117s) 95s ago 116s 15.9M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (2m) 95s ago 2m 11.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (2m) 101s ago 2m 16.3M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (114s) 101s ago 113s 13.0M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:9283,8765,8443 running (9m) 95s ago 9m 488M - 18.2.1 5be31c24972a cd8abd4f468e 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (6m) 101s ago 6m 442M - 18.2.1 5be31c24972a b8099e88d36b 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (9m) 95s ago 9m 45.4M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (6m) 101s ago 6m 38.5M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (7m) 95s ago 7m 16.2M - 1.5.0 0da6a335fe13 7c84e9bf6c1c 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (6m) 101s ago 6m 16.4M - 1.5.0 0da6a335fe13 82d41c3cc1ee 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (5m) 95s ago 5m 48.3M 3948M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (5m) 95s ago 5m 47.2M 3948M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:48:15.528 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (4m) 95s ago 4m 46.4M 3948M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:48:15.529 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (4m) 101s ago 4m 47.2M 4972M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:48:15.529 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (3m) 101s ago 3m 43.9M 4972M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:48:15.529 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (3m) 101s ago 3m 44.5M 4972M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:48:15.529 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (5m) 95s ago 7m 56.4M - 2.43.0 a07b618ecd1d 8ebaffdf2443 2024-09-16T23:48:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f935006c4e0 msgr2=0x7f935006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f935006c4e0 0x7f935006e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9368069660 tx=0x7f935c009260 comp rx=0 tx=0).stop 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93680ffc10 msgr2=0x7f9368069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93680ffc10 0x7f9368069100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9358000c00 tx=0x7f935800f690 comp rx=0 tx=0).stop 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 shutdown_connections 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f935006c4e0 0x7f935006e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93680ffc10 0x7f9368069100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 --2- 172.21.15.42:0/3921157785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9368100e10 0x7f936819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 >> 172.21.15.42:0/3921157785 conn(0x7f93680fb190 msgr2=0x7f9368104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 shutdown_connections 2024-09-16T23:48:15.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.532+0000 7f936d343700 1 -- 172.21.15.42:0/3921157785 wait complete. 2024-09-16T23:48:15.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 -- 172.21.15.42:0/4005166232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 msgr2=0x7fc2e0100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/4005166232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e0100030 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009910 tx=0x7fc2d0009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:15.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 -- 172.21.15.42:0/4005166232 shutdown_connections 2024-09-16T23:48:15.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/4005166232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e0100e10 0x7fc2e0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/4005166232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e0100030 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 -- 172.21.15.42:0/4005166232 >> 172.21.15.42:0/4005166232 conn(0x7fc2e00fb190 msgr2=0x7fc2e00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:15.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 -- 172.21.15.42:0/4005166232 shutdown_connections 2024-09-16T23:48:15.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.652+0000 7fc2e49d4700 1 -- 172.21.15.42:0/4005166232 wait complete. 2024-09-16T23:48:15.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.653+0000 7fc2e49d4700 1 Processor -- start 2024-09-16T23:48:15.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.653+0000 7fc2e49d4700 1 -- start start 2024-09-16T23:48:15.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.653+0000 7fc2e49d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e00690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:15.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.653+0000 7fc2e49d4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e0100e10 0x7fc2e019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:15.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.653+0000 7fc2e49d4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e019b800 con 0x7fc2e00ffc10 2024-09-16T23:48:15.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.653+0000 7fc2e49d4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e019b940 con 0x7fc2e0100e10 2024-09-16T23:48:15.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2de59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e00690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2de59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e00690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37860/0 (socket says 172.21.15.42:37860) 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2ddd9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e0100e10 0x7fc2e019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2de59c700 1 -- 172.21.15.42:0/279569353 learned_addr learned my addr 172.21.15.42:0/279569353 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2de59c700 1 -- 172.21.15.42:0/279569353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e0100e10 msgr2=0x7fc2e019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2de59c700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e0100e10 0x7fc2e019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2de59c700 1 -- 172.21.15.42:0/279569353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00095f0 con 0x7fc2e00ffc10 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2de59c700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e00690e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2cf7fe700 1 -- 172.21.15.42:0/279569353 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2d001d070 con 0x7fc2e00ffc10 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2cf7fe700 1 -- 172.21.15.42:0/279569353 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc2d0005dc0 con 0x7fc2e00ffc10 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e019bbc0 con 0x7fc2e00ffc10 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.654+0000 7fc2cf7fe700 1 -- 172.21.15.42:0/279569353 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2d0022470 con 0x7fc2e00ffc10 2024-09-16T23:48:15.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.655+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e019c0b0 con 0x7fc2e00ffc10 2024-09-16T23:48:15.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.656+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e0108700 con 0x7fc2e00ffc10 2024-09-16T23:48:15.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.656+0000 7fc2cf7fe700 1 -- 172.21.15.42:0/279569353 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7fc2d000fa70 con 0x7fc2e00ffc10 2024-09-16T23:48:15.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.656+0000 7fc2cf7fe700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc2c806c4e0 0x7fc2c806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:15.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.657+0000 7fc2cf7fe700 1 -- 172.21.15.42:0/279569353 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc2d008cca0 con 0x7fc2e00ffc10 2024-09-16T23:48:15.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.657+0000 7fc2ddd9b700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc2c806c4e0 0x7fc2c806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:15.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.657+0000 7fc2ddd9b700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc2c806c4e0 0x7fc2c806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0069660 tx=0x7fc2d4009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:15.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.662+0000 7fc2cf7fe700 1 -- 172.21.15.42:0/279569353 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc2d0091070 con 0x7fc2e00ffc10 2024-09-16T23:48:15.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.993+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc2e019c660 con 0x7fc2e00ffc10 2024-09-16T23:48:15.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.993+0000 7fc2cf7fe700 1 -- 172.21.15.42:0/279569353 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fc2d005af60 con 0x7fc2e00ffc10 2024-09-16T23:48:15.991 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:48:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:48:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:48:15.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:48:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:48:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:48:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:48:15.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:48:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:48:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:48:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-09-16T23:48:15.993 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:48:15.994 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:48:15.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc2c806c4e0 msgr2=0x7fc2c806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc2c806c4e0 0x7fc2c806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0069660 tx=0x7fc2d4009260 comp rx=0 tx=0).stop 2024-09-16T23:48:15.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 msgr2=0x7fc2e00690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:15.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e00690e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d000f690 comp rx=0 tx=0).stop 2024-09-16T23:48:15.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 shutdown_connections 2024-09-16T23:48:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fc2c806c4e0 0x7fc2c806e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2e00ffc10 0x7fc2e00690e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 --2- 172.21.15.42:0/279569353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2e0100e10 0x7fc2e019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 >> 172.21.15.42:0/279569353 conn(0x7fc2e00fb190 msgr2=0x7fc2e0104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 shutdown_connections 2024-09-16T23:48:15.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:15.996+0000 7fc2e49d4700 1 -- 172.21.15.42:0/279569353 wait complete. 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 -- 172.21.15.42:0/4175789496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 msgr2=0x7f7860103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 --2- 172.21.15.42:0/4175789496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860103b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7850009910 tx=0x7f7850009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 -- 172.21.15.42:0/4175789496 shutdown_connections 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 --2- 172.21.15.42:0/4175789496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78601040d0 0x7f78601064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 --2- 172.21.15.42:0/4175789496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860103b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 -- 172.21.15.42:0/4175789496 >> 172.21.15.42:0/4175789496 conn(0x7f78600fb190 msgr2=0x7f78600fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 -- 172.21.15.42:0/4175789496 shutdown_connections 2024-09-16T23:48:16.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.130+0000 7f78659d8700 1 -- 172.21.15.42:0/4175789496 wait complete. 2024-09-16T23:48:16.129 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:15 smithi042 ceph-mon[27914]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:16.129 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:15 smithi042 ceph-mon[27914]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f78659d8700 1 Processor -- start 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f78659d8700 1 -- start start 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f78659d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f78659d8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78601040d0 0x7f78600ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f78659d8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600ffc90 con 0x7f78601017a0 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f785effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f78659d8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78600ffdd0 con 0x7f78601040d0 2024-09-16T23:48:16.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f785effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37890/0 (socket says 172.21.15.42:37890) 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f785effd700 1 -- 172.21.15.42:0/4220322709 learned_addr learned my addr 172.21.15.42:0/4220322709 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.131+0000 7f785e7fc700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78601040d0 0x7f78600ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f785effd700 1 -- 172.21.15.42:0/4220322709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78601040d0 msgr2=0x7f78600ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f785effd700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78601040d0 0x7f78600ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f785effd700 1 -- 172.21.15.42:0/4220322709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78500095f0 con 0x7f78601017a0 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f785effd700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860101100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7850006010 tx=0x7f785000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f78649d6700 1 -- 172.21.15.42:0/4220322709 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f785001d070 con 0x7f78601017a0 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f78649d6700 1 -- 172.21.15.42:0/4220322709 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f785000f460 con 0x7f78601017a0 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f78649d6700 1 -- 172.21.15.42:0/4220322709 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7850021620 con 0x7f78601017a0 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7860100050 con 0x7f78601017a0 2024-09-16T23:48:16.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.132+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78601004f0 con 0x7f78601017a0 2024-09-16T23:48:16.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:15 smithi043 ceph-mon[35191]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:16.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:15 smithi043 ceph-mon[35191]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:16.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.133+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7860190610 con 0x7f78601017a0 2024-09-16T23:48:16.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.134+0000 7f78649d6700 1 -- 172.21.15.42:0/4220322709 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 19) v1 ==== 90312+0+0 (secure 0 0 0) 0x7f785000fa50 con 0x7f78601017a0 2024-09-16T23:48:16.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.135+0000 7f78649d6700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f784806c4e0 0x7f784806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:16.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.135+0000 7f78649d6700 1 -- 172.21.15.42:0/4220322709 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f785008cee0 con 0x7f78601017a0 2024-09-16T23:48:16.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.135+0000 7f785e7fc700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f784806c4e0 0x7f784806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:16.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.136+0000 7f785e7fc700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f784806c4e0 0x7f784806e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7854005ea0 tx=0x7f7854005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:16.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.140+0000 7f78649d6700 1 -- 172.21.15.42:0/4220322709 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f785005b2c0 con 0x7f78601017a0 2024-09-16T23:48:16.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.422+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78600611d0 con 0x7f784806c4e0 2024-09-16T23:48:16.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.423+0000 7f78649d6700 1 -- 172.21.15.42:0/4220322709 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f78600611d0 con 0x7f784806c4e0 2024-09-16T23:48:16.420 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:48:16.420 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:48:16.420 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:48:16.421 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:48:16.421 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-09-16T23:48:16.421 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "0/2 daemons upgraded", 2024-09-16T23:48:16.421 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T23:48:16.421 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:48:16.421 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:48:16.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.425+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f784806c4e0 msgr2=0x7f784806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f784806c4e0 0x7f784806e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7854005ea0 tx=0x7f7854005e30 comp rx=0 tx=0).stop 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 msgr2=0x7f7860101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860101100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7850006010 tx=0x7f785000be20 comp rx=0 tx=0).stop 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 shutdown_connections 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f784806c4e0 0x7f784806e9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78601017a0 0x7f7860101100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 --2- 172.21.15.42:0/4220322709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78601040d0 0x7f78600ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 >> 172.21.15.42:0/4220322709 conn(0x7f78600fb190 msgr2=0x7f78600fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 shutdown_connections 2024-09-16T23:48:16.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:16.426+0000 7f78659d8700 1 -- 172.21.15.42:0/4220322709 wait complete. 2024-09-16T23:48:17.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:16 smithi043 ceph-mon[35191]: pgmap v247: 65 pgs: 65 active+clean; 88 MiB data, 449 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 126 op/s 2024-09-16T23:48:17.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:16 smithi043 ceph-mon[35191]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:16 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/279569353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:48:17.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:16 smithi042 ceph-mon[27914]: pgmap v247: 65 pgs: 65 active+clean; 88 MiB data, 449 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 126 op/s 2024-09-16T23:48:17.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:16 smithi042 ceph-mon[27914]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:17.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:16 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/279569353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:48:18.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:17 smithi043 ceph-mon[35191]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:18.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:17 smithi042 ceph-mon[27914]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:18 smithi043 ceph-mon[35191]: pgmap v248: 65 pgs: 65 active+clean; 99 MiB data, 485 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 182 op/s 2024-09-16T23:48:19.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:18 smithi042 ceph-mon[27914]: pgmap v248: 65 pgs: 65 active+clean; 99 MiB data, 485 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 182 op/s 2024-09-16T23:48:21.004 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:20 smithi043 ceph-mon[35191]: pgmap v249: 65 pgs: 65 active+clean; 107 MiB data, 530 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 186 op/s 2024-09-16T23:48:21.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:20 smithi042 ceph-mon[27914]: pgmap v249: 65 pgs: 65 active+clean; 107 MiB data, 530 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 186 op/s 2024-09-16T23:48:23.117 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:22 smithi043 ceph-mon[35191]: pgmap v250: 65 pgs: 65 active+clean; 120 MiB data, 561 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 194 op/s 2024-09-16T23:48:23.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:22 smithi042 ceph-mon[27914]: pgmap v250: 65 pgs: 65 active+clean; 120 MiB data, 561 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.4 MiB/s wr, 194 op/s 2024-09-16T23:48:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:24 smithi043 ceph-mon[35191]: pgmap v251: 65 pgs: 65 active+clean; 129 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.9 MiB/s wr, 231 op/s 2024-09-16T23:48:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:48:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:24 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:24.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:24 smithi042 ceph-mon[27914]: pgmap v251: 65 pgs: 65 active+clean; 129 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.9 MiB/s wr, 231 op/s 2024-09-16T23:48:24.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:24.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:24.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:48:24.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:24 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:26.567 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:26 smithi043 ceph-mon[35191]: pgmap v252: 65 pgs: 65 active+clean; 129 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 194 op/s 2024-09-16T23:48:26.656 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:26 smithi042 ceph-mon[27914]: pgmap v252: 65 pgs: 65 active+clean; 129 MiB data, 614 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 194 op/s 2024-09-16T23:48:28.751 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:28 smithi043 ceph-mon[35191]: pgmap v253: 65 pgs: 65 active+clean; 135 MiB data, 636 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 251 op/s 2024-09-16T23:48:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:28 smithi042 ceph-mon[27914]: pgmap v253: 65 pgs: 65 active+clean; 135 MiB data, 636 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 251 op/s 2024-09-16T23:48:30.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:30 smithi043 ceph-mon[35191]: pgmap v254: 65 pgs: 65 active+clean; 149 MiB data, 684 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.3 MiB/s wr, 249 op/s 2024-09-16T23:48:30.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:30 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou restarted 2024-09-16T23:48:30.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:30 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:48:30.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:30 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:48:30.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:30 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:48:30.874 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:30 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:48:30.874 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:30 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:48:31.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:30 smithi042 ceph-mon[27914]: pgmap v254: 65 pgs: 65 active+clean; 149 MiB data, 684 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.3 MiB/s wr, 249 op/s 2024-09-16T23:48:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:30 smithi042 ceph-mon[27914]: Standby manager daemon smithi043.xveyou restarted 2024-09-16T23:48:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:30 smithi042 ceph-mon[27914]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:48:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:30 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:48:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:30 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:48:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:30 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:48:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:30 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:48:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:31 smithi042 ceph-mon[27914]: mgrmap e20: smithi042.zrkumz(active, since 7m), standbys: smithi043.xveyou 2024-09-16T23:48:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:31 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:32.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:31 smithi043 ceph-mon[35191]: mgrmap e20: smithi042.zrkumz(active, since 7m), standbys: smithi043.xveyou 2024-09-16T23:48:32.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:32.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:31 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:33.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:32 smithi042 ceph-mon[27914]: pgmap v255: 65 pgs: 65 active+clean; 162 MiB data, 715 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 289 op/s 2024-09-16T23:48:33.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:33.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:32 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:33.061 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:32 smithi043 ceph-mon[35191]: pgmap v255: 65 pgs: 65 active+clean; 162 MiB data, 715 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.6 MiB/s wr, 289 op/s 2024-09-16T23:48:33.061 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:33.061 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:32 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:34.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:34 smithi043 ceph-mon[35191]: pgmap v256: 65 pgs: 65 active+clean; 178 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 354 op/s 2024-09-16T23:48:35.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:34 smithi042 ceph-mon[27914]: pgmap v256: 65 pgs: 65 active+clean; 178 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 5.0 MiB/s wr, 354 op/s 2024-09-16T23:48:36.052 INFO:tasks.workunit.client.1.smithi043.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-16T23:48:36.058 INFO:tasks.workunit.client.1.smithi043.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T23:48:36.058 INFO:tasks.workunit.client.1.smithi043.stderr:+ make 2024-09-16T23:48:36.320 INFO:tasks.workunit.client.1.smithi043.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-16T23:48:36.856 INFO:tasks.workunit.client.1.smithi043.stderr:++ readlink -f fsstress 2024-09-16T23:48:36.860 INFO:tasks.workunit.client.1.smithi043.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-16T23:48:36.860 INFO:tasks.workunit.client.1.smithi043.stderr:+ popd 2024-09-16T23:48:36.862 INFO:tasks.workunit.client.1.smithi043.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T23:48:36.862 INFO:tasks.workunit.client.1.smithi043.stderr:+ popd 2024-09-16T23:48:36.863 INFO:tasks.workunit.client.1.smithi043.stdout:~/cephtest/mnt.1/client.1/tmp 2024-09-16T23:48:36.863 INFO:tasks.workunit.client.1.smithi043.stderr:++ mktemp -d -p . 2024-09-16T23:48:36.867 INFO:tasks.workunit.client.1.smithi043.stderr:+ T=./tmp.FE3mQPppf0 2024-09-16T23:48:36.867 INFO:tasks.workunit.client.1.smithi043.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.FE3mQPppf0 -l 1 -n 1000 -p 10 -v 2024-09-16T23:48:36.870 INFO:tasks.workunit.client.1.smithi043.stdout:seed = 1726323446 2024-09-16T23:48:36.874 INFO:tasks.workunit.client.1.smithi043.stdout:0/0: stat - no entries 2024-09-16T23:48:36.876 INFO:tasks.workunit.client.1.smithi043.stdout:0/1: mkdir d0 0 2024-09-16T23:48:36.877 INFO:tasks.workunit.client.1.smithi043.stdout:2/0: write - no filename 2024-09-16T23:48:36.877 INFO:tasks.workunit.client.1.smithi043.stdout:2/1: fdatasync - no filename 2024-09-16T23:48:36.877 INFO:tasks.workunit.client.1.smithi043.stdout:1/0: creat f0 x:0 0 0 2024-09-16T23:48:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:36 smithi043 ceph-mon[35191]: pgmap v257: 65 pgs: 65 active+clean; 178 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-16T23:48:36.878 INFO:tasks.workunit.client.1.smithi043.stdout:0/2: mkdir d0/d1 0 2024-09-16T23:48:36.878 INFO:tasks.workunit.client.1.smithi043.stdout:1/1: fdatasync f0 0 2024-09-16T23:48:36.879 INFO:tasks.workunit.client.1.smithi043.stdout:3/0: rename - no filename 2024-09-16T23:48:36.880 INFO:tasks.workunit.client.1.smithi043.stdout:2/2: symlink l0 0 2024-09-16T23:48:36.880 INFO:tasks.workunit.client.1.smithi043.stdout:2/3: dread - no filename 2024-09-16T23:48:36.881 INFO:tasks.workunit.client.1.smithi043.stdout:1/2: rename f0 to f1 0 2024-09-16T23:48:36.882 INFO:tasks.workunit.client.1.smithi043.stdout:1/3: chown f1 2848183 1 2024-09-16T23:48:36.883 INFO:tasks.workunit.client.1.smithi043.stdout:1/4: dread - f1 zero size 2024-09-16T23:48:36.884 INFO:tasks.workunit.client.1.smithi043.stdout:0/3: rmdir d0/d1 0 2024-09-16T23:48:36.884 INFO:tasks.workunit.client.1.smithi043.stdout:0/4: read - no filename 2024-09-16T23:48:36.884 INFO:tasks.workunit.client.1.smithi043.stdout:0/5: fdatasync - no filename 2024-09-16T23:48:36.885 INFO:tasks.workunit.client.1.smithi043.stdout:0/6: chown d0 3606039 1 2024-09-16T23:48:36.885 INFO:tasks.workunit.client.1.smithi043.stdout:0/7: fsync - no filename 2024-09-16T23:48:36.885 INFO:tasks.workunit.client.1.smithi043.stdout:1/5: read - f1 zero size 2024-09-16T23:48:36.885 INFO:tasks.workunit.client.1.smithi043.stdout:3/1: mknod c0 0 2024-09-16T23:48:36.885 INFO:tasks.workunit.client.1.smithi043.stdout:3/2: dread - no filename 2024-09-16T23:48:36.886 INFO:tasks.workunit.client.1.smithi043.stdout:4/0: creat f0 x:0 0 0 2024-09-16T23:48:36.886 INFO:tasks.workunit.client.1.smithi043.stdout:4/1: rmdir - no directory 2024-09-16T23:48:36.888 INFO:tasks.workunit.client.1.smithi043.stdout:0/8: mkdir d0/d2 0 2024-09-16T23:48:36.888 INFO:tasks.workunit.client.1.smithi043.stdout:0/9: fdatasync - no filename 2024-09-16T23:48:36.888 INFO:tasks.workunit.client.1.smithi043.stdout:0/10: write - no filename 2024-09-16T23:48:36.888 INFO:tasks.workunit.client.1.smithi043.stdout:0/11: chown d0 5929 1 2024-09-16T23:48:36.888 INFO:tasks.workunit.client.1.smithi043.stdout:0/12: dread - no filename 2024-09-16T23:48:36.888 INFO:tasks.workunit.client.1.smithi043.stdout:1/6: symlink l2 0 2024-09-16T23:48:36.889 INFO:tasks.workunit.client.1.smithi043.stdout:0/13: stat d0 0 2024-09-16T23:48:36.890 INFO:tasks.workunit.client.1.smithi043.stdout:0/14: stat d0 0 2024-09-16T23:48:36.890 INFO:tasks.workunit.client.1.smithi043.stdout:3/3: creat f1 x:0 0 0 2024-09-16T23:48:36.891 INFO:tasks.workunit.client.1.smithi043.stdout:4/2: creat f1 x:0 0 0 2024-09-16T23:48:36.892 INFO:tasks.workunit.client.1.smithi043.stdout:7/0: chown . 2 1 2024-09-16T23:48:36.892 INFO:tasks.workunit.client.1.smithi043.stdout:7/1: dread - no filename 2024-09-16T23:48:36.892 INFO:tasks.workunit.client.1.smithi043.stdout:7/2: unlink - no file 2024-09-16T23:48:36.892 INFO:tasks.workunit.client.1.smithi043.stdout:7/3: link - no file 2024-09-16T23:48:36.892 INFO:tasks.workunit.client.1.smithi043.stdout:3/4: mknod c2 0 2024-09-16T23:48:36.893 INFO:tasks.workunit.client.1.smithi043.stdout:3/5: readlink - no filename 2024-09-16T23:48:36.893 INFO:tasks.workunit.client.1.smithi043.stdout:1/7: write f1 [277572,32894] 0 2024-09-16T23:48:36.893 INFO:tasks.workunit.client.1.smithi043.stdout:6/0: mknod c0 0 2024-09-16T23:48:36.895 INFO:tasks.workunit.client.1.smithi043.stdout:1/8: chown l2 124 1 2024-09-16T23:48:36.895 INFO:tasks.workunit.client.1.smithi043.stdout:8/0: fdatasync - no filename 2024-09-16T23:48:36.895 INFO:tasks.workunit.client.1.smithi043.stdout:8/1: stat - no entries 2024-09-16T23:48:36.895 INFO:tasks.workunit.client.1.smithi043.stdout:3/6: mknod c3 0 2024-09-16T23:48:36.895 INFO:tasks.workunit.client.1.smithi043.stdout:4/3: write f0 [206512,129017] 0 2024-09-16T23:48:36.896 INFO:tasks.workunit.client.1.smithi043.stdout:6/1: creat f1 x:0 0 0 2024-09-16T23:48:36.897 INFO:tasks.workunit.client.1.smithi043.stdout:0/15: mknod d0/d2/c3 0 2024-09-16T23:48:36.898 INFO:tasks.workunit.client.1.smithi043.stdout:4/4: dread - f1 zero size 2024-09-16T23:48:36.898 INFO:tasks.workunit.client.1.smithi043.stdout:3/7: write f1 [441645,88614] 0 2024-09-16T23:48:36.899 INFO:tasks.workunit.client.1.smithi043.stdout:4/5: stat f0 0 2024-09-16T23:48:36.900 INFO:tasks.workunit.client.1.smithi043.stdout:1/9: rename l2 to l3 0 2024-09-16T23:48:36.900 INFO:tasks.workunit.client.1.smithi043.stdout:9/0: write - no filename 2024-09-16T23:48:36.900 INFO:tasks.workunit.client.1.smithi043.stdout:9/1: write - no filename 2024-09-16T23:48:36.900 INFO:tasks.workunit.client.1.smithi043.stdout:9/2: write - no filename 2024-09-16T23:48:36.900 INFO:tasks.workunit.client.1.smithi043.stdout:9/3: rename - no filename 2024-09-16T23:48:36.900 INFO:tasks.workunit.client.1.smithi043.stdout:9/4: dread - no filename 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:9/5: chown . 18907530 1 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:9/6: unlink - no file 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:9/7: rename - no filename 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:9/8: write - no filename 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:9/9: readlink - no filename 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:7/4: symlink l0 0 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:7/5: truncate - no filename 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:1/10: chown f1 755174873 1 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:7/6: readlink l0 0 2024-09-16T23:48:36.901 INFO:tasks.workunit.client.1.smithi043.stdout:7/7: dread - no filename 2024-09-16T23:48:36.902 INFO:tasks.workunit.client.1.smithi043.stdout:7/8: dread - no filename 2024-09-16T23:48:36.902 INFO:tasks.workunit.client.1.smithi043.stdout:3/8: fdatasync f1 0 2024-09-16T23:48:36.902 INFO:tasks.workunit.client.1.smithi043.stdout:7/9: readlink l0 0 2024-09-16T23:48:36.903 INFO:tasks.workunit.client.1.smithi043.stdout:1/11: truncate f1 771413 0 2024-09-16T23:48:36.903 INFO:tasks.workunit.client.1.smithi043.stdout:6/2: unlink c0 0 2024-09-16T23:48:36.903 INFO:tasks.workunit.client.1.smithi043.stdout:6/3: chown f1 12247908 1 2024-09-16T23:48:36.903 INFO:tasks.workunit.client.1.smithi043.stdout:3/9: chown c0 1675 1 2024-09-16T23:48:36.903 INFO:tasks.workunit.client.1.smithi043.stdout:0/16: symlink d0/l4 0 2024-09-16T23:48:36.903 INFO:tasks.workunit.client.1.smithi043.stdout:0/17: dwrite - no filename 2024-09-16T23:48:36.903 INFO:tasks.workunit.client.1.smithi043.stdout:0/18: fdatasync - no filename 2024-09-16T23:48:36.904 INFO:tasks.workunit.client.1.smithi043.stdout:0/19: write - no filename 2024-09-16T23:48:36.904 INFO:tasks.workunit.client.1.smithi043.stdout:0/20: dread - no filename 2024-09-16T23:48:36.904 INFO:tasks.workunit.client.1.smithi043.stdout:8/2: mknod c0 0 2024-09-16T23:48:36.904 INFO:tasks.workunit.client.1.smithi043.stdout:7/10: chown l0 1133 1 2024-09-16T23:48:36.904 INFO:tasks.workunit.client.1.smithi043.stdout:4/6: symlink l2 0 2024-09-16T23:48:36.905 INFO:tasks.workunit.client.1.smithi043.stdout:8/3: chown c0 0 1 2024-09-16T23:48:36.905 INFO:tasks.workunit.client.1.smithi043.stdout:8/4: truncate - no filename 2024-09-16T23:48:36.905 INFO:tasks.workunit.client.1.smithi043.stdout:3/10: chown f1 1287560941 1 2024-09-16T23:48:36.906 INFO:tasks.workunit.client.1.smithi043.stdout:4/7: chown f0 2431866 1 2024-09-16T23:48:36.907 INFO:tasks.workunit.client.1.smithi043.stdout:4/8: chown l2 15 1 2024-09-16T23:48:36.908 INFO:tasks.workunit.client.1.smithi043.stdout:6/4: creat f2 x:0 0 0 2024-09-16T23:48:36.908 INFO:tasks.workunit.client.1.smithi043.stdout:6/5: readlink - no filename 2024-09-16T23:48:36.909 INFO:tasks.workunit.client.1.smithi043.stdout:7/11: creat f1 x:0 0 0 2024-09-16T23:48:36.911 INFO:tasks.workunit.client.1.smithi043.stdout:0/21: symlink d0/d2/l5 0 2024-09-16T23:48:36.911 INFO:tasks.workunit.client.1.smithi043.stdout:6/6: write f1 [304264,46658] 0 2024-09-16T23:48:36.912 INFO:tasks.workunit.client.1.smithi043.stdout:9/10: mknod c0 0 2024-09-16T23:48:36.912 INFO:tasks.workunit.client.1.smithi043.stdout:9/11: dread - no filename 2024-09-16T23:48:36.913 INFO:tasks.workunit.client.1.smithi043.stdout:9/12: readlink - no filename 2024-09-16T23:48:36.913 INFO:tasks.workunit.client.1.smithi043.stdout:9/13: dread - no filename 2024-09-16T23:48:36.913 INFO:tasks.workunit.client.1.smithi043.stdout:7/12: write f1 [457481,73180] 0 2024-09-16T23:48:36.916 INFO:tasks.workunit.client.1.smithi043.stdout:3/11: mkdir d4 0 2024-09-16T23:48:36.917 INFO:tasks.workunit.client.1.smithi043.stdout:9/14: chown c0 86570619 1 2024-09-16T23:48:36.918 INFO:tasks.workunit.client.1.smithi043.stdout:9/15: dread - no filename 2024-09-16T23:48:36.918 INFO:tasks.workunit.client.1.smithi043.stdout:8/5: symlink l1 0 2024-09-16T23:48:36.918 INFO:tasks.workunit.client.1.smithi043.stdout:8/6: dread - no filename 2024-09-16T23:48:36.918 INFO:tasks.workunit.client.1.smithi043.stdout:8/7: read - no filename 2024-09-16T23:48:36.919 INFO:tasks.workunit.client.1.smithi043.stdout:6/7: truncate f2 382381 0 2024-09-16T23:48:36.921 INFO:tasks.workunit.client.1.smithi043.stdout:3/12: chown c2 7838 1 2024-09-16T23:48:36.922 INFO:tasks.workunit.client.1.smithi043.stdout:6/8: stat f1 0 2024-09-16T23:48:36.924 INFO:tasks.workunit.client.1.smithi043.stdout:1/12: dwrite f1 [0,4194304] 0 2024-09-16T23:48:36.926 INFO:tasks.workunit.client.1.smithi043.stdout:0/22: symlink d0/d2/l6 0 2024-09-16T23:48:36.931 INFO:tasks.workunit.client.1.smithi043.stdout:0/23: stat d0/d2 0 2024-09-16T23:48:36.933 INFO:tasks.workunit.client.1.smithi043.stdout:4/9: dwrite f1 [0,4194304] 0 2024-09-16T23:48:36.935 INFO:tasks.workunit.client.1.smithi043.stdout:9/16: mknod c1 0 2024-09-16T23:48:36.935 INFO:tasks.workunit.client.1.smithi043.stdout:9/17: dread - no filename 2024-09-16T23:48:36.938 INFO:tasks.workunit.client.1.smithi043.stdout:4/10: chown f1 92499 1 2024-09-16T23:48:36.938 INFO:tasks.workunit.client.1.smithi043.stdout:9/18: stat c1 0 2024-09-16T23:48:36.939 INFO:tasks.workunit.client.1.smithi043.stdout:9/19: write - no filename 2024-09-16T23:48:36.939 INFO:tasks.workunit.client.1.smithi043.stdout:9/20: truncate - no filename 2024-09-16T23:48:36.939 INFO:tasks.workunit.client.1.smithi043.stdout:9/21: truncate - no filename 2024-09-16T23:48:36.941 INFO:tasks.workunit.client.1.smithi043.stdout:9/22: chown c1 152042603 1 2024-09-16T23:48:36.941 INFO:tasks.workunit.client.1.smithi043.stdout:9/23: readlink - no filename 2024-09-16T23:48:36.941 INFO:tasks.workunit.client.1.smithi043.stdout:9/24: write - no filename 2024-09-16T23:48:36.941 INFO:tasks.workunit.client.1.smithi043.stdout:9/25: dread - no filename 2024-09-16T23:48:36.942 INFO:tasks.workunit.client.1.smithi043.stdout:6/9: creat f3 x:0 0 0 2024-09-16T23:48:36.942 INFO:tasks.workunit.client.1.smithi043.stdout:6/10: readlink - no filename 2024-09-16T23:48:36.944 INFO:tasks.workunit.client.1.smithi043.stdout:1/13: mkdir d4 0 2024-09-16T23:48:36.946 INFO:tasks.workunit.client.1.smithi043.stdout:4/11: chown f1 1038 1 2024-09-16T23:48:36.946 INFO:tasks.workunit.client.1.smithi043.stdout:7/13: dwrite f1 [0,4194304] 0 2024-09-16T23:48:36.946 INFO:tasks.workunit.client.1.smithi043.stdout:4/12: stat f0 0 2024-09-16T23:48:36.947 INFO:tasks.workunit.client.1.smithi043.stdout:7/14: chown l0 65643784 1 2024-09-16T23:48:36.954 INFO:tasks.workunit.client.1.smithi043.stdout:0/24: mkdir d0/d2/d7 0 2024-09-16T23:48:36.954 INFO:tasks.workunit.client.1.smithi043.stdout:4/13: write f1 [1780651,15747] 0 2024-09-16T23:48:36.956 INFO:tasks.workunit.client.1.smithi043.stdout:3/13: dwrite f1 [0,4194304] 0 2024-09-16T23:48:36.957 INFO:tasks.workunit.client.1.smithi043.stdout:4/14: rmdir - no directory 2024-09-16T23:48:36.958 INFO:tasks.workunit.client.1.smithi043.stdout:6/11: symlink l4 0 2024-09-16T23:48:36.959 INFO:tasks.workunit.client.1.smithi043.stdout:3/14: chown c2 239372889 1 2024-09-16T23:48:36.959 INFO:tasks.workunit.client.1.smithi043.stdout:4/15: stat f1 0 2024-09-16T23:48:36.959 INFO:tasks.workunit.client.1.smithi043.stdout:7/15: mknod c2 0 2024-09-16T23:48:36.961 INFO:tasks.workunit.client.1.smithi043.stdout:3/15: read f1 [1706846,109239] 0 2024-09-16T23:48:36.962 INFO:tasks.workunit.client.1.smithi043.stdout:0/25: mknod d0/d2/c8 0 2024-09-16T23:48:36.962 INFO:tasks.workunit.client.1.smithi043.stdout:1/14: mknod d4/c5 0 2024-09-16T23:48:36.964 INFO:tasks.workunit.client.1.smithi043.stdout:4/16: symlink l3 0 2024-09-16T23:48:36.964 INFO:tasks.workunit.client.1.smithi043.stdout:6/12: mknod c5 0 2024-09-16T23:48:36.964 INFO:tasks.workunit.client.1.smithi043.stdout:7/16: mknod c3 0 2024-09-16T23:48:36.970 INFO:tasks.workunit.client.1.smithi043.stdout:7/17: mkdir d4 0 2024-09-16T23:48:36.972 INFO:tasks.workunit.client.1.smithi043.stdout:7/18: chown c2 712448 1 2024-09-16T23:48:36.972 INFO:tasks.workunit.client.1.smithi043.stdout:1/15: dread f1 [0,4194304] 0 2024-09-16T23:48:36.973 INFO:tasks.workunit.client.1.smithi043.stdout:3/16: creat d4/f5 x:0 0 0 2024-09-16T23:48:36.977 INFO:tasks.workunit.client.1.smithi043.stdout:0/26: link d0/l4 d0/l9 0 2024-09-16T23:48:36.977 INFO:tasks.workunit.client.1.smithi043.stdout:0/27: dread - no filename 2024-09-16T23:48:36.979 INFO:tasks.workunit.client.1.smithi043.stdout:3/17: creat d4/f6 x:0 0 0 2024-09-16T23:48:36.979 INFO:tasks.workunit.client.1.smithi043.stdout:6/13: dwrite f2 [0,4194304] 0 2024-09-16T23:48:36.980 INFO:tasks.workunit.client.1.smithi043.stdout:7/19: symlink d4/l5 0 2024-09-16T23:48:36.981 INFO:tasks.workunit.client.1.smithi043.stdout:3/18: write d4/f5 [414273,28725] 0 2024-09-16T23:48:36.984 INFO:tasks.workunit.client.1.smithi043.stdout:3/19: read - d4/f6 zero size 2024-09-16T23:48:36.984 INFO:tasks.workunit.client.1.smithi043.stdout:3/20: readlink - no filename 2024-09-16T23:48:36.984 INFO:tasks.workunit.client.1.smithi043.stdout:0/28: symlink d0/d2/d7/la 0 2024-09-16T23:48:36.984 INFO:tasks.workunit.client.1.smithi043.stdout:0/29: dwrite - no filename 2024-09-16T23:48:36.985 INFO:tasks.workunit.client.1.smithi043.stdout:0/30: dwrite - no filename 2024-09-16T23:48:36.985 INFO:tasks.workunit.client.1.smithi043.stdout:0/31: read - no filename 2024-09-16T23:48:36.985 INFO:tasks.workunit.client.1.smithi043.stdout:0/32: truncate - no filename 2024-09-16T23:48:36.985 INFO:tasks.workunit.client.1.smithi043.stdout:7/20: creat d4/f6 x:0 0 0 2024-09-16T23:48:36.985 INFO:tasks.workunit.client.1.smithi043.stdout:0/33: symlink d0/lb 0 2024-09-16T23:48:36.989 INFO:tasks.workunit.client.1.smithi043.stdout:0/34: mkdir d0/dc 0 2024-09-16T23:48:36.991 INFO:tasks.workunit.client.1.smithi043.stdout:0/35: mkdir d0/d2/dd 0 2024-09-16T23:48:36.992 INFO:tasks.workunit.client.1.smithi043.stdout:0/36: chown d0/d2/c3 0 1 2024-09-16T23:48:36.994 INFO:tasks.workunit.client.1.smithi043.stdout:0/37: mkdir d0/de 0 2024-09-16T23:48:36.995 INFO:tasks.workunit.client.1.smithi043.stdout:0/38: chown d0/d2/c8 451879259 1 2024-09-16T23:48:36.996 INFO:tasks.workunit.client.1.smithi043.stdout:0/39: mkdir d0/d2/df 0 2024-09-16T23:48:36.999 INFO:tasks.workunit.client.1.smithi043.stdout:0/40: creat d0/d2/df/f10 x:0 0 0 2024-09-16T23:48:36.999 INFO:tasks.workunit.client.1.smithi043.stdout:2/4: getdents . 0 2024-09-16T23:48:37.002 INFO:tasks.workunit.client.1.smithi043.stdout:2/5: rename l0 to l1 0 2024-09-16T23:48:37.002 INFO:tasks.workunit.client.1.smithi043.stdout:2/6: write - no filename 2024-09-16T23:48:37.004 INFO:tasks.workunit.client.1.smithi043.stdout:0/41: link d0/d2/l6 d0/l11 0 2024-09-16T23:48:37.005 INFO:tasks.workunit.client.1.smithi043.stdout:3/21: fsync f1 0 2024-09-16T23:48:37.006 INFO:tasks.workunit.client.1.smithi043.stdout:0/42: truncate d0/d2/df/f10 329604 0 2024-09-16T23:48:37.007 INFO:tasks.workunit.client.1.smithi043.stdout:3/22: mknod d4/c7 0 2024-09-16T23:48:37.008 INFO:tasks.workunit.client.1.smithi043.stdout:3/23: chown d4/c7 899406 1 2024-09-16T23:48:37.015 INFO:tasks.workunit.client.1.smithi043.stdout:9/26: getdents . 0 2024-09-16T23:48:37.015 INFO:tasks.workunit.client.1.smithi043.stdout:9/27: fdatasync - no filename 2024-09-16T23:48:37.015 INFO:tasks.workunit.client.1.smithi043.stdout:9/28: dread - no filename 2024-09-16T23:48:37.016 INFO:tasks.workunit.client.1.smithi043.stdout:9/29: rename c1 to c2 0 2024-09-16T23:48:37.017 INFO:tasks.workunit.client.1.smithi043.stdout:9/30: write - no filename 2024-09-16T23:48:37.017 INFO:tasks.workunit.client.1.smithi043.stdout:9/31: chown c0 239 1 2024-09-16T23:48:37.020 INFO:tasks.workunit.client.1.smithi043.stdout:6/14: getdents . 0 2024-09-16T23:48:37.021 INFO:tasks.workunit.client.1.smithi043.stdout:6/15: write f1 [389363,62340] 0 2024-09-16T23:48:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:36 smithi042 ceph-mon[27914]: pgmap v257: 65 pgs: 65 active+clean; 178 MiB data, 740 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-16T23:48:37.025 INFO:tasks.workunit.client.1.smithi043.stdout:6/16: link f2 f6 0 2024-09-16T23:48:37.026 INFO:tasks.workunit.client.1.smithi043.stdout:6/17: dread - f3 zero size 2024-09-16T23:48:37.027 INFO:tasks.workunit.client.1.smithi043.stdout:0/43: rename d0/d2/d7 to d0/d2/d12 0 2024-09-16T23:48:37.028 INFO:tasks.workunit.client.1.smithi043.stdout:7/21: rmdir d4 39 2024-09-16T23:48:37.028 INFO:tasks.workunit.client.1.smithi043.stdout:6/18: chown f6 1970307 1 2024-09-16T23:48:37.029 INFO:tasks.workunit.client.1.smithi043.stdout:7/22: fdatasync f1 0 2024-09-16T23:48:37.030 INFO:tasks.workunit.client.1.smithi043.stdout:6/19: fdatasync f3 0 2024-09-16T23:48:37.031 INFO:tasks.workunit.client.1.smithi043.stdout:0/44: creat d0/f13 x:0 0 0 2024-09-16T23:48:37.031 INFO:tasks.workunit.client.1.smithi043.stdout:7/23: readlink l0 0 2024-09-16T23:48:37.033 INFO:tasks.workunit.client.1.smithi043.stdout:1/16: dwrite f1 [4194304,4194304] 0 2024-09-16T23:48:37.034 INFO:tasks.workunit.client.1.smithi043.stdout:6/20: mknod c7 0 2024-09-16T23:48:37.035 INFO:tasks.workunit.client.1.smithi043.stdout:2/7: unlink l1 0 2024-09-16T23:48:37.035 INFO:tasks.workunit.client.1.smithi043.stdout:2/8: link - no file 2024-09-16T23:48:37.035 INFO:tasks.workunit.client.1.smithi043.stdout:2/9: truncate - no filename 2024-09-16T23:48:37.038 INFO:tasks.workunit.client.1.smithi043.stdout:7/24: rmdir d4 39 2024-09-16T23:48:37.042 INFO:tasks.workunit.client.1.smithi043.stdout:6/21: mknod c8 0 2024-09-16T23:48:37.044 INFO:tasks.workunit.client.1.smithi043.stdout:9/32: rename c2 to c3 0 2024-09-16T23:48:37.044 INFO:tasks.workunit.client.1.smithi043.stdout:9/33: dread - no filename 2024-09-16T23:48:37.044 INFO:tasks.workunit.client.1.smithi043.stdout:9/34: rmdir - no directory 2024-09-16T23:48:37.044 INFO:tasks.workunit.client.1.smithi043.stdout:9/35: dwrite - no filename 2024-09-16T23:48:37.044 INFO:tasks.workunit.client.1.smithi043.stdout:9/36: dwrite - no filename 2024-09-16T23:48:37.044 INFO:tasks.workunit.client.1.smithi043.stdout:9/37: dread - no filename 2024-09-16T23:48:37.045 INFO:tasks.workunit.client.1.smithi043.stdout:0/45: dwrite d0/d2/df/f10 [0,4194304] 0 2024-09-16T23:48:37.047 INFO:tasks.workunit.client.1.smithi043.stdout:7/25: write d4/f6 [830630,125569] 0 2024-09-16T23:48:37.049 INFO:tasks.workunit.client.1.smithi043.stdout:7/26: rmdir d4 39 2024-09-16T23:48:37.050 INFO:tasks.workunit.client.1.smithi043.stdout:1/17: fsync f1 0 2024-09-16T23:48:37.050 INFO:tasks.workunit.client.1.smithi043.stdout:7/27: write f1 [855593,47515] 0 2024-09-16T23:48:37.051 INFO:tasks.workunit.client.1.smithi043.stdout:0/46: mknod d0/de/c14 0 2024-09-16T23:48:37.056 INFO:tasks.workunit.client.1.smithi043.stdout:1/18: creat d4/f6 x:0 0 0 2024-09-16T23:48:37.059 INFO:tasks.workunit.client.1.smithi043.stdout:9/38: getdents . 0 2024-09-16T23:48:37.059 INFO:tasks.workunit.client.1.smithi043.stdout:9/39: dwrite - no filename 2024-09-16T23:48:37.059 INFO:tasks.workunit.client.1.smithi043.stdout:7/28: symlink d4/l7 0 2024-09-16T23:48:37.062 INFO:tasks.workunit.client.1.smithi043.stdout:1/19: rename d4/f6 to d4/f7 0 2024-09-16T23:48:37.064 INFO:tasks.workunit.client.1.smithi043.stdout:7/29: write f1 [2259349,80766] 0 2024-09-16T23:48:37.064 INFO:tasks.workunit.client.1.smithi043.stdout:0/47: dwrite d0/d2/df/f10 [0,4194304] 0 2024-09-16T23:48:37.064 INFO:tasks.workunit.client.1.smithi043.stdout:9/40: mkdir d4 0 2024-09-16T23:48:37.067 INFO:tasks.workunit.client.1.smithi043.stdout:1/20: symlink d4/l8 0 2024-09-16T23:48:37.068 INFO:tasks.workunit.client.1.smithi043.stdout:7/30: symlink d4/l8 0 2024-09-16T23:48:37.069 INFO:tasks.workunit.client.1.smithi043.stdout:1/21: chown d4/f7 2536416 1 2024-09-16T23:48:37.070 INFO:tasks.workunit.client.1.smithi043.stdout:9/41: mkdir d4/d5 0 2024-09-16T23:48:37.070 INFO:tasks.workunit.client.1.smithi043.stdout:6/22: dwrite f6 [0,4194304] 0 2024-09-16T23:48:37.071 INFO:tasks.workunit.client.1.smithi043.stdout:6/23: rmdir - no directory 2024-09-16T23:48:37.072 INFO:tasks.workunit.client.1.smithi043.stdout:6/24: readlink l4 0 2024-09-16T23:48:37.073 INFO:tasks.workunit.client.1.smithi043.stdout:6/25: chown l4 4533 1 2024-09-16T23:48:37.077 INFO:tasks.workunit.client.1.smithi043.stdout:9/42: rmdir d4 39 2024-09-16T23:48:37.078 INFO:tasks.workunit.client.1.smithi043.stdout:9/43: dwrite - no filename 2024-09-16T23:48:37.078 INFO:tasks.workunit.client.1.smithi043.stdout:7/31: dread d4/f6 [0,4194304] 0 2024-09-16T23:48:37.080 INFO:tasks.workunit.client.1.smithi043.stdout:0/48: dwrite d0/d2/df/f10 [4194304,4194304] 0 2024-09-16T23:48:37.081 INFO:tasks.workunit.client.1.smithi043.stdout:7/32: chown f1 27155842 1 2024-09-16T23:48:37.082 INFO:tasks.workunit.client.1.smithi043.stdout:7/33: chown c2 265857951 1 2024-09-16T23:48:37.084 INFO:tasks.workunit.client.1.smithi043.stdout:0/49: creat d0/de/f15 x:0 0 0 2024-09-16T23:48:37.086 INFO:tasks.workunit.client.1.smithi043.stdout:7/34: creat d4/f9 x:0 0 0 2024-09-16T23:48:37.090 INFO:tasks.workunit.client.1.smithi043.stdout:0/50: creat d0/d2/df/f16 x:0 0 0 2024-09-16T23:48:37.091 INFO:tasks.workunit.client.1.smithi043.stdout:1/22: dwrite d4/f7 [0,4194304] 0 2024-09-16T23:48:37.091 INFO:tasks.workunit.client.1.smithi043.stdout:7/35: dread d4/f6 [0,4194304] 0 2024-09-16T23:48:37.092 INFO:tasks.workunit.client.1.smithi043.stdout:1/23: stat d4/l8 0 2024-09-16T23:48:37.094 INFO:tasks.workunit.client.1.smithi043.stdout:1/24: mkdir d4/d9 0 2024-09-16T23:48:37.095 INFO:tasks.workunit.client.1.smithi043.stdout:7/36: write d4/f6 [1755045,54066] 0 2024-09-16T23:48:37.096 INFO:tasks.workunit.client.1.smithi043.stdout:1/25: symlink d4/d9/la 0 2024-09-16T23:48:37.101 INFO:tasks.workunit.client.1.smithi043.stdout:7/37: dread d4/f6 [0,4194304] 0 2024-09-16T23:48:37.101 INFO:tasks.workunit.client.1.smithi043.stdout:7/38: fdatasync f1 0 2024-09-16T23:48:37.107 INFO:tasks.workunit.client.1.smithi043.stdout:1/26: truncate d4/f7 1427419 0 2024-09-16T23:48:37.111 INFO:tasks.workunit.client.1.smithi043.stdout:7/39: dwrite d4/f9 [0,4194304] 0 2024-09-16T23:48:37.112 INFO:tasks.workunit.client.1.smithi043.stdout:7/40: chown c3 719 1 2024-09-16T23:48:37.115 INFO:tasks.workunit.client.1.smithi043.stdout:7/41: dread d4/f6 [0,4194304] 0 2024-09-16T23:48:37.116 INFO:tasks.workunit.client.1.smithi043.stdout:1/27: link l3 d4/d9/lb 0 2024-09-16T23:48:37.123 INFO:tasks.workunit.client.1.smithi043.stdout:7/42: dread d4/f9 [0,4194304] 0 2024-09-16T23:48:37.125 INFO:tasks.workunit.client.1.smithi043.stdout:1/28: dwrite f1 [0,4194304] 0 2024-09-16T23:48:37.128 INFO:tasks.workunit.client.1.smithi043.stdout:7/43: link d4/f6 d4/fa 0 2024-09-16T23:48:37.129 INFO:tasks.workunit.client.1.smithi043.stdout:7/44: fsync d4/f9 0 2024-09-16T23:48:37.130 INFO:tasks.workunit.client.1.smithi043.stdout:1/29: dread f1 [0,4194304] 0 2024-09-16T23:48:37.132 INFO:tasks.workunit.client.1.smithi043.stdout:7/45: mkdir d4/db 0 2024-09-16T23:48:37.133 INFO:tasks.workunit.client.1.smithi043.stdout:1/30: write d4/f7 [567623,19423] 0 2024-09-16T23:48:37.134 INFO:tasks.workunit.client.1.smithi043.stdout:7/46: write d4/f6 [1176848,71760] 0 2024-09-16T23:48:37.135 INFO:tasks.workunit.client.1.smithi043.stdout:7/47: read f1 [4055745,82283] 0 2024-09-16T23:48:37.139 INFO:tasks.workunit.client.1.smithi043.stdout:7/48: creat d4/fc x:0 0 0 2024-09-16T23:48:37.140 INFO:tasks.workunit.client.1.smithi043.stdout:7/49: dread - d4/fc zero size 2024-09-16T23:48:37.143 INFO:tasks.workunit.client.1.smithi043.stdout:1/31: dwrite f1 [0,4194304] 0 2024-09-16T23:48:37.145 INFO:tasks.workunit.client.1.smithi043.stdout:1/32: chown d4/f7 8631 1 2024-09-16T23:48:37.148 INFO:tasks.workunit.client.1.smithi043.stdout:1/33: creat d4/fc x:0 0 0 2024-09-16T23:48:37.152 INFO:tasks.workunit.client.1.smithi043.stdout:7/50: dwrite d4/fa [0,4194304] 0 2024-09-16T23:48:37.153 INFO:tasks.workunit.client.1.smithi043.stdout:7/51: chown f1 4590199 1 2024-09-16T23:48:37.155 INFO:tasks.workunit.client.1.smithi043.stdout:1/34: rename d4/d9/lb to d4/ld 0 2024-09-16T23:48:37.155 INFO:tasks.workunit.client.1.smithi043.stdout:7/52: symlink d4/ld 0 2024-09-16T23:48:37.155 INFO:tasks.workunit.client.1.smithi043.stdout:1/35: stat d4/d9 0 2024-09-16T23:48:37.156 INFO:tasks.workunit.client.1.smithi043.stdout:7/53: chown d4 149 1 2024-09-16T23:48:37.157 INFO:tasks.workunit.client.1.smithi043.stdout:7/54: chown d4/fa 0 1 2024-09-16T23:48:37.159 INFO:tasks.workunit.client.1.smithi043.stdout:1/36: unlink l3 0 2024-09-16T23:48:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:38 smithi042 ceph-mon[27914]: pgmap v258: 65 pgs: 65 active+clean; 184 MiB data, 784 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 382 op/s 2024-09-16T23:48:39.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:38 smithi043 ceph-mon[35191]: pgmap v258: 65 pgs: 65 active+clean; 184 MiB data, 784 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 382 op/s 2024-09-16T23:48:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:39 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:40.028 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:39 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:41.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:40 smithi042 ceph-mon[27914]: pgmap v259: 65 pgs: 65 active+clean; 217 MiB data, 888 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 8.2 MiB/s wr, 390 op/s 2024-09-16T23:48:41.047 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:40 smithi043 ceph-mon[35191]: pgmap v259: 65 pgs: 65 active+clean; 217 MiB data, 888 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 8.2 MiB/s wr, 390 op/s 2024-09-16T23:48:42.031 INFO:tasks.workunit.client.1.smithi043.stdout:5/0: sync 2024-09-16T23:48:42.033 INFO:tasks.workunit.client.1.smithi043.stdout:5/1: mknod c0 0 2024-09-16T23:48:42.033 INFO:tasks.workunit.client.1.smithi043.stdout:5/2: dwrite - no filename 2024-09-16T23:48:42.034 INFO:tasks.workunit.client.1.smithi043.stdout:5/3: dwrite - no filename 2024-09-16T23:48:42.034 INFO:tasks.workunit.client.1.smithi043.stdout:5/4: dread - no filename 2024-09-16T23:48:42.034 INFO:tasks.workunit.client.1.smithi043.stdout:5/5: readlink - no filename 2024-09-16T23:48:42.034 INFO:tasks.workunit.client.1.smithi043.stdout:5/6: chown c0 557 1 2024-09-16T23:48:42.035 INFO:tasks.workunit.client.1.smithi043.stdout:5/7: unlink c0 0 2024-09-16T23:48:42.035 INFO:tasks.workunit.client.1.smithi043.stdout:5/8: write - no filename 2024-09-16T23:48:42.036 INFO:tasks.workunit.client.1.smithi043.stdout:5/9: link - no file 2024-09-16T23:48:42.037 INFO:tasks.workunit.client.1.smithi043.stdout:5/10: mknod c1 0 2024-09-16T23:48:42.038 INFO:tasks.workunit.client.1.smithi043.stdout:5/11: creat f2 x:0 0 0 2024-09-16T23:48:42.039 INFO:tasks.workunit.client.1.smithi043.stdout:5/12: write f2 [1030531,60250] 0 2024-09-16T23:48:42.042 INFO:tasks.workunit.client.1.smithi043.stdout:5/13: read f2 [814447,12187] 0 2024-09-16T23:48:42.043 INFO:tasks.workunit.client.1.smithi043.stdout:5/14: mkdir d3 0 2024-09-16T23:48:42.045 INFO:tasks.workunit.client.1.smithi043.stdout:5/15: mknod d3/c4 0 2024-09-16T23:48:42.045 INFO:tasks.workunit.client.1.smithi043.stdout:5/16: truncate f2 1158969 0 2024-09-16T23:48:42.047 INFO:tasks.workunit.client.1.smithi043.stdout:5/17: symlink d3/l5 0 2024-09-16T23:48:42.047 INFO:tasks.workunit.client.1.smithi043.stdout:5/18: rename d3 to d3/d6 22 2024-09-16T23:48:42.048 INFO:tasks.workunit.client.1.smithi043.stdout:5/19: mknod d3/c7 0 2024-09-16T23:48:42.055 INFO:tasks.workunit.client.1.smithi043.stdout:5/20: dwrite f2 [0,4194304] 0 2024-09-16T23:48:42.241 INFO:tasks.workunit.client.1.smithi043.stdout:5/21: truncate f2 1432770 0 2024-09-16T23:48:42.243 INFO:tasks.workunit.client.1.smithi043.stdout:5/22: fsync f2 0 2024-09-16T23:48:42.244 INFO:tasks.workunit.client.1.smithi043.stdout:5/23: fdatasync f2 0 2024-09-16T23:48:42.247 INFO:tasks.workunit.client.1.smithi043.stdout:5/24: link d3/c4 d3/c8 0 2024-09-16T23:48:42.249 INFO:tasks.workunit.client.1.smithi043.stdout:5/25: mkdir d3/d9 0 2024-09-16T23:48:42.249 INFO:tasks.workunit.client.1.smithi043.stdout:5/26: fdatasync f2 0 2024-09-16T23:48:42.251 INFO:tasks.workunit.client.1.smithi043.stdout:5/27: unlink d3/l5 0 2024-09-16T23:48:42.252 INFO:tasks.workunit.client.1.smithi043.stdout:5/28: read f2 [1395762,85721] 0 2024-09-16T23:48:42.255 INFO:tasks.workunit.client.1.smithi043.stdout:5/29: rename d3/c8 to d3/d9/ca 0 2024-09-16T23:48:42.256 INFO:tasks.workunit.client.1.smithi043.stdout:5/30: write f2 [1148893,26941] 0 2024-09-16T23:48:42.258 INFO:tasks.workunit.client.1.smithi043.stdout:5/31: dread f2 [0,4194304] 0 2024-09-16T23:48:42.259 INFO:tasks.workunit.client.1.smithi043.stdout:5/32: mkdir d3/db 0 2024-09-16T23:48:42.261 INFO:tasks.workunit.client.1.smithi043.stdout:5/33: dread f2 [0,4194304] 0 2024-09-16T23:48:42.263 INFO:tasks.workunit.client.1.smithi043.stdout:5/34: creat d3/fc x:0 0 0 2024-09-16T23:48:42.268 INFO:tasks.workunit.client.1.smithi043.stdout:5/35: fsync d3/fc 0 2024-09-16T23:48:42.269 INFO:tasks.workunit.client.1.smithi043.stdout:5/36: dread - d3/fc zero size 2024-09-16T23:48:42.272 INFO:tasks.workunit.client.1.smithi043.stdout:5/37: link c1 d3/db/cd 0 2024-09-16T23:48:42.273 INFO:tasks.workunit.client.1.smithi043.stdout:5/38: dread f2 [0,4194304] 0 2024-09-16T23:48:42.280 INFO:tasks.workunit.client.1.smithi043.stdout:5/39: dwrite f2 [0,4194304] 0 2024-09-16T23:48:42.281 INFO:tasks.workunit.client.1.smithi043.stdout:5/40: chown d3 0 1 2024-09-16T23:48:42.283 INFO:tasks.workunit.client.1.smithi043.stdout:5/41: symlink d3/d9/le 0 2024-09-16T23:48:42.284 INFO:tasks.workunit.client.1.smithi043.stdout:5/42: chown f2 1722 1 2024-09-16T23:48:42.287 INFO:tasks.workunit.client.1.smithi043.stdout:5/43: dread f2 [0,4194304] 0 2024-09-16T23:48:42.290 INFO:tasks.workunit.client.1.smithi043.stdout:5/44: unlink d3/fc 0 2024-09-16T23:48:42.290 INFO:tasks.workunit.client.1.smithi043.stdout:5/45: truncate f2 4433562 0 2024-09-16T23:48:42.292 INFO:tasks.workunit.client.1.smithi043.stdout:5/46: write f2 [4132246,108471] 0 2024-09-16T23:48:42.300 INFO:tasks.workunit.client.1.smithi043.stdout:5/47: dwrite f2 [0,4194304] 0 2024-09-16T23:48:42.303 INFO:tasks.workunit.client.1.smithi043.stdout:5/48: dread f2 [0,4194304] 0 2024-09-16T23:48:42.306 INFO:tasks.workunit.client.1.smithi043.stdout:5/49: symlink d3/db/lf 0 2024-09-16T23:48:42.309 INFO:tasks.workunit.client.1.smithi043.stdout:5/50: mkdir d3/db/d10 0 2024-09-16T23:48:42.312 INFO:tasks.workunit.client.1.smithi043.stdout:5/51: dread f2 [0,4194304] 0 2024-09-16T23:48:42.314 INFO:tasks.workunit.client.1.smithi043.stdout:5/52: rmdir d3/d9 39 2024-09-16T23:48:42.318 INFO:tasks.workunit.client.1.smithi043.stdout:5/53: rename d3/db/cd to d3/db/c11 0 2024-09-16T23:48:42.319 INFO:tasks.workunit.client.1.smithi043.stdout:5/54: fsync f2 0 2024-09-16T23:48:42.322 INFO:tasks.workunit.client.1.smithi043.stdout:5/55: creat d3/db/d10/f12 x:0 0 0 2024-09-16T23:48:42.322 INFO:tasks.workunit.client.1.smithi043.stdout:5/56: read f2 [1705653,29435] 0 2024-09-16T23:48:42.323 INFO:tasks.workunit.client.1.smithi043.stdout:5/57: write f2 [935068,98291] 0 2024-09-16T23:48:42.328 INFO:tasks.workunit.client.1.smithi043.stdout:5/58: dwrite f2 [0,4194304] 0 2024-09-16T23:48:42.329 INFO:tasks.workunit.client.1.smithi043.stdout:5/59: readlink d3/db/lf 0 2024-09-16T23:48:42.330 INFO:tasks.workunit.client.1.smithi043.stdout:5/60: fdatasync d3/db/d10/f12 0 2024-09-16T23:48:42.332 INFO:tasks.workunit.client.1.smithi043.stdout:5/61: symlink d3/db/d10/l13 0 2024-09-16T23:48:42.335 INFO:tasks.workunit.client.1.smithi043.stdout:5/62: dread f2 [0,4194304] 0 2024-09-16T23:48:42.341 INFO:tasks.workunit.client.1.smithi043.stdout:5/63: rename d3/db/d10/l13 to d3/d9/l14 0 2024-09-16T23:48:42.350 INFO:tasks.workunit.client.1.smithi043.stdout:5/64: unlink d3/db/c11 0 2024-09-16T23:48:42.351 INFO:tasks.workunit.client.1.smithi043.stdout:5/65: dread - d3/db/d10/f12 zero size 2024-09-16T23:48:42.356 INFO:tasks.workunit.client.1.smithi043.stdout:5/66: truncate f2 316781 0 2024-09-16T23:48:42.361 INFO:tasks.workunit.client.1.smithi043.stdout:5/67: dread f2 [0,4194304] 0 2024-09-16T23:48:42.363 INFO:tasks.workunit.client.1.smithi043.stdout:5/68: fsync f2 0 2024-09-16T23:48:42.366 INFO:tasks.workunit.client.1.smithi043.stdout:5/69: rename d3/db/d10 to d3/db/d15 0 2024-09-16T23:48:42.367 INFO:tasks.workunit.client.1.smithi043.stdout:5/70: chown d3/d9/le 574157 1 2024-09-16T23:48:42.368 INFO:tasks.workunit.client.1.smithi043.stdout:5/71: rmdir d3/db 39 2024-09-16T23:48:42.373 INFO:tasks.workunit.client.1.smithi043.stdout:5/72: rename d3/c4 to d3/db/c16 0 2024-09-16T23:48:42.374 INFO:tasks.workunit.client.1.smithi043.stdout:5/73: fsync d3/db/d15/f12 0 2024-09-16T23:48:42.376 INFO:tasks.workunit.client.1.smithi043.stdout:5/74: creat d3/f17 x:0 0 0 2024-09-16T23:48:42.381 INFO:tasks.workunit.client.1.smithi043.stdout:5/75: getdents d3/db 0 2024-09-16T23:48:42.382 INFO:tasks.workunit.client.1.smithi043.stdout:5/76: dread f2 [0,4194304] 0 2024-09-16T23:48:42.389 INFO:tasks.workunit.client.1.smithi043.stdout:5/77: dwrite d3/db/d15/f12 [0,4194304] 0 2024-09-16T23:48:42.393 INFO:tasks.workunit.client.1.smithi043.stdout:5/78: rename d3/d9/le to d3/db/l18 0 2024-09-16T23:48:42.401 INFO:tasks.workunit.client.1.smithi043.stdout:5/79: dwrite d3/db/d15/f12 [4194304,4194304] 0 2024-09-16T23:48:42.402 INFO:tasks.workunit.client.1.smithi043.stdout:5/80: truncate f2 584692 0 2024-09-16T23:48:42.406 INFO:tasks.workunit.client.1.smithi043.stdout:5/81: dread d3/db/d15/f12 [0,4194304] 0 2024-09-16T23:48:42.407 INFO:tasks.workunit.client.1.smithi043.stdout:5/82: read f2 [538720,102148] 0 2024-09-16T23:48:42.410 INFO:tasks.workunit.client.1.smithi043.stdout:5/83: rmdir d3/db 39 2024-09-16T23:48:42.413 INFO:tasks.workunit.client.1.smithi043.stdout:5/84: mkdir d3/db/d19 0 2024-09-16T23:48:42.420 INFO:tasks.workunit.client.1.smithi043.stdout:5/85: dwrite d3/f17 [0,4194304] 0 2024-09-16T23:48:42.422 INFO:tasks.workunit.client.1.smithi043.stdout:5/86: chown d3/db/d15/f12 23670646 1 2024-09-16T23:48:42.422 INFO:tasks.workunit.client.1.smithi043.stdout:5/87: chown d3/db/d19 456717 1 2024-09-16T23:48:42.424 INFO:tasks.workunit.client.1.smithi043.stdout:5/88: symlink d3/db/l1a 0 2024-09-16T23:48:42.426 INFO:tasks.workunit.client.1.smithi043.stdout:5/89: mkdir d3/d1b 0 2024-09-16T23:48:42.429 INFO:tasks.workunit.client.1.smithi043.stdout:5/90: symlink d3/l1c 0 2024-09-16T23:48:42.430 INFO:tasks.workunit.client.1.smithi043.stdout:5/91: chown d3/db/l1a 46612807 1 2024-09-16T23:48:42.431 INFO:tasks.workunit.client.1.smithi043.stdout:5/92: chown d3/db/l1a 3486 1 2024-09-16T23:48:42.432 INFO:tasks.workunit.client.1.smithi043.stdout:5/93: write d3/f17 [4805143,111747] 0 2024-09-16T23:48:42.434 INFO:tasks.workunit.client.1.smithi043.stdout:5/94: symlink d3/d1b/l1d 0 2024-09-16T23:48:42.438 INFO:tasks.workunit.client.1.smithi043.stdout:5/95: dread d3/f17 [0,4194304] 0 2024-09-16T23:48:42.441 INFO:tasks.workunit.client.1.smithi043.stdout:5/96: mknod d3/c1e 0 2024-09-16T23:48:42.443 INFO:tasks.workunit.client.1.smithi043.stdout:5/97: write d3/db/d15/f12 [7875237,68551] 0 2024-09-16T23:48:42.445 INFO:tasks.workunit.client.1.smithi043.stdout:5/98: dread f2 [0,4194304] 0 2024-09-16T23:48:42.452 INFO:tasks.workunit.client.1.smithi043.stdout:5/99: dwrite d3/f17 [0,4194304] 0 2024-09-16T23:48:42.456 INFO:tasks.workunit.client.1.smithi043.stdout:5/100: rename d3/d9/l14 to d3/d1b/l1f 0 2024-09-16T23:48:42.458 INFO:tasks.workunit.client.1.smithi043.stdout:5/101: symlink d3/l20 0 2024-09-16T23:48:42.462 INFO:tasks.workunit.client.1.smithi043.stdout:5/102: dread d3/f17 [0,4194304] 0 2024-09-16T23:48:42.464 INFO:tasks.workunit.client.1.smithi043.stdout:5/103: truncate f2 737586 0 2024-09-16T23:48:42.467 INFO:tasks.workunit.client.1.smithi043.stdout:5/104: mkdir d3/db/d19/d21 0 2024-09-16T23:48:42.468 INFO:tasks.workunit.client.1.smithi043.stdout:5/105: fdatasync d3/db/d15/f12 0 2024-09-16T23:48:42.476 INFO:tasks.workunit.client.1.smithi043.stdout:5/106: dread f2 [0,4194304] 0 2024-09-16T23:48:42.478 INFO:tasks.workunit.client.1.smithi043.stdout:5/107: mkdir d3/d9/d22 0 2024-09-16T23:48:42.480 INFO:tasks.workunit.client.1.smithi043.stdout:5/108: write d3/f17 [2398133,15133] 0 2024-09-16T23:48:42.483 INFO:tasks.workunit.client.1.smithi043.stdout:5/109: rename d3/db/d19/d21 to d3/d9/d23 0 2024-09-16T23:48:42.485 INFO:tasks.workunit.client.1.smithi043.stdout:5/110: creat d3/db/f24 x:0 0 0 2024-09-16T23:48:42.486 INFO:tasks.workunit.client.1.smithi043.stdout:5/111: mknod d3/db/c25 0 2024-09-16T23:48:42.488 INFO:tasks.workunit.client.1.smithi043.stdout:5/112: mkdir d3/db/d26 0 2024-09-16T23:48:42.489 INFO:tasks.workunit.client.1.smithi043.stdout:5/113: read d3/f17 [3891300,79] 0 2024-09-16T23:48:42.492 INFO:tasks.workunit.client.1.smithi043.stdout:5/114: dread d3/db/d15/f12 [0,4194304] 0 2024-09-16T23:48:42.493 INFO:tasks.workunit.client.1.smithi043.stdout:5/115: truncate d3/db/f24 520249 0 2024-09-16T23:48:42.494 INFO:tasks.workunit.client.1.smithi043.stdout:5/116: dread d3/db/f24 [0,4194304] 0 2024-09-16T23:48:43.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:42 smithi042 ceph-mon[27914]: pgmap v260: 65 pgs: 65 active+clean; 239 MiB data, 916 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 8.7 MiB/s wr, 396 op/s 2024-09-16T23:48:43.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:42 smithi043 ceph-mon[35191]: pgmap v260: 65 pgs: 65 active+clean; 239 MiB data, 916 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 8.7 MiB/s wr, 396 op/s 2024-09-16T23:48:43.485 INFO:tasks.workunit.client.1.smithi043.stdout:8/8: sync 2024-09-16T23:48:43.486 INFO:tasks.workunit.client.1.smithi043.stdout:8/9: dwrite - no filename 2024-09-16T23:48:43.486 INFO:tasks.workunit.client.1.smithi043.stdout:8/10: truncate - no filename 2024-09-16T23:48:43.488 INFO:tasks.workunit.client.1.smithi043.stdout:8/11: rename l1 to l2 0 2024-09-16T23:48:43.488 INFO:tasks.workunit.client.1.smithi043.stdout:8/12: dwrite - no filename 2024-09-16T23:48:43.489 INFO:tasks.workunit.client.1.smithi043.stdout:8/13: rename c0 to c3 0 2024-09-16T23:48:43.492 INFO:tasks.workunit.client.1.smithi043.stdout:8/14: creat f4 x:0 0 0 2024-09-16T23:48:43.493 INFO:tasks.workunit.client.1.smithi043.stdout:8/15: symlink l5 0 2024-09-16T23:48:43.493 INFO:tasks.workunit.client.1.smithi043.stdout:8/16: rmdir - no directory 2024-09-16T23:48:43.493 INFO:tasks.workunit.client.1.smithi043.stdout:8/17: dread - f4 zero size 2024-09-16T23:48:43.494 INFO:tasks.workunit.client.1.smithi043.stdout:8/18: read - f4 zero size 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:3/24: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:2/10: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:5/117: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:4/17: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:6/26: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:9/44: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:9/45: dread - no filename 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:7/55: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:0/51: sync 2024-09-16T23:48:43.506 INFO:tasks.workunit.client.1.smithi043.stdout:1/37: sync 2024-09-16T23:48:43.508 INFO:tasks.workunit.client.1.smithi043.stdout:6/27: fdatasync f2 0 2024-09-16T23:48:43.509 INFO:tasks.workunit.client.1.smithi043.stdout:8/19: getdents . 0 2024-09-16T23:48:43.510 INFO:tasks.workunit.client.1.smithi043.stdout:8/20: write f4 [599717,26278] 0 2024-09-16T23:48:43.511 INFO:tasks.workunit.client.1.smithi043.stdout:7/56: unlink l0 0 2024-09-16T23:48:43.512 INFO:tasks.workunit.client.1.smithi043.stdout:7/57: chown d4/f9 23 1 2024-09-16T23:48:43.513 INFO:tasks.workunit.client.1.smithi043.stdout:2/11: creat f2 x:0 0 0 2024-09-16T23:48:43.514 INFO:tasks.workunit.client.1.smithi043.stdout:2/12: fdatasync f2 0 2024-09-16T23:48:43.514 INFO:tasks.workunit.client.1.smithi043.stdout:8/21: dread f4 [0,4194304] 0 2024-09-16T23:48:43.514 INFO:tasks.workunit.client.1.smithi043.stdout:4/18: mknod c4 0 2024-09-16T23:48:43.515 INFO:tasks.workunit.client.1.smithi043.stdout:2/13: write f2 [82628,37078] 0 2024-09-16T23:48:43.516 INFO:tasks.workunit.client.1.smithi043.stdout:5/118: mkdir d3/db/d15/d27 0 2024-09-16T23:48:43.519 INFO:tasks.workunit.client.1.smithi043.stdout:6/28: read f1 [380257,85181] 0 2024-09-16T23:48:43.519 INFO:tasks.workunit.client.1.smithi043.stdout:0/52: dread d0/d2/df/f10 [0,4194304] 0 2024-09-16T23:48:43.519 INFO:tasks.workunit.client.1.smithi043.stdout:7/58: mkdir d4/de 0 2024-09-16T23:48:43.520 INFO:tasks.workunit.client.1.smithi043.stdout:8/22: symlink l6 0 2024-09-16T23:48:43.523 INFO:tasks.workunit.client.1.smithi043.stdout:2/14: mkdir d3 0 2024-09-16T23:48:43.525 INFO:tasks.workunit.client.1.smithi043.stdout:9/46: creat d4/d5/f6 x:0 0 0 2024-09-16T23:48:43.526 INFO:tasks.workunit.client.1.smithi043.stdout:9/47: write d4/d5/f6 [68769,120152] 0 2024-09-16T23:48:43.529 INFO:tasks.workunit.client.1.smithi043.stdout:1/38: mkdir d4/d9/de 0 2024-09-16T23:48:43.530 INFO:tasks.workunit.client.1.smithi043.stdout:4/19: write f0 [1060809,21918] 0 2024-09-16T23:48:43.530 INFO:tasks.workunit.client.1.smithi043.stdout:0/53: creat d0/d2/df/f17 x:0 0 0 2024-09-16T23:48:43.531 INFO:tasks.workunit.client.1.smithi043.stdout:7/59: truncate d4/fa 3448253 0 2024-09-16T23:48:43.531 INFO:tasks.workunit.client.1.smithi043.stdout:4/20: write f0 [1801856,46235] 0 2024-09-16T23:48:43.532 INFO:tasks.workunit.client.1.smithi043.stdout:0/54: dread - d0/d2/df/f17 zero size 2024-09-16T23:48:43.534 INFO:tasks.workunit.client.1.smithi043.stdout:3/25: link d4/c7 d4/c8 0 2024-09-16T23:48:43.536 INFO:tasks.workunit.client.1.smithi043.stdout:9/48: mknod d4/c7 0 2024-09-16T23:48:43.537 INFO:tasks.workunit.client.1.smithi043.stdout:8/23: truncate f4 449834 0 2024-09-16T23:48:43.537 INFO:tasks.workunit.client.1.smithi043.stdout:8/24: readlink l5 0 2024-09-16T23:48:43.537 INFO:tasks.workunit.client.1.smithi043.stdout:1/39: readlink d4/ld 0 2024-09-16T23:48:43.538 INFO:tasks.workunit.client.1.smithi043.stdout:6/29: write f1 [702077,45375] 0 2024-09-16T23:48:43.538 INFO:tasks.workunit.client.1.smithi043.stdout:9/49: dread d4/d5/f6 [0,4194304] 0 2024-09-16T23:48:43.539 INFO:tasks.workunit.client.1.smithi043.stdout:9/50: write d4/d5/f6 [369409,1027] 0 2024-09-16T23:48:43.542 INFO:tasks.workunit.client.1.smithi043.stdout:9/51: dread d4/d5/f6 [0,4194304] 0 2024-09-16T23:48:43.543 INFO:tasks.workunit.client.1.smithi043.stdout:0/55: creat d0/d2/df/f18 x:0 0 0 2024-09-16T23:48:43.543 INFO:tasks.workunit.client.1.smithi043.stdout:9/52: chown d4/d5/f6 248174 1 2024-09-16T23:48:43.543 INFO:tasks.workunit.client.1.smithi043.stdout:3/26: symlink d4/l9 0 2024-09-16T23:48:43.544 INFO:tasks.workunit.client.1.smithi043.stdout:4/21: dread f1 [0,4194304] 0 2024-09-16T23:48:43.544 INFO:tasks.workunit.client.1.smithi043.stdout:9/53: write d4/d5/f6 [437877,39717] 0 2024-09-16T23:48:43.547 INFO:tasks.workunit.client.1.smithi043.stdout:9/54: write d4/d5/f6 [1455080,107936] 0 2024-09-16T23:48:43.548 INFO:tasks.workunit.client.1.smithi043.stdout:8/25: symlink l7 0 2024-09-16T23:48:43.549 INFO:tasks.workunit.client.1.smithi043.stdout:4/22: dread f1 [0,4194304] 0 2024-09-16T23:48:43.550 INFO:tasks.workunit.client.1.smithi043.stdout:6/30: mknod c9 0 2024-09-16T23:48:43.551 INFO:tasks.workunit.client.1.smithi043.stdout:7/60: symlink d4/de/lf 0 2024-09-16T23:48:43.552 INFO:tasks.workunit.client.1.smithi043.stdout:0/56: fsync d0/f13 0 2024-09-16T23:48:43.553 INFO:tasks.workunit.client.1.smithi043.stdout:4/23: dread f1 [0,4194304] 0 2024-09-16T23:48:43.554 INFO:tasks.workunit.client.1.smithi043.stdout:2/15: rmdir d3 0 2024-09-16T23:48:43.554 INFO:tasks.workunit.client.1.smithi043.stdout:2/16: chown f2 4 1 2024-09-16T23:48:43.555 INFO:tasks.workunit.client.1.smithi043.stdout:3/27: symlink d4/la 0 2024-09-16T23:48:43.556 INFO:tasks.workunit.client.1.smithi043.stdout:5/119: getdents d3/db/d15 0 2024-09-16T23:48:43.556 INFO:tasks.workunit.client.1.smithi043.stdout:5/120: readlink d3/db/lf 0 2024-09-16T23:48:43.558 INFO:tasks.workunit.client.1.smithi043.stdout:9/55: mknod d4/c8 0 2024-09-16T23:48:43.561 INFO:tasks.workunit.client.1.smithi043.stdout:6/31: rename l4 to la 0 2024-09-16T23:48:43.561 INFO:tasks.workunit.client.1.smithi043.stdout:1/40: dwrite f1 [4194304,4194304] 0 2024-09-16T23:48:43.562 INFO:tasks.workunit.client.1.smithi043.stdout:7/61: truncate f1 2239314 0 2024-09-16T23:48:43.562 INFO:tasks.workunit.client.1.smithi043.stdout:6/32: write f1 [825169,61959] 0 2024-09-16T23:48:43.563 INFO:tasks.workunit.client.1.smithi043.stdout:4/24: mkdir d5 0 2024-09-16T23:48:43.563 INFO:tasks.workunit.client.1.smithi043.stdout:0/57: chown d0/d2/l6 45 1 2024-09-16T23:48:43.565 INFO:tasks.workunit.client.1.smithi043.stdout:3/28: creat d4/fb x:0 0 0 2024-09-16T23:48:43.567 INFO:tasks.workunit.client.1.smithi043.stdout:9/56: mknod d4/c9 0 2024-09-16T23:48:43.576 INFO:tasks.workunit.client.1.smithi043.stdout:6/33: dread f2 [0,4194304] 0 2024-09-16T23:48:43.578 INFO:tasks.workunit.client.1.smithi043.stdout:9/57: write d4/d5/f6 [2405606,104240] 0 2024-09-16T23:48:43.580 INFO:tasks.workunit.client.1.smithi043.stdout:9/58: chown d4/c7 105265 1 2024-09-16T23:48:43.582 INFO:tasks.workunit.client.1.smithi043.stdout:8/26: chown f4 917164 1 2024-09-16T23:48:43.583 INFO:tasks.workunit.client.1.smithi043.stdout:8/27: chown c3 641 1 2024-09-16T23:48:43.584 INFO:tasks.workunit.client.1.smithi043.stdout:5/121: write d3/db/f24 [1401954,60415] 0 2024-09-16T23:48:43.585 INFO:tasks.workunit.client.1.smithi043.stdout:7/62: unlink d4/f6 0 2024-09-16T23:48:43.586 INFO:tasks.workunit.client.1.smithi043.stdout:8/28: dread f4 [0,4194304] 0 2024-09-16T23:48:43.587 INFO:tasks.workunit.client.1.smithi043.stdout:6/34: dwrite f1 [0,4194304] 0 2024-09-16T23:48:43.587 INFO:tasks.workunit.client.1.smithi043.stdout:4/25: rmdir d5 0 2024-09-16T23:48:43.588 INFO:tasks.workunit.client.1.smithi043.stdout:9/59: symlink d4/la 0 2024-09-16T23:48:43.590 INFO:tasks.workunit.client.1.smithi043.stdout:5/122: mkdir d3/d9/d23/d28 0 2024-09-16T23:48:43.593 INFO:tasks.workunit.client.1.smithi043.stdout:9/60: read d4/d5/f6 [1459248,14956] 0 2024-09-16T23:48:43.593 INFO:tasks.workunit.client.1.smithi043.stdout:1/41: write d4/f7 [2097479,95494] 0 2024-09-16T23:48:43.597 INFO:tasks.workunit.client.1.smithi043.stdout:8/29: truncate f4 1483136 0 2024-09-16T23:48:43.597 INFO:tasks.workunit.client.1.smithi043.stdout:8/30: rmdir - no directory 2024-09-16T23:48:43.601 INFO:tasks.workunit.client.1.smithi043.stdout:9/61: write d4/d5/f6 [247709,113908] 0 2024-09-16T23:48:43.606 INFO:tasks.workunit.client.1.smithi043.stdout:6/35: dwrite f1 [0,4194304] 0 2024-09-16T23:48:43.608 INFO:tasks.workunit.client.1.smithi043.stdout:0/58: dwrite d0/d2/df/f10 [0,4194304] 0 2024-09-16T23:48:43.609 INFO:tasks.workunit.client.1.smithi043.stdout:8/31: creat f8 x:0 0 0 2024-09-16T23:48:43.610 INFO:tasks.workunit.client.1.smithi043.stdout:1/42: mkdir d4/df 0 2024-09-16T23:48:43.611 INFO:tasks.workunit.client.1.smithi043.stdout:3/29: dwrite d4/f6 [0,4194304] 0 2024-09-16T23:48:43.612 INFO:tasks.workunit.client.1.smithi043.stdout:0/59: readlink d0/d2/l6 0 2024-09-16T23:48:43.613 INFO:tasks.workunit.client.1.smithi043.stdout:4/26: write f1 [2397095,112135] 0 2024-09-16T23:48:43.615 INFO:tasks.workunit.client.1.smithi043.stdout:0/60: chown d0/dc 16887 1 2024-09-16T23:48:43.616 INFO:tasks.workunit.client.1.smithi043.stdout:8/32: symlink l9 0 2024-09-16T23:48:43.620 INFO:tasks.workunit.client.1.smithi043.stdout:5/123: truncate d3/db/d15/f12 7413549 0 2024-09-16T23:48:43.622 INFO:tasks.workunit.client.1.smithi043.stdout:9/62: dread d4/d5/f6 [0,4194304] 0 2024-09-16T23:48:43.624 INFO:tasks.workunit.client.1.smithi043.stdout:7/63: dwrite d4/fa [4194304,4194304] 0 2024-09-16T23:48:43.624 INFO:tasks.workunit.client.1.smithi043.stdout:6/36: dread f6 [0,4194304] 0 2024-09-16T23:48:43.625 INFO:tasks.workunit.client.1.smithi043.stdout:3/30: unlink d4/c7 0 2024-09-16T23:48:43.630 INFO:tasks.workunit.client.1.smithi043.stdout:0/61: rename d0/d2/d12/la to d0/d2/df/l19 0 2024-09-16T23:48:43.632 INFO:tasks.workunit.client.1.smithi043.stdout:8/33: dread f4 [0,4194304] 0 2024-09-16T23:48:43.634 INFO:tasks.workunit.client.1.smithi043.stdout:9/63: unlink d4/d5/f6 0 2024-09-16T23:48:43.641 INFO:tasks.workunit.client.1.smithi043.stdout:6/37: symlink lb 0 2024-09-16T23:48:43.642 INFO:tasks.workunit.client.1.smithi043.stdout:1/43: creat d4/d9/de/f10 x:0 0 0 2024-09-16T23:48:43.643 INFO:tasks.workunit.client.1.smithi043.stdout:3/31: unlink d4/f5 0 2024-09-16T23:48:43.649 INFO:tasks.workunit.client.1.smithi043.stdout:1/44: dwrite d4/f7 [0,4194304] 0 2024-09-16T23:48:43.650 INFO:tasks.workunit.client.1.smithi043.stdout:5/124: unlink d3/db/c16 0 2024-09-16T23:48:43.650 INFO:tasks.workunit.client.1.smithi043.stdout:1/45: write f1 [5309557,80481] 0 2024-09-16T23:48:43.651 INFO:tasks.workunit.client.1.smithi043.stdout:1/46: stat d4/df 0 2024-09-16T23:48:43.652 INFO:tasks.workunit.client.1.smithi043.stdout:0/62: unlink d0/de/c14 0 2024-09-16T23:48:43.653 INFO:tasks.workunit.client.1.smithi043.stdout:4/27: truncate f1 41078 0 2024-09-16T23:48:43.654 INFO:tasks.workunit.client.1.smithi043.stdout:0/63: truncate d0/d2/df/f18 635208 0 2024-09-16T23:48:43.655 INFO:tasks.workunit.client.1.smithi043.stdout:8/34: rename l5 to la 0 2024-09-16T23:48:43.655 INFO:tasks.workunit.client.1.smithi043.stdout:0/64: truncate d0/d2/df/f18 1103116 0 2024-09-16T23:48:43.655 INFO:tasks.workunit.client.1.smithi043.stdout:8/35: dread - f8 zero size 2024-09-16T23:48:43.656 INFO:tasks.workunit.client.1.smithi043.stdout:9/64: creat d4/fb x:0 0 0 2024-09-16T23:48:43.657 INFO:tasks.workunit.client.1.smithi043.stdout:8/36: chown l6 46351 1 2024-09-16T23:48:43.657 INFO:tasks.workunit.client.1.smithi043.stdout:8/37: fdatasync f8 0 2024-09-16T23:48:43.658 INFO:tasks.workunit.client.1.smithi043.stdout:6/38: creat fc x:0 0 0 2024-09-16T23:48:43.665 INFO:tasks.workunit.client.1.smithi043.stdout:1/47: creat d4/d9/f11 x:0 0 0 2024-09-16T23:48:43.669 INFO:tasks.workunit.client.1.smithi043.stdout:4/28: mknod c6 0 2024-09-16T23:48:43.675 INFO:tasks.workunit.client.1.smithi043.stdout:0/65: dwrite d0/d2/df/f10 [8388608,4194304] 0 2024-09-16T23:48:43.676 INFO:tasks.workunit.client.1.smithi043.stdout:9/65: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:43.676 INFO:tasks.workunit.client.1.smithi043.stdout:7/64: dwrite d4/fa [0,4194304] 0 2024-09-16T23:48:43.678 INFO:tasks.workunit.client.1.smithi043.stdout:3/32: dread f1 [0,4194304] 0 2024-09-16T23:48:43.678 INFO:tasks.workunit.client.1.smithi043.stdout:1/48: truncate d4/d9/de/f10 213705 0 2024-09-16T23:48:43.679 INFO:tasks.workunit.client.1.smithi043.stdout:6/39: creat fd x:0 0 0 2024-09-16T23:48:43.682 INFO:tasks.workunit.client.1.smithi043.stdout:4/29: rename f0 to f7 0 2024-09-16T23:48:43.686 INFO:tasks.workunit.client.1.smithi043.stdout:1/49: dread f1 [0,4194304] 0 2024-09-16T23:48:43.689 INFO:tasks.workunit.client.1.smithi043.stdout:9/66: symlink d4/lc 0 2024-09-16T23:48:43.690 INFO:tasks.workunit.client.1.smithi043.stdout:8/38: write f4 [931831,86315] 0 2024-09-16T23:48:43.691 INFO:tasks.workunit.client.1.smithi043.stdout:8/39: dread - f8 zero size 2024-09-16T23:48:43.691 INFO:tasks.workunit.client.1.smithi043.stdout:8/40: rmdir - no directory 2024-09-16T23:48:43.694 INFO:tasks.workunit.client.1.smithi043.stdout:0/66: mkdir d0/d2/df/d1a 0 2024-09-16T23:48:43.694 INFO:tasks.workunit.client.1.smithi043.stdout:0/67: chown d0 4132 1 2024-09-16T23:48:43.695 INFO:tasks.workunit.client.1.smithi043.stdout:5/125: truncate d3/db/f24 74831 0 2024-09-16T23:48:43.697 INFO:tasks.workunit.client.1.smithi043.stdout:0/68: dread d0/d2/df/f18 [0,4194304] 0 2024-09-16T23:48:43.704 INFO:tasks.workunit.client.1.smithi043.stdout:9/67: symlink d4/ld 0 2024-09-16T23:48:43.706 INFO:tasks.workunit.client.1.smithi043.stdout:3/33: dread d4/f6 [0,4194304] 0 2024-09-16T23:48:43.708 INFO:tasks.workunit.client.1.smithi043.stdout:6/40: write f6 [71541,82770] 0 2024-09-16T23:48:43.709 INFO:tasks.workunit.client.1.smithi043.stdout:9/68: dread d4/fb [0,4194304] 0 2024-09-16T23:48:43.712 INFO:tasks.workunit.client.1.smithi043.stdout:7/65: mknod d4/db/c10 0 2024-09-16T23:48:43.717 INFO:tasks.workunit.client.1.smithi043.stdout:3/34: rmdir d4 39 2024-09-16T23:48:43.719 INFO:tasks.workunit.client.1.smithi043.stdout:6/41: mkdir de 0 2024-09-16T23:48:43.720 INFO:tasks.workunit.client.1.smithi043.stdout:9/69: symlink d4/d5/le 0 2024-09-16T23:48:43.720 INFO:tasks.workunit.client.1.smithi043.stdout:1/50: write d4/fc [45938,54720] 0 2024-09-16T23:48:43.722 INFO:tasks.workunit.client.1.smithi043.stdout:0/69: creat d0/d2/df/d1a/f1b x:0 0 0 2024-09-16T23:48:43.722 INFO:tasks.workunit.client.1.smithi043.stdout:0/70: readlink d0/d2/l6 0 2024-09-16T23:48:43.723 INFO:tasks.workunit.client.1.smithi043.stdout:8/41: truncate f4 723759 0 2024-09-16T23:48:43.723 INFO:tasks.workunit.client.1.smithi043.stdout:8/42: dread - f8 zero size 2024-09-16T23:48:43.725 INFO:tasks.workunit.client.1.smithi043.stdout:4/30: truncate f1 36182 0 2024-09-16T23:48:43.726 INFO:tasks.workunit.client.1.smithi043.stdout:7/66: mkdir d4/db/d11 0 2024-09-16T23:48:43.729 INFO:tasks.workunit.client.1.smithi043.stdout:9/70: creat d4/ff x:0 0 0 2024-09-16T23:48:43.729 INFO:tasks.workunit.client.1.smithi043.stdout:5/126: mknod d3/d9/c29 0 2024-09-16T23:48:43.731 INFO:tasks.workunit.client.1.smithi043.stdout:1/51: rename d4/d9/de to d4/d9/d12 0 2024-09-16T23:48:43.733 INFO:tasks.workunit.client.1.smithi043.stdout:2/17: sync 2024-09-16T23:48:43.733 INFO:tasks.workunit.client.1.smithi043.stdout:0/71: dwrite d0/d2/df/f10 [12582912,4194304] 0 2024-09-16T23:48:43.734 INFO:tasks.workunit.client.1.smithi043.stdout:2/18: fsync f2 0 2024-09-16T23:48:43.735 INFO:tasks.workunit.client.1.smithi043.stdout:1/52: write d4/d9/f11 [268782,86581] 0 2024-09-16T23:48:43.735 INFO:tasks.workunit.client.1.smithi043.stdout:4/31: mkdir d8 0 2024-09-16T23:48:43.736 INFO:tasks.workunit.client.1.smithi043.stdout:2/19: read f2 [95905,70736] 0 2024-09-16T23:48:43.736 INFO:tasks.workunit.client.1.smithi043.stdout:7/67: dread d4/fa [4194304,4194304] 0 2024-09-16T23:48:43.737 INFO:tasks.workunit.client.1.smithi043.stdout:0/72: write d0/d2/df/f10 [7690241,9247] 0 2024-09-16T23:48:43.737 INFO:tasks.workunit.client.1.smithi043.stdout:2/20: chown f2 5113 1 2024-09-16T23:48:43.739 INFO:tasks.workunit.client.1.smithi043.stdout:3/35: dread - d4/fb zero size 2024-09-16T23:48:43.741 INFO:tasks.workunit.client.1.smithi043.stdout:6/42: mkdir de/df 0 2024-09-16T23:48:43.744 INFO:tasks.workunit.client.1.smithi043.stdout:9/71: rename d4/fb to d4/f10 0 2024-09-16T23:48:43.745 INFO:tasks.workunit.client.1.smithi043.stdout:5/127: creat d3/db/f2a x:0 0 0 2024-09-16T23:48:43.748 INFO:tasks.workunit.client.1.smithi043.stdout:8/43: unlink la 0 2024-09-16T23:48:43.749 INFO:tasks.workunit.client.1.smithi043.stdout:8/44: chown f8 0 1 2024-09-16T23:48:43.754 INFO:tasks.workunit.client.1.smithi043.stdout:5/128: dwrite d3/db/f2a [0,4194304] 0 2024-09-16T23:48:43.754 INFO:tasks.workunit.client.1.smithi043.stdout:2/21: symlink l4 0 2024-09-16T23:48:43.757 INFO:tasks.workunit.client.1.smithi043.stdout:0/73: creat d0/f1c x:0 0 0 2024-09-16T23:48:43.758 INFO:tasks.workunit.client.1.smithi043.stdout:7/68: dread d4/f9 [0,4194304] 0 2024-09-16T23:48:43.759 INFO:tasks.workunit.client.1.smithi043.stdout:0/74: write d0/d2/df/d1a/f1b [126221,26192] 0 2024-09-16T23:48:43.762 INFO:tasks.workunit.client.1.smithi043.stdout:9/72: write d4/f10 [4518631,74286] 0 2024-09-16T23:48:43.764 INFO:tasks.workunit.client.1.smithi043.stdout:8/45: mknod cb 0 2024-09-16T23:48:43.766 INFO:tasks.workunit.client.1.smithi043.stdout:9/73: dread d4/f10 [0,4194304] 0 2024-09-16T23:48:43.769 INFO:tasks.workunit.client.1.smithi043.stdout:1/53: mkdir d4/df/d13 0 2024-09-16T23:48:43.770 INFO:tasks.workunit.client.1.smithi043.stdout:5/129: chown c1 0 1 2024-09-16T23:48:43.771 INFO:tasks.workunit.client.1.smithi043.stdout:2/22: creat f5 x:0 0 0 2024-09-16T23:48:43.771 INFO:tasks.workunit.client.1.smithi043.stdout:2/23: rmdir - no directory 2024-09-16T23:48:43.772 INFO:tasks.workunit.client.1.smithi043.stdout:2/24: read f2 [117676,119355] 0 2024-09-16T23:48:43.772 INFO:tasks.workunit.client.1.smithi043.stdout:2/25: chown l4 655 1 2024-09-16T23:48:43.772 INFO:tasks.workunit.client.1.smithi043.stdout:3/36: unlink d4/c8 0 2024-09-16T23:48:43.773 INFO:tasks.workunit.client.1.smithi043.stdout:7/69: mknod d4/db/c12 0 2024-09-16T23:48:43.775 INFO:tasks.workunit.client.1.smithi043.stdout:0/75: rename d0/d2/l5 to d0/d2/d12/l1d 0 2024-09-16T23:48:43.777 INFO:tasks.workunit.client.1.smithi043.stdout:6/43: write f3 [76748,120029] 0 2024-09-16T23:48:43.778 INFO:tasks.workunit.client.1.smithi043.stdout:6/44: fdatasync f6 0 2024-09-16T23:48:43.781 INFO:tasks.workunit.client.1.smithi043.stdout:1/54: unlink d4/f7 0 2024-09-16T23:48:43.783 INFO:tasks.workunit.client.1.smithi043.stdout:2/26: creat f6 x:0 0 0 2024-09-16T23:48:43.784 INFO:tasks.workunit.client.1.smithi043.stdout:2/27: dread - f6 zero size 2024-09-16T23:48:43.790 INFO:tasks.workunit.client.1.smithi043.stdout:3/37: dwrite d4/f6 [0,4194304] 0 2024-09-16T23:48:43.790 INFO:tasks.workunit.client.1.smithi043.stdout:6/45: symlink de/l10 0 2024-09-16T23:48:43.791 INFO:tasks.workunit.client.1.smithi043.stdout:3/38: dread - d4/fb zero size 2024-09-16T23:48:43.792 INFO:tasks.workunit.client.1.smithi043.stdout:1/55: symlink d4/df/l14 0 2024-09-16T23:48:43.793 INFO:tasks.workunit.client.1.smithi043.stdout:1/56: readlink d4/df/l14 0 2024-09-16T23:48:43.794 INFO:tasks.workunit.client.1.smithi043.stdout:8/46: rename f4 to fc 0 2024-09-16T23:48:43.796 INFO:tasks.workunit.client.1.smithi043.stdout:7/70: creat d4/db/d11/f13 x:0 0 0 2024-09-16T23:48:43.797 INFO:tasks.workunit.client.1.smithi043.stdout:0/76: mknod d0/dc/c1e 0 2024-09-16T23:48:43.798 INFO:tasks.workunit.client.1.smithi043.stdout:1/57: dread d4/d9/f11 [0,4194304] 0 2024-09-16T23:48:43.798 INFO:tasks.workunit.client.1.smithi043.stdout:6/46: symlink de/l11 0 2024-09-16T23:48:43.799 INFO:tasks.workunit.client.1.smithi043.stdout:0/77: stat d0/f1c 0 2024-09-16T23:48:43.800 INFO:tasks.workunit.client.1.smithi043.stdout:6/47: stat c5 0 2024-09-16T23:48:43.800 INFO:tasks.workunit.client.1.smithi043.stdout:5/130: mknod d3/d9/c2b 0 2024-09-16T23:48:43.803 INFO:tasks.workunit.client.1.smithi043.stdout:0/78: stat d0/d2 0 2024-09-16T23:48:43.804 INFO:tasks.workunit.client.1.smithi043.stdout:9/74: getdents d4 0 2024-09-16T23:48:43.805 INFO:tasks.workunit.client.1.smithi043.stdout:7/71: creat d4/db/f14 x:0 0 0 2024-09-16T23:48:43.807 INFO:tasks.workunit.client.1.smithi043.stdout:3/39: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:43.809 INFO:tasks.workunit.client.1.smithi043.stdout:1/58: chown d4/c5 179308 1 2024-09-16T23:48:43.812 INFO:tasks.workunit.client.1.smithi043.stdout:9/75: write d4/ff [44806,26536] 0 2024-09-16T23:48:43.813 INFO:tasks.workunit.client.1.smithi043.stdout:1/59: fsync d4/fc 0 2024-09-16T23:48:43.816 INFO:tasks.workunit.client.1.smithi043.stdout:0/79: write d0/d2/df/f10 [1764608,118106] 0 2024-09-16T23:48:43.816 INFO:tasks.workunit.client.1.smithi043.stdout:4/32: dwrite f1 [0,4194304] 0 2024-09-16T23:48:43.819 INFO:tasks.workunit.client.1.smithi043.stdout:4/33: stat c4 0 2024-09-16T23:48:43.824 INFO:tasks.workunit.client.1.smithi043.stdout:8/47: truncate fc 991474 0 2024-09-16T23:48:43.824 INFO:tasks.workunit.client.1.smithi043.stdout:9/76: read d4/ff [23328,55022] 0 2024-09-16T23:48:43.825 INFO:tasks.workunit.client.1.smithi043.stdout:5/131: truncate d3/db/f24 349445 0 2024-09-16T23:48:43.825 INFO:tasks.workunit.client.1.smithi043.stdout:6/48: dwrite fc [0,4194304] 0 2024-09-16T23:48:43.826 INFO:tasks.workunit.client.1.smithi043.stdout:8/48: dread - f8 zero size 2024-09-16T23:48:43.826 INFO:tasks.workunit.client.1.smithi043.stdout:1/60: chown d4/l8 2621 1 2024-09-16T23:48:43.827 INFO:tasks.workunit.client.1.smithi043.stdout:0/80: mknod d0/d2/d12/c1f 0 2024-09-16T23:48:43.827 INFO:tasks.workunit.client.1.smithi043.stdout:6/49: truncate fc 4470122 0 2024-09-16T23:48:43.828 INFO:tasks.workunit.client.1.smithi043.stdout:3/40: dread d4/fb [0,4194304] 0 2024-09-16T23:48:43.829 INFO:tasks.workunit.client.1.smithi043.stdout:7/72: write f1 [142862,70753] 0 2024-09-16T23:48:43.829 INFO:tasks.workunit.client.1.smithi043.stdout:9/77: creat d4/f11 x:0 0 0 2024-09-16T23:48:43.832 INFO:tasks.workunit.client.1.smithi043.stdout:6/50: dread f3 [0,4194304] 0 2024-09-16T23:48:43.833 INFO:tasks.workunit.client.1.smithi043.stdout:1/61: dread d4/fc [0,4194304] 0 2024-09-16T23:48:43.833 INFO:tasks.workunit.client.1.smithi043.stdout:4/34: symlink d8/l9 0 2024-09-16T23:48:43.834 INFO:tasks.workunit.client.1.smithi043.stdout:4/35: read f1 [2133343,73970] 0 2024-09-16T23:48:43.834 INFO:tasks.workunit.client.1.smithi043.stdout:7/73: fsync f1 0 2024-09-16T23:48:43.834 INFO:tasks.workunit.client.1.smithi043.stdout:0/81: creat d0/de/f20 x:0 0 0 2024-09-16T23:48:43.835 INFO:tasks.workunit.client.1.smithi043.stdout:6/51: stat f3 0 2024-09-16T23:48:43.835 INFO:tasks.workunit.client.1.smithi043.stdout:9/78: mkdir d4/d12 0 2024-09-16T23:48:43.836 INFO:tasks.workunit.client.1.smithi043.stdout:0/82: read - d0/d2/df/f17 zero size 2024-09-16T23:48:43.838 INFO:tasks.workunit.client.1.smithi043.stdout:0/83: chown d0/d2/d12/c1f 259 1 2024-09-16T23:48:43.838 INFO:tasks.workunit.client.1.smithi043.stdout:7/74: rmdir d4/db 39 2024-09-16T23:48:43.839 INFO:tasks.workunit.client.1.smithi043.stdout:3/41: dread d4/fb [0,4194304] 0 2024-09-16T23:48:43.841 INFO:tasks.workunit.client.1.smithi043.stdout:4/36: write f7 [1372371,64376] 0 2024-09-16T23:48:43.844 INFO:tasks.workunit.client.1.smithi043.stdout:3/42: dread f1 [0,4194304] 0 2024-09-16T23:48:43.845 INFO:tasks.workunit.client.1.smithi043.stdout:7/75: dread d4/f9 [0,4194304] 0 2024-09-16T23:48:43.845 INFO:tasks.workunit.client.1.smithi043.stdout:0/84: rmdir d0/dc 39 2024-09-16T23:48:43.846 INFO:tasks.workunit.client.1.smithi043.stdout:3/43: read d4/fb [3235964,32762] 0 2024-09-16T23:48:43.847 INFO:tasks.workunit.client.1.smithi043.stdout:8/49: link fc fd 0 2024-09-16T23:48:43.848 INFO:tasks.workunit.client.1.smithi043.stdout:1/62: getdents d4/df/d13 0 2024-09-16T23:48:43.849 INFO:tasks.workunit.client.1.smithi043.stdout:3/44: dread f1 [0,4194304] 0 2024-09-16T23:48:43.850 INFO:tasks.workunit.client.1.smithi043.stdout:6/52: rename la to de/df/l12 0 2024-09-16T23:48:43.852 INFO:tasks.workunit.client.1.smithi043.stdout:4/37: truncate f1 1370451 0 2024-09-16T23:48:43.853 INFO:tasks.workunit.client.1.smithi043.stdout:5/132: write d3/db/f24 [1253356,83590] 0 2024-09-16T23:48:43.854 INFO:tasks.workunit.client.1.smithi043.stdout:8/50: mkdir de 0 2024-09-16T23:48:43.856 INFO:tasks.workunit.client.1.smithi043.stdout:1/63: mkdir d4/d9/d12/d15 0 2024-09-16T23:48:43.857 INFO:tasks.workunit.client.1.smithi043.stdout:3/45: rmdir d4 39 2024-09-16T23:48:43.859 INFO:tasks.workunit.client.1.smithi043.stdout:9/79: rename d4/la to d4/d5/l13 0 2024-09-16T23:48:43.860 INFO:tasks.workunit.client.1.smithi043.stdout:6/53: creat de/f13 x:0 0 0 2024-09-16T23:48:43.861 INFO:tasks.workunit.client.1.smithi043.stdout:0/85: mknod d0/dc/c21 0 2024-09-16T23:48:43.862 INFO:tasks.workunit.client.1.smithi043.stdout:6/54: chown f6 42374 1 2024-09-16T23:48:43.865 INFO:tasks.workunit.client.1.smithi043.stdout:4/38: rename d8/l9 to d8/la 0 2024-09-16T23:48:43.867 INFO:tasks.workunit.client.1.smithi043.stdout:1/64: dwrite d4/d9/d12/f10 [0,4194304] 0 2024-09-16T23:48:43.867 INFO:tasks.workunit.client.1.smithi043.stdout:7/76: truncate f1 1107690 0 2024-09-16T23:48:43.868 INFO:tasks.workunit.client.1.smithi043.stdout:4/39: write f7 [2199006,85363] 0 2024-09-16T23:48:43.868 INFO:tasks.workunit.client.1.smithi043.stdout:4/40: stat l3 0 2024-09-16T23:48:43.870 INFO:tasks.workunit.client.1.smithi043.stdout:6/55: dread fc [0,4194304] 0 2024-09-16T23:48:43.871 INFO:tasks.workunit.client.1.smithi043.stdout:9/80: symlink d4/l14 0 2024-09-16T23:48:43.873 INFO:tasks.workunit.client.1.smithi043.stdout:9/81: write d4/f11 [858151,23674] 0 2024-09-16T23:48:43.874 INFO:tasks.workunit.client.1.smithi043.stdout:9/82: chown d4/c8 60494 1 2024-09-16T23:48:43.876 INFO:tasks.workunit.client.1.smithi043.stdout:0/86: truncate d0/de/f15 885796 0 2024-09-16T23:48:43.879 INFO:tasks.workunit.client.1.smithi043.stdout:6/56: dwrite f6 [0,4194304] 0 2024-09-16T23:48:43.879 INFO:tasks.workunit.client.1.smithi043.stdout:0/87: truncate d0/de/f20 667337 0 2024-09-16T23:48:43.879 INFO:tasks.workunit.client.1.smithi043.stdout:7/77: write d4/db/f14 [553574,27761] 0 2024-09-16T23:48:43.880 INFO:tasks.workunit.client.1.smithi043.stdout:4/41: symlink d8/lb 0 2024-09-16T23:48:43.881 INFO:tasks.workunit.client.1.smithi043.stdout:3/46: stat d4/l9 0 2024-09-16T23:48:43.882 INFO:tasks.workunit.client.1.smithi043.stdout:4/42: truncate f7 3266702 0 2024-09-16T23:48:43.883 INFO:tasks.workunit.client.1.smithi043.stdout:0/88: write d0/d2/df/f17 [761892,101887] 0 2024-09-16T23:48:43.885 INFO:tasks.workunit.client.1.smithi043.stdout:0/89: write d0/f1c [790498,107342] 0 2024-09-16T23:48:43.886 INFO:tasks.workunit.client.1.smithi043.stdout:6/57: mkdir de/d14 0 2024-09-16T23:48:43.887 INFO:tasks.workunit.client.1.smithi043.stdout:6/58: write f2 [2180375,104660] 0 2024-09-16T23:48:43.889 INFO:tasks.workunit.client.1.smithi043.stdout:7/78: mkdir d4/db/d11/d15 0 2024-09-16T23:48:43.893 INFO:tasks.workunit.client.1.smithi043.stdout:3/47: write d4/fb [661622,130903] 0 2024-09-16T23:48:43.895 INFO:tasks.workunit.client.1.smithi043.stdout:9/83: mknod d4/d12/c15 0 2024-09-16T23:48:43.895 INFO:tasks.workunit.client.1.smithi043.stdout:4/43: creat d8/fc x:0 0 0 2024-09-16T23:48:43.897 INFO:tasks.workunit.client.1.smithi043.stdout:5/133: mknod d3/d9/d23/d28/c2c 0 2024-09-16T23:48:43.899 INFO:tasks.workunit.client.1.smithi043.stdout:6/59: dwrite fd [0,4194304] 0 2024-09-16T23:48:43.899 INFO:tasks.workunit.client.1.smithi043.stdout:4/44: mkdir d8/dd 0 2024-09-16T23:48:43.901 INFO:tasks.workunit.client.1.smithi043.stdout:6/60: write de/f13 [932789,75814] 0 2024-09-16T23:48:43.902 INFO:tasks.workunit.client.1.smithi043.stdout:6/61: chown fd 4 1 2024-09-16T23:48:43.902 INFO:tasks.workunit.client.1.smithi043.stdout:9/84: mkdir d4/d5/d16 0 2024-09-16T23:48:43.903 INFO:tasks.workunit.client.1.smithi043.stdout:1/65: truncate d4/d9/d12/f10 960785 0 2024-09-16T23:48:43.904 INFO:tasks.workunit.client.1.smithi043.stdout:6/62: write f6 [242312,20251] 0 2024-09-16T23:48:43.906 INFO:tasks.workunit.client.1.smithi043.stdout:5/134: creat d3/d9/d23/f2d x:0 0 0 2024-09-16T23:48:43.906 INFO:tasks.workunit.client.1.smithi043.stdout:9/85: write d4/f11 [34823,20965] 0 2024-09-16T23:48:43.911 INFO:tasks.workunit.client.1.smithi043.stdout:7/79: truncate f1 1108487 0 2024-09-16T23:48:43.914 INFO:tasks.workunit.client.1.smithi043.stdout:3/48: rename d4/l9 to d4/lc 0 2024-09-16T23:48:43.915 INFO:tasks.workunit.client.1.smithi043.stdout:4/45: dwrite d8/fc [0,4194304] 0 2024-09-16T23:48:43.916 INFO:tasks.workunit.client.1.smithi043.stdout:9/86: symlink d4/l17 0 2024-09-16T23:48:43.917 INFO:tasks.workunit.client.1.smithi043.stdout:5/135: mknod d3/db/d15/c2e 0 2024-09-16T23:48:43.923 INFO:tasks.workunit.client.1.smithi043.stdout:9/87: truncate d4/ff 407129 0 2024-09-16T23:48:43.925 INFO:tasks.workunit.client.1.smithi043.stdout:0/90: dwrite d0/f13 [0,4194304] 0 2024-09-16T23:48:43.925 INFO:tasks.workunit.client.1.smithi043.stdout:6/63: dwrite de/f13 [0,4194304] 0 2024-09-16T23:48:43.934 INFO:tasks.workunit.client.1.smithi043.stdout:9/88: read d4/ff [116862,25581] 0 2024-09-16T23:48:43.940 INFO:tasks.workunit.client.1.smithi043.stdout:1/66: write d4/d9/d12/f10 [1950344,116095] 0 2024-09-16T23:48:43.948 INFO:tasks.workunit.client.1.smithi043.stdout:9/89: dread d4/f11 [0,4194304] 0 2024-09-16T23:48:43.951 INFO:tasks.workunit.client.1.smithi043.stdout:9/90: dread d4/f11 [0,4194304] 0 2024-09-16T23:48:43.952 INFO:tasks.workunit.client.1.smithi043.stdout:1/67: dwrite d4/fc [0,4194304] 0 2024-09-16T23:48:43.953 INFO:tasks.workunit.client.1.smithi043.stdout:9/91: stat c0 0 2024-09-16T23:48:43.956 INFO:tasks.workunit.client.1.smithi043.stdout:6/64: mknod de/c15 0 2024-09-16T23:48:43.956 INFO:tasks.workunit.client.1.smithi043.stdout:0/91: dread d0/de/f15 [0,4194304] 0 2024-09-16T23:48:43.957 INFO:tasks.workunit.client.1.smithi043.stdout:9/92: dread d4/ff [0,4194304] 0 2024-09-16T23:48:43.962 INFO:tasks.workunit.client.1.smithi043.stdout:9/93: dread d4/ff [0,4194304] 0 2024-09-16T23:48:43.964 INFO:tasks.workunit.client.1.smithi043.stdout:1/68: dread d4/d9/f11 [0,4194304] 0 2024-09-16T23:48:43.966 INFO:tasks.workunit.client.1.smithi043.stdout:3/49: creat d4/fd x:0 0 0 2024-09-16T23:48:43.966 INFO:tasks.workunit.client.1.smithi043.stdout:6/65: dwrite f1 [0,4194304] 0 2024-09-16T23:48:43.970 INFO:tasks.workunit.client.1.smithi043.stdout:5/136: symlink d3/d9/d22/l2f 0 2024-09-16T23:48:43.972 INFO:tasks.workunit.client.1.smithi043.stdout:9/94: mknod d4/d12/c18 0 2024-09-16T23:48:43.976 INFO:tasks.workunit.client.1.smithi043.stdout:6/66: creat de/df/f16 x:0 0 0 2024-09-16T23:48:43.979 INFO:tasks.workunit.client.1.smithi043.stdout:5/137: unlink d3/db/lf 0 2024-09-16T23:48:43.980 INFO:tasks.workunit.client.1.smithi043.stdout:9/95: symlink d4/d12/l19 0 2024-09-16T23:48:43.982 INFO:tasks.workunit.client.1.smithi043.stdout:3/50: mkdir d4/de 0 2024-09-16T23:48:43.983 INFO:tasks.workunit.client.1.smithi043.stdout:3/51: dread - d4/fd zero size 2024-09-16T23:48:43.983 INFO:tasks.workunit.client.1.smithi043.stdout:7/80: write d4/fa [2917181,18603] 0 2024-09-16T23:48:43.983 INFO:tasks.workunit.client.1.smithi043.stdout:3/52: read - d4/fd zero size 2024-09-16T23:48:43.984 INFO:tasks.workunit.client.1.smithi043.stdout:3/53: chown d4/de 2985 1 2024-09-16T23:48:43.990 INFO:tasks.workunit.client.1.smithi043.stdout:7/81: truncate d4/f9 3367306 0 2024-09-16T23:48:43.993 INFO:tasks.workunit.client.1.smithi043.stdout:7/82: rmdir d4/db 39 2024-09-16T23:48:43.993 INFO:tasks.workunit.client.1.smithi043.stdout:9/96: dwrite d4/f11 [0,4194304] 0 2024-09-16T23:48:43.994 INFO:tasks.workunit.client.1.smithi043.stdout:4/46: write f1 [2396690,84303] 0 2024-09-16T23:48:43.995 INFO:tasks.workunit.client.1.smithi043.stdout:7/83: chown d4/l5 454 1 2024-09-16T23:48:43.996 INFO:tasks.workunit.client.1.smithi043.stdout:4/47: fsync d8/fc 0 2024-09-16T23:48:43.997 INFO:tasks.workunit.client.1.smithi043.stdout:3/54: rename d4/la to d4/lf 0 2024-09-16T23:48:43.998 INFO:tasks.workunit.client.1.smithi043.stdout:4/48: write f1 [1109823,111625] 0 2024-09-16T23:48:43.999 INFO:tasks.workunit.client.1.smithi043.stdout:3/55: dread - d4/fd zero size 2024-09-16T23:48:43.999 INFO:tasks.workunit.client.1.smithi043.stdout:9/97: creat d4/f1a x:0 0 0 2024-09-16T23:48:44.002 INFO:tasks.workunit.client.1.smithi043.stdout:7/84: stat d4/db/c10 0 2024-09-16T23:48:44.006 INFO:tasks.workunit.client.1.smithi043.stdout:3/56: dread d4/fb [0,4194304] 0 2024-09-16T23:48:44.008 INFO:tasks.workunit.client.1.smithi043.stdout:4/49: dread f7 [0,4194304] 0 2024-09-16T23:48:44.008 INFO:tasks.workunit.client.1.smithi043.stdout:3/57: truncate d4/fd 206640 0 2024-09-16T23:48:44.009 INFO:tasks.workunit.client.1.smithi043.stdout:0/92: creat d0/f22 x:0 0 0 2024-09-16T23:48:44.011 INFO:tasks.workunit.client.1.smithi043.stdout:1/69: rename d4/c5 to d4/d9/d12/d15/c16 0 2024-09-16T23:48:44.012 INFO:tasks.workunit.client.1.smithi043.stdout:9/98: rename d4/d5 to d4/d5/d16/d1b 22 2024-09-16T23:48:44.013 INFO:tasks.workunit.client.1.smithi043.stdout:0/93: mknod d0/d2/c23 0 2024-09-16T23:48:44.013 INFO:tasks.workunit.client.1.smithi043.stdout:4/50: mkdir d8/dd/de 0 2024-09-16T23:48:44.014 INFO:tasks.workunit.client.1.smithi043.stdout:1/70: creat d4/d9/f17 x:0 0 0 2024-09-16T23:48:44.015 INFO:tasks.workunit.client.1.smithi043.stdout:5/138: truncate d3/db/f2a 1576914 0 2024-09-16T23:48:44.015 INFO:tasks.workunit.client.1.smithi043.stdout:9/99: mkdir d4/d1c 0 2024-09-16T23:48:44.015 INFO:tasks.workunit.client.1.smithi043.stdout:0/94: write d0/f22 [742118,120488] 0 2024-09-16T23:48:44.021 INFO:tasks.workunit.client.1.smithi043.stdout:3/58: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:44.022 INFO:tasks.workunit.client.1.smithi043.stdout:5/139: symlink d3/d9/d23/l30 0 2024-09-16T23:48:44.023 INFO:tasks.workunit.client.1.smithi043.stdout:0/95: write d0/d2/df/f17 [1492504,32833] 0 2024-09-16T23:48:44.025 INFO:tasks.workunit.client.1.smithi043.stdout:4/51: mknod d8/dd/de/cf 0 2024-09-16T23:48:44.028 INFO:tasks.workunit.client.1.smithi043.stdout:5/140: dread - d3/d9/d23/f2d zero size 2024-09-16T23:48:44.028 INFO:tasks.workunit.client.1.smithi043.stdout:9/100: creat d4/d5/f1d x:0 0 0 2024-09-16T23:48:44.029 INFO:tasks.workunit.client.1.smithi043.stdout:1/71: dwrite d4/d9/f17 [0,4194304] 0 2024-09-16T23:48:44.032 INFO:tasks.workunit.client.1.smithi043.stdout:5/141: mknod d3/d9/d23/c31 0 2024-09-16T23:48:44.033 INFO:tasks.workunit.client.1.smithi043.stdout:4/52: creat d8/dd/f10 x:0 0 0 2024-09-16T23:48:44.035 INFO:tasks.workunit.client.1.smithi043.stdout:1/72: mknod d4/d9/c18 0 2024-09-16T23:48:44.040 INFO:tasks.workunit.client.1.smithi043.stdout:4/53: creat d8/dd/de/f11 x:0 0 0 2024-09-16T23:48:44.042 INFO:tasks.workunit.client.1.smithi043.stdout:0/96: getdents d0 0 2024-09-16T23:48:44.043 INFO:tasks.workunit.client.1.smithi043.stdout:3/59: truncate d4/fb 3091581 0 2024-09-16T23:48:44.044 INFO:tasks.workunit.client.1.smithi043.stdout:7/85: dwrite d4/f9 [0,4194304] 0 2024-09-16T23:48:44.044 INFO:tasks.workunit.client.1.smithi043.stdout:5/142: write f2 [96989,103652] 0 2024-09-16T23:48:44.045 INFO:tasks.workunit.client.1.smithi043.stdout:0/97: stat d0/dc/c1e 0 2024-09-16T23:48:44.051 INFO:tasks.workunit.client.1.smithi043.stdout:0/98: chown d0/d2/df/f10 463103 1 2024-09-16T23:48:44.052 INFO:tasks.workunit.client.1.smithi043.stdout:7/86: mknod d4/db/d11/c16 0 2024-09-16T23:48:44.052 INFO:tasks.workunit.client.1.smithi043.stdout:1/73: dwrite d4/d9/f17 [0,4194304] 0 2024-09-16T23:48:44.055 INFO:tasks.workunit.client.1.smithi043.stdout:3/60: creat d4/f10 x:0 0 0 2024-09-16T23:48:44.056 INFO:tasks.workunit.client.1.smithi043.stdout:4/54: dwrite f1 [0,4194304] 0 2024-09-16T23:48:44.056 INFO:tasks.workunit.client.1.smithi043.stdout:0/99: unlink d0/f22 0 2024-09-16T23:48:44.057 INFO:tasks.workunit.client.1.smithi043.stdout:1/74: write d4/d9/d12/f10 [1971838,121875] 0 2024-09-16T23:48:44.058 INFO:tasks.workunit.client.1.smithi043.stdout:7/87: mkdir d4/db/d17 0 2024-09-16T23:48:44.059 INFO:tasks.workunit.client.1.smithi043.stdout:1/75: chown d4/df 53787371 1 2024-09-16T23:48:44.060 INFO:tasks.workunit.client.1.smithi043.stdout:4/55: mknod d8/c12 0 2024-09-16T23:48:44.061 INFO:tasks.workunit.client.1.smithi043.stdout:5/143: creat d3/f32 x:0 0 0 2024-09-16T23:48:44.061 INFO:tasks.workunit.client.1.smithi043.stdout:1/76: dread d4/d9/f11 [0,4194304] 0 2024-09-16T23:48:44.062 INFO:tasks.workunit.client.1.smithi043.stdout:7/88: creat d4/db/f18 x:0 0 0 2024-09-16T23:48:44.063 INFO:tasks.workunit.client.1.smithi043.stdout:1/77: chown d4/ld 10988429 1 2024-09-16T23:48:44.065 INFO:tasks.workunit.client.1.smithi043.stdout:1/78: write d4/d9/d12/f10 [575290,8606] 0 2024-09-16T23:48:44.065 INFO:tasks.workunit.client.1.smithi043.stdout:5/144: rename d3/db/f24 to d3/db/d26/f33 0 2024-09-16T23:48:44.066 INFO:tasks.workunit.client.1.smithi043.stdout:4/56: creat d8/dd/de/f13 x:0 0 0 2024-09-16T23:48:44.066 INFO:tasks.workunit.client.1.smithi043.stdout:5/145: dread - d3/d9/d23/f2d zero size 2024-09-16T23:48:44.070 INFO:tasks.workunit.client.1.smithi043.stdout:1/79: mknod d4/d9/c19 0 2024-09-16T23:48:44.071 INFO:tasks.workunit.client.1.smithi043.stdout:7/89: creat d4/f19 x:0 0 0 2024-09-16T23:48:44.073 INFO:tasks.workunit.client.1.smithi043.stdout:7/90: fdatasync d4/db/d11/f13 0 2024-09-16T23:48:44.074 INFO:tasks.workunit.client.1.smithi043.stdout:7/91: fsync d4/db/d11/f13 0 2024-09-16T23:48:44.075 INFO:tasks.workunit.client.1.smithi043.stdout:5/146: dread d3/db/f2a [0,4194304] 0 2024-09-16T23:48:44.076 INFO:tasks.workunit.client.1.smithi043.stdout:4/57: truncate d8/fc 97256 0 2024-09-16T23:48:44.076 INFO:tasks.workunit.client.1.smithi043.stdout:7/92: truncate d4/db/d11/f13 761207 0 2024-09-16T23:48:44.077 INFO:tasks.workunit.client.1.smithi043.stdout:1/80: symlink d4/df/d13/l1a 0 2024-09-16T23:48:44.079 INFO:tasks.workunit.client.1.smithi043.stdout:0/100: truncate d0/f1c 910707 0 2024-09-16T23:48:44.081 INFO:tasks.workunit.client.1.smithi043.stdout:7/93: unlink f1 0 2024-09-16T23:48:44.083 INFO:tasks.workunit.client.1.smithi043.stdout:3/61: dwrite f1 [0,4194304] 0 2024-09-16T23:48:44.085 INFO:tasks.workunit.client.1.smithi043.stdout:4/58: dread f1 [0,4194304] 0 2024-09-16T23:48:44.086 INFO:tasks.workunit.client.1.smithi043.stdout:5/147: dread d3/db/d15/f12 [4194304,4194304] 0 2024-09-16T23:48:44.088 INFO:tasks.workunit.client.1.smithi043.stdout:5/148: write f2 [1710662,72806] 0 2024-09-16T23:48:44.088 INFO:tasks.workunit.client.1.smithi043.stdout:7/94: mknod d4/de/c1a 0 2024-09-16T23:48:44.089 INFO:tasks.workunit.client.1.smithi043.stdout:7/95: fdatasync d4/fa 0 2024-09-16T23:48:44.090 INFO:tasks.workunit.client.1.smithi043.stdout:0/101: creat d0/f24 x:0 0 0 2024-09-16T23:48:44.092 INFO:tasks.workunit.client.1.smithi043.stdout:4/59: creat d8/dd/de/f14 x:0 0 0 2024-09-16T23:48:44.093 INFO:tasks.workunit.client.1.smithi043.stdout:1/81: getdents d4 0 2024-09-16T23:48:44.093 INFO:tasks.workunit.client.1.smithi043.stdout:5/149: fsync d3/f17 0 2024-09-16T23:48:44.095 INFO:tasks.workunit.client.1.smithi043.stdout:3/62: creat d4/f11 x:0 0 0 2024-09-16T23:48:44.096 INFO:tasks.workunit.client.1.smithi043.stdout:3/63: rename d4 to d4/de/d12 22 2024-09-16T23:48:44.100 INFO:tasks.workunit.client.1.smithi043.stdout:1/82: mkdir d4/df/d13/d1b 0 2024-09-16T23:48:44.102 INFO:tasks.workunit.client.1.smithi043.stdout:7/96: mkdir d4/db/d17/d1b 0 2024-09-16T23:48:44.103 INFO:tasks.workunit.client.1.smithi043.stdout:5/150: mknod d3/db/c34 0 2024-09-16T23:48:44.104 INFO:tasks.workunit.client.1.smithi043.stdout:0/102: fdatasync d0/de/f15 0 2024-09-16T23:48:44.105 INFO:tasks.workunit.client.1.smithi043.stdout:3/64: dwrite d4/f11 [0,4194304] 0 2024-09-16T23:48:44.106 INFO:tasks.workunit.client.1.smithi043.stdout:1/83: mkdir d4/d9/d1c 0 2024-09-16T23:48:44.106 INFO:tasks.workunit.client.1.smithi043.stdout:5/151: truncate d3/f32 383367 0 2024-09-16T23:48:44.107 INFO:tasks.workunit.client.1.smithi043.stdout:4/60: write f7 [4053089,30056] 0 2024-09-16T23:48:44.110 INFO:tasks.workunit.client.1.smithi043.stdout:3/65: write d4/fd [545428,70599] 0 2024-09-16T23:48:44.111 INFO:tasks.workunit.client.1.smithi043.stdout:1/84: mkdir d4/df/d13/d1b/d1d 0 2024-09-16T23:48:44.113 INFO:tasks.workunit.client.1.smithi043.stdout:7/97: dwrite d4/fa [4194304,4194304] 0 2024-09-16T23:48:44.114 INFO:tasks.workunit.client.1.smithi043.stdout:5/152: rmdir d3/db/d15/d27 0 2024-09-16T23:48:44.114 INFO:tasks.workunit.client.1.smithi043.stdout:1/85: dread d4/d9/f11 [0,4194304] 0 2024-09-16T23:48:44.118 INFO:tasks.workunit.client.1.smithi043.stdout:4/61: dread f1 [0,4194304] 0 2024-09-16T23:48:44.118 INFO:tasks.workunit.client.1.smithi043.stdout:5/153: rename d3/db/d15/f12 to d3/d9/d23/f35 0 2024-09-16T23:48:44.119 INFO:tasks.workunit.client.1.smithi043.stdout:7/98: symlink d4/db/d11/d15/l1c 0 2024-09-16T23:48:44.120 INFO:tasks.workunit.client.1.smithi043.stdout:4/62: mkdir d8/d15 0 2024-09-16T23:48:44.121 INFO:tasks.workunit.client.1.smithi043.stdout:7/99: chown d4/db 20267 1 2024-09-16T23:48:44.121 INFO:tasks.workunit.client.1.smithi043.stdout:7/100: chown d4 4089303 1 2024-09-16T23:48:44.122 INFO:tasks.workunit.client.1.smithi043.stdout:1/86: rename d4/df/d13/d1b/d1d to d4/d9/d1c/d1e 0 2024-09-16T23:48:44.123 INFO:tasks.workunit.client.1.smithi043.stdout:5/154: dread - d3/d9/d23/f2d zero size 2024-09-16T23:48:44.123 INFO:tasks.workunit.client.1.smithi043.stdout:7/101: chown d4/db 3 1 2024-09-16T23:48:44.126 INFO:tasks.workunit.client.1.smithi043.stdout:3/66: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:44.128 INFO:tasks.workunit.client.1.smithi043.stdout:7/102: truncate d4/f19 620264 0 2024-09-16T23:48:44.130 INFO:tasks.workunit.client.1.smithi043.stdout:7/103: fsync d4/f9 0 2024-09-16T23:48:44.136 INFO:tasks.workunit.client.1.smithi043.stdout:3/67: mknod d4/de/c13 0 2024-09-16T23:48:44.138 INFO:tasks.workunit.client.1.smithi043.stdout:1/87: dwrite d4/fc [4194304,4194304] 0 2024-09-16T23:48:44.138 INFO:tasks.workunit.client.1.smithi043.stdout:3/68: chown d4/fb 12219 1 2024-09-16T23:48:44.138 INFO:tasks.workunit.client.1.smithi043.stdout:5/155: truncate d3/d9/d23/f35 7998996 0 2024-09-16T23:48:44.139 INFO:tasks.workunit.client.1.smithi043.stdout:4/63: dwrite d8/dd/de/f14 [0,4194304] 0 2024-09-16T23:48:44.142 INFO:tasks.workunit.client.1.smithi043.stdout:1/88: write d4/fc [9088966,124226] 0 2024-09-16T23:48:44.144 INFO:tasks.workunit.client.1.smithi043.stdout:4/64: symlink d8/l16 0 2024-09-16T23:48:44.144 INFO:tasks.workunit.client.1.smithi043.stdout:5/156: truncate d3/db/d26/f33 2230284 0 2024-09-16T23:48:44.145 INFO:tasks.workunit.client.1.smithi043.stdout:7/104: dread d4/fa [0,4194304] 0 2024-09-16T23:48:44.146 INFO:tasks.workunit.client.1.smithi043.stdout:4/65: read - d8/dd/f10 zero size 2024-09-16T23:48:44.146 INFO:tasks.workunit.client.1.smithi043.stdout:5/157: read f2 [189895,116430] 0 2024-09-16T23:48:44.146 INFO:tasks.workunit.client.1.smithi043.stdout:4/66: stat c6 0 2024-09-16T23:48:44.148 INFO:tasks.workunit.client.1.smithi043.stdout:1/89: truncate d4/d9/f11 1326710 0 2024-09-16T23:48:44.148 INFO:tasks.workunit.client.1.smithi043.stdout:5/158: stat d3/d9/c2b 0 2024-09-16T23:48:44.149 INFO:tasks.workunit.client.1.smithi043.stdout:3/69: rename d4/lf to d4/l14 0 2024-09-16T23:48:44.152 INFO:tasks.workunit.client.1.smithi043.stdout:3/70: write d4/f10 [958121,43638] 0 2024-09-16T23:48:44.153 INFO:tasks.workunit.client.1.smithi043.stdout:1/90: creat d4/d9/d12/f1f x:0 0 0 2024-09-16T23:48:44.158 INFO:tasks.workunit.client.1.smithi043.stdout:3/71: unlink f1 0 2024-09-16T23:48:44.160 INFO:tasks.workunit.client.1.smithi043.stdout:4/67: dread f7 [0,4194304] 0 2024-09-16T23:48:44.162 INFO:tasks.workunit.client.1.smithi043.stdout:5/159: dwrite d3/f32 [0,4194304] 0 2024-09-16T23:48:44.163 INFO:tasks.workunit.client.1.smithi043.stdout:4/68: fsync f7 0 2024-09-16T23:48:44.164 INFO:tasks.workunit.client.1.smithi043.stdout:7/105: dwrite d4/fa [0,4194304] 0 2024-09-16T23:48:44.167 INFO:tasks.workunit.client.1.smithi043.stdout:3/72: getdents d4/de 0 2024-09-16T23:48:44.168 INFO:tasks.workunit.client.1.smithi043.stdout:4/69: dread - d8/dd/de/f11 zero size 2024-09-16T23:48:44.169 INFO:tasks.workunit.client.1.smithi043.stdout:7/106: symlink d4/db/d17/l1d 0 2024-09-16T23:48:44.170 INFO:tasks.workunit.client.1.smithi043.stdout:3/73: read d4/fb [1932617,117775] 0 2024-09-16T23:48:44.173 INFO:tasks.workunit.client.1.smithi043.stdout:4/70: rename d8/dd to d8/d15/d17 0 2024-09-16T23:48:44.173 INFO:tasks.workunit.client.1.smithi043.stdout:5/160: dread f2 [0,4194304] 0 2024-09-16T23:48:44.173 INFO:tasks.workunit.client.1.smithi043.stdout:1/91: dwrite f1 [0,4194304] 0 2024-09-16T23:48:44.174 INFO:tasks.workunit.client.1.smithi043.stdout:1/92: stat d4/df/d13/d1b 0 2024-09-16T23:48:44.175 INFO:tasks.workunit.client.1.smithi043.stdout:3/74: creat d4/de/f15 x:0 0 0 2024-09-16T23:48:44.178 INFO:tasks.workunit.client.1.smithi043.stdout:5/161: fsync f2 0 2024-09-16T23:48:44.180 INFO:tasks.workunit.client.1.smithi043.stdout:4/71: unlink d8/lb 0 2024-09-16T23:48:44.182 INFO:tasks.workunit.client.1.smithi043.stdout:7/107: symlink d4/db/d17/d1b/l1e 0 2024-09-16T23:48:44.184 INFO:tasks.workunit.client.1.smithi043.stdout:1/93: symlink d4/d9/d1c/d1e/l20 0 2024-09-16T23:48:44.186 INFO:tasks.workunit.client.1.smithi043.stdout:3/75: creat d4/f16 x:0 0 0 2024-09-16T23:48:44.187 INFO:tasks.workunit.client.1.smithi043.stdout:7/108: write d4/db/f14 [1147204,27729] 0 2024-09-16T23:48:44.192 INFO:tasks.workunit.client.1.smithi043.stdout:3/76: mkdir d4/d17 0 2024-09-16T23:48:44.192 INFO:tasks.workunit.client.1.smithi043.stdout:5/162: dwrite d3/d9/d23/f2d [0,4194304] 0 2024-09-16T23:48:44.193 INFO:tasks.workunit.client.1.smithi043.stdout:7/109: creat d4/de/f1f x:0 0 0 2024-09-16T23:48:44.197 INFO:tasks.workunit.client.1.smithi043.stdout:1/94: truncate d4/d9/f11 613353 0 2024-09-16T23:48:44.198 INFO:tasks.workunit.client.1.smithi043.stdout:4/72: dwrite f7 [0,4194304] 0 2024-09-16T23:48:44.199 INFO:tasks.workunit.client.1.smithi043.stdout:1/95: readlink d4/df/d13/l1a 0 2024-09-16T23:48:44.199 INFO:tasks.workunit.client.1.smithi043.stdout:4/73: chown d8/d15/d17/f10 51388009 1 2024-09-16T23:48:44.200 INFO:tasks.workunit.client.1.smithi043.stdout:1/96: write d4/d9/f17 [1620893,58754] 0 2024-09-16T23:48:44.200 INFO:tasks.workunit.client.1.smithi043.stdout:3/77: rename c2 to d4/d17/c18 0 2024-09-16T23:48:44.201 INFO:tasks.workunit.client.1.smithi043.stdout:3/78: chown d4 206667 1 2024-09-16T23:48:44.202 INFO:tasks.workunit.client.1.smithi043.stdout:5/163: creat d3/d1b/f36 x:0 0 0 2024-09-16T23:48:44.203 INFO:tasks.workunit.client.1.smithi043.stdout:7/110: mknod d4/db/d17/d1b/c20 0 2024-09-16T23:48:44.203 INFO:tasks.workunit.client.1.smithi043.stdout:1/97: stat d4/l8 0 2024-09-16T23:48:44.204 INFO:tasks.workunit.client.1.smithi043.stdout:7/111: fsync d4/db/f18 0 2024-09-16T23:48:44.205 INFO:tasks.workunit.client.1.smithi043.stdout:3/79: fdatasync d4/f11 0 2024-09-16T23:48:44.205 INFO:tasks.workunit.client.1.smithi043.stdout:4/74: write f7 [2086550,122812] 0 2024-09-16T23:48:44.207 INFO:tasks.workunit.client.1.smithi043.stdout:3/80: chown d4/f11 33300837 1 2024-09-16T23:48:44.207 INFO:tasks.workunit.client.1.smithi043.stdout:5/164: creat d3/d9/d23/d28/f37 x:0 0 0 2024-09-16T23:48:44.208 INFO:tasks.workunit.client.1.smithi043.stdout:7/112: rmdir d4/db/d11 39 2024-09-16T23:48:44.209 INFO:tasks.workunit.client.1.smithi043.stdout:3/81: write d4/f10 [968368,43559] 0 2024-09-16T23:48:44.210 INFO:tasks.workunit.client.1.smithi043.stdout:7/113: chown d4/db/c10 201317 1 2024-09-16T23:48:44.211 INFO:tasks.workunit.client.1.smithi043.stdout:5/165: rmdir d3/db 39 2024-09-16T23:48:44.211 INFO:tasks.workunit.client.1.smithi043.stdout:7/114: write d4/de/f1f [290952,121058] 0 2024-09-16T23:48:44.213 INFO:tasks.workunit.client.1.smithi043.stdout:1/98: mkdir d4/df/d13/d1b/d21 0 2024-09-16T23:48:44.218 INFO:tasks.workunit.client.1.smithi043.stdout:1/99: rmdir d4/d9/d12 39 2024-09-16T23:48:44.219 INFO:tasks.workunit.client.1.smithi043.stdout:1/100: read f1 [5621229,93163] 0 2024-09-16T23:48:44.220 INFO:tasks.workunit.client.1.smithi043.stdout:3/82: dwrite d4/de/f15 [0,4194304] 0 2024-09-16T23:48:44.220 INFO:tasks.workunit.client.1.smithi043.stdout:7/115: mkdir d4/db/d11/d21 0 2024-09-16T23:48:44.224 INFO:tasks.workunit.client.1.smithi043.stdout:4/75: truncate f1 768618 0 2024-09-16T23:48:44.226 INFO:tasks.workunit.client.1.smithi043.stdout:3/83: write d4/fb [3682405,108772] 0 2024-09-16T23:48:44.227 INFO:tasks.workunit.client.1.smithi043.stdout:4/76: mknod d8/d15/d17/de/c18 0 2024-09-16T23:48:44.227 INFO:tasks.workunit.client.1.smithi043.stdout:1/101: truncate d4/d9/d12/f1f 108565 0 2024-09-16T23:48:44.230 INFO:tasks.workunit.client.1.smithi043.stdout:3/84: stat d4/d17/c18 0 2024-09-16T23:48:44.233 INFO:tasks.workunit.client.1.smithi043.stdout:4/77: getdents d8/d15 0 2024-09-16T23:48:44.233 INFO:tasks.workunit.client.1.smithi043.stdout:7/116: dwrite d4/f9 [4194304,4194304] 0 2024-09-16T23:48:44.236 INFO:tasks.workunit.client.1.smithi043.stdout:7/117: write d4/db/f18 [532994,28718] 0 2024-09-16T23:48:44.238 INFO:tasks.workunit.client.1.smithi043.stdout:3/85: link d4/f10 d4/f19 0 2024-09-16T23:48:44.238 INFO:tasks.workunit.client.1.smithi043.stdout:7/118: mkdir d4/de/d22 0 2024-09-16T23:48:44.241 INFO:tasks.workunit.client.1.smithi043.stdout:1/102: dwrite d4/fc [4194304,4194304] 0 2024-09-16T23:48:44.245 INFO:tasks.workunit.client.1.smithi043.stdout:3/86: mknod d4/c1a 0 2024-09-16T23:48:44.249 INFO:tasks.workunit.client.1.smithi043.stdout:1/103: symlink d4/d9/d12/d15/l22 0 2024-09-16T23:48:44.252 INFO:tasks.workunit.client.1.smithi043.stdout:4/78: dwrite d8/d15/d17/de/f13 [0,4194304] 0 2024-09-16T23:48:44.256 INFO:tasks.workunit.client.1.smithi043.stdout:3/87: write d4/f10 [102778,49165] 0 2024-09-16T23:48:44.258 INFO:tasks.workunit.client.1.smithi043.stdout:1/104: creat d4/d9/d1c/d1e/f23 x:0 0 0 2024-09-16T23:48:44.259 INFO:tasks.workunit.client.1.smithi043.stdout:3/88: stat d4/fd 0 2024-09-16T23:48:44.260 INFO:tasks.workunit.client.1.smithi043.stdout:7/119: dwrite d4/db/f14 [0,4194304] 0 2024-09-16T23:48:44.261 INFO:tasks.workunit.client.1.smithi043.stdout:4/79: chown d8/fc 0 1 2024-09-16T23:48:44.261 INFO:tasks.workunit.client.1.smithi043.stdout:3/89: stat d4/f16 0 2024-09-16T23:48:44.261 INFO:tasks.workunit.client.1.smithi043.stdout:6/67: sync 2024-09-16T23:48:44.262 INFO:tasks.workunit.client.1.smithi043.stdout:2/28: sync 2024-09-16T23:48:44.262 INFO:tasks.workunit.client.1.smithi043.stdout:8/51: sync 2024-09-16T23:48:44.263 INFO:tasks.workunit.client.1.smithi043.stdout:4/80: stat d8/c12 0 2024-09-16T23:48:44.264 INFO:tasks.workunit.client.1.smithi043.stdout:1/105: chown d4/d9/d1c/d1e/l20 1340662 1 2024-09-16T23:48:44.264 INFO:tasks.workunit.client.1.smithi043.stdout:3/90: rmdir d4/de 39 2024-09-16T23:48:44.264 INFO:tasks.workunit.client.1.smithi043.stdout:2/29: truncate f5 632755 0 2024-09-16T23:48:44.265 INFO:tasks.workunit.client.1.smithi043.stdout:6/68: creat de/df/f17 x:0 0 0 2024-09-16T23:48:44.268 INFO:tasks.workunit.client.1.smithi043.stdout:3/91: readlink d4/lc 0 2024-09-16T23:48:44.270 INFO:tasks.workunit.client.1.smithi043.stdout:7/120: link d4/fa d4/db/d11/d21/f23 0 2024-09-16T23:48:44.271 INFO:tasks.workunit.client.1.smithi043.stdout:2/30: creat f7 x:0 0 0 2024-09-16T23:48:44.272 INFO:tasks.workunit.client.1.smithi043.stdout:8/52: creat de/ff x:0 0 0 2024-09-16T23:48:44.273 INFO:tasks.workunit.client.1.smithi043.stdout:1/106: rename d4/ld to d4/d9/d1c/l24 0 2024-09-16T23:48:44.274 INFO:tasks.workunit.client.1.smithi043.stdout:4/81: write d8/fc [1064832,48287] 0 2024-09-16T23:48:44.274 INFO:tasks.workunit.client.1.smithi043.stdout:6/69: rename de to de/d14/d18 22 2024-09-16T23:48:44.275 INFO:tasks.workunit.client.1.smithi043.stdout:3/92: chown d4/de/c13 2727496 1 2024-09-16T23:48:44.275 INFO:tasks.workunit.client.1.smithi043.stdout:2/31: mkdir d8 0 2024-09-16T23:48:44.277 INFO:tasks.workunit.client.1.smithi043.stdout:7/121: dread d4/f9 [4194304,4194304] 0 2024-09-16T23:48:44.279 INFO:tasks.workunit.client.1.smithi043.stdout:2/32: truncate f6 493520 0 2024-09-16T23:48:44.280 INFO:tasks.workunit.client.1.smithi043.stdout:4/82: truncate d8/d15/d17/f10 412930 0 2024-09-16T23:48:44.281 INFO:tasks.workunit.client.1.smithi043.stdout:2/33: truncate f7 658138 0 2024-09-16T23:48:44.282 INFO:tasks.workunit.client.1.smithi043.stdout:8/53: getdents de 0 2024-09-16T23:48:44.282 INFO:tasks.workunit.client.1.smithi043.stdout:4/83: chown d8/d15/d17/de/f11 11453 1 2024-09-16T23:48:44.283 INFO:tasks.workunit.client.1.smithi043.stdout:4/84: readlink l3 0 2024-09-16T23:48:44.284 INFO:tasks.workunit.client.1.smithi043.stdout:8/54: write de/ff [921655,47979] 0 2024-09-16T23:48:44.286 INFO:tasks.workunit.client.1.smithi043.stdout:3/93: dread d4/de/f15 [0,4194304] 0 2024-09-16T23:48:44.288 INFO:tasks.workunit.client.1.smithi043.stdout:4/85: chown d8/fc 168 1 2024-09-16T23:48:44.289 INFO:tasks.workunit.client.1.smithi043.stdout:7/122: truncate d4/db/f14 2030001 0 2024-09-16T23:48:44.291 INFO:tasks.workunit.client.1.smithi043.stdout:2/34: truncate f7 104221 0 2024-09-16T23:48:44.291 INFO:tasks.workunit.client.1.smithi043.stdout:7/123: fdatasync d4/f19 0 2024-09-16T23:48:44.292 INFO:tasks.workunit.client.1.smithi043.stdout:3/94: unlink d4/f6 0 2024-09-16T23:48:44.295 INFO:tasks.workunit.client.1.smithi043.stdout:8/55: write fc [645568,89454] 0 2024-09-16T23:48:44.295 INFO:tasks.workunit.client.1.smithi043.stdout:6/70: dwrite fc [0,4194304] 0 2024-09-16T23:48:44.297 INFO:tasks.workunit.client.1.smithi043.stdout:4/86: rename d8/d15/d17/de to d8/d15/d19 0 2024-09-16T23:48:44.299 INFO:tasks.workunit.client.1.smithi043.stdout:8/56: rmdir de 39 2024-09-16T23:48:44.300 INFO:tasks.workunit.client.1.smithi043.stdout:8/57: readlink l6 0 2024-09-16T23:48:44.300 INFO:tasks.workunit.client.1.smithi043.stdout:6/71: stat de/df/l12 0 2024-09-16T23:48:44.303 INFO:tasks.workunit.client.1.smithi043.stdout:4/87: rename d8/d15/d19/f13 to d8/f1a 0 2024-09-16T23:48:44.304 INFO:tasks.workunit.client.1.smithi043.stdout:4/88: fdatasync f7 0 2024-09-16T23:48:44.305 INFO:tasks.workunit.client.1.smithi043.stdout:2/35: mknod d8/c9 0 2024-09-16T23:48:44.306 INFO:tasks.workunit.client.1.smithi043.stdout:6/72: creat de/f19 x:0 0 0 2024-09-16T23:48:44.307 INFO:tasks.workunit.client.1.smithi043.stdout:1/107: link d4/fc d4/df/d13/d1b/f25 0 2024-09-16T23:48:44.309 INFO:tasks.workunit.client.1.smithi043.stdout:4/89: chown f1 3250 1 2024-09-16T23:48:44.310 INFO:tasks.workunit.client.1.smithi043.stdout:7/124: creat d4/db/f24 x:0 0 0 2024-09-16T23:48:44.312 INFO:tasks.workunit.client.1.smithi043.stdout:8/58: dwrite f8 [0,4194304] 0 2024-09-16T23:48:44.312 INFO:tasks.workunit.client.1.smithi043.stdout:3/95: creat d4/f1b x:0 0 0 2024-09-16T23:48:44.313 INFO:tasks.workunit.client.1.smithi043.stdout:6/73: mkdir de/d1a 0 2024-09-16T23:48:44.316 INFO:tasks.workunit.client.1.smithi043.stdout:7/125: creat d4/db/d17/d1b/f25 x:0 0 0 2024-09-16T23:48:44.317 INFO:tasks.workunit.client.1.smithi043.stdout:1/108: unlink d4/d9/d1c/d1e/l20 0 2024-09-16T23:48:44.318 INFO:tasks.workunit.client.1.smithi043.stdout:4/90: dread d8/f1a [0,4194304] 0 2024-09-16T23:48:44.319 INFO:tasks.workunit.client.1.smithi043.stdout:3/96: mknod d4/de/c1c 0 2024-09-16T23:48:44.320 INFO:tasks.workunit.client.1.smithi043.stdout:1/109: read d4/d9/f17 [2109760,99375] 0 2024-09-16T23:48:44.320 INFO:tasks.workunit.client.1.smithi043.stdout:4/91: chown d8/c12 858 1 2024-09-16T23:48:44.320 INFO:tasks.workunit.client.1.smithi043.stdout:3/97: dread - d4/f1b zero size 2024-09-16T23:48:44.321 INFO:tasks.workunit.client.1.smithi043.stdout:1/110: stat d4/df 0 2024-09-16T23:48:44.321 INFO:tasks.workunit.client.1.smithi043.stdout:6/74: dread f1 [0,4194304] 0 2024-09-16T23:48:44.321 INFO:tasks.workunit.client.1.smithi043.stdout:8/59: mkdir de/d10 0 2024-09-16T23:48:44.322 INFO:tasks.workunit.client.1.smithi043.stdout:8/60: chown l6 10910 1 2024-09-16T23:48:44.322 INFO:tasks.workunit.client.1.smithi043.stdout:4/92: write d8/d15/d19/f14 [3548472,81477] 0 2024-09-16T23:48:44.324 INFO:tasks.workunit.client.1.smithi043.stdout:1/111: write d4/d9/d12/f1f [479458,91593] 0 2024-09-16T23:48:44.325 INFO:tasks.workunit.client.1.smithi043.stdout:3/98: truncate d4/f10 1194315 0 2024-09-16T23:48:44.326 INFO:tasks.workunit.client.1.smithi043.stdout:7/126: rename d4/db/d11/d21/f23 to d4/db/d17/f26 0 2024-09-16T23:48:44.328 INFO:tasks.workunit.client.1.smithi043.stdout:8/61: unlink fd 0 2024-09-16T23:48:44.329 INFO:tasks.workunit.client.1.smithi043.stdout:6/75: creat de/df/f1b x:0 0 0 2024-09-16T23:48:44.330 INFO:tasks.workunit.client.1.smithi043.stdout:8/62: fsync f8 0 2024-09-16T23:48:44.332 INFO:tasks.workunit.client.1.smithi043.stdout:8/63: stat l7 0 2024-09-16T23:48:44.335 INFO:tasks.workunit.client.1.smithi043.stdout:4/93: dread f7 [0,4194304] 0 2024-09-16T23:48:44.338 INFO:tasks.workunit.client.1.smithi043.stdout:4/94: readlink l2 0 2024-09-16T23:48:44.338 INFO:tasks.workunit.client.1.smithi043.stdout:3/99: dread d4/f11 [0,4194304] 0 2024-09-16T23:48:44.339 INFO:tasks.workunit.client.1.smithi043.stdout:6/76: creat de/d1a/f1c x:0 0 0 2024-09-16T23:48:44.340 INFO:tasks.workunit.client.1.smithi043.stdout:1/112: truncate d4/d9/f11 976634 0 2024-09-16T23:48:44.341 INFO:tasks.workunit.client.1.smithi043.stdout:8/64: mknod de/d10/c11 0 2024-09-16T23:48:44.344 INFO:tasks.workunit.client.1.smithi043.stdout:3/100: stat d4/l14 0 2024-09-16T23:48:44.344 INFO:tasks.workunit.client.1.smithi043.stdout:8/65: readlink l2 0 2024-09-16T23:48:44.344 INFO:tasks.workunit.client.1.smithi043.stdout:7/127: dwrite d4/db/d11/f13 [0,4194304] 0 2024-09-16T23:48:44.344 INFO:tasks.workunit.client.1.smithi043.stdout:4/95: truncate d8/d15/d19/f11 614783 0 2024-09-16T23:48:44.344 INFO:tasks.workunit.client.1.smithi043.stdout:8/66: readlink l2 0 2024-09-16T23:48:44.345 INFO:tasks.workunit.client.1.smithi043.stdout:6/77: rmdir de/d14 0 2024-09-16T23:48:44.347 INFO:tasks.workunit.client.1.smithi043.stdout:3/101: creat d4/d17/f1d x:0 0 0 2024-09-16T23:48:44.347 INFO:tasks.workunit.client.1.smithi043.stdout:7/128: fsync d4/fa 0 2024-09-16T23:48:44.348 INFO:tasks.workunit.client.1.smithi043.stdout:1/113: truncate d4/d9/d12/f1f 625638 0 2024-09-16T23:48:44.348 INFO:tasks.workunit.client.1.smithi043.stdout:6/78: rmdir de/d1a 39 2024-09-16T23:48:44.349 INFO:tasks.workunit.client.1.smithi043.stdout:3/102: fsync d4/f1b 0 2024-09-16T23:48:44.351 INFO:tasks.workunit.client.1.smithi043.stdout:7/129: fdatasync d4/db/d17/d1b/f25 0 2024-09-16T23:48:44.354 INFO:tasks.workunit.client.1.smithi043.stdout:4/96: dwrite f7 [0,4194304] 0 2024-09-16T23:48:44.356 INFO:tasks.workunit.client.1.smithi043.stdout:1/114: mkdir d4/df/d26 0 2024-09-16T23:48:44.357 INFO:tasks.workunit.client.1.smithi043.stdout:3/103: symlink d4/d17/l1e 0 2024-09-16T23:48:44.358 INFO:tasks.workunit.client.1.smithi043.stdout:3/104: stat d4/l14 0 2024-09-16T23:48:44.359 INFO:tasks.workunit.client.1.smithi043.stdout:6/79: mknod de/d1a/c1d 0 2024-09-16T23:48:44.359 INFO:tasks.workunit.client.1.smithi043.stdout:4/97: rmdir d8/d15 39 2024-09-16T23:48:44.359 INFO:tasks.workunit.client.1.smithi043.stdout:8/67: link l6 de/l12 0 2024-09-16T23:48:44.361 INFO:tasks.workunit.client.1.smithi043.stdout:7/130: mkdir d4/d27 0 2024-09-16T23:48:44.363 INFO:tasks.workunit.client.1.smithi043.stdout:7/131: write d4/f19 [1583750,7049] 0 2024-09-16T23:48:44.364 INFO:tasks.workunit.client.1.smithi043.stdout:8/68: creat de/d10/f13 x:0 0 0 2024-09-16T23:48:44.365 INFO:tasks.workunit.client.1.smithi043.stdout:4/98: rmdir d8/d15 39 2024-09-16T23:48:44.365 INFO:tasks.workunit.client.1.smithi043.stdout:1/115: dread f1 [4194304,4194304] 0 2024-09-16T23:48:44.367 INFO:tasks.workunit.client.1.smithi043.stdout:7/132: dread - d4/fc zero size 2024-09-16T23:48:44.368 INFO:tasks.workunit.client.1.smithi043.stdout:8/69: creat de/f14 x:0 0 0 2024-09-16T23:48:44.370 INFO:tasks.workunit.client.1.smithi043.stdout:4/99: rename d8 to d8/d15/d1b 22 2024-09-16T23:48:44.373 INFO:tasks.workunit.client.1.smithi043.stdout:4/100: readlink d8/la 0 2024-09-16T23:48:44.373 INFO:tasks.workunit.client.1.smithi043.stdout:6/80: write f3 [76171,130086] 0 2024-09-16T23:48:44.374 INFO:tasks.workunit.client.1.smithi043.stdout:1/116: write d4/df/d13/d1b/f25 [493083,77603] 0 2024-09-16T23:48:44.376 INFO:tasks.workunit.client.1.smithi043.stdout:6/81: dread - de/d1a/f1c zero size 2024-09-16T23:48:44.381 INFO:tasks.workunit.client.1.smithi043.stdout:7/133: dwrite d4/f19 [0,4194304] 0 2024-09-16T23:48:44.383 INFO:tasks.workunit.client.1.smithi043.stdout:8/70: dread f8 [0,4194304] 0 2024-09-16T23:48:44.386 INFO:tasks.workunit.client.1.smithi043.stdout:1/117: rename d4/d9/f11 to d4/d9/d12/f27 0 2024-09-16T23:48:44.389 INFO:tasks.workunit.client.1.smithi043.stdout:8/71: write f8 [3674424,97418] 0 2024-09-16T23:48:44.392 INFO:tasks.workunit.client.1.smithi043.stdout:1/118: readlink d4/d9/d12/d15/l22 0 2024-09-16T23:48:44.392 INFO:tasks.workunit.client.1.smithi043.stdout:7/134: truncate d4/fa 8261692 0 2024-09-16T23:48:44.396 INFO:tasks.workunit.client.1.smithi043.stdout:4/101: dwrite d8/d15/d19/f11 [0,4194304] 0 2024-09-16T23:48:44.398 INFO:tasks.workunit.client.1.smithi043.stdout:6/82: dwrite de/f13 [0,4194304] 0 2024-09-16T23:48:44.398 INFO:tasks.workunit.client.1.smithi043.stdout:6/83: write f3 [281667,63280] 0 2024-09-16T23:48:44.404 INFO:tasks.workunit.client.1.smithi043.stdout:1/119: mkdir d4/d9/d1c/d1e/d28 0 2024-09-16T23:48:44.407 INFO:tasks.workunit.client.1.smithi043.stdout:7/135: mkdir d4/db/d28 0 2024-09-16T23:48:44.408 INFO:tasks.workunit.client.1.smithi043.stdout:8/72: getdents de 0 2024-09-16T23:48:44.415 INFO:tasks.workunit.client.1.smithi043.stdout:6/84: dwrite de/df/f1b [0,4194304] 0 2024-09-16T23:48:44.419 INFO:tasks.workunit.client.1.smithi043.stdout:7/136: mkdir d4/db/d11/d15/d29 0 2024-09-16T23:48:44.420 INFO:tasks.workunit.client.1.smithi043.stdout:8/73: dread fc [0,4194304] 0 2024-09-16T23:48:44.420 INFO:tasks.workunit.client.1.smithi043.stdout:6/85: rmdir de/df 39 2024-09-16T23:48:44.424 INFO:tasks.workunit.client.1.smithi043.stdout:6/86: stat de 0 2024-09-16T23:48:44.425 INFO:tasks.workunit.client.1.smithi043.stdout:4/102: truncate d8/d15/d19/f14 140487 0 2024-09-16T23:48:44.425 INFO:tasks.workunit.client.1.smithi043.stdout:8/74: write de/f14 [859100,43083] 0 2024-09-16T23:48:44.426 INFO:tasks.workunit.client.1.smithi043.stdout:7/137: mknod d4/db/d11/c2a 0 2024-09-16T23:48:44.427 INFO:tasks.workunit.client.1.smithi043.stdout:1/120: dwrite d4/d9/d1c/d1e/f23 [0,4194304] 0 2024-09-16T23:48:44.427 INFO:tasks.workunit.client.1.smithi043.stdout:4/103: symlink d8/d15/l1c 0 2024-09-16T23:48:44.427 INFO:tasks.workunit.client.1.smithi043.stdout:9/101: sync 2024-09-16T23:48:44.429 INFO:tasks.workunit.client.1.smithi043.stdout:8/75: creat de/f15 x:0 0 0 2024-09-16T23:48:44.431 INFO:tasks.workunit.client.1.smithi043.stdout:7/138: write d4/db/d17/d1b/f25 [962660,33275] 0 2024-09-16T23:48:44.433 INFO:tasks.workunit.client.1.smithi043.stdout:9/102: write d4/f1a [524450,7485] 0 2024-09-16T23:48:44.434 INFO:tasks.workunit.client.1.smithi043.stdout:7/139: readlink d4/db/d17/l1d 0 2024-09-16T23:48:44.436 INFO:tasks.workunit.client.1.smithi043.stdout:8/76: chown de/l12 20813799 1 2024-09-16T23:48:44.438 INFO:tasks.workunit.client.1.smithi043.stdout:8/77: creat de/f16 x:0 0 0 2024-09-16T23:48:44.439 INFO:tasks.workunit.client.1.smithi043.stdout:9/103: chown d4/l17 107729 1 2024-09-16T23:48:44.439 INFO:tasks.workunit.client.1.smithi043.stdout:8/78: readlink l2 0 2024-09-16T23:48:44.442 INFO:tasks.workunit.client.1.smithi043.stdout:1/121: dwrite d4/fc [4194304,4194304] 0 2024-09-16T23:48:44.443 INFO:tasks.workunit.client.1.smithi043.stdout:1/122: chown d4/df/d13/d1b/d21 27223 1 2024-09-16T23:48:44.444 INFO:tasks.workunit.client.1.smithi043.stdout:7/140: rmdir d4/de/d22 0 2024-09-16T23:48:44.446 INFO:tasks.workunit.client.1.smithi043.stdout:9/104: unlink d4/d5/l13 0 2024-09-16T23:48:44.446 INFO:tasks.workunit.client.1.smithi043.stdout:4/104: rename d8/d15/d19/f14 to d8/f1d 0 2024-09-16T23:48:44.451 INFO:tasks.workunit.client.1.smithi043.stdout:1/123: fsync d4/d9/d12/f27 0 2024-09-16T23:48:44.452 INFO:tasks.workunit.client.1.smithi043.stdout:4/105: dread d8/f1a [0,4194304] 0 2024-09-16T23:48:44.456 INFO:tasks.workunit.client.1.smithi043.stdout:4/106: fsync f1 0 2024-09-16T23:48:44.456 INFO:tasks.workunit.client.1.smithi043.stdout:8/79: getdents de 0 2024-09-16T23:48:44.457 INFO:tasks.workunit.client.1.smithi043.stdout:8/80: stat de/d10/f13 0 2024-09-16T23:48:44.459 INFO:tasks.workunit.client.1.smithi043.stdout:4/107: truncate d8/d15/d19/f11 5037792 0 2024-09-16T23:48:44.460 INFO:tasks.workunit.client.1.smithi043.stdout:8/81: rename de/f16 to de/f17 0 2024-09-16T23:48:44.461 INFO:tasks.workunit.client.1.smithi043.stdout:7/141: write d4/fa [2391355,87504] 0 2024-09-16T23:48:44.461 INFO:tasks.workunit.client.1.smithi043.stdout:9/105: truncate d4/f11 2781504 0 2024-09-16T23:48:44.462 INFO:tasks.workunit.client.1.smithi043.stdout:8/82: fsync f8 0 2024-09-16T23:48:44.462 INFO:tasks.workunit.client.1.smithi043.stdout:4/108: symlink d8/d15/d19/l1e 0 2024-09-16T23:48:44.465 INFO:tasks.workunit.client.1.smithi043.stdout:4/109: creat d8/d15/f1f x:0 0 0 2024-09-16T23:48:44.467 INFO:tasks.workunit.client.1.smithi043.stdout:9/106: rename d4/d5/le to d4/d5/d16/l1e 0 2024-09-16T23:48:44.467 INFO:tasks.workunit.client.1.smithi043.stdout:4/110: chown d8/d15/d19/f11 10 1 2024-09-16T23:48:44.468 INFO:tasks.workunit.client.1.smithi043.stdout:8/83: getdents de/d10 0 2024-09-16T23:48:44.469 INFO:tasks.workunit.client.1.smithi043.stdout:4/111: chown d8/c12 8055409 1 2024-09-16T23:48:44.469 INFO:tasks.workunit.client.1.smithi043.stdout:8/84: readlink l9 0 2024-09-16T23:48:44.470 INFO:tasks.workunit.client.1.smithi043.stdout:1/124: dwrite d4/d9/f17 [0,4194304] 0 2024-09-16T23:48:44.470 INFO:tasks.workunit.client.1.smithi043.stdout:4/112: chown l2 1274 1 2024-09-16T23:48:44.470 INFO:tasks.workunit.client.1.smithi043.stdout:9/107: creat d4/d5/d16/f1f x:0 0 0 2024-09-16T23:48:44.471 INFO:tasks.workunit.client.1.smithi043.stdout:8/85: mkdir de/d18 0 2024-09-16T23:48:44.473 INFO:tasks.workunit.client.1.smithi043.stdout:4/113: rmdir d8/d15/d17 39 2024-09-16T23:48:44.474 INFO:tasks.workunit.client.1.smithi043.stdout:8/86: write de/f14 [503219,47979] 0 2024-09-16T23:48:44.475 INFO:tasks.workunit.client.1.smithi043.stdout:1/125: symlink d4/d9/d1c/d1e/l29 0 2024-09-16T23:48:44.475 INFO:tasks.workunit.client.1.smithi043.stdout:8/87: chown l2 0 1 2024-09-16T23:48:44.476 INFO:tasks.workunit.client.1.smithi043.stdout:7/142: truncate d4/f19 2453036 0 2024-09-16T23:48:44.478 INFO:tasks.workunit.client.1.smithi043.stdout:7/143: fdatasync d4/db/d17/f26 0 2024-09-16T23:48:44.478 INFO:tasks.workunit.client.1.smithi043.stdout:4/114: mkdir d8/d15/d20 0 2024-09-16T23:48:44.480 INFO:tasks.workunit.client.1.smithi043.stdout:4/115: write d8/d15/f1f [907312,62682] 0 2024-09-16T23:48:44.482 INFO:tasks.workunit.client.1.smithi043.stdout:1/126: fsync d4/d9/f17 0 2024-09-16T23:48:44.483 INFO:tasks.workunit.client.1.smithi043.stdout:8/88: dwrite f8 [4194304,4194304] 0 2024-09-16T23:48:44.483 INFO:tasks.workunit.client.1.smithi043.stdout:7/144: creat d4/f2b x:0 0 0 2024-09-16T23:48:44.484 INFO:tasks.workunit.client.1.smithi043.stdout:8/89: read - de/f15 zero size 2024-09-16T23:48:44.486 INFO:tasks.workunit.client.1.smithi043.stdout:1/127: creat d4/d9/d1c/d1e/f2a x:0 0 0 2024-09-16T23:48:44.486 INFO:tasks.workunit.client.1.smithi043.stdout:4/116: dread d8/d15/d17/f10 [0,4194304] 0 2024-09-16T23:48:44.488 INFO:tasks.workunit.client.1.smithi043.stdout:8/90: write de/f17 [67808,73817] 0 2024-09-16T23:48:44.489 INFO:tasks.workunit.client.1.smithi043.stdout:1/128: mkdir d4/d9/d12/d2b 0 2024-09-16T23:48:44.494 INFO:tasks.workunit.client.1.smithi043.stdout:4/117: dread d8/d15/d19/f11 [0,4194304] 0 2024-09-16T23:48:44.495 INFO:tasks.workunit.client.1.smithi043.stdout:8/91: dread de/f14 [0,4194304] 0 2024-09-16T23:48:44.499 INFO:tasks.workunit.client.1.smithi043.stdout:7/145: truncate d4/de/f1f 314417 0 2024-09-16T23:48:44.501 INFO:tasks.workunit.client.1.smithi043.stdout:1/129: dwrite d4/d9/d1c/d1e/f2a [0,4194304] 0 2024-09-16T23:48:44.502 INFO:tasks.workunit.client.1.smithi043.stdout:4/118: link d8/f1a d8/d15/f21 0 2024-09-16T23:48:44.503 INFO:tasks.workunit.client.1.smithi043.stdout:7/146: unlink d4/fa 0 2024-09-16T23:48:44.505 INFO:tasks.workunit.client.1.smithi043.stdout:1/130: write d4/fc [2613772,34465] 0 2024-09-16T23:48:44.505 INFO:tasks.workunit.client.1.smithi043.stdout:8/92: dread f8 [0,4194304] 0 2024-09-16T23:48:44.505 INFO:tasks.workunit.client.1.smithi043.stdout:7/147: dread - d4/fc zero size 2024-09-16T23:48:44.506 INFO:tasks.workunit.client.1.smithi043.stdout:7/148: dread - d4/fc zero size 2024-09-16T23:48:44.507 INFO:tasks.workunit.client.1.smithi043.stdout:8/93: stat de/ff 0 2024-09-16T23:48:44.510 INFO:tasks.workunit.client.1.smithi043.stdout:8/94: symlink de/d10/l19 0 2024-09-16T23:48:44.511 INFO:tasks.workunit.client.1.smithi043.stdout:8/95: truncate de/f15 894158 0 2024-09-16T23:48:44.511 INFO:tasks.workunit.client.1.smithi043.stdout:7/149: mknod d4/c2c 0 2024-09-16T23:48:44.512 INFO:tasks.workunit.client.1.smithi043.stdout:8/96: dread de/f14 [0,4194304] 0 2024-09-16T23:48:44.514 INFO:tasks.workunit.client.1.smithi043.stdout:7/150: creat d4/db/d28/f2d x:0 0 0 2024-09-16T23:48:44.515 INFO:tasks.workunit.client.1.smithi043.stdout:8/97: write f8 [5077008,88515] 0 2024-09-16T23:48:44.516 INFO:tasks.workunit.client.1.smithi043.stdout:4/119: write f1 [1614027,46241] 0 2024-09-16T23:48:44.516 INFO:tasks.workunit.client.1.smithi043.stdout:8/98: rename de/d18 to de/d18/d1a 22 2024-09-16T23:48:44.517 INFO:tasks.workunit.client.1.smithi043.stdout:4/120: write d8/fc [1166091,6836] 0 2024-09-16T23:48:44.520 INFO:tasks.workunit.client.1.smithi043.stdout:8/99: creat de/d10/f1b x:0 0 0 2024-09-16T23:48:44.521 INFO:tasks.workunit.client.1.smithi043.stdout:8/100: stat de/ff 0 2024-09-16T23:48:44.524 INFO:tasks.workunit.client.1.smithi043.stdout:1/131: dwrite f1 [4194304,4194304] 0 2024-09-16T23:48:44.529 INFO:tasks.workunit.client.1.smithi043.stdout:4/121: truncate f7 1944987 0 2024-09-16T23:48:44.529 INFO:tasks.workunit.client.1.smithi043.stdout:1/132: dread d4/fc [8388608,4194304] 0 2024-09-16T23:48:44.529 INFO:tasks.workunit.client.1.smithi043.stdout:8/101: link c3 de/d18/c1c 0 2024-09-16T23:48:44.531 INFO:tasks.workunit.client.1.smithi043.stdout:8/102: write de/d10/f1b [661125,81970] 0 2024-09-16T23:48:44.531 INFO:tasks.workunit.client.1.smithi043.stdout:4/122: mkdir d8/d15/d17/d22 0 2024-09-16T23:48:44.532 INFO:tasks.workunit.client.1.smithi043.stdout:8/103: dread fc [0,4194304] 0 2024-09-16T23:48:44.534 INFO:tasks.workunit.client.1.smithi043.stdout:8/104: write de/f15 [1219725,130368] 0 2024-09-16T23:48:44.534 INFO:tasks.workunit.client.1.smithi043.stdout:8/105: chown de/f15 10436 1 2024-09-16T23:48:44.534 INFO:tasks.workunit.client.1.smithi043.stdout:1/133: rename d4/d9/d1c/l24 to d4/df/d26/l2c 0 2024-09-16T23:48:44.536 INFO:tasks.workunit.client.1.smithi043.stdout:4/123: mkdir d8/d15/d17/d23 0 2024-09-16T23:48:44.537 INFO:tasks.workunit.client.1.smithi043.stdout:8/106: symlink de/l1d 0 2024-09-16T23:48:44.538 INFO:tasks.workunit.client.1.smithi043.stdout:8/107: fsync de/ff 0 2024-09-16T23:48:44.539 INFO:tasks.workunit.client.1.smithi043.stdout:1/134: dread d4/d9/d12/f10 [0,4194304] 0 2024-09-16T23:48:44.541 INFO:tasks.workunit.client.1.smithi043.stdout:1/135: truncate d4/d9/d12/f1f 705168 0 2024-09-16T23:48:44.541 INFO:tasks.workunit.client.1.smithi043.stdout:8/108: link cb de/d18/c1e 0 2024-09-16T23:48:44.544 INFO:tasks.workunit.client.1.smithi043.stdout:1/136: symlink d4/df/l2d 0 2024-09-16T23:48:44.546 INFO:tasks.workunit.client.1.smithi043.stdout:1/137: symlink d4/d9/l2e 0 2024-09-16T23:48:44.552 INFO:tasks.workunit.client.1.smithi043.stdout:1/138: creat d4/df/d13/d1b/f2f x:0 0 0 2024-09-16T23:48:44.554 INFO:tasks.workunit.client.1.smithi043.stdout:8/109: dwrite de/d10/f13 [0,4194304] 0 2024-09-16T23:48:44.557 INFO:tasks.workunit.client.1.smithi043.stdout:8/110: write de/ff [885545,106967] 0 2024-09-16T23:48:44.558 INFO:tasks.workunit.client.1.smithi043.stdout:1/139: creat d4/df/d13/d1b/f30 x:0 0 0 2024-09-16T23:48:44.558 INFO:tasks.workunit.client.1.smithi043.stdout:4/124: dwrite d8/d15/d17/f10 [0,4194304] 0 2024-09-16T23:48:44.564 INFO:tasks.workunit.client.1.smithi043.stdout:1/140: write d4/df/d13/d1b/f30 [141847,19441] 0 2024-09-16T23:48:44.565 INFO:tasks.workunit.client.1.smithi043.stdout:8/111: mkdir de/d18/d1f 0 2024-09-16T23:48:44.570 INFO:tasks.workunit.client.1.smithi043.stdout:1/141: dread d4/d9/d1c/d1e/f2a [0,4194304] 0 2024-09-16T23:48:44.578 INFO:tasks.workunit.client.1.smithi043.stdout:1/142: truncate d4/df/d13/d1b/f25 3184405 0 2024-09-16T23:48:44.579 INFO:tasks.workunit.client.1.smithi043.stdout:4/125: dwrite d8/f1a [4194304,4194304] 0 2024-09-16T23:48:44.582 INFO:tasks.workunit.client.1.smithi043.stdout:1/143: creat d4/d9/d12/d15/f31 x:0 0 0 2024-09-16T23:48:44.587 INFO:tasks.workunit.client.1.smithi043.stdout:1/144: dread d4/d9/f17 [0,4194304] 0 2024-09-16T23:48:44.591 INFO:tasks.workunit.client.1.smithi043.stdout:1/145: symlink d4/d9/d12/d15/l32 0 2024-09-16T23:48:44.592 INFO:tasks.workunit.client.1.smithi043.stdout:4/126: dwrite d8/d15/d19/f11 [0,4194304] 0 2024-09-16T23:48:44.594 INFO:tasks.workunit.client.1.smithi043.stdout:1/146: dread d4/d9/d12/f27 [0,4194304] 0 2024-09-16T23:48:44.595 INFO:tasks.workunit.client.1.smithi043.stdout:4/127: rmdir d8 39 2024-09-16T23:48:44.596 INFO:tasks.workunit.client.1.smithi043.stdout:1/147: fdatasync d4/d9/d12/d15/f31 0 2024-09-16T23:48:44.598 INFO:tasks.workunit.client.1.smithi043.stdout:0/103: sync 2024-09-16T23:48:44.599 INFO:tasks.workunit.client.1.smithi043.stdout:4/128: unlink d8/f1a 0 2024-09-16T23:48:44.602 INFO:tasks.workunit.client.1.smithi043.stdout:1/148: rename d4/d9/d12/f27 to d4/d9/d12/f33 0 2024-09-16T23:48:44.605 INFO:tasks.workunit.client.1.smithi043.stdout:4/129: creat d8/d15/d17/d22/f24 x:0 0 0 2024-09-16T23:48:44.607 INFO:tasks.workunit.client.1.smithi043.stdout:4/130: fsync f7 0 2024-09-16T23:48:44.607 INFO:tasks.workunit.client.1.smithi043.stdout:4/131: chown c6 148 1 2024-09-16T23:48:44.611 INFO:tasks.workunit.client.1.smithi043.stdout:4/132: dread d8/f1d [0,4194304] 0 2024-09-16T23:48:44.615 INFO:tasks.workunit.client.1.smithi043.stdout:4/133: mkdir d8/d25 0 2024-09-16T23:48:44.620 INFO:tasks.workunit.client.1.smithi043.stdout:4/134: creat d8/d15/d17/f26 x:0 0 0 2024-09-16T23:48:44.623 INFO:tasks.workunit.client.1.smithi043.stdout:4/135: read d8/d15/d19/f11 [2442874,116368] 0 2024-09-16T23:48:44.624 INFO:tasks.workunit.client.1.smithi043.stdout:1/149: dwrite d4/d9/d12/f33 [0,4194304] 0 2024-09-16T23:48:44.627 INFO:tasks.workunit.client.1.smithi043.stdout:4/136: mkdir d8/d15/d17/d23/d27 0 2024-09-16T23:48:44.630 INFO:tasks.workunit.client.1.smithi043.stdout:1/150: rename d4/d9/d1c/d1e/d28 to d4/d34 0 2024-09-16T23:48:44.631 INFO:tasks.workunit.client.1.smithi043.stdout:4/137: creat d8/d15/d17/f28 x:0 0 0 2024-09-16T23:48:44.632 INFO:tasks.workunit.client.1.smithi043.stdout:4/138: chown d8 14 1 2024-09-16T23:48:44.633 INFO:tasks.workunit.client.1.smithi043.stdout:4/139: write d8/d15/d17/f28 [558774,117803] 0 2024-09-16T23:48:44.636 INFO:tasks.workunit.client.1.smithi043.stdout:4/140: symlink d8/d15/d20/l29 0 2024-09-16T23:48:44.637 INFO:tasks.workunit.client.1.smithi043.stdout:1/151: rmdir d4/d9 39 2024-09-16T23:48:44.640 INFO:tasks.workunit.client.1.smithi043.stdout:4/141: dread d8/d15/d17/f28 [0,4194304] 0 2024-09-16T23:48:44.642 INFO:tasks.workunit.client.1.smithi043.stdout:4/142: creat d8/d15/d19/f2a x:0 0 0 2024-09-16T23:48:44.644 INFO:tasks.workunit.client.1.smithi043.stdout:1/152: link d4/d9/c19 d4/df/d13/d1b/c35 0 2024-09-16T23:48:44.644 INFO:tasks.workunit.client.1.smithi043.stdout:1/153: stat d4/d9/d12/f1f 0 2024-09-16T23:48:44.646 INFO:tasks.workunit.client.1.smithi043.stdout:4/143: rename d8/d15/d17 to d8/d15/d20/d2b 0 2024-09-16T23:48:44.650 INFO:tasks.workunit.client.1.smithi043.stdout:4/144: dread f1 [0,4194304] 0 2024-09-16T23:48:44.651 INFO:tasks.workunit.client.1.smithi043.stdout:4/145: chown d8/d15/d20/d2b/d22 3 1 2024-09-16T23:48:44.651 INFO:tasks.workunit.client.1.smithi043.stdout:1/154: truncate d4/d9/d12/f33 2207295 0 2024-09-16T23:48:44.651 INFO:tasks.workunit.client.1.smithi043.stdout:4/146: chown l2 845844672 1 2024-09-16T23:48:44.651 INFO:tasks.workunit.client.1.smithi043.stdout:4/147: fdatasync d8/d15/d19/f2a 0 2024-09-16T23:48:44.654 INFO:tasks.workunit.client.1.smithi043.stdout:4/148: rename d8/d15/d19/f11 to d8/d25/f2c 0 2024-09-16T23:48:44.658 INFO:tasks.workunit.client.1.smithi043.stdout:4/149: creat d8/d15/d20/d2b/f2d x:0 0 0 2024-09-16T23:48:44.661 INFO:tasks.workunit.client.1.smithi043.stdout:4/150: symlink d8/d25/l2e 0 2024-09-16T23:48:44.661 INFO:tasks.workunit.client.1.smithi043.stdout:4/151: readlink l2 0 2024-09-16T23:48:44.663 INFO:tasks.workunit.client.1.smithi043.stdout:1/155: dwrite d4/fc [0,4194304] 0 2024-09-16T23:48:44.665 INFO:tasks.workunit.client.1.smithi043.stdout:4/152: mknod d8/d15/d20/d2b/d22/c2f 0 2024-09-16T23:48:44.665 INFO:tasks.workunit.client.1.smithi043.stdout:1/156: rmdir d4/d9/d1c 39 2024-09-16T23:48:44.669 INFO:tasks.workunit.client.1.smithi043.stdout:4/153: symlink d8/d15/d20/d2b/d23/d27/l30 0 2024-09-16T23:48:44.670 INFO:tasks.workunit.client.1.smithi043.stdout:1/157: creat d4/d9/d12/d2b/f36 x:0 0 0 2024-09-16T23:48:44.672 INFO:tasks.workunit.client.1.smithi043.stdout:1/158: write d4/d9/d12/f1f [1353245,30198] 0 2024-09-16T23:48:44.672 INFO:tasks.workunit.client.1.smithi043.stdout:4/154: creat d8/f31 x:0 0 0 2024-09-16T23:48:44.673 INFO:tasks.workunit.client.1.smithi043.stdout:4/155: write d8/d15/d20/d2b/f10 [1050872,38849] 0 2024-09-16T23:48:44.675 INFO:tasks.workunit.client.1.smithi043.stdout:1/159: chown d4/df/d26/l2c 1 1 2024-09-16T23:48:44.675 INFO:tasks.workunit.client.1.smithi043.stdout:4/156: read f7 [1718397,48310] 0 2024-09-16T23:48:44.675 INFO:tasks.workunit.client.1.smithi043.stdout:1/160: chown d4 7487059 1 2024-09-16T23:48:44.676 INFO:tasks.workunit.client.1.smithi043.stdout:1/161: read - d4/d9/d12/d2b/f36 zero size 2024-09-16T23:48:44.676 INFO:tasks.workunit.client.1.smithi043.stdout:4/157: stat d8/d15/f21 0 2024-09-16T23:48:44.678 INFO:tasks.workunit.client.1.smithi043.stdout:1/162: truncate d4/d9/d12/d2b/f36 714695 0 2024-09-16T23:48:44.679 INFO:tasks.workunit.client.1.smithi043.stdout:4/158: creat d8/d15/d20/d2b/f32 x:0 0 0 2024-09-16T23:48:44.682 INFO:tasks.workunit.client.1.smithi043.stdout:1/163: creat d4/d9/d1c/f37 x:0 0 0 2024-09-16T23:48:44.682 INFO:tasks.workunit.client.1.smithi043.stdout:4/159: fsync d8/d15/d20/d2b/f32 0 2024-09-16T23:48:44.683 INFO:tasks.workunit.client.1.smithi043.stdout:1/164: chown d4/d9/d1c/d1e/f2a 1 1 2024-09-16T23:48:44.684 INFO:tasks.workunit.client.1.smithi043.stdout:1/165: chown d4/d9/d1c/d1e 3601 1 2024-09-16T23:48:44.685 INFO:tasks.workunit.client.1.smithi043.stdout:1/166: creat d4/df/f38 x:0 0 0 2024-09-16T23:48:44.686 INFO:tasks.workunit.client.1.smithi043.stdout:4/160: write f7 [904836,114592] 0 2024-09-16T23:48:44.688 INFO:tasks.workunit.client.1.smithi043.stdout:1/167: unlink d4/d9/c18 0 2024-09-16T23:48:44.689 INFO:tasks.workunit.client.1.smithi043.stdout:4/161: unlink d8/d15/d20/d2b/d22/c2f 0 2024-09-16T23:48:44.691 INFO:tasks.workunit.client.1.smithi043.stdout:4/162: dread f7 [0,4194304] 0 2024-09-16T23:48:44.692 INFO:tasks.workunit.client.1.smithi043.stdout:1/168: link d4/df/d13/d1b/f30 d4/df/d26/f39 0 2024-09-16T23:48:44.694 INFO:tasks.workunit.client.1.smithi043.stdout:4/163: rename l3 to d8/d15/d20/l33 0 2024-09-16T23:48:44.696 INFO:tasks.workunit.client.1.smithi043.stdout:4/164: mknod d8/d15/d20/d2b/d23/d27/c34 0 2024-09-16T23:48:44.696 INFO:tasks.workunit.client.1.smithi043.stdout:1/169: dread d4/d9/d12/f33 [0,4194304] 0 2024-09-16T23:48:44.697 INFO:tasks.workunit.client.1.smithi043.stdout:4/165: fdatasync d8/fc 0 2024-09-16T23:48:44.698 INFO:tasks.workunit.client.1.smithi043.stdout:4/166: dread d8/f1d [0,4194304] 0 2024-09-16T23:48:44.702 INFO:tasks.workunit.client.1.smithi043.stdout:1/170: getdents d4/df 0 2024-09-16T23:48:44.703 INFO:tasks.workunit.client.1.smithi043.stdout:1/171: fsync d4/df/f38 0 2024-09-16T23:48:44.706 INFO:tasks.workunit.client.1.smithi043.stdout:1/172: creat d4/d34/f3a x:0 0 0 2024-09-16T23:48:44.708 INFO:tasks.workunit.client.1.smithi043.stdout:4/167: dwrite d8/d15/d20/d2b/f28 [0,4194304] 0 2024-09-16T23:48:44.708 INFO:tasks.workunit.client.1.smithi043.stdout:1/173: chown d4/d9/d1c/f37 6932 1 2024-09-16T23:48:44.709 INFO:tasks.workunit.client.1.smithi043.stdout:4/168: write f7 [2601910,3754] 0 2024-09-16T23:48:44.710 INFO:tasks.workunit.client.1.smithi043.stdout:1/174: fsync d4/df/d13/d1b/f30 0 2024-09-16T23:48:44.711 INFO:tasks.workunit.client.1.smithi043.stdout:4/169: chown d8/la 909513 1 2024-09-16T23:48:44.711 INFO:tasks.workunit.client.1.smithi043.stdout:1/175: fdatasync f1 0 2024-09-16T23:48:44.716 INFO:tasks.workunit.client.1.smithi043.stdout:1/176: dread f1 [0,4194304] 0 2024-09-16T23:48:44.717 INFO:tasks.workunit.client.1.smithi043.stdout:4/170: write d8/f1d [178249,98500] 0 2024-09-16T23:48:44.719 INFO:tasks.workunit.client.1.smithi043.stdout:4/171: write d8/f31 [297237,22845] 0 2024-09-16T23:48:44.719 INFO:tasks.workunit.client.1.smithi043.stdout:1/177: mknod d4/d9/c3b 0 2024-09-16T23:48:44.720 INFO:tasks.workunit.client.1.smithi043.stdout:4/172: write d8/f31 [322857,1340] 0 2024-09-16T23:48:44.722 INFO:tasks.workunit.client.1.smithi043.stdout:4/173: dread - d8/d15/d20/d2b/d22/f24 zero size 2024-09-16T23:48:44.723 INFO:tasks.workunit.client.1.smithi043.stdout:1/178: dread d4/df/d13/d1b/f30 [0,4194304] 0 2024-09-16T23:48:44.724 INFO:tasks.workunit.client.1.smithi043.stdout:4/174: mkdir d8/d15/d20/d35 0 2024-09-16T23:48:44.725 INFO:tasks.workunit.client.1.smithi043.stdout:5/166: sync 2024-09-16T23:48:44.729 INFO:tasks.workunit.client.1.smithi043.stdout:1/179: getdents d4/d9/d12/d15 0 2024-09-16T23:48:44.729 INFO:tasks.workunit.client.1.smithi043.stdout:4/175: dread d8/f31 [0,4194304] 0 2024-09-16T23:48:44.729 INFO:tasks.workunit.client.1.smithi043.stdout:5/167: write d3/d9/d23/f2d [4509948,88828] 0 2024-09-16T23:48:44.731 INFO:tasks.workunit.client.1.smithi043.stdout:4/176: dread - d8/d15/d20/d2b/d22/f24 zero size 2024-09-16T23:48:44.732 INFO:tasks.workunit.client.1.smithi043.stdout:1/180: mknod d4/df/c3c 0 2024-09-16T23:48:44.734 INFO:tasks.workunit.client.1.smithi043.stdout:4/177: creat d8/d15/d20/d2b/d22/f36 x:0 0 0 2024-09-16T23:48:44.735 INFO:tasks.workunit.client.1.smithi043.stdout:1/181: unlink d4/df/d13/l1a 0 2024-09-16T23:48:44.737 INFO:tasks.workunit.client.1.smithi043.stdout:4/178: unlink f1 0 2024-09-16T23:48:44.738 INFO:tasks.workunit.client.1.smithi043.stdout:1/182: mkdir d4/d9/d12/d2b/d3d 0 2024-09-16T23:48:44.738 INFO:tasks.workunit.client.1.smithi043.stdout:1/183: chown d4/d9/d12/d2b/f36 254 1 2024-09-16T23:48:44.739 INFO:tasks.workunit.client.1.smithi043.stdout:5/168: truncate d3/db/d26/f33 1049318 0 2024-09-16T23:48:44.740 INFO:tasks.workunit.client.1.smithi043.stdout:1/184: mkdir d4/d3e 0 2024-09-16T23:48:44.741 INFO:tasks.workunit.client.1.smithi043.stdout:5/169: creat d3/f38 x:0 0 0 2024-09-16T23:48:44.742 INFO:tasks.workunit.client.1.smithi043.stdout:1/185: chown d4/d9/d12/f33 3779372 1 2024-09-16T23:48:44.747 INFO:tasks.workunit.client.1.smithi043.stdout:5/170: dread d3/f32 [0,4194304] 0 2024-09-16T23:48:44.749 INFO:tasks.workunit.client.1.smithi043.stdout:1/186: dwrite d4/d9/d12/f1f [0,4194304] 0 2024-09-16T23:48:44.752 INFO:tasks.workunit.client.1.smithi043.stdout:1/187: write d4/fc [4287538,127486] 0 2024-09-16T23:48:44.753 INFO:tasks.workunit.client.1.smithi043.stdout:5/171: creat d3/d9/d23/d28/f39 x:0 0 0 2024-09-16T23:48:44.755 INFO:tasks.workunit.client.1.smithi043.stdout:1/188: symlink d4/d3e/l3f 0 2024-09-16T23:48:44.757 INFO:tasks.workunit.client.1.smithi043.stdout:5/172: symlink d3/d9/l3a 0 2024-09-16T23:48:44.758 INFO:tasks.workunit.client.1.smithi043.stdout:1/189: dread d4/d9/d12/f10 [0,4194304] 0 2024-09-16T23:48:44.759 INFO:tasks.workunit.client.1.smithi043.stdout:1/190: dread - d4/d9/d1c/f37 zero size 2024-09-16T23:48:44.762 INFO:tasks.workunit.client.1.smithi043.stdout:5/173: creat d3/d9/f3b x:0 0 0 2024-09-16T23:48:44.765 INFO:tasks.workunit.client.1.smithi043.stdout:1/191: link d4/df/d26/f39 d4/df/d13/f40 0 2024-09-16T23:48:44.766 INFO:tasks.workunit.client.1.smithi043.stdout:1/192: dread - d4/d9/d1c/f37 zero size 2024-09-16T23:48:44.768 INFO:tasks.workunit.client.1.smithi043.stdout:5/174: dread d3/d9/d23/f35 [0,4194304] 0 2024-09-16T23:48:44.771 INFO:tasks.workunit.client.1.smithi043.stdout:5/175: creat d3/db/d15/f3c x:0 0 0 2024-09-16T23:48:44.772 INFO:tasks.workunit.client.1.smithi043.stdout:5/176: readlink d3/d9/d22/l2f 0 2024-09-16T23:48:44.772 INFO:tasks.workunit.client.1.smithi043.stdout:1/193: dwrite f1 [0,4194304] 0 2024-09-16T23:48:44.778 INFO:tasks.workunit.client.1.smithi043.stdout:1/194: dread d4/df/d13/d1b/f25 [0,4194304] 0 2024-09-16T23:48:44.780 INFO:tasks.workunit.client.1.smithi043.stdout:1/195: creat d4/d34/f41 x:0 0 0 2024-09-16T23:48:44.782 INFO:tasks.workunit.client.1.smithi043.stdout:5/177: getdents d3/db/d15 0 2024-09-16T23:48:44.783 INFO:tasks.workunit.client.1.smithi043.stdout:1/196: stat d4/df/d13/d1b/f30 0 2024-09-16T23:48:44.783 INFO:tasks.workunit.client.1.smithi043.stdout:5/178: chown d3/d9/d23/d28/f39 17 1 2024-09-16T23:48:44.786 INFO:tasks.workunit.client.1.smithi043.stdout:1/197: write d4/df/d13/d1b/f25 [642663,44344] 0 2024-09-16T23:48:44.789 INFO:tasks.workunit.client.1.smithi043.stdout:5/179: dread d3/f17 [0,4194304] 0 2024-09-16T23:48:44.794 INFO:tasks.workunit.client.1.smithi043.stdout:1/198: dwrite d4/fc [0,4194304] 0 2024-09-16T23:48:44.794 INFO:tasks.workunit.client.1.smithi043.stdout:5/180: unlink d3/d1b/l1f 0 2024-09-16T23:48:44.795 INFO:tasks.workunit.client.1.smithi043.stdout:5/181: chown d3/d9/l3a 4100413 1 2024-09-16T23:48:44.796 INFO:tasks.workunit.client.1.smithi043.stdout:5/182: chown d3/d9/f3b 2818 1 2024-09-16T23:48:44.803 INFO:tasks.workunit.client.1.smithi043.stdout:1/199: dwrite d4/d9/d1c/f37 [0,4194304] 0 2024-09-16T23:48:44.807 INFO:tasks.workunit.client.1.smithi043.stdout:1/200: dread d4/d9/d12/f10 [0,4194304] 0 2024-09-16T23:48:44.807 INFO:tasks.workunit.client.1.smithi043.stdout:5/183: write d3/db/d26/f33 [2001689,5247] 0 2024-09-16T23:48:44.812 INFO:tasks.workunit.client.1.smithi043.stdout:5/184: creat d3/f3d x:0 0 0 2024-09-16T23:48:44.816 INFO:tasks.workunit.client.1.smithi043.stdout:1/201: dwrite d4/fc [0,4194304] 0 2024-09-16T23:48:44.817 INFO:tasks.workunit.client.1.smithi043.stdout:1/202: fsync d4/d34/f41 0 2024-09-16T23:48:44.823 INFO:tasks.workunit.client.1.smithi043.stdout:5/185: dwrite d3/f17 [0,4194304] 0 2024-09-16T23:48:44.825 INFO:tasks.workunit.client.1.smithi043.stdout:5/186: write d3/db/d15/f3c [908184,72562] 0 2024-09-16T23:48:44.829 INFO:tasks.workunit.client.1.smithi043.stdout:5/187: truncate d3/d9/d23/f35 8207491 0 2024-09-16T23:48:44.829 INFO:tasks.workunit.client.1.smithi043.stdout:5/188: dread - d3/d1b/f36 zero size 2024-09-16T23:48:44.830 INFO:tasks.workunit.client.1.smithi043.stdout:5/189: dread - d3/d1b/f36 zero size 2024-09-16T23:48:44.830 INFO:tasks.workunit.client.1.smithi043.stdout:5/190: stat c1 0 2024-09-16T23:48:44.831 INFO:tasks.workunit.client.1.smithi043.stdout:5/191: read d3/d9/d23/f2d [1798898,64836] 0 2024-09-16T23:48:44.838 INFO:tasks.workunit.client.1.smithi043.stdout:5/192: rename d3/d9/d23 to d3/db/d19/d3e 0 2024-09-16T23:48:44.846 INFO:tasks.workunit.client.1.smithi043.stdout:5/193: dwrite d3/d9/f3b [0,4194304] 0 2024-09-16T23:48:44.848 INFO:tasks.workunit.client.1.smithi043.stdout:5/194: chown d3/db/d19/d3e/d28/c2c 15326915 1 2024-09-16T23:48:44.848 INFO:tasks.workunit.client.1.smithi043.stdout:5/195: chown d3/db/d15 88755185 1 2024-09-16T23:48:44.852 INFO:tasks.workunit.client.1.smithi043.stdout:5/196: symlink d3/d9/l3f 0 2024-09-16T23:48:44.853 INFO:tasks.workunit.client.1.smithi043.stdout:5/197: chown d3/f17 257202882 1 2024-09-16T23:48:44.854 INFO:tasks.workunit.client.1.smithi043.stdout:5/198: fsync d3/f38 0 2024-09-16T23:48:44.865 INFO:tasks.workunit.client.1.smithi043.stdout:5/199: dwrite d3/f3d [0,4194304] 0 2024-09-16T23:48:44.865 INFO:tasks.workunit.client.1.smithi043.stdout:5/200: mknod d3/db/d19/d3e/d28/c40 0 2024-09-16T23:48:44.867 INFO:tasks.workunit.client.1.smithi043.stdout:5/201: chown d3/db/d19/d3e/l30 17122 1 2024-09-16T23:48:44.868 INFO:tasks.workunit.client.1.smithi043.stdout:5/202: readlink d3/d9/l3f 0 2024-09-16T23:48:44.870 INFO:tasks.workunit.client.1.smithi043.stdout:5/203: unlink d3/l1c 0 2024-09-16T23:48:44.871 INFO:tasks.workunit.client.1.smithi043.stdout:5/204: mknod d3/db/d15/c41 0 2024-09-16T23:48:44.875 INFO:tasks.workunit.client.1.smithi043.stdout:5/205: rename d3/f3d to d3/db/d19/d3e/d28/f42 0 2024-09-16T23:48:44.876 INFO:tasks.workunit.client.1.smithi043.stdout:5/206: read d3/db/f2a [700014,89130] 0 2024-09-16T23:48:44.880 INFO:tasks.workunit.client.1.smithi043.stdout:5/207: dread d3/db/d15/f3c [0,4194304] 0 2024-09-16T23:48:44.882 INFO:tasks.workunit.client.1.smithi043.stdout:5/208: stat d3/db/d19/d3e/f35 0 2024-09-16T23:48:44.883 INFO:tasks.workunit.client.1.smithi043.stdout:5/209: mkdir d3/db/d15/d43 0 2024-09-16T23:48:44.888 INFO:tasks.workunit.client.1.smithi043.stdout:5/210: dread d3/d9/f3b [0,4194304] 0 2024-09-16T23:48:44.892 INFO:tasks.workunit.client.1.smithi043.stdout:5/211: truncate d3/f32 4048209 0 2024-09-16T23:48:44.896 INFO:tasks.workunit.client.1.smithi043.stdout:5/212: link d3/c1e d3/db/d19/d3e/d28/c44 0 2024-09-16T23:48:44.897 INFO:tasks.workunit.client.1.smithi043.stdout:5/213: chown d3/db/d15/f3c 344148990 1 2024-09-16T23:48:44.899 INFO:tasks.workunit.client.1.smithi043.stdout:5/214: fdatasync f2 0 2024-09-16T23:48:44.903 INFO:tasks.workunit.client.1.smithi043.stdout:5/215: truncate d3/db/d19/d3e/f35 8457975 0 2024-09-16T23:48:44.906 INFO:tasks.workunit.client.1.smithi043.stdout:5/216: creat d3/db/d19/d3e/f45 x:0 0 0 2024-09-16T23:48:44.907 INFO:tasks.workunit.client.1.smithi043.stdout:5/217: mknod d3/d9/d22/c46 0 2024-09-16T23:48:44.908 INFO:tasks.workunit.client.1.smithi043.stdout:5/218: read d3/d9/f3b [1116875,82158] 0 2024-09-16T23:48:44.913 INFO:tasks.workunit.client.1.smithi043.stdout:5/219: truncate d3/f32 2861474 0 2024-09-16T23:48:44.916 INFO:tasks.workunit.client.1.smithi043.stdout:5/220: getdents d3/db/d19 0 2024-09-16T23:48:44.917 INFO:tasks.workunit.client.1.smithi043.stdout:5/221: mknod d3/db/d15/d43/c47 0 2024-09-16T23:48:44.921 INFO:tasks.workunit.client.1.smithi043.stdout:5/222: link d3/d9/c29 d3/db/d19/d3e/c48 0 2024-09-16T23:48:44.928 INFO:tasks.workunit.client.1.smithi043.stdout:5/223: dwrite d3/d1b/f36 [0,4194304] 0 2024-09-16T23:48:44.937 INFO:tasks.workunit.client.1.smithi043.stdout:5/224: dwrite d3/f38 [0,4194304] 0 2024-09-16T23:48:44.943 INFO:tasks.workunit.client.1.smithi043.stdout:5/225: rename d3/db/l18 to d3/d9/l49 0 2024-09-16T23:48:44.949 INFO:tasks.workunit.client.1.smithi043.stdout:5/226: dread d3/db/d19/d3e/f35 [4194304,4194304] 0 2024-09-16T23:48:44.950 INFO:tasks.workunit.client.1.smithi043.stdout:5/227: mkdir d3/d9/d22/d4a 0 2024-09-16T23:48:44.955 INFO:tasks.workunit.client.1.smithi043.stdout:5/228: rename d3/d9/f3b to d3/d9/f4b 0 2024-09-16T23:48:44.958 INFO:tasks.workunit.client.1.smithi043.stdout:5/229: fsync d3/db/d19/d3e/f2d 0 2024-09-16T23:48:44.965 INFO:tasks.workunit.client.1.smithi043.stdout:5/230: dwrite d3/db/d19/d3e/d28/f37 [0,4194304] 0 2024-09-16T23:48:44.971 INFO:tasks.workunit.client.1.smithi043.stdout:5/231: write d3/db/d19/d3e/f2d [4393793,81458] 0 2024-09-16T23:48:44.973 INFO:tasks.workunit.client.1.smithi043.stdout:5/232: dread d3/db/d15/f3c [0,4194304] 0 2024-09-16T23:48:44.976 INFO:tasks.workunit.client.1.smithi043.stdout:5/233: getdents d3/d9/d22 0 2024-09-16T23:48:44.976 INFO:tasks.workunit.client.1.smithi043.stdout:5/234: chown d3/d1b 60795222 1 2024-09-16T23:48:44.978 INFO:tasks.workunit.client.1.smithi043.stdout:5/235: mkdir d3/d9/d22/d4a/d4c 0 2024-09-16T23:48:44.978 INFO:tasks.workunit.client.1.smithi043.stdout:5/236: stat c1 0 2024-09-16T23:48:44.979 INFO:tasks.workunit.client.1.smithi043.stdout:5/237: chown d3/db 358 1 2024-09-16T23:48:44.980 INFO:tasks.workunit.client.1.smithi043.stdout:5/238: mkdir d3/db/d15/d4d 0 2024-09-16T23:48:44.982 INFO:tasks.workunit.client.1.smithi043.stdout:5/239: creat d3/d9/d22/f4e x:0 0 0 2024-09-16T23:48:44.983 INFO:tasks.workunit.client.1.smithi043.stdout:5/240: symlink d3/db/d15/d43/l4f 0 2024-09-16T23:48:44.987 INFO:tasks.workunit.client.1.smithi043.stdout:5/241: rmdir d3/d9/d22 39 2024-09-16T23:48:44.987 INFO:tasks.workunit.client.1.smithi043.stdout:5/242: readlink d3/l20 0 2024-09-16T23:48:44.989 INFO:tasks.workunit.client.1.smithi043.stdout:5/243: creat d3/f50 x:0 0 0 2024-09-16T23:48:44.990 INFO:tasks.workunit.client.1.smithi043.stdout:5/244: mknod d3/db/d19/d3e/c51 0 2024-09-16T23:48:44.991 INFO:tasks.workunit.client.1.smithi043.stdout:5/245: write d3/f17 [2930993,72022] 0 2024-09-16T23:48:44.993 INFO:tasks.workunit.client.1.smithi043.stdout:5/246: chown d3/d9/ca 55967599 1 2024-09-16T23:48:44.994 INFO:tasks.workunit.client.1.smithi043.stdout:5/247: write d3/db/d19/d3e/d28/f39 [518331,1681] 0 2024-09-16T23:48:44.997 INFO:tasks.workunit.client.1.smithi043.stdout:5/248: dread d3/d9/f4b [0,4194304] 0 2024-09-16T23:48:44.998 INFO:tasks.workunit.client.1.smithi043.stdout:5/249: fsync d3/d9/f4b 0 2024-09-16T23:48:44.999 INFO:tasks.workunit.client.1.smithi043.stdout:2/36: sync 2024-09-16T23:48:45.000 INFO:tasks.workunit.client.1.smithi043.stdout:5/250: unlink d3/db/d19/d3e/c51 0 2024-09-16T23:48:45.001 INFO:tasks.workunit.client.1.smithi043.stdout:2/37: symlink d8/la 0 2024-09-16T23:48:45.001 INFO:tasks.workunit.client.1.smithi043.stdout:2/38: stat d8/c9 0 2024-09-16T23:48:45.001 INFO:tasks.workunit.client.1.smithi043.stdout:5/251: truncate d3/db/d19/d3e/f45 154967 0 2024-09-16T23:48:45.003 INFO:tasks.workunit.client.1.smithi043.stdout:5/252: creat d3/db/f52 x:0 0 0 2024-09-16T23:48:45.008 INFO:tasks.workunit.client.1.smithi043.stdout:2/39: dwrite f2 [0,4194304] 0 2024-09-16T23:48:45.009 INFO:tasks.workunit.client.1.smithi043.stdout:5/253: getdents d3 0 2024-09-16T23:48:45.011 INFO:tasks.workunit.client.1.smithi043.stdout:2/40: creat d8/fb x:0 0 0 2024-09-16T23:48:45.012 INFO:tasks.workunit.client.1.smithi043.stdout:5/254: rename d3/db/d19/d3e/c31 to d3/d1b/c53 0 2024-09-16T23:48:45.012 INFO:tasks.workunit.client.1.smithi043.stdout:2/41: dread f6 [0,4194304] 0 2024-09-16T23:48:45.013 INFO:tasks.workunit.client.1.smithi043.stdout:2/42: dread - d8/fb zero size 2024-09-16T23:48:45.015 INFO:tasks.workunit.client.1.smithi043.stdout:2/43: dread f6 [0,4194304] 0 2024-09-16T23:48:45.016 INFO:tasks.workunit.client.1.smithi043.stdout:2/44: chown f5 44 1 2024-09-16T23:48:45.016 INFO:tasks.workunit.client.1.smithi043.stdout:5/255: dread d3/db/d19/d3e/f2d [4194304,4194304] 0 2024-09-16T23:48:45.018 INFO:tasks.workunit.client.1.smithi043.stdout:5/256: stat d3/d9/d22 0 2024-09-16T23:48:45.020 INFO:tasks.workunit.client.1.smithi043.stdout:5/257: creat d3/db/d19/f54 x:0 0 0 2024-09-16T23:48:45.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:44 smithi042 ceph-mon[27914]: pgmap v261: 65 pgs: 65 active+clean; 267 MiB data, 1004 MiB used, 535 GiB / 536 GiB avail; 184 KiB/s rd, 12 MiB/s wr, 376 op/s 2024-09-16T23:48:45.022 INFO:tasks.workunit.client.1.smithi043.stdout:5/258: mknod d3/db/d19/d3e/d28/c55 0 2024-09-16T23:48:45.023 INFO:tasks.workunit.client.1.smithi043.stdout:5/259: dread - d3/db/f52 zero size 2024-09-16T23:48:45.028 INFO:tasks.workunit.client.1.smithi043.stdout:5/260: dwrite d3/f38 [0,4194304] 0 2024-09-16T23:48:45.033 INFO:tasks.workunit.client.1.smithi043.stdout:5/261: rename d3/db/d19/d3e/d28/f39 to d3/d9/f56 0 2024-09-16T23:48:45.035 INFO:tasks.workunit.client.1.smithi043.stdout:5/262: write d3/db/d19/d3e/f45 [838658,114600] 0 2024-09-16T23:48:45.035 INFO:tasks.workunit.client.1.smithi043.stdout:5/263: readlink d3/l20 0 2024-09-16T23:48:45.037 INFO:tasks.workunit.client.1.smithi043.stdout:5/264: symlink d3/db/d26/l57 0 2024-09-16T23:48:45.039 INFO:tasks.workunit.client.1.smithi043.stdout:5/265: symlink d3/d1b/l58 0 2024-09-16T23:48:45.040 INFO:tasks.workunit.client.1.smithi043.stdout:5/266: mknod d3/db/c59 0 2024-09-16T23:48:45.045 INFO:tasks.workunit.client.1.smithi043.stdout:3/105: sync 2024-09-16T23:48:45.046 INFO:tasks.workunit.client.1.smithi043.stdout:3/106: dread - d4/d17/f1d zero size 2024-09-16T23:48:45.046 INFO:tasks.workunit.client.1.smithi043.stdout:5/267: truncate d3/db/d26/f33 415508 0 2024-09-16T23:48:45.047 INFO:tasks.workunit.client.1.smithi043.stdout:3/107: mkdir d4/de/d1f 0 2024-09-16T23:48:45.054 INFO:tasks.workunit.client.1.smithi043.stdout:5/268: dwrite d3/d9/d22/f4e [0,4194304] 0 2024-09-16T23:48:45.056 INFO:tasks.workunit.client.1.smithi043.stdout:5/269: chown d3/d9/l49 514 1 2024-09-16T23:48:45.058 INFO:tasks.workunit.client.1.smithi043.stdout:5/270: mkdir d3/d1b/d5a 0 2024-09-16T23:48:45.060 INFO:tasks.workunit.client.1.smithi043.stdout:5/271: chown d3/d9/c29 29519 1 2024-09-16T23:48:45.063 INFO:tasks.workunit.client.1.smithi043.stdout:5/272: write d3/db/d19/d3e/f2d [1790996,57220] 0 2024-09-16T23:48:45.064 INFO:tasks.workunit.client.1.smithi043.stdout:5/273: chown d3/db/d19/d3e/d28/c40 46066 1 2024-09-16T23:48:45.070 INFO:tasks.workunit.client.1.smithi043.stdout:5/274: truncate d3/db/d19/d3e/f45 545252 0 2024-09-16T23:48:45.073 INFO:tasks.workunit.client.1.smithi043.stdout:5/275: mknod d3/db/d15/d4d/c5b 0 2024-09-16T23:48:45.073 INFO:tasks.workunit.client.1.smithi043.stdout:5/276: write d3/f38 [2527132,59770] 0 2024-09-16T23:48:45.077 INFO:tasks.workunit.client.1.smithi043.stdout:5/277: link d3/db/d19/d3e/l30 d3/d9/d22/d4a/l5c 0 2024-09-16T23:48:45.081 INFO:tasks.workunit.client.1.smithi043.stdout:5/278: rename d3/c1e to d3/d9/c5d 0 2024-09-16T23:48:45.086 INFO:tasks.workunit.client.1.smithi043.stdout:5/279: dread - d3/f50 zero size 2024-09-16T23:48:45.090 INFO:tasks.workunit.client.1.smithi043.stdout:5/280: dread d3/f17 [0,4194304] 0 2024-09-16T23:48:45.093 INFO:tasks.workunit.client.1.smithi043.stdout:5/281: rename d3/d9/l3a to d3/d9/d22/d4a/d4c/l5e 0 2024-09-16T23:48:45.095 INFO:tasks.workunit.client.1.smithi043.stdout:5/282: getdents d3/d9/d22 0 2024-09-16T23:48:45.097 INFO:tasks.workunit.client.1.smithi043.stdout:5/283: chown d3/d1b/c53 3 1 2024-09-16T23:48:45.105 INFO:tasks.workunit.client.1.smithi043.stdout:5/284: dwrite d3/db/d19/d3e/d28/f42 [4194304,4194304] 0 2024-09-16T23:48:45.107 INFO:tasks.workunit.client.1.smithi043.stdout:5/285: creat d3/db/d15/f5f x:0 0 0 2024-09-16T23:48:45.116 INFO:tasks.workunit.client.1.smithi043.stdout:5/286: dwrite d3/f17 [4194304,4194304] 0 2024-09-16T23:48:45.126 INFO:tasks.workunit.client.1.smithi043.stdout:5/287: dwrite d3/db/f2a [0,4194304] 0 2024-09-16T23:48:45.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:44 smithi043 ceph-mon[35191]: pgmap v261: 65 pgs: 65 active+clean; 267 MiB data, 1004 MiB used, 535 GiB / 536 GiB avail; 184 KiB/s rd, 12 MiB/s wr, 376 op/s 2024-09-16T23:48:45.132 INFO:tasks.workunit.client.1.smithi043.stdout:5/288: dread d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:45.135 INFO:tasks.workunit.client.1.smithi043.stdout:5/289: mkdir d3/d9/d22/d60 0 2024-09-16T23:48:45.136 INFO:tasks.workunit.client.1.smithi043.stdout:5/290: dread - d3/db/f52 zero size 2024-09-16T23:48:45.140 INFO:tasks.workunit.client.1.smithi043.stdout:5/291: rename d3/f32 to d3/d9/d22/f61 0 2024-09-16T23:48:45.146 INFO:tasks.workunit.client.1.smithi043.stdout:5/292: write d3/f38 [4439646,128045] 0 2024-09-16T23:48:45.147 INFO:tasks.workunit.client.1.smithi043.stdout:5/293: stat d3/d9/c2b 0 2024-09-16T23:48:45.149 INFO:tasks.workunit.client.1.smithi043.stdout:5/294: mknod d3/db/d15/d4d/c62 0 2024-09-16T23:48:45.152 INFO:tasks.workunit.client.1.smithi043.stdout:5/295: creat d3/d9/d22/d60/f63 x:0 0 0 2024-09-16T23:48:45.153 INFO:tasks.workunit.client.1.smithi043.stdout:5/296: truncate d3/d9/f56 880878 0 2024-09-16T23:48:45.157 INFO:tasks.workunit.client.1.smithi043.stdout:5/297: dread d3/d9/d22/f4e [0,4194304] 0 2024-09-16T23:48:45.157 INFO:tasks.workunit.client.1.smithi043.stdout:5/298: chown d3/db/c59 1087162557 1 2024-09-16T23:48:45.160 INFO:tasks.workunit.client.1.smithi043.stdout:5/299: unlink d3/l20 0 2024-09-16T23:48:45.164 INFO:tasks.workunit.client.1.smithi043.stdout:5/300: dread d3/f17 [4194304,4194304] 0 2024-09-16T23:48:45.167 INFO:tasks.workunit.client.1.smithi043.stdout:5/301: mkdir d3/d9/d22/d4a/d64 0 2024-09-16T23:48:45.174 INFO:tasks.workunit.client.1.smithi043.stdout:5/302: dwrite d3/db/f52 [0,4194304] 0 2024-09-16T23:48:45.179 INFO:tasks.workunit.client.1.smithi043.stdout:5/303: unlink d3/db/d19/d3e/d28/c2c 0 2024-09-16T23:48:45.179 INFO:tasks.workunit.client.1.smithi043.stdout:6/87: sync 2024-09-16T23:48:45.180 INFO:tasks.workunit.client.1.smithi043.stdout:6/88: write f3 [355665,101617] 0 2024-09-16T23:48:45.183 INFO:tasks.workunit.client.1.smithi043.stdout:6/89: mknod de/df/c1e 0 2024-09-16T23:48:45.188 INFO:tasks.workunit.client.1.smithi043.stdout:6/90: link de/f19 de/df/f1f 0 2024-09-16T23:48:45.188 INFO:tasks.workunit.client.1.smithi043.stdout:5/304: dwrite d3/d9/f56 [0,4194304] 0 2024-09-16T23:48:45.190 INFO:tasks.workunit.client.1.smithi043.stdout:6/91: truncate de/df/f16 344409 0 2024-09-16T23:48:45.193 INFO:tasks.workunit.client.1.smithi043.stdout:6/92: creat de/d1a/f20 x:0 0 0 2024-09-16T23:48:45.194 INFO:tasks.workunit.client.1.smithi043.stdout:6/93: stat de/d1a/f20 0 2024-09-16T23:48:45.194 INFO:tasks.workunit.client.1.smithi043.stdout:5/305: dread d3/db/f52 [0,4194304] 0 2024-09-16T23:48:45.195 INFO:tasks.workunit.client.1.smithi043.stdout:6/94: dread - de/df/f1f zero size 2024-09-16T23:48:45.199 INFO:tasks.workunit.client.1.smithi043.stdout:6/95: mknod de/c21 0 2024-09-16T23:48:45.201 INFO:tasks.workunit.client.1.smithi043.stdout:6/96: symlink de/df/l22 0 2024-09-16T23:48:45.208 INFO:tasks.workunit.client.1.smithi043.stdout:5/306: dwrite f2 [0,4194304] 0 2024-09-16T23:48:45.208 INFO:tasks.workunit.client.1.smithi043.stdout:6/97: dread de/df/f1b [0,4194304] 0 2024-09-16T23:48:45.211 INFO:tasks.workunit.client.1.smithi043.stdout:5/307: rmdir d3/d9 39 2024-09-16T23:48:45.215 INFO:tasks.workunit.client.1.smithi043.stdout:6/98: dwrite f1 [0,4194304] 0 2024-09-16T23:48:45.220 INFO:tasks.workunit.client.1.smithi043.stdout:6/99: mkdir de/d23 0 2024-09-16T23:48:45.224 INFO:tasks.workunit.client.1.smithi043.stdout:6/100: rename f6 to de/f24 0 2024-09-16T23:48:45.227 INFO:tasks.workunit.client.1.smithi043.stdout:6/101: unlink de/d1a/f1c 0 2024-09-16T23:48:45.227 INFO:tasks.workunit.client.1.smithi043.stdout:5/308: dwrite d3/d9/d22/f61 [0,4194304] 0 2024-09-16T23:48:45.229 INFO:tasks.workunit.client.1.smithi043.stdout:5/309: write d3/db/d19/d3e/f2d [2178772,92287] 0 2024-09-16T23:48:45.230 INFO:tasks.workunit.client.1.smithi043.stdout:5/310: read f2 [1232017,115522] 0 2024-09-16T23:48:45.233 INFO:tasks.workunit.client.1.smithi043.stdout:6/102: dread fc [0,4194304] 0 2024-09-16T23:48:45.234 INFO:tasks.workunit.client.1.smithi043.stdout:5/311: write d3/d9/d22/f61 [1513642,100176] 0 2024-09-16T23:48:45.246 INFO:tasks.workunit.client.1.smithi043.stdout:6/103: dwrite de/df/f1f [0,4194304] 0 2024-09-16T23:48:45.246 INFO:tasks.workunit.client.1.smithi043.stdout:6/104: stat de/f13 0 2024-09-16T23:48:45.249 INFO:tasks.workunit.client.1.smithi043.stdout:6/105: symlink de/l25 0 2024-09-16T23:48:45.251 INFO:tasks.workunit.client.1.smithi043.stdout:6/106: rename de/df/c1e to de/d1a/c26 0 2024-09-16T23:48:45.252 INFO:tasks.workunit.client.1.smithi043.stdout:6/107: fsync de/f13 0 2024-09-16T23:48:45.259 INFO:tasks.workunit.client.1.smithi043.stdout:6/108: truncate de/f13 870187 0 2024-09-16T23:48:45.259 INFO:tasks.workunit.client.1.smithi043.stdout:6/109: stat de/f13 0 2024-09-16T23:48:45.261 INFO:tasks.workunit.client.1.smithi043.stdout:6/110: mknod de/d23/c27 0 2024-09-16T23:48:45.263 INFO:tasks.workunit.client.1.smithi043.stdout:6/111: mkdir de/d1a/d28 0 2024-09-16T23:48:45.264 INFO:tasks.workunit.client.1.smithi043.stdout:6/112: readlink de/df/l22 0 2024-09-16T23:48:45.265 INFO:tasks.workunit.client.0.smithi042.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-16T23:48:45.267 INFO:tasks.workunit.client.1.smithi043.stdout:6/113: dread de/f19 [0,4194304] 0 2024-09-16T23:48:45.269 INFO:tasks.workunit.client.1.smithi043.stdout:6/114: write de/d1a/f20 [992548,77978] 0 2024-09-16T23:48:45.276 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T23:48:45.276 INFO:tasks.workunit.client.0.smithi042.stderr:+ make 2024-09-16T23:48:45.276 INFO:tasks.workunit.client.1.smithi043.stdout:6/115: dwrite de/df/f17 [0,4194304] 0 2024-09-16T23:48:45.280 INFO:tasks.workunit.client.1.smithi043.stdout:6/116: link de/d23/c27 de/d23/c29 0 2024-09-16T23:48:45.281 INFO:tasks.workunit.client.1.smithi043.stdout:6/117: readlink de/df/l22 0 2024-09-16T23:48:45.284 INFO:tasks.workunit.client.1.smithi043.stdout:6/118: mkdir de/d2a 0 2024-09-16T23:48:45.286 INFO:tasks.workunit.client.1.smithi043.stdout:6/119: write de/f24 [2974728,61534] 0 2024-09-16T23:48:45.290 INFO:tasks.workunit.client.1.smithi043.stdout:6/120: dread de/f19 [0,4194304] 0 2024-09-16T23:48:45.291 INFO:tasks.workunit.client.1.smithi043.stdout:6/121: write de/df/f1b [2433857,109493] 0 2024-09-16T23:48:45.294 INFO:tasks.workunit.client.1.smithi043.stdout:6/122: creat de/df/f2b x:0 0 0 2024-09-16T23:48:45.297 INFO:tasks.workunit.client.1.smithi043.stdout:6/123: rename de/df/l22 to de/d23/l2c 0 2024-09-16T23:48:45.298 INFO:tasks.workunit.client.1.smithi043.stdout:6/124: read fd [1726988,71018] 0 2024-09-16T23:48:45.299 INFO:tasks.workunit.client.1.smithi043.stdout:6/125: read de/df/f1b [85099,31414] 0 2024-09-16T23:48:45.303 INFO:tasks.workunit.client.1.smithi043.stdout:6/126: dread f3 [0,4194304] 0 2024-09-16T23:48:45.307 INFO:tasks.workunit.client.1.smithi043.stdout:6/127: dread de/d1a/f20 [0,4194304] 0 2024-09-16T23:48:45.310 INFO:tasks.workunit.client.1.smithi043.stdout:6/128: link de/l25 de/d23/l2d 0 2024-09-16T23:48:45.311 INFO:tasks.workunit.client.1.smithi043.stdout:6/129: unlink de/df/l12 0 2024-09-16T23:48:45.313 INFO:tasks.workunit.client.1.smithi043.stdout:6/130: unlink fc 0 2024-09-16T23:48:45.315 INFO:tasks.workunit.client.1.smithi043.stdout:6/131: mknod de/d23/c2e 0 2024-09-16T23:48:45.317 INFO:tasks.workunit.client.1.smithi043.stdout:6/132: rename de/l11 to de/l2f 0 2024-09-16T23:48:45.319 INFO:tasks.workunit.client.1.smithi043.stdout:6/133: unlink de/d23/c2e 0 2024-09-16T23:48:45.321 INFO:tasks.workunit.client.1.smithi043.stdout:6/134: symlink de/df/l30 0 2024-09-16T23:48:45.328 INFO:tasks.workunit.client.1.smithi043.stdout:6/135: rmdir de/df 39 2024-09-16T23:48:45.329 INFO:tasks.workunit.client.1.smithi043.stdout:6/136: write de/f19 [5108843,36418] 0 2024-09-16T23:48:45.329 INFO:tasks.workunit.client.1.smithi043.stdout:6/137: read f1 [1967759,19923] 0 2024-09-16T23:48:45.331 INFO:tasks.workunit.client.1.smithi043.stdout:6/138: rename c9 to de/d2a/c31 0 2024-09-16T23:48:45.332 INFO:tasks.workunit.client.1.smithi043.stdout:6/139: write fd [1890464,33080] 0 2024-09-16T23:48:45.334 INFO:tasks.workunit.client.1.smithi043.stdout:6/140: truncate de/df/f2b 324646 0 2024-09-16T23:48:45.334 INFO:tasks.workunit.client.1.smithi043.stdout:7/151: sync 2024-09-16T23:48:45.335 INFO:tasks.workunit.client.1.smithi043.stdout:9/108: sync 2024-09-16T23:48:45.336 INFO:tasks.workunit.client.1.smithi043.stdout:9/109: write d4/d5/d16/f1f [73500,35488] 0 2024-09-16T23:48:45.336 INFO:tasks.workunit.client.1.smithi043.stdout:6/141: rmdir de/d23 39 2024-09-16T23:48:45.337 INFO:tasks.workunit.client.1.smithi043.stdout:9/110: chown d4/d5/d16/f1f 1811791 1 2024-09-16T23:48:45.338 INFO:tasks.workunit.client.1.smithi043.stdout:7/152: rmdir d4/db/d17 39 2024-09-16T23:48:45.340 INFO:tasks.workunit.client.1.smithi043.stdout:6/142: truncate f1 4142559 0 2024-09-16T23:48:45.341 INFO:tasks.workunit.client.1.smithi043.stdout:9/111: creat d4/d1c/f20 x:0 0 0 2024-09-16T23:48:45.342 INFO:tasks.workunit.client.1.smithi043.stdout:6/143: rename de/df/f1b to de/d1a/f32 0 2024-09-16T23:48:45.345 INFO:tasks.workunit.client.1.smithi043.stdout:6/144: rename de/df/f2b to de/d2a/f33 0 2024-09-16T23:48:45.349 INFO:tasks.workunit.client.1.smithi043.stdout:7/153: truncate d4/db/d11/f13 1573628 0 2024-09-16T23:48:45.352 INFO:tasks.workunit.client.1.smithi043.stdout:9/112: write d4/f11 [632914,129239] 0 2024-09-16T23:48:45.354 INFO:tasks.workunit.client.1.smithi043.stdout:9/113: chown d4/d1c 490068 1 2024-09-16T23:48:45.357 INFO:tasks.workunit.client.1.smithi043.stdout:6/145: dwrite de/d1a/f20 [0,4194304] 0 2024-09-16T23:48:45.357 INFO:tasks.workunit.client.1.smithi043.stdout:9/114: truncate d4/ff 644087 0 2024-09-16T23:48:45.358 INFO:tasks.workunit.client.1.smithi043.stdout:9/115: write d4/d5/f1d [131782,45817] 0 2024-09-16T23:48:45.362 INFO:tasks.workunit.client.1.smithi043.stdout:9/116: mkdir d4/d21 0 2024-09-16T23:48:45.364 INFO:tasks.workunit.client.1.smithi043.stdout:9/117: readlink d4/l14 0 2024-09-16T23:48:45.364 INFO:tasks.workunit.client.1.smithi043.stdout:6/146: mkdir de/d23/d34 0 2024-09-16T23:48:45.364 INFO:tasks.workunit.client.1.smithi043.stdout:9/118: dread d4/d5/d16/f1f [0,4194304] 0 2024-09-16T23:48:45.367 INFO:tasks.workunit.client.1.smithi043.stdout:9/119: mkdir d4/d22 0 2024-09-16T23:48:45.369 INFO:tasks.workunit.client.1.smithi043.stdout:9/120: truncate d4/d5/f1d 316944 0 2024-09-16T23:48:45.372 INFO:tasks.workunit.client.1.smithi043.stdout:7/154: dwrite d4/db/d11/f13 [0,4194304] 0 2024-09-16T23:48:45.372 INFO:tasks.workunit.client.1.smithi043.stdout:9/121: mknod d4/d1c/c23 0 2024-09-16T23:48:45.373 INFO:tasks.workunit.client.1.smithi043.stdout:9/122: read - d4/d1c/f20 zero size 2024-09-16T23:48:45.375 INFO:tasks.workunit.client.1.smithi043.stdout:9/123: creat d4/f24 x:0 0 0 2024-09-16T23:48:45.376 INFO:tasks.workunit.client.1.smithi043.stdout:9/124: chown d4/ld 22 1 2024-09-16T23:48:45.376 INFO:tasks.workunit.client.1.smithi043.stdout:9/125: readlink d4/ld 0 2024-09-16T23:48:45.377 INFO:tasks.workunit.client.1.smithi043.stdout:7/155: getdents d4/db 0 2024-09-16T23:48:45.377 INFO:tasks.workunit.client.1.smithi043.stdout:9/126: truncate d4/ff 1112215 0 2024-09-16T23:48:45.380 INFO:tasks.workunit.client.1.smithi043.stdout:9/127: mkdir d4/d1c/d25 0 2024-09-16T23:48:45.381 INFO:tasks.workunit.client.1.smithi043.stdout:7/156: symlink d4/db/d17/d1b/l2e 0 2024-09-16T23:48:45.382 INFO:tasks.workunit.client.1.smithi043.stdout:7/157: fsync d4/db/f24 0 2024-09-16T23:48:45.384 INFO:tasks.workunit.client.1.smithi043.stdout:7/158: creat d4/db/d11/d21/f2f x:0 0 0 2024-09-16T23:48:45.385 INFO:tasks.workunit.client.0.smithi042.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-16T23:48:45.386 INFO:tasks.workunit.client.1.smithi043.stdout:7/159: mkdir d4/db/d11/d21/d30 0 2024-09-16T23:48:45.388 INFO:tasks.workunit.client.1.smithi043.stdout:7/160: getdents d4/db/d11/d21/d30 0 2024-09-16T23:48:45.389 INFO:tasks.workunit.client.1.smithi043.stdout:7/161: truncate d4/db/d11/d21/f2f 1031686 0 2024-09-16T23:48:45.392 INFO:tasks.workunit.client.1.smithi043.stdout:7/162: rmdir d4/db/d11/d21/d30 0 2024-09-16T23:48:45.395 INFO:tasks.workunit.client.1.smithi043.stdout:7/163: unlink d4/db/d11/c2a 0 2024-09-16T23:48:45.398 INFO:tasks.workunit.client.1.smithi043.stdout:7/164: unlink d4/db/d17/d1b/l1e 0 2024-09-16T23:48:45.398 INFO:tasks.workunit.client.1.smithi043.stdout:7/165: dread - d4/db/d28/f2d zero size 2024-09-16T23:48:45.400 INFO:tasks.workunit.client.1.smithi043.stdout:7/166: mkdir d4/db/d11/d21/d31 0 2024-09-16T23:48:45.406 INFO:tasks.workunit.client.1.smithi043.stdout:7/167: dwrite d4/db/f18 [0,4194304] 0 2024-09-16T23:48:45.409 INFO:tasks.workunit.client.1.smithi043.stdout:7/168: fsync d4/f19 0 2024-09-16T23:48:45.417 INFO:tasks.workunit.client.1.smithi043.stdout:7/169: dwrite d4/db/d11/f13 [0,4194304] 0 2024-09-16T23:48:45.516 INFO:tasks.workunit.client.1.smithi043.stdout:8/112: sync 2024-09-16T23:48:45.516 INFO:tasks.workunit.client.1.smithi043.stdout:8/113: chown l6 478541 1 2024-09-16T23:48:45.521 INFO:tasks.workunit.client.1.smithi043.stdout:8/114: dwrite de/d10/f13 [0,4194304] 0 2024-09-16T23:48:45.524 INFO:tasks.workunit.client.1.smithi043.stdout:8/115: mknod de/c20 0 2024-09-16T23:48:45.525 INFO:tasks.workunit.client.1.smithi043.stdout:8/116: creat de/d18/f21 x:0 0 0 2024-09-16T23:48:45.527 INFO:tasks.workunit.client.1.smithi043.stdout:8/117: chown de/d18/c1c 461 1 2024-09-16T23:48:45.528 INFO:tasks.workunit.client.1.smithi043.stdout:8/118: read de/f15 [1260043,36552] 0 2024-09-16T23:48:45.533 INFO:tasks.workunit.client.1.smithi043.stdout:8/119: dwrite de/d10/f13 [0,4194304] 0 2024-09-16T23:48:45.536 INFO:tasks.workunit.client.1.smithi043.stdout:8/120: mknod de/d10/c22 0 2024-09-16T23:48:45.540 INFO:tasks.workunit.client.1.smithi043.stdout:8/121: dread de/d10/f1b [0,4194304] 0 2024-09-16T23:48:45.546 INFO:tasks.workunit.client.1.smithi043.stdout:8/122: getdents de/d18 0 2024-09-16T23:48:45.555 INFO:tasks.workunit.client.1.smithi043.stdout:8/123: dwrite de/f15 [0,4194304] 0 2024-09-16T23:48:45.557 INFO:tasks.workunit.client.1.smithi043.stdout:8/124: creat de/f23 x:0 0 0 2024-09-16T23:48:45.559 INFO:tasks.workunit.client.1.smithi043.stdout:8/125: getdents de/d18 0 2024-09-16T23:48:45.563 INFO:tasks.workunit.client.1.smithi043.stdout:8/126: dread de/d10/f13 [0,4194304] 0 2024-09-16T23:48:45.565 INFO:tasks.workunit.client.1.smithi043.stdout:8/127: creat de/d10/f24 x:0 0 0 2024-09-16T23:48:45.568 INFO:tasks.workunit.client.1.smithi043.stdout:8/128: mkdir de/d18/d1f/d25 0 2024-09-16T23:48:45.568 INFO:tasks.workunit.client.1.smithi043.stdout:8/129: dread - de/d18/f21 zero size 2024-09-16T23:48:45.574 INFO:tasks.workunit.client.1.smithi043.stdout:8/130: dwrite de/d10/f1b [0,4194304] 0 2024-09-16T23:48:45.576 INFO:tasks.workunit.client.1.smithi043.stdout:0/104: sync 2024-09-16T23:48:45.576 INFO:tasks.workunit.client.1.smithi043.stdout:1/203: sync 2024-09-16T23:48:45.577 INFO:tasks.workunit.client.1.smithi043.stdout:4/179: sync 2024-09-16T23:48:45.577 INFO:tasks.workunit.client.1.smithi043.stdout:8/131: rmdir de/d10 39 2024-09-16T23:48:45.578 INFO:tasks.workunit.client.1.smithi043.stdout:4/180: read - d8/d15/d20/d2b/d22/f24 zero size 2024-09-16T23:48:45.579 INFO:tasks.workunit.client.1.smithi043.stdout:4/181: dread - d8/d15/d20/d2b/d22/f24 zero size 2024-09-16T23:48:45.583 INFO:tasks.workunit.client.1.smithi043.stdout:8/132: creat de/d10/f26 x:0 0 0 2024-09-16T23:48:45.587 INFO:tasks.workunit.client.1.smithi043.stdout:8/133: mkdir de/d18/d27 0 2024-09-16T23:48:45.587 INFO:tasks.workunit.client.1.smithi043.stdout:8/134: fsync de/d18/f21 0 2024-09-16T23:48:45.588 INFO:tasks.workunit.client.1.smithi043.stdout:1/204: rmdir d4/df/d13/d1b/d21 0 2024-09-16T23:48:45.589 INFO:tasks.workunit.client.1.smithi043.stdout:8/135: dread - de/d10/f24 zero size 2024-09-16T23:48:45.589 INFO:tasks.workunit.client.1.smithi043.stdout:4/182: truncate f7 336003 0 2024-09-16T23:48:45.591 INFO:tasks.workunit.client.1.smithi043.stdout:1/205: mkdir d4/d9/d1c/d1e/d42 0 2024-09-16T23:48:45.592 INFO:tasks.workunit.client.1.smithi043.stdout:4/183: write d8/f31 [1104427,14256] 0 2024-09-16T23:48:45.594 INFO:tasks.workunit.client.1.smithi043.stdout:8/136: dread de/f15 [0,4194304] 0 2024-09-16T23:48:45.596 INFO:tasks.workunit.client.1.smithi043.stdout:0/105: dread d0/f13 [0,4194304] 0 2024-09-16T23:48:45.597 INFO:tasks.workunit.client.1.smithi043.stdout:4/184: mknod d8/d15/d20/c37 0 2024-09-16T23:48:45.598 INFO:tasks.workunit.client.1.smithi043.stdout:8/137: mknod de/d18/d27/c28 0 2024-09-16T23:48:45.598 INFO:tasks.workunit.client.1.smithi043.stdout:8/138: stat de 0 2024-09-16T23:48:45.600 INFO:tasks.workunit.client.1.smithi043.stdout:8/139: fsync de/d10/f1b 0 2024-09-16T23:48:45.601 INFO:tasks.workunit.client.1.smithi043.stdout:1/206: dwrite d4/d9/d1c/f37 [0,4194304] 0 2024-09-16T23:48:45.602 INFO:tasks.workunit.client.1.smithi043.stdout:0/106: rmdir d0/d2 39 2024-09-16T23:48:45.605 INFO:tasks.workunit.client.1.smithi043.stdout:8/140: mknod de/c29 0 2024-09-16T23:48:45.605 INFO:tasks.workunit.client.1.smithi043.stdout:1/207: read d4/d9/d12/f10 [692406,29886] 0 2024-09-16T23:48:45.605 INFO:tasks.workunit.client.1.smithi043.stdout:0/107: chown d0/d2 11 1 2024-09-16T23:48:45.610 INFO:tasks.workunit.client.1.smithi043.stdout:1/208: fsync d4/df/d13/d1b/f2f 0 2024-09-16T23:48:45.610 INFO:tasks.workunit.client.1.smithi043.stdout:4/185: dwrite d8/d15/d20/d2b/f2d [0,4194304] 0 2024-09-16T23:48:45.610 INFO:tasks.workunit.client.1.smithi043.stdout:0/108: write d0/d2/df/f10 [13497779,31134] 0 2024-09-16T23:48:45.614 INFO:tasks.workunit.client.1.smithi043.stdout:0/109: dread - d0/d2/df/f16 zero size 2024-09-16T23:48:45.615 INFO:tasks.workunit.client.1.smithi043.stdout:8/141: link de/d10/c11 de/d18/d1f/d25/c2a 0 2024-09-16T23:48:45.617 INFO:tasks.workunit.client.1.smithi043.stdout:8/142: mkdir de/d18/d2b 0 2024-09-16T23:48:45.618 INFO:tasks.workunit.client.1.smithi043.stdout:9/128: sync 2024-09-16T23:48:45.618 INFO:tasks.workunit.client.1.smithi043.stdout:7/170: sync 2024-09-16T23:48:45.618 INFO:tasks.workunit.client.1.smithi043.stdout:2/45: sync 2024-09-16T23:48:45.618 INFO:tasks.workunit.client.1.smithi043.stdout:3/108: sync 2024-09-16T23:48:45.618 INFO:tasks.workunit.client.1.smithi043.stdout:6/147: sync 2024-09-16T23:48:45.619 INFO:tasks.workunit.client.1.smithi043.stdout:5/312: sync 2024-09-16T23:48:45.620 INFO:tasks.workunit.client.1.smithi043.stdout:1/209: dread d4/d9/d1c/d1e/f2a [0,4194304] 0 2024-09-16T23:48:45.621 INFO:tasks.workunit.client.1.smithi043.stdout:0/110: dread d0/d2/df/d1a/f1b [0,4194304] 0 2024-09-16T23:48:45.622 INFO:tasks.workunit.client.1.smithi043.stdout:2/46: creat d8/fc x:0 0 0 2024-09-16T23:48:45.624 INFO:tasks.workunit.client.1.smithi043.stdout:7/171: write d4/db/f24 [495362,47721] 0 2024-09-16T23:48:45.625 INFO:tasks.workunit.client.1.smithi043.stdout:8/143: rename de/c29 to de/d18/d1f/c2c 0 2024-09-16T23:48:45.626 INFO:tasks.workunit.client.1.smithi043.stdout:2/47: dread - d8/fb zero size 2024-09-16T23:48:45.626 INFO:tasks.workunit.client.1.smithi043.stdout:4/186: dwrite d8/d15/d20/d2b/f28 [4194304,4194304] 0 2024-09-16T23:48:45.626 INFO:tasks.workunit.client.1.smithi043.stdout:5/313: mknod d3/d9/d22/d60/c65 0 2024-09-16T23:48:45.630 INFO:tasks.workunit.client.1.smithi043.stdout:9/129: creat d4/d1c/d25/f26 x:0 0 0 2024-09-16T23:48:45.632 INFO:tasks.workunit.client.1.smithi043.stdout:2/48: chown f5 34153192 1 2024-09-16T23:48:45.636 INFO:tasks.workunit.client.1.smithi043.stdout:4/187: dread - d8/d15/d20/d2b/f32 zero size 2024-09-16T23:48:45.637 INFO:tasks.workunit.client.1.smithi043.stdout:9/130: creat d4/d1c/f27 x:0 0 0 2024-09-16T23:48:45.639 INFO:tasks.workunit.client.1.smithi043.stdout:6/148: dwrite de/d1a/f20 [0,4194304] 0 2024-09-16T23:48:45.639 INFO:tasks.workunit.client.1.smithi043.stdout:5/314: write d3/d1b/f36 [1432379,80724] 0 2024-09-16T23:48:45.641 INFO:tasks.workunit.client.1.smithi043.stdout:2/49: creat d8/fd x:0 0 0 2024-09-16T23:48:45.644 INFO:tasks.workunit.client.1.smithi043.stdout:4/188: chown d8/d15/d20/d2b/d23/d27 577530 1 2024-09-16T23:48:45.645 INFO:tasks.workunit.client.1.smithi043.stdout:3/109: dwrite d4/de/f15 [0,4194304] 0 2024-09-16T23:48:45.645 INFO:tasks.workunit.client.1.smithi043.stdout:7/172: creat d4/f32 x:0 0 0 2024-09-16T23:48:45.648 INFO:tasks.workunit.client.1.smithi043.stdout:0/111: link d0/lb d0/de/l25 0 2024-09-16T23:48:45.649 INFO:tasks.workunit.client.1.smithi043.stdout:3/110: readlink d4/lc 0 2024-09-16T23:48:45.650 INFO:tasks.workunit.client.1.smithi043.stdout:8/144: dwrite de/f23 [0,4194304] 0 2024-09-16T23:48:45.654 INFO:tasks.workunit.client.1.smithi043.stdout:6/149: creat de/d1a/d28/f35 x:0 0 0 2024-09-16T23:48:45.657 INFO:tasks.workunit.client.1.smithi043.stdout:3/111: dread - d4/f1b zero size 2024-09-16T23:48:45.657 INFO:tasks.workunit.client.1.smithi043.stdout:1/210: dwrite d4/df/d13/d1b/f30 [0,4194304] 0 2024-09-16T23:48:45.658 INFO:tasks.workunit.client.1.smithi043.stdout:4/189: write d8/d15/d20/d2b/f10 [657226,42619] 0 2024-09-16T23:48:45.661 INFO:tasks.workunit.client.1.smithi043.stdout:8/145: read de/f15 [981062,93976] 0 2024-09-16T23:48:45.665 INFO:tasks.workunit.client.1.smithi043.stdout:1/211: fdatasync d4/df/d13/d1b/f2f 0 2024-09-16T23:48:45.666 INFO:tasks.workunit.client.1.smithi043.stdout:4/190: read - d8/d15/d19/f2a zero size 2024-09-16T23:48:45.667 INFO:tasks.workunit.client.1.smithi043.stdout:7/173: rename d4/de to d4/db/d11/d21/d31/d33 0 2024-09-16T23:48:45.669 INFO:tasks.workunit.client.1.smithi043.stdout:3/112: rename d4/de to d4/de/d1f/d20 22 2024-09-16T23:48:45.669 INFO:tasks.workunit.client.1.smithi043.stdout:5/315: truncate d3/d1b/f36 402248 0 2024-09-16T23:48:45.672 INFO:tasks.workunit.client.1.smithi043.stdout:9/131: dwrite d4/d1c/d25/f26 [0,4194304] 0 2024-09-16T23:48:45.672 INFO:tasks.workunit.client.1.smithi043.stdout:7/174: symlink d4/db/d11/d21/l34 0 2024-09-16T23:48:45.677 INFO:tasks.workunit.client.1.smithi043.stdout:4/191: creat d8/d15/d20/d2b/d23/f38 x:0 0 0 2024-09-16T23:48:45.678 INFO:tasks.workunit.client.1.smithi043.stdout:6/150: dread de/f13 [0,4194304] 0 2024-09-16T23:48:45.682 INFO:tasks.workunit.client.1.smithi043.stdout:0/112: dwrite d0/d2/df/d1a/f1b [0,4194304] 0 2024-09-16T23:48:45.685 INFO:tasks.workunit.client.1.smithi043.stdout:6/151: chown de/d1a/c1d 4 1 2024-09-16T23:48:45.686 INFO:tasks.workunit.client.1.smithi043.stdout:9/132: dread d4/f1a [0,4194304] 0 2024-09-16T23:48:45.691 INFO:tasks.workunit.client.1.smithi043.stdout:8/146: dwrite de/d18/f21 [0,4194304] 0 2024-09-16T23:48:45.691 INFO:tasks.workunit.client.1.smithi043.stdout:9/133: mkdir d4/d28 0 2024-09-16T23:48:45.693 INFO:tasks.workunit.client.1.smithi043.stdout:0/113: symlink d0/d2/df/l26 0 2024-09-16T23:48:45.694 INFO:tasks.workunit.client.1.smithi043.stdout:6/152: write de/df/f17 [1129583,71856] 0 2024-09-16T23:48:45.695 INFO:tasks.workunit.client.1.smithi043.stdout:3/113: dwrite d4/d17/f1d [0,4194304] 0 2024-09-16T23:48:45.696 INFO:tasks.workunit.client.1.smithi043.stdout:5/316: dread d3/f38 [0,4194304] 0 2024-09-16T23:48:45.700 INFO:tasks.workunit.client.1.smithi043.stdout:8/147: creat de/d18/d1f/f2d x:0 0 0 2024-09-16T23:48:45.701 INFO:tasks.workunit.client.1.smithi043.stdout:5/317: chown d3/db/d15 717 1 2024-09-16T23:48:45.701 INFO:tasks.workunit.client.1.smithi043.stdout:0/114: write d0/d2/df/d1a/f1b [1609663,90641] 0 2024-09-16T23:48:45.704 INFO:tasks.workunit.client.1.smithi043.stdout:3/114: creat d4/de/f21 x:0 0 0 2024-09-16T23:48:45.706 INFO:tasks.workunit.client.1.smithi043.stdout:5/318: fsync d3/f17 0 2024-09-16T23:48:45.709 INFO:tasks.workunit.client.1.smithi043.stdout:4/192: dwrite d8/d15/f1f [0,4194304] 0 2024-09-16T23:48:45.709 INFO:tasks.workunit.client.1.smithi043.stdout:3/115: chown d4/de/f15 34834118 1 2024-09-16T23:48:45.716 INFO:tasks.workunit.client.1.smithi043.stdout:3/116: creat d4/f22 x:0 0 0 2024-09-16T23:48:45.718 INFO:tasks.workunit.client.1.smithi043.stdout:6/153: dwrite de/d1a/d28/f35 [0,4194304] 0 2024-09-16T23:48:45.718 INFO:tasks.workunit.client.1.smithi043.stdout:0/115: dread d0/f1c [0,4194304] 0 2024-09-16T23:48:45.720 INFO:tasks.workunit.client.1.smithi043.stdout:4/193: getdents d8/d15/d20/d2b 0 2024-09-16T23:48:45.721 INFO:tasks.workunit.client.1.smithi043.stdout:3/117: stat d4/de/f15 0 2024-09-16T23:48:45.723 INFO:tasks.workunit.client.1.smithi043.stdout:8/148: dwrite de/d10/f26 [0,4194304] 0 2024-09-16T23:48:45.723 INFO:tasks.workunit.client.1.smithi043.stdout:6/154: chown de/c21 32 1 2024-09-16T23:48:45.724 INFO:tasks.workunit.client.1.smithi043.stdout:6/155: chown f3 1049 1 2024-09-16T23:48:45.726 INFO:tasks.workunit.client.1.smithi043.stdout:0/116: unlink d0/f1c 0 2024-09-16T23:48:45.727 INFO:tasks.workunit.client.1.smithi043.stdout:5/319: dwrite d3/f38 [4194304,4194304] 0 2024-09-16T23:48:45.728 INFO:tasks.workunit.client.1.smithi043.stdout:4/194: mkdir d8/d25/d39 0 2024-09-16T23:48:45.734 INFO:tasks.workunit.client.1.smithi043.stdout:6/156: readlink de/d23/l2c 0 2024-09-16T23:48:45.735 INFO:tasks.workunit.client.1.smithi043.stdout:7/175: write d4/f19 [1021504,104810] 0 2024-09-16T23:48:45.744 INFO:tasks.workunit.client.1.smithi043.stdout:3/118: dwrite d4/f10 [0,4194304] 0 2024-09-16T23:48:45.744 INFO:tasks.workunit.client.1.smithi043.stdout:0/117: unlink d0/l9 0 2024-09-16T23:48:45.746 INFO:tasks.workunit.client.1.smithi043.stdout:7/176: mknod d4/d27/c35 0 2024-09-16T23:48:45.746 INFO:tasks.workunit.client.1.smithi043.stdout:5/320: dread d3/db/f2a [0,4194304] 0 2024-09-16T23:48:45.746 INFO:tasks.workunit.client.1.smithi043.stdout:4/195: dread d8/d15/d20/d2b/f28 [4194304,4194304] 0 2024-09-16T23:48:45.746 INFO:tasks.workunit.client.1.smithi043.stdout:6/157: rename de/d23/c27 to de/df/c36 0 2024-09-16T23:48:45.747 INFO:tasks.workunit.client.1.smithi043.stdout:9/134: dwrite d4/f10 [0,4194304] 0 2024-09-16T23:48:45.749 INFO:tasks.workunit.client.1.smithi043.stdout:0/118: unlink d0/de/f20 0 2024-09-16T23:48:45.754 INFO:tasks.workunit.client.1.smithi043.stdout:5/321: symlink d3/d1b/l66 0 2024-09-16T23:48:45.756 INFO:tasks.workunit.client.1.smithi043.stdout:5/322: write d3/db/d19/f54 [904434,67098] 0 2024-09-16T23:48:45.757 INFO:tasks.workunit.client.1.smithi043.stdout:0/119: mkdir d0/d2/d12/d27 0 2024-09-16T23:48:45.759 INFO:tasks.workunit.client.1.smithi043.stdout:3/119: creat d4/f23 x:0 0 0 2024-09-16T23:48:45.759 INFO:tasks.workunit.client.1.smithi043.stdout:7/177: dwrite d4/db/d11/d21/f2f [0,4194304] 0 2024-09-16T23:48:45.760 INFO:tasks.workunit.client.1.smithi043.stdout:6/158: mkdir de/d37 0 2024-09-16T23:48:45.760 INFO:tasks.workunit.client.1.smithi043.stdout:3/120: write d4/f11 [1126175,40676] 0 2024-09-16T23:48:45.761 INFO:tasks.workunit.client.1.smithi043.stdout:3/121: fsync d4/d17/f1d 0 2024-09-16T23:48:45.764 INFO:tasks.workunit.client.1.smithi043.stdout:0/120: creat d0/d2/d12/f28 x:0 0 0 2024-09-16T23:48:45.764 INFO:tasks.workunit.client.1.smithi043.stdout:3/122: chown d4/de/c13 7031563 1 2024-09-16T23:48:45.771 INFO:tasks.workunit.client.1.smithi043.stdout:6/159: dread f2 [0,4194304] 0 2024-09-16T23:48:45.771 INFO:tasks.workunit.client.1.smithi043.stdout:3/123: rename d4/f11 to d4/de/f24 0 2024-09-16T23:48:45.775 INFO:tasks.workunit.client.1.smithi043.stdout:5/323: link d3/db/d15/d43/c47 d3/db/d15/d43/c67 0 2024-09-16T23:48:45.775 INFO:tasks.workunit.client.1.smithi043.stdout:5/324: chown d3/db/d15/c41 17 1 2024-09-16T23:48:45.779 INFO:tasks.workunit.client.1.smithi043.stdout:0/121: getdents d0/d2/d12 0 2024-09-16T23:48:45.780 INFO:tasks.workunit.client.1.smithi043.stdout:0/122: dread - d0/f24 zero size 2024-09-16T23:48:45.780 INFO:tasks.workunit.client.1.smithi043.stdout:3/124: link c3 d4/de/c25 0 2024-09-16T23:48:45.781 INFO:tasks.workunit.client.1.smithi043.stdout:0/123: chown d0/d2/d12 379416806 1 2024-09-16T23:48:45.782 INFO:tasks.workunit.client.1.smithi043.stdout:5/325: link d3/d9/l49 d3/db/d19/l68 0 2024-09-16T23:48:45.783 INFO:tasks.workunit.client.1.smithi043.stdout:0/124: mknod d0/d2/d12/c29 0 2024-09-16T23:48:45.786 INFO:tasks.workunit.client.1.smithi043.stdout:0/125: rmdir d0/d2/df 39 2024-09-16T23:48:45.787 INFO:tasks.workunit.client.1.smithi043.stdout:0/126: unlink d0/d2/d12/f28 0 2024-09-16T23:48:45.789 INFO:tasks.workunit.client.1.smithi043.stdout:0/127: creat d0/d2/d12/f2a x:0 0 0 2024-09-16T23:48:45.791 INFO:tasks.workunit.client.1.smithi043.stdout:0/128: fdatasync d0/d2/df/f10 0 2024-09-16T23:48:45.792 INFO:tasks.workunit.client.1.smithi043.stdout:0/129: chown d0/l4 1204481404 1 2024-09-16T23:48:45.794 INFO:tasks.workunit.client.1.smithi043.stdout:0/130: creat d0/dc/f2b x:0 0 0 2024-09-16T23:48:45.794 INFO:tasks.workunit.client.1.smithi043.stdout:0/131: fsync d0/d2/df/d1a/f1b 0 2024-09-16T23:48:45.804 INFO:tasks.workunit.client.1.smithi043.stdout:0/132: dwrite d0/de/f15 [0,4194304] 0 2024-09-16T23:48:45.806 INFO:tasks.workunit.client.1.smithi043.stdout:0/133: fsync d0/d2/df/f18 0 2024-09-16T23:48:45.817 INFO:tasks.workunit.client.1.smithi043.stdout:0/134: dwrite d0/d2/df/f16 [0,4194304] 0 2024-09-16T23:48:45.817 INFO:tasks.workunit.client.1.smithi043.stdout:0/135: fdatasync d0/d2/df/f17 0 2024-09-16T23:48:45.818 INFO:tasks.workunit.client.1.smithi043.stdout:0/136: chown d0/d2/df 29803 1 2024-09-16T23:48:45.822 INFO:tasks.workunit.client.1.smithi043.stdout:0/137: dread d0/de/f15 [0,4194304] 0 2024-09-16T23:48:45.824 INFO:tasks.workunit.client.1.smithi043.stdout:0/138: fsync d0/f13 0 2024-09-16T23:48:45.825 INFO:tasks.workunit.client.1.smithi043.stdout:0/139: truncate d0/dc/f2b 9735 0 2024-09-16T23:48:45.828 INFO:tasks.workunit.client.1.smithi043.stdout:0/140: dread d0/de/f15 [0,4194304] 0 2024-09-16T23:48:45.829 INFO:tasks.workunit.client.1.smithi043.stdout:0/141: fdatasync d0/d2/d12/f2a 0 2024-09-16T23:48:45.832 INFO:tasks.workunit.client.1.smithi043.stdout:0/142: dread d0/de/f15 [0,4194304] 0 2024-09-16T23:48:45.832 INFO:tasks.workunit.client.1.smithi043.stdout:0/143: chown d0/d2 0 1 2024-09-16T23:48:45.833 INFO:tasks.workunit.client.1.smithi043.stdout:0/144: read d0/d2/df/d1a/f1b [1839781,91485] 0 2024-09-16T23:48:45.834 INFO:tasks.workunit.client.1.smithi043.stdout:0/145: write d0/d2/df/f10 [14206889,11101] 0 2024-09-16T23:48:45.835 INFO:tasks.workunit.client.1.smithi043.stdout:0/146: write d0/f24 [912596,112981] 0 2024-09-16T23:48:45.836 INFO:tasks.workunit.client.1.smithi043.stdout:0/147: fdatasync d0/d2/df/d1a/f1b 0 2024-09-16T23:48:45.837 INFO:tasks.workunit.client.1.smithi043.stdout:0/148: fsync d0/d2/df/d1a/f1b 0 2024-09-16T23:48:45.839 INFO:tasks.workunit.client.1.smithi043.stdout:0/149: unlink d0/d2/d12/c1f 0 2024-09-16T23:48:45.842 INFO:tasks.workunit.client.1.smithi043.stdout:0/150: dread d0/d2/df/f16 [0,4194304] 0 2024-09-16T23:48:45.844 INFO:tasks.workunit.client.1.smithi043.stdout:0/151: creat d0/d2/df/d1a/f2c x:0 0 0 2024-09-16T23:48:45.844 INFO:tasks.workunit.client.1.smithi043.stdout:0/152: chown d0/d2/d12 1 1 2024-09-16T23:48:45.845 INFO:tasks.workunit.client.1.smithi043.stdout:0/153: creat d0/de/f2d x:0 0 0 2024-09-16T23:48:45.846 INFO:tasks.workunit.client.1.smithi043.stdout:0/154: truncate d0/f13 4352807 0 2024-09-16T23:48:45.849 INFO:tasks.workunit.client.1.smithi043.stdout:0/155: dread d0/d2/df/f16 [0,4194304] 0 2024-09-16T23:48:45.850 INFO:tasks.workunit.client.1.smithi043.stdout:0/156: chown d0/lb 309 1 2024-09-16T23:48:45.853 INFO:tasks.workunit.client.1.smithi043.stdout:0/157: write d0/d2/df/f18 [822556,116400] 0 2024-09-16T23:48:45.853 INFO:tasks.workunit.client.1.smithi043.stdout:0/158: chown d0/d2/d12 148 1 2024-09-16T23:48:45.854 INFO:tasks.workunit.client.1.smithi043.stdout:0/159: chown d0/de/f15 2536063 1 2024-09-16T23:48:45.857 INFO:tasks.workunit.client.1.smithi043.stdout:0/160: link d0/d2/df/f17 d0/d2/df/d1a/f2e 0 2024-09-16T23:48:45.862 INFO:tasks.workunit.client.1.smithi043.stdout:0/161: dwrite d0/f24 [0,4194304] 0 2024-09-16T23:48:45.864 INFO:tasks.workunit.client.1.smithi043.stdout:0/162: creat d0/d2/df/d1a/f2f x:0 0 0 2024-09-16T23:48:45.866 INFO:tasks.workunit.client.1.smithi043.stdout:0/163: truncate d0/d2/d12/f2a 582429 0 2024-09-16T23:48:45.866 INFO:tasks.workunit.client.1.smithi043.stdout:0/164: chown d0/d2/df/l26 127570 1 2024-09-16T23:48:45.873 INFO:tasks.workunit.client.1.smithi043.stdout:0/165: rmdir d0/d2/df/d1a 39 2024-09-16T23:48:45.874 INFO:tasks.workunit.client.1.smithi043.stdout:0/166: write d0/de/f2d [196212,77863] 0 2024-09-16T23:48:45.877 INFO:tasks.workunit.client.1.smithi043.stdout:0/167: creat d0/d2/df/d1a/f30 x:0 0 0 2024-09-16T23:48:45.882 INFO:tasks.workunit.client.1.smithi043.stdout:0/168: dwrite d0/d2/df/f18 [0,4194304] 0 2024-09-16T23:48:45.885 INFO:tasks.workunit.client.1.smithi043.stdout:0/169: creat d0/d2/df/d1a/f31 x:0 0 0 2024-09-16T23:48:45.892 INFO:tasks.workunit.client.1.smithi043.stdout:0/170: truncate d0/d2/df/f16 1084963 0 2024-09-16T23:48:45.894 INFO:tasks.workunit.client.1.smithi043.stdout:0/171: unlink d0/dc/f2b 0 2024-09-16T23:48:45.895 INFO:tasks.workunit.client.1.smithi043.stdout:0/172: dread - d0/d2/df/d1a/f2f zero size 2024-09-16T23:48:45.903 INFO:tasks.workunit.client.1.smithi043.stdout:0/173: dwrite d0/d2/df/d1a/f2f [0,4194304] 0 2024-09-16T23:48:45.909 INFO:tasks.workunit.client.1.smithi043.stdout:0/174: write d0/d2/df/d1a/f2e [1496960,114260] 0 2024-09-16T23:48:45.911 INFO:tasks.workunit.client.1.smithi043.stdout:0/175: creat d0/d2/d12/d27/f32 x:0 0 0 2024-09-16T23:48:45.915 INFO:tasks.workunit.client.1.smithi043.stdout:0/176: rename d0/d2/df/d1a/f1b to d0/d2/f33 0 2024-09-16T23:48:45.915 INFO:tasks.workunit.client.1.smithi043.stdout:0/177: chown d0/d2/l6 144061648 1 2024-09-16T23:48:45.917 INFO:tasks.workunit.client.1.smithi043.stdout:0/178: mkdir d0/d2/df/d1a/d34 0 2024-09-16T23:48:45.923 INFO:tasks.workunit.client.1.smithi043.stdout:0/179: getdents d0/d2/df/d1a 0 2024-09-16T23:48:45.925 INFO:tasks.workunit.client.1.smithi043.stdout:0/180: creat d0/d2/df/d1a/d34/f35 x:0 0 0 2024-09-16T23:48:45.925 INFO:tasks.workunit.client.1.smithi043.stdout:0/181: stat d0/lb 0 2024-09-16T23:48:45.926 INFO:tasks.workunit.client.1.smithi043.stdout:0/182: fsync d0/d2/df/f17 0 2024-09-16T23:48:45.928 INFO:tasks.workunit.client.1.smithi043.stdout:0/183: creat d0/d2/df/d1a/f36 x:0 0 0 2024-09-16T23:48:45.929 INFO:tasks.workunit.client.1.smithi043.stdout:0/184: mkdir d0/de/d37 0 2024-09-16T23:48:45.930 INFO:tasks.workunit.client.1.smithi043.stdout:0/185: write d0/d2/df/d1a/f2f [2580442,30432] 0 2024-09-16T23:48:45.933 INFO:tasks.workunit.client.1.smithi043.stdout:0/186: symlink d0/d2/df/l38 0 2024-09-16T23:48:45.935 INFO:tasks.workunit.client.1.smithi043.stdout:0/187: unlink d0/d2/df/d1a/f2e 0 2024-09-16T23:48:45.939 INFO:tasks.workunit.client.0.smithi042.stderr:++ readlink -f fsstress 2024-09-16T23:48:45.941 INFO:tasks.workunit.client.0.smithi042.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-16T23:48:45.941 INFO:tasks.workunit.client.0.smithi042.stderr:+ popd 2024-09-16T23:48:45.941 INFO:tasks.workunit.client.1.smithi043.stdout:0/188: dwrite d0/d2/df/d1a/f30 [0,4194304] 0 2024-09-16T23:48:45.942 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T23:48:45.942 INFO:tasks.workunit.client.0.smithi042.stderr:+ popd 2024-09-16T23:48:45.942 INFO:tasks.workunit.client.1.smithi043.stdout:0/189: fdatasync d0/f24 0 2024-09-16T23:48:45.943 INFO:tasks.workunit.client.0.smithi042.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-16T23:48:45.943 INFO:tasks.workunit.client.0.smithi042.stderr:++ mktemp -d -p . 2024-09-16T23:48:45.943 INFO:tasks.workunit.client.1.smithi043.stdout:0/190: write d0/d2/df/d1a/d34/f35 [622918,55280] 0 2024-09-16T23:48:45.948 INFO:tasks.workunit.client.1.smithi043.stdout:0/191: rename d0/de/d37 to d0/d2/df/d1a/d34/d39 0 2024-09-16T23:48:45.948 INFO:tasks.workunit.client.0.smithi042.stderr:+ T=./tmp.0BFvYjQHmP 2024-09-16T23:48:45.949 INFO:tasks.workunit.client.0.smithi042.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.0BFvYjQHmP -l 1 -n 1000 -p 10 -v 2024-09-16T23:48:45.950 INFO:tasks.workunit.client.1.smithi043.stdout:0/192: symlink d0/l3a 0 2024-09-16T23:48:45.952 INFO:tasks.workunit.client.0.smithi042.stdout:seed = 1726372488 2024-09-16T23:48:45.957 INFO:tasks.workunit.client.1.smithi043.stdout:0/193: dwrite d0/d2/df/d1a/f2c [0,4194304] 0 2024-09-16T23:48:45.958 INFO:tasks.workunit.client.1.smithi043.stdout:0/194: chown d0/de/f2d 5482 1 2024-09-16T23:48:45.960 INFO:tasks.workunit.client.0.smithi042.stdout:1/0: chown . 82 1 2024-09-16T23:48:45.960 INFO:tasks.workunit.client.0.smithi042.stdout:1/1: truncate - no filename 2024-09-16T23:48:45.960 INFO:tasks.workunit.client.0.smithi042.stdout:1/2: chown . 19 1 2024-09-16T23:48:45.960 INFO:tasks.workunit.client.0.smithi042.stdout:1/3: readlink - no filename 2024-09-16T23:48:45.961 INFO:tasks.workunit.client.0.smithi042.stdout:2/0: link - no file 2024-09-16T23:48:45.962 INFO:tasks.workunit.client.0.smithi042.stdout:1/4: mkdir d0 0 2024-09-16T23:48:45.962 INFO:tasks.workunit.client.0.smithi042.stdout:1/5: unlink - no file 2024-09-16T23:48:45.962 INFO:tasks.workunit.client.0.smithi042.stdout:1/6: readlink - no filename 2024-09-16T23:48:45.962 INFO:tasks.workunit.client.0.smithi042.stdout:1/7: chown d0 4796407 1 2024-09-16T23:48:45.963 INFO:tasks.workunit.client.0.smithi042.stdout:3/0: link - no file 2024-09-16T23:48:45.963 INFO:tasks.workunit.client.0.smithi042.stdout:3/1: rename - no filename 2024-09-16T23:48:45.963 INFO:tasks.workunit.client.0.smithi042.stdout:3/2: read - no filename 2024-09-16T23:48:45.963 INFO:tasks.workunit.client.0.smithi042.stdout:3/3: link - no file 2024-09-16T23:48:45.964 INFO:tasks.workunit.client.0.smithi042.stdout:2/1: mkdir d0 0 2024-09-16T23:48:45.964 INFO:tasks.workunit.client.0.smithi042.stdout:2/2: write - no filename 2024-09-16T23:48:45.976 INFO:tasks.workunit.client.0.smithi042.stdout:1/8: creat d0/f1 x:0 0 0 2024-09-16T23:48:45.976 INFO:tasks.workunit.client.0.smithi042.stdout:1/9: readlink - no filename 2024-09-16T23:48:45.976 INFO:tasks.workunit.client.0.smithi042.stdout:1/10: read - d0/f1 zero size 2024-09-16T23:48:45.981 INFO:tasks.workunit.client.1.smithi043.stdout:2/50: sync 2024-09-16T23:48:45.982 INFO:tasks.workunit.client.1.smithi043.stdout:2/51: dread - d8/fb zero size 2024-09-16T23:48:45.983 INFO:tasks.workunit.client.1.smithi043.stdout:2/52: fdatasync f5 0 2024-09-16T23:48:45.985 INFO:tasks.workunit.client.1.smithi043.stdout:2/53: symlink d8/le 0 2024-09-16T23:48:45.986 INFO:tasks.workunit.client.1.smithi043.stdout:0/195: rmdir d0/d2/df/d1a/d34 39 2024-09-16T23:48:45.987 INFO:tasks.workunit.client.1.smithi043.stdout:2/54: creat d8/ff x:0 0 0 2024-09-16T23:48:45.993 INFO:tasks.workunit.client.1.smithi043.stdout:0/196: dwrite d0/f24 [0,4194304] 0 2024-09-16T23:48:45.996 INFO:tasks.workunit.client.1.smithi043.stdout:2/55: dwrite d8/fc [0,4194304] 0 2024-09-16T23:48:45.997 INFO:tasks.workunit.client.1.smithi043.stdout:2/56: readlink d8/le 0 2024-09-16T23:48:46.003 INFO:tasks.workunit.client.1.smithi043.stdout:2/57: dread f2 [0,4194304] 0 2024-09-16T23:48:46.003 INFO:tasks.workunit.client.1.smithi043.stdout:0/197: rename d0/d2/d12 to d0/d2/df/d1a/d3b 0 2024-09-16T23:48:46.005 INFO:tasks.workunit.client.1.smithi043.stdout:0/198: chown d0/d2/df/d1a/d3b/f2a 701348 1 2024-09-16T23:48:46.010 INFO:tasks.workunit.client.1.smithi043.stdout:0/199: creat d0/f3c x:0 0 0 2024-09-16T23:48:46.011 INFO:tasks.workunit.client.1.smithi043.stdout:2/58: dwrite f5 [0,4194304] 0 2024-09-16T23:48:46.013 INFO:tasks.workunit.client.1.smithi043.stdout:0/200: write d0/d2/df/d1a/f2f [3922777,124754] 0 2024-09-16T23:48:46.013 INFO:tasks.workunit.client.1.smithi043.stdout:0/201: readlink d0/l3a 0 2024-09-16T23:48:46.015 INFO:tasks.workunit.client.1.smithi043.stdout:2/59: creat d8/f10 x:0 0 0 2024-09-16T23:48:46.017 INFO:tasks.workunit.client.1.smithi043.stdout:2/60: mkdir d8/d11 0 2024-09-16T23:48:46.018 INFO:tasks.workunit.client.1.smithi043.stdout:0/202: creat d0/d2/df/d1a/d3b/f3d x:0 0 0 2024-09-16T23:48:46.020 INFO:tasks.workunit.client.1.smithi043.stdout:2/61: truncate f7 922615 0 2024-09-16T23:48:46.021 INFO:tasks.workunit.client.1.smithi043.stdout:2/62: symlink d8/d11/l12 0 2024-09-16T23:48:46.022 INFO:tasks.workunit.client.1.smithi043.stdout:2/63: fdatasync d8/f10 0 2024-09-16T23:48:46.023 INFO:tasks.workunit.client.1.smithi043.stdout:2/64: creat d8/f13 x:0 0 0 2024-09-16T23:48:46.025 INFO:tasks.workunit.client.1.smithi043.stdout:2/65: unlink d8/f13 0 2024-09-16T23:48:46.027 INFO:tasks.workunit.client.1.smithi043.stdout:2/66: unlink f5 0 2024-09-16T23:48:46.033 INFO:tasks.workunit.client.1.smithi043.stdout:2/67: dwrite d8/fb [0,4194304] 0 2024-09-16T23:48:46.133 INFO:tasks.workunit.client.0.smithi042.stdout:3/4: creat f0 x:0 0 0 2024-09-16T23:48:46.133 INFO:tasks.workunit.client.0.smithi042.stdout:3/5: dread - f0 zero size 2024-09-16T23:48:46.134 INFO:tasks.workunit.client.0.smithi042.stdout:5/0: dread - no filename 2024-09-16T23:48:46.134 INFO:tasks.workunit.client.0.smithi042.stdout:1/11: creat d0/f2 x:0 0 0 2024-09-16T23:48:46.134 INFO:tasks.workunit.client.0.smithi042.stdout:5/1: write - no filename 2024-09-16T23:48:46.134 INFO:tasks.workunit.client.0.smithi042.stdout:3/6: dread - f0 zero size 2024-09-16T23:48:46.134 INFO:tasks.workunit.client.0.smithi042.stdout:2/3: mknod d0/c1 0 2024-09-16T23:48:46.135 INFO:tasks.workunit.client.0.smithi042.stdout:4/0: getdents . 0 2024-09-16T23:48:46.135 INFO:tasks.workunit.client.0.smithi042.stdout:4/1: rmdir - no directory 2024-09-16T23:48:46.135 INFO:tasks.workunit.client.0.smithi042.stdout:4/2: write - no filename 2024-09-16T23:48:46.135 INFO:tasks.workunit.client.0.smithi042.stdout:4/3: dwrite - no filename 2024-09-16T23:48:46.135 INFO:tasks.workunit.client.0.smithi042.stdout:4/4: chown . 266518774 1 2024-09-16T23:48:46.135 INFO:tasks.workunit.client.0.smithi042.stdout:4/5: rmdir - no directory 2024-09-16T23:48:46.136 INFO:tasks.workunit.client.0.smithi042.stdout:4/6: dread - no filename 2024-09-16T23:48:46.136 INFO:tasks.workunit.client.0.smithi042.stdout:4/7: link - no file 2024-09-16T23:48:46.136 INFO:tasks.workunit.client.0.smithi042.stdout:4/8: dread - no filename 2024-09-16T23:48:46.136 INFO:tasks.workunit.client.0.smithi042.stdout:6/0: truncate - no filename 2024-09-16T23:48:46.136 INFO:tasks.workunit.client.0.smithi042.stdout:6/1: dwrite - no filename 2024-09-16T23:48:46.136 INFO:tasks.workunit.client.0.smithi042.stdout:6/2: stat - no entries 2024-09-16T23:48:46.137 INFO:tasks.workunit.client.0.smithi042.stdout:2/4: mknod d0/c2 0 2024-09-16T23:48:46.138 INFO:tasks.workunit.client.0.smithi042.stdout:7/0: dread - no filename 2024-09-16T23:48:46.139 INFO:tasks.workunit.client.0.smithi042.stdout:5/2: symlink l0 0 2024-09-16T23:48:46.139 INFO:tasks.workunit.client.0.smithi042.stdout:5/3: truncate - no filename 2024-09-16T23:48:46.139 INFO:tasks.workunit.client.0.smithi042.stdout:4/9: symlink l0 0 2024-09-16T23:48:46.140 INFO:tasks.workunit.client.0.smithi042.stdout:2/5: creat d0/f3 x:0 0 0 2024-09-16T23:48:46.141 INFO:tasks.workunit.client.0.smithi042.stdout:6/3: creat f0 x:0 0 0 2024-09-16T23:48:46.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/4: symlink l1 0 2024-09-16T23:48:46.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/5: write - no filename 2024-09-16T23:48:46.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/6: dwrite - no filename 2024-09-16T23:48:46.143 INFO:tasks.workunit.client.0.smithi042.stdout:4/10: unlink l0 0 2024-09-16T23:48:46.144 INFO:tasks.workunit.client.0.smithi042.stdout:2/6: symlink d0/l4 0 2024-09-16T23:48:46.145 INFO:tasks.workunit.client.0.smithi042.stdout:7/1: symlink l0 0 2024-09-16T23:48:46.146 INFO:tasks.workunit.client.0.smithi042.stdout:9/0: readlink - no filename 2024-09-16T23:48:46.147 INFO:tasks.workunit.client.0.smithi042.stdout:5/7: unlink l1 0 2024-09-16T23:48:46.147 INFO:tasks.workunit.client.0.smithi042.stdout:5/8: truncate - no filename 2024-09-16T23:48:46.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/11: mknod c1 0 2024-09-16T23:48:46.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/12: write - no filename 2024-09-16T23:48:46.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/13: dread - no filename 2024-09-16T23:48:46.149 INFO:tasks.workunit.client.0.smithi042.stdout:6/4: link f0 f1 0 2024-09-16T23:48:46.149 INFO:tasks.workunit.client.0.smithi042.stdout:6/5: dread - f1 zero size 2024-09-16T23:48:46.180 INFO:tasks.workunit.client.1.smithi043.stdout:8/149: sync 2024-09-16T23:48:46.180 INFO:tasks.workunit.client.1.smithi043.stdout:1/212: sync 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.1.smithi043.stdout:8/150: chown l9 1 1 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.1.smithi043.stdout:8/151: creat de/d18/d2b/f2e x:0 0 0 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.0.smithi042.stdout:2/7: rename d0/c2 to d0/c5 0 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.0.smithi042.stdout:7/2: mknod c1 0 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.0.smithi042.stdout:7/3: rmdir - no directory 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.0.smithi042.stdout:8/0: symlink l0 0 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.0.smithi042.stdout:8/1: stat l0 0 2024-09-16T23:48:46.181 INFO:tasks.workunit.client.0.smithi042.stdout:8/2: write - no filename 2024-09-16T23:48:46.182 INFO:tasks.workunit.client.0.smithi042.stdout:5/9: unlink l0 0 2024-09-16T23:48:46.182 INFO:tasks.workunit.client.0.smithi042.stdout:5/10: rename - no filename 2024-09-16T23:48:46.182 INFO:tasks.workunit.client.1.smithi043.stdout:8/152: mkdir de/d18/d1f/d2f 0 2024-09-16T23:48:46.182 INFO:tasks.workunit.client.1.smithi043.stdout:8/153: chown de/d18 8304040 1 2024-09-16T23:48:46.183 INFO:tasks.workunit.client.0.smithi042.stdout:4/14: creat f2 x:0 0 0 2024-09-16T23:48:46.183 INFO:tasks.workunit.client.1.smithi043.stdout:8/154: fdatasync de/f15 0 2024-09-16T23:48:46.185 INFO:tasks.workunit.client.1.smithi043.stdout:8/155: symlink de/d18/d1f/l30 0 2024-09-16T23:48:46.187 INFO:tasks.workunit.client.0.smithi042.stdout:6/6: write f1 [926997,95563] 0 2024-09-16T23:48:46.187 INFO:tasks.workunit.client.1.smithi043.stdout:8/156: symlink de/d18/d1f/l31 0 2024-09-16T23:48:46.189 INFO:tasks.workunit.client.0.smithi042.stdout:9/1: getdents . 0 2024-09-16T23:48:46.189 INFO:tasks.workunit.client.0.smithi042.stdout:9/2: dread - no filename 2024-09-16T23:48:46.189 INFO:tasks.workunit.client.0.smithi042.stdout:9/3: chown . 10839 1 2024-09-16T23:48:46.190 INFO:tasks.workunit.client.1.smithi043.stdout:8/157: creat de/d18/d1f/f32 x:0 0 0 2024-09-16T23:48:46.191 INFO:tasks.workunit.client.0.smithi042.stdout:8/3: rename l0 to l1 0 2024-09-16T23:48:46.191 INFO:tasks.workunit.client.0.smithi042.stdout:8/4: write - no filename 2024-09-16T23:48:46.191 INFO:tasks.workunit.client.1.smithi043.stdout:1/213: dread f1 [4194304,4194304] 0 2024-09-16T23:48:46.192 INFO:tasks.workunit.client.1.smithi043.stdout:8/158: mknod de/d10/c33 0 2024-09-16T23:48:46.192 INFO:tasks.workunit.client.0.smithi042.stdout:5/11: creat f2 x:0 0 0 2024-09-16T23:48:46.196 INFO:tasks.workunit.client.0.smithi042.stdout:6/7: rename f0 to f2 0 2024-09-16T23:48:46.200 INFO:tasks.workunit.client.1.smithi043.stdout:8/159: dwrite de/f15 [0,4194304] 0 2024-09-16T23:48:46.200 INFO:tasks.workunit.client.0.smithi042.stdout:5/12: dwrite f2 [0,4194304] 0 2024-09-16T23:48:46.200 INFO:tasks.workunit.client.0.smithi042.stdout:5/13: stat f2 0 2024-09-16T23:48:46.201 INFO:tasks.workunit.client.1.smithi043.stdout:1/214: dwrite d4/d9/d12/f10 [0,4194304] 0 2024-09-16T23:48:46.202 INFO:tasks.workunit.client.0.smithi042.stdout:9/4: creat f0 x:0 0 0 2024-09-16T23:48:46.204 INFO:tasks.workunit.client.0.smithi042.stdout:4/15: link c1 c3 0 2024-09-16T23:48:46.205 INFO:tasks.workunit.client.1.smithi043.stdout:1/215: symlink d4/d3e/l43 0 2024-09-16T23:48:46.208 INFO:tasks.workunit.client.0.smithi042.stdout:9/5: mknod c1 0 2024-09-16T23:48:46.212 INFO:tasks.workunit.client.0.smithi042.stdout:6/8: link f2 f3 0 2024-09-16T23:48:46.213 INFO:tasks.workunit.client.0.smithi042.stdout:4/16: symlink l4 0 2024-09-16T23:48:46.216 INFO:tasks.workunit.client.0.smithi042.stdout:6/9: symlink l4 0 2024-09-16T23:48:46.216 INFO:tasks.workunit.client.1.smithi043.stdout:1/216: getdents d4/df/d13/d1b 0 2024-09-16T23:48:46.218 INFO:tasks.workunit.client.0.smithi042.stdout:4/17: symlink l5 0 2024-09-16T23:48:46.220 INFO:tasks.workunit.client.0.smithi042.stdout:6/10: creat f5 x:0 0 0 2024-09-16T23:48:46.220 INFO:tasks.workunit.client.0.smithi042.stdout:6/11: rmdir - no directory 2024-09-16T23:48:46.220 INFO:tasks.workunit.client.1.smithi043.stdout:1/217: mknod d4/df/d26/c44 0 2024-09-16T23:48:46.221 INFO:tasks.workunit.client.0.smithi042.stdout:4/18: creat f6 x:0 0 0 2024-09-16T23:48:46.221 INFO:tasks.workunit.client.1.smithi043.stdout:8/160: truncate de/f23 2779685 0 2024-09-16T23:48:46.222 INFO:tasks.workunit.client.0.smithi042.stdout:9/6: dwrite f0 [0,4194304] 0 2024-09-16T23:48:46.224 INFO:tasks.workunit.client.0.smithi042.stdout:4/19: creat f7 x:0 0 0 2024-09-16T23:48:46.224 INFO:tasks.workunit.client.1.smithi043.stdout:1/218: getdents d4/d9/d1c/d1e/d42 0 2024-09-16T23:48:46.226 INFO:tasks.workunit.client.0.smithi042.stdout:9/7: dread f0 [0,4194304] 0 2024-09-16T23:48:46.226 INFO:tasks.workunit.client.1.smithi043.stdout:1/219: write d4/d34/f3a [394612,67566] 0 2024-09-16T23:48:46.227 INFO:tasks.workunit.client.0.smithi042.stdout:6/12: link f5 f6 0 2024-09-16T23:48:46.227 INFO:tasks.workunit.client.0.smithi042.stdout:6/13: chown f2 127507748 1 2024-09-16T23:48:46.228 INFO:tasks.workunit.client.0.smithi042.stdout:4/20: symlink l8 0 2024-09-16T23:48:46.228 INFO:tasks.workunit.client.1.smithi043.stdout:1/220: fsync d4/df/d13/d1b/f25 0 2024-09-16T23:48:46.229 INFO:tasks.workunit.client.1.smithi043.stdout:8/161: dwrite de/f15 [0,4194304] 0 2024-09-16T23:48:46.229 INFO:tasks.workunit.client.0.smithi042.stdout:4/21: write f6 [745765,113374] 0 2024-09-16T23:48:46.232 INFO:tasks.workunit.client.1.smithi043.stdout:8/162: mkdir de/d18/d27/d34 0 2024-09-16T23:48:46.232 INFO:tasks.workunit.client.0.smithi042.stdout:9/8: rename f0 to f2 0 2024-09-16T23:48:46.234 INFO:tasks.workunit.client.0.smithi042.stdout:6/14: fdatasync f5 0 2024-09-16T23:48:46.236 INFO:tasks.workunit.client.0.smithi042.stdout:4/22: mknod c9 0 2024-09-16T23:48:46.237 INFO:tasks.workunit.client.0.smithi042.stdout:4/23: truncate f7 203077 0 2024-09-16T23:48:46.238 INFO:tasks.workunit.client.0.smithi042.stdout:9/9: dread f2 [0,4194304] 0 2024-09-16T23:48:46.238 INFO:tasks.workunit.client.0.smithi042.stdout:4/24: dread f7 [0,4194304] 0 2024-09-16T23:48:46.240 INFO:tasks.workunit.client.0.smithi042.stdout:4/25: creat fa x:0 0 0 2024-09-16T23:48:46.240 INFO:tasks.workunit.client.1.smithi043.stdout:1/221: dwrite d4/d9/d12/d15/f31 [0,4194304] 0 2024-09-16T23:48:46.241 INFO:tasks.workunit.client.0.smithi042.stdout:9/10: symlink l3 0 2024-09-16T23:48:46.242 INFO:tasks.workunit.client.1.smithi043.stdout:1/222: chown d4/df 5952008 1 2024-09-16T23:48:46.245 INFO:tasks.workunit.client.1.smithi043.stdout:8/163: dwrite de/f14 [0,4194304] 0 2024-09-16T23:48:46.245 INFO:tasks.workunit.client.0.smithi042.stdout:4/26: unlink f6 0 2024-09-16T23:48:46.246 INFO:tasks.workunit.client.1.smithi043.stdout:1/223: creat d4/d9/d12/d2b/f45 x:0 0 0 2024-09-16T23:48:46.246 INFO:tasks.workunit.client.1.smithi043.stdout:8/164: write de/f15 [641357,40908] 0 2024-09-16T23:48:46.249 INFO:tasks.workunit.client.1.smithi043.stdout:1/224: write d4/df/d13/d1b/f25 [5287951,38403] 0 2024-09-16T23:48:46.249 INFO:tasks.workunit.client.0.smithi042.stdout:9/11: write f2 [2839487,90325] 0 2024-09-16T23:48:46.250 INFO:tasks.workunit.client.1.smithi043.stdout:8/165: symlink de/d18/d2b/l35 0 2024-09-16T23:48:46.252 INFO:tasks.workunit.client.1.smithi043.stdout:8/166: chown de/d18/c1e 992 1 2024-09-16T23:48:46.253 INFO:tasks.workunit.client.0.smithi042.stdout:4/27: symlink lb 0 2024-09-16T23:48:46.254 INFO:tasks.workunit.client.0.smithi042.stdout:9/12: mkdir d4 0 2024-09-16T23:48:46.256 INFO:tasks.workunit.client.1.smithi043.stdout:8/167: rename f8 to de/d18/f36 0 2024-09-16T23:48:46.259 INFO:tasks.workunit.client.0.smithi042.stdout:9/13: creat d4/f5 x:0 0 0 2024-09-16T23:48:46.260 INFO:tasks.workunit.client.1.smithi043.stdout:8/168: dread de/d10/f13 [0,4194304] 0 2024-09-16T23:48:46.261 INFO:tasks.workunit.client.0.smithi042.stdout:9/14: mknod d4/c6 0 2024-09-16T23:48:46.263 INFO:tasks.workunit.client.1.smithi043.stdout:8/169: mkdir de/d10/d37 0 2024-09-16T23:48:46.263 INFO:tasks.workunit.client.0.smithi042.stdout:9/15: mknod d4/c7 0 2024-09-16T23:48:46.264 INFO:tasks.workunit.client.1.smithi043.stdout:8/170: symlink de/d10/l38 0 2024-09-16T23:48:46.266 INFO:tasks.workunit.client.0.smithi042.stdout:9/16: dread f2 [0,4194304] 0 2024-09-16T23:48:46.267 INFO:tasks.workunit.client.0.smithi042.stdout:9/17: readlink l3 0 2024-09-16T23:48:46.267 INFO:tasks.workunit.client.1.smithi043.stdout:1/225: dread f1 [0,4194304] 0 2024-09-16T23:48:46.267 INFO:tasks.workunit.client.1.smithi043.stdout:8/171: mkdir de/d18/d1f/d2f/d39 0 2024-09-16T23:48:46.269 INFO:tasks.workunit.client.1.smithi043.stdout:8/172: mknod de/d18/d1f/c3a 0 2024-09-16T23:48:46.270 INFO:tasks.workunit.client.0.smithi042.stdout:9/18: link l3 d4/l8 0 2024-09-16T23:48:46.272 INFO:tasks.workunit.client.1.smithi043.stdout:8/173: creat de/d18/d1f/d25/f3b x:0 0 0 2024-09-16T23:48:46.275 INFO:tasks.workunit.client.1.smithi043.stdout:1/226: dwrite d4/df/f38 [0,4194304] 0 2024-09-16T23:48:46.277 INFO:tasks.workunit.client.1.smithi043.stdout:1/227: chown d4/d9/d12/d15/c16 1 1 2024-09-16T23:48:46.283 INFO:tasks.workunit.client.1.smithi043.stdout:8/174: dwrite de/f15 [0,4194304] 0 2024-09-16T23:48:46.286 INFO:tasks.workunit.client.1.smithi043.stdout:8/175: symlink de/d18/l3c 0 2024-09-16T23:48:46.295 INFO:tasks.workunit.client.1.smithi043.stdout:1/228: dread d4/fc [0,4194304] 0 2024-09-16T23:48:46.298 INFO:tasks.workunit.client.1.smithi043.stdout:1/229: creat d4/d9/f46 x:0 0 0 2024-09-16T23:48:46.300 INFO:tasks.workunit.client.1.smithi043.stdout:1/230: mkdir d4/d9/d1c/d1e/d47 0 2024-09-16T23:48:46.301 INFO:tasks.workunit.client.1.smithi043.stdout:1/231: dread - d4/df/d13/d1b/f2f zero size 2024-09-16T23:48:46.307 INFO:tasks.workunit.client.1.smithi043.stdout:1/232: truncate d4/df/d13/f40 3193274 0 2024-09-16T23:48:46.308 INFO:tasks.workunit.client.1.smithi043.stdout:1/233: read - d4/d9/d12/d2b/f45 zero size 2024-09-16T23:48:46.308 INFO:tasks.workunit.client.1.smithi043.stdout:1/234: chown d4/d9/d12/f10 130115 1 2024-09-16T23:48:46.311 INFO:tasks.workunit.client.1.smithi043.stdout:1/235: getdents d4/d34 0 2024-09-16T23:48:46.323 INFO:tasks.workunit.client.1.smithi043.stdout:1/236: dread d4/d9/d1c/d1e/f23 [0,4194304] 0 2024-09-16T23:48:46.325 INFO:tasks.workunit.client.1.smithi043.stdout:1/237: mknod d4/df/d13/c48 0 2024-09-16T23:48:46.327 INFO:tasks.workunit.client.1.smithi043.stdout:1/238: symlink d4/d9/d1c/d1e/d47/l49 0 2024-09-16T23:48:46.330 INFO:tasks.workunit.client.1.smithi043.stdout:1/239: link d4/df/d26/l2c d4/d9/d12/d15/l4a 0 2024-09-16T23:48:46.332 INFO:tasks.workunit.client.1.smithi043.stdout:1/240: mknod d4/d9/d1c/d1e/d47/c4b 0 2024-09-16T23:48:46.333 INFO:tasks.workunit.client.0.smithi042.stdout:1/12: rmdir d0 39 2024-09-16T23:48:46.334 INFO:tasks.workunit.client.1.smithi043.stdout:1/241: symlink d4/d9/l4c 0 2024-09-16T23:48:46.336 INFO:tasks.workunit.client.1.smithi043.stdout:1/242: creat d4/d34/f4d x:0 0 0 2024-09-16T23:48:46.337 INFO:tasks.workunit.client.0.smithi042.stdout:1/13: getdents d0 0 2024-09-16T23:48:46.338 INFO:tasks.workunit.client.1.smithi043.stdout:1/243: creat d4/d9/f4e x:0 0 0 2024-09-16T23:48:46.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/14: rename d0/f2 to d0/f3 0 2024-09-16T23:48:46.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/15: rename d0 to d0/d4 22 2024-09-16T23:48:46.342 INFO:tasks.workunit.client.1.smithi043.stdout:1/244: dread d4/fc [0,4194304] 0 2024-09-16T23:48:46.344 INFO:tasks.workunit.client.1.smithi043.stdout:1/245: creat d4/d9/d1c/d1e/d47/f4f x:0 0 0 2024-09-16T23:48:46.345 INFO:tasks.workunit.client.0.smithi042.stdout:2/8: rmdir d0 39 2024-09-16T23:48:46.345 INFO:tasks.workunit.client.1.smithi043.stdout:1/246: mknod d4/d3e/c50 0 2024-09-16T23:48:46.346 INFO:tasks.workunit.client.0.smithi042.stdout:7/4: getdents . 0 2024-09-16T23:48:46.346 INFO:tasks.workunit.client.0.smithi042.stdout:7/5: dread - no filename 2024-09-16T23:48:46.347 INFO:tasks.workunit.client.1.smithi043.stdout:1/247: fsync d4/d9/d12/f33 0 2024-09-16T23:48:46.348 INFO:tasks.workunit.client.0.smithi042.stdout:2/9: write d0/f3 [1036247,119199] 0 2024-09-16T23:48:46.348 INFO:tasks.workunit.client.0.smithi042.stdout:2/10: readlink d0/l4 0 2024-09-16T23:48:46.348 INFO:tasks.workunit.client.0.smithi042.stdout:7/6: rename l0 to l2 0 2024-09-16T23:48:46.349 INFO:tasks.workunit.client.1.smithi043.stdout:1/248: dread - d4/d9/f4e zero size 2024-09-16T23:48:46.349 INFO:tasks.workunit.client.1.smithi043.stdout:1/249: readlink d4/l8 0 2024-09-16T23:48:46.350 INFO:tasks.workunit.client.0.smithi042.stdout:2/11: fdatasync d0/f3 0 2024-09-16T23:48:46.351 INFO:tasks.workunit.client.0.smithi042.stdout:7/7: mkdir d3 0 2024-09-16T23:48:46.353 INFO:tasks.workunit.client.1.smithi043.stdout:1/250: dread f1 [4194304,4194304] 0 2024-09-16T23:48:46.353 INFO:tasks.workunit.client.0.smithi042.stdout:8/5: chown l1 0 1 2024-09-16T23:48:46.353 INFO:tasks.workunit.client.0.smithi042.stdout:2/12: dread d0/f3 [0,4194304] 0 2024-09-16T23:48:46.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/8: mknod d3/c4 0 2024-09-16T23:48:46.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/9: write - no filename 2024-09-16T23:48:46.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/10: fsync - no filename 2024-09-16T23:48:46.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/11: stat d3 0 2024-09-16T23:48:46.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/12: read - no filename 2024-09-16T23:48:46.355 INFO:tasks.workunit.client.0.smithi042.stdout:7/13: write - no filename 2024-09-16T23:48:46.355 INFO:tasks.workunit.client.1.smithi043.stdout:1/251: symlink d4/d9/d12/l51 0 2024-09-16T23:48:46.356 INFO:tasks.workunit.client.0.smithi042.stdout:6/15: unlink f2 0 2024-09-16T23:48:46.357 INFO:tasks.workunit.client.0.smithi042.stdout:8/6: mknod c2 0 2024-09-16T23:48:46.358 INFO:tasks.workunit.client.1.smithi043.stdout:1/252: write d4/d9/d12/f1f [3718568,2057] 0 2024-09-16T23:48:46.358 INFO:tasks.workunit.client.0.smithi042.stdout:2/13: rmdir d0 39 2024-09-16T23:48:46.360 INFO:tasks.workunit.client.0.smithi042.stdout:7/14: rename c1 to d3/c5 0 2024-09-16T23:48:46.360 INFO:tasks.workunit.client.0.smithi042.stdout:7/15: fdatasync - no filename 2024-09-16T23:48:46.360 INFO:tasks.workunit.client.0.smithi042.stdout:7/16: dwrite - no filename 2024-09-16T23:48:46.360 INFO:tasks.workunit.client.1.smithi043.stdout:1/253: truncate d4/df/d26/f39 4141482 0 2024-09-16T23:48:46.361 INFO:tasks.workunit.client.0.smithi042.stdout:6/16: creat f7 x:0 0 0 2024-09-16T23:48:46.361 INFO:tasks.workunit.client.0.smithi042.stdout:5/14: write f2 [4847304,14345] 0 2024-09-16T23:48:46.362 INFO:tasks.workunit.client.0.smithi042.stdout:8/7: symlink l3 0 2024-09-16T23:48:46.363 INFO:tasks.workunit.client.1.smithi043.stdout:1/254: write d4/d9/d12/f33 [2062576,47026] 0 2024-09-16T23:48:46.365 INFO:tasks.workunit.client.1.smithi043.stdout:1/255: write d4/d9/d12/f1f [2840861,67785] 0 2024-09-16T23:48:46.365 INFO:tasks.workunit.client.0.smithi042.stdout:2/14: chown d0/f3 239 1 2024-09-16T23:48:46.367 INFO:tasks.workunit.client.0.smithi042.stdout:8/8: mknod c4 0 2024-09-16T23:48:46.367 INFO:tasks.workunit.client.0.smithi042.stdout:8/9: rmdir - no directory 2024-09-16T23:48:46.367 INFO:tasks.workunit.client.1.smithi043.stdout:1/256: creat d4/d9/d12/d2b/f52 x:0 0 0 2024-09-16T23:48:46.371 INFO:tasks.workunit.client.0.smithi042.stdout:8/10: rename c2 to c5 0 2024-09-16T23:48:46.371 INFO:tasks.workunit.client.0.smithi042.stdout:4/28: fsync f7 0 2024-09-16T23:48:46.374 INFO:tasks.workunit.client.0.smithi042.stdout:4/29: dread - fa zero size 2024-09-16T23:48:46.374 INFO:tasks.workunit.client.1.smithi043.stdout:1/257: dwrite d4/d9/d12/f10 [0,4194304] 0 2024-09-16T23:48:46.375 INFO:tasks.workunit.client.0.smithi042.stdout:5/15: dwrite f2 [0,4194304] 0 2024-09-16T23:48:46.376 INFO:tasks.workunit.client.0.smithi042.stdout:8/11: creat f6 x:0 0 0 2024-09-16T23:48:46.376 INFO:tasks.workunit.client.0.smithi042.stdout:4/30: dread - fa zero size 2024-09-16T23:48:46.379 INFO:tasks.workunit.client.0.smithi042.stdout:8/12: write f6 [190025,47844] 0 2024-09-16T23:48:46.381 INFO:tasks.workunit.client.0.smithi042.stdout:4/31: write f2 [604573,50306] 0 2024-09-16T23:48:46.383 INFO:tasks.workunit.client.0.smithi042.stdout:2/15: dwrite d0/f3 [0,4194304] 0 2024-09-16T23:48:46.384 INFO:tasks.workunit.client.0.smithi042.stdout:6/17: dwrite f6 [0,4194304] 0 2024-09-16T23:48:46.386 INFO:tasks.workunit.client.0.smithi042.stdout:8/13: write f6 [773657,66332] 0 2024-09-16T23:48:46.387 INFO:tasks.workunit.client.0.smithi042.stdout:6/18: write f5 [3196029,102049] 0 2024-09-16T23:48:46.390 INFO:tasks.workunit.client.0.smithi042.stdout:9/19: fsync d4/f5 0 2024-09-16T23:48:46.393 INFO:tasks.workunit.client.0.smithi042.stdout:1/16: getdents d0 0 2024-09-16T23:48:46.399 INFO:tasks.workunit.client.0.smithi042.stdout:9/20: creat d4/f9 x:0 0 0 2024-09-16T23:48:46.400 INFO:tasks.workunit.client.0.smithi042.stdout:2/16: dread d0/f3 [0,4194304] 0 2024-09-16T23:48:46.400 INFO:tasks.workunit.client.0.smithi042.stdout:4/32: dwrite f7 [0,4194304] 0 2024-09-16T23:48:46.407 INFO:tasks.workunit.client.0.smithi042.stdout:6/19: mkdir d8 0 2024-09-16T23:48:46.410 INFO:tasks.workunit.client.0.smithi042.stdout:8/14: chown c5 147159895 1 2024-09-16T23:48:46.410 INFO:tasks.workunit.client.0.smithi042.stdout:8/15: stat c5 0 2024-09-16T23:48:46.410 INFO:tasks.workunit.client.0.smithi042.stdout:5/16: getdents . 0 2024-09-16T23:48:46.411 INFO:tasks.workunit.client.0.smithi042.stdout:9/21: rename d4/f9 to d4/fa 0 2024-09-16T23:48:46.412 INFO:tasks.workunit.client.0.smithi042.stdout:4/33: creat fc x:0 0 0 2024-09-16T23:48:46.412 INFO:tasks.workunit.client.0.smithi042.stdout:4/34: chown c9 226875 1 2024-09-16T23:48:46.412 INFO:tasks.workunit.client.0.smithi042.stdout:2/17: mknod d0/c6 0 2024-09-16T23:48:46.414 INFO:tasks.workunit.client.0.smithi042.stdout:1/17: getdents d0 0 2024-09-16T23:48:46.419 INFO:tasks.workunit.client.0.smithi042.stdout:8/16: mknod c7 0 2024-09-16T23:48:46.420 INFO:tasks.workunit.client.0.smithi042.stdout:5/17: rename f2 to f3 0 2024-09-16T23:48:46.420 INFO:tasks.workunit.client.0.smithi042.stdout:5/18: read f3 [3394781,3120] 0 2024-09-16T23:48:46.421 INFO:tasks.workunit.client.0.smithi042.stdout:4/35: creat fd x:0 0 0 2024-09-16T23:48:46.422 INFO:tasks.workunit.client.0.smithi042.stdout:1/18: creat d0/f5 x:0 0 0 2024-09-16T23:48:46.423 INFO:tasks.workunit.client.0.smithi042.stdout:7/17: link l2 d3/l6 0 2024-09-16T23:48:46.423 INFO:tasks.workunit.client.0.smithi042.stdout:7/18: write - no filename 2024-09-16T23:48:46.423 INFO:tasks.workunit.client.0.smithi042.stdout:7/19: dread - no filename 2024-09-16T23:48:46.423 INFO:tasks.workunit.client.0.smithi042.stdout:7/20: fsync - no filename 2024-09-16T23:48:46.424 INFO:tasks.workunit.client.0.smithi042.stdout:8/17: rename c4 to c8 0 2024-09-16T23:48:46.425 INFO:tasks.workunit.client.0.smithi042.stdout:5/19: mknod c4 0 2024-09-16T23:48:46.427 INFO:tasks.workunit.client.0.smithi042.stdout:4/36: mkdir de 0 2024-09-16T23:48:46.428 INFO:tasks.workunit.client.0.smithi042.stdout:1/19: creat d0/f6 x:0 0 0 2024-09-16T23:48:46.429 INFO:tasks.workunit.client.0.smithi042.stdout:7/21: rmdir d3 39 2024-09-16T23:48:46.429 INFO:tasks.workunit.client.1.smithi043.stdout:7/178: sync 2024-09-16T23:48:46.430 INFO:tasks.workunit.client.1.smithi043.stdout:9/135: sync 2024-09-16T23:48:46.430 INFO:tasks.workunit.client.1.smithi043.stdout:4/196: sync 2024-09-16T23:48:46.431 INFO:tasks.workunit.client.0.smithi042.stdout:6/20: write f1 [786772,95922] 0 2024-09-16T23:48:46.431 INFO:tasks.workunit.client.1.smithi043.stdout:7/179: rmdir d4/d27 39 2024-09-16T23:48:46.431 INFO:tasks.workunit.client.1.smithi043.stdout:9/136: truncate d4/d1c/f20 497500 0 2024-09-16T23:48:46.433 INFO:tasks.workunit.client.1.smithi043.stdout:4/197: creat d8/d25/d39/f3a x:0 0 0 2024-09-16T23:48:46.433 INFO:tasks.workunit.client.0.smithi042.stdout:1/20: symlink d0/l7 0 2024-09-16T23:48:46.434 INFO:tasks.workunit.client.0.smithi042.stdout:8/18: dread f6 [0,4194304] 0 2024-09-16T23:48:46.434 INFO:tasks.workunit.client.0.smithi042.stdout:9/22: dwrite d4/fa [0,4194304] 0 2024-09-16T23:48:46.435 INFO:tasks.workunit.client.0.smithi042.stdout:2/18: dwrite d0/f3 [0,4194304] 0 2024-09-16T23:48:46.436 INFO:tasks.workunit.client.0.smithi042.stdout:7/22: chown d3/c4 1055571971 1 2024-09-16T23:48:46.436 INFO:tasks.workunit.client.0.smithi042.stdout:7/23: dread - no filename 2024-09-16T23:48:46.437 INFO:tasks.workunit.client.1.smithi043.stdout:4/198: dread - d8/d15/d20/d2b/d23/f38 zero size 2024-09-16T23:48:46.437 INFO:tasks.workunit.client.0.smithi042.stdout:8/19: read f6 [141069,15580] 0 2024-09-16T23:48:46.438 INFO:tasks.workunit.client.0.smithi042.stdout:1/21: dread - d0/f3 zero size 2024-09-16T23:48:46.440 INFO:tasks.workunit.client.1.smithi043.stdout:4/199: read - d8/d25/d39/f3a zero size 2024-09-16T23:48:46.443 INFO:tasks.workunit.client.1.smithi043.stdout:7/180: dwrite d4/f2b [0,4194304] 0 2024-09-16T23:48:46.443 INFO:tasks.workunit.client.0.smithi042.stdout:9/23: creat d4/fb x:0 0 0 2024-09-16T23:48:46.443 INFO:tasks.workunit.client.0.smithi042.stdout:8/20: chown c7 0 1 2024-09-16T23:48:46.445 INFO:tasks.workunit.client.1.smithi043.stdout:9/137: dwrite d4/d1c/f27 [0,4194304] 0 2024-09-16T23:48:46.446 INFO:tasks.workunit.client.0.smithi042.stdout:2/19: rename d0/l4 to d0/l7 0 2024-09-16T23:48:46.449 INFO:tasks.workunit.client.0.smithi042.stdout:5/20: dwrite f3 [0,4194304] 0 2024-09-16T23:48:46.449 INFO:tasks.workunit.client.1.smithi043.stdout:7/181: mkdir d4/db/d11/d21/d36 0 2024-09-16T23:48:46.450 INFO:tasks.workunit.client.0.smithi042.stdout:4/37: dwrite f7 [0,4194304] 0 2024-09-16T23:48:46.450 INFO:tasks.workunit.client.0.smithi042.stdout:1/22: write d0/f3 [699838,58713] 0 2024-09-16T23:48:46.450 INFO:tasks.workunit.client.1.smithi043.stdout:7/182: dread - d4/fc zero size 2024-09-16T23:48:46.455 INFO:tasks.workunit.client.1.smithi043.stdout:4/200: link d8/d15/d19/f2a d8/f3b 0 2024-09-16T23:48:46.456 INFO:tasks.workunit.client.1.smithi043.stdout:4/201: stat d8/fc 0 2024-09-16T23:48:46.464 INFO:tasks.workunit.client.1.smithi043.stdout:9/138: getdents d4 0 2024-09-16T23:48:46.465 INFO:tasks.workunit.client.0.smithi042.stdout:1/23: dwrite d0/f3 [0,4194304] 0 2024-09-16T23:48:46.468 INFO:tasks.workunit.client.1.smithi043.stdout:4/202: rename d8/d15/f1f to d8/d15/f3c 0 2024-09-16T23:48:46.468 INFO:tasks.workunit.client.0.smithi042.stdout:2/20: unlink d0/f3 0 2024-09-16T23:48:46.470 INFO:tasks.workunit.client.1.smithi043.stdout:4/203: write d8/d15/d20/d2b/f26 [331826,7564] 0 2024-09-16T23:48:46.470 INFO:tasks.workunit.client.1.smithi043.stdout:9/139: symlink d4/l29 0 2024-09-16T23:48:46.472 INFO:tasks.workunit.client.0.smithi042.stdout:5/21: symlink l5 0 2024-09-16T23:48:46.472 INFO:tasks.workunit.client.0.smithi042.stdout:6/21: truncate f6 1404266 0 2024-09-16T23:48:46.473 INFO:tasks.workunit.client.0.smithi042.stdout:5/22: stat c4 0 2024-09-16T23:48:46.473 INFO:tasks.workunit.client.0.smithi042.stdout:6/22: stat f1 0 2024-09-16T23:48:46.474 INFO:tasks.workunit.client.0.smithi042.stdout:1/24: mknod d0/c8 0 2024-09-16T23:48:46.475 INFO:tasks.workunit.client.1.smithi043.stdout:7/183: write d4/fc [698829,57537] 0 2024-09-16T23:48:46.475 INFO:tasks.workunit.client.0.smithi042.stdout:9/24: dwrite f2 [0,4194304] 0 2024-09-16T23:48:46.476 INFO:tasks.workunit.client.1.smithi043.stdout:9/140: mkdir d4/d21/d2a 0 2024-09-16T23:48:46.477 INFO:tasks.workunit.client.0.smithi042.stdout:7/24: link d3/l6 d3/l7 0 2024-09-16T23:48:46.477 INFO:tasks.workunit.client.0.smithi042.stdout:2/21: mknod d0/c8 0 2024-09-16T23:48:46.478 INFO:tasks.workunit.client.0.smithi042.stdout:8/21: getdents . 0 2024-09-16T23:48:46.478 INFO:tasks.workunit.client.0.smithi042.stdout:5/23: rename f3 to f6 0 2024-09-16T23:48:46.479 INFO:tasks.workunit.client.0.smithi042.stdout:5/24: rmdir - no directory 2024-09-16T23:48:46.480 INFO:tasks.workunit.client.1.smithi043.stdout:9/141: mkdir d4/d1c/d25/d2b 0 2024-09-16T23:48:46.481 INFO:tasks.workunit.client.0.smithi042.stdout:9/25: mkdir d4/dc 0 2024-09-16T23:48:46.483 INFO:tasks.workunit.client.1.smithi043.stdout:9/142: mkdir d4/d21/d2a/d2c 0 2024-09-16T23:48:46.484 INFO:tasks.workunit.client.0.smithi042.stdout:8/22: write f6 [1110792,45047] 0 2024-09-16T23:48:46.485 INFO:tasks.workunit.client.0.smithi042.stdout:4/38: fsync f7 0 2024-09-16T23:48:46.487 INFO:tasks.workunit.client.0.smithi042.stdout:4/39: dread - fa zero size 2024-09-16T23:48:46.488 INFO:tasks.workunit.client.1.smithi043.stdout:9/143: creat d4/d21/d2a/d2c/f2d x:0 0 0 2024-09-16T23:48:46.489 INFO:tasks.workunit.client.0.smithi042.stdout:8/23: symlink l9 0 2024-09-16T23:48:46.490 INFO:tasks.workunit.client.1.smithi043.stdout:4/204: dread d8/d15/f21 [4194304,4194304] 0 2024-09-16T23:48:46.491 INFO:tasks.workunit.client.0.smithi042.stdout:6/23: dwrite f1 [0,4194304] 0 2024-09-16T23:48:46.491 INFO:tasks.workunit.client.1.smithi043.stdout:4/205: chown d8/d15/d20/d2b/f28 106557991 1 2024-09-16T23:48:46.492 INFO:tasks.workunit.client.0.smithi042.stdout:9/26: dread d4/fa [0,4194304] 0 2024-09-16T23:48:46.492 INFO:tasks.workunit.client.1.smithi043.stdout:9/144: rename d4/d12 to d4/d28/d2e 0 2024-09-16T23:48:46.492 INFO:tasks.workunit.client.1.smithi043.stdout:4/206: chown c4 1 1 2024-09-16T23:48:46.493 INFO:tasks.workunit.client.0.smithi042.stdout:8/24: mkdir da 0 2024-09-16T23:48:46.494 INFO:tasks.workunit.client.1.smithi043.stdout:9/145: write d4/d21/d2a/d2c/f2d [53571,29725] 0 2024-09-16T23:48:46.495 INFO:tasks.workunit.client.0.smithi042.stdout:4/40: mknod de/cf 0 2024-09-16T23:48:46.498 INFO:tasks.workunit.client.1.smithi043.stdout:4/207: mkdir d8/d3d 0 2024-09-16T23:48:46.498 INFO:tasks.workunit.client.0.smithi042.stdout:8/25: unlink l1 0 2024-09-16T23:48:46.499 INFO:tasks.workunit.client.0.smithi042.stdout:2/22: rmdir d0 39 2024-09-16T23:48:46.500 INFO:tasks.workunit.client.1.smithi043.stdout:9/146: write d4/f1a [300513,97185] 0 2024-09-16T23:48:46.500 INFO:tasks.workunit.client.0.smithi042.stdout:8/26: fdatasync f6 0 2024-09-16T23:48:46.500 INFO:tasks.workunit.client.0.smithi042.stdout:6/24: mkdir d8/d9 0 2024-09-16T23:48:46.502 INFO:tasks.workunit.client.1.smithi043.stdout:7/184: dwrite d4/db/d11/d21/d31/d33/f1f [0,4194304] 0 2024-09-16T23:48:46.502 INFO:tasks.workunit.client.0.smithi042.stdout:8/27: write f6 [1126200,23135] 0 2024-09-16T23:48:46.503 INFO:tasks.workunit.client.0.smithi042.stdout:8/28: readlink l9 0 2024-09-16T23:48:46.503 INFO:tasks.workunit.client.0.smithi042.stdout:9/27: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:46.506 INFO:tasks.workunit.client.0.smithi042.stdout:4/41: rename fc to de/f10 0 2024-09-16T23:48:46.506 INFO:tasks.workunit.client.0.smithi042.stdout:8/29: write f6 [452623,9780] 0 2024-09-16T23:48:46.506 INFO:tasks.workunit.client.0.smithi042.stdout:4/42: chown de/cf 1819891 1 2024-09-16T23:48:46.509 INFO:tasks.workunit.client.1.smithi043.stdout:7/185: creat d4/db/d17/d1b/f37 x:0 0 0 2024-09-16T23:48:46.510 INFO:tasks.workunit.client.1.smithi043.stdout:4/208: dwrite d8/d15/d20/d2b/f26 [0,4194304] 0 2024-09-16T23:48:46.510 INFO:tasks.workunit.client.0.smithi042.stdout:1/25: truncate d0/f3 1779162 0 2024-09-16T23:48:46.512 INFO:tasks.workunit.client.1.smithi043.stdout:9/147: mknod d4/d1c/d25/d2b/c2f 0 2024-09-16T23:48:46.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/25: stat f7 0 2024-09-16T23:48:46.513 INFO:tasks.workunit.client.0.smithi042.stdout:1/26: chown d0/f6 105097 1 2024-09-16T23:48:46.513 INFO:tasks.workunit.client.1.smithi043.stdout:9/148: write d4/d1c/d25/f26 [1498965,129490] 0 2024-09-16T23:48:46.514 INFO:tasks.workunit.client.1.smithi043.stdout:4/209: mknod d8/d25/d39/c3e 0 2024-09-16T23:48:46.515 INFO:tasks.workunit.client.1.smithi043.stdout:9/149: chown d4/d21/d2a/d2c/f2d 1243165303 1 2024-09-16T23:48:46.516 INFO:tasks.workunit.client.0.smithi042.stdout:5/25: truncate f6 192349 0 2024-09-16T23:48:46.516 INFO:tasks.workunit.client.1.smithi043.stdout:9/150: write d4/d5/f1d [339967,35330] 0 2024-09-16T23:48:46.517 INFO:tasks.workunit.client.0.smithi042.stdout:8/30: rename l9 to da/lb 0 2024-09-16T23:48:46.517 INFO:tasks.workunit.client.0.smithi042.stdout:9/28: write d4/fa [1514949,52562] 0 2024-09-16T23:48:46.518 INFO:tasks.workunit.client.0.smithi042.stdout:4/43: symlink de/l11 0 2024-09-16T23:48:46.519 INFO:tasks.workunit.client.1.smithi043.stdout:7/186: read d4/db/f24 [166308,73476] 0 2024-09-16T23:48:46.519 INFO:tasks.workunit.client.1.smithi043.stdout:9/151: truncate d4/f24 250435 0 2024-09-16T23:48:46.521 INFO:tasks.workunit.client.0.smithi042.stdout:6/26: dread f1 [0,4194304] 0 2024-09-16T23:48:46.524 INFO:tasks.workunit.client.0.smithi042.stdout:1/27: mkdir d0/d9 0 2024-09-16T23:48:46.525 INFO:tasks.workunit.client.1.smithi043.stdout:7/187: symlink d4/db/d11/d15/l38 0 2024-09-16T23:48:46.525 INFO:tasks.workunit.client.0.smithi042.stdout:1/28: chown d0/f3 167839 1 2024-09-16T23:48:46.525 INFO:tasks.workunit.client.1.smithi043.stdout:4/210: dwrite d8/fc [0,4194304] 0 2024-09-16T23:48:46.528 INFO:tasks.workunit.client.1.smithi043.stdout:4/211: chown d8/d15/d20/d2b 0 1 2024-09-16T23:48:46.528 INFO:tasks.workunit.client.0.smithi042.stdout:5/26: rename c4 to c7 0 2024-09-16T23:48:46.528 INFO:tasks.workunit.client.0.smithi042.stdout:5/27: readlink l5 0 2024-09-16T23:48:46.529 INFO:tasks.workunit.client.0.smithi042.stdout:9/29: readlink l3 0 2024-09-16T23:48:46.531 INFO:tasks.workunit.client.1.smithi043.stdout:7/188: getdents d4/db/d11/d15/d29 0 2024-09-16T23:48:46.531 INFO:tasks.workunit.client.1.smithi043.stdout:4/212: mknod d8/d15/d20/d2b/c3f 0 2024-09-16T23:48:46.532 INFO:tasks.workunit.client.0.smithi042.stdout:4/44: write de/f10 [436425,128769] 0 2024-09-16T23:48:46.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/45: chown l8 39266 1 2024-09-16T23:48:46.535 INFO:tasks.workunit.client.0.smithi042.stdout:4/46: fdatasync fd 0 2024-09-16T23:48:46.536 INFO:tasks.workunit.client.1.smithi043.stdout:7/189: mkdir d4/db/d11/d15/d29/d39 0 2024-09-16T23:48:46.537 INFO:tasks.workunit.client.1.smithi043.stdout:9/152: write d4/d5/d16/f1f [503734,10294] 0 2024-09-16T23:48:46.540 INFO:tasks.workunit.client.1.smithi043.stdout:9/153: write d4/d1c/f20 [242011,78910] 0 2024-09-16T23:48:46.541 INFO:tasks.workunit.client.0.smithi042.stdout:9/30: dwrite d4/f5 [0,4194304] 0 2024-09-16T23:48:46.542 INFO:tasks.workunit.client.1.smithi043.stdout:4/213: dwrite d8/d15/d20/d2b/d22/f24 [0,4194304] 0 2024-09-16T23:48:46.543 INFO:tasks.workunit.client.0.smithi042.stdout:4/47: mkdir de/d12 0 2024-09-16T23:48:46.543 INFO:tasks.workunit.client.0.smithi042.stdout:9/31: chown d4/dc 16802061 1 2024-09-16T23:48:46.544 INFO:tasks.workunit.client.1.smithi043.stdout:9/154: write d4/d5/f1d [149778,91970] 0 2024-09-16T23:48:46.546 INFO:tasks.workunit.client.1.smithi043.stdout:3/125: sync 2024-09-16T23:48:46.546 INFO:tasks.workunit.client.1.smithi043.stdout:6/160: sync 2024-09-16T23:48:46.546 INFO:tasks.workunit.client.1.smithi043.stdout:0/203: sync 2024-09-16T23:48:46.546 INFO:tasks.workunit.client.1.smithi043.stdout:5/326: sync 2024-09-16T23:48:46.546 INFO:tasks.workunit.client.1.smithi043.stdout:2/68: sync 2024-09-16T23:48:46.546 INFO:tasks.workunit.client.1.smithi043.stdout:8/176: sync 2024-09-16T23:48:46.546 INFO:tasks.workunit.client.1.smithi043.stdout:4/214: chown d8/f31 15 1 2024-09-16T23:48:46.548 INFO:tasks.workunit.client.1.smithi043.stdout:9/155: dread d4/f1a [0,4194304] 0 2024-09-16T23:48:46.548 INFO:tasks.workunit.client.0.smithi042.stdout:6/27: creat d8/d9/fa x:0 0 0 2024-09-16T23:48:46.548 INFO:tasks.workunit.client.0.smithi042.stdout:4/48: readlink l5 0 2024-09-16T23:48:46.549 INFO:tasks.workunit.client.0.smithi042.stdout:6/28: dread - d8/d9/fa zero size 2024-09-16T23:48:46.551 INFO:tasks.workunit.client.0.smithi042.stdout:2/23: getdents d0 0 2024-09-16T23:48:46.551 INFO:tasks.workunit.client.0.smithi042.stdout:2/24: write - no filename 2024-09-16T23:48:46.551 INFO:tasks.workunit.client.0.smithi042.stdout:1/29: dwrite d0/f3 [0,4194304] 0 2024-09-16T23:48:46.552 INFO:tasks.workunit.client.0.smithi042.stdout:2/25: chown d0/c8 510944308 1 2024-09-16T23:48:46.552 INFO:tasks.workunit.client.1.smithi043.stdout:8/177: fsync fc 0 2024-09-16T23:48:46.553 INFO:tasks.workunit.client.0.smithi042.stdout:1/30: dread - d0/f1 zero size 2024-09-16T23:48:46.553 INFO:tasks.workunit.client.0.smithi042.stdout:2/26: chown d0/c8 14329895 1 2024-09-16T23:48:46.554 INFO:tasks.workunit.client.0.smithi042.stdout:1/31: chown d0/c8 31692 1 2024-09-16T23:48:46.554 INFO:tasks.workunit.client.1.smithi043.stdout:6/161: creat de/d1a/f38 x:0 0 0 2024-09-16T23:48:46.555 INFO:tasks.workunit.client.0.smithi042.stdout:8/31: truncate f6 563751 0 2024-09-16T23:48:46.556 INFO:tasks.workunit.client.0.smithi042.stdout:1/32: write d0/f6 [833141,70879] 0 2024-09-16T23:48:46.556 INFO:tasks.workunit.client.0.smithi042.stdout:2/27: symlink d0/l9 0 2024-09-16T23:48:46.557 INFO:tasks.workunit.client.0.smithi042.stdout:8/32: mknod da/cc 0 2024-09-16T23:48:46.558 INFO:tasks.workunit.client.1.smithi043.stdout:3/126: mknod d4/c26 0 2024-09-16T23:48:46.558 INFO:tasks.workunit.client.0.smithi042.stdout:2/28: chown d0/c8 195463 1 2024-09-16T23:48:46.558 INFO:tasks.workunit.client.0.smithi042.stdout:9/32: rmdir d4/dc 0 2024-09-16T23:48:46.559 INFO:tasks.workunit.client.1.smithi043.stdout:4/215: dread d8/fc [0,4194304] 0 2024-09-16T23:48:46.559 INFO:tasks.workunit.client.1.smithi043.stdout:7/190: dwrite d4/db/f14 [0,4194304] 0 2024-09-16T23:48:46.560 INFO:tasks.workunit.client.1.smithi043.stdout:9/156: mkdir d4/d5/d30 0 2024-09-16T23:48:46.561 INFO:tasks.workunit.client.0.smithi042.stdout:1/33: creat d0/fa x:0 0 0 2024-09-16T23:48:46.562 INFO:tasks.workunit.client.0.smithi042.stdout:1/34: stat d0/c8 0 2024-09-16T23:48:46.563 INFO:tasks.workunit.client.0.smithi042.stdout:1/35: truncate d0/f5 33099 0 2024-09-16T23:48:46.564 INFO:tasks.workunit.client.0.smithi042.stdout:1/36: rename d0 to d0/d9/db 22 2024-09-16T23:48:46.565 INFO:tasks.workunit.client.0.smithi042.stdout:2/29: mkdir d0/da 0 2024-09-16T23:48:46.565 INFO:tasks.workunit.client.0.smithi042.stdout:2/30: write - no filename 2024-09-16T23:48:46.566 INFO:tasks.workunit.client.0.smithi042.stdout:9/33: unlink d4/c7 0 2024-09-16T23:48:46.566 INFO:tasks.workunit.client.0.smithi042.stdout:2/31: chown d0/c8 2469 1 2024-09-16T23:48:46.566 INFO:tasks.workunit.client.0.smithi042.stdout:2/32: write - no filename 2024-09-16T23:48:46.566 INFO:tasks.workunit.client.0.smithi042.stdout:2/33: truncate - no filename 2024-09-16T23:48:46.567 INFO:tasks.workunit.client.1.smithi043.stdout:5/327: rename d3/d1b/f36 to d3/db/d15/f69 0 2024-09-16T23:48:46.569 INFO:tasks.workunit.client.1.smithi043.stdout:5/328: readlink d3/db/d26/l57 0 2024-09-16T23:48:46.570 INFO:tasks.workunit.client.1.smithi043.stdout:9/157: dread d4/d5/f1d [0,4194304] 0 2024-09-16T23:48:46.570 INFO:tasks.workunit.client.1.smithi043.stdout:5/329: read d3/db/f52 [2402394,71503] 0 2024-09-16T23:48:46.570 INFO:tasks.workunit.client.1.smithi043.stdout:6/162: dread f2 [0,4194304] 0 2024-09-16T23:48:46.571 INFO:tasks.workunit.client.0.smithi042.stdout:1/37: rename d0/c8 to d0/cc 0 2024-09-16T23:48:46.572 INFO:tasks.workunit.client.1.smithi043.stdout:4/216: creat d8/d15/d20/d2b/d23/f40 x:0 0 0 2024-09-16T23:48:46.573 INFO:tasks.workunit.client.0.smithi042.stdout:9/34: symlink d4/ld 0 2024-09-16T23:48:46.573 INFO:tasks.workunit.client.0.smithi042.stdout:1/38: dread - d0/f1 zero size 2024-09-16T23:48:46.573 INFO:tasks.workunit.client.0.smithi042.stdout:2/34: symlink d0/da/lb 0 2024-09-16T23:48:46.573 INFO:tasks.workunit.client.0.smithi042.stdout:2/35: dwrite - no filename 2024-09-16T23:48:46.574 INFO:tasks.workunit.client.0.smithi042.stdout:6/29: getdents d8 0 2024-09-16T23:48:46.575 INFO:tasks.workunit.client.1.smithi043.stdout:8/178: mkdir de/d18/d27/d34/d3d 0 2024-09-16T23:48:46.576 INFO:tasks.workunit.client.0.smithi042.stdout:1/39: write d0/f5 [270269,46935] 0 2024-09-16T23:48:46.576 INFO:tasks.workunit.client.0.smithi042.stdout:2/36: creat d0/fc x:0 0 0 2024-09-16T23:48:46.577 INFO:tasks.workunit.client.0.smithi042.stdout:1/40: chown d0/fa 121812886 1 2024-09-16T23:48:46.578 INFO:tasks.workunit.client.1.smithi043.stdout:9/158: creat d4/f31 x:0 0 0 2024-09-16T23:48:46.581 INFO:tasks.workunit.client.0.smithi042.stdout:2/37: creat d0/da/fd x:0 0 0 2024-09-16T23:48:46.581 INFO:tasks.workunit.client.0.smithi042.stdout:1/41: fdatasync d0/f3 0 2024-09-16T23:48:46.582 INFO:tasks.workunit.client.1.smithi043.stdout:6/163: rename de/df to de/d1a/d39 0 2024-09-16T23:48:46.585 INFO:tasks.workunit.client.1.smithi043.stdout:5/330: mkdir d3/d9/d22/d4a/d6a 0 2024-09-16T23:48:46.585 INFO:tasks.workunit.client.1.smithi043.stdout:5/331: chown d3/db/d15/d4d 14556005 1 2024-09-16T23:48:46.588 INFO:tasks.workunit.client.1.smithi043.stdout:4/217: mkdir d8/d25/d39/d41 0 2024-09-16T23:48:46.589 INFO:tasks.workunit.client.0.smithi042.stdout:5/28: truncate f6 44655 0 2024-09-16T23:48:46.589 INFO:tasks.workunit.client.0.smithi042.stdout:2/38: dwrite d0/fc [0,4194304] 0 2024-09-16T23:48:46.590 INFO:tasks.workunit.client.1.smithi043.stdout:7/191: dread d4/db/d11/f13 [0,4194304] 0 2024-09-16T23:48:46.590 INFO:tasks.workunit.client.0.smithi042.stdout:9/35: link d4/ld d4/le 0 2024-09-16T23:48:46.590 INFO:tasks.workunit.client.1.smithi043.stdout:8/179: creat de/d18/d1f/d2f/f3e x:0 0 0 2024-09-16T23:48:46.591 INFO:tasks.workunit.client.1.smithi043.stdout:9/159: rmdir d4/d28/d2e 39 2024-09-16T23:48:46.591 INFO:tasks.workunit.client.1.smithi043.stdout:8/180: write de/d10/f26 [4999315,88041] 0 2024-09-16T23:48:46.594 INFO:tasks.workunit.client.1.smithi043.stdout:5/332: mkdir d3/db/d15/d4d/d6b 0 2024-09-16T23:48:46.596 INFO:tasks.workunit.client.1.smithi043.stdout:3/127: getdents d4 0 2024-09-16T23:48:46.597 INFO:tasks.workunit.client.0.smithi042.stdout:2/39: mknod d0/da/ce 0 2024-09-16T23:48:46.598 INFO:tasks.workunit.client.1.smithi043.stdout:0/204: truncate d0/d2/df/d1a/d34/f35 522800 0 2024-09-16T23:48:46.600 INFO:tasks.workunit.client.1.smithi043.stdout:3/128: dread - d4/de/f21 zero size 2024-09-16T23:48:46.600 INFO:tasks.workunit.client.1.smithi043.stdout:7/192: mkdir d4/db/d28/d3a 0 2024-09-16T23:48:46.600 INFO:tasks.workunit.client.0.smithi042.stdout:9/36: symlink d4/lf 0 2024-09-16T23:48:46.600 INFO:tasks.workunit.client.1.smithi043.stdout:5/333: fdatasync d3/db/d19/d3e/d28/f37 0 2024-09-16T23:48:46.601 INFO:tasks.workunit.client.0.smithi042.stdout:6/30: getdents d8/d9 0 2024-09-16T23:48:46.602 INFO:tasks.workunit.client.0.smithi042.stdout:2/40: dread d0/fc [0,4194304] 0 2024-09-16T23:48:46.603 INFO:tasks.workunit.client.0.smithi042.stdout:2/41: fdatasync d0/da/fd 0 2024-09-16T23:48:46.604 INFO:tasks.workunit.client.1.smithi043.stdout:6/164: creat de/d37/f3a x:0 0 0 2024-09-16T23:48:46.605 INFO:tasks.workunit.client.0.smithi042.stdout:8/33: write f6 [1414346,124537] 0 2024-09-16T23:48:46.606 INFO:tasks.workunit.client.1.smithi043.stdout:9/160: readlink d4/d28/d2e/l19 0 2024-09-16T23:48:46.607 INFO:tasks.workunit.client.0.smithi042.stdout:8/34: write f6 [1700176,48096] 0 2024-09-16T23:48:46.608 INFO:tasks.workunit.client.1.smithi043.stdout:3/129: creat d4/de/f27 x:0 0 0 2024-09-16T23:48:46.610 INFO:tasks.workunit.client.1.smithi043.stdout:6/165: fsync de/d1a/d39/f1f 0 2024-09-16T23:48:46.610 INFO:tasks.workunit.client.1.smithi043.stdout:9/161: write d4/d1c/f20 [1217351,105028] 0 2024-09-16T23:48:46.610 INFO:tasks.workunit.client.1.smithi043.stdout:5/334: unlink d3/db/d15/d4d/c5b 0 2024-09-16T23:48:46.611 INFO:tasks.workunit.client.1.smithi043.stdout:8/181: dwrite de/d10/f24 [0,4194304] 0 2024-09-16T23:48:46.611 INFO:tasks.workunit.client.0.smithi042.stdout:9/37: dwrite d4/fa [0,4194304] 0 2024-09-16T23:48:46.611 INFO:tasks.workunit.client.0.smithi042.stdout:9/38: readlink l3 0 2024-09-16T23:48:46.612 INFO:tasks.workunit.client.0.smithi042.stdout:9/39: read d4/f5 [1741903,31295] 0 2024-09-16T23:48:46.612 INFO:tasks.workunit.client.1.smithi043.stdout:4/218: dread d8/d25/f2c [0,4194304] 0 2024-09-16T23:48:46.612 INFO:tasks.workunit.client.0.smithi042.stdout:5/29: unlink f6 0 2024-09-16T23:48:46.613 INFO:tasks.workunit.client.1.smithi043.stdout:7/193: mknod d4/db/d28/c3b 0 2024-09-16T23:48:46.613 INFO:tasks.workunit.client.1.smithi043.stdout:8/182: chown de/f15 0 1 2024-09-16T23:48:46.614 INFO:tasks.workunit.client.1.smithi043.stdout:7/194: stat d4/f32 0 2024-09-16T23:48:46.615 INFO:tasks.workunit.client.1.smithi043.stdout:5/335: stat d3/d9/c29 0 2024-09-16T23:48:46.615 INFO:tasks.workunit.client.0.smithi042.stdout:2/42: creat d0/ff x:0 0 0 2024-09-16T23:48:46.620 INFO:tasks.workunit.client.0.smithi042.stdout:8/35: creat da/fd x:0 0 0 2024-09-16T23:48:46.621 INFO:tasks.workunit.client.1.smithi043.stdout:6/166: creat de/d1a/f3b x:0 0 0 2024-09-16T23:48:46.621 INFO:tasks.workunit.client.0.smithi042.stdout:8/36: chown l3 94034 1 2024-09-16T23:48:46.624 INFO:tasks.workunit.client.1.smithi043.stdout:8/183: dread de/f15 [0,4194304] 0 2024-09-16T23:48:46.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.628+0000 7fe251240700 1 -- 172.21.15.42:0/159271038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 msgr2=0x7fe24c101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:46.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.628+0000 7fe251240700 1 --2- 172.21.15.42:0/159271038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 0x7fe24c101640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009910 tx=0x7fe23c009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:46.626 INFO:tasks.workunit.client.1.smithi043.stdout:3/130: mknod d4/de/d1f/c28 0 2024-09-16T23:48:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.628+0000 7fe251240700 1 -- 172.21.15.42:0/159271038 shutdown_connections 2024-09-16T23:48:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.628+0000 7fe251240700 1 --2- 172.21.15.42:0/159271038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 0x7fe24c100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.628+0000 7fe251240700 1 --2- 172.21.15.42:0/159271038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 0x7fe24c101640 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.628+0000 7fe251240700 1 -- 172.21.15.42:0/159271038 >> 172.21.15.42:0/159271038 conn(0x7fe24c0fa8c0 msgr2=0x7fe24c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.629+0000 7fe251240700 1 -- 172.21.15.42:0/159271038 shutdown_connections 2024-09-16T23:48:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.629+0000 7fe251240700 1 -- 172.21.15.42:0/159271038 wait complete. 2024-09-16T23:48:46.627 INFO:tasks.workunit.client.0.smithi042.stdout:9/40: stat d4/le 0 2024-09-16T23:48:46.627 INFO:tasks.workunit.client.1.smithi043.stdout:4/219: rename d8/d15/d20/d2b/f2d to d8/d25/f42 0 2024-09-16T23:48:46.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.630+0000 7fe251240700 1 Processor -- start 2024-09-16T23:48:46.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.630+0000 7fe251240700 1 -- start start 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.630+0000 7fe251240700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 0x7fe24c0718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.630+0000 7fe251240700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 0x7fe24c071e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe251240700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c072370 con 0x7fe24c101220 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe251240700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0724b0 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe24ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 0x7fe24c0718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe24ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 0x7fe24c0718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:60264/0 (socket says 172.21.15.42:60264) 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe24ad9d700 1 -- 172.21.15.42:0/2123978463 learned_addr learned my addr 172.21.15.42:0/2123978463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:46.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe24259c700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 0x7fe24c071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:46.629 INFO:tasks.workunit.client.0.smithi042.stdout:5/30: symlink l8 0 2024-09-16T23:48:46.629 INFO:tasks.workunit.client.0.smithi042.stdout:5/31: dread - no filename 2024-09-16T23:48:46.629 INFO:tasks.workunit.client.0.smithi042.stdout:5/32: dwrite - no filename 2024-09-16T23:48:46.630 INFO:tasks.workunit.client.0.smithi042.stdout:5/33: write - no filename 2024-09-16T23:48:46.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe24ad9d700 1 -- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 msgr2=0x7fe24c071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:46.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe24ad9d700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 0x7fe24c071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:46.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.631+0000 7fe24ad9d700 1 -- 172.21.15.42:0/2123978463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0095f0 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.632+0000 7fe24ad9d700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 0x7fe24c0718f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0098e0 tx=0x7fe23c0048c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:46.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.632+0000 7fe248d99700 1 -- 172.21.15.42:0/2123978463 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe23c01d070 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.632+0000 7fe248d99700 1 -- 172.21.15.42:0/2123978463 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe23c004ca0 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.632+0000 7fe248d99700 1 -- 172.21.15.42:0/2123978463 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe23c00f690 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.632+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c1a63a0 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.631 INFO:tasks.workunit.client.0.smithi042.stdout:1/42: link d0/cc d0/d9/cd 0 2024-09-16T23:48:46.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.632+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c1a6890 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.634+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c1076a0 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.632 INFO:tasks.workunit.client.0.smithi042.stdout:8/37: symlink da/le 0 2024-09-16T23:48:46.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.634+0000 7fe248d99700 1 -- 172.21.15.42:0/2123978463 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 20) v1 ==== 91358+0+0 (secure 0 0 0) 0x7fe23c022470 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.635+0000 7fe248d99700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe23806d440 0x7fe23806f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:46.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.635+0000 7fe248d99700 1 -- 172.21.15.42:0/2123978463 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fe23c08e2b0 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.635+0000 7fe24259c700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe23806d440 0x7fe23806f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:46.633 INFO:tasks.workunit.client.1.smithi043.stdout:9/162: mknod d4/d28/c32 0 2024-09-16T23:48:46.633 INFO:tasks.workunit.client.1.smithi043.stdout:4/220: chown d8/d25/f42 56848 1 2024-09-16T23:48:46.633 INFO:tasks.workunit.client.1.smithi043.stdout:9/163: truncate d4/f31 135516 0 2024-09-16T23:48:46.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.636+0000 7fe24259c700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe23806d440 0x7fe23806f900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe234009550 tx=0x7fe2340091b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:46.635 INFO:tasks.workunit.client.1.smithi043.stdout:3/131: dwrite d4/de/f21 [0,4194304] 0 2024-09-16T23:48:46.635 INFO:tasks.workunit.client.1.smithi043.stdout:3/132: stat d4/f19 0 2024-09-16T23:48:46.636 INFO:tasks.workunit.client.1.smithi043.stdout:6/167: rename de/d1a/c1d to de/d1a/d39/c3c 0 2024-09-16T23:48:46.638 INFO:tasks.workunit.client.0.smithi042.stdout:5/34: mkdir d9 0 2024-09-16T23:48:46.638 INFO:tasks.workunit.client.0.smithi042.stdout:5/35: dread - no filename 2024-09-16T23:48:46.638 INFO:tasks.workunit.client.0.smithi042.stdout:8/38: dwrite f6 [0,4194304] 0 2024-09-16T23:48:46.639 INFO:tasks.workunit.client.1.smithi043.stdout:5/336: dread d3/db/f52 [0,4194304] 0 2024-09-16T23:48:46.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:46.643+0000 7fe248d99700 1 -- 172.21.15.42:0/2123978463 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe23c05c600 con 0x7fe24c0ffdf0 2024-09-16T23:48:46.641 INFO:tasks.workunit.client.1.smithi043.stdout:4/221: symlink d8/d15/d19/l43 0 2024-09-16T23:48:46.643 INFO:tasks.workunit.client.0.smithi042.stdout:1/43: rename d0/f1 to d0/d9/fe 0 2024-09-16T23:48:46.643 INFO:tasks.workunit.client.1.smithi043.stdout:8/184: link de/d18/d2b/f2e de/d10/d37/f3f 0 2024-09-16T23:48:46.646 INFO:tasks.workunit.client.1.smithi043.stdout:5/337: dwrite d3/d9/f4b [0,4194304] 0 2024-09-16T23:48:46.651 INFO:tasks.workunit.client.1.smithi043.stdout:6/168: chown de/l25 95290 1 2024-09-16T23:48:46.651 INFO:tasks.workunit.client.1.smithi043.stdout:3/133: rename d4/f22 to d4/de/d1f/f29 0 2024-09-16T23:48:46.653 INFO:tasks.workunit.client.0.smithi042.stdout:2/43: getdents d0 0 2024-09-16T23:48:46.653 INFO:tasks.workunit.client.1.smithi043.stdout:4/222: symlink d8/d15/d20/d2b/d23/l44 0 2024-09-16T23:48:46.656 INFO:tasks.workunit.client.1.smithi043.stdout:7/195: getdents d4/d27 0 2024-09-16T23:48:46.658 INFO:tasks.workunit.client.0.smithi042.stdout:5/36: rename c7 to d9/ca 0 2024-09-16T23:48:46.658 INFO:tasks.workunit.client.1.smithi043.stdout:7/196: fdatasync d4/db/d11/d21/d31/d33/f1f 0 2024-09-16T23:48:46.659 INFO:tasks.workunit.client.0.smithi042.stdout:8/39: dwrite da/fd [0,4194304] 0 2024-09-16T23:48:46.660 INFO:tasks.workunit.client.1.smithi043.stdout:5/338: creat d3/db/d15/d4d/f6c x:0 0 0 2024-09-16T23:48:46.660 INFO:tasks.workunit.client.1.smithi043.stdout:6/169: dwrite fd [0,4194304] 0 2024-09-16T23:48:46.661 INFO:tasks.workunit.client.0.smithi042.stdout:6/31: dwrite f6 [0,4194304] 0 2024-09-16T23:48:46.662 INFO:tasks.workunit.client.1.smithi043.stdout:6/170: stat de/f19 0 2024-09-16T23:48:46.664 INFO:tasks.workunit.client.1.smithi043.stdout:6/171: chown de/f19 21625 1 2024-09-16T23:48:46.664 INFO:tasks.workunit.client.0.smithi042.stdout:5/37: mkdir d9/db 0 2024-09-16T23:48:46.664 INFO:tasks.workunit.client.0.smithi042.stdout:5/38: truncate - no filename 2024-09-16T23:48:46.664 INFO:tasks.workunit.client.0.smithi042.stdout:5/39: dwrite - no filename 2024-09-16T23:48:46.664 INFO:tasks.workunit.client.1.smithi043.stdout:5/339: symlink d3/db/d15/l6d 0 2024-09-16T23:48:46.666 INFO:tasks.workunit.client.0.smithi042.stdout:5/40: readlink l8 0 2024-09-16T23:48:46.666 INFO:tasks.workunit.client.0.smithi042.stdout:5/41: dwrite - no filename 2024-09-16T23:48:46.666 INFO:tasks.workunit.client.0.smithi042.stdout:1/44: dwrite d0/f5 [0,4194304] 0 2024-09-16T23:48:46.667 INFO:tasks.workunit.client.0.smithi042.stdout:5/42: readlink l5 0 2024-09-16T23:48:46.669 INFO:tasks.workunit.client.0.smithi042.stdout:1/45: chown d0/d9 0 1 2024-09-16T23:48:46.670 INFO:tasks.workunit.client.1.smithi043.stdout:8/185: getdents de/d18/d1f 0 2024-09-16T23:48:46.671 INFO:tasks.workunit.client.0.smithi042.stdout:1/46: truncate d0/f5 4435459 0 2024-09-16T23:48:46.673 INFO:tasks.workunit.client.0.smithi042.stdout:1/47: dread - d0/d9/fe zero size 2024-09-16T23:48:46.674 INFO:tasks.workunit.client.0.smithi042.stdout:5/43: mknod d9/cc 0 2024-09-16T23:48:46.675 INFO:tasks.workunit.client.1.smithi043.stdout:6/172: symlink de/d23/d34/l3d 0 2024-09-16T23:48:46.675 INFO:tasks.workunit.client.0.smithi042.stdout:6/32: dread f3 [0,4194304] 0 2024-09-16T23:48:46.676 INFO:tasks.workunit.client.0.smithi042.stdout:6/33: dread - d8/d9/fa zero size 2024-09-16T23:48:46.679 INFO:tasks.workunit.client.1.smithi043.stdout:1/258: sync 2024-09-16T23:48:46.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/34: mkdir d8/db 0 2024-09-16T23:48:46.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/44: link l5 d9/db/ld 0 2024-09-16T23:48:46.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/45: write - no filename 2024-09-16T23:48:46.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/46: write - no filename 2024-09-16T23:48:46.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/47: dwrite - no filename 2024-09-16T23:48:46.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/48: dwrite - no filename 2024-09-16T23:48:46.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/49: write - no filename 2024-09-16T23:48:46.684 INFO:tasks.workunit.client.0.smithi042.stdout:2/44: truncate d0/fc 263147 0 2024-09-16T23:48:46.684 INFO:tasks.workunit.client.1.smithi043.stdout:3/134: truncate d4/f23 2277 0 2024-09-16T23:48:46.685 INFO:tasks.workunit.client.1.smithi043.stdout:7/197: getdents d4/db 0 2024-09-16T23:48:46.686 INFO:tasks.workunit.client.1.smithi043.stdout:1/259: creat d4/d9/d1c/d1e/d47/f53 x:0 0 0 2024-09-16T23:48:46.687 INFO:tasks.workunit.client.1.smithi043.stdout:3/135: chown d4/de 50 1 2024-09-16T23:48:46.687 INFO:tasks.workunit.client.1.smithi043.stdout:8/186: link cb de/d18/d27/d34/c40 0 2024-09-16T23:48:46.688 INFO:tasks.workunit.client.0.smithi042.stdout:2/45: mknod d0/da/c10 0 2024-09-16T23:48:46.689 INFO:tasks.workunit.client.1.smithi043.stdout:7/198: stat d4/db/d17/l1d 0 2024-09-16T23:48:46.691 INFO:tasks.workunit.client.0.smithi042.stdout:5/50: mkdir d9/db/de 0 2024-09-16T23:48:46.691 INFO:tasks.workunit.client.0.smithi042.stdout:5/51: dwrite - no filename 2024-09-16T23:48:46.691 INFO:tasks.workunit.client.0.smithi042.stdout:5/52: truncate - no filename 2024-09-16T23:48:46.692 INFO:tasks.workunit.client.0.smithi042.stdout:1/48: dread d0/f6 [0,4194304] 0 2024-09-16T23:48:46.692 INFO:tasks.workunit.client.1.smithi043.stdout:8/187: rmdir de/d18/d1f/d25 39 2024-09-16T23:48:46.696 INFO:tasks.workunit.client.1.smithi043.stdout:5/340: truncate d3/d9/d22/f61 647950 0 2024-09-16T23:48:46.696 INFO:tasks.workunit.client.1.smithi043.stdout:6/173: dwrite f2 [0,4194304] 0 2024-09-16T23:48:46.697 INFO:tasks.workunit.client.0.smithi042.stdout:8/40: dwrite f6 [4194304,4194304] 0 2024-09-16T23:48:46.697 INFO:tasks.workunit.client.0.smithi042.stdout:6/35: dwrite f6 [0,4194304] 0 2024-09-16T23:48:46.698 INFO:tasks.workunit.client.0.smithi042.stdout:1/49: mkdir d0/d9/df 0 2024-09-16T23:48:46.701 INFO:tasks.workunit.client.0.smithi042.stdout:5/53: creat d9/db/ff x:0 0 0 2024-09-16T23:48:46.702 INFO:tasks.workunit.client.0.smithi042.stdout:5/54: dread - d9/db/ff zero size 2024-09-16T23:48:46.703 INFO:tasks.workunit.client.0.smithi042.stdout:8/41: symlink da/lf 0 2024-09-16T23:48:46.704 INFO:tasks.workunit.client.1.smithi043.stdout:6/174: dread de/d2a/f33 [0,4194304] 0 2024-09-16T23:48:46.705 INFO:tasks.workunit.client.1.smithi043.stdout:1/260: dwrite d4/d9/d12/f1f [0,4194304] 0 2024-09-16T23:48:46.707 INFO:tasks.workunit.client.0.smithi042.stdout:6/36: creat d8/db/fc x:0 0 0 2024-09-16T23:48:46.708 INFO:tasks.workunit.client.0.smithi042.stdout:8/42: readlink da/lb 0 2024-09-16T23:48:46.708 INFO:tasks.workunit.client.1.smithi043.stdout:3/136: dwrite d4/de/f21 [0,4194304] 0 2024-09-16T23:48:46.708 INFO:tasks.workunit.client.0.smithi042.stdout:6/37: write f7 [265334,19460] 0 2024-09-16T23:48:46.710 INFO:tasks.workunit.client.0.smithi042.stdout:1/50: rename d0/cc to d0/d9/df/c10 0 2024-09-16T23:48:46.711 INFO:tasks.workunit.client.1.smithi043.stdout:6/175: fsync f3 0 2024-09-16T23:48:46.713 INFO:tasks.workunit.client.0.smithi042.stdout:8/43: dread f6 [4194304,4194304] 0 2024-09-16T23:48:46.715 INFO:tasks.workunit.client.1.smithi043.stdout:1/261: rename f1 to d4/d9/d12/f54 0 2024-09-16T23:48:46.715 INFO:tasks.workunit.client.1.smithi043.stdout:8/188: write fc [1455403,9943] 0 2024-09-16T23:48:46.716 INFO:tasks.workunit.client.0.smithi042.stdout:8/44: write da/fd [4603340,107835] 0 2024-09-16T23:48:46.718 INFO:tasks.workunit.client.0.smithi042.stdout:6/38: mkdir d8/db/dd 0 2024-09-16T23:48:46.718 INFO:tasks.workunit.client.1.smithi043.stdout:6/176: write de/d37/f3a [467338,15261] 0 2024-09-16T23:48:46.718 INFO:tasks.workunit.client.1.smithi043.stdout:5/341: dwrite d3/d9/d22/d60/f63 [0,4194304] 0 2024-09-16T23:48:46.719 INFO:tasks.workunit.client.1.smithi043.stdout:6/177: chown f2 3283156 1 2024-09-16T23:48:46.721 INFO:tasks.workunit.client.1.smithi043.stdout:6/178: fdatasync de/d1a/f38 0 2024-09-16T23:48:46.722 INFO:tasks.workunit.client.1.smithi043.stdout:3/137: rmdir d4/de/d1f 39 2024-09-16T23:48:46.726 INFO:tasks.workunit.client.1.smithi043.stdout:8/189: mkdir de/d18/d27/d41 0 2024-09-16T23:48:46.727 INFO:tasks.workunit.client.0.smithi042.stdout:2/46: dwrite d0/fc [0,4194304] 0 2024-09-16T23:48:46.728 INFO:tasks.workunit.client.1.smithi043.stdout:6/179: dread de/d1a/d28/f35 [0,4194304] 0 2024-09-16T23:48:46.729 INFO:tasks.workunit.client.0.smithi042.stdout:2/47: dread - d0/ff zero size 2024-09-16T23:48:46.729 INFO:tasks.workunit.client.1.smithi043.stdout:6/180: readlink de/d1a/d39/l30 0 2024-09-16T23:48:46.730 INFO:tasks.workunit.client.1.smithi043.stdout:1/262: creat d4/d9/d1c/d1e/f55 x:0 0 0 2024-09-16T23:48:46.730 INFO:tasks.workunit.client.1.smithi043.stdout:6/181: stat de/d23 0 2024-09-16T23:48:46.730 INFO:tasks.workunit.client.0.smithi042.stdout:6/39: rename f1 to d8/d9/fe 0 2024-09-16T23:48:46.732 INFO:tasks.workunit.client.1.smithi043.stdout:6/182: chown de/d1a/f20 179624612 1 2024-09-16T23:48:46.736 INFO:tasks.workunit.client.0.smithi042.stdout:6/40: write d8/d9/fe [972245,90067] 0 2024-09-16T23:48:46.738 INFO:tasks.workunit.client.1.smithi043.stdout:8/190: creat de/d18/d27/d34/d3d/f42 x:0 0 0 2024-09-16T23:48:46.738 INFO:tasks.workunit.client.0.smithi042.stdout:6/41: dread - d8/db/fc zero size 2024-09-16T23:48:46.738 INFO:tasks.workunit.client.0.smithi042.stdout:6/42: stat d8/db/dd 0 2024-09-16T23:48:46.740 INFO:tasks.workunit.client.0.smithi042.stdout:1/51: truncate d0/f6 580456 0 2024-09-16T23:48:46.741 INFO:tasks.workunit.client.1.smithi043.stdout:3/138: truncate d4/f16 520071 0 2024-09-16T23:48:46.741 INFO:tasks.workunit.client.1.smithi043.stdout:8/191: stat de/l12 0 2024-09-16T23:48:46.742 INFO:tasks.workunit.client.1.smithi043.stdout:5/342: dwrite d3/db/f52 [0,4194304] 0 2024-09-16T23:48:46.743 INFO:tasks.workunit.client.1.smithi043.stdout:3/139: fsync d4/f23 0 2024-09-16T23:48:46.743 INFO:tasks.workunit.client.0.smithi042.stdout:8/45: getdents da 0 2024-09-16T23:48:46.744 INFO:tasks.workunit.client.0.smithi042.stdout:8/46: chown da/lf 2 1 2024-09-16T23:48:46.745 INFO:tasks.workunit.client.0.smithi042.stdout:1/52: rmdir d0/d9/df 39 2024-09-16T23:48:46.748 INFO:tasks.workunit.client.1.smithi043.stdout:1/263: creat d4/d9/f56 x:0 0 0 2024-09-16T23:48:46.749 INFO:tasks.workunit.client.1.smithi043.stdout:5/343: symlink d3/d1b/l6e 0 2024-09-16T23:48:46.752 INFO:tasks.workunit.client.0.smithi042.stdout:2/48: truncate d0/fc 1150188 0 2024-09-16T23:48:46.752 INFO:tasks.workunit.client.1.smithi043.stdout:6/183: dwrite f3 [0,4194304] 0 2024-09-16T23:48:46.753 INFO:tasks.workunit.client.0.smithi042.stdout:2/49: write d0/ff [786442,81214] 0 2024-09-16T23:48:46.754 INFO:tasks.workunit.client.0.smithi042.stdout:2/50: readlink d0/l7 0 2024-09-16T23:48:46.755 INFO:tasks.workunit.client.1.smithi043.stdout:6/184: chown de 34529412 1 2024-09-16T23:48:46.755 INFO:tasks.workunit.client.1.smithi043.stdout:7/199: dread d4/db/f18 [0,4194304] 0 2024-09-16T23:48:46.759 INFO:tasks.workunit.client.1.smithi043.stdout:1/264: creat d4/df/d13/f57 x:0 0 0 2024-09-16T23:48:46.759 INFO:tasks.workunit.client.1.smithi043.stdout:6/185: write de/d1a/f20 [195232,56530] 0 2024-09-16T23:48:46.759 INFO:tasks.workunit.client.0.smithi042.stdout:1/53: truncate d0/f6 945105 0 2024-09-16T23:48:46.760 INFO:tasks.workunit.client.1.smithi043.stdout:1/265: dread - d4/d9/d12/d2b/f52 zero size 2024-09-16T23:48:46.761 INFO:tasks.workunit.client.1.smithi043.stdout:8/192: rename de/d10/d37/f3f to de/d18/d1f/d2f/f43 0 2024-09-16T23:48:46.762 INFO:tasks.workunit.client.1.smithi043.stdout:3/140: creat d4/de/f2a x:0 0 0 2024-09-16T23:48:46.763 INFO:tasks.workunit.client.0.smithi042.stdout:8/47: truncate f6 3437546 0 2024-09-16T23:48:46.764 INFO:tasks.workunit.client.1.smithi043.stdout:5/344: mkdir d3/d9/d6f 0 2024-09-16T23:48:46.766 INFO:tasks.workunit.client.0.smithi042.stdout:1/54: dwrite d0/f5 [0,4194304] 0 2024-09-16T23:48:46.767 INFO:tasks.workunit.client.0.smithi042.stdout:8/48: rename da/cc to da/c10 0 2024-09-16T23:48:46.771 INFO:tasks.workunit.client.1.smithi043.stdout:1/266: symlink d4/d34/l58 0 2024-09-16T23:48:46.772 INFO:tasks.workunit.client.1.smithi043.stdout:3/141: chown d4/fd 571404883 1 2024-09-16T23:48:46.773 INFO:tasks.workunit.client.1.smithi043.stdout:5/345: creat d3/db/d19/d3e/f70 x:0 0 0 2024-09-16T23:48:46.774 INFO:tasks.workunit.client.1.smithi043.stdout:3/142: dread - d4/de/f2a zero size 2024-09-16T23:48:46.777 INFO:tasks.workunit.client.1.smithi043.stdout:3/143: write d4/fd [84221,23411] 0 2024-09-16T23:48:46.777 INFO:tasks.workunit.client.1.smithi043.stdout:8/193: symlink de/d18/d27/d41/l44 0 2024-09-16T23:48:46.778 INFO:tasks.workunit.client.1.smithi043.stdout:1/267: creat d4/df/d13/d1b/f59 x:0 0 0 2024-09-16T23:48:46.779 INFO:tasks.workunit.client.0.smithi042.stdout:8/49: write f6 [334969,101514] 0 2024-09-16T23:48:46.780 INFO:tasks.workunit.client.1.smithi043.stdout:7/200: rmdir d4/db/d11/d15/d29/d39 0 2024-09-16T23:48:46.780 INFO:tasks.workunit.client.0.smithi042.stdout:8/50: read f6 [2428644,81668] 0 2024-09-16T23:48:46.782 INFO:tasks.workunit.client.1.smithi043.stdout:6/186: link de/d1a/d28/f35 de/d1a/d28/f3e 0 2024-09-16T23:48:46.783 INFO:tasks.workunit.client.1.smithi043.stdout:3/144: fsync d4/fb 0 2024-09-16T23:48:46.784 INFO:tasks.workunit.client.1.smithi043.stdout:3/145: fdatasync d4/de/f2a 0 2024-09-16T23:48:46.785 INFO:tasks.workunit.client.0.smithi042.stdout:8/51: unlink da/le 0 2024-09-16T23:48:46.785 INFO:tasks.workunit.client.1.smithi043.stdout:7/201: symlink d4/db/d11/d15/l3c 0 2024-09-16T23:48:46.785 INFO:tasks.workunit.client.1.smithi043.stdout:7/202: chown d4/f32 12451 1 2024-09-16T23:48:46.786 INFO:tasks.workunit.client.0.smithi042.stdout:8/52: write da/fd [152238,51378] 0 2024-09-16T23:48:46.786 INFO:tasks.workunit.client.1.smithi043.stdout:6/187: symlink de/d23/d34/l3f 0 2024-09-16T23:48:46.787 INFO:tasks.workunit.client.1.smithi043.stdout:6/188: readlink de/d23/l2c 0 2024-09-16T23:48:46.787 INFO:tasks.workunit.client.0.smithi042.stdout:1/55: fsync d0/f5 0 2024-09-16T23:48:46.788 INFO:tasks.workunit.client.1.smithi043.stdout:7/203: truncate d4/db/f24 772249 0 2024-09-16T23:48:46.788 INFO:tasks.workunit.client.1.smithi043.stdout:1/268: mknod d4/d9/d1c/d1e/d42/c5a 0 2024-09-16T23:48:46.788 INFO:tasks.workunit.client.1.smithi043.stdout:6/189: fsync de/d37/f3a 0 2024-09-16T23:48:46.789 INFO:tasks.workunit.client.0.smithi042.stdout:8/53: mknod da/c11 0 2024-09-16T23:48:46.789 INFO:tasks.workunit.client.0.smithi042.stdout:8/54: stat da 0 2024-09-16T23:48:46.789 INFO:tasks.workunit.client.1.smithi043.stdout:8/194: creat de/f45 x:0 0 0 2024-09-16T23:48:46.789 INFO:tasks.workunit.client.1.smithi043.stdout:7/204: stat d4/db/d17/d1b/f37 0 2024-09-16T23:48:46.790 INFO:tasks.workunit.client.0.smithi042.stdout:8/55: fsync da/fd 0 2024-09-16T23:48:46.790 INFO:tasks.workunit.client.1.smithi043.stdout:3/146: write d4/f1b [984694,22871] 0 2024-09-16T23:48:46.791 INFO:tasks.workunit.client.1.smithi043.stdout:3/147: stat d4/de/f24 0 2024-09-16T23:48:46.792 INFO:tasks.workunit.client.1.smithi043.stdout:8/195: chown de/d18/d27/d41/l44 421 1 2024-09-16T23:48:46.792 INFO:tasks.workunit.client.0.smithi042.stdout:1/56: creat d0/d9/f11 x:0 0 0 2024-09-16T23:48:46.792 INFO:tasks.workunit.client.1.smithi043.stdout:8/196: readlink de/d18/l3c 0 2024-09-16T23:48:46.793 INFO:tasks.workunit.client.1.smithi043.stdout:6/190: creat de/f40 x:0 0 0 2024-09-16T23:48:46.795 INFO:tasks.workunit.client.1.smithi043.stdout:6/191: write de/d1a/d39/f16 [289668,121874] 0 2024-09-16T23:48:46.797 INFO:tasks.workunit.client.1.smithi043.stdout:3/148: mkdir d4/d17/d2b 0 2024-09-16T23:48:46.797 INFO:tasks.workunit.client.0.smithi042.stdout:8/56: dwrite da/fd [0,4194304] 0 2024-09-16T23:48:46.799 INFO:tasks.workunit.client.0.smithi042.stdout:8/57: fdatasync f6 0 2024-09-16T23:48:46.800 INFO:tasks.workunit.client.1.smithi043.stdout:8/197: truncate de/d10/f1b 3119106 0 2024-09-16T23:48:46.800 INFO:tasks.workunit.client.1.smithi043.stdout:7/205: getdents d4/db 0 2024-09-16T23:48:46.801 INFO:tasks.workunit.client.0.smithi042.stdout:1/57: getdents d0/d9/df 0 2024-09-16T23:48:46.801 INFO:tasks.workunit.client.0.smithi042.stdout:1/58: fsync d0/f3 0 2024-09-16T23:48:46.803 INFO:tasks.workunit.client.1.smithi043.stdout:1/269: dread d4/df/d13/d1b/f30 [0,4194304] 0 2024-09-16T23:48:46.803 INFO:tasks.workunit.client.0.smithi042.stdout:1/59: read - d0/d9/f11 zero size 2024-09-16T23:48:46.803 INFO:tasks.workunit.client.1.smithi043.stdout:8/198: mkdir de/d18/d27/d46 0 2024-09-16T23:48:46.804 INFO:tasks.workunit.client.1.smithi043.stdout:8/199: chown de/d18/d27/c28 498985 1 2024-09-16T23:48:46.804 INFO:tasks.workunit.client.0.smithi042.stdout:8/58: dread da/fd [0,4194304] 0 2024-09-16T23:48:46.806 INFO:tasks.workunit.client.1.smithi043.stdout:3/149: mkdir d4/de/d1f/d2c 0 2024-09-16T23:48:46.807 INFO:tasks.workunit.client.0.smithi042.stdout:1/60: creat d0/d9/df/f12 x:0 0 0 2024-09-16T23:48:46.808 INFO:tasks.workunit.client.1.smithi043.stdout:7/206: unlink d4/db/d11/d21/d31/d33/lf 0 2024-09-16T23:48:46.809 INFO:tasks.workunit.client.0.smithi042.stdout:8/59: unlink c7 0 2024-09-16T23:48:46.809 INFO:tasks.workunit.client.1.smithi043.stdout:1/270: chown d4/d9/d12/d15/l4a 130951 1 2024-09-16T23:48:46.810 INFO:tasks.workunit.client.1.smithi043.stdout:7/207: fsync d4/db/d17/d1b/f37 0 2024-09-16T23:48:46.811 INFO:tasks.workunit.client.0.smithi042.stdout:1/61: creat d0/d9/df/f13 x:0 0 0 2024-09-16T23:48:46.812 INFO:tasks.workunit.client.0.smithi042.stdout:8/60: mknod da/c12 0 2024-09-16T23:48:46.814 INFO:tasks.workunit.client.0.smithi042.stdout:1/62: dread d0/f5 [0,4194304] 0 2024-09-16T23:48:46.815 INFO:tasks.workunit.client.1.smithi043.stdout:8/200: rename cb to de/d18/d27/c47 0 2024-09-16T23:48:46.815 INFO:tasks.workunit.client.0.smithi042.stdout:1/63: stat d0/d9/df/f12 0 2024-09-16T23:48:46.816 INFO:tasks.workunit.client.1.smithi043.stdout:6/192: dread de/d1a/d39/f1f [0,4194304] 0 2024-09-16T23:48:46.816 INFO:tasks.workunit.client.1.smithi043.stdout:3/150: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:46.818 INFO:tasks.workunit.client.1.smithi043.stdout:3/151: readlink d4/d17/l1e 0 2024-09-16T23:48:46.818 INFO:tasks.workunit.client.0.smithi042.stdout:1/64: mknod d0/d9/c14 0 2024-09-16T23:48:46.819 INFO:tasks.workunit.client.0.smithi042.stdout:1/65: dread - d0/fa zero size 2024-09-16T23:48:46.819 INFO:tasks.workunit.client.1.smithi043.stdout:3/152: write d4/fb [4426633,35531] 0 2024-09-16T23:48:46.820 INFO:tasks.workunit.client.0.smithi042.stdout:1/66: readlink d0/l7 0 2024-09-16T23:48:46.820 INFO:tasks.workunit.client.1.smithi043.stdout:7/208: rename d4/fc to d4/db/d11/d15/f3d 0 2024-09-16T23:48:46.822 INFO:tasks.workunit.client.0.smithi042.stdout:1/67: mkdir d0/d15 0 2024-09-16T23:48:46.823 INFO:tasks.workunit.client.1.smithi043.stdout:6/193: dread de/d1a/d39/f16 [0,4194304] 0 2024-09-16T23:48:46.826 INFO:tasks.workunit.client.0.smithi042.stdout:1/68: creat d0/d9/f16 x:0 0 0 2024-09-16T23:48:46.827 INFO:tasks.workunit.client.1.smithi043.stdout:3/153: fsync d4/f19 0 2024-09-16T23:48:46.827 INFO:tasks.workunit.client.0.smithi042.stdout:1/69: readlink d0/l7 0 2024-09-16T23:48:46.827 INFO:tasks.workunit.client.0.smithi042.stdout:8/61: rmdir da 39 2024-09-16T23:48:46.828 INFO:tasks.workunit.client.1.smithi043.stdout:8/201: symlink de/d18/d1f/l48 0 2024-09-16T23:48:46.828 INFO:tasks.workunit.client.1.smithi043.stdout:8/202: readlink de/l1d 0 2024-09-16T23:48:46.829 INFO:tasks.workunit.client.0.smithi042.stdout:1/70: mkdir d0/d17 0 2024-09-16T23:48:46.829 INFO:tasks.workunit.client.1.smithi043.stdout:1/271: getdents d4/d9/d12 0 2024-09-16T23:48:46.830 INFO:tasks.workunit.client.0.smithi042.stdout:8/62: rename da/c12 to da/c13 0 2024-09-16T23:48:46.831 INFO:tasks.workunit.client.0.smithi042.stdout:1/71: mkdir d0/d17/d18 0 2024-09-16T23:48:46.832 INFO:tasks.workunit.client.0.smithi042.stdout:8/63: dread da/fd [4194304,4194304] 0 2024-09-16T23:48:46.833 INFO:tasks.workunit.client.0.smithi042.stdout:1/72: write d0/f5 [5163056,102241] 0 2024-09-16T23:48:46.833 INFO:tasks.workunit.client.0.smithi042.stdout:1/73: stat d0/d9/f11 0 2024-09-16T23:48:46.835 INFO:tasks.workunit.client.0.smithi042.stdout:1/74: write d0/f3 [3788254,51655] 0 2024-09-16T23:48:46.835 INFO:tasks.workunit.client.1.smithi043.stdout:3/154: readlink d4/lc 0 2024-09-16T23:48:46.838 INFO:tasks.workunit.client.0.smithi042.stdout:1/75: symlink d0/d17/l19 0 2024-09-16T23:48:46.839 INFO:tasks.workunit.client.1.smithi043.stdout:1/272: mkdir d4/d3e/d5b 0 2024-09-16T23:48:46.839 INFO:tasks.workunit.client.1.smithi043.stdout:1/273: readlink d4/l8 0 2024-09-16T23:48:46.840 INFO:tasks.workunit.client.1.smithi043.stdout:3/155: dread d4/f1b [0,4194304] 0 2024-09-16T23:48:46.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/64: dwrite da/fd [0,4194304] 0 2024-09-16T23:48:46.843 INFO:tasks.workunit.client.0.smithi042.stdout:8/65: dread da/fd [4194304,4194304] 0 2024-09-16T23:48:46.845 INFO:tasks.workunit.client.0.smithi042.stdout:8/66: write f6 [3652761,72104] 0 2024-09-16T23:48:46.846 INFO:tasks.workunit.client.1.smithi043.stdout:3/156: dwrite d4/de/f15 [0,4194304] 0 2024-09-16T23:48:46.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/76: dwrite d0/f5 [0,4194304] 0 2024-09-16T23:48:46.847 INFO:tasks.workunit.client.0.smithi042.stdout:1/77: readlink d0/d17/l19 0 2024-09-16T23:48:46.847 INFO:tasks.workunit.client.0.smithi042.stdout:1/78: chown d0/d17/d18 342916636 1 2024-09-16T23:48:46.849 INFO:tasks.workunit.client.1.smithi043.stdout:8/203: fdatasync de/f23 0 2024-09-16T23:48:46.849 INFO:tasks.workunit.client.0.smithi042.stdout:8/67: creat da/f14 x:0 0 0 2024-09-16T23:48:46.849 INFO:tasks.workunit.client.0.smithi042.stdout:8/68: chown da/c11 496 1 2024-09-16T23:48:46.850 INFO:tasks.workunit.client.1.smithi043.stdout:1/274: rename d4/df/d13/f40 to d4/d9/d12/f5c 0 2024-09-16T23:48:46.853 INFO:tasks.workunit.client.0.smithi042.stdout:8/69: symlink da/l15 0 2024-09-16T23:48:46.855 INFO:tasks.workunit.client.1.smithi043.stdout:6/194: link de/f13 de/d1a/f41 0 2024-09-16T23:48:46.855 INFO:tasks.workunit.client.0.smithi042.stdout:1/79: link d0/d9/cd d0/d15/c1a 0 2024-09-16T23:48:46.856 INFO:tasks.workunit.client.0.smithi042.stdout:1/80: write d0/d9/df/f12 [230572,28244] 0 2024-09-16T23:48:46.857 INFO:tasks.workunit.client.0.smithi042.stdout:1/81: dread - d0/fa zero size 2024-09-16T23:48:46.862 INFO:tasks.workunit.client.1.smithi043.stdout:1/275: mkdir d4/d9/d1c/d1e/d42/d5d 0 2024-09-16T23:48:46.862 INFO:tasks.workunit.client.0.smithi042.stdout:1/82: symlink d0/d17/l1b 0 2024-09-16T23:48:46.864 INFO:tasks.workunit.client.1.smithi043.stdout:3/157: symlink d4/de/d1f/d2c/l2d 0 2024-09-16T23:48:46.866 INFO:tasks.workunit.client.0.smithi042.stdout:1/83: fsync d0/f6 0 2024-09-16T23:48:46.869 INFO:tasks.workunit.client.1.smithi043.stdout:6/195: rename f1 to de/d1a/f42 0 2024-09-16T23:48:46.870 INFO:tasks.workunit.client.1.smithi043.stdout:7/209: dwrite d4/db/d17/f26 [4194304,4194304] 0 2024-09-16T23:48:46.872 INFO:tasks.workunit.client.1.smithi043.stdout:1/276: symlink d4/d9/d1c/d1e/d42/l5e 0 2024-09-16T23:48:46.873 INFO:tasks.workunit.client.1.smithi043.stdout:6/196: mknod de/d23/c43 0 2024-09-16T23:48:46.873 INFO:tasks.workunit.client.0.smithi042.stdout:1/84: creat d0/d9/f1c x:0 0 0 2024-09-16T23:48:46.875 INFO:tasks.workunit.client.1.smithi043.stdout:3/158: dwrite d4/fb [4194304,4194304] 0 2024-09-16T23:48:46.876 INFO:tasks.workunit.client.1.smithi043.stdout:7/210: mknod d4/db/d17/c3e 0 2024-09-16T23:48:46.877 INFO:tasks.workunit.client.0.smithi042.stdout:1/85: unlink d0/fa 0 2024-09-16T23:48:46.878 INFO:tasks.workunit.client.1.smithi043.stdout:1/277: mknod d4/d9/c5f 0 2024-09-16T23:48:46.880 INFO:tasks.workunit.client.0.smithi042.stdout:1/86: write d0/d9/f11 [788418,7731] 0 2024-09-16T23:48:46.880 INFO:tasks.workunit.client.1.smithi043.stdout:8/204: write de/d10/f1b [4064831,96979] 0 2024-09-16T23:48:46.881 INFO:tasks.workunit.client.1.smithi043.stdout:3/159: dread d4/f16 [0,4194304] 0 2024-09-16T23:48:46.881 INFO:tasks.workunit.client.1.smithi043.stdout:2/69: sync 2024-09-16T23:48:46.882 INFO:tasks.workunit.client.1.smithi043.stdout:3/160: chown d4/lc 11765 1 2024-09-16T23:48:46.883 INFO:tasks.workunit.client.1.smithi043.stdout:3/161: truncate d4/f23 390584 0 2024-09-16T23:48:46.883 INFO:tasks.workunit.client.1.smithi043.stdout:7/211: creat d4/d27/f3f x:0 0 0 2024-09-16T23:48:46.885 INFO:tasks.workunit.client.0.smithi042.stdout:8/70: getdents da 0 2024-09-16T23:48:46.886 INFO:tasks.workunit.client.1.smithi043.stdout:6/197: rename de/d1a/d39/c36 to de/d1a/c44 0 2024-09-16T23:48:46.887 INFO:tasks.workunit.client.0.smithi042.stdout:1/87: dwrite d0/f5 [4194304,4194304] 0 2024-09-16T23:48:46.888 INFO:tasks.workunit.client.1.smithi043.stdout:8/205: chown de/d18/d1f/d25/f3b 0 1 2024-09-16T23:48:46.889 INFO:tasks.workunit.client.0.smithi042.stdout:8/71: creat da/f16 x:0 0 0 2024-09-16T23:48:46.889 INFO:tasks.workunit.client.0.smithi042.stdout:1/88: creat d0/f1d x:0 0 0 2024-09-16T23:48:46.889 INFO:tasks.workunit.client.1.smithi043.stdout:8/206: stat de/d10/f26 0 2024-09-16T23:48:46.890 INFO:tasks.workunit.client.0.smithi042.stdout:8/72: chown da/f14 656 1 2024-09-16T23:48:46.891 INFO:tasks.workunit.client.1.smithi043.stdout:3/162: unlink d4/fd 0 2024-09-16T23:48:46.891 INFO:tasks.workunit.client.1.smithi043.stdout:8/207: read de/d18/f36 [8344900,4173] 0 2024-09-16T23:48:46.892 INFO:tasks.workunit.client.1.smithi043.stdout:7/212: read d4/db/d17/f26 [4681993,53150] 0 2024-09-16T23:48:46.893 INFO:tasks.workunit.client.1.smithi043.stdout:2/70: truncate f7 902910 0 2024-09-16T23:48:46.894 INFO:tasks.workunit.client.1.smithi043.stdout:6/198: mknod de/d23/c45 0 2024-09-16T23:48:46.894 INFO:tasks.workunit.client.0.smithi042.stdout:1/89: dread d0/f5 [0,4194304] 0 2024-09-16T23:48:46.894 INFO:tasks.workunit.client.0.smithi042.stdout:8/73: unlink da/fd 0 2024-09-16T23:48:46.895 INFO:tasks.workunit.client.1.smithi043.stdout:6/199: chown c5 12483 1 2024-09-16T23:48:46.895 INFO:tasks.workunit.client.1.smithi043.stdout:1/278: truncate d4/d9/f17 19530 0 2024-09-16T23:48:46.898 INFO:tasks.workunit.client.1.smithi043.stdout:8/208: stat de/d18/d27/c47 0 2024-09-16T23:48:46.899 INFO:tasks.workunit.client.1.smithi043.stdout:2/71: mknod d8/c14 0 2024-09-16T23:48:46.900 INFO:tasks.workunit.client.0.smithi042.stdout:8/74: rename f6 to da/f17 0 2024-09-16T23:48:46.901 INFO:tasks.workunit.client.1.smithi043.stdout:6/200: dread de/d1a/d39/f16 [0,4194304] 0 2024-09-16T23:48:46.901 INFO:tasks.workunit.client.0.smithi042.stdout:1/90: dwrite d0/f3 [0,4194304] 0 2024-09-16T23:48:46.903 INFO:tasks.workunit.client.0.smithi042.stdout:8/75: creat da/f18 x:0 0 0 2024-09-16T23:48:46.903 INFO:tasks.workunit.client.1.smithi043.stdout:1/279: unlink d4/d9/d12/d2b/f45 0 2024-09-16T23:48:46.904 INFO:tasks.workunit.client.0.smithi042.stdout:1/91: rmdir d0/d9/df 39 2024-09-16T23:48:46.905 INFO:tasks.workunit.client.1.smithi043.stdout:1/280: write d4/d9/d1c/f37 [3358866,57506] 0 2024-09-16T23:48:46.905 INFO:tasks.workunit.client.0.smithi042.stdout:1/92: fsync d0/d9/f11 0 2024-09-16T23:48:46.907 INFO:tasks.workunit.client.1.smithi043.stdout:3/163: symlink d4/d17/l2e 0 2024-09-16T23:48:46.907 INFO:tasks.workunit.client.0.smithi042.stdout:8/76: mkdir da/d19 0 2024-09-16T23:48:46.909 INFO:tasks.workunit.client.1.smithi043.stdout:8/209: rename de/d18/d1f/l31 to de/d18/d1f/d25/l49 0 2024-09-16T23:48:46.911 INFO:tasks.workunit.client.0.smithi042.stdout:8/77: dread da/f17 [0,4194304] 0 2024-09-16T23:48:46.911 INFO:tasks.workunit.client.1.smithi043.stdout:2/72: mknod d8/d11/c15 0 2024-09-16T23:48:46.914 INFO:tasks.workunit.client.1.smithi043.stdout:6/201: creat de/d1a/d28/f46 x:0 0 0 2024-09-16T23:48:46.915 INFO:tasks.workunit.client.0.smithi042.stdout:8/78: creat da/d19/f1a x:0 0 0 2024-09-16T23:48:46.917 INFO:tasks.workunit.client.0.smithi042.stdout:8/79: mkdir da/d1b 0 2024-09-16T23:48:46.917 INFO:tasks.workunit.client.1.smithi043.stdout:1/281: mknod d4/df/d26/c60 0 2024-09-16T23:48:46.919 INFO:tasks.workunit.client.0.smithi042.stdout:1/93: dwrite d0/d9/df/f13 [0,4194304] 0 2024-09-16T23:48:46.921 INFO:tasks.workunit.client.0.smithi042.stdout:1/94: chown d0/d15/c1a 2049422538 1 2024-09-16T23:48:46.922 INFO:tasks.workunit.client.0.smithi042.stdout:1/95: chown d0/d9/c14 32440161 1 2024-09-16T23:48:46.923 INFO:tasks.workunit.client.0.smithi042.stdout:1/96: write d0/f1d [428407,92508] 0 2024-09-16T23:48:46.923 INFO:tasks.workunit.client.0.smithi042.stdout:8/80: rmdir da/d1b 0 2024-09-16T23:48:46.927 INFO:tasks.workunit.client.0.smithi042.stdout:8/81: mknod da/d19/c1c 0 2024-09-16T23:48:46.928 INFO:tasks.workunit.client.0.smithi042.stdout:8/82: dread - da/f14 zero size 2024-09-16T23:48:46.928 INFO:tasks.workunit.client.1.smithi043.stdout:8/210: dwrite de/d18/f36 [0,4194304] 0 2024-09-16T23:48:46.930 INFO:tasks.workunit.client.1.smithi043.stdout:1/282: creat d4/d9/d12/d2b/d3d/f61 x:0 0 0 2024-09-16T23:48:46.931 INFO:tasks.workunit.client.0.smithi042.stdout:1/97: link d0/f5 d0/d9/df/f1e 0 2024-09-16T23:48:46.932 INFO:tasks.workunit.client.0.smithi042.stdout:8/83: mkdir da/d19/d1d 0 2024-09-16T23:48:46.932 INFO:tasks.workunit.client.1.smithi043.stdout:6/202: creat de/f47 x:0 0 0 2024-09-16T23:48:46.933 INFO:tasks.workunit.client.1.smithi043.stdout:1/283: write d4/d9/d1c/d1e/d47/f4f [266779,59843] 0 2024-09-16T23:48:46.934 INFO:tasks.workunit.client.1.smithi043.stdout:1/284: chown d4/df/d26/c44 1 1 2024-09-16T23:48:46.935 INFO:tasks.workunit.client.0.smithi042.stdout:8/84: creat da/d19/f1e x:0 0 0 2024-09-16T23:48:46.935 INFO:tasks.workunit.client.1.smithi043.stdout:1/285: write d4/df/f38 [369600,5637] 0 2024-09-16T23:48:46.935 INFO:tasks.workunit.client.1.smithi043.stdout:8/211: read de/d10/f1b [1788038,96693] 0 2024-09-16T23:48:46.936 INFO:tasks.workunit.client.1.smithi043.stdout:1/286: fdatasync d4/df/d13/d1b/f59 0 2024-09-16T23:48:46.937 INFO:tasks.workunit.client.1.smithi043.stdout:8/212: truncate de/d18/d1f/d2f/f43 668624 0 2024-09-16T23:48:46.938 INFO:tasks.workunit.client.1.smithi043.stdout:3/164: creat d4/de/f2f x:0 0 0 2024-09-16T23:48:46.939 INFO:tasks.workunit.client.1.smithi043.stdout:3/165: chown d4/d17/l2e 54 1 2024-09-16T23:48:46.940 INFO:tasks.workunit.client.1.smithi043.stdout:6/203: chown de/l2f 0 1 2024-09-16T23:48:46.945 INFO:tasks.workunit.client.0.smithi042.stdout:8/85: dwrite da/f18 [0,4194304] 0 2024-09-16T23:48:46.945 INFO:tasks.workunit.client.0.smithi042.stdout:8/86: chown c5 0 1 2024-09-16T23:48:46.946 INFO:tasks.workunit.client.1.smithi043.stdout:3/166: write d4/fb [1619602,96824] 0 2024-09-16T23:48:46.946 INFO:tasks.workunit.client.0.smithi042.stdout:8/87: write da/f16 [158817,12895] 0 2024-09-16T23:48:46.950 INFO:tasks.workunit.client.1.smithi043.stdout:6/204: dread de/d1a/d28/f35 [0,4194304] 0 2024-09-16T23:48:46.951 INFO:tasks.workunit.client.0.smithi042.stdout:8/88: creat da/d19/f1f x:0 0 0 2024-09-16T23:48:46.953 INFO:tasks.workunit.client.1.smithi043.stdout:1/287: rename d4/d9/d12/f54 to d4/f62 0 2024-09-16T23:48:46.955 INFO:tasks.workunit.client.1.smithi043.stdout:1/288: chown d4/d9/d1c/d1e/d42/c5a 1 1 2024-09-16T23:48:46.956 INFO:tasks.workunit.client.1.smithi043.stdout:6/205: symlink de/d2a/l48 0 2024-09-16T23:48:46.958 INFO:tasks.workunit.client.1.smithi043.stdout:8/213: rename de/f14 to de/d10/f4a 0 2024-09-16T23:48:46.958 INFO:tasks.workunit.client.0.smithi042.stdout:8/89: dwrite da/f18 [0,4194304] 0 2024-09-16T23:48:46.959 INFO:tasks.workunit.client.1.smithi043.stdout:6/206: dread de/d1a/d39/f16 [0,4194304] 0 2024-09-16T23:48:46.959 INFO:tasks.workunit.client.0.smithi042.stdout:8/90: chown c8 251048 1 2024-09-16T23:48:46.959 INFO:tasks.workunit.client.1.smithi043.stdout:1/289: symlink d4/d9/d12/d15/l63 0 2024-09-16T23:48:46.959 INFO:tasks.workunit.client.0.smithi042.stdout:8/91: truncate da/f18 4547133 0 2024-09-16T23:48:46.961 INFO:tasks.workunit.client.0.smithi042.stdout:8/92: write da/d19/f1f [670317,7802] 0 2024-09-16T23:48:46.961 INFO:tasks.workunit.client.1.smithi043.stdout:6/207: write fd [949604,7310] 0 2024-09-16T23:48:46.962 INFO:tasks.workunit.client.1.smithi043.stdout:3/167: rmdir d4/d17/d2b 0 2024-09-16T23:48:46.962 INFO:tasks.workunit.client.0.smithi042.stdout:8/93: chown da/d19 7110 1 2024-09-16T23:48:46.964 INFO:tasks.workunit.client.1.smithi043.stdout:1/290: chown d4/d9/d12/d15/l22 160 1 2024-09-16T23:48:46.965 INFO:tasks.workunit.client.1.smithi043.stdout:6/208: chown de/l2f 990219 1 2024-09-16T23:48:46.965 INFO:tasks.workunit.client.0.smithi042.stdout:8/94: mknod da/d19/d1d/c20 0 2024-09-16T23:48:46.965 INFO:tasks.workunit.client.1.smithi043.stdout:2/73: dwrite f7 [0,4194304] 0 2024-09-16T23:48:46.965 INFO:tasks.workunit.client.0.smithi042.stdout:8/95: read da/f18 [1473974,115270] 0 2024-09-16T23:48:46.968 INFO:tasks.workunit.client.0.smithi042.stdout:8/96: symlink da/d19/l21 0 2024-09-16T23:48:46.968 INFO:tasks.workunit.client.1.smithi043.stdout:1/291: write d4/d34/f3a [469,45182] 0 2024-09-16T23:48:46.971 INFO:tasks.workunit.client.1.smithi043.stdout:8/214: dwrite de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:48:46.972 INFO:tasks.workunit.client.1.smithi043.stdout:8/215: write de/f45 [579585,74675] 0 2024-09-16T23:48:46.974 INFO:tasks.workunit.client.1.smithi043.stdout:2/74: mkdir d8/d11/d16 0 2024-09-16T23:48:46.981 INFO:tasks.workunit.client.1.smithi043.stdout:1/292: rename d4/d9/d12/f10 to d4/df/d13/d1b/f64 0 2024-09-16T23:48:46.985 INFO:tasks.workunit.client.1.smithi043.stdout:1/293: mknod d4/df/c65 0 2024-09-16T23:48:46.988 INFO:tasks.workunit.client.1.smithi043.stdout:2/75: dwrite d8/fb [0,4194304] 0 2024-09-16T23:48:46.994 INFO:tasks.workunit.client.0.smithi042.stdout:8/97: dwrite da/d19/f1e [0,4194304] 0 2024-09-16T23:48:46.995 INFO:tasks.workunit.client.0.smithi042.stdout:8/98: fdatasync da/f14 0 2024-09-16T23:48:46.996 INFO:tasks.workunit.client.0.smithi042.stdout:8/99: write da/f14 [835361,97918] 0 2024-09-16T23:48:46.996 INFO:tasks.workunit.client.1.smithi043.stdout:8/216: dwrite de/d10/f4a [4194304,4194304] 0 2024-09-16T23:48:47.002 INFO:tasks.workunit.client.1.smithi043.stdout:8/217: rmdir de/d18/d2b 39 2024-09-16T23:48:47.003 INFO:tasks.workunit.client.1.smithi043.stdout:3/168: dread d4/f19 [0,4194304] 0 2024-09-16T23:48:47.004 INFO:tasks.workunit.client.1.smithi043.stdout:6/209: dwrite de/d1a/f32 [0,4194304] 0 2024-09-16T23:48:47.005 INFO:tasks.workunit.client.1.smithi043.stdout:1/294: dread d4/d9/d12/d15/f31 [0,4194304] 0 2024-09-16T23:48:47.005 INFO:tasks.workunit.client.1.smithi043.stdout:3/169: fsync d4/de/f21 0 2024-09-16T23:48:47.008 INFO:tasks.workunit.client.1.smithi043.stdout:2/76: dwrite d8/fc [0,4194304] 0 2024-09-16T23:48:47.014 INFO:tasks.workunit.client.1.smithi043.stdout:1/295: creat d4/d9/d1c/d1e/f66 x:0 0 0 2024-09-16T23:48:47.017 INFO:tasks.workunit.client.1.smithi043.stdout:6/210: fsync de/d1a/f41 0 2024-09-16T23:48:47.018 INFO:tasks.workunit.client.1.smithi043.stdout:2/77: dread f7 [0,4194304] 0 2024-09-16T23:48:47.021 INFO:tasks.workunit.client.1.smithi043.stdout:1/296: fsync d4/d9/d12/f5c 0 2024-09-16T23:48:47.022 INFO:tasks.workunit.client.1.smithi043.stdout:6/211: mkdir de/d23/d49 0 2024-09-16T23:48:47.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:46 smithi042 ceph-mon[27914]: pgmap v262: 65 pgs: 65 active+clean; 267 MiB data, 1004 MiB used, 535 GiB / 536 GiB avail; 184 KiB/s rd, 11 MiB/s wr, 272 op/s 2024-09-16T23:48:47.024 INFO:tasks.workunit.client.1.smithi043.stdout:6/212: readlink de/d1a/d39/l30 0 2024-09-16T23:48:47.024 INFO:tasks.workunit.client.1.smithi043.stdout:3/170: dwrite d4/de/d1f/f29 [0,4194304] 0 2024-09-16T23:48:47.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.027+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe24c105230 con 0x7fe23806d440 2024-09-16T23:48:47.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.029+0000 7fe248d99700 1 -- 172.21.15.42:0/2123978463 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe24c105230 con 0x7fe23806d440 2024-09-16T23:48:47.027 INFO:tasks.workunit.client.1.smithi043.stdout:3/171: chown d4/l14 58 1 2024-09-16T23:48:47.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe23806d440 msgr2=0x7fe23806f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe23806d440 0x7fe23806f900 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe234009550 tx=0x7fe2340091b0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 msgr2=0x7fe24c0718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 0x7fe24c0718f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0098e0 tx=0x7fe23c0048c0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 shutdown_connections 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7fe23806d440 0x7fe23806f900 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe24c0ffdf0 0x7fe24c0718f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 --2- 172.21.15.42:0/2123978463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe24c101220 0x7fe24c071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 >> 172.21.15.42:0/2123978463 conn(0x7fe24c0fa8c0 msgr2=0x7fe24c0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 shutdown_connections 2024-09-16T23:48:47.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.032+0000 7fe251240700 1 -- 172.21.15.42:0/2123978463 wait complete. 2024-09-16T23:48:47.032 INFO:tasks.workunit.client.1.smithi043.stdout:3/172: mkdir d4/d30 0 2024-09-16T23:48:47.033 INFO:tasks.workunit.client.1.smithi043.stdout:6/213: link de/d23/d34/l3d de/l4a 0 2024-09-16T23:48:47.033 INFO:tasks.workunit.client.1.smithi043.stdout:1/297: getdents d4/d9 0 2024-09-16T23:48:47.034 INFO:tasks.workunit.client.1.smithi043.stdout:2/78: write f6 [945934,58740] 0 2024-09-16T23:48:47.034 INFO:tasks.workunit.client.1.smithi043.stdout:3/173: write d4/d17/f1d [4044639,12547] 0 2024-09-16T23:48:47.036 INFO:tasks.workunit.client.1.smithi043.stdout:2/79: stat d8/c14 0 2024-09-16T23:48:47.038 INFO:tasks.workunit.client.1.smithi043.stdout:3/174: write d4/f23 [74654,50531] 0 2024-09-16T23:48:47.040 INFO:tasks.workunit.client.1.smithi043.stdout:6/214: dread de/d1a/f41 [0,4194304] 0 2024-09-16T23:48:47.040 INFO:tasks.workunit.client.1.smithi043.stdout:2/80: rename d8/fb to d8/d11/f17 0 2024-09-16T23:48:47.041 INFO:tasks.workunit.client.1.smithi043.stdout:1/298: dread d4/d34/f3a [0,4194304] 0 2024-09-16T23:48:47.042 INFO:tasks.workunit.client.1.smithi043.stdout:6/215: symlink de/d2a/l4b 0 2024-09-16T23:48:47.043 INFO:tasks.workunit.client.1.smithi043.stdout:6/216: stat de/d1a/d39/f16 0 2024-09-16T23:48:47.043 INFO:tasks.workunit.client.1.smithi043.stdout:2/81: mknod d8/d11/c18 0 2024-09-16T23:48:47.043 INFO:tasks.workunit.client.1.smithi043.stdout:2/82: dread - d8/ff zero size 2024-09-16T23:48:47.044 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:48:47.044 INFO:tasks.workunit.client.1.smithi043.stdout:1/299: symlink d4/d9/d1c/d1e/d42/l67 0 2024-09-16T23:48:47.045 INFO:tasks.workunit.client.1.smithi043.stdout:1/300: fsync d4/df/d13/d1b/f2f 0 2024-09-16T23:48:47.046 INFO:tasks.workunit.client.1.smithi043.stdout:6/217: link de/d1a/f38 de/d23/d49/f4c 0 2024-09-16T23:48:47.046 INFO:tasks.workunit.client.1.smithi043.stdout:3/175: link d4/c26 d4/d30/c31 0 2024-09-16T23:48:47.047 INFO:tasks.workunit.client.1.smithi043.stdout:3/176: stat d4 0 2024-09-16T23:48:47.047 INFO:tasks.workunit.client.1.smithi043.stdout:2/83: creat d8/d11/d16/f19 x:0 0 0 2024-09-16T23:48:47.048 INFO:tasks.workunit.client.1.smithi043.stdout:6/218: mkdir de/d1a/d4d 0 2024-09-16T23:48:47.049 INFO:tasks.workunit.client.1.smithi043.stdout:3/177: rmdir d4/d30 39 2024-09-16T23:48:47.050 INFO:tasks.workunit.client.1.smithi043.stdout:2/84: unlink d8/le 0 2024-09-16T23:48:47.053 INFO:tasks.workunit.client.1.smithi043.stdout:2/85: read - d8/fd zero size 2024-09-16T23:48:47.053 INFO:tasks.workunit.client.1.smithi043.stdout:6/219: creat de/d23/f4e x:0 0 0 2024-09-16T23:48:47.054 INFO:tasks.workunit.client.1.smithi043.stdout:1/301: dwrite d4/df/d13/d1b/f2f [0,4194304] 0 2024-09-16T23:48:47.058 INFO:tasks.workunit.client.1.smithi043.stdout:2/86: creat d8/f1a x:0 0 0 2024-09-16T23:48:47.059 INFO:tasks.workunit.client.1.smithi043.stdout:6/220: mknod de/d23/c4f 0 2024-09-16T23:48:47.059 INFO:tasks.workunit.client.1.smithi043.stdout:1/302: mknod d4/d9/d1c/d1e/d47/c68 0 2024-09-16T23:48:47.060 INFO:tasks.workunit.client.1.smithi043.stdout:3/178: dwrite d4/de/d1f/f29 [0,4194304] 0 2024-09-16T23:48:47.064 INFO:tasks.workunit.client.1.smithi043.stdout:6/221: fdatasync de/d1a/f42 0 2024-09-16T23:48:47.067 INFO:tasks.workunit.client.1.smithi043.stdout:6/222: chown de/d23/d34/l3f 4542961 1 2024-09-16T23:48:47.068 INFO:tasks.workunit.client.1.smithi043.stdout:3/179: creat d4/de/d1f/d2c/f32 x:0 0 0 2024-09-16T23:48:47.072 INFO:tasks.workunit.client.1.smithi043.stdout:6/223: unlink f3 0 2024-09-16T23:48:47.074 INFO:tasks.workunit.client.1.smithi043.stdout:3/180: dread d4/f1b [0,4194304] 0 2024-09-16T23:48:47.075 INFO:tasks.workunit.client.1.smithi043.stdout:1/303: link d4/d9/d12/d15/f31 d4/f69 0 2024-09-16T23:48:47.079 INFO:tasks.workunit.client.1.smithi043.stdout:2/87: truncate f2 2855851 0 2024-09-16T23:48:47.079 INFO:tasks.workunit.client.1.smithi043.stdout:1/304: fdatasync d4/fc 0 2024-09-16T23:48:47.082 INFO:tasks.workunit.client.1.smithi043.stdout:2/88: chown d8/d11/l12 334 1 2024-09-16T23:48:47.082 INFO:tasks.workunit.client.1.smithi043.stdout:1/305: chown d4/d9/f46 144610945 1 2024-09-16T23:48:47.082 INFO:tasks.workunit.client.1.smithi043.stdout:6/224: write de/d1a/f41 [1304122,3276] 0 2024-09-16T23:48:47.084 INFO:tasks.workunit.client.1.smithi043.stdout:3/181: dwrite d4/de/f27 [0,4194304] 0 2024-09-16T23:48:47.085 INFO:tasks.workunit.client.1.smithi043.stdout:3/182: fdatasync d4/de/f21 0 2024-09-16T23:48:47.086 INFO:tasks.workunit.client.1.smithi043.stdout:1/306: write d4/d9/d12/d2b/f52 [123038,121458] 0 2024-09-16T23:48:47.087 INFO:tasks.workunit.client.1.smithi043.stdout:2/89: mknod d8/d11/c1b 0 2024-09-16T23:48:47.092 INFO:tasks.workunit.client.1.smithi043.stdout:2/90: rename d8/d11/c15 to d8/d11/d16/c1c 0 2024-09-16T23:48:47.092 INFO:tasks.workunit.client.1.smithi043.stdout:3/183: dread d4/de/f21 [0,4194304] 0 2024-09-16T23:48:47.092 INFO:tasks.workunit.client.1.smithi043.stdout:2/91: chown d8/f1a 53 1 2024-09-16T23:48:47.095 INFO:tasks.workunit.client.1.smithi043.stdout:1/307: getdents d4/d9/d1c/d1e/d47 0 2024-09-16T23:48:47.096 INFO:tasks.workunit.client.1.smithi043.stdout:2/92: mkdir d8/d1d 0 2024-09-16T23:48:47.097 INFO:tasks.workunit.client.1.smithi043.stdout:6/225: truncate fd 3382243 0 2024-09-16T23:48:47.101 INFO:tasks.workunit.client.1.smithi043.stdout:2/93: dread f6 [0,4194304] 0 2024-09-16T23:48:47.102 INFO:tasks.workunit.client.1.smithi043.stdout:3/184: dwrite d4/f10 [0,4194304] 0 2024-09-16T23:48:47.105 INFO:tasks.workunit.client.1.smithi043.stdout:2/94: mkdir d8/d11/d16/d1e 0 2024-09-16T23:48:47.105 INFO:tasks.workunit.client.1.smithi043.stdout:6/226: write de/d1a/d28/f3e [3001995,71995] 0 2024-09-16T23:48:47.108 INFO:tasks.workunit.client.1.smithi043.stdout:2/95: mkdir d8/d1f 0 2024-09-16T23:48:47.109 INFO:tasks.workunit.client.1.smithi043.stdout:2/96: dread - d8/ff zero size 2024-09-16T23:48:47.110 INFO:tasks.workunit.client.1.smithi043.stdout:2/97: stat d8/d11/c1b 0 2024-09-16T23:48:47.113 INFO:tasks.workunit.client.1.smithi043.stdout:6/227: dwrite de/d1a/d28/f46 [0,4194304] 0 2024-09-16T23:48:47.113 INFO:tasks.workunit.client.1.smithi043.stdout:2/98: mkdir d8/d1d/d20 0 2024-09-16T23:48:47.114 INFO:tasks.workunit.client.1.smithi043.stdout:6/228: write f2 [1994598,99746] 0 2024-09-16T23:48:47.116 INFO:tasks.workunit.client.1.smithi043.stdout:6/229: write de/d23/f4e [295828,25813] 0 2024-09-16T23:48:47.117 INFO:tasks.workunit.client.1.smithi043.stdout:2/99: symlink d8/d1f/l21 0 2024-09-16T23:48:47.118 INFO:tasks.workunit.client.1.smithi043.stdout:6/230: fsync f2 0 2024-09-16T23:48:47.120 INFO:tasks.workunit.client.1.smithi043.stdout:2/100: mknod d8/d11/d16/c22 0 2024-09-16T23:48:47.122 INFO:tasks.workunit.client.1.smithi043.stdout:6/231: rename de/d37 to de/d1a/d4d/d50 0 2024-09-16T23:48:47.123 INFO:tasks.workunit.client.1.smithi043.stdout:6/232: chown c7 61 1 2024-09-16T23:48:47.125 INFO:tasks.workunit.client.1.smithi043.stdout:2/101: mkdir d8/d1d/d20/d23 0 2024-09-16T23:48:47.125 INFO:tasks.workunit.client.1.smithi043.stdout:6/233: mkdir de/d1a/d4d/d51 0 2024-09-16T23:48:47.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:46 smithi043 ceph-mon[35191]: pgmap v262: 65 pgs: 65 active+clean; 267 MiB data, 1004 MiB used, 535 GiB / 536 GiB avail; 184 KiB/s rd, 11 MiB/s wr, 272 op/s 2024-09-16T23:48:47.127 INFO:tasks.workunit.client.1.smithi043.stdout:2/102: write d8/d11/f17 [895817,28800] 0 2024-09-16T23:48:47.128 INFO:tasks.workunit.client.1.smithi043.stdout:2/103: dread - d8/f10 zero size 2024-09-16T23:48:47.130 INFO:tasks.workunit.client.1.smithi043.stdout:2/104: creat d8/f24 x:0 0 0 2024-09-16T23:48:47.130 INFO:tasks.workunit.client.1.smithi043.stdout:2/105: dread - d8/fd zero size 2024-09-16T23:48:47.133 INFO:tasks.workunit.client.1.smithi043.stdout:6/234: dread fd [0,4194304] 0 2024-09-16T23:48:47.133 INFO:tasks.workunit.client.1.smithi043.stdout:2/106: mkdir d8/d1d/d25 0 2024-09-16T23:48:47.133 INFO:tasks.workunit.client.1.smithi043.stdout:6/235: stat de/f24 0 2024-09-16T23:48:47.134 INFO:tasks.workunit.client.1.smithi043.stdout:2/107: write f7 [2151051,72948] 0 2024-09-16T23:48:47.135 INFO:tasks.workunit.client.1.smithi043.stdout:6/236: write de/d1a/f41 [249819,76980] 0 2024-09-16T23:48:47.136 INFO:tasks.workunit.client.1.smithi043.stdout:2/108: write d8/f1a [880635,37667] 0 2024-09-16T23:48:47.137 INFO:tasks.workunit.client.1.smithi043.stdout:6/237: symlink de/d23/d49/l52 0 2024-09-16T23:48:47.139 INFO:tasks.workunit.client.1.smithi043.stdout:2/109: unlink f7 0 2024-09-16T23:48:47.140 INFO:tasks.workunit.client.1.smithi043.stdout:6/238: fsync fd 0 2024-09-16T23:48:47.141 INFO:tasks.workunit.client.1.smithi043.stdout:2/110: rmdir d8/d1f 39 2024-09-16T23:48:47.141 INFO:tasks.workunit.client.1.smithi043.stdout:2/111: stat d8/c14 0 2024-09-16T23:48:47.142 INFO:tasks.workunit.client.1.smithi043.stdout:2/112: chown d8/la 972957655 1 2024-09-16T23:48:47.142 INFO:tasks.workunit.client.1.smithi043.stdout:2/113: truncate f6 1418981 0 2024-09-16T23:48:47.145 INFO:tasks.workunit.client.1.smithi043.stdout:6/239: rename de/c21 to de/d23/d49/c53 0 2024-09-16T23:48:47.146 INFO:tasks.workunit.client.1.smithi043.stdout:2/114: mkdir d8/d1d/d20/d26 0 2024-09-16T23:48:47.148 INFO:tasks.workunit.client.1.smithi043.stdout:6/240: symlink de/d23/d49/l54 0 2024-09-16T23:48:47.148 INFO:tasks.workunit.client.1.smithi043.stdout:2/115: dread f6 [0,4194304] 0 2024-09-16T23:48:47.151 INFO:tasks.workunit.client.1.smithi043.stdout:2/116: creat d8/d1d/d20/d26/f27 x:0 0 0 2024-09-16T23:48:47.153 INFO:tasks.workunit.client.1.smithi043.stdout:2/117: write d8/ff [771837,17850] 0 2024-09-16T23:48:47.154 INFO:tasks.workunit.client.1.smithi043.stdout:2/118: truncate d8/f1a 1496225 0 2024-09-16T23:48:47.156 INFO:tasks.workunit.client.1.smithi043.stdout:2/119: creat d8/d11/d16/f28 x:0 0 0 2024-09-16T23:48:47.157 INFO:tasks.workunit.client.1.smithi043.stdout:2/120: mknod d8/d11/d16/c29 0 2024-09-16T23:48:47.158 INFO:tasks.workunit.client.1.smithi043.stdout:2/121: write d8/fc [1110703,66784] 0 2024-09-16T23:48:47.160 INFO:tasks.workunit.client.1.smithi043.stdout:6/241: dread de/d1a/f42 [0,4194304] 0 2024-09-16T23:48:47.160 INFO:tasks.workunit.client.1.smithi043.stdout:2/122: truncate d8/d11/d16/f28 995270 0 2024-09-16T23:48:47.161 INFO:tasks.workunit.client.1.smithi043.stdout:2/123: write d8/d1d/d20/d26/f27 [164237,101470] 0 2024-09-16T23:48:47.165 INFO:tasks.workunit.client.1.smithi043.stdout:6/242: dread de/d1a/d28/f35 [0,4194304] 0 2024-09-16T23:48:47.165 INFO:tasks.workunit.client.1.smithi043.stdout:2/124: rename d8/d11/f17 to d8/d1d/f2a 0 2024-09-16T23:48:47.166 INFO:tasks.workunit.client.1.smithi043.stdout:6/243: chown de/d1a/d39/c3c 0 1 2024-09-16T23:48:47.167 INFO:tasks.workunit.client.1.smithi043.stdout:2/125: creat d8/d1d/d25/f2b x:0 0 0 2024-09-16T23:48:47.168 INFO:tasks.workunit.client.1.smithi043.stdout:2/126: read - d8/f10 zero size 2024-09-16T23:48:47.174 INFO:tasks.workunit.client.1.smithi043.stdout:2/127: getdents d8/d1d 0 2024-09-16T23:48:47.175 INFO:tasks.workunit.client.1.smithi043.stdout:6/244: write de/d1a/f42 [244334,18500] 0 2024-09-16T23:48:47.176 INFO:tasks.workunit.client.1.smithi043.stdout:2/128: mknod d8/d1d/d25/c2c 0 2024-09-16T23:48:47.177 INFO:tasks.workunit.client.1.smithi043.stdout:6/245: fsync de/d1a/d28/f3e 0 2024-09-16T23:48:47.177 INFO:tasks.workunit.client.1.smithi043.stdout:2/129: truncate d8/f10 583878 0 2024-09-16T23:48:47.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.180+0000 7f5505d09700 1 -- 172.21.15.42:0/728387902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001017a0 msgr2=0x7f5500103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.180+0000 7f5505d09700 1 --2- 172.21.15.42:0/728387902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001017a0 0x7f5500103b90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009910 tx=0x7f54f0009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.180+0000 7f5505d09700 1 -- 172.21.15.42:0/728387902 shutdown_connections 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.180+0000 7f5505d09700 1 --2- 172.21.15.42:0/728387902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55001040d0 0x7f55001064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.180+0000 7f5505d09700 1 --2- 172.21.15.42:0/728387902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001017a0 0x7f5500103b90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.180+0000 7f5505d09700 1 -- 172.21.15.42:0/728387902 >> 172.21.15.42:0/728387902 conn(0x7f55000fb190 msgr2=0x7f55000fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.181+0000 7f5505d09700 1 -- 172.21.15.42:0/728387902 shutdown_connections 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.181+0000 7f5505d09700 1 -- 172.21.15.42:0/728387902 wait complete. 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.181+0000 7f5505d09700 1 Processor -- start 2024-09-16T23:48:47.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.181+0000 7f5505d09700 1 -- start start 2024-09-16T23:48:47.180 INFO:tasks.workunit.client.1.smithi043.stdout:2/130: read d8/f1a [1306860,31747] 0 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f5505d09700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55001017a0 0x7f5500101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f5505d09700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001040d0 0x7f55000ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f5505d09700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55000ffc90 con 0x7f55001040d0 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f5505d09700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55000ffdd0 con 0x7f55001017a0 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54feffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001040d0 0x7f55000ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54feffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001040d0 0x7f55000ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59676/0 (socket says 172.21.15.42:59676) 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54feffd700 1 -- 172.21.15.42:0/4040411427 learned_addr learned my addr 172.21.15.42:0/4040411427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54ff7fe700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55001017a0 0x7f5500101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54feffd700 1 -- 172.21.15.42:0/4040411427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55001017a0 msgr2=0x7f5500101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54feffd700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55001017a0 0x7f5500101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54feffd700 1 -- 172.21.15.42:0/4040411427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00095f0 con 0x7f55001040d0 2024-09-16T23:48:47.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54feffd700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001040d0 0x7f55000ff750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f54f400d8d0 tx=0x7f54f400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:47.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.182+0000 7f54fcff9700 1 -- 172.21.15.42:0/4040411427 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f40098a0 con 0x7f55001040d0 2024-09-16T23:48:47.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.183+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55001000b0 con 0x7f55001040d0 2024-09-16T23:48:47.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.183+0000 7f54fcff9700 1 -- 172.21.15.42:0/4040411427 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f54f4010460 con 0x7f55001040d0 2024-09-16T23:48:47.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.183+0000 7f54fcff9700 1 -- 172.21.15.42:0/4040411427 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54f400f5f0 con 0x7f55001040d0 2024-09-16T23:48:47.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.183+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550019e4c0 con 0x7f55001040d0 2024-09-16T23:48:47.183 INFO:tasks.workunit.client.1.smithi043.stdout:6/246: dread fd [0,4194304] 0 2024-09-16T23:48:47.183 INFO:tasks.workunit.client.1.smithi043.stdout:2/131: rmdir d8/d11/d16/d1e 0 2024-09-16T23:48:47.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.184+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5500190610 con 0x7f55001040d0 2024-09-16T23:48:47.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.185+0000 7f54fcff9700 1 -- 172.21.15.42:0/4040411427 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) v1 ==== 91358+0+0 (secure 0 0 0) 0x7f54f400f750 con 0x7f55001040d0 2024-09-16T23:48:47.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.185+0000 7f54fcff9700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f54e806d440 0x7f54e806f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:47.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.185+0000 7f54fcff9700 1 -- 172.21.15.42:0/4040411427 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f54f408cb00 con 0x7f55001040d0 2024-09-16T23:48:47.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.186+0000 7f54ff7fe700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f54e806d440 0x7f54e806f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:47.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.186+0000 7f54ff7fe700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f54e806d440 0x7f54e806f900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54f0006010 tx=0x7f54f0005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:47.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.190+0000 7f54fcff9700 1 -- 172.21.15.42:0/4040411427 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f54f405bd20 con 0x7f55001040d0 2024-09-16T23:48:47.188 INFO:tasks.workunit.client.1.smithi043.stdout:6/247: dwrite de/d1a/d39/f1f [0,4194304] 0 2024-09-16T23:48:47.190 INFO:tasks.workunit.client.1.smithi043.stdout:2/132: dwrite d8/f24 [0,4194304] 0 2024-09-16T23:48:47.194 INFO:tasks.workunit.client.1.smithi043.stdout:2/133: write d8/f1a [1945647,79702] 0 2024-09-16T23:48:47.196 INFO:tasks.workunit.client.1.smithi043.stdout:6/248: rename de/d23/d34/l3d to de/d23/d34/l55 0 2024-09-16T23:48:47.198 INFO:tasks.workunit.client.1.smithi043.stdout:6/249: rename de/d1a/d4d to de/d1a/d4d/d50/d56 22 2024-09-16T23:48:47.199 INFO:tasks.workunit.client.1.smithi043.stdout:2/134: mknod d8/c2d 0 2024-09-16T23:48:47.204 INFO:tasks.workunit.client.1.smithi043.stdout:6/250: write de/d2a/f33 [1074371,61027] 0 2024-09-16T23:48:47.204 INFO:tasks.workunit.client.1.smithi043.stdout:6/251: chown f2 1059 1 2024-09-16T23:48:47.207 INFO:tasks.workunit.client.1.smithi043.stdout:6/252: mknod de/d1a/d4d/c57 0 2024-09-16T23:48:47.209 INFO:tasks.workunit.client.1.smithi043.stdout:2/135: dwrite f6 [0,4194304] 0 2024-09-16T23:48:47.209 INFO:tasks.workunit.client.1.smithi043.stdout:6/253: truncate de/f40 244820 0 2024-09-16T23:48:47.213 INFO:tasks.workunit.client.1.smithi043.stdout:6/254: rename de/d1a/d39/c3c to de/d23/d34/c58 0 2024-09-16T23:48:47.215 INFO:tasks.workunit.client.1.smithi043.stdout:2/136: creat d8/d1f/f2e x:0 0 0 2024-09-16T23:48:47.216 INFO:tasks.workunit.client.1.smithi043.stdout:6/255: fsync de/d1a/d39/f16 0 2024-09-16T23:48:47.220 INFO:tasks.workunit.client.1.smithi043.stdout:6/256: symlink de/l59 0 2024-09-16T23:48:47.221 INFO:tasks.workunit.client.1.smithi043.stdout:6/257: fdatasync de/f40 0 2024-09-16T23:48:47.222 INFO:tasks.workunit.client.1.smithi043.stdout:6/258: chown lb 1745 1 2024-09-16T23:48:47.223 INFO:tasks.workunit.client.1.smithi043.stdout:2/137: dwrite d8/ff [0,4194304] 0 2024-09-16T23:48:47.224 INFO:tasks.workunit.client.1.smithi043.stdout:6/259: chown de/d23/d49/f4c 169514033 1 2024-09-16T23:48:47.224 INFO:tasks.workunit.client.1.smithi043.stdout:6/260: fsync de/d1a/f3b 0 2024-09-16T23:48:47.225 INFO:tasks.workunit.client.1.smithi043.stdout:6/261: chown de/d2a 0 1 2024-09-16T23:48:47.225 INFO:tasks.workunit.client.1.smithi043.stdout:6/262: chown de/d1a/f20 1 1 2024-09-16T23:48:47.227 INFO:tasks.workunit.client.1.smithi043.stdout:2/138: rmdir d8/d11/d16 39 2024-09-16T23:48:47.229 INFO:tasks.workunit.client.1.smithi043.stdout:6/263: rename de/d1a/d4d/d51 to de/d1a/d4d/d5a 0 2024-09-16T23:48:47.231 INFO:tasks.workunit.client.1.smithi043.stdout:2/139: symlink d8/d1d/d20/l2f 0 2024-09-16T23:48:47.234 INFO:tasks.workunit.client.1.smithi043.stdout:4/223: sync 2024-09-16T23:48:47.234 INFO:tasks.workunit.client.1.smithi043.stdout:9/164: sync 2024-09-16T23:48:47.234 INFO:tasks.workunit.client.1.smithi043.stdout:0/205: sync 2024-09-16T23:48:47.234 INFO:tasks.workunit.client.1.smithi043.stdout:5/346: sync 2024-09-16T23:48:47.234 INFO:tasks.workunit.client.1.smithi043.stdout:7/213: sync 2024-09-16T23:48:47.235 INFO:tasks.workunit.client.1.smithi043.stdout:2/140: creat d8/d1f/f30 x:0 0 0 2024-09-16T23:48:47.236 INFO:tasks.workunit.client.0.smithi042.stdout:8/100: creat da/d19/d1d/f22 x:0 0 0 2024-09-16T23:48:47.240 INFO:tasks.workunit.client.1.smithi043.stdout:2/141: mknod d8/d1d/d20/d26/c31 0 2024-09-16T23:48:47.241 INFO:tasks.workunit.client.1.smithi043.stdout:5/347: dread d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:47.243 INFO:tasks.workunit.client.1.smithi043.stdout:0/206: rename d0/d2/df/d1a/d3b/c29 to d0/d2/df/d1a/d3b/d27/c3e 0 2024-09-16T23:48:47.249 INFO:tasks.workunit.client.1.smithi043.stdout:5/348: mknod d3/db/d15/c71 0 2024-09-16T23:48:47.252 INFO:tasks.workunit.client.1.smithi043.stdout:5/349: readlink d3/db/l1a 0 2024-09-16T23:48:47.255 INFO:tasks.workunit.client.1.smithi043.stdout:2/142: rename d8/f10 to d8/d11/f32 0 2024-09-16T23:48:47.257 INFO:tasks.workunit.client.1.smithi043.stdout:9/165: dwrite d4/ff [0,4194304] 0 2024-09-16T23:48:47.259 INFO:tasks.workunit.client.1.smithi043.stdout:7/214: dwrite d4/db/d17/d1b/f37 [0,4194304] 0 2024-09-16T23:48:47.261 INFO:tasks.workunit.client.1.smithi043.stdout:0/207: chown d0/d2/df/d1a/d3b/f2a 13493540 1 2024-09-16T23:48:47.262 INFO:tasks.workunit.client.1.smithi043.stdout:6/264: dwrite de/d1a/d39/f16 [0,4194304] 0 2024-09-16T23:48:47.263 INFO:tasks.workunit.client.1.smithi043.stdout:4/224: dwrite d8/d15/f3c [0,4194304] 0 2024-09-16T23:48:47.264 INFO:tasks.workunit.client.1.smithi043.stdout:7/215: fdatasync d4/f2b 0 2024-09-16T23:48:47.266 INFO:tasks.workunit.client.1.smithi043.stdout:4/225: read d8/d25/f2c [1071790,116372] 0 2024-09-16T23:48:47.268 INFO:tasks.workunit.client.1.smithi043.stdout:0/208: write d0/f24 [3921646,60742] 0 2024-09-16T23:48:47.269 INFO:tasks.workunit.client.1.smithi043.stdout:4/226: dread - d8/d15/d19/f2a zero size 2024-09-16T23:48:47.331 INFO:tasks.workunit.client.0.smithi042.stdout:8/101: rmdir da/d19 39 2024-09-16T23:48:47.332 INFO:tasks.workunit.client.0.smithi042.stdout:8/102: truncate da/f18 2889050 0 2024-09-16T23:48:47.335 INFO:tasks.workunit.client.0.smithi042.stdout:8/103: creat da/d19/f23 x:0 0 0 2024-09-16T23:48:47.336 INFO:tasks.workunit.client.1.smithi043.stdout:5/350: creat d3/d9/d22/d4a/d4c/f72 x:0 0 0 2024-09-16T23:48:47.336 INFO:tasks.workunit.client.1.smithi043.stdout:4/227: rmdir d8/d15 39 2024-09-16T23:48:47.338 INFO:tasks.workunit.client.1.smithi043.stdout:0/209: write d0/d2/df/d1a/d3b/d27/f32 [763993,43008] 0 2024-09-16T23:48:47.339 INFO:tasks.workunit.client.1.smithi043.stdout:5/351: readlink d3/d9/l49 0 2024-09-16T23:48:47.339 INFO:tasks.workunit.client.1.smithi043.stdout:5/352: chown d3/db/d26 267686579 1 2024-09-16T23:48:47.341 INFO:tasks.workunit.client.1.smithi043.stdout:5/353: readlink d3/db/d26/l57 0 2024-09-16T23:48:47.341 INFO:tasks.workunit.client.1.smithi043.stdout:2/143: rename d8/d11/d16/c22 to d8/c33 0 2024-09-16T23:48:47.342 INFO:tasks.workunit.client.1.smithi043.stdout:7/216: creat d4/db/d17/f40 x:0 0 0 2024-09-16T23:48:47.343 INFO:tasks.workunit.client.1.smithi043.stdout:0/210: truncate d0/d2/df/d1a/d3b/f3d 35117 0 2024-09-16T23:48:47.344 INFO:tasks.workunit.client.1.smithi043.stdout:4/228: dread f7 [0,4194304] 0 2024-09-16T23:48:47.344 INFO:tasks.workunit.client.0.smithi042.stdout:8/104: truncate da/d19/f1e 1547486 0 2024-09-16T23:48:47.345 INFO:tasks.workunit.client.1.smithi043.stdout:6/265: truncate de/d1a/d39/f1f 2382140 0 2024-09-16T23:48:47.345 INFO:tasks.workunit.client.1.smithi043.stdout:0/211: write d0/de/f2d [763638,70696] 0 2024-09-16T23:48:47.345 INFO:tasks.workunit.client.1.smithi043.stdout:6/266: readlink de/d23/d49/l52 0 2024-09-16T23:48:47.346 INFO:tasks.workunit.client.1.smithi043.stdout:2/144: rmdir d8/d1f 39 2024-09-16T23:48:47.346 INFO:tasks.workunit.client.0.smithi042.stdout:8/105: unlink c5 0 2024-09-16T23:48:47.347 INFO:tasks.workunit.client.0.smithi042.stdout:8/106: chown da/d19/f23 61721754 1 2024-09-16T23:48:47.347 INFO:tasks.workunit.client.1.smithi043.stdout:5/354: fsync d3/db/d26/f33 0 2024-09-16T23:48:47.351 INFO:tasks.workunit.client.1.smithi043.stdout:0/212: mknod d0/d2/df/d1a/d3b/c3f 0 2024-09-16T23:48:47.353 INFO:tasks.workunit.client.0.smithi042.stdout:8/107: dwrite da/f14 [0,4194304] 0 2024-09-16T23:48:47.354 INFO:tasks.workunit.client.0.smithi042.stdout:8/108: dread - da/d19/d1d/f22 zero size 2024-09-16T23:48:47.356 INFO:tasks.workunit.client.1.smithi043.stdout:6/267: creat de/d1a/d28/f5b x:0 0 0 2024-09-16T23:48:47.357 INFO:tasks.workunit.client.0.smithi042.stdout:8/109: unlink da/c11 0 2024-09-16T23:48:47.359 INFO:tasks.workunit.client.1.smithi043.stdout:4/229: symlink d8/d15/d20/l45 0 2024-09-16T23:48:47.361 INFO:tasks.workunit.client.0.smithi042.stdout:8/110: creat da/f24 x:0 0 0 2024-09-16T23:48:47.362 INFO:tasks.workunit.client.1.smithi043.stdout:0/213: dread d0/d2/df/d1a/f30 [0,4194304] 0 2024-09-16T23:48:47.362 INFO:tasks.workunit.client.1.smithi043.stdout:2/145: mknod d8/d1f/c34 0 2024-09-16T23:48:47.363 INFO:tasks.workunit.client.0.smithi042.stdout:8/111: rmdir da/d19/d1d 39 2024-09-16T23:48:47.363 INFO:tasks.workunit.client.1.smithi043.stdout:0/214: write d0/d2/df/d1a/f2f [4107046,53297] 0 2024-09-16T23:48:47.364 INFO:tasks.workunit.client.1.smithi043.stdout:0/215: write d0/f24 [4770582,24540] 0 2024-09-16T23:48:47.367 INFO:tasks.workunit.client.1.smithi043.stdout:6/268: rename lb to de/d1a/d28/l5c 0 2024-09-16T23:48:47.367 INFO:tasks.workunit.client.0.smithi042.stdout:8/112: dread da/f14 [0,4194304] 0 2024-09-16T23:48:47.370 INFO:tasks.workunit.client.1.smithi043.stdout:4/230: symlink d8/d15/d20/d2b/l46 0 2024-09-16T23:48:47.371 INFO:tasks.workunit.client.0.smithi042.stdout:8/113: write da/d19/d1d/f22 [801562,107514] 0 2024-09-16T23:48:47.372 INFO:tasks.workunit.client.0.smithi042.stdout:8/114: write da/f24 [766423,60389] 0 2024-09-16T23:48:47.372 INFO:tasks.workunit.client.1.smithi043.stdout:0/216: mknod d0/de/c40 0 2024-09-16T23:48:47.373 INFO:tasks.workunit.client.1.smithi043.stdout:6/269: rmdir de/d1a/d4d/d50 39 2024-09-16T23:48:47.374 INFO:tasks.workunit.client.1.smithi043.stdout:5/355: rmdir d3/d1b/d5a 0 2024-09-16T23:48:47.376 INFO:tasks.workunit.client.1.smithi043.stdout:2/146: mkdir d8/d35 0 2024-09-16T23:48:47.377 INFO:tasks.workunit.client.0.smithi042.stdout:8/115: rename da/c13 to da/d19/c25 0 2024-09-16T23:48:47.377 INFO:tasks.workunit.client.1.smithi043.stdout:2/147: fdatasync d8/d1d/d25/f2b 0 2024-09-16T23:48:47.379 INFO:tasks.workunit.client.1.smithi043.stdout:2/148: truncate d8/d1d/d20/d26/f27 1112769 0 2024-09-16T23:48:47.379 INFO:tasks.workunit.client.1.smithi043.stdout:0/217: fsync d0/d2/df/f16 0 2024-09-16T23:48:47.380 INFO:tasks.workunit.client.1.smithi043.stdout:4/231: getdents d8 0 2024-09-16T23:48:47.382 INFO:tasks.workunit.client.1.smithi043.stdout:0/218: write d0/f24 [4296770,91388] 0 2024-09-16T23:48:47.382 INFO:tasks.workunit.client.0.smithi042.stdout:8/116: rename da/c10 to da/c26 0 2024-09-16T23:48:47.382 INFO:tasks.workunit.client.0.smithi042.stdout:8/117: readlink da/lb 0 2024-09-16T23:48:47.385 INFO:tasks.workunit.client.1.smithi043.stdout:2/149: dread d8/d1d/d20/d26/f27 [0,4194304] 0 2024-09-16T23:48:47.385 INFO:tasks.workunit.client.1.smithi043.stdout:2/150: read f6 [1736459,44976] 0 2024-09-16T23:48:47.389 INFO:tasks.workunit.client.0.smithi042.stdout:8/118: dwrite da/f24 [0,4194304] 0 2024-09-16T23:48:47.390 INFO:tasks.workunit.client.1.smithi043.stdout:5/356: dread d3/db/d19/d3e/f2d [0,4194304] 0 2024-09-16T23:48:47.391 INFO:tasks.workunit.client.1.smithi043.stdout:5/357: chown d3/db/d19/d3e/d28/f42 33 1 2024-09-16T23:48:47.392 INFO:tasks.workunit.client.1.smithi043.stdout:5/358: chown d3/db/d15/d4d/d6b 1 1 2024-09-16T23:48:47.394 INFO:tasks.workunit.client.1.smithi043.stdout:0/219: link d0/d2/df/d1a/d3b/d27/f32 d0/f41 0 2024-09-16T23:48:47.397 INFO:tasks.workunit.client.1.smithi043.stdout:5/359: unlink d3/d9/d22/l2f 0 2024-09-16T23:48:47.399 INFO:tasks.workunit.client.1.smithi043.stdout:0/220: rename d0/f13 to d0/dc/f42 0 2024-09-16T23:48:47.401 INFO:tasks.workunit.client.1.smithi043.stdout:2/151: dread f2 [0,4194304] 0 2024-09-16T23:48:47.402 INFO:tasks.workunit.client.1.smithi043.stdout:5/360: truncate d3/db/d26/f33 638692 0 2024-09-16T23:48:47.402 INFO:tasks.workunit.client.0.smithi042.stdout:8/119: write da/d19/f1e [521484,21640] 0 2024-09-16T23:48:47.402 INFO:tasks.workunit.client.1.smithi043.stdout:4/232: truncate d8/d25/f42 934271 0 2024-09-16T23:48:47.403 INFO:tasks.workunit.client.0.smithi042.stdout:8/120: chown da/d19/l21 164 1 2024-09-16T23:48:47.405 INFO:tasks.workunit.client.1.smithi043.stdout:0/221: mknod d0/d2/df/d1a/d34/d39/c43 0 2024-09-16T23:48:47.406 INFO:tasks.workunit.client.1.smithi043.stdout:8/218: sync 2024-09-16T23:48:47.406 INFO:tasks.workunit.client.1.smithi043.stdout:1/308: sync 2024-09-16T23:48:47.407 INFO:tasks.workunit.client.1.smithi043.stdout:8/219: fdatasync de/ff 0 2024-09-16T23:48:47.408 INFO:tasks.workunit.client.1.smithi043.stdout:5/361: rename d3/d9/d22/d4a/d6a to d3/db/d15/d4d/d6b/d73 0 2024-09-16T23:48:47.412 INFO:tasks.workunit.client.1.smithi043.stdout:8/220: chown de/d18/d1f/d25/f3b 245356951 1 2024-09-16T23:48:47.412 INFO:tasks.workunit.client.1.smithi043.stdout:1/309: truncate d4/d9/d1c/d1e/f55 329628 0 2024-09-16T23:48:47.412 INFO:tasks.workunit.client.1.smithi043.stdout:2/152: creat d8/d35/f36 x:0 0 0 2024-09-16T23:48:47.412 INFO:tasks.workunit.client.1.smithi043.stdout:0/222: symlink d0/d2/df/d1a/d3b/l44 0 2024-09-16T23:48:47.413 INFO:tasks.workunit.client.1.smithi043.stdout:8/221: write de/d10/f24 [1881820,24560] 0 2024-09-16T23:48:47.413 INFO:tasks.workunit.client.1.smithi043.stdout:4/233: dwrite d8/d25/f2c [4194304,4194304] 0 2024-09-16T23:48:47.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.480+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 --> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55000611d0 con 0x7f54e806d440 2024-09-16T23:48:47.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.483+0000 7f54fcff9700 1 -- 172.21.15.42:0/4040411427 <== mgr.14213 v2:172.21.15.42:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f55000611d0 con 0x7f54e806d440 2024-09-16T23:48:47.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f54e806d440 msgr2=0x7f54e806f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f54e806d440 0x7f54e806f900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54f0006010 tx=0x7f54f0005a90 comp rx=0 tx=0).stop 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001040d0 msgr2=0x7f55000ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001040d0 0x7f55000ff750 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f54f400d8d0 tx=0x7f54f400dbe0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 shutdown_connections 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7f54e806d440 0x7f54e806f900 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55001017a0 0x7f5500101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 --2- 172.21.15.42:0/4040411427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55001040d0 0x7f55000ff750 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 >> 172.21.15.42:0/4040411427 conn(0x7f55000fb190 msgr2=0x7f55000fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 shutdown_connections 2024-09-16T23:48:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.485+0000 7f5505d09700 1 -- 172.21.15.42:0/4040411427 wait complete. 2024-09-16T23:48:47.515 INFO:tasks.workunit.client.1.smithi043.stdout:5/362: rename d3/db/l1a to d3/d9/d6f/l74 0 2024-09-16T23:48:47.517 INFO:tasks.workunit.client.1.smithi043.stdout:8/222: mknod de/d18/d1f/d25/c4b 0 2024-09-16T23:48:47.519 INFO:tasks.workunit.client.1.smithi043.stdout:2/153: dread d8/f1a [0,4194304] 0 2024-09-16T23:48:47.519 INFO:tasks.workunit.client.1.smithi043.stdout:1/310: rename d4/d9/l2e to d4/d9/d12/d2b/d3d/l6a 0 2024-09-16T23:48:47.520 INFO:tasks.workunit.client.1.smithi043.stdout:4/234: symlink d8/d25/d39/l47 0 2024-09-16T23:48:47.520 INFO:tasks.workunit.client.1.smithi043.stdout:4/235: chown d8/d15/d19/f2a 2221711 1 2024-09-16T23:48:47.522 INFO:tasks.workunit.client.1.smithi043.stdout:0/223: rename d0/d2/df/d1a/d3b/f3d to d0/d2/df/d1a/d34/d39/f45 0 2024-09-16T23:48:47.523 INFO:tasks.workunit.client.1.smithi043.stdout:2/154: creat d8/d35/f37 x:0 0 0 2024-09-16T23:48:47.524 INFO:tasks.workunit.client.1.smithi043.stdout:2/155: chown d8/d35 317381 1 2024-09-16T23:48:47.525 INFO:tasks.workunit.client.1.smithi043.stdout:8/223: creat de/d18/f4c x:0 0 0 2024-09-16T23:48:47.528 INFO:tasks.workunit.client.1.smithi043.stdout:8/224: stat de/d18/d1f/d2f/f3e 0 2024-09-16T23:48:47.530 INFO:tasks.workunit.client.1.smithi043.stdout:2/156: mkdir d8/d1d/d20/d23/d38 0 2024-09-16T23:48:47.530 INFO:tasks.workunit.client.1.smithi043.stdout:4/236: dwrite d8/d25/d39/f3a [0,4194304] 0 2024-09-16T23:48:47.531 INFO:tasks.workunit.client.1.smithi043.stdout:1/311: creat d4/d9/d12/d2b/f6b x:0 0 0 2024-09-16T23:48:47.533 INFO:tasks.workunit.client.1.smithi043.stdout:0/224: rename d0/d2/df/d1a/d34 to d0/d2/df/d1a/d46 0 2024-09-16T23:48:47.533 INFO:tasks.workunit.client.1.smithi043.stdout:2/157: dread - d8/d11/d16/f19 zero size 2024-09-16T23:48:47.536 INFO:tasks.workunit.client.1.smithi043.stdout:1/312: mknod d4/d3e/c6c 0 2024-09-16T23:48:47.539 INFO:tasks.workunit.client.1.smithi043.stdout:5/363: dwrite d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:47.541 INFO:tasks.workunit.client.1.smithi043.stdout:1/313: dread - d4/d9/f46 zero size 2024-09-16T23:48:47.545 INFO:tasks.workunit.client.1.smithi043.stdout:2/158: mkdir d8/d1d/d20/d23/d39 0 2024-09-16T23:48:47.547 INFO:tasks.workunit.client.1.smithi043.stdout:8/225: dwrite de/d10/f24 [0,4194304] 0 2024-09-16T23:48:47.548 INFO:tasks.workunit.client.1.smithi043.stdout:2/159: chown d8/c9 62 1 2024-09-16T23:48:47.549 INFO:tasks.workunit.client.1.smithi043.stdout:1/314: unlink d4/d9/d12/d2b/f36 0 2024-09-16T23:48:47.549 INFO:tasks.workunit.client.1.smithi043.stdout:5/364: write d3/db/f2a [4392414,12039] 0 2024-09-16T23:48:47.552 INFO:tasks.workunit.client.1.smithi043.stdout:2/160: rmdir d8/d1f 39 2024-09-16T23:48:47.553 INFO:tasks.workunit.client.1.smithi043.stdout:0/225: dwrite d0/de/f15 [4194304,4194304] 0 2024-09-16T23:48:47.558 INFO:tasks.workunit.client.1.smithi043.stdout:3/185: sync 2024-09-16T23:48:47.559 INFO:tasks.workunit.client.1.smithi043.stdout:0/226: chown d0/d2/df/d1a/f2f 28276 1 2024-09-16T23:48:47.561 INFO:tasks.workunit.client.1.smithi043.stdout:2/161: mkdir d8/d1d/d20/d23/d39/d3a 0 2024-09-16T23:48:47.561 INFO:tasks.workunit.client.1.smithi043.stdout:1/315: truncate d4/d9/d12/f1f 4343862 0 2024-09-16T23:48:47.561 INFO:tasks.workunit.client.1.smithi043.stdout:0/227: readlink d0/de/l25 0 2024-09-16T23:48:47.564 INFO:tasks.workunit.client.1.smithi043.stdout:4/237: dread d8/d15/d20/d2b/f26 [0,4194304] 0 2024-09-16T23:48:47.564 INFO:tasks.workunit.client.1.smithi043.stdout:0/228: stat d0/d2/df/l19 0 2024-09-16T23:48:47.567 INFO:tasks.workunit.client.1.smithi043.stdout:1/316: read d4/df/f38 [2179991,17127] 0 2024-09-16T23:48:47.568 INFO:tasks.workunit.client.1.smithi043.stdout:3/186: write d4/f1b [2009470,96712] 0 2024-09-16T23:48:47.568 INFO:tasks.workunit.client.1.smithi043.stdout:5/365: dwrite d3/db/d15/f5f [0,4194304] 0 2024-09-16T23:48:47.569 INFO:tasks.workunit.client.1.smithi043.stdout:1/317: write d4/d34/f4d [760181,108292] 0 2024-09-16T23:48:47.570 INFO:tasks.workunit.client.1.smithi043.stdout:2/162: symlink d8/d1d/d20/d23/d39/d3a/l3b 0 2024-09-16T23:48:47.571 INFO:tasks.workunit.client.1.smithi043.stdout:8/226: fsync de/d10/f24 0 2024-09-16T23:48:47.573 INFO:tasks.workunit.client.1.smithi043.stdout:4/238: symlink d8/d3d/l48 0 2024-09-16T23:48:47.590 INFO:tasks.workunit.client.1.smithi043.stdout:5/366: creat d3/db/d15/f75 x:0 0 0 2024-09-16T23:48:47.590 INFO:tasks.workunit.client.1.smithi043.stdout:2/163: creat d8/d1d/d20/d26/f3c x:0 0 0 2024-09-16T23:48:47.591 INFO:tasks.workunit.client.1.smithi043.stdout:0/229: creat d0/d2/df/d1a/f47 x:0 0 0 2024-09-16T23:48:47.592 INFO:tasks.workunit.client.1.smithi043.stdout:1/318: dwrite d4/df/d13/d1b/f25 [4194304,4194304] 0 2024-09-16T23:48:47.593 INFO:tasks.workunit.client.1.smithi043.stdout:4/239: write d8/d15/f21 [5345236,13772] 0 2024-09-16T23:48:47.593 INFO:tasks.workunit.client.1.smithi043.stdout:8/227: mkdir de/d18/d1f/d2f/d39/d4d 0 2024-09-16T23:48:47.595 INFO:tasks.workunit.client.1.smithi043.stdout:1/319: fsync d4/d9/d12/f1f 0 2024-09-16T23:48:47.595 INFO:tasks.workunit.client.1.smithi043.stdout:0/230: fdatasync d0/d2/df/d1a/d3b/f2a 0 2024-09-16T23:48:47.597 INFO:tasks.workunit.client.1.smithi043.stdout:0/231: read d0/f24 [139641,42602] 0 2024-09-16T23:48:47.602 INFO:tasks.workunit.client.1.smithi043.stdout:8/228: symlink de/d18/l4e 0 2024-09-16T23:48:47.604 INFO:tasks.workunit.client.1.smithi043.stdout:2/164: link d8/ff d8/d1d/d20/d23/d39/f3d 0 2024-09-16T23:48:47.606 INFO:tasks.workunit.client.1.smithi043.stdout:1/320: write d4/d9/d1c/d1e/f2a [4598806,39419] 0 2024-09-16T23:48:47.608 INFO:tasks.workunit.client.1.smithi043.stdout:5/367: dread d3/db/d19/d3e/d28/f42 [4194304,4194304] 0 2024-09-16T23:48:47.608 INFO:tasks.workunit.client.1.smithi043.stdout:8/229: read de/d18/d2b/f2e [106280,123841] 0 2024-09-16T23:48:47.609 INFO:tasks.workunit.client.1.smithi043.stdout:8/230: write de/d18/f4c [223895,23223] 0 2024-09-16T23:48:47.611 INFO:tasks.workunit.client.1.smithi043.stdout:8/231: write de/d18/f4c [917826,71184] 0 2024-09-16T23:48:47.611 INFO:tasks.workunit.client.1.smithi043.stdout:1/321: mknod d4/d34/c6d 0 2024-09-16T23:48:47.611 INFO:tasks.workunit.client.1.smithi043.stdout:5/368: read d3/d9/f56 [1157659,61421] 0 2024-09-16T23:48:47.616 INFO:tasks.workunit.client.1.smithi043.stdout:4/240: getdents d8/d15/d20/d2b 0 2024-09-16T23:48:47.616 INFO:tasks.workunit.client.1.smithi043.stdout:1/322: truncate d4/d9/d12/d2b/f6b 889430 0 2024-09-16T23:48:47.618 INFO:tasks.workunit.client.1.smithi043.stdout:5/369: write d3/db/d15/f3c [1713732,20163] 0 2024-09-16T23:48:47.619 INFO:tasks.workunit.client.1.smithi043.stdout:4/241: rmdir d8/d3d 39 2024-09-16T23:48:47.620 INFO:tasks.workunit.client.1.smithi043.stdout:2/165: truncate d8/d1d/d20/d23/d39/f3d 3134938 0 2024-09-16T23:48:47.621 INFO:tasks.workunit.client.1.smithi043.stdout:8/232: dwrite de/f45 [0,4194304] 0 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.623+0000 7ff6f19c1700 1 -- 172.21.15.42:0/522785603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 msgr2=0x7ff6ec100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.623+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/522785603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec100030 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009910 tx=0x7ff6dc009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.623+0000 7ff6f19c1700 1 -- 172.21.15.42:0/522785603 shutdown_connections 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.624+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/522785603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec100e10 0x7ff6ec101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.624+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/522785603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec100030 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.624+0000 7ff6f19c1700 1 -- 172.21.15.42:0/522785603 >> 172.21.15.42:0/522785603 conn(0x7ff6ec0fb190 msgr2=0x7ff6ec0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.624+0000 7ff6f19c1700 1 -- 172.21.15.42:0/522785603 shutdown_connections 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.624+0000 7ff6f19c1700 1 -- 172.21.15.42:0/522785603 wait complete. 2024-09-16T23:48:47.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.624+0000 7ff6f19c1700 1 Processor -- start 2024-09-16T23:48:47.623 INFO:tasks.workunit.client.1.smithi043.stdout:1/323: creat d4/d34/f6e x:0 0 0 2024-09-16T23:48:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6f19c1700 1 -- start start 2024-09-16T23:48:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6f19c1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec069140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6f19c1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec100e10 0x7ff6ec19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6f19c1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec19b790 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6f19c1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec19b8d0 con 0x7ff6ec100e10 2024-09-16T23:48:47.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6eaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec069140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6eaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec069140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59694/0 (socket says 172.21.15.42:59694) 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.625+0000 7ff6eaffd700 1 -- 172.21.15.42:0/891775267 learned_addr learned my addr 172.21.15.42:0/891775267 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6ea7fc700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec100e10 0x7ff6ec19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6eaffd700 1 -- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec100e10 msgr2=0x7ff6ec19b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6eaffd700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec100e10 0x7ff6ec19b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6eaffd700 1 -- 172.21.15.42:0/891775267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc0095f0 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6ea7fc700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec100e10 0x7ff6ec19b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6eaffd700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec069140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc005100 tx=0x7ff6dc0051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6dc01d070 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.626+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec19bb50 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.627+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec19c040 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.627+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc005640 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.627+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6dc004a00 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.628+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d8005320 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.629+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 20) v1 ==== 91358+0+0 (secure 0 0 0) 0x7ff6dc003d30 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.627 INFO:tasks.workunit.client.1.smithi043.stdout:8/233: truncate de/d18/d1f/d2f/f3e 335268 0 2024-09-16T23:48:47.628 INFO:tasks.workunit.client.1.smithi043.stdout:0/232: truncate d0/de/f15 5427193 0 2024-09-16T23:48:47.628 INFO:tasks.workunit.client.1.smithi043.stdout:0/233: read d0/de/f2d [680809,240] 0 2024-09-16T23:48:47.629 INFO:tasks.workunit.client.1.smithi043.stdout:1/324: dread d4/d9/d12/d15/f31 [0,4194304] 0 2024-09-16T23:48:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.630+0000 7ff6f09bf700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff6d406d3f0 0x7ff6d406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.630+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff6dc08e1e0 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.630+0000 7ff6ea7fc700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff6d406d3f0 0x7ff6d406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:47.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.631+0000 7ff6ea7fc700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff6d406d3f0 0x7ff6d406f8b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009cb0 tx=0x7ff6e0009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:47.630 INFO:tasks.workunit.client.1.smithi043.stdout:1/325: write d4/fc [1508605,30800] 0 2024-09-16T23:48:47.631 INFO:tasks.workunit.client.1.smithi043.stdout:1/326: dread - d4/d9/d1c/d1e/f66 zero size 2024-09-16T23:48:47.633 INFO:tasks.workunit.client.1.smithi043.stdout:5/370: dread d3/f38 [4194304,4194304] 0 2024-09-16T23:48:47.633 INFO:tasks.workunit.client.1.smithi043.stdout:2/166: symlink d8/d1f/l3e 0 2024-09-16T23:48:47.634 INFO:tasks.workunit.client.1.smithi043.stdout:0/234: write d0/d2/f33 [4606347,15739] 0 2024-09-16T23:48:47.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.637+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff6dc05c520 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.644+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 21) v1 ==== 46123+0+0 (secure 0 0 0) 0x7ff6dc06fdc0 con 0x7ff6ec0ffc10 2024-09-16T23:48:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.644+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff6d406d3f0 msgr2=0x7ff6d406f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:47.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:47.644+0000 7ff6f09bf700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff6d406d3f0 0x7ff6d406f8b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009cb0 tx=0x7ff6e0009260 comp rx=0 tx=0).stop 2024-09-16T23:48:47.645 INFO:tasks.workunit.client.1.smithi043.stdout:1/327: symlink d4/df/d26/l6f 0 2024-09-16T23:48:47.648 INFO:tasks.workunit.client.1.smithi043.stdout:5/371: mkdir d3/db/d19/d76 0 2024-09-16T23:48:47.650 INFO:tasks.workunit.client.1.smithi043.stdout:9/166: sync 2024-09-16T23:48:47.650 INFO:tasks.workunit.client.1.smithi043.stdout:2/167: rename f2 to d8/d1d/d20/d26/f3f 0 2024-09-16T23:48:47.654 INFO:tasks.workunit.client.1.smithi043.stdout:9/167: read d4/d5/d16/f1f [92394,20807] 0 2024-09-16T23:48:47.655 INFO:tasks.workunit.client.1.smithi043.stdout:9/168: stat d4/d1c 0 2024-09-16T23:48:47.656 INFO:tasks.workunit.client.1.smithi043.stdout:2/168: rmdir d8/d1d/d20/d23/d39/d3a 39 2024-09-16T23:48:47.657 INFO:tasks.workunit.client.1.smithi043.stdout:8/234: link de/d18/d27/c47 de/c4f 0 2024-09-16T23:48:47.659 INFO:tasks.workunit.client.1.smithi043.stdout:4/242: dwrite d8/d15/d19/f2a [0,4194304] 0 2024-09-16T23:48:47.660 INFO:tasks.workunit.client.1.smithi043.stdout:5/372: unlink d3/d9/d22/d4a/l5c 0 2024-09-16T23:48:47.667 INFO:tasks.workunit.client.1.smithi043.stdout:8/235: rmdir de/d18/d1f/d2f 39 2024-09-16T23:48:47.667 INFO:tasks.workunit.client.1.smithi043.stdout:0/235: dread d0/f24 [0,4194304] 0 2024-09-16T23:48:47.667 INFO:tasks.workunit.client.1.smithi043.stdout:9/169: dwrite d4/d1c/d25/f26 [0,4194304] 0 2024-09-16T23:48:47.669 INFO:tasks.workunit.client.1.smithi043.stdout:8/236: fdatasync de/d18/f21 0 2024-09-16T23:48:47.669 INFO:tasks.workunit.client.1.smithi043.stdout:4/243: dread d8/d15/d19/f2a [0,4194304] 0 2024-09-16T23:48:47.671 INFO:tasks.workunit.client.1.smithi043.stdout:1/328: getdents d4/df/d13/d1b 0 2024-09-16T23:48:47.672 INFO:tasks.workunit.client.1.smithi043.stdout:4/244: write d8/d15/f21 [4976735,32315] 0 2024-09-16T23:48:47.679 INFO:tasks.workunit.client.1.smithi043.stdout:5/373: unlink d3/c7 0 2024-09-16T23:48:47.682 INFO:tasks.workunit.client.1.smithi043.stdout:4/245: dwrite d8/d15/d20/d2b/d23/f38 [0,4194304] 0 2024-09-16T23:48:47.682 INFO:tasks.workunit.client.1.smithi043.stdout:8/237: rmdir de/d18/d27 39 2024-09-16T23:48:47.685 INFO:tasks.workunit.client.1.smithi043.stdout:9/170: dread d4/f10 [0,4194304] 0 2024-09-16T23:48:47.686 INFO:tasks.workunit.client.1.smithi043.stdout:1/329: fdatasync d4/df/f38 0 2024-09-16T23:48:47.686 INFO:tasks.workunit.client.1.smithi043.stdout:0/236: symlink d0/l48 0 2024-09-16T23:48:47.688 INFO:tasks.workunit.client.1.smithi043.stdout:8/238: dread de/d18/f4c [0,4194304] 0 2024-09-16T23:48:47.689 INFO:tasks.workunit.client.1.smithi043.stdout:2/169: write d8/ff [2665462,72660] 0 2024-09-16T23:48:47.695 INFO:tasks.workunit.client.1.smithi043.stdout:1/330: dread d4/fc [0,4194304] 0 2024-09-16T23:48:47.695 INFO:tasks.workunit.client.1.smithi043.stdout:4/246: dread d8/f1d [0,4194304] 0 2024-09-16T23:48:47.696 INFO:tasks.workunit.client.1.smithi043.stdout:9/171: readlink d4/d5/d16/l1e 0 2024-09-16T23:48:47.705 INFO:tasks.workunit.client.1.smithi043.stdout:2/170: fsync f6 0 2024-09-16T23:48:47.705 INFO:tasks.workunit.client.1.smithi043.stdout:2/171: chown d8/d1d/d20 1135319492 1 2024-09-16T23:48:47.706 INFO:tasks.workunit.client.1.smithi043.stdout:2/172: write d8/d1f/f30 [565520,120948] 0 2024-09-16T23:48:47.709 INFO:tasks.workunit.client.1.smithi043.stdout:4/247: mkdir d8/d15/d19/d49 0 2024-09-16T23:48:47.709 INFO:tasks.workunit.client.1.smithi043.stdout:9/172: dread d4/f11 [0,4194304] 0 2024-09-16T23:48:47.711 INFO:tasks.workunit.client.1.smithi043.stdout:9/173: truncate d4/f10 4595571 0 2024-09-16T23:48:47.712 INFO:tasks.workunit.client.1.smithi043.stdout:5/374: dwrite d3/db/d26/f33 [0,4194304] 0 2024-09-16T23:48:47.716 INFO:tasks.workunit.client.1.smithi043.stdout:2/173: creat d8/d1d/d20/f40 x:0 0 0 2024-09-16T23:48:47.718 INFO:tasks.workunit.client.1.smithi043.stdout:9/174: dwrite d4/d21/d2a/d2c/f2d [0,4194304] 0 2024-09-16T23:48:47.719 INFO:tasks.workunit.client.1.smithi043.stdout:9/175: chown d4/d28/d2e/c18 5 1 2024-09-16T23:48:47.721 INFO:tasks.workunit.client.1.smithi043.stdout:9/176: dread d4/f10 [4194304,4194304] 0 2024-09-16T23:48:47.722 INFO:tasks.workunit.client.1.smithi043.stdout:4/248: write d8/d15/d20/d2b/f26 [2990406,29724] 0 2024-09-16T23:48:47.723 INFO:tasks.workunit.client.1.smithi043.stdout:0/237: write d0/d2/df/d1a/d46/f35 [417322,104001] 0 2024-09-16T23:48:47.725 INFO:tasks.workunit.client.1.smithi043.stdout:0/238: readlink d0/d2/df/d1a/d3b/l1d 0 2024-09-16T23:48:47.730 INFO:tasks.workunit.client.1.smithi043.stdout:9/177: mkdir d4/d21/d33 0 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:7/25: sync 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:9/41: sync 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:3/7: sync 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:4/49: sync 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:0/0: sync 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:0/1: write - no filename 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:0/2: truncate - no filename 2024-09-16T23:48:47.732 INFO:tasks.workunit.client.0.smithi042.stdout:5/55: sync 2024-09-16T23:48:47.733 INFO:tasks.workunit.client.0.smithi042.stdout:1/98: sync 2024-09-16T23:48:47.733 INFO:tasks.workunit.client.0.smithi042.stdout:8/121: sync 2024-09-16T23:48:47.733 INFO:tasks.workunit.client.1.smithi043.stdout:8/239: rmdir de/d18/d27/d46 0 2024-09-16T23:48:47.734 INFO:tasks.workunit.client.1.smithi043.stdout:9/178: dread d4/d1c/d25/f26 [0,4194304] 0 2024-09-16T23:48:47.735 INFO:tasks.workunit.client.1.smithi043.stdout:0/239: fdatasync d0/d2/df/d1a/d3b/d27/f32 0 2024-09-16T23:48:47.735 INFO:tasks.workunit.client.0.smithi042.stdout:6/43: sync 2024-09-16T23:48:47.736 INFO:tasks.workunit.client.0.smithi042.stdout:2/51: sync 2024-09-16T23:48:47.736 INFO:tasks.workunit.client.0.smithi042.stdout:7/26: creat d3/f8 x:0 0 0 2024-09-16T23:48:47.736 INFO:tasks.workunit.client.0.smithi042.stdout:8/122: fdatasync da/f16 0 2024-09-16T23:48:47.738 INFO:tasks.workunit.client.0.smithi042.stdout:5/56: chown d9/db/de 2 1 2024-09-16T23:48:47.738 INFO:tasks.workunit.client.0.smithi042.stdout:1/99: fsync d0/f6 0 2024-09-16T23:48:47.739 INFO:tasks.workunit.client.1.smithi043.stdout:2/174: rename l4 to d8/d1d/d20/d23/d39/d3a/l41 0 2024-09-16T23:48:47.740 INFO:tasks.workunit.client.1.smithi043.stdout:4/249: creat d8/d15/d20/d35/f4a x:0 0 0 2024-09-16T23:48:47.741 INFO:tasks.workunit.client.1.smithi043.stdout:2/175: stat d8/d1d/d25/c2c 0 2024-09-16T23:48:47.741 INFO:tasks.workunit.client.1.smithi043.stdout:8/240: dread de/f45 [0,4194304] 0 2024-09-16T23:48:47.742 INFO:tasks.workunit.client.0.smithi042.stdout:6/44: symlink d8/db/lf 0 2024-09-16T23:48:47.742 INFO:tasks.workunit.client.0.smithi042.stdout:6/45: chown d8 29 1 2024-09-16T23:48:47.744 INFO:tasks.workunit.client.1.smithi043.stdout:9/179: rmdir d4/d21 39 2024-09-16T23:48:47.745 INFO:tasks.workunit.client.0.smithi042.stdout:3/8: link f0 f1 0 2024-09-16T23:48:47.745 INFO:tasks.workunit.client.0.smithi042.stdout:3/9: rmdir - no directory 2024-09-16T23:48:47.745 INFO:tasks.workunit.client.0.smithi042.stdout:8/123: mknod da/d19/d1d/c27 0 2024-09-16T23:48:47.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/3: mkdir d0 0 2024-09-16T23:48:47.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/4: truncate - no filename 2024-09-16T23:48:47.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/5: readlink - no filename 2024-09-16T23:48:47.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/6: write - no filename 2024-09-16T23:48:47.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/7: fsync - no filename 2024-09-16T23:48:47.747 INFO:tasks.workunit.client.0.smithi042.stdout:0/8: write - no filename 2024-09-16T23:48:47.748 INFO:tasks.workunit.client.0.smithi042.stdout:0/9: chown d0 2994044 1 2024-09-16T23:48:47.748 INFO:tasks.workunit.client.0.smithi042.stdout:0/10: fdatasync - no filename 2024-09-16T23:48:47.748 INFO:tasks.workunit.client.0.smithi042.stdout:0/11: dwrite - no filename 2024-09-16T23:48:47.748 INFO:tasks.workunit.client.0.smithi042.stdout:0/12: write - no filename 2024-09-16T23:48:47.749 INFO:tasks.workunit.client.0.smithi042.stdout:5/57: mkdir d9/db/de/d10 0 2024-09-16T23:48:47.751 INFO:tasks.workunit.client.0.smithi042.stdout:3/10: mkdir d2 0 2024-09-16T23:48:47.751 INFO:tasks.workunit.client.0.smithi042.stdout:1/100: mknod d0/d17/d18/c1f 0 2024-09-16T23:48:47.752 INFO:tasks.workunit.client.0.smithi042.stdout:9/42: dread d4/fa [0,4194304] 0 2024-09-16T23:48:47.754 INFO:tasks.workunit.client.0.smithi042.stdout:6/46: rename d8/d9/fe to d8/d9/f10 0 2024-09-16T23:48:47.754 INFO:tasks.workunit.client.0.smithi042.stdout:3/11: chown d2 816 1 2024-09-16T23:48:47.755 INFO:tasks.workunit.client.1.smithi043.stdout:2/176: symlink d8/d1d/d20/d23/l42 0 2024-09-16T23:48:47.755 INFO:tasks.workunit.client.1.smithi043.stdout:4/250: dwrite d8/f31 [0,4194304] 0 2024-09-16T23:48:47.755 INFO:tasks.workunit.client.0.smithi042.stdout:4/50: link de/cf de/d12/c13 0 2024-09-16T23:48:47.756 INFO:tasks.workunit.client.0.smithi042.stdout:6/47: read - d8/d9/fa zero size 2024-09-16T23:48:47.756 INFO:tasks.workunit.client.1.smithi043.stdout:8/241: truncate de/d18/f4c 1581399 0 2024-09-16T23:48:47.757 INFO:tasks.workunit.client.1.smithi043.stdout:4/251: read - d8/d15/d20/d35/f4a zero size 2024-09-16T23:48:47.759 INFO:tasks.workunit.client.0.smithi042.stdout:7/27: dwrite d3/f8 [0,4194304] 0 2024-09-16T23:48:47.759 INFO:tasks.workunit.client.0.smithi042.stdout:0/13: symlink d0/l1 0 2024-09-16T23:48:47.760 INFO:tasks.workunit.client.0.smithi042.stdout:0/14: dwrite - no filename 2024-09-16T23:48:47.760 INFO:tasks.workunit.client.0.smithi042.stdout:5/58: truncate d9/db/ff 650406 0 2024-09-16T23:48:47.763 INFO:tasks.workunit.client.1.smithi043.stdout:4/252: dread d8/d15/f3c [0,4194304] 0 2024-09-16T23:48:47.763 INFO:tasks.workunit.client.0.smithi042.stdout:2/52: dwrite d0/fc [0,4194304] 0 2024-09-16T23:48:47.765 INFO:tasks.workunit.client.1.smithi043.stdout:0/240: mkdir d0/d2/df/d1a/d49 0 2024-09-16T23:48:47.765 INFO:tasks.workunit.client.0.smithi042.stdout:6/48: dread - d8/d9/fa zero size 2024-09-16T23:48:47.767 INFO:tasks.workunit.client.1.smithi043.stdout:0/241: chown d0/d2/df/d1a/d46/d39/f45 14524 1 2024-09-16T23:48:47.767 INFO:tasks.workunit.client.0.smithi042.stdout:6/49: readlink d8/db/lf 0 2024-09-16T23:48:47.768 INFO:tasks.workunit.client.0.smithi042.stdout:1/101: creat d0/d17/f20 x:0 0 0 2024-09-16T23:48:47.769 INFO:tasks.workunit.client.0.smithi042.stdout:1/102: chown d0 0 1 2024-09-16T23:48:47.769 INFO:tasks.workunit.client.1.smithi043.stdout:8/242: write de/d18/d1f/d2f/f3e [912835,17144] 0 2024-09-16T23:48:47.770 INFO:tasks.workunit.client.1.smithi043.stdout:8/243: stat de/d18/d27 0 2024-09-16T23:48:47.770 INFO:tasks.workunit.client.0.smithi042.stdout:9/43: mkdir d4/d10 0 2024-09-16T23:48:47.771 INFO:tasks.workunit.client.0.smithi042.stdout:8/124: symlink da/d19/l28 0 2024-09-16T23:48:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:48:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:48:47.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr fail", "who": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "mgr fail", "who": "smithi042.zrkumz"}]': finished 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: mgrmap e21: smithi043.xveyou(active, starting, since 0.017473s) 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:48:47.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:48:47.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:47 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:48:47.775 INFO:tasks.workunit.client.0.smithi042.stdout:2/53: creat d0/f11 x:0 0 0 2024-09-16T23:48:47.777 INFO:tasks.workunit.client.0.smithi042.stdout:5/59: unlink d9/db/ld 0 2024-09-16T23:48:47.778 INFO:tasks.workunit.client.0.smithi042.stdout:4/51: dread f2 [0,4194304] 0 2024-09-16T23:48:47.780 INFO:tasks.workunit.client.0.smithi042.stdout:3/12: getdents d2 0 2024-09-16T23:48:47.781 INFO:tasks.workunit.client.1.smithi043.stdout:4/253: mkdir d8/d15/d20/d2b/d23/d27/d4b 0 2024-09-16T23:48:47.783 INFO:tasks.workunit.client.0.smithi042.stdout:5/60: dread d9/db/ff [0,4194304] 0 2024-09-16T23:48:47.784 INFO:tasks.workunit.client.1.smithi043.stdout:4/254: write d8/d15/f21 [1975546,58838] 0 2024-09-16T23:48:47.785 INFO:tasks.workunit.client.0.smithi042.stdout:1/103: creat d0/d17/f21 x:0 0 0 2024-09-16T23:48:47.788 INFO:tasks.workunit.client.0.smithi042.stdout:7/28: rename d3/l6 to d3/l9 0 2024-09-16T23:48:47.790 INFO:tasks.workunit.client.1.smithi043.stdout:0/242: dwrite d0/d2/df/d1a/d3b/d27/f32 [0,4194304] 0 2024-09-16T23:48:47.792 INFO:tasks.workunit.client.0.smithi042.stdout:5/61: mkdir d9/db/d11 0 2024-09-16T23:48:47.794 INFO:tasks.workunit.client.1.smithi043.stdout:9/180: creat d4/d21/d33/f34 x:0 0 0 2024-09-16T23:48:47.794 INFO:tasks.workunit.client.1.smithi043.stdout:0/243: chown d0 92608030 1 2024-09-16T23:48:47.796 INFO:tasks.workunit.client.0.smithi042.stdout:4/52: dwrite de/f10 [0,4194304] 0 2024-09-16T23:48:47.796 INFO:tasks.workunit.client.1.smithi043.stdout:2/177: creat d8/f43 x:0 0 0 2024-09-16T23:48:47.797 INFO:tasks.workunit.client.1.smithi043.stdout:9/181: write d4/d21/d33/f34 [515985,5550] 0 2024-09-16T23:48:47.797 INFO:tasks.workunit.client.1.smithi043.stdout:2/178: stat d8/d1d 0 2024-09-16T23:48:47.797 INFO:tasks.workunit.client.1.smithi043.stdout:4/255: dread d8/f31 [0,4194304] 0 2024-09-16T23:48:47.801 INFO:tasks.workunit.client.0.smithi042.stdout:6/50: link f3 d8/db/dd/f11 0 2024-09-16T23:48:47.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/15: link d0/l1 d0/l2 0 2024-09-16T23:48:47.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/16: write - no filename 2024-09-16T23:48:47.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/17: dwrite - no filename 2024-09-16T23:48:47.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/18: dwrite - no filename 2024-09-16T23:48:47.802 INFO:tasks.workunit.client.0.smithi042.stdout:7/29: creat d3/fa x:0 0 0 2024-09-16T23:48:47.802 INFO:tasks.workunit.client.1.smithi043.stdout:8/244: dread de/d10/f4a [0,4194304] 0 2024-09-16T23:48:47.803 INFO:tasks.workunit.client.0.smithi042.stdout:1/104: dread d0/d9/df/f12 [0,4194304] 0 2024-09-16T23:48:47.806 INFO:tasks.workunit.client.0.smithi042.stdout:5/62: dread d9/db/ff [0,4194304] 0 2024-09-16T23:48:47.807 INFO:tasks.workunit.client.0.smithi042.stdout:1/105: read - d0/d17/f20 zero size 2024-09-16T23:48:47.809 INFO:tasks.workunit.client.1.smithi043.stdout:8/245: dwrite de/d18/f21 [4194304,4194304] 0 2024-09-16T23:48:47.811 INFO:tasks.workunit.client.0.smithi042.stdout:6/51: dread d8/d9/f10 [0,4194304] 0 2024-09-16T23:48:47.811 INFO:tasks.workunit.client.1.smithi043.stdout:9/182: mkdir d4/d21/d35 0 2024-09-16T23:48:47.813 INFO:tasks.workunit.client.0.smithi042.stdout:2/54: rmdir d0 39 2024-09-16T23:48:47.814 INFO:tasks.workunit.client.0.smithi042.stdout:0/19: mknod d0/c3 0 2024-09-16T23:48:47.814 INFO:tasks.workunit.client.1.smithi043.stdout:9/183: chown d4/d21/d2a/d2c/f2d 58405417 1 2024-09-16T23:48:47.816 INFO:tasks.workunit.client.0.smithi042.stdout:7/30: chown d3/c5 1149 1 2024-09-16T23:48:47.817 INFO:tasks.workunit.client.0.smithi042.stdout:3/13: write f1 [1043662,85861] 0 2024-09-16T23:48:47.817 INFO:tasks.workunit.client.0.smithi042.stdout:6/52: creat d8/db/f12 x:0 0 0 2024-09-16T23:48:47.817 INFO:tasks.workunit.client.1.smithi043.stdout:0/244: mknod d0/d2/c4a 0 2024-09-16T23:48:47.819 INFO:tasks.workunit.client.1.smithi043.stdout:8/246: creat de/d18/d27/d41/f50 x:0 0 0 2024-09-16T23:48:47.821 INFO:tasks.workunit.client.0.smithi042.stdout:8/125: creat da/f29 x:0 0 0 2024-09-16T23:48:47.824 INFO:tasks.workunit.client.1.smithi043.stdout:8/247: fdatasync de/d10/f13 0 2024-09-16T23:48:47.824 INFO:tasks.workunit.client.0.smithi042.stdout:0/20: creat d0/f4 x:0 0 0 2024-09-16T23:48:47.824 INFO:tasks.workunit.client.1.smithi043.stdout:0/245: symlink d0/l4b 0 2024-09-16T23:48:47.825 INFO:tasks.workunit.client.1.smithi043.stdout:9/184: rmdir d4/d22 0 2024-09-16T23:48:47.826 INFO:tasks.workunit.client.1.smithi043.stdout:8/248: chown de/d18/d1f/d25/l49 10 1 2024-09-16T23:48:47.827 INFO:tasks.workunit.client.0.smithi042.stdout:1/106: mknod d0/d9/c22 0 2024-09-16T23:48:47.827 INFO:tasks.workunit.client.1.smithi043.stdout:8/249: truncate de/d18/d27/d41/f50 918441 0 2024-09-16T23:48:47.828 INFO:tasks.workunit.client.1.smithi043.stdout:8/250: fsync de/f17 0 2024-09-16T23:48:47.831 INFO:tasks.workunit.client.0.smithi042.stdout:9/44: truncate f2 2282071 0 2024-09-16T23:48:47.832 INFO:tasks.workunit.client.1.smithi043.stdout:6/270: sync 2024-09-16T23:48:47.832 INFO:tasks.workunit.client.1.smithi043.stdout:7/217: sync 2024-09-16T23:48:47.833 INFO:tasks.workunit.client.0.smithi042.stdout:2/55: truncate d0/da/fd 1009034 0 2024-09-16T23:48:47.835 INFO:tasks.workunit.client.0.smithi042.stdout:0/21: creat d0/f5 x:0 0 0 2024-09-16T23:48:47.835 INFO:tasks.workunit.client.0.smithi042.stdout:7/31: dwrite d3/fa [0,4194304] 0 2024-09-16T23:48:47.838 INFO:tasks.workunit.client.1.smithi043.stdout:8/251: dread de/ff [0,4194304] 0 2024-09-16T23:48:47.840 INFO:tasks.workunit.client.1.smithi043.stdout:4/256: dwrite f7 [0,4194304] 0 2024-09-16T23:48:47.840 INFO:tasks.workunit.client.0.smithi042.stdout:3/14: mknod d2/c3 0 2024-09-16T23:48:47.842 INFO:tasks.workunit.client.0.smithi042.stdout:5/63: dwrite d9/db/ff [0,4194304] 0 2024-09-16T23:48:47.843 INFO:tasks.workunit.client.0.smithi042.stdout:0/22: dread - d0/f4 zero size 2024-09-16T23:48:47.846 INFO:tasks.workunit.client.1.smithi043.stdout:2/179: dwrite d8/d1d/d20/d26/f27 [0,4194304] 0 2024-09-16T23:48:47.848 INFO:tasks.workunit.client.1.smithi043.stdout:9/185: write d4/d5/d16/f1f [603010,67442] 0 2024-09-16T23:48:47.849 INFO:tasks.workunit.client.1.smithi043.stdout:0/246: dwrite d0/dc/f42 [4194304,4194304] 0 2024-09-16T23:48:47.849 INFO:tasks.workunit.client.0.smithi042.stdout:2/56: rename d0/fc to d0/f12 0 2024-09-16T23:48:47.851 INFO:tasks.workunit.client.1.smithi043.stdout:7/218: write d4/db/d11/d15/f3d [1484758,62618] 0 2024-09-16T23:48:47.856 INFO:tasks.workunit.client.0.smithi042.stdout:1/107: dread d0/f3 [0,4194304] 0 2024-09-16T23:48:47.856 INFO:tasks.workunit.client.0.smithi042.stdout:1/108: chown d0/d9 1 1 2024-09-16T23:48:47.856 INFO:tasks.workunit.client.1.smithi043.stdout:4/257: mkdir d8/d15/d19/d49/d4c 0 2024-09-16T23:48:47.857 INFO:tasks.workunit.client.1.smithi043.stdout:0/247: write d0/d2/df/d1a/d46/f35 [1188308,122249] 0 2024-09-16T23:48:47.859 INFO:tasks.workunit.client.0.smithi042.stdout:8/126: dread da/f14 [0,4194304] 0 2024-09-16T23:48:47.859 INFO:tasks.workunit.client.0.smithi042.stdout:6/53: link f7 d8/d9/f13 0 2024-09-16T23:48:47.859 INFO:tasks.workunit.client.0.smithi042.stdout:0/23: symlink d0/l6 0 2024-09-16T23:48:47.860 INFO:tasks.workunit.client.1.smithi043.stdout:8/252: dwrite de/d10/f24 [0,4194304] 0 2024-09-16T23:48:47.865 INFO:tasks.workunit.client.1.smithi043.stdout:7/219: dwrite d4/f32 [0,4194304] 0 2024-09-16T23:48:47.865 INFO:tasks.workunit.client.1.smithi043.stdout:9/186: mknod d4/d21/d33/c36 0 2024-09-16T23:48:47.868 INFO:tasks.workunit.client.0.smithi042.stdout:7/32: dwrite d3/fa [0,4194304] 0 2024-09-16T23:48:47.870 INFO:tasks.workunit.client.0.smithi042.stdout:2/57: mknod d0/da/c13 0 2024-09-16T23:48:47.870 INFO:tasks.workunit.client.0.smithi042.stdout:7/33: stat d3/f8 0 2024-09-16T23:48:47.871 INFO:tasks.workunit.client.1.smithi043.stdout:0/248: dwrite d0/d2/df/d1a/d3b/d27/f32 [0,4194304] 0 2024-09-16T23:48:47.873 INFO:tasks.workunit.client.1.smithi043.stdout:6/271: getdents de/d23 0 2024-09-16T23:48:47.875 INFO:tasks.workunit.client.0.smithi042.stdout:2/58: stat d0/l9 0 2024-09-16T23:48:47.879 INFO:tasks.workunit.client.0.smithi042.stdout:5/64: dwrite d9/db/ff [0,4194304] 0 2024-09-16T23:48:47.879 INFO:tasks.workunit.client.0.smithi042.stdout:3/15: dwrite f0 [0,4194304] 0 2024-09-16T23:48:47.882 INFO:tasks.workunit.client.0.smithi042.stdout:5/65: readlink l8 0 2024-09-16T23:48:47.885 INFO:tasks.workunit.client.0.smithi042.stdout:2/59: mknod d0/da/c14 0 2024-09-16T23:48:47.885 INFO:tasks.workunit.client.0.smithi042.stdout:9/45: write d4/f5 [2720941,4754] 0 2024-09-16T23:48:47.888 INFO:tasks.workunit.client.0.smithi042.stdout:1/109: mkdir d0/d15/d23 0 2024-09-16T23:48:47.891 INFO:tasks.workunit.client.0.smithi042.stdout:3/16: creat d2/f4 x:0 0 0 2024-09-16T23:48:47.891 INFO:tasks.workunit.client.0.smithi042.stdout:3/17: stat f0 0 2024-09-16T23:48:47.891 INFO:tasks.workunit.client.0.smithi042.stdout:3/18: chown f1 30499 1 2024-09-16T23:48:47.891 INFO:tasks.workunit.client.0.smithi042.stdout:0/24: unlink d0/l2 0 2024-09-16T23:48:47.897 INFO:tasks.workunit.client.0.smithi042.stdout:6/54: dread f7 [0,4194304] 0 2024-09-16T23:48:47.897 INFO:tasks.workunit.client.0.smithi042.stdout:3/19: dread - d2/f4 zero size 2024-09-16T23:48:47.897 INFO:tasks.workunit.client.0.smithi042.stdout:5/66: dread d9/db/ff [0,4194304] 0 2024-09-16T23:48:47.897 INFO:tasks.workunit.client.1.smithi043.stdout:4/258: rename d8/d15/d20/d35 to d8/d15/d20/d4d 0 2024-09-16T23:48:47.898 INFO:tasks.workunit.client.0.smithi042.stdout:7/34: dread d3/f8 [0,4194304] 0 2024-09-16T23:48:47.898 INFO:tasks.workunit.client.1.smithi043.stdout:4/259: dread - d8/d15/d20/d2b/f32 zero size 2024-09-16T23:48:47.898 INFO:tasks.workunit.client.0.smithi042.stdout:9/46: symlink d4/d10/l11 0 2024-09-16T23:48:47.899 INFO:tasks.workunit.client.1.smithi043.stdout:4/260: chown d8/d15/d20/d2b/d22 14035081 1 2024-09-16T23:48:47.901 INFO:tasks.workunit.client.1.smithi043.stdout:4/261: write d8/d15/d20/d2b/d23/f40 [407319,115285] 0 2024-09-16T23:48:47.901 INFO:tasks.workunit.client.0.smithi042.stdout:3/20: symlink d2/l5 0 2024-09-16T23:48:47.901 INFO:tasks.workunit.client.0.smithi042.stdout:0/25: chown d0/f5 426 1 2024-09-16T23:48:47.905 INFO:tasks.workunit.client.0.smithi042.stdout:6/55: truncate f5 4795833 0 2024-09-16T23:48:47.907 INFO:tasks.workunit.client.0.smithi042.stdout:8/127: getdents da/d19/d1d 0 2024-09-16T23:48:47.909 INFO:tasks.workunit.client.0.smithi042.stdout:9/47: stat d4/ld 0 2024-09-16T23:48:47.909 INFO:tasks.workunit.client.0.smithi042.stdout:3/21: symlink d2/l6 0 2024-09-16T23:48:47.909 INFO:tasks.workunit.client.0.smithi042.stdout:9/48: stat d4/d10 0 2024-09-16T23:48:47.909 INFO:tasks.workunit.client.1.smithi043.stdout:4/262: dwrite d8/d15/d20/d2b/d22/f36 [0,4194304] 0 2024-09-16T23:48:47.911 INFO:tasks.workunit.client.0.smithi042.stdout:6/56: chown d8/d9/f13 1195529370 1 2024-09-16T23:48:47.911 INFO:tasks.workunit.client.0.smithi042.stdout:2/60: link d0/c6 d0/c15 0 2024-09-16T23:48:47.912 INFO:tasks.workunit.client.0.smithi042.stdout:2/61: fsync d0/f11 0 2024-09-16T23:48:47.913 INFO:tasks.workunit.client.1.smithi043.stdout:8/253: rmdir de/d18/d1f 39 2024-09-16T23:48:47.916 INFO:tasks.workunit.client.0.smithi042.stdout:9/49: stat d4/lf 0 2024-09-16T23:48:47.916 INFO:tasks.workunit.client.0.smithi042.stdout:0/26: write d0/f5 [859335,122922] 0 2024-09-16T23:48:47.917 INFO:tasks.workunit.client.0.smithi042.stdout:7/35: rename d3/l9 to d3/lb 0 2024-09-16T23:48:47.918 INFO:tasks.workunit.client.0.smithi042.stdout:9/50: mkdir d4/d12 0 2024-09-16T23:48:47.918 INFO:tasks.workunit.client.0.smithi042.stdout:3/22: write f0 [993233,115372] 0 2024-09-16T23:48:47.919 INFO:tasks.workunit.client.1.smithi043.stdout:7/220: rename d4/db/d17 to d4/d27/d41 0 2024-09-16T23:48:47.920 INFO:tasks.workunit.client.1.smithi043.stdout:7/221: write d4/d27/d41/f40 [182329,113769] 0 2024-09-16T23:48:47.920 INFO:tasks.workunit.client.0.smithi042.stdout:9/51: write d4/f5 [1814845,34976] 0 2024-09-16T23:48:47.921 INFO:tasks.workunit.client.0.smithi042.stdout:1/110: dread d0/d9/f11 [0,4194304] 0 2024-09-16T23:48:47.921 INFO:tasks.workunit.client.1.smithi043.stdout:9/187: truncate d4/f11 1902229 0 2024-09-16T23:48:47.924 INFO:tasks.workunit.client.0.smithi042.stdout:2/62: readlink d0/da/lb 0 2024-09-16T23:48:47.925 INFO:tasks.workunit.client.0.smithi042.stdout:6/57: symlink d8/db/dd/l14 0 2024-09-16T23:48:47.925 INFO:tasks.workunit.client.0.smithi042.stdout:7/36: creat d3/fc x:0 0 0 2024-09-16T23:48:47.926 INFO:tasks.workunit.client.1.smithi043.stdout:4/263: symlink d8/d15/d19/d49/l4e 0 2024-09-16T23:48:47.927 INFO:tasks.workunit.client.1.smithi043.stdout:8/254: creat de/d18/d27/d34/d3d/f51 x:0 0 0 2024-09-16T23:48:47.931 INFO:tasks.workunit.client.0.smithi042.stdout:8/128: creat da/f2a x:0 0 0 2024-09-16T23:48:47.931 INFO:tasks.workunit.client.0.smithi042.stdout:1/111: read d0/d9/df/f12 [234214,73599] 0 2024-09-16T23:48:47.931 INFO:tasks.workunit.client.0.smithi042.stdout:9/52: symlink d4/d10/l13 0 2024-09-16T23:48:47.933 INFO:tasks.workunit.client.0.smithi042.stdout:9/53: chown d4/ld 16 1 2024-09-16T23:48:47.934 INFO:tasks.workunit.client.0.smithi042.stdout:5/67: dwrite d9/db/ff [0,4194304] 0 2024-09-16T23:48:47.935 INFO:tasks.workunit.client.1.smithi043.stdout:9/188: symlink d4/d21/d33/l37 0 2024-09-16T23:48:47.937 INFO:tasks.workunit.client.0.smithi042.stdout:6/58: fdatasync f3 0 2024-09-16T23:48:47.942 INFO:tasks.workunit.client.1.smithi043.stdout:7/222: creat d4/db/d11/d21/d36/f42 x:0 0 0 2024-09-16T23:48:47.942 INFO:tasks.workunit.client.0.smithi042.stdout:3/23: dwrite f1 [0,4194304] 0 2024-09-16T23:48:47.942 INFO:tasks.workunit.client.0.smithi042.stdout:7/37: dread d3/fa [0,4194304] 0 2024-09-16T23:48:47.943 INFO:tasks.workunit.client.1.smithi043.stdout:0/249: truncate d0/dc/f42 750490 0 2024-09-16T23:48:47.943 INFO:tasks.workunit.client.0.smithi042.stdout:2/63: rename d0/da/c13 to d0/c16 0 2024-09-16T23:48:47.944 INFO:tasks.workunit.client.0.smithi042.stdout:0/27: creat d0/f7 x:0 0 0 2024-09-16T23:48:47.947 INFO:tasks.workunit.client.1.smithi043.stdout:0/250: dread d0/f41 [0,4194304] 0 2024-09-16T23:48:47.948 INFO:tasks.workunit.client.0.smithi042.stdout:1/112: unlink d0/d9/c14 0 2024-09-16T23:48:47.948 INFO:tasks.workunit.client.0.smithi042.stdout:6/59: rmdir d8/db/dd 39 2024-09-16T23:48:47.950 INFO:tasks.workunit.client.0.smithi042.stdout:7/38: mknod d3/cd 0 2024-09-16T23:48:47.950 INFO:tasks.workunit.client.0.smithi042.stdout:3/24: mkdir d2/d7 0 2024-09-16T23:48:47.952 INFO:tasks.workunit.client.1.smithi043.stdout:4/264: creat d8/d25/f4f x:0 0 0 2024-09-16T23:48:47.953 INFO:tasks.workunit.client.0.smithi042.stdout:7/39: read d3/f8 [1463217,56174] 0 2024-09-16T23:48:47.953 INFO:tasks.workunit.client.0.smithi042.stdout:5/68: dwrite d9/db/ff [0,4194304] 0 2024-09-16T23:48:47.954 INFO:tasks.workunit.client.1.smithi043.stdout:4/265: stat d8/d15/d20/d2b/f28 0 2024-09-16T23:48:47.954 INFO:tasks.workunit.client.0.smithi042.stdout:0/28: creat d0/f8 x:0 0 0 2024-09-16T23:48:47.956 INFO:tasks.workunit.client.0.smithi042.stdout:0/29: readlink d0/l6 0 2024-09-16T23:48:47.958 INFO:tasks.workunit.client.0.smithi042.stdout:6/60: chown f6 54254 1 2024-09-16T23:48:47.961 INFO:tasks.workunit.client.0.smithi042.stdout:3/25: creat d2/d7/f8 x:0 0 0 2024-09-16T23:48:47.962 INFO:tasks.workunit.client.1.smithi043.stdout:8/255: dread de/d18/f36 [4194304,4194304] 0 2024-09-16T23:48:47.966 INFO:tasks.workunit.client.0.smithi042.stdout:6/61: write d8/d9/f10 [3750850,6360] 0 2024-09-16T23:48:47.967 INFO:tasks.workunit.client.1.smithi043.stdout:0/251: symlink d0/d2/df/d1a/d46/l4c 0 2024-09-16T23:48:47.968 INFO:tasks.workunit.client.1.smithi043.stdout:6/272: getdents de/d1a/d28 0 2024-09-16T23:48:47.968 INFO:tasks.workunit.client.1.smithi043.stdout:6/273: stat de 0 2024-09-16T23:48:47.969 INFO:tasks.workunit.client.1.smithi043.stdout:6/274: stat de/d23/c4f 0 2024-09-16T23:48:47.969 INFO:tasks.workunit.client.0.smithi042.stdout:1/113: getdents d0/d9 0 2024-09-16T23:48:47.972 INFO:tasks.workunit.client.1.smithi043.stdout:4/266: mkdir d8/d15/d20/d2b/d22/d50 0 2024-09-16T23:48:47.972 INFO:tasks.workunit.client.0.smithi042.stdout:9/54: truncate d4/fa 3924588 0 2024-09-16T23:48:47.973 INFO:tasks.workunit.client.1.smithi043.stdout:8/256: write de/d18/f36 [2039809,39662] 0 2024-09-16T23:48:47.974 INFO:tasks.workunit.client.1.smithi043.stdout:9/189: link d4/l29 d4/d5/l38 0 2024-09-16T23:48:47.975 INFO:tasks.workunit.client.1.smithi043.stdout:7/223: rmdir d4/db/d28/d3a 0 2024-09-16T23:48:47.977 INFO:tasks.workunit.client.1.smithi043.stdout:7/224: dread - d4/d27/f3f zero size 2024-09-16T23:48:47.977 INFO:tasks.workunit.client.1.smithi043.stdout:0/252: dwrite d0/d2/df/d1a/d46/f35 [0,4194304] 0 2024-09-16T23:48:47.978 INFO:tasks.workunit.client.0.smithi042.stdout:1/114: read d0/f3 [86384,77458] 0 2024-09-16T23:48:47.978 INFO:tasks.workunit.client.0.smithi042.stdout:2/64: chown d0/c16 31533 1 2024-09-16T23:48:47.978 INFO:tasks.workunit.client.1.smithi043.stdout:6/275: rmdir de/d23/d49 39 2024-09-16T23:48:47.979 INFO:tasks.workunit.client.0.smithi042.stdout:5/69: rename d9/db/d11 to d9/db/d11/d12 22 2024-09-16T23:48:47.979 INFO:tasks.workunit.client.0.smithi042.stdout:8/129: dwrite da/f14 [0,4194304] 0 2024-09-16T23:48:47.981 INFO:tasks.workunit.client.1.smithi043.stdout:6/276: chown de/d23/d34/l3f 13 1 2024-09-16T23:48:47.982 INFO:tasks.workunit.client.1.smithi043.stdout:9/190: write d4/f10 [3564350,69662] 0 2024-09-16T23:48:47.984 INFO:tasks.workunit.client.1.smithi043.stdout:8/257: symlink de/l52 0 2024-09-16T23:48:47.985 INFO:tasks.workunit.client.1.smithi043.stdout:4/267: dwrite d8/d15/d20/d2b/d22/f24 [4194304,4194304] 0 2024-09-16T23:48:47.987 INFO:tasks.workunit.client.1.smithi043.stdout:6/277: creat de/d23/d34/f5d x:0 0 0 2024-09-16T23:48:47.989 INFO:tasks.workunit.client.0.smithi042.stdout:7/40: dwrite d3/fa [0,4194304] 0 2024-09-16T23:48:47.989 INFO:tasks.workunit.client.0.smithi042.stdout:0/30: dwrite d0/f8 [0,4194304] 0 2024-09-16T23:48:47.990 INFO:tasks.workunit.client.1.smithi043.stdout:6/278: dread de/f40 [0,4194304] 0 2024-09-16T23:48:47.991 INFO:tasks.workunit.client.0.smithi042.stdout:2/65: mkdir d0/da/d17 0 2024-09-16T23:48:47.991 INFO:tasks.workunit.client.1.smithi043.stdout:0/253: fdatasync d0/d2/df/f17 0 2024-09-16T23:48:47.992 INFO:tasks.workunit.client.0.smithi042.stdout:3/26: truncate f0 4109298 0 2024-09-16T23:48:47.993 INFO:tasks.workunit.client.0.smithi042.stdout:5/70: read d9/db/ff [2745145,104503] 0 2024-09-16T23:48:47.993 INFO:tasks.workunit.client.0.smithi042.stdout:1/115: read - d0/d9/f16 zero size 2024-09-16T23:48:47.993 INFO:tasks.workunit.client.0.smithi042.stdout:9/55: dwrite d4/f5 [0,4194304] 0 2024-09-16T23:48:47.994 INFO:tasks.workunit.client.1.smithi043.stdout:0/254: chown d0/l3a 56 1 2024-09-16T23:48:47.994 INFO:tasks.workunit.client.1.smithi043.stdout:4/268: chown d8/c12 40 1 2024-09-16T23:48:47.996 INFO:tasks.workunit.client.1.smithi043.stdout:9/191: creat d4/d5/d30/f39 x:0 0 0 2024-09-16T23:48:47.996 INFO:tasks.workunit.client.0.smithi042.stdout:0/31: stat d0/f8 0 2024-09-16T23:48:47.996 INFO:tasks.workunit.client.1.smithi043.stdout:7/225: dwrite d4/db/f24 [0,4194304] 0 2024-09-16T23:48:47.997 INFO:tasks.workunit.client.0.smithi042.stdout:7/41: write d3/fc [1038468,63081] 0 2024-09-16T23:48:47.997 INFO:tasks.workunit.client.0.smithi042.stdout:4/53: sync 2024-09-16T23:48:47.998 INFO:tasks.workunit.client.1.smithi043.stdout:8/258: dread de/d10/f24 [0,4194304] 0 2024-09-16T23:48:47.999 INFO:tasks.workunit.client.1.smithi043.stdout:9/192: fsync d4/d1c/f27 0 2024-09-16T23:48:48.014 INFO:tasks.workunit.client.1.smithi043.stdout:4/269: rename d8/d15/d20/d2b/l46 to d8/d15/d19/l51 0 2024-09-16T23:48:48.021 INFO:tasks.workunit.client.0.smithi042.stdout:3/27: chown f1 5 1 2024-09-16T23:48:48.021 INFO:tasks.workunit.client.1.smithi043.stdout:4/270: dwrite d8/d25/d39/f3a [0,4194304] 0 2024-09-16T23:48:48.023 INFO:tasks.workunit.client.0.smithi042.stdout:1/116: chown d0/d17/d18/c1f 3 1 2024-09-16T23:48:48.023 INFO:tasks.workunit.client.0.smithi042.stdout:0/32: truncate d0/f7 552307 0 2024-09-16T23:48:48.024 INFO:tasks.workunit.client.1.smithi043.stdout:6/279: truncate de/d1a/d39/f1f 2424158 0 2024-09-16T23:48:48.026 INFO:tasks.workunit.client.1.smithi043.stdout:7/226: dwrite d4/db/f18 [0,4194304] 0 2024-09-16T23:48:48.027 INFO:tasks.workunit.client.0.smithi042.stdout:0/33: chown d0 0 1 2024-09-16T23:48:48.028 INFO:tasks.workunit.client.0.smithi042.stdout:2/66: chown d0/f12 98859127 1 2024-09-16T23:48:48.029 INFO:tasks.workunit.client.0.smithi042.stdout:8/130: creat da/d19/d1d/f2b x:0 0 0 2024-09-16T23:48:48.031 INFO:tasks.workunit.client.1.smithi043.stdout:4/271: truncate d8/d15/d20/d2b/d22/f24 8467491 0 2024-09-16T23:48:48.032 INFO:tasks.workunit.client.1.smithi043.stdout:0/255: creat d0/f4d x:0 0 0 2024-09-16T23:48:48.033 INFO:tasks.workunit.client.0.smithi042.stdout:6/62: dread d8/db/dd/f11 [0,4194304] 0 2024-09-16T23:48:48.033 INFO:tasks.workunit.client.1.smithi043.stdout:4/272: readlink d8/d15/d19/l43 0 2024-09-16T23:48:48.034 INFO:tasks.workunit.client.1.smithi043.stdout:6/280: dread fd [0,4194304] 0 2024-09-16T23:48:48.034 INFO:tasks.workunit.client.1.smithi043.stdout:6/281: stat de/d1a/d39 0 2024-09-16T23:48:48.035 INFO:tasks.workunit.client.0.smithi042.stdout:6/63: chown d8/db/dd 996969 1 2024-09-16T23:48:48.037 INFO:tasks.workunit.client.1.smithi043.stdout:8/259: mknod de/d18/d1f/d2f/d39/d4d/c53 0 2024-09-16T23:48:48.037 INFO:tasks.workunit.client.1.smithi043.stdout:7/227: mkdir d4/db/d43 0 2024-09-16T23:48:48.039 INFO:tasks.workunit.client.0.smithi042.stdout:5/71: dread d9/db/ff [0,4194304] 0 2024-09-16T23:48:48.040 INFO:tasks.workunit.client.0.smithi042.stdout:6/64: read f3 [2665268,47293] 0 2024-09-16T23:48:48.042 INFO:tasks.workunit.client.1.smithi043.stdout:3/187: sync 2024-09-16T23:48:48.043 INFO:tasks.workunit.client.1.smithi043.stdout:0/256: creat d0/d2/df/d1a/d3b/f4e x:0 0 0 2024-09-16T23:48:48.043 INFO:tasks.workunit.client.1.smithi043.stdout:9/193: dread d4/d1c/f20 [0,4194304] 0 2024-09-16T23:48:48.043 INFO:tasks.workunit.client.1.smithi043.stdout:3/188: chown d4/de/f21 42501 1 2024-09-16T23:48:48.044 INFO:tasks.workunit.client.1.smithi043.stdout:6/282: mknod de/d1a/d39/c5e 0 2024-09-16T23:48:48.045 INFO:tasks.workunit.client.1.smithi043.stdout:3/189: fsync d4/d17/f1d 0 2024-09-16T23:48:48.046 INFO:tasks.workunit.client.1.smithi043.stdout:7/228: creat d4/d27/d41/d1b/f44 x:0 0 0 2024-09-16T23:48:48.046 INFO:tasks.workunit.client.0.smithi042.stdout:2/67: dwrite d0/da/fd [0,4194304] 0 2024-09-16T23:48:48.047 INFO:tasks.workunit.client.1.smithi043.stdout:7/229: dread - d4/d27/d41/d1b/f44 zero size 2024-09-16T23:48:48.050 INFO:tasks.workunit.client.1.smithi043.stdout:3/190: chown d4/c1a 1 1 2024-09-16T23:48:48.053 INFO:tasks.workunit.client.1.smithi043.stdout:4/273: dwrite d8/f3b [0,4194304] 0 2024-09-16T23:48:48.053 INFO:tasks.workunit.client.1.smithi043.stdout:9/194: rename d4/d1c/d25/d2b/c2f to d4/d1c/c3a 0 2024-09-16T23:48:48.054 INFO:tasks.workunit.client.1.smithi043.stdout:4/274: fdatasync d8/d15/d20/d2b/d23/f38 0 2024-09-16T23:48:48.055 INFO:tasks.workunit.client.1.smithi043.stdout:0/257: mkdir d0/d2/dd/d4f 0 2024-09-16T23:48:48.059 INFO:tasks.workunit.client.1.smithi043.stdout:9/195: write d4/d21/d2a/d2c/f2d [2920278,112388] 0 2024-09-16T23:48:48.060 INFO:tasks.workunit.client.1.smithi043.stdout:4/275: mknod d8/d15/d20/d2b/c52 0 2024-09-16T23:48:48.063 INFO:tasks.workunit.client.1.smithi043.stdout:7/230: creat d4/f45 x:0 0 0 2024-09-16T23:48:48.064 INFO:tasks.workunit.client.1.smithi043.stdout:0/258: read d0/d2/df/f17 [1489471,91367] 0 2024-09-16T23:48:48.065 INFO:tasks.workunit.client.1.smithi043.stdout:3/191: link d4/f1b d4/f33 0 2024-09-16T23:48:48.066 INFO:tasks.workunit.client.1.smithi043.stdout:0/259: chown d0/dc 28600 1 2024-09-16T23:48:48.067 INFO:tasks.workunit.client.1.smithi043.stdout:7/231: mknod d4/db/d43/c46 0 2024-09-16T23:48:48.068 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:48.068 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr fail", "who": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.14213 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "mgr fail", "who": "smithi042.zrkumz"}]': finished 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: mgrmap e21: smithi043.xveyou(active, starting, since 0.017473s) 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:48:48.069 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:48:48.070 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:47 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:48:48.071 INFO:tasks.workunit.client.1.smithi043.stdout:8/260: getdents de/d18/d1f/d2f/d39/d4d 0 2024-09-16T23:48:48.071 INFO:tasks.workunit.client.1.smithi043.stdout:9/196: truncate d4/f11 2880564 0 2024-09-16T23:48:48.071 INFO:tasks.workunit.client.1.smithi043.stdout:0/260: creat d0/dc/f50 x:0 0 0 2024-09-16T23:48:48.072 INFO:tasks.workunit.client.1.smithi043.stdout:3/192: dread - d4/de/d1f/d2c/f32 zero size 2024-09-16T23:48:48.073 INFO:tasks.workunit.client.1.smithi043.stdout:6/283: write de/d23/d49/f4c [485574,112237] 0 2024-09-16T23:48:48.074 INFO:tasks.workunit.client.1.smithi043.stdout:4/276: dwrite d8/d15/f21 [0,4194304] 0 2024-09-16T23:48:48.075 INFO:tasks.workunit.client.1.smithi043.stdout:0/261: dread - d0/dc/f50 zero size 2024-09-16T23:48:48.076 INFO:tasks.workunit.client.1.smithi043.stdout:9/197: read d4/ff [3086698,67742] 0 2024-09-16T23:48:48.079 INFO:tasks.workunit.client.1.smithi043.stdout:0/262: chown d0/f3c 4534 1 2024-09-16T23:48:48.081 INFO:tasks.workunit.client.1.smithi043.stdout:7/232: rename d4/d27/d41/d1b/f37 to d4/db/d11/d15/f47 0 2024-09-16T23:48:48.083 INFO:tasks.workunit.client.1.smithi043.stdout:6/284: write de/d1a/f38 [533904,59830] 0 2024-09-16T23:48:48.086 INFO:tasks.workunit.client.1.smithi043.stdout:2/180: sync 2024-09-16T23:48:48.086 INFO:tasks.workunit.client.1.smithi043.stdout:1/331: sync 2024-09-16T23:48:48.086 INFO:tasks.workunit.client.1.smithi043.stdout:5/375: sync 2024-09-16T23:48:48.086 INFO:tasks.workunit.client.1.smithi043.stdout:9/198: symlink d4/d28/l3b 0 2024-09-16T23:48:48.092 INFO:tasks.workunit.client.1.smithi043.stdout:5/376: unlink d3/d9/ca 0 2024-09-16T23:48:48.095 INFO:tasks.workunit.client.1.smithi043.stdout:1/332: write d4/d9/d12/f1f [3994997,102579] 0 2024-09-16T23:48:48.095 INFO:tasks.workunit.client.1.smithi043.stdout:6/285: creat de/d1a/d28/f5f x:0 0 0 2024-09-16T23:48:48.095 INFO:tasks.workunit.client.1.smithi043.stdout:4/277: dwrite d8/d15/d20/d2b/d22/f24 [0,4194304] 0 2024-09-16T23:48:48.100 INFO:tasks.workunit.client.1.smithi043.stdout:7/233: rename d4/db/d11/c16 to d4/db/d11/d21/d31/c48 0 2024-09-16T23:48:48.100 INFO:tasks.workunit.client.1.smithi043.stdout:6/286: dread - de/d1a/f3b zero size 2024-09-16T23:48:48.101 INFO:tasks.workunit.client.1.smithi043.stdout:0/263: dread d0/d2/df/d1a/d46/f35 [0,4194304] 0 2024-09-16T23:48:48.102 INFO:tasks.workunit.client.1.smithi043.stdout:1/333: write d4/d9/d12/f33 [232660,8624] 0 2024-09-16T23:48:48.102 INFO:tasks.workunit.client.1.smithi043.stdout:5/377: write d3/db/f2a [2344037,41295] 0 2024-09-16T23:48:48.108 INFO:tasks.workunit.client.1.smithi043.stdout:8/261: dread de/d10/f13 [0,4194304] 0 2024-09-16T23:48:48.108 INFO:tasks.workunit.client.1.smithi043.stdout:2/181: dwrite d8/d1d/d25/f2b [0,4194304] 0 2024-09-16T23:48:48.109 INFO:tasks.workunit.client.1.smithi043.stdout:6/287: mknod de/d1a/c60 0 2024-09-16T23:48:48.110 INFO:tasks.workunit.client.1.smithi043.stdout:8/262: stat l7 0 2024-09-16T23:48:48.111 INFO:tasks.workunit.client.1.smithi043.stdout:0/264: fsync d0/d2/df/f10 0 2024-09-16T23:48:48.111 INFO:tasks.workunit.client.1.smithi043.stdout:2/182: fsync d8/d35/f36 0 2024-09-16T23:48:48.113 INFO:tasks.workunit.client.1.smithi043.stdout:7/234: mknod d4/db/d28/c49 0 2024-09-16T23:48:48.114 INFO:tasks.workunit.client.1.smithi043.stdout:4/278: symlink d8/l53 0 2024-09-16T23:48:48.118 INFO:tasks.workunit.client.1.smithi043.stdout:5/378: creat d3/db/d19/f77 x:0 0 0 2024-09-16T23:48:48.122 INFO:tasks.workunit.client.1.smithi043.stdout:4/279: mkdir d8/d15/d20/d2b/d54 0 2024-09-16T23:48:48.123 INFO:tasks.workunit.client.1.smithi043.stdout:8/263: write de/d10/f4a [163922,63085] 0 2024-09-16T23:48:48.123 INFO:tasks.workunit.client.1.smithi043.stdout:7/235: creat d4/db/d11/d21/d31/f4a x:0 0 0 2024-09-16T23:48:48.124 INFO:tasks.workunit.client.1.smithi043.stdout:6/288: dwrite de/d1a/d39/f16 [0,4194304] 0 2024-09-16T23:48:48.128 INFO:tasks.workunit.client.1.smithi043.stdout:5/379: mkdir d3/db/d19/d76/d78 0 2024-09-16T23:48:48.134 INFO:tasks.workunit.client.1.smithi043.stdout:8/264: dread de/ff [0,4194304] 0 2024-09-16T23:48:48.136 INFO:tasks.workunit.client.1.smithi043.stdout:1/334: dwrite d4/d9/f17 [0,4194304] 0 2024-09-16T23:48:48.141 INFO:tasks.workunit.client.1.smithi043.stdout:1/335: write d4/d9/d12/d2b/f52 [40454,67062] 0 2024-09-16T23:48:48.142 INFO:tasks.workunit.client.1.smithi043.stdout:5/380: mkdir d3/d79 0 2024-09-16T23:48:48.145 INFO:tasks.workunit.client.1.smithi043.stdout:7/236: dwrite d4/f45 [0,4194304] 0 2024-09-16T23:48:48.146 INFO:tasks.workunit.client.1.smithi043.stdout:0/265: link d0/d2/df/d1a/d3b/l1d d0/d2/df/d1a/d46/d39/l51 0 2024-09-16T23:48:48.149 INFO:tasks.workunit.client.1.smithi043.stdout:5/381: readlink d3/db/d19/d3e/l30 0 2024-09-16T23:48:48.149 INFO:tasks.workunit.client.1.smithi043.stdout:6/289: rename de/f19 to de/d1a/d4d/d50/f61 0 2024-09-16T23:48:48.151 INFO:tasks.workunit.client.1.smithi043.stdout:8/265: creat de/f54 x:0 0 0 2024-09-16T23:48:48.153 INFO:tasks.workunit.client.1.smithi043.stdout:7/237: mkdir d4/db/d43/d4b 0 2024-09-16T23:48:48.155 INFO:tasks.workunit.client.1.smithi043.stdout:1/336: creat d4/d9/d1c/d1e/d42/f70 x:0 0 0 2024-09-16T23:48:48.156 INFO:tasks.workunit.client.1.smithi043.stdout:5/382: mkdir d3/d9/d22/d4a/d4c/d7a 0 2024-09-16T23:48:48.158 INFO:tasks.workunit.client.1.smithi043.stdout:0/266: mknod d0/dc/c52 0 2024-09-16T23:48:48.159 INFO:tasks.workunit.client.1.smithi043.stdout:6/290: dwrite de/d1a/f41 [0,4194304] 0 2024-09-16T23:48:48.159 INFO:tasks.workunit.client.1.smithi043.stdout:8/266: symlink de/d18/d1f/d2f/l55 0 2024-09-16T23:48:48.162 INFO:tasks.workunit.client.1.smithi043.stdout:0/267: fdatasync d0/d2/df/d1a/d3b/f4e 0 2024-09-16T23:48:48.168 INFO:tasks.workunit.client.1.smithi043.stdout:6/291: chown de/l4a 973972 1 2024-09-16T23:48:48.170 INFO:tasks.workunit.client.1.smithi043.stdout:7/238: dwrite d4/d27/f3f [0,4194304] 0 2024-09-16T23:48:48.171 INFO:tasks.workunit.client.1.smithi043.stdout:8/267: rmdir de/d18/d1f/d2f/d39 39 2024-09-16T23:48:48.177 INFO:tasks.workunit.client.1.smithi043.stdout:1/337: dwrite d4/d9/d1c/d1e/f2a [0,4194304] 0 2024-09-16T23:48:48.181 INFO:tasks.workunit.client.1.smithi043.stdout:8/268: chown de/d10/f1b 7 1 2024-09-16T23:48:48.186 INFO:tasks.workunit.client.1.smithi043.stdout:7/239: dwrite d4/d27/f3f [0,4194304] 0 2024-09-16T23:48:48.192 INFO:tasks.workunit.client.1.smithi043.stdout:8/269: chown de/d18/d1f/d2f/f3e 3541348 1 2024-09-16T23:48:48.193 INFO:tasks.workunit.client.1.smithi043.stdout:0/268: dwrite d0/d2/df/d1a/d46/f35 [0,4194304] 0 2024-09-16T23:48:48.196 INFO:tasks.workunit.client.1.smithi043.stdout:5/383: dread d3/d9/d22/f4e [0,4194304] 0 2024-09-16T23:48:48.198 INFO:tasks.workunit.client.1.smithi043.stdout:0/269: read - d0/d2/df/d1a/d3b/f4e zero size 2024-09-16T23:48:48.203 INFO:tasks.workunit.client.1.smithi043.stdout:8/270: fsync de/d18/d1f/d2f/f3e 0 2024-09-16T23:48:48.203 INFO:tasks.workunit.client.1.smithi043.stdout:7/240: creat d4/d27/d41/f4c x:0 0 0 2024-09-16T23:48:48.204 INFO:tasks.workunit.client.1.smithi043.stdout:0/270: rmdir d0/d2/df 39 2024-09-16T23:48:48.206 INFO:tasks.workunit.client.1.smithi043.stdout:1/338: dwrite d4/d9/d1c/d1e/d42/f70 [0,4194304] 0 2024-09-16T23:48:48.208 INFO:tasks.workunit.client.1.smithi043.stdout:5/384: creat d3/d9/d22/d4a/d64/f7b x:0 0 0 2024-09-16T23:48:48.213 INFO:tasks.workunit.client.1.smithi043.stdout:6/292: dread de/f24 [0,4194304] 0 2024-09-16T23:48:48.214 INFO:tasks.workunit.client.1.smithi043.stdout:0/271: dread - d0/d2/df/d1a/d3b/f4e zero size 2024-09-16T23:48:48.216 INFO:tasks.workunit.client.1.smithi043.stdout:7/241: dwrite d4/d27/f3f [0,4194304] 0 2024-09-16T23:48:48.218 INFO:tasks.workunit.client.1.smithi043.stdout:1/339: write d4/d34/f41 [726808,7577] 0 2024-09-16T23:48:48.225 INFO:tasks.workunit.client.1.smithi043.stdout:7/242: chown d4/d27/d41/l1d 47858026 1 2024-09-16T23:48:48.227 INFO:tasks.workunit.client.1.smithi043.stdout:6/293: mknod de/c62 0 2024-09-16T23:48:48.227 INFO:tasks.workunit.client.1.smithi043.stdout:8/271: link de/f45 de/d10/f56 0 2024-09-16T23:48:48.228 INFO:tasks.workunit.client.1.smithi043.stdout:1/340: truncate d4/d9/f4e 741516 0 2024-09-16T23:48:48.228 INFO:tasks.workunit.client.1.smithi043.stdout:8/272: stat l6 0 2024-09-16T23:48:48.230 INFO:tasks.workunit.client.1.smithi043.stdout:7/243: fsync d4/d27/d41/f26 0 2024-09-16T23:48:48.234 INFO:tasks.workunit.client.1.smithi043.stdout:6/294: chown de/l2f 734355 1 2024-09-16T23:48:48.236 INFO:tasks.workunit.client.1.smithi043.stdout:1/341: mkdir d4/d9/d1c/d1e/d42/d5d/d71 0 2024-09-16T23:48:48.238 INFO:tasks.workunit.client.1.smithi043.stdout:7/244: creat d4/db/d28/f4d x:0 0 0 2024-09-16T23:48:48.239 INFO:tasks.workunit.client.1.smithi043.stdout:6/295: mkdir de/d23/d63 0 2024-09-16T23:48:48.242 INFO:tasks.workunit.client.1.smithi043.stdout:5/385: dwrite f2 [0,4194304] 0 2024-09-16T23:48:48.244 INFO:tasks.workunit.client.1.smithi043.stdout:8/273: dwrite de/f15 [0,4194304] 0 2024-09-16T23:48:48.245 INFO:tasks.workunit.client.1.smithi043.stdout:1/342: fsync d4/d9/d1c/d1e/d47/f53 0 2024-09-16T23:48:48.247 INFO:tasks.workunit.client.1.smithi043.stdout:1/343: rmdir d4 39 2024-09-16T23:48:48.255 INFO:tasks.workunit.client.1.smithi043.stdout:1/344: mkdir d4/d9/d72 0 2024-09-16T23:48:48.257 INFO:tasks.workunit.client.1.smithi043.stdout:7/245: dread d4/d27/f3f [0,4194304] 0 2024-09-16T23:48:48.259 INFO:tasks.workunit.client.1.smithi043.stdout:0/272: dwrite d0/d2/df/d1a/d3b/d27/f32 [0,4194304] 0 2024-09-16T23:48:48.263 INFO:tasks.workunit.client.1.smithi043.stdout:1/345: creat d4/d9/d72/f73 x:0 0 0 2024-09-16T23:48:48.267 INFO:tasks.workunit.client.1.smithi043.stdout:1/346: mknod d4/d3e/d5b/c74 0 2024-09-16T23:48:48.276 INFO:tasks.workunit.client.1.smithi043.stdout:1/347: mkdir d4/d9/d75 0 2024-09-16T23:48:48.277 INFO:tasks.workunit.client.1.smithi043.stdout:5/386: dread d3/db/d19/d3e/d28/f42 [0,4194304] 0 2024-09-16T23:48:48.280 INFO:tasks.workunit.client.1.smithi043.stdout:5/387: mkdir d3/d9/d6f/d7c 0 2024-09-16T23:48:48.280 INFO:tasks.workunit.client.1.smithi043.stdout:5/388: write d3/db/f2a [1598328,53012] 0 2024-09-16T23:48:48.287 INFO:tasks.workunit.client.1.smithi043.stdout:0/273: dwrite d0/d2/df/d1a/f30 [0,4194304] 0 2024-09-16T23:48:48.287 INFO:tasks.workunit.client.1.smithi043.stdout:1/348: read d4/d9/d12/f33 [1309298,109751] 0 2024-09-16T23:48:48.291 INFO:tasks.workunit.client.1.smithi043.stdout:0/274: fdatasync d0/d2/df/d1a/f36 0 2024-09-16T23:48:48.291 INFO:tasks.workunit.client.1.smithi043.stdout:7/246: truncate d4/f9 4092354 0 2024-09-16T23:48:48.294 INFO:tasks.workunit.client.1.smithi043.stdout:0/275: chown d0/l3a 217320411 1 2024-09-16T23:48:48.294 INFO:tasks.workunit.client.1.smithi043.stdout:7/247: creat d4/d27/d41/d1b/f4e x:0 0 0 2024-09-16T23:48:48.298 INFO:tasks.workunit.client.1.smithi043.stdout:0/276: creat d0/dc/f53 x:0 0 0 2024-09-16T23:48:48.298 INFO:tasks.workunit.client.1.smithi043.stdout:7/248: mknod d4/db/d28/c4f 0 2024-09-16T23:48:48.305 INFO:tasks.workunit.client.1.smithi043.stdout:0/277: dwrite d0/d2/df/d1a/d3b/d27/f32 [0,4194304] 0 2024-09-16T23:48:48.308 INFO:tasks.workunit.client.1.smithi043.stdout:7/249: mkdir d4/db/d50 0 2024-09-16T23:48:48.308 INFO:tasks.workunit.client.1.smithi043.stdout:0/278: chown d0/d2/df/d1a/d3b/d27/c3e 16882471 1 2024-09-16T23:48:48.311 INFO:tasks.workunit.client.1.smithi043.stdout:7/250: creat d4/db/f51 x:0 0 0 2024-09-16T23:48:48.312 INFO:tasks.workunit.client.1.smithi043.stdout:7/251: truncate d4/db/f51 188745 0 2024-09-16T23:48:48.313 INFO:tasks.workunit.client.1.smithi043.stdout:0/279: rename d0/d2/c3 to d0/d2/dd/c54 0 2024-09-16T23:48:48.315 INFO:tasks.workunit.client.1.smithi043.stdout:7/252: write d4/db/d28/f2d [210417,36543] 0 2024-09-16T23:48:48.320 INFO:tasks.workunit.client.1.smithi043.stdout:1/349: truncate d4/d9/f17 4013773 0 2024-09-16T23:48:48.321 INFO:tasks.workunit.client.1.smithi043.stdout:5/389: dwrite d3/db/d19/d3e/f35 [0,4194304] 0 2024-09-16T23:48:48.321 INFO:tasks.workunit.client.1.smithi043.stdout:0/280: creat d0/d2/df/d1a/d3b/d27/f55 x:0 0 0 2024-09-16T23:48:48.324 INFO:tasks.workunit.client.1.smithi043.stdout:5/390: write d3/d9/d22/d4a/d64/f7b [607766,113742] 0 2024-09-16T23:48:48.325 INFO:tasks.workunit.client.1.smithi043.stdout:1/350: stat d4/d9/d1c/d1e/d42/l5e 0 2024-09-16T23:48:48.325 INFO:tasks.workunit.client.1.smithi043.stdout:7/253: mknod d4/db/d50/c52 0 2024-09-16T23:48:48.328 INFO:tasks.workunit.client.1.smithi043.stdout:1/351: truncate d4/df/f38 4825712 0 2024-09-16T23:48:48.330 INFO:tasks.workunit.client.1.smithi043.stdout:5/391: symlink d3/db/d15/d4d/d6b/d73/l7d 0 2024-09-16T23:48:48.330 INFO:tasks.workunit.client.1.smithi043.stdout:1/352: fsync d4/d9/d1c/f37 0 2024-09-16T23:48:48.336 INFO:tasks.workunit.client.1.smithi043.stdout:0/281: creat d0/d2/df/f56 x:0 0 0 2024-09-16T23:48:48.337 INFO:tasks.workunit.client.1.smithi043.stdout:5/392: mknod d3/db/d19/d3e/d28/c7e 0 2024-09-16T23:48:48.337 INFO:tasks.workunit.client.1.smithi043.stdout:7/254: dwrite d4/f45 [4194304,4194304] 0 2024-09-16T23:48:48.342 INFO:tasks.workunit.client.1.smithi043.stdout:7/255: write d4/db/d28/f2d [546974,16862] 0 2024-09-16T23:48:48.343 INFO:tasks.workunit.client.1.smithi043.stdout:1/353: dread d4/d9/d1c/d1e/f2a [4194304,4194304] 0 2024-09-16T23:48:48.344 INFO:tasks.workunit.client.1.smithi043.stdout:1/354: write d4/d34/f4d [893471,44702] 0 2024-09-16T23:48:48.348 INFO:tasks.workunit.client.1.smithi043.stdout:5/393: creat d3/d9/d22/d4a/f7f x:0 0 0 2024-09-16T23:48:48.352 INFO:tasks.workunit.client.1.smithi043.stdout:7/256: write d4/d27/f3f [1562823,17585] 0 2024-09-16T23:48:48.358 INFO:tasks.workunit.client.1.smithi043.stdout:7/257: mknod d4/db/d11/d21/d36/c53 0 2024-09-16T23:48:48.361 INFO:tasks.workunit.client.1.smithi043.stdout:5/394: mkdir d3/d9/d6f/d7c/d80 0 2024-09-16T23:48:48.362 INFO:tasks.workunit.client.1.smithi043.stdout:7/258: symlink d4/db/d11/l54 0 2024-09-16T23:48:48.363 INFO:tasks.workunit.client.1.smithi043.stdout:1/355: dwrite d4/d9/d1c/d1e/f23 [4194304,4194304] 0 2024-09-16T23:48:48.363 INFO:tasks.workunit.client.1.smithi043.stdout:5/395: dread - d3/db/d19/d3e/f70 zero size 2024-09-16T23:48:48.365 INFO:tasks.workunit.client.1.smithi043.stdout:5/396: readlink d3/d1b/l6e 0 2024-09-16T23:48:48.366 INFO:tasks.workunit.client.1.smithi043.stdout:5/397: write d3/db/d26/f33 [1010364,121683] 0 2024-09-16T23:48:48.368 INFO:tasks.workunit.client.1.smithi043.stdout:1/356: truncate d4/d9/f4e 1657826 0 2024-09-16T23:48:48.372 INFO:tasks.workunit.client.1.smithi043.stdout:5/398: rmdir d3/db/d19/d3e/d28 39 2024-09-16T23:48:48.375 INFO:tasks.workunit.client.1.smithi043.stdout:1/357: mkdir d4/d9/d1c/d1e/d42/d5d/d71/d76 0 2024-09-16T23:48:48.380 INFO:tasks.workunit.client.1.smithi043.stdout:1/358: dread d4/d9/f4e [0,4194304] 0 2024-09-16T23:48:48.381 INFO:tasks.workunit.client.1.smithi043.stdout:7/259: dwrite d4/db/d11/f13 [0,4194304] 0 2024-09-16T23:48:48.387 INFO:tasks.workunit.client.1.smithi043.stdout:5/399: dwrite d3/db/d19/d3e/f70 [0,4194304] 0 2024-09-16T23:48:48.389 INFO:tasks.workunit.client.1.smithi043.stdout:1/359: write d4/df/d13/d1b/f2f [2913264,77656] 0 2024-09-16T23:48:48.391 INFO:tasks.workunit.client.1.smithi043.stdout:7/260: dread - d4/d27/d41/d1b/f44 zero size 2024-09-16T23:48:48.395 INFO:tasks.workunit.client.1.smithi043.stdout:5/400: readlink d3/d9/d6f/l74 0 2024-09-16T23:48:48.400 INFO:tasks.workunit.client.1.smithi043.stdout:1/360: symlink d4/df/d13/d1b/l77 0 2024-09-16T23:48:48.401 INFO:tasks.workunit.client.1.smithi043.stdout:5/401: mknod d3/db/d19/c81 0 2024-09-16T23:48:48.401 INFO:tasks.workunit.client.1.smithi043.stdout:7/261: creat d4/db/d11/d21/d31/d33/f55 x:0 0 0 2024-09-16T23:48:48.404 INFO:tasks.workunit.client.1.smithi043.stdout:1/361: write d4/d9/f4e [383995,115484] 0 2024-09-16T23:48:48.404 INFO:tasks.workunit.client.1.smithi043.stdout:7/262: symlink d4/db/d28/l56 0 2024-09-16T23:48:48.408 INFO:tasks.workunit.client.1.smithi043.stdout:7/263: creat d4/db/d11/d21/d36/f57 x:0 0 0 2024-09-16T23:48:48.410 INFO:tasks.workunit.client.1.smithi043.stdout:1/362: truncate d4/d9/d12/d2b/d3d/f61 309132 0 2024-09-16T23:48:48.410 INFO:tasks.workunit.client.1.smithi043.stdout:5/402: dread d3/db/d26/f33 [0,4194304] 0 2024-09-16T23:48:48.412 INFO:tasks.workunit.client.1.smithi043.stdout:5/403: readlink d3/db/d15/d43/l4f 0 2024-09-16T23:48:48.418 INFO:tasks.workunit.client.1.smithi043.stdout:7/264: rename d4/db/d11/d21/d31/c48 to d4/db/d11/d21/d31/d33/c58 0 2024-09-16T23:48:48.418 INFO:tasks.workunit.client.1.smithi043.stdout:1/363: dwrite d4/f62 [4194304,4194304] 0 2024-09-16T23:48:48.420 INFO:tasks.workunit.client.1.smithi043.stdout:7/265: read d4/db/f18 [1378186,114152] 0 2024-09-16T23:48:48.422 INFO:tasks.workunit.client.1.smithi043.stdout:5/404: getdents d3/d9/d22/d4a/d4c 0 2024-09-16T23:48:48.424 INFO:tasks.workunit.client.1.smithi043.stdout:1/364: symlink d4/d9/d12/l78 0 2024-09-16T23:48:48.426 INFO:tasks.workunit.client.1.smithi043.stdout:7/266: mknod d4/db/c59 0 2024-09-16T23:48:48.427 INFO:tasks.workunit.client.1.smithi043.stdout:1/365: read d4/df/d13/d1b/f2f [3370379,70878] 0 2024-09-16T23:48:48.431 INFO:tasks.workunit.client.1.smithi043.stdout:5/405: truncate d3/d9/f56 4332172 0 2024-09-16T23:48:48.431 INFO:tasks.workunit.client.1.smithi043.stdout:7/267: rename d4/db/d28/c49 to d4/db/d11/d15/d29/c5a 0 2024-09-16T23:48:48.432 INFO:tasks.workunit.client.1.smithi043.stdout:7/268: dread - d4/d27/d41/d1b/f4e zero size 2024-09-16T23:48:48.434 INFO:tasks.workunit.client.1.smithi043.stdout:7/269: fsync d4/db/f14 0 2024-09-16T23:48:48.435 INFO:tasks.workunit.client.1.smithi043.stdout:7/270: chown d4/f2b 120480 1 2024-09-16T23:48:48.441 INFO:tasks.workunit.client.1.smithi043.stdout:5/406: dwrite d3/d9/d22/f4e [0,4194304] 0 2024-09-16T23:48:48.441 INFO:tasks.workunit.client.1.smithi043.stdout:5/407: dread - d3/db/d15/f75 zero size 2024-09-16T23:48:48.446 INFO:tasks.workunit.client.1.smithi043.stdout:5/408: unlink d3/db/d15/d43/l4f 0 2024-09-16T23:48:48.446 INFO:tasks.workunit.client.1.smithi043.stdout:3/193: sync 2024-09-16T23:48:48.447 INFO:tasks.workunit.client.1.smithi043.stdout:7/271: dwrite d4/db/d11/d21/f2f [0,4194304] 0 2024-09-16T23:48:48.451 INFO:tasks.workunit.client.1.smithi043.stdout:7/272: write d4/d27/f3f [949806,57287] 0 2024-09-16T23:48:48.452 INFO:tasks.workunit.client.1.smithi043.stdout:1/366: write d4/fc [6706271,90866] 0 2024-09-16T23:48:48.453 INFO:tasks.workunit.client.1.smithi043.stdout:3/194: fdatasync d4/f10 0 2024-09-16T23:48:48.454 INFO:tasks.workunit.client.1.smithi043.stdout:5/409: chown d3/db/d15/d43/c67 1 1 2024-09-16T23:48:48.456 INFO:tasks.workunit.client.1.smithi043.stdout:5/410: dread - d3/d9/d22/d4a/f7f zero size 2024-09-16T23:48:48.456 INFO:tasks.workunit.client.1.smithi043.stdout:5/411: chown d3/db/d19/c81 458 1 2024-09-16T23:48:48.460 INFO:tasks.workunit.client.1.smithi043.stdout:1/367: mkdir d4/df/d26/d79 0 2024-09-16T23:48:48.464 INFO:tasks.workunit.client.0.smithi042.stdout:1/117: creat d0/d9/f24 x:0 0 0 2024-09-16T23:48:48.468 INFO:tasks.workunit.client.1.smithi043.stdout:3/195: dwrite d4/de/f21 [0,4194304] 0 2024-09-16T23:48:48.469 INFO:tasks.workunit.client.1.smithi043.stdout:1/368: mkdir d4/d9/d1c/d1e/d42/d5d/d7a 0 2024-09-16T23:48:48.470 INFO:tasks.workunit.client.0.smithi042.stdout:3/28: dwrite f1 [0,4194304] 0 2024-09-16T23:48:48.470 INFO:tasks.workunit.client.1.smithi043.stdout:7/273: truncate d4/d27/d41/d1b/f25 927910 0 2024-09-16T23:48:48.471 INFO:tasks.workunit.client.0.smithi042.stdout:3/29: fdatasync d2/d7/f8 0 2024-09-16T23:48:48.473 INFO:tasks.workunit.client.1.smithi043.stdout:1/369: write d4/d9/d1c/d1e/f66 [360069,67313] 0 2024-09-16T23:48:48.475 INFO:tasks.workunit.client.0.smithi042.stdout:7/42: unlink d3/c5 0 2024-09-16T23:48:48.479 INFO:tasks.workunit.client.0.smithi042.stdout:9/56: mkdir d4/d10/d14 0 2024-09-16T23:48:48.480 INFO:tasks.workunit.client.0.smithi042.stdout:7/43: dread d3/fc [0,4194304] 0 2024-09-16T23:48:48.480 INFO:tasks.workunit.client.1.smithi043.stdout:7/274: creat d4/db/d11/d21/d31/d33/f5b x:0 0 0 2024-09-16T23:48:48.481 INFO:tasks.workunit.client.1.smithi043.stdout:3/196: dwrite d4/f10 [0,4194304] 0 2024-09-16T23:48:48.482 INFO:tasks.workunit.client.1.smithi043.stdout:5/412: dread d3/f38 [4194304,4194304] 0 2024-09-16T23:48:48.487 INFO:tasks.workunit.client.1.smithi043.stdout:1/370: write d4/df/d13/d1b/f64 [2683296,130845] 0 2024-09-16T23:48:48.487 INFO:tasks.workunit.client.0.smithi042.stdout:8/131: read da/f18 [2693017,55589] 0 2024-09-16T23:48:48.487 INFO:tasks.workunit.client.1.smithi043.stdout:7/275: symlink d4/d27/d41/d1b/l5c 0 2024-09-16T23:48:48.489 INFO:tasks.workunit.client.1.smithi043.stdout:1/371: dread - d4/d9/d72/f73 zero size 2024-09-16T23:48:48.489 INFO:tasks.workunit.client.1.smithi043.stdout:5/413: unlink d3/db/d19/d3e/d28/c7e 0 2024-09-16T23:48:48.491 INFO:tasks.workunit.client.1.smithi043.stdout:3/197: mkdir d4/de/d1f/d34 0 2024-09-16T23:48:48.492 INFO:tasks.workunit.client.0.smithi042.stdout:6/65: fsync d8/d9/f13 0 2024-09-16T23:48:48.494 INFO:tasks.workunit.client.0.smithi042.stdout:6/66: truncate d8/db/dd/f11 4857230 0 2024-09-16T23:48:48.496 INFO:tasks.workunit.client.1.smithi043.stdout:1/372: rename d4/d9/d12/f33 to d4/d34/f7b 0 2024-09-16T23:48:48.497 INFO:tasks.workunit.client.1.smithi043.stdout:5/414: fdatasync d3/f38 0 2024-09-16T23:48:48.500 INFO:tasks.workunit.client.0.smithi042.stdout:6/67: dwrite f5 [0,4194304] 0 2024-09-16T23:48:48.500 INFO:tasks.workunit.client.1.smithi043.stdout:7/276: link d4/db/d11/d21/d31/d33/c1a d4/db/c5d 0 2024-09-16T23:48:48.501 INFO:tasks.workunit.client.0.smithi042.stdout:5/72: rename d9/db/ff to d9/db/d11/f13 0 2024-09-16T23:48:48.501 INFO:tasks.workunit.client.0.smithi042.stdout:5/73: readlink l8 0 2024-09-16T23:48:48.502 INFO:tasks.workunit.client.1.smithi043.stdout:5/415: rmdir d3/db/d15/d43 39 2024-09-16T23:48:48.502 INFO:tasks.workunit.client.1.smithi043.stdout:5/416: chown d3/f17 486 1 2024-09-16T23:48:48.506 INFO:tasks.workunit.client.1.smithi043.stdout:3/198: rename d4/d17 to d4/de/d35 0 2024-09-16T23:48:48.509 INFO:tasks.workunit.client.0.smithi042.stdout:3/30: rmdir d2 39 2024-09-16T23:48:48.510 INFO:tasks.workunit.client.1.smithi043.stdout:5/417: dread d3/d9/d22/f4e [0,4194304] 0 2024-09-16T23:48:48.513 INFO:tasks.workunit.client.0.smithi042.stdout:7/44: creat d3/fe x:0 0 0 2024-09-16T23:48:48.513 INFO:tasks.workunit.client.1.smithi043.stdout:3/199: mknod d4/de/c36 0 2024-09-16T23:48:48.515 INFO:tasks.workunit.client.0.smithi042.stdout:4/54: creat de/d12/f14 x:0 0 0 2024-09-16T23:48:48.515 INFO:tasks.workunit.client.0.smithi042.stdout:4/55: stat c9 0 2024-09-16T23:48:48.516 INFO:tasks.workunit.client.1.smithi043.stdout:7/277: getdents d4/db/d50 0 2024-09-16T23:48:48.516 INFO:tasks.workunit.client.1.smithi043.stdout:5/418: dread - d3/f50 zero size 2024-09-16T23:48:48.518 INFO:tasks.workunit.client.1.smithi043.stdout:3/200: creat d4/de/d1f/d2c/f37 x:0 0 0 2024-09-16T23:48:48.519 INFO:tasks.workunit.client.1.smithi043.stdout:5/419: stat d3/db/d26/l57 0 2024-09-16T23:48:48.519 INFO:tasks.workunit.client.0.smithi042.stdout:4/56: dread f7 [0,4194304] 0 2024-09-16T23:48:48.519 INFO:tasks.workunit.client.1.smithi043.stdout:7/278: creat d4/db/f5e x:0 0 0 2024-09-16T23:48:48.522 INFO:tasks.workunit.client.1.smithi043.stdout:1/373: dwrite d4/f69 [0,4194304] 0 2024-09-16T23:48:48.523 INFO:tasks.workunit.client.0.smithi042.stdout:6/68: creat d8/db/dd/f15 x:0 0 0 2024-09-16T23:48:48.524 INFO:tasks.workunit.client.1.smithi043.stdout:3/201: fsync d4/de/f24 0 2024-09-16T23:48:48.525 INFO:tasks.workunit.client.0.smithi042.stdout:6/69: dread - d8/db/f12 zero size 2024-09-16T23:48:48.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/57: dwrite de/f10 [0,4194304] 0 2024-09-16T23:48:48.526 INFO:tasks.workunit.client.1.smithi043.stdout:7/279: creat d4/db/d11/d15/f5f x:0 0 0 2024-09-16T23:48:48.528 INFO:tasks.workunit.client.1.smithi043.stdout:1/374: rename d4/df/d26/c44 to d4/df/d26/c7c 0 2024-09-16T23:48:48.529 INFO:tasks.workunit.client.1.smithi043.stdout:7/280: read d4/db/d28/f2d [265721,110902] 0 2024-09-16T23:48:48.537 INFO:tasks.workunit.client.1.smithi043.stdout:5/420: write d3/d9/d22/f61 [662263,5391] 0 2024-09-16T23:48:48.537 INFO:tasks.workunit.client.1.smithi043.stdout:3/202: getdents d4/d30 0 2024-09-16T23:48:48.540 INFO:tasks.workunit.client.1.smithi043.stdout:9/199: sync 2024-09-16T23:48:48.540 INFO:tasks.workunit.client.1.smithi043.stdout:7/281: creat d4/f60 x:0 0 0 2024-09-16T23:48:48.544 INFO:tasks.workunit.client.1.smithi043.stdout:5/421: creat d3/d9/d22/d60/f82 x:0 0 0 2024-09-16T23:48:48.544 INFO:tasks.workunit.client.1.smithi043.stdout:3/203: mknod d4/c38 0 2024-09-16T23:48:48.545 INFO:tasks.workunit.client.1.smithi043.stdout:1/375: dread d4/d9/d1c/d1e/f23 [0,4194304] 0 2024-09-16T23:48:48.546 INFO:tasks.workunit.client.1.smithi043.stdout:9/200: truncate d4/ff 4581368 0 2024-09-16T23:48:48.546 INFO:tasks.workunit.client.1.smithi043.stdout:7/282: link d4/f2b d4/db/d28/f61 0 2024-09-16T23:48:48.547 INFO:tasks.workunit.client.0.smithi042.stdout:8/132: mkdir da/d19/d2c 0 2024-09-16T23:48:48.548 INFO:tasks.workunit.client.1.smithi043.stdout:9/201: dread d4/f24 [0,4194304] 0 2024-09-16T23:48:48.548 INFO:tasks.workunit.client.1.smithi043.stdout:3/204: symlink d4/d30/l39 0 2024-09-16T23:48:48.549 INFO:tasks.workunit.client.1.smithi043.stdout:5/422: rmdir d3/d9/d22/d4a/d4c 39 2024-09-16T23:48:48.550 INFO:tasks.workunit.client.0.smithi042.stdout:6/70: mknod d8/db/dd/c16 0 2024-09-16T23:48:48.552 INFO:tasks.workunit.client.0.smithi042.stdout:4/58: creat de/d12/f15 x:0 0 0 2024-09-16T23:48:48.554 INFO:tasks.workunit.client.1.smithi043.stdout:9/202: creat d4/d5/d30/f3c x:0 0 0 2024-09-16T23:48:48.555 INFO:tasks.workunit.client.1.smithi043.stdout:5/423: dread d3/db/d15/f69 [0,4194304] 0 2024-09-16T23:48:48.556 INFO:tasks.workunit.client.0.smithi042.stdout:9/57: rename f2 to d4/d10/f15 0 2024-09-16T23:48:48.557 INFO:tasks.workunit.client.1.smithi043.stdout:7/283: rename d4/db/d11/d21/d31/d33/f1f to d4/f62 0 2024-09-16T23:48:48.558 INFO:tasks.workunit.client.1.smithi043.stdout:9/203: mknod d4/d5/c3d 0 2024-09-16T23:48:48.559 INFO:tasks.workunit.client.0.smithi042.stdout:2/68: link d0/da/ce d0/c18 0 2024-09-16T23:48:48.559 INFO:tasks.workunit.client.0.smithi042.stdout:4/59: dwrite fa [0,4194304] 0 2024-09-16T23:48:48.562 INFO:tasks.workunit.client.1.smithi043.stdout:3/205: rename d4/de/d1f/d2c/f37 to d4/de/d1f/d34/f3a 0 2024-09-16T23:48:48.563 INFO:tasks.workunit.client.0.smithi042.stdout:3/31: mkdir d2/d9 0 2024-09-16T23:48:48.565 INFO:tasks.workunit.client.1.smithi043.stdout:9/204: symlink d4/d1c/d25/d2b/l3e 0 2024-09-16T23:48:48.566 INFO:tasks.workunit.client.0.smithi042.stdout:0/34: getdents d0 0 2024-09-16T23:48:48.566 INFO:tasks.workunit.client.1.smithi043.stdout:5/424: symlink d3/l83 0 2024-09-16T23:48:48.569 INFO:tasks.workunit.client.1.smithi043.stdout:5/425: write d3/db/d15/f5f [4154964,100318] 0 2024-09-16T23:48:48.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/32: dwrite d2/d7/f8 [0,4194304] 0 2024-09-16T23:48:48.572 INFO:tasks.workunit.client.0.smithi042.stdout:6/71: write d8/db/dd/f11 [990967,77167] 0 2024-09-16T23:48:48.576 INFO:tasks.workunit.client.1.smithi043.stdout:9/205: symlink d4/d21/d2a/l3f 0 2024-09-16T23:48:48.578 INFO:tasks.workunit.client.1.smithi043.stdout:7/284: dread d4/db/d11/d15/f47 [0,4194304] 0 2024-09-16T23:48:48.580 INFO:tasks.workunit.client.0.smithi042.stdout:6/72: dwrite d8/d9/fa [0,4194304] 0 2024-09-16T23:48:48.580 INFO:tasks.workunit.client.1.smithi043.stdout:3/206: rename d4/de/f2f to d4/f3b 0 2024-09-16T23:48:48.580 INFO:tasks.workunit.client.1.smithi043.stdout:5/426: dwrite d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:48.583 INFO:tasks.workunit.client.1.smithi043.stdout:5/427: fdatasync d3/f17 0 2024-09-16T23:48:48.587 INFO:tasks.workunit.client.0.smithi042.stdout:4/60: symlink de/l16 0 2024-09-16T23:48:48.595 INFO:tasks.workunit.client.0.smithi042.stdout:2/69: rename d0/da/lb to d0/da/d17/l19 0 2024-09-16T23:48:48.595 INFO:tasks.workunit.client.1.smithi043.stdout:1/376: dwrite d4/df/d13/d1b/f30 [0,4194304] 0 2024-09-16T23:48:48.596 INFO:tasks.workunit.client.0.smithi042.stdout:0/35: creat d0/f9 x:0 0 0 2024-09-16T23:48:48.596 INFO:tasks.workunit.client.0.smithi042.stdout:0/36: write d0/f4 [128451,18999] 0 2024-09-16T23:48:48.598 INFO:tasks.workunit.client.0.smithi042.stdout:8/133: creat da/d19/d2c/f2d x:0 0 0 2024-09-16T23:48:48.602 INFO:tasks.workunit.client.1.smithi043.stdout:7/285: dwrite d4/f60 [0,4194304] 0 2024-09-16T23:48:48.605 INFO:tasks.workunit.client.0.smithi042.stdout:6/73: readlink l4 0 2024-09-16T23:48:48.608 INFO:tasks.workunit.client.0.smithi042.stdout:4/61: symlink de/d12/l17 0 2024-09-16T23:48:48.612 INFO:tasks.workunit.client.0.smithi042.stdout:2/70: creat d0/f1a x:0 0 0 2024-09-16T23:48:48.613 INFO:tasks.workunit.client.1.smithi043.stdout:3/207: dread d4/de/f24 [0,4194304] 0 2024-09-16T23:48:48.616 INFO:tasks.workunit.client.1.smithi043.stdout:7/286: mknod d4/d27/d41/c63 0 2024-09-16T23:48:48.620 INFO:tasks.workunit.client.0.smithi042.stdout:8/134: chown da/c26 92758121 1 2024-09-16T23:48:48.622 INFO:tasks.workunit.client.0.smithi042.stdout:8/135: stat da/d19/d2c 0 2024-09-16T23:48:48.622 INFO:tasks.workunit.client.1.smithi043.stdout:7/287: mkdir d4/db/d11/d21/d31/d33/d64 0 2024-09-16T23:48:48.622 INFO:tasks.workunit.client.1.smithi043.stdout:7/288: chown d4/d27 3634 1 2024-09-16T23:48:48.623 INFO:tasks.workunit.client.1.smithi043.stdout:9/206: truncate d4/d1c/f20 931111 0 2024-09-16T23:48:48.624 INFO:tasks.workunit.client.0.smithi042.stdout:3/33: link d2/f4 d2/fa 0 2024-09-16T23:48:48.624 INFO:tasks.workunit.client.0.smithi042.stdout:6/74: symlink d8/db/dd/l17 0 2024-09-16T23:48:48.624 INFO:tasks.workunit.client.0.smithi042.stdout:3/34: write f0 [317110,68150] 0 2024-09-16T23:48:48.625 INFO:tasks.workunit.client.1.smithi043.stdout:5/428: truncate d3/db/d19/f54 743917 0 2024-09-16T23:48:48.625 INFO:tasks.workunit.client.1.smithi043.stdout:7/289: fdatasync d4/db/d11/d21/d36/f57 0 2024-09-16T23:48:48.626 INFO:tasks.workunit.client.0.smithi042.stdout:4/62: mknod de/c18 0 2024-09-16T23:48:48.628 INFO:tasks.workunit.client.0.smithi042.stdout:2/71: mkdir d0/da/d1b 0 2024-09-16T23:48:48.630 INFO:tasks.workunit.client.1.smithi043.stdout:3/208: dwrite d4/de/d1f/d34/f3a [0,4194304] 0 2024-09-16T23:48:48.631 INFO:tasks.workunit.client.1.smithi043.stdout:5/429: mkdir d3/db/d84 0 2024-09-16T23:48:48.632 INFO:tasks.workunit.client.0.smithi042.stdout:8/136: dwrite da/d19/d1d/f22 [0,4194304] 0 2024-09-16T23:48:48.632 INFO:tasks.workunit.client.1.smithi043.stdout:7/290: truncate d4/d27/d41/f40 885219 0 2024-09-16T23:48:48.634 INFO:tasks.workunit.client.1.smithi043.stdout:1/377: truncate d4/d34/f41 212918 0 2024-09-16T23:48:48.634 INFO:tasks.workunit.client.0.smithi042.stdout:4/63: unlink l8 0 2024-09-16T23:48:48.635 INFO:tasks.workunit.client.0.smithi042.stdout:8/137: chown da/d19/d1d/c20 1498 1 2024-09-16T23:48:48.636 INFO:tasks.workunit.client.0.smithi042.stdout:3/35: dwrite f1 [0,4194304] 0 2024-09-16T23:48:48.637 INFO:tasks.workunit.client.0.smithi042.stdout:3/36: readlink d2/l5 0 2024-09-16T23:48:48.639 INFO:tasks.workunit.client.0.smithi042.stdout:3/37: write f1 [4950428,107039] 0 2024-09-16T23:48:48.639 INFO:tasks.workunit.client.1.smithi043.stdout:1/378: write d4/d9/f4e [2655246,71879] 0 2024-09-16T23:48:48.639 INFO:tasks.workunit.client.1.smithi043.stdout:9/207: dwrite d4/f31 [0,4194304] 0 2024-09-16T23:48:48.640 INFO:tasks.workunit.client.0.smithi042.stdout:3/38: fdatasync d2/d7/f8 0 2024-09-16T23:48:48.642 INFO:tasks.workunit.client.1.smithi043.stdout:9/208: truncate d4/d5/d30/f3c 49134 0 2024-09-16T23:48:48.643 INFO:tasks.workunit.client.0.smithi042.stdout:8/138: creat da/d19/d1d/f2e x:0 0 0 2024-09-16T23:48:48.643 INFO:tasks.workunit.client.0.smithi042.stdout:8/139: truncate da/f16 553869 0 2024-09-16T23:48:48.644 INFO:tasks.workunit.client.1.smithi043.stdout:9/209: fsync d4/d5/d16/f1f 0 2024-09-16T23:48:48.644 INFO:tasks.workunit.client.0.smithi042.stdout:8/140: fsync da/d19/d1d/f2b 0 2024-09-16T23:48:48.648 INFO:tasks.workunit.client.0.smithi042.stdout:2/72: stat d0/c18 0 2024-09-16T23:48:48.649 INFO:tasks.workunit.client.1.smithi043.stdout:3/209: rename d4/c38 to d4/de/d1f/d34/c3c 0 2024-09-16T23:48:48.649 INFO:tasks.workunit.client.1.smithi043.stdout:9/210: write d4/f31 [2481967,34095] 0 2024-09-16T23:48:48.650 INFO:tasks.workunit.client.1.smithi043.stdout:3/210: chown d4/f19 116206612 1 2024-09-16T23:48:48.652 INFO:tasks.workunit.client.0.smithi042.stdout:4/64: link f7 de/f19 0 2024-09-16T23:48:48.655 INFO:tasks.workunit.client.0.smithi042.stdout:8/141: dwrite da/d19/f1a [0,4194304] 0 2024-09-16T23:48:48.656 INFO:tasks.workunit.client.1.smithi043.stdout:7/291: mkdir d4/db/d11/d21/d31/d65 0 2024-09-16T23:48:48.657 INFO:tasks.workunit.client.0.smithi042.stdout:3/39: write d2/f4 [805043,37066] 0 2024-09-16T23:48:48.658 INFO:tasks.workunit.client.0.smithi042.stdout:3/40: write d2/f4 [53116,38743] 0 2024-09-16T23:48:48.659 INFO:tasks.workunit.client.0.smithi042.stdout:3/41: chown d2/l5 215 1 2024-09-16T23:48:48.660 INFO:tasks.workunit.client.1.smithi043.stdout:1/379: symlink d4/d9/d12/d2b/l7d 0 2024-09-16T23:48:48.660 INFO:tasks.workunit.client.1.smithi043.stdout:3/211: dwrite d4/de/f21 [4194304,4194304] 0 2024-09-16T23:48:48.661 INFO:tasks.workunit.client.0.smithi042.stdout:2/73: creat d0/da/d17/f1c x:0 0 0 2024-09-16T23:48:48.662 INFO:tasks.workunit.client.0.smithi042.stdout:2/74: write d0/ff [736891,110722] 0 2024-09-16T23:48:48.663 INFO:tasks.workunit.client.1.smithi043.stdout:2/183: sync 2024-09-16T23:48:48.663 INFO:tasks.workunit.client.1.smithi043.stdout:5/430: rename d3/db/d19/d76/d78 to d3/d9/d6f/d85 0 2024-09-16T23:48:48.666 INFO:tasks.workunit.client.1.smithi043.stdout:9/211: read d4/ff [3081591,72097] 0 2024-09-16T23:48:48.667 INFO:tasks.workunit.client.1.smithi043.stdout:7/292: mkdir d4/d27/d41/d1b/d66 0 2024-09-16T23:48:48.668 INFO:tasks.workunit.client.0.smithi042.stdout:4/65: fdatasync f2 0 2024-09-16T23:48:48.670 INFO:tasks.workunit.client.1.smithi043.stdout:7/293: readlink d4/db/d11/d15/l3c 0 2024-09-16T23:48:48.670 INFO:tasks.workunit.client.1.smithi043.stdout:2/184: creat d8/d1d/d25/f44 x:0 0 0 2024-09-16T23:48:48.675 INFO:tasks.workunit.client.1.smithi043.stdout:3/212: symlink d4/de/d1f/d2c/l3d 0 2024-09-16T23:48:48.677 INFO:tasks.workunit.client.1.smithi043.stdout:1/380: dread d4/d9/d1c/d1e/f66 [0,4194304] 0 2024-09-16T23:48:48.677 INFO:tasks.workunit.client.1.smithi043.stdout:9/212: symlink d4/d28/d2e/l40 0 2024-09-16T23:48:48.678 INFO:tasks.workunit.client.1.smithi043.stdout:2/185: write d8/d1d/d20/d26/f3c [265759,40263] 0 2024-09-16T23:48:48.678 INFO:tasks.workunit.client.1.smithi043.stdout:7/294: rmdir d4/db/d50 39 2024-09-16T23:48:48.679 INFO:tasks.workunit.client.0.smithi042.stdout:8/142: mkdir da/d2f 0 2024-09-16T23:48:48.679 INFO:tasks.workunit.client.0.smithi042.stdout:4/66: dwrite de/d12/f14 [0,4194304] 0 2024-09-16T23:48:48.680 INFO:tasks.workunit.client.0.smithi042.stdout:4/67: fdatasync fa 0 2024-09-16T23:48:48.685 INFO:tasks.workunit.client.1.smithi043.stdout:2/186: unlink f6 0 2024-09-16T23:48:48.685 INFO:tasks.workunit.client.1.smithi043.stdout:9/213: mkdir d4/d1c/d41 0 2024-09-16T23:48:48.688 INFO:tasks.workunit.client.0.smithi042.stdout:8/143: fsync da/f17 0 2024-09-16T23:48:48.689 INFO:tasks.workunit.client.0.smithi042.stdout:4/68: dwrite f2 [0,4194304] 0 2024-09-16T23:48:48.689 INFO:tasks.workunit.client.0.smithi042.stdout:8/144: stat da/f2a 0 2024-09-16T23:48:48.691 INFO:tasks.workunit.client.1.smithi043.stdout:2/187: mkdir d8/d1d/d20/d23/d45 0 2024-09-16T23:48:48.692 INFO:tasks.workunit.client.1.smithi043.stdout:1/381: dwrite d4/f62 [4194304,4194304] 0 2024-09-16T23:48:48.692 INFO:tasks.workunit.client.1.smithi043.stdout:7/295: dread d4/db/d11/d21/f2f [0,4194304] 0 2024-09-16T23:48:48.694 INFO:tasks.workunit.client.1.smithi043.stdout:2/188: read d8/d1f/f30 [649463,23993] 0 2024-09-16T23:48:48.695 INFO:tasks.workunit.client.1.smithi043.stdout:3/213: getdents d4 0 2024-09-16T23:48:48.696 INFO:tasks.workunit.client.0.smithi042.stdout:4/69: dread de/d12/f14 [0,4194304] 0 2024-09-16T23:48:48.698 INFO:tasks.workunit.client.1.smithi043.stdout:1/382: fsync d4/df/d13/d1b/f64 0 2024-09-16T23:48:48.701 INFO:tasks.workunit.client.0.smithi042.stdout:8/145: dwrite da/f2a [0,4194304] 0 2024-09-16T23:48:48.701 INFO:tasks.workunit.client.1.smithi043.stdout:9/214: read d4/f1a [97215,42704] 0 2024-09-16T23:48:48.705 INFO:tasks.workunit.client.1.smithi043.stdout:3/214: creat d4/de/d1f/f3e x:0 0 0 2024-09-16T23:48:48.705 INFO:tasks.workunit.client.1.smithi043.stdout:2/189: dwrite d8/d1f/f30 [0,4194304] 0 2024-09-16T23:48:48.707 INFO:tasks.workunit.client.1.smithi043.stdout:7/296: mknod d4/db/c67 0 2024-09-16T23:48:48.709 INFO:tasks.workunit.client.0.smithi042.stdout:8/146: dread da/f17 [0,4194304] 0 2024-09-16T23:48:48.712 INFO:tasks.workunit.client.0.smithi042.stdout:8/147: mkdir da/d19/d1d/d30 0 2024-09-16T23:48:48.718 INFO:tasks.workunit.client.1.smithi043.stdout:1/383: mknod d4/df/d13/c7e 0 2024-09-16T23:48:48.719 INFO:tasks.workunit.client.0.smithi042.stdout:8/148: rename da/d19/d1d/f22 to da/d19/d1d/d30/f31 0 2024-09-16T23:48:48.722 INFO:tasks.workunit.client.0.smithi042.stdout:8/149: creat da/d19/d1d/f32 x:0 0 0 2024-09-16T23:48:48.722 INFO:tasks.workunit.client.1.smithi043.stdout:1/384: write d4/df/d13/d1b/f59 [975711,101466] 0 2024-09-16T23:48:48.723 INFO:tasks.workunit.client.0.smithi042.stdout:8/150: fdatasync da/d19/d1d/f2e 0 2024-09-16T23:48:48.725 INFO:tasks.workunit.client.0.smithi042.stdout:8/151: creat da/d19/d1d/f33 x:0 0 0 2024-09-16T23:48:48.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.730+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 22) v1 ==== 46250+0+0 (secure 0 0 0) 0x7ff6dc0743a0 con 0x7ff6ec0ffc10 2024-09-16T23:48:48.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.731+0000 7ff6f09bf700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7ff6d4005d60 0x7ff6d4034ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:48.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.731+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6d4036ac0 con 0x7ff6d4005d60 2024-09-16T23:48:48.729 INFO:tasks.workunit.client.1.smithi043.stdout:1/385: dwrite d4/d9/d1c/f37 [0,4194304] 0 2024-09-16T23:48:48.729 INFO:tasks.workunit.client.1.smithi043.stdout:0/282: sync 2024-09-16T23:48:48.729 INFO:tasks.workunit.client.1.smithi043.stdout:4/280: sync 2024-09-16T23:48:48.729 INFO:tasks.workunit.client.1.smithi043.stdout:8/274: sync 2024-09-16T23:48:48.729 INFO:tasks.workunit.client.1.smithi043.stdout:6/296: sync 2024-09-16T23:48:48.730 INFO:tasks.workunit.client.0.smithi042.stdout:8/152: rename c8 to da/d19/d1d/c34 0 2024-09-16T23:48:48.731 INFO:tasks.workunit.client.0.smithi042.stdout:8/153: write da/d19/f23 [16737,42403] 0 2024-09-16T23:48:48.732 INFO:tasks.workunit.client.0.smithi042.stdout:8/154: stat da/f17 0 2024-09-16T23:48:48.733 INFO:tasks.workunit.client.0.smithi042.stdout:8/155: stat da/d19/d2c/f2d 0 2024-09-16T23:48:48.733 INFO:tasks.workunit.client.1.smithi043.stdout:8/275: fdatasync de/d18/f36 0 2024-09-16T23:48:48.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.737+0000 7ff6ea7fc700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7ff6d4005d60 0x7ff6d4034ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:48.735 INFO:tasks.workunit.client.1.smithi043.stdout:8/276: write de/f54 [744978,56844] 0 2024-09-16T23:48:48.736 INFO:tasks.workunit.client.0.smithi042.stdout:8/156: dread da/f16 [0,4194304] 0 2024-09-16T23:48:48.736 INFO:tasks.workunit.client.1.smithi043.stdout:8/277: chown de/d18/d1f/d25/l49 309039012 1 2024-09-16T23:48:48.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.739+0000 7ff6ea7fc700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7ff6d4005d60 0x7ff6d4034ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00092a0 tx=0x7ff6e0017000 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:48.738 INFO:tasks.workunit.client.1.smithi043.stdout:8/278: dread de/d18/d1f/d2f/f43 [0,4194304] 0 2024-09-16T23:48:48.747 INFO:tasks.workunit.client.0.smithi042.stdout:8/157: mknod da/d19/c35 0 2024-09-16T23:48:48.749 INFO:tasks.workunit.client.0.smithi042.stdout:8/158: creat da/d19/d2c/f36 x:0 0 0 2024-09-16T23:48:48.753 INFO:tasks.workunit.client.0.smithi042.stdout:8/159: dread da/d19/f1f [0,4194304] 0 2024-09-16T23:48:48.754 INFO:tasks.workunit.client.0.smithi042.stdout:1/118: write d0/d9/f24 [337411,34777] 0 2024-09-16T23:48:48.755 INFO:tasks.workunit.client.0.smithi042.stdout:8/160: dread da/d19/f23 [0,4194304] 0 2024-09-16T23:48:48.758 INFO:tasks.workunit.client.0.smithi042.stdout:8/161: dread da/f18 [0,4194304] 0 2024-09-16T23:48:48.759 INFO:tasks.workunit.client.0.smithi042.stdout:8/162: rename da to da/d19/d1d/d37 22 2024-09-16T23:48:48.760 INFO:tasks.workunit.client.0.smithi042.stdout:8/163: fdatasync da/d19/d1d/f32 0 2024-09-16T23:48:48.761 INFO:tasks.workunit.client.0.smithi042.stdout:1/119: dwrite d0/d9/df/f13 [0,4194304] 0 2024-09-16T23:48:48.762 INFO:tasks.workunit.client.0.smithi042.stdout:1/120: dread - d0/d17/f20 zero size 2024-09-16T23:48:48.762 INFO:tasks.workunit.client.0.smithi042.stdout:8/164: dread da/f16 [0,4194304] 0 2024-09-16T23:48:48.771 INFO:tasks.workunit.client.0.smithi042.stdout:1/121: rename d0/d9/cd to d0/d15/c25 0 2024-09-16T23:48:48.771 INFO:tasks.workunit.client.0.smithi042.stdout:7/45: truncate d3/fc 1464939 0 2024-09-16T23:48:48.772 INFO:tasks.workunit.client.0.smithi042.stdout:8/165: creat da/d2f/f38 x:0 0 0 2024-09-16T23:48:48.775 INFO:tasks.workunit.client.0.smithi042.stdout:7/46: mknod d3/cf 0 2024-09-16T23:48:48.782 INFO:tasks.workunit.client.0.smithi042.stdout:5/74: truncate d9/db/d11/f13 1750122 0 2024-09-16T23:48:48.782 INFO:tasks.workunit.client.0.smithi042.stdout:5/75: chown d9 14556567 1 2024-09-16T23:48:48.783 INFO:tasks.workunit.client.0.smithi042.stdout:8/166: truncate da/f17 3618160 0 2024-09-16T23:48:48.783 INFO:tasks.workunit.client.0.smithi042.stdout:8/167: readlink l3 0 2024-09-16T23:48:48.786 INFO:tasks.workunit.client.0.smithi042.stdout:5/76: mknod d9/c14 0 2024-09-16T23:48:48.788 INFO:tasks.workunit.client.0.smithi042.stdout:8/168: write da/d19/f23 [910558,70717] 0 2024-09-16T23:48:48.793 INFO:tasks.workunit.client.0.smithi042.stdout:8/169: creat da/d19/f39 x:0 0 0 2024-09-16T23:48:48.794 INFO:tasks.workunit.client.0.smithi042.stdout:3/42: fsync d2/fa 0 2024-09-16T23:48:48.794 INFO:tasks.workunit.client.0.smithi042.stdout:8/170: write da/f14 [2975792,85656] 0 2024-09-16T23:48:48.795 INFO:tasks.workunit.client.0.smithi042.stdout:3/43: write f0 [3483247,117036] 0 2024-09-16T23:48:48.797 INFO:tasks.workunit.client.0.smithi042.stdout:8/171: dread da/f16 [0,4194304] 0 2024-09-16T23:48:48.802 INFO:tasks.workunit.client.0.smithi042.stdout:8/172: dwrite da/d19/d2c/f36 [0,4194304] 0 2024-09-16T23:48:48.806 INFO:tasks.workunit.client.0.smithi042.stdout:8/173: mknod da/c3a 0 2024-09-16T23:48:48.806 INFO:tasks.workunit.client.0.smithi042.stdout:2/75: rmdir d0 39 2024-09-16T23:48:48.808 INFO:tasks.workunit.client.0.smithi042.stdout:3/44: truncate d2/fa 275695 0 2024-09-16T23:48:48.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/45: creat d2/fb x:0 0 0 2024-09-16T23:48:48.813 INFO:tasks.workunit.client.0.smithi042.stdout:4/70: write de/d12/f14 [4928637,130] 0 2024-09-16T23:48:48.813 INFO:tasks.workunit.client.0.smithi042.stdout:4/71: readlink de/l11 0 2024-09-16T23:48:48.814 INFO:tasks.workunit.client.0.smithi042.stdout:3/46: read f1 [4641094,114421] 0 2024-09-16T23:48:48.815 INFO:tasks.workunit.client.0.smithi042.stdout:8/174: creat da/d19/f3b x:0 0 0 2024-09-16T23:48:48.815 INFO:tasks.workunit.client.0.smithi042.stdout:3/47: chown d2/l6 1960 1 2024-09-16T23:48:48.816 INFO:tasks.workunit.client.0.smithi042.stdout:2/76: symlink d0/da/d1b/l1d 0 2024-09-16T23:48:48.816 INFO:tasks.workunit.client.0.smithi042.stdout:3/48: dread - d2/fb zero size 2024-09-16T23:48:48.817 INFO:tasks.workunit.client.0.smithi042.stdout:2/77: write d0/ff [1378216,125571] 0 2024-09-16T23:48:48.818 INFO:tasks.workunit.client.0.smithi042.stdout:4/72: mkdir de/d1a 0 2024-09-16T23:48:48.823 INFO:tasks.workunit.client.0.smithi042.stdout:3/49: mkdir d2/d7/dc 0 2024-09-16T23:48:48.824 INFO:tasks.workunit.client.0.smithi042.stdout:4/73: mkdir de/d12/d1b 0 2024-09-16T23:48:48.825 INFO:tasks.workunit.client.0.smithi042.stdout:8/175: dwrite da/d19/d1d/f2b [0,4194304] 0 2024-09-16T23:48:48.825 INFO:tasks.workunit.client.0.smithi042.stdout:4/74: readlink de/l16 0 2024-09-16T23:48:48.826 INFO:tasks.workunit.client.0.smithi042.stdout:8/176: stat da/f14 0 2024-09-16T23:48:48.828 INFO:tasks.workunit.client.0.smithi042.stdout:2/78: dwrite d0/da/d17/f1c [0,4194304] 0 2024-09-16T23:48:48.829 INFO:tasks.workunit.client.0.smithi042.stdout:2/79: chown d0/da/d1b/l1d 2 1 2024-09-16T23:48:48.830 INFO:tasks.workunit.client.0.smithi042.stdout:3/50: mkdir d2/d9/dd 0 2024-09-16T23:48:48.834 INFO:tasks.workunit.client.0.smithi042.stdout:8/177: write da/d19/f1f [1569537,97064] 0 2024-09-16T23:48:48.834 INFO:tasks.workunit.client.0.smithi042.stdout:8/178: stat da/d19/d2c/f36 0 2024-09-16T23:48:48.837 INFO:tasks.workunit.client.1.smithi043.stdout:7/297: mkdir d4/db/d11/d21/d68 0 2024-09-16T23:48:48.839 INFO:tasks.workunit.client.1.smithi043.stdout:7/298: stat d4/db/d11/d21/d68 0 2024-09-16T23:48:48.841 INFO:tasks.workunit.client.1.smithi043.stdout:9/215: symlink d4/d1c/d41/l42 0 2024-09-16T23:48:48.845 INFO:tasks.workunit.client.1.smithi043.stdout:5/431: dwrite d3/d9/f56 [0,4194304] 0 2024-09-16T23:48:48.846 INFO:tasks.workunit.client.0.smithi042.stdout:7/47: rmdir d3 39 2024-09-16T23:48:48.850 INFO:tasks.workunit.client.1.smithi043.stdout:1/386: fsync d4/d9/d1c/d1e/f23 0 2024-09-16T23:48:48.851 INFO:tasks.workunit.client.1.smithi043.stdout:6/297: rename de/d2a/l48 to de/d23/d34/l64 0 2024-09-16T23:48:48.853 INFO:tasks.workunit.client.0.smithi042.stdout:5/77: unlink d9/db/d11/f13 0 2024-09-16T23:48:48.854 INFO:tasks.workunit.client.0.smithi042.stdout:1/122: mkdir d0/d9/d26 0 2024-09-16T23:48:48.854 INFO:tasks.workunit.client.1.smithi043.stdout:8/279: symlink de/d18/d1f/d25/l57 0 2024-09-16T23:48:48.856 INFO:tasks.workunit.client.0.smithi042.stdout:9/58: write d4/d10/f15 [683851,10066] 0 2024-09-16T23:48:48.857 INFO:tasks.workunit.client.1.smithi043.stdout:9/216: dwrite d4/d21/d2a/d2c/f2d [0,4194304] 0 2024-09-16T23:48:48.857 INFO:tasks.workunit.client.1.smithi043.stdout:1/387: write d4/d9/d12/d2b/f6b [743981,101832] 0 2024-09-16T23:48:48.860 INFO:tasks.workunit.client.1.smithi043.stdout:0/283: symlink d0/l57 0 2024-09-16T23:48:48.860 INFO:tasks.workunit.client.1.smithi043.stdout:2/190: creat d8/d11/f46 x:0 0 0 2024-09-16T23:48:48.861 INFO:tasks.workunit.client.0.smithi042.stdout:2/80: getdents d0 0 2024-09-16T23:48:48.861 INFO:tasks.workunit.client.0.smithi042.stdout:5/78: symlink d9/l15 0 2024-09-16T23:48:48.863 INFO:tasks.workunit.client.0.smithi042.stdout:9/59: dread d4/f5 [0,4194304] 0 2024-09-16T23:48:48.863 INFO:tasks.workunit.client.0.smithi042.stdout:2/81: dread - d0/f1a zero size 2024-09-16T23:48:48.863 INFO:tasks.workunit.client.0.smithi042.stdout:5/79: chown l8 1 1 2024-09-16T23:48:48.863 INFO:tasks.workunit.client.0.smithi042.stdout:5/80: dwrite - no filename 2024-09-16T23:48:48.863 INFO:tasks.workunit.client.1.smithi043.stdout:2/191: fdatasync d8/d1d/d25/f44 0 2024-09-16T23:48:48.864 INFO:tasks.workunit.client.1.smithi043.stdout:8/280: dread - de/d18/d27/d34/d3d/f51 zero size 2024-09-16T23:48:48.865 INFO:tasks.workunit.client.0.smithi042.stdout:7/48: write d3/f8 [152585,88829] 0 2024-09-16T23:48:48.865 INFO:tasks.workunit.client.0.smithi042.stdout:5/81: readlink d9/l15 0 2024-09-16T23:48:48.865 INFO:tasks.workunit.client.1.smithi043.stdout:7/299: symlink d4/l69 0 2024-09-16T23:48:48.866 INFO:tasks.workunit.client.1.smithi043.stdout:2/192: fsync d8/d1d/d20/f40 0 2024-09-16T23:48:48.866 INFO:tasks.workunit.client.0.smithi042.stdout:1/123: creat d0/d9/d26/f27 x:0 0 0 2024-09-16T23:48:48.867 INFO:tasks.workunit.client.1.smithi043.stdout:4/281: rename d8/d15/d19 to d8/d15/d20/d2b/d54/d55 0 2024-09-16T23:48:48.870 INFO:tasks.workunit.client.1.smithi043.stdout:6/298: dread de/f24 [0,4194304] 0 2024-09-16T23:48:48.870 INFO:tasks.workunit.client.1.smithi043.stdout:3/215: getdents d4/d30 0 2024-09-16T23:48:48.871 INFO:tasks.workunit.client.0.smithi042.stdout:2/82: rename d0/c5 to d0/c1e 0 2024-09-16T23:48:48.872 INFO:tasks.workunit.client.1.smithi043.stdout:0/284: creat d0/de/f58 x:0 0 0 2024-09-16T23:48:48.872 INFO:tasks.workunit.client.0.smithi042.stdout:2/83: read d0/da/d17/f1c [1129201,36398] 0 2024-09-16T23:48:48.873 INFO:tasks.workunit.client.1.smithi043.stdout:0/285: chown d0/d2/df/d1a 2863828 1 2024-09-16T23:48:48.876 INFO:tasks.workunit.client.1.smithi043.stdout:9/217: dread d4/f24 [0,4194304] 0 2024-09-16T23:48:48.876 INFO:tasks.workunit.client.1.smithi043.stdout:3/216: chown d4/de/c1c 41349 1 2024-09-16T23:48:48.877 INFO:tasks.workunit.client.0.smithi042.stdout:5/82: readlink l5 0 2024-09-16T23:48:48.877 INFO:tasks.workunit.client.0.smithi042.stdout:5/83: dwrite - no filename 2024-09-16T23:48:48.877 INFO:tasks.workunit.client.0.smithi042.stdout:5/84: write - no filename 2024-09-16T23:48:48.877 INFO:tasks.workunit.client.0.smithi042.stdout:5/85: write - no filename 2024-09-16T23:48:48.878 INFO:tasks.workunit.client.0.smithi042.stdout:5/86: dwrite - no filename 2024-09-16T23:48:48.878 INFO:tasks.workunit.client.0.smithi042.stdout:5/87: dwrite - no filename 2024-09-16T23:48:48.878 INFO:tasks.workunit.client.0.smithi042.stdout:5/88: truncate - no filename 2024-09-16T23:48:48.878 INFO:tasks.workunit.client.0.smithi042.stdout:5/89: write - no filename 2024-09-16T23:48:48.878 INFO:tasks.workunit.client.1.smithi043.stdout:1/388: creat d4/d3e/d5b/f7f x:0 0 0 2024-09-16T23:48:48.879 INFO:tasks.workunit.client.1.smithi043.stdout:6/299: dread de/d1a/f38 [0,4194304] 0 2024-09-16T23:48:48.880 INFO:tasks.workunit.client.0.smithi042.stdout:7/49: rmdir d3 39 2024-09-16T23:48:48.880 INFO:tasks.workunit.client.0.smithi042.stdout:1/124: stat d0/d9/df/f13 0 2024-09-16T23:48:48.884 INFO:tasks.workunit.client.1.smithi043.stdout:7/300: mknod d4/db/d11/d21/d31/c6a 0 2024-09-16T23:48:48.886 INFO:tasks.workunit.client.0.smithi042.stdout:2/84: creat d0/da/f1f x:0 0 0 2024-09-16T23:48:48.886 INFO:tasks.workunit.client.1.smithi043.stdout:4/282: write d8/d15/d20/d2b/f28 [4874039,14937] 0 2024-09-16T23:48:48.887 INFO:tasks.workunit.client.0.smithi042.stdout:5/90: creat d9/db/f16 x:0 0 0 2024-09-16T23:48:48.888 INFO:tasks.workunit.client.1.smithi043.stdout:2/193: creat d8/d1d/d20/d23/d38/f47 x:0 0 0 2024-09-16T23:48:48.888 INFO:tasks.workunit.client.0.smithi042.stdout:5/91: chown d9/cc 30470269 1 2024-09-16T23:48:48.889 INFO:tasks.workunit.client.1.smithi043.stdout:1/389: symlink d4/d9/d12/l80 0 2024-09-16T23:48:48.890 INFO:tasks.workunit.client.0.smithi042.stdout:4/75: truncate f2 3926640 0 2024-09-16T23:48:48.894 INFO:tasks.workunit.client.1.smithi043.stdout:4/283: read d8/d15/d20/d2b/d54/d55/f2a [2631166,59434] 0 2024-09-16T23:48:48.896 INFO:tasks.workunit.client.0.smithi042.stdout:2/85: creat d0/f20 x:0 0 0 2024-09-16T23:48:48.898 INFO:tasks.workunit.client.0.smithi042.stdout:5/92: mkdir d9/db/d17 0 2024-09-16T23:48:48.898 INFO:tasks.workunit.client.1.smithi043.stdout:5/432: getdents d3/d9/d22 0 2024-09-16T23:48:48.898 INFO:tasks.workunit.client.1.smithi043.stdout:2/194: creat d8/d1d/d20/f48 x:0 0 0 2024-09-16T23:48:48.899 INFO:tasks.workunit.client.1.smithi043.stdout:6/300: dwrite f2 [0,4194304] 0 2024-09-16T23:48:48.899 INFO:tasks.workunit.client.1.smithi043.stdout:3/217: dwrite d4/de/f2a [0,4194304] 0 2024-09-16T23:48:48.900 INFO:tasks.workunit.client.1.smithi043.stdout:7/301: creat d4/db/d11/d21/d31/f6b x:0 0 0 2024-09-16T23:48:48.900 INFO:tasks.workunit.client.1.smithi043.stdout:8/281: dread de/d10/f1b [0,4194304] 0 2024-09-16T23:48:48.901 INFO:tasks.workunit.client.0.smithi042.stdout:1/125: symlink d0/d9/l28 0 2024-09-16T23:48:48.901 INFO:tasks.workunit.client.0.smithi042.stdout:7/50: dwrite d3/fe [0,4194304] 0 2024-09-16T23:48:48.902 INFO:tasks.workunit.client.1.smithi043.stdout:6/301: chown c8 3924286 1 2024-09-16T23:48:48.902 INFO:tasks.workunit.client.1.smithi043.stdout:7/302: write d4/d27/f3f [4478223,51449] 0 2024-09-16T23:48:48.903 INFO:tasks.workunit.client.0.smithi042.stdout:9/60: getdents d4/d10 0 2024-09-16T23:48:48.904 INFO:tasks.workunit.client.0.smithi042.stdout:8/179: dwrite da/f18 [0,4194304] 0 2024-09-16T23:48:48.905 INFO:tasks.workunit.client.1.smithi043.stdout:5/433: write d3/d9/d22/d60/f82 [251577,107954] 0 2024-09-16T23:48:48.906 INFO:tasks.workunit.client.0.smithi042.stdout:9/61: chown d4/l8 39 1 2024-09-16T23:48:48.906 INFO:tasks.workunit.client.1.smithi043.stdout:1/390: creat d4/d9/d1c/d1e/d42/f81 x:0 0 0 2024-09-16T23:48:48.908 INFO:tasks.workunit.client.1.smithi043.stdout:3/218: read d4/de/d1f/d34/f3a [204514,46766] 0 2024-09-16T23:48:48.908 INFO:tasks.workunit.client.1.smithi043.stdout:9/218: write d4/d1c/f20 [1784704,21289] 0 2024-09-16T23:48:48.909 INFO:tasks.workunit.client.1.smithi043.stdout:0/286: truncate d0/d2/df/f10 15009129 0 2024-09-16T23:48:48.909 INFO:tasks.workunit.client.0.smithi042.stdout:8/180: read da/d19/f23 [391608,89218] 0 2024-09-16T23:48:48.911 INFO:tasks.workunit.client.1.smithi043.stdout:7/303: read - d4/db/d28/f4d zero size 2024-09-16T23:48:48.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.915+0000 7ff6f09bf700 1 -- 172.21.15.42:0/891775267 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7ff6d4036ac0 con 0x7ff6d4005d60 2024-09-16T23:48:48.916 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:48:48.916 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (6m) 2m ago 8m 25.5M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:48:48.916 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (8m) 2m ago 8m 7453k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:48:48.917 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (7m) 17s ago 7m 7537k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:48:48.917 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (8m) 2m ago 8m 6660k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:48:48.917 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (7m) 17s ago 7m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:48:48.917 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (6m) 2m ago 7m 83.1M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:48:48.917 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (2m) 2m ago 2m 15.9M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:48:48.917 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (2m) 2m ago 2m 11.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:48:48.917 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (2m) 17s ago 2m 112M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (2m) 17s ago 2m 14.0M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:9283,8765,8443 running (9m) 2m ago 9m 488M - 18.2.1 5be31c24972a cd8abd4f468e 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (27s) 17s ago 7m 73.2M - 19.1.1-330-gc2ddf972 ebb0615149de 50141468f451 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (9m) 2m ago 9m 45.4M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (7m) 17s ago 7m 38.4M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (8m) 2m ago 8m 16.2M - 1.5.0 0da6a335fe13 7c84e9bf6c1c 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (7m) 17s ago 7m 16.8M - 1.5.0 0da6a335fe13 82d41c3cc1ee 2024-09-16T23:48:48.918 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 2m ago 6m 48.3M 3948M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:48:48.919 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (5m) 2m ago 5m 47.2M 3948M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:48:48.919 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (5m) 2m ago 5m 46.4M 3948M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:48:48.919 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (4m) 17s ago 4m 86.2M 4972M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:48:48.919 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (4m) 17s ago 4m 74.8M 4972M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:48:48.919 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (4m) 17s ago 4m 75.3M 4972M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:48:48.919 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (6m) 2m ago 7m 56.4M - 2.43.0 a07b618ecd1d 8ebaffdf2443 2024-09-16T23:48:48.920 INFO:tasks.workunit.client.1.smithi043.stdout:3/219: dwrite d4/de/d1f/f29 [0,4194304] 0 2024-09-16T23:48:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.919+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7ff6d4005d60 msgr2=0x7ff6d4034ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.919+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7ff6d4005d60 0x7ff6d4034ac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00092a0 tx=0x7ff6e0017000 comp rx=0 tx=0).stop 2024-09-16T23:48:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.920+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 msgr2=0x7ff6ec069140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:48.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.920+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec069140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc005100 tx=0x7ff6dc0051e0 comp rx=0 tx=0).stop 2024-09-16T23:48:48.921 INFO:tasks.workunit.client.0.smithi042.stdout:7/51: symlink d3/l10 0 2024-09-16T23:48:48.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 shutdown_connections 2024-09-16T23:48:48.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7ff6d4005d60 0x7ff6d4034ac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:48.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:6800/2,v1:172.21.15.42:6801/2] conn(0x7ff6d406d3f0 0x7ff6d406f8b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:48.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6ec0ffc10 0x7ff6ec069140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:48.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 --2- 172.21.15.42:0/891775267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6ec100e10 0x7ff6ec19b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:48.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 >> 172.21.15.42:0/891775267 conn(0x7ff6ec0fb190 msgr2=0x7ff6ec104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:48.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 shutdown_connections 2024-09-16T23:48:48.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:48.923+0000 7ff6f19c1700 1 -- 172.21.15.42:0/891775267 wait complete. 2024-09-16T23:48:48.922 INFO:tasks.workunit.client.1.smithi043.stdout:7/304: write d4/db/d11/d21/d31/d33/f55 [1038371,108720] 0 2024-09-16T23:48:48.923 INFO:tasks.workunit.client.0.smithi042.stdout:3/51: dwrite d2/fa [0,4194304] 0 2024-09-16T23:48:48.925 INFO:tasks.workunit.client.0.smithi042.stdout:3/52: chown d2/f4 5848 1 2024-09-16T23:48:48.927 INFO:tasks.workunit.client.1.smithi043.stdout:7/305: truncate d4/f19 3368151 0 2024-09-16T23:48:48.927 INFO:tasks.workunit.client.1.smithi043.stdout:7/306: chown d4/d27/d41/d1b/l2e 2051499 1 2024-09-16T23:48:48.927 INFO:tasks.workunit.client.0.smithi042.stdout:3/53: write d2/d7/f8 [1138669,121547] 0 2024-09-16T23:48:48.928 INFO:tasks.workunit.client.1.smithi043.stdout:9/219: dwrite d4/f31 [0,4194304] 0 2024-09-16T23:48:48.929 INFO:tasks.workunit.client.0.smithi042.stdout:8/181: creat da/d19/d1d/f3c x:0 0 0 2024-09-16T23:48:48.931 INFO:tasks.workunit.client.0.smithi042.stdout:2/86: dwrite d0/da/d17/f1c [4194304,4194304] 0 2024-09-16T23:48:48.933 INFO:tasks.workunit.client.1.smithi043.stdout:0/287: stat d0/d2/df/d1a/d3b/l44 0 2024-09-16T23:48:48.933 INFO:tasks.workunit.client.0.smithi042.stdout:3/54: truncate d2/fb 578088 0 2024-09-16T23:48:48.933 INFO:tasks.workunit.client.0.smithi042.stdout:2/87: stat d0 0 2024-09-16T23:48:48.934 INFO:tasks.workunit.client.1.smithi043.stdout:7/307: read - d4/db/d11/d21/d36/f42 zero size 2024-09-16T23:48:48.934 INFO:tasks.workunit.client.1.smithi043.stdout:2/195: creat d8/d11/d16/f49 x:0 0 0 2024-09-16T23:48:48.935 INFO:tasks.workunit.client.1.smithi043.stdout:4/284: mknod d8/d25/c56 0 2024-09-16T23:48:48.938 INFO:tasks.workunit.client.1.smithi043.stdout:4/285: chown f7 219 1 2024-09-16T23:48:48.939 INFO:tasks.workunit.client.1.smithi043.stdout:7/308: write d4/db/d11/d21/d36/f57 [515281,90927] 0 2024-09-16T23:48:48.946 INFO:tasks.workunit.client.1.smithi043.stdout:3/220: read d4/fb [3973843,95986] 0 2024-09-16T23:48:48.947 INFO:tasks.workunit.client.0.smithi042.stdout:1/126: write d0/f6 [823539,15361] 0 2024-09-16T23:48:48.949 INFO:tasks.workunit.client.1.smithi043.stdout:7/309: stat d4/db/d11/d15/d29 0 2024-09-16T23:48:48.950 INFO:tasks.workunit.client.1.smithi043.stdout:5/434: symlink d3/db/d19/d76/l86 0 2024-09-16T23:48:48.951 INFO:tasks.workunit.client.0.smithi042.stdout:7/52: fdatasync d3/fc 0 2024-09-16T23:48:48.952 INFO:tasks.workunit.client.1.smithi043.stdout:4/286: write d8/d15/d20/d2b/d22/f36 [1272716,38618] 0 2024-09-16T23:48:48.953 INFO:tasks.workunit.client.0.smithi042.stdout:7/53: chown d3/f8 7 1 2024-09-16T23:48:48.954 INFO:tasks.workunit.client.0.smithi042.stdout:8/182: chown da/f17 9647 1 2024-09-16T23:48:48.955 INFO:tasks.workunit.client.1.smithi043.stdout:0/288: dwrite d0/d2/df/d1a/d46/f35 [0,4194304] 0 2024-09-16T23:48:48.955 INFO:tasks.workunit.client.0.smithi042.stdout:5/93: link d9/cc d9/db/de/d10/c18 0 2024-09-16T23:48:48.956 INFO:tasks.workunit.client.1.smithi043.stdout:2/196: dread d8/d1f/f30 [0,4194304] 0 2024-09-16T23:48:48.958 INFO:tasks.workunit.client.0.smithi042.stdout:1/127: symlink d0/d17/d18/l29 0 2024-09-16T23:48:48.959 INFO:tasks.workunit.client.0.smithi042.stdout:1/128: fdatasync d0/f1d 0 2024-09-16T23:48:48.959 INFO:tasks.workunit.client.1.smithi043.stdout:6/302: symlink de/d23/d63/l65 0 2024-09-16T23:48:48.960 INFO:tasks.workunit.client.0.smithi042.stdout:1/129: dread - d0/d17/f21 zero size 2024-09-16T23:48:48.961 INFO:tasks.workunit.client.0.smithi042.stdout:7/54: dwrite d3/fe [0,4194304] 0 2024-09-16T23:48:48.962 INFO:tasks.workunit.client.1.smithi043.stdout:6/303: dread - de/f47 zero size 2024-09-16T23:48:48.963 INFO:tasks.workunit.client.0.smithi042.stdout:8/183: mknod da/d19/d2c/c3d 0 2024-09-16T23:48:48.963 INFO:tasks.workunit.client.0.smithi042.stdout:5/94: mkdir d9/db/d19 0 2024-09-16T23:48:48.964 INFO:tasks.workunit.client.0.smithi042.stdout:5/95: chown d9/db/de/d10 57363478 1 2024-09-16T23:48:48.965 INFO:tasks.workunit.client.1.smithi043.stdout:1/391: dread d4/d9/d12/f1f [0,4194304] 0 2024-09-16T23:48:48.968 INFO:tasks.workunit.client.0.smithi042.stdout:7/55: creat d3/f11 x:0 0 0 2024-09-16T23:48:48.969 INFO:tasks.workunit.client.1.smithi043.stdout:3/221: fsync d4/f16 0 2024-09-16T23:48:48.969 INFO:tasks.workunit.client.0.smithi042.stdout:8/184: creat da/d19/d1d/f3e x:0 0 0 2024-09-16T23:48:48.970 INFO:tasks.workunit.client.1.smithi043.stdout:3/222: chown d4/de/f21 321552136 1 2024-09-16T23:48:48.971 INFO:tasks.workunit.client.0.smithi042.stdout:8/185: dread - da/d19/d1d/f2e zero size 2024-09-16T23:48:48.971 INFO:tasks.workunit.client.0.smithi042.stdout:8/186: stat da/d19/f23 0 2024-09-16T23:48:48.972 INFO:tasks.workunit.client.0.smithi042.stdout:3/55: fsync d2/f4 0 2024-09-16T23:48:48.973 INFO:tasks.workunit.client.0.smithi042.stdout:1/130: link d0/f1d d0/d9/df/f2a 0 2024-09-16T23:48:48.975 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: Manager daemon smithi043.xveyou is now available 2024-09-16T23:48:48.975 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: Migrating agent root cert to cert store 2024-09-16T23:48:48.975 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.975 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: Migrating agent root key to cert store 2024-09-16T23:48:48.975 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.975 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: Checking for cert/key for grafana.smithi042 2024-09-16T23:48:48.975 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: Migrating grafana.smithi042 cert to cert store 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: Migrating grafana.smithi042 key to cert store 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/trash_purge_schedule"}]: dispatch 2024-09-16T23:48:48.976 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:48 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/trash_purge_schedule"}]: dispatch 2024-09-16T23:48:48.980 INFO:tasks.workunit.client.1.smithi043.stdout:7/310: dwrite d4/db/d11/d15/f47 [0,4194304] 0 2024-09-16T23:48:48.983 INFO:tasks.workunit.client.1.smithi043.stdout:5/435: symlink d3/db/d26/l87 0 2024-09-16T23:48:48.986 INFO:tasks.workunit.client.1.smithi043.stdout:5/436: stat d3/db/d15/f69 0 2024-09-16T23:48:48.986 INFO:tasks.workunit.client.1.smithi043.stdout:6/304: mkdir de/d23/d34/d66 0 2024-09-16T23:48:48.987 INFO:tasks.workunit.client.0.smithi042.stdout:5/96: creat d9/db/de/d10/f1a x:0 0 0 2024-09-16T23:48:48.987 INFO:tasks.workunit.client.1.smithi043.stdout:1/392: creat d4/d9/d12/f82 x:0 0 0 2024-09-16T23:48:48.991 INFO:tasks.workunit.client.1.smithi043.stdout:4/287: write d8/fc [3951671,124749] 0 2024-09-16T23:48:48.991 INFO:tasks.workunit.client.1.smithi043.stdout:3/223: fdatasync d4/fb 0 2024-09-16T23:48:48.992 INFO:tasks.workunit.client.1.smithi043.stdout:7/311: readlink d4/l7 0 2024-09-16T23:48:48.994 INFO:tasks.workunit.client.0.smithi042.stdout:8/187: fdatasync da/f16 0 2024-09-16T23:48:48.995 INFO:tasks.workunit.client.0.smithi042.stdout:8/188: readlink da/lb 0 2024-09-16T23:48:48.996 INFO:tasks.workunit.client.0.smithi042.stdout:4/76: write f2 [2402997,6361] 0 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: Manager daemon smithi043.xveyou is now available 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: Migrating agent root cert to cert store 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: Migrating agent root key to cert store 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: Checking for cert/key for grafana.smithi042 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: Migrating grafana.smithi042 cert to cert store 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: Migrating grafana.smithi042 key to cert store 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.996 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:48:48.997 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:48:48.997 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:48:48.997 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:48:48.997 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:48:48.997 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/trash_purge_schedule"}]: dispatch 2024-09-16T23:48:48.997 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:48 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi043.xveyou/trash_purge_schedule"}]: dispatch 2024-09-16T23:48:48.997 INFO:tasks.workunit.client.1.smithi043.stdout:4/288: write d8/fc [5178348,94368] 0 2024-09-16T23:48:48.998 INFO:tasks.workunit.client.1.smithi043.stdout:5/437: dread - d3/d9/d22/d4a/d4c/f72 zero size 2024-09-16T23:48:48.999 INFO:tasks.workunit.client.1.smithi043.stdout:4/289: stat d8/d15/f3c 0 2024-09-16T23:48:49.004 INFO:tasks.workunit.client.1.smithi043.stdout:3/224: mknod d4/de/d1f/d34/c3f 0 2024-09-16T23:48:49.004 INFO:tasks.workunit.client.0.smithi042.stdout:3/56: creat d2/d7/fe x:0 0 0 2024-09-16T23:48:49.005 INFO:tasks.workunit.client.0.smithi042.stdout:3/57: stat d2/d7/fe 0 2024-09-16T23:48:49.007 INFO:tasks.workunit.client.1.smithi043.stdout:4/290: read d8/d15/d20/d2b/f28 [8053538,84979] 0 2024-09-16T23:48:49.010 INFO:tasks.workunit.client.0.smithi042.stdout:7/56: rename d3/l7 to d3/l12 0 2024-09-16T23:48:49.010 INFO:tasks.workunit.client.1.smithi043.stdout:1/393: getdents d4/d9/d1c/d1e/d42/d5d/d71/d76 0 2024-09-16T23:48:49.014 INFO:tasks.workunit.client.0.smithi042.stdout:1/131: creat d0/d9/df/f2b x:0 0 0 2024-09-16T23:48:49.015 INFO:tasks.workunit.client.0.smithi042.stdout:1/132: chown d0/f5 25 1 2024-09-16T23:48:49.016 INFO:tasks.workunit.client.0.smithi042.stdout:5/97: mkdir d9/db/d1b 0 2024-09-16T23:48:49.016 INFO:tasks.workunit.client.0.smithi042.stdout:1/133: chown d0 8080 1 2024-09-16T23:48:49.017 INFO:tasks.workunit.client.1.smithi043.stdout:6/305: rename de/d23/f4e to de/f67 0 2024-09-16T23:48:49.018 INFO:tasks.workunit.client.0.smithi042.stdout:7/57: dwrite d3/fa [0,4194304] 0 2024-09-16T23:48:49.018 INFO:tasks.workunit.client.0.smithi042.stdout:7/58: chown d3/f11 0 1 2024-09-16T23:48:49.020 INFO:tasks.workunit.client.1.smithi043.stdout:4/291: unlink d8/d15/d20/d2b/f32 0 2024-09-16T23:48:49.027 INFO:tasks.workunit.client.1.smithi043.stdout:3/225: mkdir d4/de/d40 0 2024-09-16T23:48:49.027 INFO:tasks.workunit.client.1.smithi043.stdout:4/292: dread d8/d15/d20/d2b/d23/f40 [0,4194304] 0 2024-09-16T23:48:49.028 INFO:tasks.workunit.client.1.smithi043.stdout:3/226: write d4/f23 [1364067,114398] 0 2024-09-16T23:48:49.029 INFO:tasks.workunit.client.1.smithi043.stdout:6/306: rename de/d1a/d39/c5e to de/d1a/d4d/d5a/c68 0 2024-09-16T23:48:49.030 INFO:tasks.workunit.client.1.smithi043.stdout:3/227: readlink d4/de/d35/l2e 0 2024-09-16T23:48:49.031 INFO:tasks.workunit.client.1.smithi043.stdout:6/307: write de/d1a/d28/f35 [3739688,100439] 0 2024-09-16T23:48:49.037 INFO:tasks.workunit.client.1.smithi043.stdout:3/228: dwrite d4/de/d35/f1d [0,4194304] 0 2024-09-16T23:48:49.037 INFO:tasks.workunit.client.1.smithi043.stdout:6/308: dread de/d1a/f41 [0,4194304] 0 2024-09-16T23:48:49.040 INFO:tasks.workunit.client.1.smithi043.stdout:4/293: mkdir d8/d25/d39/d41/d57 0 2024-09-16T23:48:49.040 INFO:tasks.workunit.client.1.smithi043.stdout:8/282: truncate de/d18/d27/d41/f50 770234 0 2024-09-16T23:48:49.041 INFO:tasks.workunit.client.1.smithi043.stdout:4/294: chown d8/d25/f2c 9898528 1 2024-09-16T23:48:49.045 INFO:tasks.workunit.client.0.smithi042.stdout:9/62: truncate d4/fb 4153222 0 2024-09-16T23:48:49.045 INFO:tasks.workunit.client.1.smithi043.stdout:6/309: stat de/d23/c29 0 2024-09-16T23:48:49.048 INFO:tasks.workunit.client.0.smithi042.stdout:9/63: fsync d4/d10/f15 0 2024-09-16T23:48:49.048 INFO:tasks.workunit.client.0.smithi042.stdout:1/134: dread d0/f1d [0,4194304] 0 2024-09-16T23:48:49.049 INFO:tasks.workunit.client.1.smithi043.stdout:3/229: dread d4/de/f2a [0,4194304] 0 2024-09-16T23:48:49.049 INFO:tasks.workunit.client.0.smithi042.stdout:4/77: write f7 [3292498,59723] 0 2024-09-16T23:48:49.050 INFO:tasks.workunit.client.1.smithi043.stdout:9/220: write d4/f11 [2776829,51031] 0 2024-09-16T23:48:49.051 INFO:tasks.workunit.client.0.smithi042.stdout:9/64: chown d4/d10 23735 1 2024-09-16T23:48:49.052 INFO:tasks.workunit.client.1.smithi043.stdout:8/283: dwrite de/d10/f13 [0,4194304] 0 2024-09-16T23:48:49.054 INFO:tasks.workunit.client.1.smithi043.stdout:3/230: truncate d4/de/f2a 4766834 0 2024-09-16T23:48:49.055 INFO:tasks.workunit.client.0.smithi042.stdout:1/135: chown d0/d15/d23 1087 1 2024-09-16T23:48:49.055 INFO:tasks.workunit.client.0.smithi042.stdout:7/59: dwrite d3/fe [0,4194304] 0 2024-09-16T23:48:49.056 INFO:tasks.workunit.client.1.smithi043.stdout:8/284: dread de/d18/d1f/d2f/f43 [0,4194304] 0 2024-09-16T23:48:49.058 INFO:tasks.workunit.client.0.smithi042.stdout:4/78: dread de/f10 [0,4194304] 0 2024-09-16T23:48:49.064 INFO:tasks.workunit.client.1.smithi043.stdout:6/310: rename de/l25 to de/l69 0 2024-09-16T23:48:49.065 INFO:tasks.workunit.client.1.smithi043.stdout:8/285: dwrite de/f54 [0,4194304] 0 2024-09-16T23:48:49.065 INFO:tasks.workunit.client.1.smithi043.stdout:2/197: truncate d8/fc 3156169 0 2024-09-16T23:48:49.067 INFO:tasks.workunit.client.1.smithi043.stdout:2/198: write d8/d1d/d20/d23/d39/f3d [3563908,123875] 0 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 -- 172.21.15.42:0/1439194650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4100da0 msgr2=0x7fd8e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 --2- 172.21.15.42:0/1439194650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4100da0 0x7fd8e4101200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4009910 tx=0x7fd8d4009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 -- 172.21.15.42:0/1439194650 shutdown_connections 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 --2- 172.21.15.42:0/1439194650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4100da0 0x7fd8e4101200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 --2- 172.21.15.42:0/1439194650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8e40ffba0 0x7fd8e40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 -- 172.21.15.42:0/1439194650 >> 172.21.15.42:0/1439194650 conn(0x7fd8e40fb120 msgr2=0x7fd8e40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 -- 172.21.15.42:0/1439194650 shutdown_connections 2024-09-16T23:48:49.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.100+0000 7fd8ec099700 1 -- 172.21.15.42:0/1439194650 wait complete. 2024-09-16T23:48:49.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.101+0000 7fd8ec099700 1 Processor -- start 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.101+0000 7fd8ec099700 1 -- start start 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.101+0000 7fd8ec099700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8e40ffba0 0x7fd8e4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.101+0000 7fd8ec099700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4198e60 0x7fd8e419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.101+0000 7fd8ec099700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4199370 con 0x7fd8e4198e60 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.101+0000 7fd8ec099700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e41994e0 con 0x7fd8e40ffba0 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9634700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4198e60 0x7fd8e419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9634700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4198e60 0x7fd8e419ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59732/0 (socket says 172.21.15.42:59732) 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9634700 1 -- 172.21.15.42:0/937573044 learned_addr learned my addr 172.21.15.42:0/937573044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9e35700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8e40ffba0 0x7fd8e4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9634700 1 -- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8e40ffba0 msgr2=0x7fd8e4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9634700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8e40ffba0 0x7fd8e4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9634700 1 -- 172.21.15.42:0/937573044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d40095f0 con 0x7fd8e4198e60 2024-09-16T23:48:49.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9e35700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8e40ffba0 0x7fd8e4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:48:49.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.102+0000 7fd8e9634700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4198e60 0x7fd8e419ded0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4015000 tx=0x7fd8d400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:49.101 INFO:tasks.workunit.client.1.smithi043.stdout:8/286: symlink de/d18/d27/l58 0 2024-09-16T23:48:49.101 INFO:tasks.workunit.client.1.smithi043.stdout:2/199: readlink d8/d1f/l3e 0 2024-09-16T23:48:49.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.103+0000 7fd8daffd700 1 -- 172.21.15.42:0/937573044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8d401d070 con 0x7fd8e4198e60 2024-09-16T23:48:49.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.103+0000 7fd8daffd700 1 -- 172.21.15.42:0/937573044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd8d4005dc0 con 0x7fd8e4198e60 2024-09-16T23:48:49.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.103+0000 7fd8daffd700 1 -- 172.21.15.42:0/937573044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8d4017400 con 0x7fd8e4198e60 2024-09-16T23:48:49.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.103+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e419e410 con 0x7fd8e4198e60 2024-09-16T23:48:49.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.103+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e419e900 con 0x7fd8e4198e60 2024-09-16T23:48:49.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.104+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e4108700 con 0x7fd8e4198e60 2024-09-16T23:48:49.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.104+0000 7fd8daffd700 1 -- 172.21.15.42:0/937573044 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) v1 ==== 46250+0+0 (secure 0 0 0) 0x7fd8d4003780 con 0x7fd8e4198e60 2024-09-16T23:48:49.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.105+0000 7fd8daffd700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fd8d0039190 0x7fd8d003b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:49.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.105+0000 7fd8daffd700 1 -- 172.21.15.42:0/937573044 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd8d404e300 con 0x7fd8e4198e60 2024-09-16T23:48:49.103 INFO:tasks.workunit.client.1.smithi043.stdout:2/200: chown d8/d1d/d20/d23 20990 1 2024-09-16T23:48:49.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.107+0000 7fd8e9e35700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fd8d0039190 0x7fd8d003b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:49.104 INFO:tasks.workunit.client.1.smithi043.stdout:9/221: dread d4/f11 [0,4194304] 0 2024-09-16T23:48:49.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.110+0000 7fd8e9e35700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fd8d0039190 0x7fd8d003b650 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4100c00 tx=0x7fd8e0009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:49.107 INFO:tasks.workunit.client.1.smithi043.stdout:2/201: fsync d8/ff 0 2024-09-16T23:48:49.107 INFO:tasks.workunit.client.1.smithi043.stdout:9/222: chown d4/d1c/d25/d2b/l3e 61485 1 2024-09-16T23:48:49.109 INFO:tasks.workunit.client.1.smithi043.stdout:2/202: read - d8/d35/f37 zero size 2024-09-16T23:48:49.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.112+0000 7fd8daffd700 1 -- 172.21.15.42:0/937573044 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fd8d40052a0 con 0x7fd8e4198e60 2024-09-16T23:48:49.113 INFO:tasks.workunit.client.1.smithi043.stdout:8/287: dwrite de/f54 [0,4194304] 0 2024-09-16T23:48:49.122 INFO:tasks.workunit.client.1.smithi043.stdout:8/288: dwrite de/d18/d27/d34/d3d/f42 [0,4194304] 0 2024-09-16T23:48:49.142 INFO:tasks.workunit.client.0.smithi042.stdout:6/75: sync 2024-09-16T23:48:49.142 INFO:tasks.workunit.client.0.smithi042.stdout:0/37: sync 2024-09-16T23:48:49.143 INFO:tasks.workunit.client.0.smithi042.stdout:6/76: dread - d8/db/f12 zero size 2024-09-16T23:48:49.144 INFO:tasks.workunit.client.0.smithi042.stdout:6/77: chown d8/db/dd/l14 3 1 2024-09-16T23:48:49.155 INFO:tasks.workunit.client.1.smithi043.stdout:4/295: getdents d8/d3d 0 2024-09-16T23:48:49.157 INFO:tasks.workunit.client.1.smithi043.stdout:1/394: rmdir d4/d9/d75 0 2024-09-16T23:48:49.160 INFO:tasks.workunit.client.1.smithi043.stdout:5/438: link d3/db/d15/c71 d3/c88 0 2024-09-16T23:48:49.161 INFO:tasks.workunit.client.1.smithi043.stdout:0/289: creat d0/d2/df/f59 x:0 0 0 2024-09-16T23:48:49.165 INFO:tasks.workunit.client.1.smithi043.stdout:6/311: getdents de/d23/d63 0 2024-09-16T23:48:49.165 INFO:tasks.workunit.client.1.smithi043.stdout:1/395: write d4/d9/f46 [8585,44138] 0 2024-09-16T23:48:49.170 INFO:tasks.workunit.client.1.smithi043.stdout:6/312: write de/d1a/d28/f5b [175090,53714] 0 2024-09-16T23:48:49.170 INFO:tasks.workunit.client.1.smithi043.stdout:2/203: creat d8/d11/f4a x:0 0 0 2024-09-16T23:48:49.174 INFO:tasks.workunit.client.1.smithi043.stdout:9/223: getdents d4 0 2024-09-16T23:48:49.175 INFO:tasks.workunit.client.1.smithi043.stdout:3/231: dwrite d4/f1b [0,4194304] 0 2024-09-16T23:48:49.175 INFO:tasks.workunit.client.1.smithi043.stdout:1/396: mkdir d4/d9/d12/d2b/d3d/d83 0 2024-09-16T23:48:49.177 INFO:tasks.workunit.client.1.smithi043.stdout:3/232: write d4/f23 [782518,58007] 0 2024-09-16T23:48:49.177 INFO:tasks.workunit.client.1.smithi043.stdout:9/224: stat d4/d5/d16/l1e 0 2024-09-16T23:48:49.177 INFO:tasks.workunit.client.1.smithi043.stdout:0/290: mkdir d0/d2/df/d5a 0 2024-09-16T23:48:49.178 INFO:tasks.workunit.client.1.smithi043.stdout:3/233: stat d4/de/f2a 0 2024-09-16T23:48:49.179 INFO:tasks.workunit.client.1.smithi043.stdout:0/291: dread - d0/dc/f50 zero size 2024-09-16T23:48:49.181 INFO:tasks.workunit.client.1.smithi043.stdout:6/313: creat de/d23/d49/f6a x:0 0 0 2024-09-16T23:48:49.182 INFO:tasks.workunit.client.1.smithi043.stdout:0/292: dread - d0/d2/df/d1a/d3b/f4e zero size 2024-09-16T23:48:49.183 INFO:tasks.workunit.client.1.smithi043.stdout:3/234: truncate d4/de/d1f/d2c/f32 878082 0 2024-09-16T23:48:49.185 INFO:tasks.workunit.client.1.smithi043.stdout:5/439: rename d3/db/d19/d3e/d28/c40 to d3/d9/c89 0 2024-09-16T23:48:49.186 INFO:tasks.workunit.client.1.smithi043.stdout:8/289: dwrite de/f23 [0,4194304] 0 2024-09-16T23:48:49.188 INFO:tasks.workunit.client.1.smithi043.stdout:1/397: rmdir d4 39 2024-09-16T23:48:49.188 INFO:tasks.workunit.client.1.smithi043.stdout:8/290: truncate de/f17 1178799 0 2024-09-16T23:48:49.190 INFO:tasks.workunit.client.1.smithi043.stdout:2/204: mkdir d8/d4b 0 2024-09-16T23:48:49.191 INFO:tasks.workunit.client.1.smithi043.stdout:8/291: chown de/d18/f21 1600 1 2024-09-16T23:48:49.192 INFO:tasks.workunit.client.1.smithi043.stdout:6/314: mknod de/d23/d49/c6b 0 2024-09-16T23:48:49.196 INFO:tasks.workunit.client.1.smithi043.stdout:3/235: rmdir d4/de/d1f/d2c 39 2024-09-16T23:48:49.201 INFO:tasks.workunit.client.1.smithi043.stdout:3/236: chown d4/d30 1841162 1 2024-09-16T23:48:49.201 INFO:tasks.workunit.client.1.smithi043.stdout:5/440: dwrite d3/db/d15/f3c [0,4194304] 0 2024-09-16T23:48:49.201 INFO:tasks.workunit.client.1.smithi043.stdout:9/225: rename d4/l14 to d4/d28/d2e/l43 0 2024-09-16T23:48:49.207 INFO:tasks.workunit.client.1.smithi043.stdout:0/293: symlink d0/d2/df/l5b 0 2024-09-16T23:48:49.208 INFO:tasks.workunit.client.1.smithi043.stdout:5/441: stat d3/db/d19/d3e/d28/c55 0 2024-09-16T23:48:49.210 INFO:tasks.workunit.client.1.smithi043.stdout:4/296: dwrite d8/d25/f42 [0,4194304] 0 2024-09-16T23:48:49.210 INFO:tasks.workunit.client.1.smithi043.stdout:3/237: creat d4/d30/f41 x:0 0 0 2024-09-16T23:48:49.216 INFO:tasks.workunit.client.1.smithi043.stdout:2/205: rename d8/d1d/d20/d26/c31 to d8/d1d/d20/d23/d45/c4c 0 2024-09-16T23:48:49.220 INFO:tasks.workunit.client.1.smithi043.stdout:2/206: read - d8/d1d/d20/f48 zero size 2024-09-16T23:48:49.223 INFO:tasks.workunit.client.1.smithi043.stdout:0/294: mknod d0/d2/dd/c5c 0 2024-09-16T23:48:49.226 INFO:tasks.workunit.client.1.smithi043.stdout:8/292: dread de/d10/f1b [0,4194304] 0 2024-09-16T23:48:49.226 INFO:tasks.workunit.client.1.smithi043.stdout:2/207: read - d8/fd zero size 2024-09-16T23:48:49.226 INFO:tasks.workunit.client.1.smithi043.stdout:0/295: chown d0/d2/df/f18 0 1 2024-09-16T23:48:49.230 INFO:tasks.workunit.client.1.smithi043.stdout:1/398: creat d4/d9/f84 x:0 0 0 2024-09-16T23:48:49.230 INFO:tasks.workunit.client.1.smithi043.stdout:6/315: dread de/d1a/d39/f17 [0,4194304] 0 2024-09-16T23:48:49.231 INFO:tasks.workunit.client.1.smithi043.stdout:5/442: truncate d3/f38 199201 0 2024-09-16T23:48:49.231 INFO:tasks.workunit.client.0.smithi042.stdout:2/88: sync 2024-09-16T23:48:49.232 INFO:tasks.workunit.client.1.smithi043.stdout:9/226: mknod d4/d1c/d25/c44 0 2024-09-16T23:48:49.236 INFO:tasks.workunit.client.1.smithi043.stdout:3/238: dwrite d4/f23 [0,4194304] 0 2024-09-16T23:48:49.238 INFO:tasks.workunit.client.1.smithi043.stdout:9/227: mkdir d4/d1c/d41/d45 0 2024-09-16T23:48:49.240 INFO:tasks.workunit.client.1.smithi043.stdout:0/296: creat d0/d2/df/d1a/d3b/f5d x:0 0 0 2024-09-16T23:48:49.242 INFO:tasks.workunit.client.1.smithi043.stdout:5/443: mknod d3/db/d15/d4d/c8a 0 2024-09-16T23:48:49.242 INFO:tasks.workunit.client.1.smithi043.stdout:6/316: unlink de/d1a/d28/f35 0 2024-09-16T23:48:49.244 INFO:tasks.workunit.client.1.smithi043.stdout:4/297: dread d8/f31 [0,4194304] 0 2024-09-16T23:48:49.247 INFO:tasks.workunit.client.1.smithi043.stdout:6/317: write de/d1a/d39/f16 [677241,47948] 0 2024-09-16T23:48:49.247 INFO:tasks.workunit.client.1.smithi043.stdout:9/228: write d4/d5/d30/f39 [892059,62847] 0 2024-09-16T23:48:49.252 INFO:tasks.workunit.client.1.smithi043.stdout:0/297: creat d0/d2/df/d1a/d46/f5e x:0 0 0 2024-09-16T23:48:49.254 INFO:tasks.workunit.client.1.smithi043.stdout:8/293: rename de/d18/d1f/d2f/f3e to de/d18/d1f/f59 0 2024-09-16T23:48:49.254 INFO:tasks.workunit.client.1.smithi043.stdout:4/298: chown d8/d15/d20/d2b/d23/d27/c34 105187851 1 2024-09-16T23:48:49.254 INFO:tasks.workunit.client.1.smithi043.stdout:0/298: mknod d0/de/c5f 0 2024-09-16T23:48:49.256 INFO:tasks.workunit.client.1.smithi043.stdout:3/239: rename d4/de to d4/de/d1f/d34/d42 22 2024-09-16T23:48:49.257 INFO:tasks.workunit.client.1.smithi043.stdout:1/399: dwrite d4/d9/f56 [0,4194304] 0 2024-09-16T23:48:49.260 INFO:tasks.workunit.client.1.smithi043.stdout:3/240: read d4/f33 [2933647,60814] 0 2024-09-16T23:48:49.264 INFO:tasks.workunit.client.1.smithi043.stdout:9/229: creat d4/d5/f46 x:0 0 0 2024-09-16T23:48:49.266 INFO:tasks.workunit.client.1.smithi043.stdout:5/444: creat d3/db/d19/d3e/f8b x:0 0 0 2024-09-16T23:48:49.268 INFO:tasks.workunit.client.1.smithi043.stdout:4/299: unlink d8/f31 0 2024-09-16T23:48:49.276 INFO:tasks.workunit.client.1.smithi043.stdout:0/299: dread d0/d2/df/f17 [0,4194304] 0 2024-09-16T23:48:49.276 INFO:tasks.workunit.client.1.smithi043.stdout:8/294: rename de/d10 to de/d18/d27/d41/d5a 0 2024-09-16T23:48:49.277 INFO:tasks.workunit.client.1.smithi043.stdout:2/208: dread d8/d1d/d20/d26/f3f [0,4194304] 0 2024-09-16T23:48:49.277 INFO:tasks.workunit.client.1.smithi043.stdout:8/295: fdatasync de/f54 0 2024-09-16T23:48:49.277 INFO:tasks.workunit.client.1.smithi043.stdout:4/300: write d8/d15/d20/d2b/d22/f24 [5811818,127543] 0 2024-09-16T23:48:49.280 INFO:tasks.workunit.client.1.smithi043.stdout:5/445: mkdir d3/d9/d22/d4a/d4c/d8c 0 2024-09-16T23:48:49.280 INFO:tasks.workunit.client.1.smithi043.stdout:3/241: dwrite d4/f10 [4194304,4194304] 0 2024-09-16T23:48:49.281 INFO:tasks.workunit.client.1.smithi043.stdout:6/318: creat de/f6c x:0 0 0 2024-09-16T23:48:49.283 INFO:tasks.workunit.client.1.smithi043.stdout:0/300: rename d0/dc/f53 to d0/d2/dd/f60 0 2024-09-16T23:48:49.289 INFO:tasks.workunit.client.1.smithi043.stdout:6/319: write de/f24 [1525314,86466] 0 2024-09-16T23:48:49.290 INFO:tasks.workunit.client.1.smithi043.stdout:1/400: dwrite d4/d9/d1c/d1e/f66 [0,4194304] 0 2024-09-16T23:48:49.291 INFO:tasks.workunit.client.1.smithi043.stdout:6/320: chown de/d1a/f42 247930 1 2024-09-16T23:48:49.293 INFO:tasks.workunit.client.1.smithi043.stdout:5/446: unlink d3/db/c34 0 2024-09-16T23:48:49.294 INFO:tasks.workunit.client.1.smithi043.stdout:3/242: symlink d4/d30/l43 0 2024-09-16T23:48:49.296 INFO:tasks.workunit.client.1.smithi043.stdout:0/301: truncate d0/d2/dd/f60 589542 0 2024-09-16T23:48:49.298 INFO:tasks.workunit.client.1.smithi043.stdout:0/302: dread - d0/d2/df/d1a/f47 zero size 2024-09-16T23:48:49.299 INFO:tasks.workunit.client.1.smithi043.stdout:1/401: fsync d4/d9/d12/d15/f31 0 2024-09-16T23:48:49.300 INFO:tasks.workunit.client.1.smithi043.stdout:6/321: rmdir de/d23/d63 39 2024-09-16T23:48:49.306 INFO:tasks.workunit.client.1.smithi043.stdout:5/447: dread d3/d9/d22/f61 [0,4194304] 0 2024-09-16T23:48:49.309 INFO:tasks.workunit.client.1.smithi043.stdout:8/296: dwrite de/d18/d27/d41/d5a/f1b [0,4194304] 0 2024-09-16T23:48:49.309 INFO:tasks.workunit.client.1.smithi043.stdout:1/402: rmdir d4/d34 39 2024-09-16T23:48:49.311 INFO:tasks.workunit.client.1.smithi043.stdout:9/230: write d4/f24 [779109,73649] 0 2024-09-16T23:48:49.312 INFO:tasks.workunit.client.1.smithi043.stdout:3/243: dwrite d4/f19 [0,4194304] 0 2024-09-16T23:48:49.316 INFO:tasks.workunit.client.1.smithi043.stdout:6/322: symlink de/d1a/d4d/d50/l6d 0 2024-09-16T23:48:49.316 INFO:tasks.workunit.client.1.smithi043.stdout:9/231: rename d4/d5 to d4/d5/d30/d47 22 2024-09-16T23:48:49.316 INFO:tasks.workunit.client.1.smithi043.stdout:2/209: link d8/d1d/d20/d23/d39/d3a/l41 d8/d1d/d20/d23/d38/l4d 0 2024-09-16T23:48:49.322 INFO:tasks.workunit.client.1.smithi043.stdout:0/303: unlink d0/de/f15 0 2024-09-16T23:48:49.323 INFO:tasks.workunit.client.1.smithi043.stdout:0/304: readlink d0/d2/df/l5b 0 2024-09-16T23:48:49.325 INFO:tasks.workunit.client.1.smithi043.stdout:4/301: truncate d8/d15/d20/d2b/f26 4003427 0 2024-09-16T23:48:49.326 INFO:tasks.workunit.client.1.smithi043.stdout:5/448: mkdir d3/db/d15/d4d/d8d 0 2024-09-16T23:48:49.331 INFO:tasks.workunit.client.1.smithi043.stdout:3/244: dwrite d4/f19 [4194304,4194304] 0 2024-09-16T23:48:49.335 INFO:tasks.workunit.client.1.smithi043.stdout:2/210: write d8/f1a [1494356,39253] 0 2024-09-16T23:48:49.335 INFO:tasks.workunit.client.1.smithi043.stdout:8/297: dwrite de/d18/d27/d34/d3d/f51 [0,4194304] 0 2024-09-16T23:48:49.344 INFO:tasks.workunit.client.1.smithi043.stdout:5/449: mkdir d3/d9/d22/d8e 0 2024-09-16T23:48:49.345 INFO:tasks.workunit.client.1.smithi043.stdout:5/450: fsync d3/db/d19/f77 0 2024-09-16T23:48:49.345 INFO:tasks.workunit.client.1.smithi043.stdout:9/232: dread d4/f31 [0,4194304] 0 2024-09-16T23:48:49.346 INFO:tasks.workunit.client.1.smithi043.stdout:2/211: symlink d8/d1d/d20/d23/d38/l4e 0 2024-09-16T23:48:49.346 INFO:tasks.workunit.client.1.smithi043.stdout:1/403: dread d4/d34/f7b [0,4194304] 0 2024-09-16T23:48:49.349 INFO:tasks.workunit.client.1.smithi043.stdout:3/245: creat d4/de/d1f/d34/f44 x:0 0 0 2024-09-16T23:48:49.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/60: symlink d3/l13 0 2024-09-16T23:48:49.357 INFO:tasks.workunit.client.1.smithi043.stdout:0/305: rmdir d0/d2/dd/d4f 0 2024-09-16T23:48:49.358 INFO:tasks.workunit.client.0.smithi042.stdout:1/136: mkdir d0/d9/d26/d2c 0 2024-09-16T23:48:49.359 INFO:tasks.workunit.client.0.smithi042.stdout:0/38: creat d0/fa x:0 0 0 2024-09-16T23:48:49.360 INFO:tasks.workunit.client.0.smithi042.stdout:6/78: creat d8/f18 x:0 0 0 2024-09-16T23:48:49.361 INFO:tasks.workunit.client.0.smithi042.stdout:6/79: fdatasync d8/db/fc 0 2024-09-16T23:48:49.361 INFO:tasks.workunit.client.0.smithi042.stdout:2/89: creat d0/da/f21 x:0 0 0 2024-09-16T23:48:49.362 INFO:tasks.workunit.client.0.smithi042.stdout:6/80: chown d8/db/dd/f11 157113 1 2024-09-16T23:48:49.363 INFO:tasks.workunit.client.1.smithi043.stdout:4/302: write d8/d15/d20/d2b/d54/d55/f2a [1610701,130205] 0 2024-09-16T23:48:49.363 INFO:tasks.workunit.client.1.smithi043.stdout:1/404: write d4/d34/f3a [854746,24235] 0 2024-09-16T23:48:49.363 INFO:tasks.workunit.client.0.smithi042.stdout:2/90: read d0/ff [458121,115239] 0 2024-09-16T23:48:49.364 INFO:tasks.workunit.client.1.smithi043.stdout:0/306: creat d0/d2/df/d1a/d46/d39/f61 x:0 0 0 2024-09-16T23:48:49.364 INFO:tasks.workunit.client.0.smithi042.stdout:9/65: mkdir d4/d12/d16 0 2024-09-16T23:48:49.365 INFO:tasks.workunit.client.0.smithi042.stdout:7/61: mknod d3/c14 0 2024-09-16T23:48:49.366 INFO:tasks.workunit.client.0.smithi042.stdout:1/137: creat d0/d17/f2d x:0 0 0 2024-09-16T23:48:49.366 INFO:tasks.workunit.client.1.smithi043.stdout:0/307: dread d0/d2/dd/f60 [0,4194304] 0 2024-09-16T23:48:49.367 INFO:tasks.workunit.client.0.smithi042.stdout:0/39: unlink d0/f5 0 2024-09-16T23:48:49.368 INFO:tasks.workunit.client.0.smithi042.stdout:3/58: getdents d2/d9 0 2024-09-16T23:48:49.368 INFO:tasks.workunit.client.0.smithi042.stdout:0/40: read - d0/f9 zero size 2024-09-16T23:48:49.371 INFO:tasks.workunit.client.0.smithi042.stdout:9/66: unlink c1 0 2024-09-16T23:48:49.372 INFO:tasks.workunit.client.1.smithi043.stdout:2/212: dread d8/ff [0,4194304] 0 2024-09-16T23:48:49.373 INFO:tasks.workunit.client.1.smithi043.stdout:9/233: creat d4/f48 x:0 0 0 2024-09-16T23:48:49.374 INFO:tasks.workunit.client.1.smithi043.stdout:6/323: write de/d1a/d28/f3e [2215158,118309] 0 2024-09-16T23:48:49.377 INFO:tasks.workunit.client.1.smithi043.stdout:1/405: creat d4/d9/d1c/d1e/f85 x:0 0 0 2024-09-16T23:48:49.378 INFO:tasks.workunit.client.0.smithi042.stdout:1/138: dwrite d0/d9/d26/f27 [0,4194304] 0 2024-09-16T23:48:49.378 INFO:tasks.workunit.client.1.smithi043.stdout:2/213: write d8/d11/f4a [603719,126800] 0 2024-09-16T23:48:49.379 INFO:tasks.workunit.client.1.smithi043.stdout:5/451: write d3/db/d19/f54 [202810,79855] 0 2024-09-16T23:48:49.379 INFO:tasks.workunit.client.1.smithi043.stdout:0/308: creat d0/d2/df/d1a/d46/d39/f62 x:0 0 0 2024-09-16T23:48:49.380 INFO:tasks.workunit.client.0.smithi042.stdout:3/59: read f0 [440612,112661] 0 2024-09-16T23:48:49.381 INFO:tasks.workunit.client.0.smithi042.stdout:0/41: creat d0/fb x:0 0 0 2024-09-16T23:48:49.383 INFO:tasks.workunit.client.1.smithi043.stdout:9/234: chown d4/f11 23865 1 2024-09-16T23:48:49.383 INFO:tasks.workunit.client.0.smithi042.stdout:6/81: link d8/db/fc d8/db/dd/f19 0 2024-09-16T23:48:49.391 INFO:tasks.workunit.client.1.smithi043.stdout:0/309: dread - d0/d2/df/d1a/d46/d39/f61 zero size 2024-09-16T23:48:49.391 INFO:tasks.workunit.client.1.smithi043.stdout:4/303: getdents d8/d15/d20 0 2024-09-16T23:48:49.396 INFO:tasks.workunit.client.1.smithi043.stdout:8/298: dwrite de/d18/d27/d41/f50 [0,4194304] 0 2024-09-16T23:48:49.402 INFO:tasks.workunit.client.1.smithi043.stdout:2/214: dwrite d8/fd [0,4194304] 0 2024-09-16T23:48:49.406 INFO:tasks.workunit.client.1.smithi043.stdout:8/299: chown de/d18/d1f/l48 75 1 2024-09-16T23:48:49.407 INFO:tasks.workunit.client.1.smithi043.stdout:4/304: rename d8/d15/d20/d4d to d8/d25/d39/d58 0 2024-09-16T23:48:49.409 INFO:tasks.workunit.client.1.smithi043.stdout:9/235: mkdir d4/d28/d49 0 2024-09-16T23:48:49.409 INFO:tasks.workunit.client.1.smithi043.stdout:5/452: dread d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:49.412 INFO:tasks.workunit.client.1.smithi043.stdout:1/406: dwrite d4/d34/f7b [0,4194304] 0 2024-09-16T23:48:49.416 INFO:tasks.workunit.client.1.smithi043.stdout:9/236: truncate d4/f48 643853 0 2024-09-16T23:48:49.416 INFO:tasks.workunit.client.1.smithi043.stdout:4/305: write d8/d15/f21 [8945497,40318] 0 2024-09-16T23:48:49.416 INFO:tasks.workunit.client.1.smithi043.stdout:5/453: readlink d3/db/d19/d76/l86 0 2024-09-16T23:48:49.419 INFO:tasks.workunit.client.1.smithi043.stdout:2/215: mkdir d8/d4b/d4f 0 2024-09-16T23:48:49.421 INFO:tasks.workunit.client.1.smithi043.stdout:5/454: read - d3/d9/d22/d4a/d4c/f72 zero size 2024-09-16T23:48:49.425 INFO:tasks.workunit.client.1.smithi043.stdout:6/324: truncate de/d1a/f32 1284910 0 2024-09-16T23:48:49.427 INFO:tasks.workunit.client.1.smithi043.stdout:1/407: link d4/d9/d12/d2b/f52 d4/d9/d1c/d1e/d47/f86 0 2024-09-16T23:48:49.428 INFO:tasks.workunit.client.1.smithi043.stdout:4/306: mknod d8/d15/d20/d2b/d54/d55/d49/d4c/c59 0 2024-09-16T23:48:49.429 INFO:tasks.workunit.client.1.smithi043.stdout:2/216: dread d8/d11/d16/f28 [0,4194304] 0 2024-09-16T23:48:49.430 INFO:tasks.workunit.client.1.smithi043.stdout:1/408: dread - d4/df/d13/f57 zero size 2024-09-16T23:48:49.430 INFO:tasks.workunit.client.1.smithi043.stdout:4/307: chown d8/d25/f42 961 1 2024-09-16T23:48:49.430 INFO:tasks.workunit.client.1.smithi043.stdout:9/237: rename d4/d28/d2e/l19 to d4/d21/d33/l4a 0 2024-09-16T23:48:49.431 INFO:tasks.workunit.client.1.smithi043.stdout:1/409: dread - d4/d9/f84 zero size 2024-09-16T23:48:49.434 INFO:tasks.workunit.client.1.smithi043.stdout:2/217: write d8/d11/d16/f49 [789931,106613] 0 2024-09-16T23:48:49.435 INFO:tasks.workunit.client.1.smithi043.stdout:4/308: chown d8/d15/d20/d2b/d54/d55 93780932 1 2024-09-16T23:48:49.438 INFO:tasks.workunit.client.1.smithi043.stdout:5/455: rename d3/db/d15/d4d/d8d to d3/d9/d22/d4a/d64/d8f 0 2024-09-16T23:48:49.441 INFO:tasks.workunit.client.1.smithi043.stdout:8/300: write de/d18/d27/d41/d5a/f56 [3852916,73421] 0 2024-09-16T23:48:49.444 INFO:tasks.workunit.client.0.smithi042.stdout:3/60: unlink d2/fb 0 2024-09-16T23:48:49.444 INFO:tasks.workunit.client.1.smithi043.stdout:6/325: creat de/d23/d34/d66/f6e x:0 0 0 2024-09-16T23:48:49.449 INFO:tasks.workunit.client.0.smithi042.stdout:6/82: symlink d8/l1a 0 2024-09-16T23:48:49.449 INFO:tasks.workunit.client.0.smithi042.stdout:1/139: dwrite d0/d9/df/f12 [0,4194304] 0 2024-09-16T23:48:49.449 INFO:tasks.workunit.client.0.smithi042.stdout:6/83: fdatasync f3 0 2024-09-16T23:48:49.451 INFO:tasks.workunit.client.0.smithi042.stdout:6/84: write d8/f18 [248399,105884] 0 2024-09-16T23:48:49.453 INFO:tasks.workunit.client.1.smithi043.stdout:9/238: dwrite d4/d1c/f27 [0,4194304] 0 2024-09-16T23:48:49.453 INFO:tasks.workunit.client.1.smithi043.stdout:2/218: symlink d8/d1d/d20/d23/d38/l50 0 2024-09-16T23:48:49.455 INFO:tasks.workunit.client.1.smithi043.stdout:9/239: write d4/f48 [529654,78762] 0 2024-09-16T23:48:49.459 INFO:tasks.workunit.client.0.smithi042.stdout:1/140: dwrite d0/d17/f21 [0,4194304] 0 2024-09-16T23:48:49.464 INFO:tasks.workunit.client.1.smithi043.stdout:1/410: dwrite d4/d9/d12/d2b/f52 [0,4194304] 0 2024-09-16T23:48:49.464 INFO:tasks.workunit.client.1.smithi043.stdout:6/326: dread de/d1a/f38 [0,4194304] 0 2024-09-16T23:48:49.469 INFO:tasks.workunit.client.1.smithi043.stdout:5/456: dwrite d3/d9/f4b [0,4194304] 0 2024-09-16T23:48:49.469 INFO:tasks.workunit.client.1.smithi043.stdout:6/327: read de/d1a/d39/f17 [3012479,40823] 0 2024-09-16T23:48:49.478 INFO:tasks.workunit.client.1.smithi043.stdout:6/328: dread de/d1a/f41 [0,4194304] 0 2024-09-16T23:48:49.478 INFO:tasks.workunit.client.0.smithi042.stdout:5/98: sync 2024-09-16T23:48:49.479 INFO:tasks.workunit.client.0.smithi042.stdout:6/85: mkdir d8/db/dd/d1b 0 2024-09-16T23:48:49.480 INFO:tasks.workunit.client.1.smithi043.stdout:8/301: dread de/d18/f36 [0,4194304] 0 2024-09-16T23:48:49.480 INFO:tasks.workunit.client.0.smithi042.stdout:6/86: readlink l4 0 2024-09-16T23:48:49.481 INFO:tasks.workunit.client.0.smithi042.stdout:5/99: dread - d9/db/f16 zero size 2024-09-16T23:48:49.487 INFO:tasks.workunit.client.1.smithi043.stdout:4/309: rmdir d8/d15/d20/d2b/d22/d50 0 2024-09-16T23:48:49.487 INFO:tasks.workunit.client.1.smithi043.stdout:2/219: rename d8/d1d/d20/d23/l42 to d8/d1d/l51 0 2024-09-16T23:48:49.488 INFO:tasks.workunit.client.0.smithi042.stdout:6/87: write d8/db/dd/f19 [222216,33802] 0 2024-09-16T23:48:49.489 INFO:tasks.workunit.client.0.smithi042.stdout:5/100: creat d9/db/d11/f1c x:0 0 0 2024-09-16T23:48:49.491 INFO:tasks.workunit.client.1.smithi043.stdout:5/457: mknod d3/d9/d22/d4a/d4c/c90 0 2024-09-16T23:48:49.492 INFO:tasks.workunit.client.0.smithi042.stdout:6/88: creat d8/db/dd/f1c x:0 0 0 2024-09-16T23:48:49.492 INFO:tasks.workunit.client.1.smithi043.stdout:8/302: dwrite de/d18/d27/d41/d5a/f26 [4194304,4194304] 0 2024-09-16T23:48:49.494 INFO:tasks.workunit.client.0.smithi042.stdout:6/89: fdatasync d8/db/fc 0 2024-09-16T23:48:49.498 INFO:tasks.workunit.client.1.smithi043.stdout:2/220: creat d8/d1d/f52 x:0 0 0 2024-09-16T23:48:49.499 INFO:tasks.workunit.client.1.smithi043.stdout:6/329: creat de/d23/d63/f6f x:0 0 0 2024-09-16T23:48:49.499 INFO:tasks.workunit.client.1.smithi043.stdout:8/303: symlink de/d18/d27/d41/d5a/d37/l5b 0 2024-09-16T23:48:49.501 INFO:tasks.workunit.client.1.smithi043.stdout:5/458: mkdir d3/d79/d91 0 2024-09-16T23:48:49.502 INFO:tasks.workunit.client.1.smithi043.stdout:8/304: read de/d18/d2b/f2e [37167,122800] 0 2024-09-16T23:48:49.502 INFO:tasks.workunit.client.1.smithi043.stdout:1/411: link d4/df/d13/c48 d4/d9/c87 0 2024-09-16T23:48:49.502 INFO:tasks.workunit.client.1.smithi043.stdout:2/221: write d8/d35/f37 [641063,31845] 0 2024-09-16T23:48:49.503 INFO:tasks.workunit.client.0.smithi042.stdout:5/101: getdents d9/db/de 0 2024-09-16T23:48:49.505 INFO:tasks.workunit.client.1.smithi043.stdout:8/305: write de/d18/d1f/d25/f3b [417565,116622] 0 2024-09-16T23:48:49.506 INFO:tasks.workunit.client.0.smithi042.stdout:5/102: chown d9/ca 772455534 1 2024-09-16T23:48:49.508 INFO:tasks.workunit.client.1.smithi043.stdout:8/306: fdatasync de/d18/d27/d41/d5a/f13 0 2024-09-16T23:48:49.509 INFO:tasks.workunit.client.1.smithi043.stdout:9/240: truncate d4/d1c/f20 1297265 0 2024-09-16T23:48:49.513 INFO:tasks.workunit.client.1.smithi043.stdout:2/222: rename d8/d1d/d20/d23/d39/d3a/l41 to d8/d11/d16/l53 0 2024-09-16T23:48:49.515 INFO:tasks.workunit.client.1.smithi043.stdout:8/307: symlink de/d18/d27/d34/l5c 0 2024-09-16T23:48:49.516 INFO:tasks.workunit.client.0.smithi042.stdout:5/103: dwrite d9/db/de/d10/f1a [0,4194304] 0 2024-09-16T23:48:49.517 INFO:tasks.workunit.client.0.smithi042.stdout:5/104: stat d9/db/d19 0 2024-09-16T23:48:49.518 INFO:tasks.workunit.client.0.smithi042.stdout:5/105: fsync d9/db/f16 0 2024-09-16T23:48:49.519 INFO:tasks.workunit.client.1.smithi043.stdout:1/412: dwrite d4/df/d13/d1b/f59 [0,4194304] 0 2024-09-16T23:48:49.520 INFO:tasks.workunit.client.1.smithi043.stdout:6/330: write de/d1a/f38 [1236919,71989] 0 2024-09-16T23:48:49.523 INFO:tasks.workunit.client.1.smithi043.stdout:9/241: dwrite d4/f10 [0,4194304] 0 2024-09-16T23:48:49.530 INFO:tasks.workunit.client.0.smithi042.stdout:5/106: dwrite d9/db/f16 [0,4194304] 0 2024-09-16T23:48:49.533 INFO:tasks.workunit.client.1.smithi043.stdout:8/308: creat de/d18/d1f/d2f/d39/d4d/f5d x:0 0 0 2024-09-16T23:48:49.535 INFO:tasks.workunit.client.1.smithi043.stdout:8/309: readlink de/d18/d1f/l30 0 2024-09-16T23:48:49.535 INFO:tasks.workunit.client.0.smithi042.stdout:5/107: stat d9/db/de/d10 0 2024-09-16T23:48:49.537 INFO:tasks.workunit.client.1.smithi043.stdout:9/242: rmdir d4/d21/d2a 39 2024-09-16T23:48:49.537 INFO:tasks.workunit.client.1.smithi043.stdout:9/243: chown d4/d1c 0 1 2024-09-16T23:48:49.538 INFO:tasks.workunit.client.1.smithi043.stdout:9/244: stat d4/d21/d33 0 2024-09-16T23:48:49.540 INFO:tasks.workunit.client.0.smithi042.stdout:5/108: symlink d9/db/d19/l1d 0 2024-09-16T23:48:49.541 INFO:tasks.workunit.client.1.smithi043.stdout:6/331: fdatasync de/d1a/d4d/d50/f61 0 2024-09-16T23:48:49.542 INFO:tasks.workunit.client.1.smithi043.stdout:8/310: dread de/d18/d27/d34/d3d/f51 [0,4194304] 0 2024-09-16T23:48:49.544 INFO:tasks.workunit.client.0.smithi042.stdout:5/109: mknod d9/db/d19/c1e 0 2024-09-16T23:48:49.546 INFO:tasks.workunit.client.0.smithi042.stdout:5/110: write d9/db/f16 [4019415,86100] 0 2024-09-16T23:48:49.546 INFO:tasks.workunit.client.1.smithi043.stdout:2/223: dread d8/f24 [0,4194304] 0 2024-09-16T23:48:49.547 INFO:tasks.workunit.client.1.smithi043.stdout:6/332: chown de/d1a/d28 6 1 2024-09-16T23:48:49.549 INFO:tasks.workunit.client.1.smithi043.stdout:1/413: link d4/df/d13/d1b/l77 d4/d9/d1c/d1e/d42/d5d/d71/d76/l88 0 2024-09-16T23:48:49.551 INFO:tasks.workunit.client.1.smithi043.stdout:1/414: readlink d4/d9/d1c/d1e/d47/l49 0 2024-09-16T23:48:49.552 INFO:tasks.workunit.client.1.smithi043.stdout:6/333: rmdir de/d23/d63 39 2024-09-16T23:48:49.552 INFO:tasks.workunit.client.1.smithi043.stdout:8/311: fdatasync de/d18/d1f/d2f/d39/d4d/f5d 0 2024-09-16T23:48:49.554 INFO:tasks.workunit.client.0.smithi042.stdout:5/111: dwrite d9/db/f16 [0,4194304] 0 2024-09-16T23:48:49.555 INFO:tasks.workunit.client.1.smithi043.stdout:2/224: mknod d8/d1d/d25/c54 0 2024-09-16T23:48:49.556 INFO:tasks.workunit.client.1.smithi043.stdout:9/245: dwrite d4/d21/d33/f34 [0,4194304] 0 2024-09-16T23:48:49.558 INFO:tasks.workunit.client.1.smithi043.stdout:6/334: write de/f67 [979240,124636] 0 2024-09-16T23:48:49.559 INFO:tasks.workunit.client.0.smithi042.stdout:5/112: symlink d9/db/d17/l1f 0 2024-09-16T23:48:49.565 INFO:tasks.workunit.client.1.smithi043.stdout:6/335: readlink de/d23/d63/l65 0 2024-09-16T23:48:49.567 INFO:tasks.workunit.client.1.smithi043.stdout:8/312: dwrite de/ff [0,4194304] 0 2024-09-16T23:48:49.567 INFO:tasks.workunit.client.0.smithi042.stdout:5/113: getdents d9/db/de/d10 0 2024-09-16T23:48:49.568 INFO:tasks.workunit.client.1.smithi043.stdout:1/415: read d4/df/d13/d1b/f25 [8203765,113197] 0 2024-09-16T23:48:49.570 INFO:tasks.workunit.client.1.smithi043.stdout:2/225: creat d8/d1d/d20/d23/d39/f55 x:0 0 0 2024-09-16T23:48:49.572 INFO:tasks.workunit.client.1.smithi043.stdout:1/416: symlink d4/d9/d1c/l89 0 2024-09-16T23:48:49.573 INFO:tasks.workunit.client.1.smithi043.stdout:9/246: write d4/d21/d33/f34 [4610467,45899] 0 2024-09-16T23:48:49.578 INFO:tasks.workunit.client.1.smithi043.stdout:2/226: truncate d8/d1d/f52 837964 0 2024-09-16T23:48:49.582 INFO:tasks.workunit.client.1.smithi043.stdout:1/417: rename d4/df/d13/d1b to d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a 0 2024-09-16T23:48:49.583 INFO:tasks.workunit.client.1.smithi043.stdout:8/313: getdents de/d18/d27/d41 0 2024-09-16T23:48:49.584 INFO:tasks.workunit.client.1.smithi043.stdout:9/247: chown d4/f1a 0 1 2024-09-16T23:48:49.587 INFO:tasks.workunit.client.1.smithi043.stdout:1/418: unlink d4/d9/f56 0 2024-09-16T23:48:49.589 INFO:tasks.workunit.client.1.smithi043.stdout:8/314: symlink de/d18/d1f/d2f/d39/d4d/l5e 0 2024-09-16T23:48:49.589 INFO:tasks.workunit.client.1.smithi043.stdout:8/315: fdatasync fc 0 2024-09-16T23:48:49.590 INFO:tasks.workunit.client.1.smithi043.stdout:2/227: mkdir d8/d4b/d4f/d56 0 2024-09-16T23:48:49.600 INFO:tasks.workunit.client.1.smithi043.stdout:8/316: rename de/ff to de/d18/d1f/d25/f5f 0 2024-09-16T23:48:49.602 INFO:tasks.workunit.client.1.smithi043.stdout:8/317: chown de/d18/d1f/d25/l49 109217694 1 2024-09-16T23:48:49.603 INFO:tasks.workunit.client.1.smithi043.stdout:1/419: write d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f59 [4180378,89581] 0 2024-09-16T23:48:49.606 INFO:tasks.workunit.client.1.smithi043.stdout:8/318: creat de/d18/d27/d41/d5a/d37/f60 x:0 0 0 2024-09-16T23:48:49.607 INFO:tasks.workunit.client.1.smithi043.stdout:2/228: link d8/d1f/f30 d8/d1d/d20/d23/d39/d3a/f57 0 2024-09-16T23:48:49.611 INFO:tasks.workunit.client.1.smithi043.stdout:2/229: creat d8/d1d/d20/f58 x:0 0 0 2024-09-16T23:48:49.612 INFO:tasks.workunit.client.1.smithi043.stdout:1/420: link d4/d34/c6d d4/d9/d12/d15/c8b 0 2024-09-16T23:48:49.613 INFO:tasks.workunit.client.0.smithi042.stdout:8/189: truncate da/d19/f1f 1174878 0 2024-09-16T23:48:49.614 INFO:tasks.workunit.client.0.smithi042.stdout:8/190: chown da/f2a 186 1 2024-09-16T23:48:49.615 INFO:tasks.workunit.client.1.smithi043.stdout:2/230: read - d8/d1d/d20/d23/d39/f55 zero size 2024-09-16T23:48:49.615 INFO:tasks.workunit.client.1.smithi043.stdout:9/248: dwrite d4/d1c/f20 [0,4194304] 0 2024-09-16T23:48:49.616 INFO:tasks.workunit.client.1.smithi043.stdout:8/319: dread de/d18/f36 [4194304,4194304] 0 2024-09-16T23:48:49.616 INFO:tasks.workunit.client.0.smithi042.stdout:4/79: sync 2024-09-16T23:48:49.617 INFO:tasks.workunit.client.0.smithi042.stdout:8/191: mkdir da/d19/d1d/d30/d3f 0 2024-09-16T23:48:49.618 INFO:tasks.workunit.client.1.smithi043.stdout:9/249: fdatasync d4/d21/d33/f34 0 2024-09-16T23:48:49.619 INFO:tasks.workunit.client.1.smithi043.stdout:8/320: dread - de/d18/d1f/d2f/d39/d4d/f5d zero size 2024-09-16T23:48:49.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/192: mknod da/d19/d1d/d30/d3f/c40 0 2024-09-16T23:48:49.623 INFO:tasks.workunit.client.1.smithi043.stdout:2/231: dread d8/d11/f4a [0,4194304] 0 2024-09-16T23:48:49.625 INFO:tasks.workunit.client.0.smithi042.stdout:4/80: dread de/f10 [0,4194304] 0 2024-09-16T23:48:49.625 INFO:tasks.workunit.client.1.smithi043.stdout:2/232: write d8/d1d/d20/d23/d38/f47 [744090,43336] 0 2024-09-16T23:48:49.626 INFO:tasks.workunit.client.1.smithi043.stdout:8/321: rmdir de/d18/d1f/d2f/d39/d4d 39 2024-09-16T23:48:49.626 INFO:tasks.workunit.client.1.smithi043.stdout:8/322: chown fc 29328673 1 2024-09-16T23:48:49.628 INFO:tasks.workunit.client.0.smithi042.stdout:8/193: creat da/d19/d1d/d30/f41 x:0 0 0 2024-09-16T23:48:49.630 INFO:tasks.workunit.client.0.smithi042.stdout:8/194: write da/d19/d1d/f3e [771413,94918] 0 2024-09-16T23:48:49.631 INFO:tasks.workunit.client.1.smithi043.stdout:1/421: truncate d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f64 3408281 0 2024-09-16T23:48:49.632 INFO:tasks.workunit.client.0.smithi042.stdout:8/195: fsync da/f14 0 2024-09-16T23:48:49.633 INFO:tasks.workunit.client.1.smithi043.stdout:2/233: truncate d8/d1f/f30 3771537 0 2024-09-16T23:48:49.635 INFO:tasks.workunit.client.0.smithi042.stdout:7/62: dwrite d3/fc [0,4194304] 0 2024-09-16T23:48:49.635 INFO:tasks.workunit.client.0.smithi042.stdout:7/63: stat d3/c4 0 2024-09-16T23:48:49.637 INFO:tasks.workunit.client.1.smithi043.stdout:9/250: dwrite d4/d5/f1d [0,4194304] 0 2024-09-16T23:48:49.637 INFO:tasks.workunit.client.1.smithi043.stdout:1/422: write d4/d9/d12/d15/f31 [365882,16001] 0 2024-09-16T23:48:49.639 INFO:tasks.workunit.client.1.smithi043.stdout:7/312: sync 2024-09-16T23:48:49.639 INFO:tasks.workunit.client.0.smithi042.stdout:8/196: symlink da/d19/d2c/l42 0 2024-09-16T23:48:49.640 INFO:tasks.workunit.client.0.smithi042.stdout:8/197: dread - da/d19/d1d/f32 zero size 2024-09-16T23:48:49.641 INFO:tasks.workunit.client.0.smithi042.stdout:7/64: mkdir d3/d15 0 2024-09-16T23:48:49.644 INFO:tasks.workunit.client.1.smithi043.stdout:9/251: dread d4/f10 [0,4194304] 0 2024-09-16T23:48:49.648 INFO:tasks.workunit.client.0.smithi042.stdout:7/65: mkdir d3/d15/d16 0 2024-09-16T23:48:49.648 INFO:tasks.workunit.client.1.smithi043.stdout:1/423: mknod d4/d9/d1c/d1e/d47/c8c 0 2024-09-16T23:48:49.649 INFO:tasks.workunit.client.0.smithi042.stdout:8/198: getdents da/d19/d1d 0 2024-09-16T23:48:49.649 INFO:tasks.workunit.client.1.smithi043.stdout:7/313: dwrite d4/d27/d41/f4c [0,4194304] 0 2024-09-16T23:48:49.652 INFO:tasks.workunit.client.0.smithi042.stdout:9/67: dwrite d4/f5 [4194304,4194304] 0 2024-09-16T23:48:49.652 INFO:tasks.workunit.client.0.smithi042.stdout:8/199: write da/d19/d1d/d30/f41 [122554,51613] 0 2024-09-16T23:48:49.654 INFO:tasks.workunit.client.0.smithi042.stdout:8/200: read da/d19/f1a [3870353,72440] 0 2024-09-16T23:48:49.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.670+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd8e41088f0 con 0x7fd8e4198e60 2024-09-16T23:48:49.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.670+0000 7fd8daffd700 1 -- 172.21.15.42:0/937573044 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fd8d402ac30 con 0x7fd8e4198e60 2024-09-16T23:48:49.668 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:48:49.668 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:48:49.668 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:48:49.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:49.668 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:48:49.668 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:48:49.669 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:48:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fd8d0039190 msgr2=0x7fd8d003b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fd8d0039190 0x7fd8d003b650 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4100c00 tx=0x7fd8e0009260 comp rx=0 tx=0).stop 2024-09-16T23:48:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4198e60 msgr2=0x7fd8e419ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4198e60 0x7fd8e419ded0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4015000 tx=0x7fd8d400f690 comp rx=0 tx=0).stop 2024-09-16T23:48:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 shutdown_connections 2024-09-16T23:48:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fd8d0039190 0x7fd8d003b650 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8e40ffba0 0x7fd8e4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 --2- 172.21.15.42:0/937573044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e4198e60 0x7fd8e419ded0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.672+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 >> 172.21.15.42:0/937573044 conn(0x7fd8e40fb120 msgr2=0x7fd8e4068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.673+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 shutdown_connections 2024-09-16T23:48:49.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.673+0000 7fd8ec099700 1 -- 172.21.15.42:0/937573044 wait complete. 2024-09-16T23:48:49.680 INFO:tasks.workunit.client.1.smithi043.stdout:8/323: truncate de/d18/d27/d41/d5a/f1b 3177527 0 2024-09-16T23:48:49.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/66: symlink d3/l17 0 2024-09-16T23:48:49.690 INFO:tasks.workunit.client.1.smithi043.stdout:9/252: dwrite d4/d5/d16/f1f [0,4194304] 0 2024-09-16T23:48:49.692 INFO:tasks.workunit.client.0.smithi042.stdout:8/201: mknod da/d19/d1d/d30/c43 0 2024-09-16T23:48:49.693 INFO:tasks.workunit.client.0.smithi042.stdout:8/202: chown da/d19/l21 5185 1 2024-09-16T23:48:49.694 INFO:tasks.workunit.client.0.smithi042.stdout:7/67: creat d3/f18 x:0 0 0 2024-09-16T23:48:49.695 INFO:tasks.workunit.client.1.smithi043.stdout:7/314: getdents d4/db/d11/d15/d29 0 2024-09-16T23:48:49.695 INFO:tasks.workunit.client.1.smithi043.stdout:9/253: read d4/d21/d33/f34 [305120,11279] 0 2024-09-16T23:48:49.695 INFO:tasks.workunit.client.0.smithi042.stdout:7/68: read d3/f8 [584931,24368] 0 2024-09-16T23:48:49.695 INFO:tasks.workunit.client.1.smithi043.stdout:1/424: dwrite d4/df/d13/f57 [0,4194304] 0 2024-09-16T23:48:49.697 INFO:tasks.workunit.client.0.smithi042.stdout:9/68: creat d4/d12/d16/f17 x:0 0 0 2024-09-16T23:48:49.698 INFO:tasks.workunit.client.0.smithi042.stdout:8/203: mknod da/d19/d1d/d30/d3f/c44 0 2024-09-16T23:48:49.698 INFO:tasks.workunit.client.1.smithi043.stdout:8/324: dwrite de/d18/d27/d34/d3d/f51 [0,4194304] 0 2024-09-16T23:48:49.699 INFO:tasks.workunit.client.1.smithi043.stdout:1/425: fdatasync d4/d9/f84 0 2024-09-16T23:48:49.700 INFO:tasks.workunit.client.1.smithi043.stdout:1/426: fdatasync d4/d9/d1c/d1e/f23 0 2024-09-16T23:48:49.700 INFO:tasks.workunit.client.0.smithi042.stdout:8/204: write da/f29 [225194,61908] 0 2024-09-16T23:48:49.703 INFO:tasks.workunit.client.1.smithi043.stdout:9/254: write d4/f24 [782378,51848] 0 2024-09-16T23:48:49.706 INFO:tasks.workunit.client.0.smithi042.stdout:7/69: dwrite d3/fa [0,4194304] 0 2024-09-16T23:48:49.707 INFO:tasks.workunit.client.1.smithi043.stdout:7/315: rmdir d4/db/d11 39 2024-09-16T23:48:49.710 INFO:tasks.workunit.client.0.smithi042.stdout:8/205: dread da/f24 [0,4194304] 0 2024-09-16T23:48:49.710 INFO:tasks.workunit.client.1.smithi043.stdout:2/234: truncate d8/d1d/f2a 2930534 0 2024-09-16T23:48:49.711 INFO:tasks.workunit.client.1.smithi043.stdout:1/427: fsync d4/d9/d1c/d1e/f2a 0 2024-09-16T23:48:49.711 INFO:tasks.workunit.client.0.smithi042.stdout:9/69: truncate d4/fb 512512 0 2024-09-16T23:48:49.713 INFO:tasks.workunit.client.0.smithi042.stdout:9/70: write d4/d12/d16/f17 [714764,72735] 0 2024-09-16T23:48:49.717 INFO:tasks.workunit.client.0.smithi042.stdout:7/70: dread d3/fa [0,4194304] 0 2024-09-16T23:48:49.717 INFO:tasks.workunit.client.0.smithi042.stdout:8/206: creat da/d19/d1d/d30/f45 x:0 0 0 2024-09-16T23:48:49.717 INFO:tasks.workunit.client.0.smithi042.stdout:9/71: rename d4/d10/d14 to d4/d10/d14/d18 22 2024-09-16T23:48:49.719 INFO:tasks.workunit.client.1.smithi043.stdout:7/316: creat d4/d27/f6c x:0 0 0 2024-09-16T23:48:49.721 INFO:tasks.workunit.client.0.smithi042.stdout:2/91: truncate d0/da/fd 808505 0 2024-09-16T23:48:49.724 INFO:tasks.workunit.client.1.smithi043.stdout:9/255: mkdir d4/d21/d35/d4b 0 2024-09-16T23:48:49.725 INFO:tasks.workunit.client.0.smithi042.stdout:2/92: chown d0/f11 0 1 2024-09-16T23:48:49.725 INFO:tasks.workunit.client.1.smithi043.stdout:9/256: chown d4/f24 1 1 2024-09-16T23:48:49.726 INFO:tasks.workunit.client.1.smithi043.stdout:7/317: chown d4/db/d11/d15/f5f 23823530 1 2024-09-16T23:48:49.726 INFO:tasks.workunit.client.0.smithi042.stdout:1/141: dwrite d0/d9/d26/f27 [4194304,4194304] 0 2024-09-16T23:48:49.726 INFO:tasks.workunit.client.1.smithi043.stdout:9/257: write d4/d5/f46 [53111,60726] 0 2024-09-16T23:48:49.734 INFO:tasks.workunit.client.0.smithi042.stdout:2/93: write d0/da/f1f [554884,44685] 0 2024-09-16T23:48:49.734 INFO:tasks.workunit.client.0.smithi042.stdout:0/42: truncate d0/f8 3228147 0 2024-09-16T23:48:49.735 INFO:tasks.workunit.client.1.smithi043.stdout:2/235: rename d8/f24 to d8/d1d/d20/d23/d39/f59 0 2024-09-16T23:48:49.736 INFO:tasks.workunit.client.0.smithi042.stdout:1/142: write d0/d9/d26/f27 [3832335,28443] 0 2024-09-16T23:48:49.737 INFO:tasks.workunit.client.0.smithi042.stdout:8/207: dwrite da/d19/f3b [0,4194304] 0 2024-09-16T23:48:49.738 INFO:tasks.workunit.client.1.smithi043.stdout:9/258: mkdir d4/d1c/d25/d2b/d4c 0 2024-09-16T23:48:49.739 INFO:tasks.workunit.client.0.smithi042.stdout:0/43: chown d0/fa 963 1 2024-09-16T23:48:49.742 INFO:tasks.workunit.client.1.smithi043.stdout:2/236: rename d8/d1d/d25/f44 to d8/d1d/f5a 0 2024-09-16T23:48:49.745 INFO:tasks.workunit.client.0.smithi042.stdout:5/114: fsync d9/db/d11/f1c 0 2024-09-16T23:48:49.748 INFO:tasks.workunit.client.0.smithi042.stdout:1/143: dread d0/d9/df/f1e [4194304,4194304] 0 2024-09-16T23:48:49.748 INFO:tasks.workunit.client.0.smithi042.stdout:8/208: fdatasync da/d19/f23 0 2024-09-16T23:48:49.748 INFO:tasks.workunit.client.1.smithi043.stdout:2/237: creat d8/d4b/f5b x:0 0 0 2024-09-16T23:48:49.750 INFO:tasks.workunit.client.0.smithi042.stdout:6/90: sync 2024-09-16T23:48:49.750 INFO:tasks.workunit.client.0.smithi042.stdout:3/61: sync 2024-09-16T23:48:49.751 INFO:tasks.workunit.client.1.smithi043.stdout:9/259: creat d4/d21/f4d x:0 0 0 2024-09-16T23:48:49.753 INFO:tasks.workunit.client.0.smithi042.stdout:2/94: dwrite d0/f11 [0,4194304] 0 2024-09-16T23:48:49.754 INFO:tasks.workunit.client.0.smithi042.stdout:7/71: mkdir d3/d15/d16/d19 0 2024-09-16T23:48:49.756 INFO:tasks.workunit.client.0.smithi042.stdout:5/115: dread d9/db/de/d10/f1a [0,4194304] 0 2024-09-16T23:48:49.757 INFO:tasks.workunit.client.1.smithi043.stdout:7/318: dread d4/f19 [0,4194304] 0 2024-09-16T23:48:49.760 INFO:tasks.workunit.client.1.smithi043.stdout:7/319: symlink d4/d27/l6d 0 2024-09-16T23:48:49.760 INFO:tasks.workunit.client.1.smithi043.stdout:9/260: dwrite d4/f24 [0,4194304] 0 2024-09-16T23:48:49.762 INFO:tasks.workunit.client.1.smithi043.stdout:9/261: write d4/d5/f1d [4663683,66969] 0 2024-09-16T23:48:49.764 INFO:tasks.workunit.client.0.smithi042.stdout:1/144: mkdir d0/d2e 0 2024-09-16T23:48:49.773 INFO:tasks.workunit.client.1.smithi043.stdout:9/262: dread d4/f1a [0,4194304] 0 2024-09-16T23:48:49.773 INFO:tasks.workunit.client.0.smithi042.stdout:2/95: chown d0/c1e 235 1 2024-09-16T23:48:49.773 INFO:tasks.workunit.client.0.smithi042.stdout:7/72: unlink d3/c14 0 2024-09-16T23:48:49.775 INFO:tasks.workunit.client.0.smithi042.stdout:9/72: rmdir d4/d10/d14 0 2024-09-16T23:48:49.778 INFO:tasks.workunit.client.0.smithi042.stdout:5/116: creat d9/db/de/d10/f20 x:0 0 0 2024-09-16T23:48:49.781 INFO:tasks.workunit.client.0.smithi042.stdout:8/209: mkdir da/d19/d46 0 2024-09-16T23:48:49.782 INFO:tasks.workunit.client.0.smithi042.stdout:8/210: fdatasync da/d19/f1e 0 2024-09-16T23:48:49.783 INFO:tasks.workunit.client.0.smithi042.stdout:9/73: dread d4/f5 [4194304,4194304] 0 2024-09-16T23:48:49.785 INFO:tasks.workunit.client.0.smithi042.stdout:3/62: link d2/d7/f8 d2/d7/ff 0 2024-09-16T23:48:49.785 INFO:tasks.workunit.client.1.smithi043.stdout:7/320: truncate d4/db/d11/d21/d36/f57 381476 0 2024-09-16T23:48:49.786 INFO:tasks.workunit.client.1.smithi043.stdout:7/321: chown d4/d27/d41/f40 56 1 2024-09-16T23:48:49.786 INFO:tasks.workunit.client.1.smithi043.stdout:9/263: truncate d4/f31 1954428 0 2024-09-16T23:48:49.788 INFO:tasks.workunit.client.1.smithi043.stdout:9/264: truncate d4/d21/d33/f34 4971512 0 2024-09-16T23:48:49.788 INFO:tasks.workunit.client.0.smithi042.stdout:2/96: creat d0/f22 x:0 0 0 2024-09-16T23:48:49.789 INFO:tasks.workunit.client.0.smithi042.stdout:2/97: read - d0/f20 zero size 2024-09-16T23:48:49.790 INFO:tasks.workunit.client.0.smithi042.stdout:7/73: dread d3/f8 [0,4194304] 0 2024-09-16T23:48:49.792 INFO:tasks.workunit.client.0.smithi042.stdout:2/98: write d0/da/f21 [333226,126445] 0 2024-09-16T23:48:49.792 INFO:tasks.workunit.client.1.smithi043.stdout:9/265: rename d4/f24 to d4/d21/d33/f4e 0 2024-09-16T23:48:49.794 INFO:tasks.workunit.client.1.smithi043.stdout:9/266: write d4/d21/d2a/d2c/f2d [579049,3017] 0 2024-09-16T23:48:49.796 INFO:tasks.workunit.client.0.smithi042.stdout:6/91: creat d8/db/f1d x:0 0 0 2024-09-16T23:48:49.797 INFO:tasks.workunit.client.1.smithi043.stdout:9/267: dread d4/d5/f46 [0,4194304] 0 2024-09-16T23:48:49.797 INFO:tasks.workunit.client.0.smithi042.stdout:6/92: stat d8/db/dd/l14 0 2024-09-16T23:48:49.800 INFO:tasks.workunit.client.0.smithi042.stdout:9/74: creat d4/f19 x:0 0 0 2024-09-16T23:48:49.801 INFO:tasks.workunit.client.0.smithi042.stdout:3/63: mkdir d2/d9/d10 0 2024-09-16T23:48:49.801 INFO:tasks.workunit.client.0.smithi042.stdout:7/74: dwrite d3/fc [0,4194304] 0 2024-09-16T23:48:49.801 INFO:tasks.workunit.client.0.smithi042.stdout:5/117: dread d9/db/de/d10/f1a [0,4194304] 0 2024-09-16T23:48:49.806 INFO:tasks.workunit.client.0.smithi042.stdout:0/44: dread d0/f8 [0,4194304] 0 2024-09-16T23:48:49.808 INFO:tasks.workunit.client.0.smithi042.stdout:6/93: rename d8/db/dd/c16 to d8/c1e 0 2024-09-16T23:48:49.809 INFO:tasks.workunit.client.0.smithi042.stdout:0/45: fdatasync d0/fa 0 2024-09-16T23:48:49.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/75: truncate d3/f8 1940296 0 2024-09-16T23:48:49.810 INFO:tasks.workunit.client.0.smithi042.stdout:5/118: mkdir d9/db/d19/d21 0 2024-09-16T23:48:49.812 INFO:tasks.workunit.client.1.smithi043.stdout:9/268: getdents d4/d21/d2a/d2c 0 2024-09-16T23:48:49.813 INFO:tasks.workunit.client.0.smithi042.stdout:7/76: write d3/f18 [512249,16469] 0 2024-09-16T23:48:49.813 INFO:tasks.workunit.client.0.smithi042.stdout:2/99: chown d0/c15 0 1 2024-09-16T23:48:49.814 INFO:tasks.workunit.client.0.smithi042.stdout:8/211: creat da/f47 x:0 0 0 2024-09-16T23:48:49.815 INFO:tasks.workunit.client.0.smithi042.stdout:2/100: readlink d0/l7 0 2024-09-16T23:48:49.816 INFO:tasks.workunit.client.0.smithi042.stdout:3/64: dwrite d2/f4 [0,4194304] 0 2024-09-16T23:48:49.818 INFO:tasks.workunit.client.0.smithi042.stdout:0/46: rmdir d0 39 2024-09-16T23:48:49.818 INFO:tasks.workunit.client.0.smithi042.stdout:7/77: write d3/f11 [453542,70377] 0 2024-09-16T23:48:49.821 INFO:tasks.workunit.client.0.smithi042.stdout:3/65: mknod d2/c11 0 2024-09-16T23:48:49.821 INFO:tasks.workunit.client.0.smithi042.stdout:7/78: write d3/f11 [1571592,62509] 0 2024-09-16T23:48:49.823 INFO:tasks.workunit.client.0.smithi042.stdout:5/119: write d9/db/d11/f1c [488417,55963] 0 2024-09-16T23:48:49.825 INFO:tasks.workunit.client.0.smithi042.stdout:2/101: creat d0/da/f23 x:0 0 0 2024-09-16T23:48:49.827 INFO:tasks.workunit.client.0.smithi042.stdout:2/102: chown d0/c15 11 1 2024-09-16T23:48:49.827 INFO:tasks.workunit.client.0.smithi042.stdout:8/212: symlink da/d19/d1d/d30/l48 0 2024-09-16T23:48:49.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/94: truncate d8/db/fc 199077 0 2024-09-16T23:48:49.830 INFO:tasks.workunit.client.0.smithi042.stdout:7/79: dread d3/fa [0,4194304] 0 2024-09-16T23:48:49.831 INFO:tasks.workunit.client.0.smithi042.stdout:7/80: stat d3/l10 0 2024-09-16T23:48:49.832 INFO:tasks.workunit.client.0.smithi042.stdout:7/81: write d3/fe [5186307,100883] 0 2024-09-16T23:48:49.832 INFO:tasks.workunit.client.0.smithi042.stdout:2/103: dread d0/f11 [0,4194304] 0 2024-09-16T23:48:49.834 INFO:tasks.workunit.client.0.smithi042.stdout:7/82: read d3/f18 [278785,63396] 0 2024-09-16T23:48:49.838 INFO:tasks.workunit.client.1.smithi043.stdout:9/269: dread d4/d1c/d25/f26 [0,4194304] 0 2024-09-16T23:48:49.847 INFO:tasks.workunit.client.0.smithi042.stdout:5/120: creat d9/db/de/d10/f22 x:0 0 0 2024-09-16T23:48:49.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.850+0000 7f15e159a700 1 -- 172.21.15.42:0/193387633 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15d400baf0 con 0x7f15e4100ca0 2024-09-16T23:48:49.848 INFO:tasks.workunit.client.0.smithi042.stdout:5/121: dread - d9/db/de/d10/f22 zero size 2024-09-16T23:48:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.851+0000 7f15e8843700 1 -- 172.21.15.42:0/193387633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4100ca0 msgr2=0x7f15e41010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.851+0000 7f15e8843700 1 --2- 172.21.15.42:0/193387633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4100ca0 0x7f15e41010c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009910 tx=0x7f15d4009c20 comp rx=0 tx=0).stop 2024-09-16T23:48:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.851+0000 7f15e8843700 1 -- 172.21.15.42:0/193387633 shutdown_connections 2024-09-16T23:48:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.851+0000 7f15e8843700 1 --2- 172.21.15.42:0/193387633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e4101ea0 0x7f15e4102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.851+0000 7f15e8843700 1 --2- 172.21.15.42:0/193387633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4100ca0 0x7f15e41010c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.851+0000 7f15e8843700 1 -- 172.21.15.42:0/193387633 >> 172.21.15.42:0/193387633 conn(0x7f15e40fc260 msgr2=0x7f15e40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:49.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.851+0000 7f15e8843700 1 -- 172.21.15.42:0/193387633 shutdown_connections 2024-09-16T23:48:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.852+0000 7f15e8843700 1 -- 172.21.15.42:0/193387633 wait complete. 2024-09-16T23:48:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.852+0000 7f15e8843700 1 Processor -- start 2024-09-16T23:48:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.852+0000 7f15e8843700 1 -- start start 2024-09-16T23:48:49.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e8843700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e4100ca0 0x7f15e41927e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e8843700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4101ea0 0x7f15e4192d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e8843700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e418f2d0 con 0x7f15e4101ea0 2024-09-16T23:48:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e8843700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e418f440 con 0x7f15e4100ca0 2024-09-16T23:48:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4101ea0 0x7f15e4192d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e4100ca0 0x7f15e41927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4101ea0 0x7f15e4192d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59750/0 (socket says 172.21.15.42:59750) 2024-09-16T23:48:49.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e1d9b700 1 -- 172.21.15.42:0/4117915817 learned_addr learned my addr 172.21.15.42:0/4117915817 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e1d9b700 1 -- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e4100ca0 msgr2=0x7f15e41927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e1d9b700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e4100ca0 0x7f15e41927e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e1d9b700 1 -- 172.21.15.42:0/4117915817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40095f0 con 0x7f15e4101ea0 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.853+0000 7f15e1d9b700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4101ea0 0x7f15e4192d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f15d800e920 tx=0x7f15d800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.854+0000 7f15d37fe700 1 -- 172.21.15.42:0/4117915817 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15d800c9d0 con 0x7f15e4101ea0 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.854+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e418f720 con 0x7f15e4101ea0 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.854+0000 7f15d37fe700 1 -- 172.21.15.42:0/4117915817 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f15d8010430 con 0x7f15e4101ea0 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.854+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e418fc70 con 0x7f15e4101ea0 2024-09-16T23:48:49.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.854+0000 7f15d37fe700 1 -- 172.21.15.42:0/4117915817 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15d8018650 con 0x7f15e4101ea0 2024-09-16T23:48:49.853 INFO:tasks.workunit.client.0.smithi042.stdout:3/66: creat d2/d9/dd/f12 x:0 0 0 2024-09-16T23:48:49.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.855+0000 7f15d37fe700 1 -- 172.21.15.42:0/4117915817 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 22) v1 ==== 46250+0+0 (secure 0 0 0) 0x7f15d80188c0 con 0x7f15e4101ea0 2024-09-16T23:48:49.853 INFO:tasks.workunit.client.0.smithi042.stdout:6/95: creat d8/db/dd/f1f x:0 0 0 2024-09-16T23:48:49.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.856+0000 7f15d37fe700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f15cc0391e0 0x7f15cc03b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:48:49.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.856+0000 7f15d37fe700 1 -- 172.21.15.42:0/4117915817 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f15d8014070 con 0x7f15e4101ea0 2024-09-16T23:48:49.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.856+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e4105f80 con 0x7f15e4101ea0 2024-09-16T23:48:49.854 INFO:tasks.workunit.client.0.smithi042.stdout:6/96: stat f5 0 2024-09-16T23:48:49.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.856+0000 7f15e259c700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f15cc0391e0 0x7f15cc03b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:48:49.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.857+0000 7f15e259c700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f15cc0391e0 0x7f15cc03b6a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009fd0 tx=0x7f15d4005fd0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:48:49.855 INFO:tasks.workunit.client.0.smithi042.stdout:6/97: write d8/db/dd/f15 [252282,62488] 0 2024-09-16T23:48:49.855 INFO:tasks.workunit.client.0.smithi042.stdout:2/104: symlink d0/l24 0 2024-09-16T23:48:49.857 INFO:tasks.workunit.client.0.smithi042.stdout:7/83: creat d3/f1a x:0 0 0 2024-09-16T23:48:49.858 INFO:tasks.workunit.client.0.smithi042.stdout:0/47: unlink d0/l1 0 2024-09-16T23:48:49.859 INFO:tasks.workunit.client.0.smithi042.stdout:0/48: fdatasync d0/f9 0 2024-09-16T23:48:49.859 INFO:tasks.workunit.client.0.smithi042.stdout:0/49: chown d0/l6 7809 1 2024-09-16T23:48:49.860 INFO:tasks.workunit.client.0.smithi042.stdout:2/105: dread d0/f12 [0,4194304] 0 2024-09-16T23:48:49.860 INFO:tasks.workunit.client.0.smithi042.stdout:5/122: mknod d9/c23 0 2024-09-16T23:48:49.860 INFO:tasks.workunit.client.0.smithi042.stdout:0/50: fdatasync d0/fa 0 2024-09-16T23:48:49.861 INFO:tasks.workunit.client.0.smithi042.stdout:0/51: chown d0/fb 1 1 2024-09-16T23:48:49.863 INFO:tasks.workunit.client.0.smithi042.stdout:5/123: write d9/db/de/d10/f22 [712178,69180] 0 2024-09-16T23:48:49.863 INFO:tasks.workunit.client.0.smithi042.stdout:2/106: chown d0/ff 4935 1 2024-09-16T23:48:49.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:49.865+0000 7f15d37fe700 1 -- 172.21.15.42:0/4117915817 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f15d80152d0 con 0x7f15e4101ea0 2024-09-16T23:48:49.864 INFO:tasks.workunit.client.0.smithi042.stdout:0/52: truncate d0/f9 559366 0 2024-09-16T23:48:49.865 INFO:tasks.workunit.client.0.smithi042.stdout:0/53: write d0/f9 [590008,58195] 0 2024-09-16T23:48:49.866 INFO:tasks.workunit.client.0.smithi042.stdout:0/54: chown d0/f8 246 1 2024-09-16T23:48:49.867 INFO:tasks.workunit.client.0.smithi042.stdout:0/55: read - d0/fb zero size 2024-09-16T23:48:49.867 INFO:tasks.workunit.client.0.smithi042.stdout:3/67: fdatasync d2/d7/ff 0 2024-09-16T23:48:49.869 INFO:tasks.workunit.client.0.smithi042.stdout:6/98: fdatasync d8/db/dd/f19 0 2024-09-16T23:48:49.871 INFO:tasks.workunit.client.0.smithi042.stdout:7/84: creat d3/d15/f1b x:0 0 0 2024-09-16T23:48:49.874 INFO:tasks.workunit.client.0.smithi042.stdout:2/107: fdatasync d0/ff 0 2024-09-16T23:48:49.876 INFO:tasks.workunit.client.0.smithi042.stdout:2/108: dread - d0/da/f23 zero size 2024-09-16T23:48:49.877 INFO:tasks.workunit.client.0.smithi042.stdout:0/56: dread d0/f8 [0,4194304] 0 2024-09-16T23:48:49.878 INFO:tasks.workunit.client.0.smithi042.stdout:3/68: rename f1 to d2/d9/d10/f13 0 2024-09-16T23:48:49.881 INFO:tasks.workunit.client.1.smithi043.stdout:9/270: chown d4/f31 42766381 1 2024-09-16T23:48:49.884 INFO:tasks.workunit.client.0.smithi042.stdout:7/85: dwrite d3/d15/f1b [0,4194304] 0 2024-09-16T23:48:49.885 INFO:tasks.workunit.client.0.smithi042.stdout:7/86: read d3/fe [3365345,24000] 0 2024-09-16T23:48:49.886 INFO:tasks.workunit.client.0.smithi042.stdout:7/87: fdatasync d3/fc 0 2024-09-16T23:48:49.887 INFO:tasks.workunit.client.0.smithi042.stdout:0/57: mknod d0/cc 0 2024-09-16T23:48:49.888 INFO:tasks.workunit.client.0.smithi042.stdout:5/124: getdents d9/db/de 0 2024-09-16T23:48:49.892 INFO:tasks.workunit.client.0.smithi042.stdout:4/81: sync 2024-09-16T23:48:49.894 INFO:tasks.workunit.client.0.smithi042.stdout:3/69: rename d2/d7/dc to d2/d9/d14 0 2024-09-16T23:48:49.895 INFO:tasks.workunit.client.0.smithi042.stdout:5/125: write d9/db/de/d10/f22 [1180138,31863] 0 2024-09-16T23:48:49.895 INFO:tasks.workunit.client.0.smithi042.stdout:4/82: dread - de/d12/f15 zero size 2024-09-16T23:48:49.899 INFO:tasks.workunit.client.0.smithi042.stdout:3/70: chown d2/d9/d14 9028551 1 2024-09-16T23:48:49.899 INFO:tasks.workunit.client.0.smithi042.stdout:0/58: dwrite d0/fb [0,4194304] 0 2024-09-16T23:48:49.901 INFO:tasks.workunit.client.0.smithi042.stdout:7/88: creat d3/d15/d16/f1c x:0 0 0 2024-09-16T23:48:49.902 INFO:tasks.workunit.client.0.smithi042.stdout:6/99: creat d8/db/f20 x:0 0 0 2024-09-16T23:48:49.904 INFO:tasks.workunit.client.0.smithi042.stdout:5/126: write d9/db/f16 [1595781,12124] 0 2024-09-16T23:48:49.904 INFO:tasks.workunit.client.0.smithi042.stdout:6/100: fdatasync d8/db/f12 0 2024-09-16T23:48:49.906 INFO:tasks.workunit.client.0.smithi042.stdout:8/213: truncate da/d19/f3b 3765819 0 2024-09-16T23:48:49.910 INFO:tasks.workunit.client.0.smithi042.stdout:7/89: symlink d3/l1d 0 2024-09-16T23:48:49.911 INFO:tasks.workunit.client.0.smithi042.stdout:5/127: read d9/db/d11/f1c [364634,91066] 0 2024-09-16T23:48:49.912 INFO:tasks.workunit.client.1.smithi043.stdout:9/271: write d4/ff [2270317,14056] 0 2024-09-16T23:48:49.913 INFO:tasks.workunit.client.1.smithi043.stdout:9/272: chown d4/f10 44663 1 2024-09-16T23:48:49.914 INFO:tasks.workunit.client.1.smithi043.stdout:9/273: chown d4/d28/c32 38125638 1 2024-09-16T23:48:49.917 INFO:tasks.workunit.client.1.smithi043.stdout:9/274: rename d4/d1c/d41 to d4/d1c/d4f 0 2024-09-16T23:48:49.917 INFO:tasks.workunit.client.1.smithi043.stdout:9/275: chown d4/f10 27777 1 2024-09-16T23:48:49.920 INFO:tasks.workunit.client.0.smithi042.stdout:8/214: write da/d19/d1d/d30/f41 [826586,90115] 0 2024-09-16T23:48:49.921 INFO:tasks.workunit.client.1.smithi043.stdout:9/276: getdents d4/d21 0 2024-09-16T23:48:49.921 INFO:tasks.workunit.client.0.smithi042.stdout:5/128: creat d9/db/d17/f24 x:0 0 0 2024-09-16T23:48:49.922 INFO:tasks.workunit.client.1.smithi043.stdout:9/277: stat d4/d21/d35/d4b 0 2024-09-16T23:48:49.923 INFO:tasks.workunit.client.0.smithi042.stdout:0/59: dwrite d0/f9 [0,4194304] 0 2024-09-16T23:48:49.924 INFO:tasks.workunit.client.1.smithi043.stdout:9/278: write d4/f10 [4805351,83723] 0 2024-09-16T23:48:49.929 INFO:tasks.workunit.client.1.smithi043.stdout:9/279: rename d4/d28/d2e to d4/d1c/d25/d2b/d4c/d50 0 2024-09-16T23:48:49.931 INFO:tasks.workunit.client.0.smithi042.stdout:7/90: getdents d3/d15/d16/d19 0 2024-09-16T23:48:49.931 INFO:tasks.workunit.client.0.smithi042.stdout:8/215: fdatasync da/f16 0 2024-09-16T23:48:49.932 INFO:tasks.workunit.client.0.smithi042.stdout:2/109: dwrite d0/ff [0,4194304] 0 2024-09-16T23:48:49.935 INFO:tasks.workunit.client.0.smithi042.stdout:6/101: dwrite d8/db/f1d [0,4194304] 0 2024-09-16T23:48:49.936 INFO:tasks.workunit.client.0.smithi042.stdout:6/102: rename d8 to d8/d21 22 2024-09-16T23:48:49.940 INFO:tasks.workunit.client.0.smithi042.stdout:2/110: write d0/da/d17/f1c [1136191,69555] 0 2024-09-16T23:48:49.941 INFO:tasks.workunit.client.0.smithi042.stdout:8/216: dread da/f17 [0,4194304] 0 2024-09-16T23:48:49.941 INFO:tasks.workunit.client.0.smithi042.stdout:2/111: chown d0/l24 218656 1 2024-09-16T23:48:49.942 INFO:tasks.workunit.client.0.smithi042.stdout:8/217: dread - da/d19/d1d/f33 zero size 2024-09-16T23:48:49.943 INFO:tasks.workunit.client.0.smithi042.stdout:8/218: read - da/d19/d1d/d30/f45 zero size 2024-09-16T23:48:49.963 INFO:tasks.workunit.client.1.smithi043.stdout:3/246: sync 2024-09-16T23:48:49.975 INFO:tasks.workunit.client.1.smithi043.stdout:3/247: getdents d4/d30 0 2024-09-16T23:48:49.979 INFO:tasks.workunit.client.1.smithi043.stdout:3/248: write d4/f16 [1132276,60030] 0 2024-09-16T23:48:49.981 INFO:tasks.workunit.client.0.smithi042.stdout:0/60: symlink d0/ld 0 2024-09-16T23:48:49.982 INFO:tasks.workunit.client.1.smithi043.stdout:9/280: dwrite d4/f1a [0,4194304] 0 2024-09-16T23:48:49.983 INFO:tasks.workunit.client.0.smithi042.stdout:5/129: creat d9/db/f25 x:0 0 0 2024-09-16T23:48:49.983 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:49 smithi042 ceph-mon[27914]: mgrmap e22: smithi043.xveyou(active, since 1.10438s) 2024-09-16T23:48:49.983 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:49 smithi042 ceph-mon[27914]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:49.984 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:49 smithi042 ceph-mon[27914]: pgmap v3: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:49.984 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:49 smithi042 ceph-mon[27914]: Deploying cephadm binary to smithi042 2024-09-16T23:48:49.984 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:49 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/937573044' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:48:49.987 INFO:tasks.workunit.client.1.smithi043.stdout:9/281: dread d4/f10 [0,4194304] 0 2024-09-16T23:48:49.991 INFO:tasks.workunit.client.1.smithi043.stdout:3/249: creat d4/de/d40/f45 x:0 0 0 2024-09-16T23:48:49.992 INFO:tasks.workunit.client.1.smithi043.stdout:9/282: symlink d4/d21/d35/l51 0 2024-09-16T23:48:49.993 INFO:tasks.workunit.client.0.smithi042.stdout:7/91: dread d3/f18 [0,4194304] 0 2024-09-16T23:48:49.994 INFO:tasks.workunit.client.1.smithi043.stdout:3/250: fsync d4/de/d1f/d34/f3a 0 2024-09-16T23:48:49.996 INFO:tasks.workunit.client.1.smithi043.stdout:3/251: chown d4/f16 12 1 2024-09-16T23:48:49.998 INFO:tasks.workunit.client.0.smithi042.stdout:8/219: mkdir da/d19/d1d/d30/d49 0 2024-09-16T23:48:49.998 INFO:tasks.workunit.client.1.smithi043.stdout:3/252: read d4/de/f21 [7732006,31151] 0 2024-09-16T23:48:50.000 INFO:tasks.workunit.client.1.smithi043.stdout:9/283: dwrite d4/d21/d2a/d2c/f2d [0,4194304] 0 2024-09-16T23:48:50.025 INFO:tasks.workunit.client.1.smithi043.stdout:9/284: rename d4/ff to d4/d1c/d4f/f52 0 2024-09-16T23:48:50.026 INFO:tasks.workunit.client.1.smithi043.stdout:3/253: creat d4/de/d1f/f46 x:0 0 0 2024-09-16T23:48:50.026 INFO:tasks.workunit.client.1.smithi043.stdout:9/285: fdatasync d4/d1c/f20 0 2024-09-16T23:48:50.027 INFO:tasks.workunit.client.0.smithi042.stdout:1/145: sync 2024-09-16T23:48:50.033 INFO:tasks.workunit.client.1.smithi043.stdout:3/254: creat d4/de/d1f/d2c/f47 x:0 0 0 2024-09-16T23:48:50.038 INFO:tasks.workunit.client.0.smithi042.stdout:0/61: symlink d0/le 0 2024-09-16T23:48:50.039 INFO:tasks.workunit.client.0.smithi042.stdout:0/62: truncate d0/fa 241210 0 2024-09-16T23:48:50.040 INFO:tasks.workunit.client.1.smithi043.stdout:3/255: creat d4/de/d40/f48 x:0 0 0 2024-09-16T23:48:50.041 INFO:tasks.workunit.client.1.smithi043.stdout:9/286: link d4/d1c/c3a d4/c53 0 2024-09-16T23:48:50.041 INFO:tasks.workunit.client.1.smithi043.stdout:3/256: readlink d4/d30/l39 0 2024-09-16T23:48:50.042 INFO:tasks.workunit.client.0.smithi042.stdout:3/71: dwrite d2/d9/d10/f13 [0,4194304] 0 2024-09-16T23:48:50.043 INFO:tasks.workunit.client.1.smithi043.stdout:3/257: rmdir d4/de/d35 39 2024-09-16T23:48:50.044 INFO:tasks.workunit.client.0.smithi042.stdout:6/103: write d8/d9/f13 [879427,101709] 0 2024-09-16T23:48:50.046 INFO:tasks.workunit.client.1.smithi043.stdout:3/258: unlink d4/f16 0 2024-09-16T23:48:50.050 INFO:tasks.workunit.client.1.smithi043.stdout:3/259: mknod d4/de/d1f/d34/c49 0 2024-09-16T23:48:50.056 INFO:tasks.workunit.client.0.smithi042.stdout:2/112: link d0/da/d1b/l1d d0/da/d1b/l25 0 2024-09-16T23:48:50.057 INFO:tasks.workunit.client.0.smithi042.stdout:2/113: stat d0/f11 0 2024-09-16T23:48:50.058 INFO:tasks.workunit.client.1.smithi043.stdout:3/260: rename d4/f10 to d4/de/d1f/d2c/f4a 0 2024-09-16T23:48:50.059 INFO:tasks.workunit.client.0.smithi042.stdout:6/104: dwrite f5 [4194304,4194304] 0 2024-09-16T23:48:50.059 INFO:tasks.workunit.client.0.smithi042.stdout:7/92: link d3/fc d3/d15/d16/f1e 0 2024-09-16T23:48:50.060 INFO:tasks.workunit.client.0.smithi042.stdout:7/93: readlink d3/l10 0 2024-09-16T23:48:50.061 INFO:tasks.workunit.client.0.smithi042.stdout:7/94: fdatasync d3/d15/f1b 0 2024-09-16T23:48:50.061 INFO:tasks.workunit.client.0.smithi042.stdout:5/130: truncate d9/db/f16 487510 0 2024-09-16T23:48:50.061 INFO:tasks.workunit.client.0.smithi042.stdout:6/105: chown d8/db/dd/l17 84910 1 2024-09-16T23:48:50.063 INFO:tasks.workunit.client.0.smithi042.stdout:6/106: write d8/db/dd/f19 [1100955,51637] 0 2024-09-16T23:48:50.067 INFO:tasks.workunit.client.1.smithi043.stdout:9/287: truncate d4/d5/d30/f39 794646 0 2024-09-16T23:48:50.068 INFO:tasks.workunit.client.1.smithi043.stdout:9/288: stat d4/d21/d2a/d2c/f2d 0 2024-09-16T23:48:50.069 INFO:tasks.workunit.client.1.smithi043.stdout:9/289: write d4/f1a [692521,41469] 0 2024-09-16T23:48:50.070 INFO:tasks.workunit.client.1.smithi043.stdout:3/261: symlink d4/de/l4b 0 2024-09-16T23:48:50.071 INFO:tasks.workunit.client.1.smithi043.stdout:3/262: write d4/de/d1f/f3e [581332,117780] 0 2024-09-16T23:48:50.072 INFO:tasks.workunit.client.1.smithi043.stdout:3/263: readlink d4/lc 0 2024-09-16T23:48:50.073 INFO:tasks.workunit.client.1.smithi043.stdout:9/290: creat d4/d5/d16/f54 x:0 0 0 2024-09-16T23:48:50.074 INFO:tasks.workunit.client.1.smithi043.stdout:9/291: readlink d4/d5/d16/l1e 0 2024-09-16T23:48:50.078 INFO:tasks.workunit.client.1.smithi043.stdout:3/264: dread d4/f23 [0,4194304] 0 2024-09-16T23:48:50.082 INFO:tasks.workunit.client.1.smithi043.stdout:3/265: write d4/f19 [3951779,33963] 0 2024-09-16T23:48:50.082 INFO:tasks.workunit.client.1.smithi043.stdout:3/266: stat c0 0 2024-09-16T23:48:50.084 INFO:tasks.workunit.client.1.smithi043.stdout:9/292: dwrite d4/d5/f1d [0,4194304] 0 2024-09-16T23:48:50.088 INFO:tasks.workunit.client.1.smithi043.stdout:9/293: creat d4/d21/d33/f55 x:0 0 0 2024-09-16T23:48:50.093 INFO:tasks.workunit.client.1.smithi043.stdout:3/267: dwrite d4/de/f27 [4194304,4194304] 0 2024-09-16T23:48:50.094 INFO:tasks.workunit.client.1.smithi043.stdout:9/294: mknod d4/d21/d2a/c56 0 2024-09-16T23:48:50.098 INFO:tasks.workunit.client.1.smithi043.stdout:3/268: creat d4/de/d35/f4c x:0 0 0 2024-09-16T23:48:50.101 INFO:tasks.workunit.client.1.smithi043.stdout:9/295: write d4/d21/d33/f4e [1031590,124735] 0 2024-09-16T23:48:50.102 INFO:tasks.workunit.client.1.smithi043.stdout:3/269: mkdir d4/de/d40/d4d 0 2024-09-16T23:48:50.102 INFO:tasks.workunit.client.1.smithi043.stdout:9/296: readlink d4/d21/d33/l37 0 2024-09-16T23:48:50.103 INFO:tasks.workunit.client.1.smithi043.stdout:3/270: readlink d4/de/d1f/d2c/l2d 0 2024-09-16T23:48:50.105 INFO:tasks.workunit.client.1.smithi043.stdout:9/297: creat d4/d5/d30/f57 x:0 0 0 2024-09-16T23:48:50.106 INFO:tasks.workunit.client.1.smithi043.stdout:3/271: truncate d4/de/d1f/d34/f44 436123 0 2024-09-16T23:48:50.107 INFO:tasks.workunit.client.1.smithi043.stdout:9/298: chown d4/c9 217428 1 2024-09-16T23:48:50.108 INFO:tasks.workunit.client.0.smithi042.stdout:6/107: creat d8/db/dd/f22 x:0 0 0 2024-09-16T23:48:50.108 INFO:tasks.workunit.client.0.smithi042.stdout:6/108: stat d8/db/dd/f11 0 2024-09-16T23:48:50.109 INFO:tasks.workunit.client.1.smithi043.stdout:3/272: symlink d4/d30/l4e 0 2024-09-16T23:48:50.111 INFO:tasks.workunit.client.1.smithi043.stdout:9/299: unlink d4/d21/d2a/d2c/f2d 0 2024-09-16T23:48:50.111 INFO:tasks.workunit.client.0.smithi042.stdout:3/72: rmdir d2/d9/d14 0 2024-09-16T23:48:50.113 INFO:tasks.workunit.client.1.smithi043.stdout:3/273: mknod d4/de/d1f/c4f 0 2024-09-16T23:48:50.114 INFO:tasks.workunit.client.1.smithi043.stdout:3/274: truncate d4/de/d1f/f29 4519106 0 2024-09-16T23:48:50.115 INFO:tasks.workunit.client.0.smithi042.stdout:5/131: creat d9/db/d19/d21/f26 x:0 0 0 2024-09-16T23:48:50.116 INFO:tasks.workunit.client.1.smithi043.stdout:3/275: truncate d4/de/d1f/f46 752436 0 2024-09-16T23:48:50.116 INFO:tasks.workunit.client.0.smithi042.stdout:5/132: readlink d9/db/d17/l1f 0 2024-09-16T23:48:50.117 INFO:tasks.workunit.client.0.smithi042.stdout:6/109: dwrite d8/db/fc [0,4194304] 0 2024-09-16T23:48:50.120 INFO:tasks.workunit.client.1.smithi043.stdout:3/276: write d4/f3b [849435,67278] 0 2024-09-16T23:48:50.121 INFO:tasks.workunit.client.1.smithi043.stdout:3/277: chown d4/de/d1f 0 1 2024-09-16T23:48:50.123 INFO:tasks.workunit.client.1.smithi043.stdout:3/278: symlink d4/l50 0 2024-09-16T23:48:50.127 INFO:tasks.workunit.client.1.smithi043.stdout:3/279: creat d4/de/f51 x:0 0 0 2024-09-16T23:48:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:49 smithi043 ceph-mon[35191]: mgrmap e22: smithi043.xveyou(active, since 1.10438s) 2024-09-16T23:48:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:49 smithi043 ceph-mon[35191]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:49 smithi043 ceph-mon[35191]: pgmap v3: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:49 smithi043 ceph-mon[35191]: Deploying cephadm binary to smithi042 2024-09-16T23:48:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:49 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/937573044' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:48:50.128 INFO:tasks.workunit.client.1.smithi043.stdout:3/280: write d4/de/d1f/f3e [346094,63000] 0 2024-09-16T23:48:50.130 INFO:tasks.workunit.client.1.smithi043.stdout:3/281: write d4/de/d1f/d2c/f4a [1665976,18336] 0 2024-09-16T23:48:50.131 INFO:tasks.workunit.client.1.smithi043.stdout:3/282: write d4/f19 [312687,128936] 0 2024-09-16T23:48:50.145 INFO:tasks.workunit.client.1.smithi043.stdout:3/283: dread d4/de/d1f/d34/f3a [0,4194304] 0 2024-09-16T23:48:50.148 INFO:tasks.workunit.client.1.smithi043.stdout:3/284: dread d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:48:50.149 INFO:tasks.workunit.client.1.smithi043.stdout:3/285: write d4/de/d40/f45 [381870,44457] 0 2024-09-16T23:48:50.151 INFO:tasks.workunit.client.1.smithi043.stdout:3/286: truncate d4/de/d1f/f46 881208 0 2024-09-16T23:48:50.152 INFO:tasks.workunit.client.1.smithi043.stdout:3/287: write d4/d30/f41 [719976,54762] 0 2024-09-16T23:48:50.154 INFO:tasks.workunit.client.1.smithi043.stdout:3/288: symlink d4/de/d1f/l52 0 2024-09-16T23:48:50.161 INFO:tasks.workunit.client.0.smithi042.stdout:9/75: sync 2024-09-16T23:48:50.163 INFO:tasks.workunit.client.1.smithi043.stdout:3/289: getdents d4 0 2024-09-16T23:48:50.167 INFO:tasks.workunit.client.1.smithi043.stdout:3/290: mkdir d4/d30/d53 0 2024-09-16T23:48:50.172 INFO:tasks.workunit.client.1.smithi043.stdout:3/291: dwrite d4/de/d1f/d34/f44 [0,4194304] 0 2024-09-16T23:48:50.175 INFO:tasks.workunit.client.1.smithi043.stdout:3/292: getdents d4/de/d40/d4d 0 2024-09-16T23:48:50.176 INFO:tasks.workunit.client.1.smithi043.stdout:3/293: write d4/de/d35/f4c [816438,108530] 0 2024-09-16T23:48:50.179 INFO:tasks.workunit.client.1.smithi043.stdout:3/294: creat d4/f54 x:0 0 0 2024-09-16T23:48:50.183 INFO:tasks.workunit.client.1.smithi043.stdout:3/295: getdents d4/de 0 2024-09-16T23:48:50.185 INFO:tasks.workunit.client.1.smithi043.stdout:3/296: write d4/f23 [100101,114562] 0 2024-09-16T23:48:50.196 INFO:tasks.workunit.client.1.smithi043.stdout:3/297: dread d4/fb [4194304,4194304] 0 2024-09-16T23:48:50.201 INFO:tasks.workunit.client.1.smithi043.stdout:3/298: dwrite d4/de/f15 [0,4194304] 0 2024-09-16T23:48:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.204+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15e40611d0 con 0x7f15cc0391e0 2024-09-16T23:48:50.202 INFO:tasks.workunit.client.1.smithi043.stdout:3/299: write d4/de/d1f/f46 [1491766,72222] 0 2024-09-16T23:48:50.207 INFO:tasks.workunit.client.1.smithi043.stdout:3/300: dread d4/de/d1f/d2c/f4a [0,4194304] 0 2024-09-16T23:48:50.211 INFO:tasks.workunit.client.1.smithi043.stdout:3/301: link d4/f33 d4/de/f55 0 2024-09-16T23:48:50.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.218+0000 7f15d37fe700 1 -- 172.21.15.42:0/4117915817 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f15e40611d0 con 0x7f15cc0391e0 2024-09-16T23:48:50.215 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:48:50.215 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:48:50.215 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:48:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:48:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-09-16T23:48:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "1/2 daemons upgraded", 2024-09-16T23:48:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "", 2024-09-16T23:48:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:48:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:48:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f15cc0391e0 msgr2=0x7f15cc03b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f15cc0391e0 0x7f15cc03b6a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009fd0 tx=0x7f15d4005fd0 comp rx=0 tx=0).stop 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4101ea0 msgr2=0x7f15e4192d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4101ea0 0x7f15e4192d20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f15d800e920 tx=0x7f15d800ec30 comp rx=0 tx=0).stop 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 shutdown_connections 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f15cc0391e0 0x7f15cc03b6a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15e4100ca0 0x7f15e41927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 --2- 172.21.15.42:0/4117915817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f15e4101ea0 0x7f15e4192d20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 >> 172.21.15.42:0/4117915817 conn(0x7f15e40fc260 msgr2=0x7f15e41030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 shutdown_connections 2024-09-16T23:48:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:48:50.221+0000 7f15e8843700 1 -- 172.21.15.42:0/4117915817 wait complete. 2024-09-16T23:48:50.220 INFO:tasks.workunit.client.1.smithi043.stdout:3/302: creat d4/de/d1f/f56 x:0 0 0 2024-09-16T23:48:50.220 INFO:tasks.workunit.client.1.smithi043.stdout:3/303: write d4/fb [5674374,2587] 0 2024-09-16T23:48:50.222 INFO:tasks.workunit.client.1.smithi043.stdout:3/304: symlink d4/d30/l57 0 2024-09-16T23:48:50.226 INFO:tasks.workunit.client.1.smithi043.stdout:3/305: unlink d4/de/d40/f48 0 2024-09-16T23:48:50.230 INFO:tasks.workunit.client.1.smithi043.stdout:3/306: creat d4/de/f58 x:0 0 0 2024-09-16T23:48:50.231 INFO:tasks.workunit.client.1.smithi043.stdout:3/307: mknod d4/d30/c59 0 2024-09-16T23:48:50.233 INFO:tasks.workunit.client.0.smithi042.stdout:4/83: sync 2024-09-16T23:48:50.234 INFO:tasks.workunit.client.0.smithi042.stdout:4/84: chown de/c18 259 1 2024-09-16T23:48:50.234 INFO:tasks.workunit.client.1.smithi043.stdout:3/308: symlink d4/de/l5a 0 2024-09-16T23:48:50.236 INFO:tasks.workunit.client.1.smithi043.stdout:3/309: rmdir d4/d30 39 2024-09-16T23:48:50.237 INFO:tasks.workunit.client.1.smithi043.stdout:3/310: read d4/f19 [471212,28669] 0 2024-09-16T23:48:50.239 INFO:tasks.workunit.client.1.smithi043.stdout:3/311: read d4/de/d1f/f46 [606052,262] 0 2024-09-16T23:48:50.245 INFO:tasks.workunit.client.0.smithi042.stdout:1/146: getdents d0/d17 0 2024-09-16T23:48:50.247 INFO:tasks.workunit.client.1.smithi043.stdout:3/312: getdents d4/de 0 2024-09-16T23:48:50.248 INFO:tasks.workunit.client.1.smithi043.stdout:3/313: fdatasync d4/de/d35/f1d 0 2024-09-16T23:48:50.250 INFO:tasks.workunit.client.0.smithi042.stdout:3/73: creat d2/d9/d10/f15 x:0 0 0 2024-09-16T23:48:50.257 INFO:tasks.workunit.client.1.smithi043.stdout:3/314: dwrite d4/de/d35/f4c [0,4194304] 0 2024-09-16T23:48:50.262 INFO:tasks.workunit.client.1.smithi043.stdout:3/315: dread d4/de/f27 [4194304,4194304] 0 2024-09-16T23:48:50.272 INFO:tasks.workunit.client.1.smithi043.stdout:3/316: truncate d4/d30/f41 533525 0 2024-09-16T23:48:50.275 INFO:tasks.workunit.client.0.smithi042.stdout:9/76: mknod d4/d12/c1a 0 2024-09-16T23:48:50.275 INFO:tasks.workunit.client.1.smithi043.stdout:3/317: symlink d4/de/d1f/d34/l5b 0 2024-09-16T23:48:50.275 INFO:tasks.workunit.client.0.smithi042.stdout:9/77: chown d4/lf 15071149 1 2024-09-16T23:48:50.276 INFO:tasks.workunit.client.0.smithi042.stdout:0/63: truncate d0/f7 251045 0 2024-09-16T23:48:50.281 INFO:tasks.workunit.client.0.smithi042.stdout:3/74: rename d2/d9/d10/f13 to d2/d9/dd/f16 0 2024-09-16T23:48:50.283 INFO:tasks.workunit.client.0.smithi042.stdout:3/75: fdatasync d2/d9/dd/f12 0 2024-09-16T23:48:50.284 INFO:tasks.workunit.client.0.smithi042.stdout:7/95: getdents d3/d15 0 2024-09-16T23:48:50.286 INFO:tasks.workunit.client.1.smithi043.stdout:3/318: dwrite d4/de/d1f/d34/f3a [0,4194304] 0 2024-09-16T23:48:50.288 INFO:tasks.workunit.client.0.smithi042.stdout:2/114: dwrite d0/da/fd [0,4194304] 0 2024-09-16T23:48:50.289 INFO:tasks.workunit.client.0.smithi042.stdout:4/85: mkdir de/d1a/d1c 0 2024-09-16T23:48:50.290 INFO:tasks.workunit.client.0.smithi042.stdout:2/115: read d0/f12 [695594,38268] 0 2024-09-16T23:48:50.290 INFO:tasks.workunit.client.0.smithi042.stdout:2/116: dread - d0/f20 zero size 2024-09-16T23:48:50.291 INFO:tasks.workunit.client.0.smithi042.stdout:9/78: creat d4/d10/f1b x:0 0 0 2024-09-16T23:48:50.292 INFO:tasks.workunit.client.1.smithi043.stdout:3/319: dread d4/de/f55 [0,4194304] 0 2024-09-16T23:48:50.292 INFO:tasks.workunit.client.0.smithi042.stdout:0/64: mkdir d0/df 0 2024-09-16T23:48:50.293 INFO:tasks.workunit.client.1.smithi043.stdout:0/310: sync 2024-09-16T23:48:50.293 INFO:tasks.workunit.client.0.smithi042.stdout:2/117: write d0/f20 [918420,49167] 0 2024-09-16T23:48:50.295 INFO:tasks.workunit.client.0.smithi042.stdout:3/76: mkdir d2/d9/dd/d17 0 2024-09-16T23:48:50.296 INFO:tasks.workunit.client.0.smithi042.stdout:7/96: mkdir d3/d1f 0 2024-09-16T23:48:50.300 INFO:tasks.workunit.client.0.smithi042.stdout:5/133: link l8 d9/db/d19/l27 0 2024-09-16T23:48:50.301 INFO:tasks.workunit.client.1.smithi043.stdout:3/320: rename d4/de/f15 to d4/d30/d53/f5c 0 2024-09-16T23:48:50.302 INFO:tasks.workunit.client.0.smithi042.stdout:7/97: chown d3/d15/d16 39743 1 2024-09-16T23:48:50.302 INFO:tasks.workunit.client.0.smithi042.stdout:4/86: dread de/f19 [0,4194304] 0 2024-09-16T23:48:50.302 INFO:tasks.workunit.client.0.smithi042.stdout:6/110: creat d8/db/f23 x:0 0 0 2024-09-16T23:48:50.302 INFO:tasks.workunit.client.0.smithi042.stdout:9/79: chown d4/fb 527569839 1 2024-09-16T23:48:50.303 INFO:tasks.workunit.client.1.smithi043.stdout:0/311: rename d0/d2/df/f56 to d0/d2/df/d5a/f63 0 2024-09-16T23:48:50.304 INFO:tasks.workunit.client.0.smithi042.stdout:2/118: dread d0/da/fd [0,4194304] 0 2024-09-16T23:48:50.305 INFO:tasks.workunit.client.0.smithi042.stdout:5/134: fdatasync d9/db/de/d10/f20 0 2024-09-16T23:48:50.306 INFO:tasks.workunit.client.0.smithi042.stdout:0/65: unlink d0/f8 0 2024-09-16T23:48:50.310 INFO:tasks.workunit.client.0.smithi042.stdout:6/111: dwrite d8/db/fc [0,4194304] 0 2024-09-16T23:48:50.310 INFO:tasks.workunit.client.0.smithi042.stdout:3/77: mknod d2/d9/c18 0 2024-09-16T23:48:50.311 INFO:tasks.workunit.client.1.smithi043.stdout:3/321: truncate d4/de/f55 3340810 0 2024-09-16T23:48:50.312 INFO:tasks.workunit.client.1.smithi043.stdout:0/312: dwrite d0/d2/df/d1a/d46/f35 [0,4194304] 0 2024-09-16T23:48:50.312 INFO:tasks.workunit.client.0.smithi042.stdout:3/78: dread - d2/d9/d10/f15 zero size 2024-09-16T23:48:50.313 INFO:tasks.workunit.client.0.smithi042.stdout:4/87: creat de/d1a/f1d x:0 0 0 2024-09-16T23:48:50.315 INFO:tasks.workunit.client.1.smithi043.stdout:0/313: symlink d0/d2/dd/l64 0 2024-09-16T23:48:50.320 INFO:tasks.workunit.client.1.smithi043.stdout:0/314: rename d0/d2/df/d1a/f2f to d0/d2/df/d1a/d3b/d27/f65 0 2024-09-16T23:48:50.323 INFO:tasks.workunit.client.0.smithi042.stdout:9/80: read d4/f5 [3391457,104134] 0 2024-09-16T23:48:50.325 INFO:tasks.workunit.client.1.smithi043.stdout:0/315: dread d0/d2/df/d1a/d46/f35 [0,4194304] 0 2024-09-16T23:48:50.326 INFO:tasks.workunit.client.0.smithi042.stdout:7/98: symlink d3/d15/l20 0 2024-09-16T23:48:50.337 INFO:tasks.workunit.client.0.smithi042.stdout:2/119: mkdir d0/d26 0 2024-09-16T23:48:50.337 INFO:tasks.workunit.client.1.smithi043.stdout:3/322: write d4/de/d1f/f46 [1162974,54352] 0 2024-09-16T23:48:50.338 INFO:tasks.workunit.client.0.smithi042.stdout:2/120: write d0/f1a [362826,13308] 0 2024-09-16T23:48:50.339 INFO:tasks.workunit.client.0.smithi042.stdout:2/121: read - d0/da/f23 zero size 2024-09-16T23:48:50.339 INFO:tasks.workunit.client.1.smithi043.stdout:0/316: link d0/de/c40 d0/dc/c66 0 2024-09-16T23:48:50.343 INFO:tasks.workunit.client.1.smithi043.stdout:3/323: fdatasync d4/de/f21 0 2024-09-16T23:48:50.343 INFO:tasks.workunit.client.0.smithi042.stdout:3/79: rmdir d2/d7 39 2024-09-16T23:48:50.346 INFO:tasks.workunit.client.0.smithi042.stdout:4/88: rename f2 to de/d1a/d1c/f1e 0 2024-09-16T23:48:50.347 INFO:tasks.workunit.client.1.smithi043.stdout:0/317: link d0/dc/c21 d0/d2/df/c67 0 2024-09-16T23:48:50.349 INFO:tasks.workunit.client.1.smithi043.stdout:3/324: read d4/de/f55 [963958,50598] 0 2024-09-16T23:48:50.350 INFO:tasks.workunit.client.1.smithi043.stdout:0/318: mknod d0/d2/c68 0 2024-09-16T23:48:50.351 INFO:tasks.workunit.client.0.smithi042.stdout:5/135: unlink d9/db/f16 0 2024-09-16T23:48:50.352 INFO:tasks.workunit.client.1.smithi043.stdout:0/319: write d0/dc/f50 [1035590,38857] 0 2024-09-16T23:48:50.352 INFO:tasks.workunit.client.1.smithi043.stdout:3/325: mkdir d4/de/d5d 0 2024-09-16T23:48:50.353 INFO:tasks.workunit.client.1.smithi043.stdout:0/320: stat d0/d2/df/d1a/f47 0 2024-09-16T23:48:50.354 INFO:tasks.workunit.client.0.smithi042.stdout:7/99: dwrite d3/fa [0,4194304] 0 2024-09-16T23:48:50.354 INFO:tasks.workunit.client.1.smithi043.stdout:0/321: read d0/d2/df/d1a/f30 [1503711,52518] 0 2024-09-16T23:48:50.355 INFO:tasks.workunit.client.0.smithi042.stdout:7/100: stat d3/fe 0 2024-09-16T23:48:50.356 INFO:tasks.workunit.client.0.smithi042.stdout:7/101: readlink d3/l1d 0 2024-09-16T23:48:50.356 INFO:tasks.workunit.client.0.smithi042.stdout:0/66: creat d0/df/f10 x:0 0 0 2024-09-16T23:48:50.356 INFO:tasks.workunit.client.1.smithi043.stdout:7/322: sync 2024-09-16T23:48:50.357 INFO:tasks.workunit.client.1.smithi043.stdout:2/238: sync 2024-09-16T23:48:50.357 INFO:tasks.workunit.client.1.smithi043.stdout:5/459: sync 2024-09-16T23:48:50.357 INFO:tasks.workunit.client.1.smithi043.stdout:4/310: sync 2024-09-16T23:48:50.357 INFO:tasks.workunit.client.1.smithi043.stdout:6/336: sync 2024-09-16T23:48:50.357 INFO:tasks.workunit.client.1.smithi043.stdout:1/428: sync 2024-09-16T23:48:50.357 INFO:tasks.workunit.client.1.smithi043.stdout:8/325: sync 2024-09-16T23:48:50.357 INFO:tasks.workunit.client.0.smithi042.stdout:0/67: truncate d0/df/f10 704107 0 2024-09-16T23:48:50.358 INFO:tasks.workunit.client.0.smithi042.stdout:6/112: creat d8/db/dd/d1b/f24 x:0 0 0 2024-09-16T23:48:50.358 INFO:tasks.workunit.client.1.smithi043.stdout:5/460: readlink d3/d1b/l6e 0 2024-09-16T23:48:50.358 INFO:tasks.workunit.client.1.smithi043.stdout:2/239: dread - d8/d1d/d20/f40 zero size 2024-09-16T23:48:50.360 INFO:tasks.workunit.client.1.smithi043.stdout:3/326: dread d4/de/d1f/d2c/f32 [0,4194304] 0 2024-09-16T23:48:50.360 INFO:tasks.workunit.client.0.smithi042.stdout:3/80: write d2/d7/fe [333462,129421] 0 2024-09-16T23:48:50.360 INFO:tasks.workunit.client.1.smithi043.stdout:2/240: stat d8/d4b/f5b 0 2024-09-16T23:48:50.362 INFO:tasks.workunit.client.0.smithi042.stdout:7/102: dread d3/d15/d16/f1e [0,4194304] 0 2024-09-16T23:48:50.362 INFO:tasks.workunit.client.1.smithi043.stdout:5/461: chown d3/d9/d22/d8e 983619 1 2024-09-16T23:48:50.363 INFO:tasks.workunit.client.0.smithi042.stdout:4/89: creat de/d1a/d1c/f1f x:0 0 0 2024-09-16T23:48:50.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/136: rmdir d9/db/de 39 2024-09-16T23:48:50.365 INFO:tasks.workunit.client.1.smithi043.stdout:2/241: fdatasync d8/fd 0 2024-09-16T23:48:50.369 INFO:tasks.workunit.client.1.smithi043.stdout:0/322: chown d0/d2/df/f10 36 1 2024-09-16T23:48:50.369 INFO:tasks.workunit.client.0.smithi042.stdout:8/220: truncate da/d19/d1d/f32 598720 0 2024-09-16T23:48:50.371 INFO:tasks.workunit.client.0.smithi042.stdout:6/113: rename d8/db/dd/d1b/f24 to d8/db/dd/f25 0 2024-09-16T23:48:50.371 INFO:tasks.workunit.client.0.smithi042.stdout:8/221: write da/f18 [3584677,111692] 0 2024-09-16T23:48:50.372 INFO:tasks.workunit.client.1.smithi043.stdout:1/429: symlink d4/d9/d72/l8d 0 2024-09-16T23:48:50.372 INFO:tasks.workunit.client.0.smithi042.stdout:6/114: write d8/db/fc [2483125,95820] 0 2024-09-16T23:48:50.377 INFO:tasks.workunit.client.1.smithi043.stdout:6/337: dwrite de/d23/d34/d66/f6e [0,4194304] 0 2024-09-16T23:48:50.377 INFO:tasks.workunit.client.1.smithi043.stdout:8/326: unlink de/d18/d1f/d25/c4b 0 2024-09-16T23:48:50.378 INFO:tasks.workunit.client.1.smithi043.stdout:4/311: unlink d8/d15/d20/d2b/d54/d55/c18 0 2024-09-16T23:48:50.378 INFO:tasks.workunit.client.1.smithi043.stdout:3/327: dwrite d4/f54 [0,4194304] 0 2024-09-16T23:48:50.382 INFO:tasks.workunit.client.1.smithi043.stdout:2/242: creat d8/d1f/f5c x:0 0 0 2024-09-16T23:48:50.383 INFO:tasks.workunit.client.1.smithi043.stdout:0/323: creat d0/de/f69 x:0 0 0 2024-09-16T23:48:50.383 INFO:tasks.workunit.client.0.smithi042.stdout:6/115: dwrite d8/db/dd/f22 [0,4194304] 0 2024-09-16T23:48:50.383 INFO:tasks.workunit.client.0.smithi042.stdout:4/90: unlink de/d1a/f1d 0 2024-09-16T23:48:50.383 INFO:tasks.workunit.client.0.smithi042.stdout:3/81: write d2/d7/f8 [2409185,91274] 0 2024-09-16T23:48:50.384 INFO:tasks.workunit.client.1.smithi043.stdout:6/338: dread de/f40 [0,4194304] 0 2024-09-16T23:48:50.385 INFO:tasks.workunit.client.0.smithi042.stdout:3/82: chown d2/fa 2 1 2024-09-16T23:48:50.386 INFO:tasks.workunit.client.0.smithi042.stdout:2/122: link d0/l7 d0/l27 0 2024-09-16T23:48:50.388 INFO:tasks.workunit.client.0.smithi042.stdout:5/137: truncate d9/db/de/d10/f22 1276377 0 2024-09-16T23:48:50.391 INFO:tasks.workunit.client.1.smithi043.stdout:2/243: dread d8/d1d/d20/d23/d39/f59 [0,4194304] 0 2024-09-16T23:48:50.394 INFO:tasks.workunit.client.0.smithi042.stdout:6/116: mkdir d8/d9/d26 0 2024-09-16T23:48:50.394 INFO:tasks.workunit.client.1.smithi043.stdout:3/328: rmdir d4/de/d1f/d34 39 2024-09-16T23:48:50.396 INFO:tasks.workunit.client.1.smithi043.stdout:8/327: dread fc [0,4194304] 0 2024-09-16T23:48:50.396 INFO:tasks.workunit.client.1.smithi043.stdout:7/323: write d4/f9 [462118,78965] 0 2024-09-16T23:48:50.396 INFO:tasks.workunit.client.0.smithi042.stdout:4/91: rename l4 to de/d1a/l20 0 2024-09-16T23:48:50.397 INFO:tasks.workunit.client.1.smithi043.stdout:6/339: mkdir de/d23/d34/d66/d70 0 2024-09-16T23:48:50.400 INFO:tasks.workunit.client.0.smithi042.stdout:2/123: creat d0/da/d17/f28 x:0 0 0 2024-09-16T23:48:50.400 INFO:tasks.workunit.client.1.smithi043.stdout:5/462: dread d3/db/d15/f5f [0,4194304] 0 2024-09-16T23:48:50.402 INFO:tasks.workunit.client.1.smithi043.stdout:0/324: rename d0/d2/df/d1a/d46/f35 to d0/d2/df/d1a/d46/d39/f6a 0 2024-09-16T23:48:50.402 INFO:tasks.workunit.client.1.smithi043.stdout:5/463: read d3/db/f2a [301257,77002] 0 2024-09-16T23:48:50.404 INFO:tasks.workunit.client.0.smithi042.stdout:2/124: dread d0/da/f21 [0,4194304] 0 2024-09-16T23:48:50.405 INFO:tasks.workunit.client.1.smithi043.stdout:2/244: creat d8/d1f/f5d x:0 0 0 2024-09-16T23:48:50.406 INFO:tasks.workunit.client.1.smithi043.stdout:4/312: dread d8/d15/d20/d2b/f10 [0,4194304] 0 2024-09-16T23:48:50.409 INFO:tasks.workunit.client.1.smithi043.stdout:4/313: write d8/d25/f4f [499834,107673] 0 2024-09-16T23:48:50.409 INFO:tasks.workunit.client.0.smithi042.stdout:0/68: truncate d0/fb 4171067 0 2024-09-16T23:48:50.410 INFO:tasks.workunit.client.0.smithi042.stdout:3/83: creat d2/d9/dd/d17/f19 x:0 0 0 2024-09-16T23:48:50.411 INFO:tasks.workunit.client.0.smithi042.stdout:8/222: write da/f16 [1416430,55916] 0 2024-09-16T23:48:50.413 INFO:tasks.workunit.client.0.smithi042.stdout:4/92: fdatasync de/f19 0 2024-09-16T23:48:50.413 INFO:tasks.workunit.client.0.smithi042.stdout:3/84: truncate d2/d9/dd/f12 831830 0 2024-09-16T23:48:50.414 INFO:tasks.workunit.client.0.smithi042.stdout:3/85: read f0 [3029426,29493] 0 2024-09-16T23:48:50.416 INFO:tasks.workunit.client.0.smithi042.stdout:3/86: truncate d2/d7/fe 1197279 0 2024-09-16T23:48:50.418 INFO:tasks.workunit.client.0.smithi042.stdout:2/125: creat d0/da/d1b/f29 x:0 0 0 2024-09-16T23:48:50.419 INFO:tasks.workunit.client.0.smithi042.stdout:2/126: stat d0/da 0 2024-09-16T23:48:50.421 INFO:tasks.workunit.client.0.smithi042.stdout:3/87: dread d2/d9/dd/f16 [0,4194304] 0 2024-09-16T23:48:50.422 INFO:tasks.workunit.client.1.smithi043.stdout:5/464: mknod d3/d9/d22/d8e/c92 0 2024-09-16T23:48:50.424 INFO:tasks.workunit.client.1.smithi043.stdout:7/324: symlink d4/d27/d41/l6e 0 2024-09-16T23:48:50.424 INFO:tasks.workunit.client.1.smithi043.stdout:0/325: mknod d0/d2/df/d1a/c6b 0 2024-09-16T23:48:50.426 INFO:tasks.workunit.client.0.smithi042.stdout:0/69: symlink d0/l11 0 2024-09-16T23:48:50.428 INFO:tasks.workunit.client.1.smithi043.stdout:4/314: dwrite d8/d15/d20/d2b/f10 [0,4194304] 0 2024-09-16T23:48:50.428 INFO:tasks.workunit.client.0.smithi042.stdout:4/93: creat de/d1a/d1c/f21 x:0 0 0 2024-09-16T23:48:50.428 INFO:tasks.workunit.client.1.smithi043.stdout:3/329: dread d4/de/f24 [0,4194304] 0 2024-09-16T23:48:50.429 INFO:tasks.workunit.client.0.smithi042.stdout:1/147: readlink d0/l7 0 2024-09-16T23:48:50.429 INFO:tasks.workunit.client.0.smithi042.stdout:1/148: fdatasync d0/d9/df/f2b 0 2024-09-16T23:48:50.430 INFO:tasks.workunit.client.0.smithi042.stdout:4/94: truncate de/d1a/d1c/f1f 51158 0 2024-09-16T23:48:50.430 INFO:tasks.workunit.client.1.smithi043.stdout:3/330: readlink d4/de/d1f/d2c/l2d 0 2024-09-16T23:48:50.431 INFO:tasks.workunit.client.0.smithi042.stdout:1/149: write d0/d9/df/f2a [897261,36798] 0 2024-09-16T23:48:50.431 INFO:tasks.workunit.client.1.smithi043.stdout:6/340: creat de/f71 x:0 0 0 2024-09-16T23:48:50.432 INFO:tasks.workunit.client.0.smithi042.stdout:2/127: unlink d0/l7 0 2024-09-16T23:48:50.434 INFO:tasks.workunit.client.0.smithi042.stdout:1/150: mkdir d0/d9/d26/d2c/d2f 0 2024-09-16T23:48:50.436 INFO:tasks.workunit.client.0.smithi042.stdout:3/88: creat d2/d9/d10/f1a x:0 0 0 2024-09-16T23:48:50.437 INFO:tasks.workunit.client.1.smithi043.stdout:8/328: getdents de/d18/d1f/d2f 0 2024-09-16T23:48:50.438 INFO:tasks.workunit.client.0.smithi042.stdout:3/89: truncate d2/d9/dd/f12 1132289 0 2024-09-16T23:48:50.438 INFO:tasks.workunit.client.0.smithi042.stdout:5/138: creat d9/db/f28 x:0 0 0 2024-09-16T23:48:50.438 INFO:tasks.workunit.client.1.smithi043.stdout:2/245: rename d8/d1d/d20/d23/d45 to d8/d5e 0 2024-09-16T23:48:50.439 INFO:tasks.workunit.client.0.smithi042.stdout:1/151: unlink d0/d9/c22 0 2024-09-16T23:48:50.439 INFO:tasks.workunit.client.1.smithi043.stdout:8/329: readlink de/d18/d27/d34/l5c 0 2024-09-16T23:48:50.439 INFO:tasks.workunit.client.1.smithi043.stdout:6/341: symlink de/d2a/l72 0 2024-09-16T23:48:50.440 INFO:tasks.workunit.client.0.smithi042.stdout:5/139: write d9/db/d17/f24 [359003,107673] 0 2024-09-16T23:48:50.441 INFO:tasks.workunit.client.0.smithi042.stdout:1/152: write d0/d17/f21 [192803,16920] 0 2024-09-16T23:48:50.443 INFO:tasks.workunit.client.0.smithi042.stdout:2/128: creat d0/f2a x:0 0 0 2024-09-16T23:48:50.445 INFO:tasks.workunit.client.0.smithi042.stdout:5/140: truncate d9/db/f25 778585 0 2024-09-16T23:48:50.446 INFO:tasks.workunit.client.1.smithi043.stdout:5/465: mknod d3/db/d19/d3e/c93 0 2024-09-16T23:48:50.446 INFO:tasks.workunit.client.1.smithi043.stdout:2/246: mkdir d8/d4b/d5f 0 2024-09-16T23:48:50.448 INFO:tasks.workunit.client.0.smithi042.stdout:3/90: dwrite d2/f4 [0,4194304] 0 2024-09-16T23:48:50.448 INFO:tasks.workunit.client.1.smithi043.stdout:7/325: dwrite d4/f9 [0,4194304] 0 2024-09-16T23:48:50.449 INFO:tasks.workunit.client.1.smithi043.stdout:6/342: rmdir de/d1a/d39 39 2024-09-16T23:48:50.449 INFO:tasks.workunit.client.1.smithi043.stdout:4/315: dwrite d8/d15/d20/d2b/d23/f38 [4194304,4194304] 0 2024-09-16T23:48:50.450 INFO:tasks.workunit.client.0.smithi042.stdout:3/91: write d2/d9/d10/f15 [649398,97063] 0 2024-09-16T23:48:50.450 INFO:tasks.workunit.client.0.smithi042.stdout:8/223: rename da/d19/c35 to da/d19/d1d/d30/d49/c4a 0 2024-09-16T23:48:50.450 INFO:tasks.workunit.client.1.smithi043.stdout:8/330: rmdir de/d18/d27/d41 39 2024-09-16T23:48:50.451 INFO:tasks.workunit.client.0.smithi042.stdout:1/153: truncate d0/d9/df/f1e 1090809 0 2024-09-16T23:48:50.453 INFO:tasks.workunit.client.0.smithi042.stdout:1/154: write d0/d9/f24 [154639,84611] 0 2024-09-16T23:48:50.454 INFO:tasks.workunit.client.0.smithi042.stdout:1/155: chown d0/d9/df/f12 32522031 1 2024-09-16T23:48:50.455 INFO:tasks.workunit.client.1.smithi043.stdout:6/343: fsync de/f24 0 2024-09-16T23:48:50.457 INFO:tasks.workunit.client.0.smithi042.stdout:5/141: link l5 d9/db/d19/d21/l29 0 2024-09-16T23:48:50.459 INFO:tasks.workunit.client.0.smithi042.stdout:1/156: truncate d0/d9/f1c 309308 0 2024-09-16T23:48:50.462 INFO:tasks.workunit.client.0.smithi042.stdout:5/142: mknod d9/c2a 0 2024-09-16T23:48:50.462 INFO:tasks.workunit.client.1.smithi043.stdout:4/316: fdatasync d8/d25/f42 0 2024-09-16T23:48:50.464 INFO:tasks.workunit.client.0.smithi042.stdout:3/92: rmdir d2 39 2024-09-16T23:48:50.466 INFO:tasks.workunit.client.1.smithi043.stdout:7/326: dread - d4/d27/d41/d1b/f4e zero size 2024-09-16T23:48:50.466 INFO:tasks.workunit.client.1.smithi043.stdout:5/466: symlink d3/db/d26/l94 0 2024-09-16T23:48:50.467 INFO:tasks.workunit.client.0.smithi042.stdout:1/157: creat d0/d9/df/f30 x:0 0 0 2024-09-16T23:48:50.468 INFO:tasks.workunit.client.0.smithi042.stdout:1/158: chown d0/d17/f2d 10640 1 2024-09-16T23:48:50.468 INFO:tasks.workunit.client.1.smithi043.stdout:3/331: dwrite d4/de/f58 [0,4194304] 0 2024-09-16T23:48:50.469 INFO:tasks.workunit.client.0.smithi042.stdout:0/70: dread d0/fb [0,4194304] 0 2024-09-16T23:48:50.474 INFO:tasks.workunit.client.0.smithi042.stdout:5/143: creat d9/db/d19/d21/f2b x:0 0 0 2024-09-16T23:48:50.477 INFO:tasks.workunit.client.1.smithi043.stdout:0/326: dwrite d0/d2/df/d1a/d46/d39/f45 [0,4194304] 0 2024-09-16T23:48:50.477 INFO:tasks.workunit.client.0.smithi042.stdout:0/71: mknod d0/df/c12 0 2024-09-16T23:48:50.480 INFO:tasks.workunit.client.1.smithi043.stdout:8/331: dwrite fc [0,4194304] 0 2024-09-16T23:48:50.481 INFO:tasks.workunit.client.1.smithi043.stdout:2/247: dwrite d8/d1f/f5c [0,4194304] 0 2024-09-16T23:48:50.482 INFO:tasks.workunit.client.0.smithi042.stdout:0/72: write d0/f9 [2169994,101309] 0 2024-09-16T23:48:50.484 INFO:tasks.workunit.client.1.smithi043.stdout:0/327: stat d0/d2/df/d1a/d46/d39/c43 0 2024-09-16T23:48:50.484 INFO:tasks.workunit.client.1.smithi043.stdout:5/467: chown d3/db/d26/l94 11173867 1 2024-09-16T23:48:50.485 INFO:tasks.workunit.client.0.smithi042.stdout:3/93: dread d2/d7/f8 [0,4194304] 0 2024-09-16T23:48:50.485 INFO:tasks.workunit.client.0.smithi042.stdout:1/159: fsync d0/f5 0 2024-09-16T23:48:50.486 INFO:tasks.workunit.client.0.smithi042.stdout:3/94: chown f0 46 1 2024-09-16T23:48:50.488 INFO:tasks.workunit.client.1.smithi043.stdout:4/317: dwrite d8/fc [4194304,4194304] 0 2024-09-16T23:48:50.491 INFO:tasks.workunit.client.0.smithi042.stdout:5/144: dwrite d9/db/d19/d21/f2b [0,4194304] 0 2024-09-16T23:48:50.493 INFO:tasks.workunit.client.0.smithi042.stdout:3/95: dread d2/d9/dd/f16 [4194304,4194304] 0 2024-09-16T23:48:50.494 INFO:tasks.workunit.client.0.smithi042.stdout:0/73: dread d0/f9 [0,4194304] 0 2024-09-16T23:48:50.495 INFO:tasks.workunit.client.1.smithi043.stdout:7/327: read d4/db/f18 [2366759,25064] 0 2024-09-16T23:48:50.497 INFO:tasks.workunit.client.1.smithi043.stdout:3/332: write d4/de/d35/f1d [4649020,122778] 0 2024-09-16T23:48:50.498 INFO:tasks.workunit.client.1.smithi043.stdout:4/318: dread d8/d15/d20/d2b/f10 [0,4194304] 0 2024-09-16T23:48:50.498 INFO:tasks.workunit.client.1.smithi043.stdout:5/468: write d3/d9/d22/d60/f82 [414145,15728] 0 2024-09-16T23:48:50.500 INFO:tasks.workunit.client.1.smithi043.stdout:5/469: write d3/db/d19/f54 [251179,76776] 0 2024-09-16T23:48:50.500 INFO:tasks.workunit.client.0.smithi042.stdout:5/145: mkdir d9/d2c 0 2024-09-16T23:48:50.503 INFO:tasks.workunit.client.0.smithi042.stdout:0/74: mkdir d0/df/d13 0 2024-09-16T23:48:50.504 INFO:tasks.workunit.client.1.smithi043.stdout:8/332: mknod de/c61 0 2024-09-16T23:48:50.504 INFO:tasks.workunit.client.0.smithi042.stdout:5/146: creat d9/db/d19/f2d x:0 0 0 2024-09-16T23:48:50.505 INFO:tasks.workunit.client.0.smithi042.stdout:3/96: dwrite d2/d9/d10/f15 [0,4194304] 0 2024-09-16T23:48:50.506 INFO:tasks.workunit.client.1.smithi043.stdout:8/333: fdatasync de/d18/f21 0 2024-09-16T23:48:50.506 INFO:tasks.workunit.client.1.smithi043.stdout:4/319: chown d8/d15/d20/d2b/f10 32378 1 2024-09-16T23:48:50.507 INFO:tasks.workunit.client.0.smithi042.stdout:0/75: write d0/df/f10 [887157,73301] 0 2024-09-16T23:48:50.507 INFO:tasks.workunit.client.0.smithi042.stdout:1/160: rmdir d0/d15/d23 0 2024-09-16T23:48:50.509 INFO:tasks.workunit.client.1.smithi043.stdout:7/328: write d4/f62 [5081898,109206] 0 2024-09-16T23:48:50.510 INFO:tasks.workunit.client.0.smithi042.stdout:5/147: unlink d9/c2a 0 2024-09-16T23:48:50.512 INFO:tasks.workunit.client.1.smithi043.stdout:0/328: write d0/d2/df/f17 [368547,72275] 0 2024-09-16T23:48:50.514 INFO:tasks.workunit.client.0.smithi042.stdout:0/76: creat d0/df/f14 x:0 0 0 2024-09-16T23:48:50.515 INFO:tasks.workunit.client.1.smithi043.stdout:2/248: link d8/d1d/d20/d26/f3c d8/d1d/d20/d26/f60 0 2024-09-16T23:48:50.518 INFO:tasks.workunit.client.1.smithi043.stdout:8/334: chown de/d18/d27/d41/d5a/f26 0 1 2024-09-16T23:48:50.523 INFO:tasks.workunit.client.1.smithi043.stdout:4/320: unlink d8/d15/d20/d2b/d23/f40 0 2024-09-16T23:48:50.524 INFO:tasks.workunit.client.0.smithi042.stdout:5/148: dread d9/db/d11/f1c [0,4194304] 0 2024-09-16T23:48:50.525 INFO:tasks.workunit.client.0.smithi042.stdout:1/161: dwrite d0/f6 [0,4194304] 0 2024-09-16T23:48:50.526 INFO:tasks.workunit.client.1.smithi043.stdout:8/335: dwrite de/f17 [0,4194304] 0 2024-09-16T23:48:50.527 INFO:tasks.workunit.client.1.smithi043.stdout:9/300: sync 2024-09-16T23:48:50.529 INFO:tasks.workunit.client.1.smithi043.stdout:7/329: creat d4/d27/d41/d1b/f6f x:0 0 0 2024-09-16T23:48:50.530 INFO:tasks.workunit.client.1.smithi043.stdout:7/330: write d4/f60 [4617279,38620] 0 2024-09-16T23:48:50.532 INFO:tasks.workunit.client.1.smithi043.stdout:3/333: stat d4/de/d1f/d34/c3c 0 2024-09-16T23:48:50.534 INFO:tasks.workunit.client.0.smithi042.stdout:5/149: creat d9/db/d19/d21/f2e x:0 0 0 2024-09-16T23:48:50.534 INFO:tasks.workunit.client.0.smithi042.stdout:3/97: dwrite d2/d7/f8 [0,4194304] 0 2024-09-16T23:48:50.535 INFO:tasks.workunit.client.1.smithi043.stdout:0/329: creat d0/d2/df/d1a/d3b/f6c x:0 0 0 2024-09-16T23:48:50.536 INFO:tasks.workunit.client.1.smithi043.stdout:0/330: chown d0/lb 58587 1 2024-09-16T23:48:50.537 INFO:tasks.workunit.client.0.smithi042.stdout:0/77: dwrite d0/df/f14 [0,4194304] 0 2024-09-16T23:48:50.537 INFO:tasks.workunit.client.1.smithi043.stdout:3/334: dread d4/de/f58 [0,4194304] 0 2024-09-16T23:48:50.539 INFO:tasks.workunit.client.1.smithi043.stdout:2/249: unlink d8/d1d/d20/d23/d39/f3d 0 2024-09-16T23:48:50.540 INFO:tasks.workunit.client.1.smithi043.stdout:1/430: sync 2024-09-16T23:48:50.540 INFO:tasks.workunit.client.0.smithi042.stdout:5/150: mkdir d9/db/de/d10/d2f 0 2024-09-16T23:48:50.540 INFO:tasks.workunit.client.1.smithi043.stdout:6/344: link de/d1a/d39/f1f de/d1a/d4d/f73 0 2024-09-16T23:48:50.541 INFO:tasks.workunit.client.0.smithi042.stdout:0/78: mkdir d0/d15 0 2024-09-16T23:48:50.542 INFO:tasks.workunit.client.0.smithi042.stdout:0/79: chown d0/f9 47389 1 2024-09-16T23:48:50.543 INFO:tasks.workunit.client.0.smithi042.stdout:3/98: mkdir d2/d1b 0 2024-09-16T23:48:50.545 INFO:tasks.workunit.client.0.smithi042.stdout:0/80: symlink d0/df/l16 0 2024-09-16T23:48:50.546 INFO:tasks.workunit.client.0.smithi042.stdout:3/99: rmdir d2/d9 39 2024-09-16T23:48:50.547 INFO:tasks.workunit.client.0.smithi042.stdout:1/162: symlink d0/d9/d26/d2c/d2f/l31 0 2024-09-16T23:48:50.549 INFO:tasks.workunit.client.0.smithi042.stdout:5/151: rename d9/cc to d9/db/d19/c30 0 2024-09-16T23:48:50.551 INFO:tasks.workunit.client.1.smithi043.stdout:9/301: rmdir d4/d21 39 2024-09-16T23:48:50.553 INFO:tasks.workunit.client.1.smithi043.stdout:8/336: creat de/d18/d27/d34/d3d/f62 x:0 0 0 2024-09-16T23:48:50.553 INFO:tasks.workunit.client.0.smithi042.stdout:5/152: fdatasync d9/db/de/d10/f1a 0 2024-09-16T23:48:50.557 INFO:tasks.workunit.client.0.smithi042.stdout:0/81: dwrite d0/f4 [0,4194304] 0 2024-09-16T23:48:50.558 INFO:tasks.workunit.client.0.smithi042.stdout:5/153: read d9/db/d11/f1c [162354,66014] 0 2024-09-16T23:48:50.558 INFO:tasks.workunit.client.0.smithi042.stdout:3/100: link d2/c3 d2/d9/d10/c1c 0 2024-09-16T23:48:50.561 INFO:tasks.workunit.client.0.smithi042.stdout:0/82: mknod d0/df/c17 0 2024-09-16T23:48:50.564 INFO:tasks.workunit.client.1.smithi043.stdout:0/331: write d0/d2/dd/f60 [203231,118438] 0 2024-09-16T23:48:50.564 INFO:tasks.workunit.client.0.smithi042.stdout:1/163: getdents d0/d17/d18 0 2024-09-16T23:48:50.567 INFO:tasks.workunit.client.0.smithi042.stdout:0/83: creat d0/df/f18 x:0 0 0 2024-09-16T23:48:50.568 INFO:tasks.workunit.client.1.smithi043.stdout:6/345: symlink de/d23/d63/l74 0 2024-09-16T23:48:50.570 INFO:tasks.workunit.client.0.smithi042.stdout:0/84: read d0/df/f10 [834395,124436] 0 2024-09-16T23:48:50.571 INFO:tasks.workunit.client.1.smithi043.stdout:5/470: link d3/db/d15/d4d/d6b/d73/l7d d3/d9/d6f/l95 0 2024-09-16T23:48:50.573 INFO:tasks.workunit.client.1.smithi043.stdout:4/321: fsync d8/d15/d20/d2b/f26 0 2024-09-16T23:48:50.578 INFO:tasks.workunit.client.1.smithi043.stdout:6/346: dwrite de/d1a/f38 [0,4194304] 0 2024-09-16T23:48:50.579 INFO:tasks.workunit.client.0.smithi042.stdout:0/85: dread d0/f7 [0,4194304] 0 2024-09-16T23:48:50.581 INFO:tasks.workunit.client.0.smithi042.stdout:1/164: truncate d0/d9/df/f13 1100229 0 2024-09-16T23:48:50.581 INFO:tasks.workunit.client.0.smithi042.stdout:5/154: truncate d9/db/de/d10/f1a 1518635 0 2024-09-16T23:48:50.582 INFO:tasks.workunit.client.1.smithi043.stdout:9/302: write d4/f31 [2756192,108805] 0 2024-09-16T23:48:50.585 INFO:tasks.workunit.client.0.smithi042.stdout:3/101: rename d2/d9/d10 to d2/d1b/d1d 0 2024-09-16T23:48:50.587 INFO:tasks.workunit.client.1.smithi043.stdout:2/250: creat d8/d4b/d5f/f61 x:0 0 0 2024-09-16T23:48:50.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/165: truncate d0/d9/fe 105710 0 2024-09-16T23:48:50.588 INFO:tasks.workunit.client.0.smithi042.stdout:0/86: getdents d0/df/d13 0 2024-09-16T23:48:50.591 INFO:tasks.workunit.client.0.smithi042.stdout:3/102: rename d2/d9/dd/f16 to d2/d9/dd/f1e 0 2024-09-16T23:48:50.592 INFO:tasks.workunit.client.0.smithi042.stdout:3/103: write d2/f4 [122036,51664] 0 2024-09-16T23:48:50.592 INFO:tasks.workunit.client.1.smithi043.stdout:5/471: unlink d3/d9/f56 0 2024-09-16T23:48:50.593 INFO:tasks.workunit.client.0.smithi042.stdout:1/166: dread d0/d9/f11 [0,4194304] 0 2024-09-16T23:48:50.594 INFO:tasks.workunit.client.1.smithi043.stdout:5/472: write d3/db/f52 [5062384,116724] 0 2024-09-16T23:48:50.599 INFO:tasks.workunit.client.0.smithi042.stdout:5/155: write d9/db/d11/f1c [1143427,86446] 0 2024-09-16T23:48:50.599 INFO:tasks.workunit.client.0.smithi042.stdout:1/167: dread d0/d9/d26/f27 [4194304,4194304] 0 2024-09-16T23:48:50.604 INFO:tasks.workunit.client.1.smithi043.stdout:3/335: symlink d4/de/d40/d4d/l5e 0 2024-09-16T23:48:50.605 INFO:tasks.workunit.client.1.smithi043.stdout:3/336: chown d4/de/f55 0 1 2024-09-16T23:48:50.606 INFO:tasks.workunit.client.1.smithi043.stdout:9/303: chown d4/d21/d33/f55 24227 1 2024-09-16T23:48:50.606 INFO:tasks.workunit.client.0.smithi042.stdout:1/168: write d0/d9/fe [762656,85566] 0 2024-09-16T23:48:50.607 INFO:tasks.workunit.client.0.smithi042.stdout:2/129: dread d0/f1a [0,4194304] 0 2024-09-16T23:48:50.607 INFO:tasks.workunit.client.0.smithi042.stdout:0/87: dwrite d0/fb [4194304,4194304] 0 2024-09-16T23:48:50.610 INFO:tasks.workunit.client.0.smithi042.stdout:3/104: creat d2/f1f x:0 0 0 2024-09-16T23:48:50.610 INFO:tasks.workunit.client.0.smithi042.stdout:0/88: write d0/df/f14 [3257640,52608] 0 2024-09-16T23:48:50.611 INFO:tasks.workunit.client.0.smithi042.stdout:5/156: fsync d9/db/de/d10/f1a 0 2024-09-16T23:48:50.611 INFO:tasks.workunit.client.0.smithi042.stdout:3/105: chown d2/f1f 57232705 1 2024-09-16T23:48:50.612 INFO:tasks.workunit.client.0.smithi042.stdout:2/130: write d0/da/d1b/f29 [424565,15437] 0 2024-09-16T23:48:50.614 INFO:tasks.workunit.client.0.smithi042.stdout:5/157: readlink d9/db/d19/l1d 0 2024-09-16T23:48:50.615 INFO:tasks.workunit.client.1.smithi043.stdout:1/431: rename d4/d3e/d5b/f7f to d4/d9/f8e 0 2024-09-16T23:48:50.615 INFO:tasks.workunit.client.0.smithi042.stdout:5/158: write d9/db/d19/d21/f2b [1291614,94954] 0 2024-09-16T23:48:50.617 INFO:tasks.workunit.client.1.smithi043.stdout:1/432: truncate d4/d9/d12/d2b/f6b 1566107 0 2024-09-16T23:48:50.618 INFO:tasks.workunit.client.0.smithi042.stdout:0/89: rename d0/fb to d0/df/f19 0 2024-09-16T23:48:50.619 INFO:tasks.workunit.client.0.smithi042.stdout:1/169: getdents d0/d17 0 2024-09-16T23:48:50.620 INFO:tasks.workunit.client.1.smithi043.stdout:5/473: write d3/db/d15/f69 [129643,71585] 0 2024-09-16T23:48:50.621 INFO:tasks.workunit.client.0.smithi042.stdout:2/131: creat d0/f2b x:0 0 0 2024-09-16T23:48:50.623 INFO:tasks.workunit.client.0.smithi042.stdout:0/90: write d0/f9 [3754851,106666] 0 2024-09-16T23:48:50.624 INFO:tasks.workunit.client.1.smithi043.stdout:3/337: symlink d4/de/d40/l5f 0 2024-09-16T23:48:50.625 INFO:tasks.workunit.client.0.smithi042.stdout:0/91: chown d0/le 21098222 1 2024-09-16T23:48:50.626 INFO:tasks.workunit.client.0.smithi042.stdout:3/106: rename d2/c11 to d2/d1b/c20 0 2024-09-16T23:48:50.628 INFO:tasks.workunit.client.0.smithi042.stdout:5/159: mkdir d9/db/d1b/d31 0 2024-09-16T23:48:50.630 INFO:tasks.workunit.client.0.smithi042.stdout:2/132: link d0/f2a d0/da/f2c 0 2024-09-16T23:48:50.630 INFO:tasks.workunit.client.0.smithi042.stdout:5/160: read - d9/db/d19/d21/f26 zero size 2024-09-16T23:48:50.630 INFO:tasks.workunit.client.0.smithi042.stdout:5/161: chown d9 1396704 1 2024-09-16T23:48:50.631 INFO:tasks.workunit.client.0.smithi042.stdout:3/107: mkdir d2/d9/dd/d17/d21 0 2024-09-16T23:48:50.631 INFO:tasks.workunit.client.0.smithi042.stdout:5/162: fdatasync d9/db/de/d10/f20 0 2024-09-16T23:48:50.632 INFO:tasks.workunit.client.0.smithi042.stdout:3/108: write d2/d7/f8 [3758684,70768] 0 2024-09-16T23:48:50.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/170: rename d0/d9/d26/d2c to d0/d9/df/d32 0 2024-09-16T23:48:50.633 INFO:tasks.workunit.client.0.smithi042.stdout:0/92: unlink d0/fa 0 2024-09-16T23:48:50.634 INFO:tasks.workunit.client.0.smithi042.stdout:0/93: fsync d0/df/f14 0 2024-09-16T23:48:50.636 INFO:tasks.workunit.client.0.smithi042.stdout:5/163: creat d9/db/d11/f32 x:0 0 0 2024-09-16T23:48:50.636 INFO:tasks.workunit.client.0.smithi042.stdout:0/94: chown d0/cc 1 1 2024-09-16T23:48:50.637 INFO:tasks.workunit.client.0.smithi042.stdout:1/171: dread d0/d9/df/f2a [0,4194304] 0 2024-09-16T23:48:50.637 INFO:tasks.workunit.client.0.smithi042.stdout:2/133: creat d0/da/d17/f2d x:0 0 0 2024-09-16T23:48:50.638 INFO:tasks.workunit.client.0.smithi042.stdout:0/95: dread d0/f7 [0,4194304] 0 2024-09-16T23:48:50.639 INFO:tasks.workunit.client.0.smithi042.stdout:5/164: chown d9/db/d19/c30 1173987 1 2024-09-16T23:48:50.640 INFO:tasks.workunit.client.1.smithi043.stdout:4/322: truncate d8/d15/d20/d2b/f26 3498591 0 2024-09-16T23:48:50.640 INFO:tasks.workunit.client.0.smithi042.stdout:5/165: fsync d9/db/d11/f1c 0 2024-09-16T23:48:50.641 INFO:tasks.workunit.client.1.smithi043.stdout:6/347: write de/d1a/d4d/d50/f61 [1498883,98415] 0 2024-09-16T23:48:50.641 INFO:tasks.workunit.client.0.smithi042.stdout:5/166: rename d9/db/d19 to d9/db/d19/d33 22 2024-09-16T23:48:50.642 INFO:tasks.workunit.client.1.smithi043.stdout:2/251: write d8/d1d/d20/d23/d39/d3a/f57 [256650,62354] 0 2024-09-16T23:48:50.643 INFO:tasks.workunit.client.1.smithi043.stdout:6/348: write de/d1a/d4d/d50/f61 [2361549,24316] 0 2024-09-16T23:48:50.644 INFO:tasks.workunit.client.1.smithi043.stdout:6/349: chown de/d2a/l4b 185 1 2024-09-16T23:48:50.644 INFO:tasks.workunit.client.0.smithi042.stdout:2/134: rename d0/da/c14 to d0/da/c2e 0 2024-09-16T23:48:50.645 INFO:tasks.workunit.client.1.smithi043.stdout:0/332: getdents d0/d2/dd 0 2024-09-16T23:48:50.647 INFO:tasks.workunit.client.1.smithi043.stdout:0/333: fsync d0/d2/df/d1a/f47 0 2024-09-16T23:48:50.647 INFO:tasks.workunit.client.0.smithi042.stdout:3/109: truncate f0 3834645 0 2024-09-16T23:48:50.647 INFO:tasks.workunit.client.0.smithi042.stdout:2/135: truncate d0/da/d1b/f29 1124789 0 2024-09-16T23:48:50.648 INFO:tasks.workunit.client.1.smithi043.stdout:1/433: mkdir d4/d9/d8f 0 2024-09-16T23:48:50.649 INFO:tasks.workunit.client.0.smithi042.stdout:5/167: creat d9/db/de/d10/d2f/f34 x:0 0 0 2024-09-16T23:48:50.649 INFO:tasks.workunit.client.0.smithi042.stdout:1/172: dwrite d0/f3 [0,4194304] 0 2024-09-16T23:48:50.649 INFO:tasks.workunit.client.0.smithi042.stdout:5/168: chown d9/db/de/d10/d2f 176016639 1 2024-09-16T23:48:50.651 INFO:tasks.workunit.client.1.smithi043.stdout:6/350: dwrite de/d1a/d28/f3e [0,4194304] 0 2024-09-16T23:48:50.652 INFO:tasks.workunit.client.0.smithi042.stdout:3/110: link d2/d7/fe d2/d9/f22 0 2024-09-16T23:48:50.657 INFO:tasks.workunit.client.0.smithi042.stdout:9/81: sync 2024-09-16T23:48:50.657 INFO:tasks.workunit.client.0.smithi042.stdout:6/117: sync 2024-09-16T23:48:50.657 INFO:tasks.workunit.client.0.smithi042.stdout:7/103: sync 2024-09-16T23:48:50.658 INFO:tasks.workunit.client.0.smithi042.stdout:6/118: chown d8/d9/f13 994549804 1 2024-09-16T23:48:50.659 INFO:tasks.workunit.client.1.smithi043.stdout:4/323: creat d8/d15/d20/d2b/f5a x:0 0 0 2024-09-16T23:48:50.659 INFO:tasks.workunit.client.0.smithi042.stdout:7/104: chown d3/d15/f1b 154876158 1 2024-09-16T23:48:50.660 INFO:tasks.workunit.client.1.smithi043.stdout:3/338: mknod d4/de/d5d/c60 0 2024-09-16T23:48:50.662 INFO:tasks.workunit.client.1.smithi043.stdout:3/339: chown d4/d30/l4e 1049398 1 2024-09-16T23:48:50.662 INFO:tasks.workunit.client.0.smithi042.stdout:3/111: creat d2/d9/dd/d17/d21/f23 x:0 0 0 2024-09-16T23:48:50.662 INFO:tasks.workunit.client.0.smithi042.stdout:1/173: chown d0/d15/c1a 449 1 2024-09-16T23:48:50.664 INFO:tasks.workunit.client.1.smithi043.stdout:6/351: dwrite de/d23/d63/f6f [0,4194304] 0 2024-09-16T23:48:50.664 INFO:tasks.workunit.client.1.smithi043.stdout:2/252: creat d8/d1d/d20/d23/d39/d3a/f62 x:0 0 0 2024-09-16T23:48:50.664 INFO:tasks.workunit.client.0.smithi042.stdout:7/105: creat d3/d15/d16/d19/f21 x:0 0 0 2024-09-16T23:48:50.665 INFO:tasks.workunit.client.1.smithi043.stdout:3/340: fdatasync d4/de/d1f/d34/f44 0 2024-09-16T23:48:50.667 INFO:tasks.workunit.client.0.smithi042.stdout:3/112: write d2/d9/f22 [553943,105012] 0 2024-09-16T23:48:50.667 INFO:tasks.workunit.client.1.smithi043.stdout:9/304: link d4/d1c/d25/d2b/l3e d4/d21/d2a/d2c/l58 0 2024-09-16T23:48:50.669 INFO:tasks.workunit.client.0.smithi042.stdout:7/106: chown l2 9409736 1 2024-09-16T23:48:50.669 INFO:tasks.workunit.client.1.smithi043.stdout:8/337: rename de/d18/l3c to de/d18/d1f/l63 0 2024-09-16T23:48:50.670 INFO:tasks.workunit.client.0.smithi042.stdout:7/107: fsync d3/f1a 0 2024-09-16T23:48:50.672 INFO:tasks.workunit.client.1.smithi043.stdout:4/324: dwrite d8/d15/f21 [0,4194304] 0 2024-09-16T23:48:50.672 INFO:tasks.workunit.client.1.smithi043.stdout:1/434: mknod d4/d9/d1c/d1e/d42/c90 0 2024-09-16T23:48:50.674 INFO:tasks.workunit.client.0.smithi042.stdout:6/119: dwrite f6 [4194304,4194304] 0 2024-09-16T23:48:50.676 INFO:tasks.workunit.client.0.smithi042.stdout:6/120: rename d8 to d8/d27 22 2024-09-16T23:48:50.680 INFO:tasks.workunit.client.0.smithi042.stdout:2/136: stat d0/da/c2e 0 2024-09-16T23:48:50.680 INFO:tasks.workunit.client.1.smithi043.stdout:4/325: readlink d8/d15/d20/d2b/d54/d55/l43 0 2024-09-16T23:48:50.681 INFO:tasks.workunit.client.0.smithi042.stdout:2/137: fdatasync d0/f2b 0 2024-09-16T23:48:50.682 INFO:tasks.workunit.client.0.smithi042.stdout:9/82: rename d4/d12 to d4/d12/d1c 22 2024-09-16T23:48:50.684 INFO:tasks.workunit.client.0.smithi042.stdout:9/83: readlink d4/ld 0 2024-09-16T23:48:50.684 INFO:tasks.workunit.client.1.smithi043.stdout:1/435: read d4/d9/d12/f5c [2718890,123225] 0 2024-09-16T23:48:50.684 INFO:tasks.workunit.client.1.smithi043.stdout:9/305: dwrite d4/d5/d30/f57 [0,4194304] 0 2024-09-16T23:48:50.684 INFO:tasks.workunit.client.0.smithi042.stdout:7/108: dwrite d3/f1a [0,4194304] 0 2024-09-16T23:48:50.685 INFO:tasks.workunit.client.0.smithi042.stdout:3/113: creat d2/f24 x:0 0 0 2024-09-16T23:48:50.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/109: read d3/d15/d16/f1e [4113276,51669] 0 2024-09-16T23:48:50.687 INFO:tasks.workunit.client.0.smithi042.stdout:2/138: truncate d0/da/f23 478113 0 2024-09-16T23:48:50.689 INFO:tasks.workunit.client.1.smithi043.stdout:6/352: creat de/d1a/d39/f75 x:0 0 0 2024-09-16T23:48:50.694 INFO:tasks.workunit.client.0.smithi042.stdout:7/110: dread d3/f18 [0,4194304] 0 2024-09-16T23:48:50.699 INFO:tasks.workunit.client.0.smithi042.stdout:1/174: dread d0/f5 [0,4194304] 0 2024-09-16T23:48:50.700 INFO:tasks.workunit.client.0.smithi042.stdout:9/84: dwrite d4/d12/d16/f17 [0,4194304] 0 2024-09-16T23:48:50.701 INFO:tasks.workunit.client.0.smithi042.stdout:2/139: creat d0/d26/f2f x:0 0 0 2024-09-16T23:48:50.704 INFO:tasks.workunit.client.0.smithi042.stdout:7/111: mknod d3/d1f/c22 0 2024-09-16T23:48:50.705 INFO:tasks.workunit.client.1.smithi043.stdout:8/338: symlink de/d18/d1f/d2f/l64 0 2024-09-16T23:48:50.706 INFO:tasks.workunit.client.1.smithi043.stdout:8/339: fdatasync de/f17 0 2024-09-16T23:48:50.708 INFO:tasks.workunit.client.0.smithi042.stdout:9/85: chown d4/d10 1 1 2024-09-16T23:48:50.708 INFO:tasks.workunit.client.0.smithi042.stdout:6/121: link d8/l1a d8/db/l28 0 2024-09-16T23:48:50.709 INFO:tasks.workunit.client.0.smithi042.stdout:3/114: dwrite d2/d7/f8 [0,4194304] 0 2024-09-16T23:48:50.714 INFO:tasks.workunit.client.0.smithi042.stdout:7/112: creat d3/d1f/f23 x:0 0 0 2024-09-16T23:48:50.716 INFO:tasks.workunit.client.1.smithi043.stdout:4/326: creat d8/d25/f5b x:0 0 0 2024-09-16T23:48:50.716 INFO:tasks.workunit.client.1.smithi043.stdout:1/436: creat d4/df/f91 x:0 0 0 2024-09-16T23:48:50.716 INFO:tasks.workunit.client.0.smithi042.stdout:6/122: write d8/db/dd/f1f [415699,96755] 0 2024-09-16T23:48:50.716 INFO:tasks.workunit.client.0.smithi042.stdout:1/175: creat d0/d9/df/d32/f33 x:0 0 0 2024-09-16T23:48:50.717 INFO:tasks.workunit.client.1.smithi043.stdout:4/327: stat d8/d15/d20/d2b/d22/f24 0 2024-09-16T23:48:50.718 INFO:tasks.workunit.client.1.smithi043.stdout:1/437: fsync d4/d9/d1c/d1e/d47/f4f 0 2024-09-16T23:48:50.718 INFO:tasks.workunit.client.0.smithi042.stdout:9/86: rename d4/l8 to d4/d12/l1d 0 2024-09-16T23:48:50.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/176: chown d0/d17 54833 1 2024-09-16T23:48:50.723 INFO:tasks.workunit.client.1.smithi043.stdout:3/341: symlink d4/de/l61 0 2024-09-16T23:48:50.725 INFO:tasks.workunit.client.1.smithi043.stdout:3/342: write d4/de/d35/f1d [3304602,127806] 0 2024-09-16T23:48:50.725 INFO:tasks.workunit.client.1.smithi043.stdout:3/343: stat d4/de/f58 0 2024-09-16T23:48:50.729 INFO:tasks.workunit.client.0.smithi042.stdout:7/113: dwrite d3/f1a [0,4194304] 0 2024-09-16T23:48:50.729 INFO:tasks.workunit.client.0.smithi042.stdout:2/140: dwrite d0/da/d17/f1c [0,4194304] 0 2024-09-16T23:48:50.731 INFO:tasks.workunit.client.0.smithi042.stdout:9/87: unlink d4/lf 0 2024-09-16T23:48:50.732 INFO:tasks.workunit.client.0.smithi042.stdout:3/115: dread d2/d9/dd/f1e [0,4194304] 0 2024-09-16T23:48:50.732 INFO:tasks.workunit.client.1.smithi043.stdout:3/344: dwrite d4/de/d40/f45 [0,4194304] 0 2024-09-16T23:48:50.733 INFO:tasks.workunit.client.0.smithi042.stdout:1/177: creat d0/d9/df/d32/d2f/f34 x:0 0 0 2024-09-16T23:48:50.735 INFO:tasks.workunit.client.0.smithi042.stdout:6/123: dwrite d8/db/dd/f15 [0,4194304] 0 2024-09-16T23:48:50.736 INFO:tasks.workunit.client.1.smithi043.stdout:8/340: rmdir de/d18/d27/d41/d5a 39 2024-09-16T23:48:50.737 INFO:tasks.workunit.client.0.smithi042.stdout:7/114: mkdir d3/d15/d24 0 2024-09-16T23:48:50.739 INFO:tasks.workunit.client.1.smithi043.stdout:7/331: sync 2024-09-16T23:48:50.740 INFO:tasks.workunit.client.0.smithi042.stdout:3/116: rename d2/fa to d2/d1b/d1d/f25 0 2024-09-16T23:48:50.745 INFO:tasks.workunit.client.0.smithi042.stdout:7/115: dread d3/f8 [0,4194304] 0 2024-09-16T23:48:50.747 INFO:tasks.workunit.client.1.smithi043.stdout:2/253: dwrite d8/d1d/f2a [0,4194304] 0 2024-09-16T23:48:50.749 INFO:tasks.workunit.client.0.smithi042.stdout:3/117: mknod d2/c26 0 2024-09-16T23:48:50.751 INFO:tasks.workunit.client.0.smithi042.stdout:2/141: mkdir d0/d30 0 2024-09-16T23:48:50.752 INFO:tasks.workunit.client.1.smithi043.stdout:2/254: fdatasync d8/d4b/d5f/f61 0 2024-09-16T23:48:50.756 INFO:tasks.workunit.client.0.smithi042.stdout:7/116: dwrite d3/d15/d16/f1c [0,4194304] 0 2024-09-16T23:48:50.758 INFO:tasks.workunit.client.1.smithi043.stdout:1/438: write d4/d9/d12/f5c [3574506,130076] 0 2024-09-16T23:48:50.759 INFO:tasks.workunit.client.0.smithi042.stdout:3/118: write d2/f4 [1707754,24298] 0 2024-09-16T23:48:50.760 INFO:tasks.workunit.client.0.smithi042.stdout:1/178: mkdir d0/d9/d35 0 2024-09-16T23:48:50.761 INFO:tasks.workunit.client.0.smithi042.stdout:2/142: symlink d0/da/d1b/l31 0 2024-09-16T23:48:50.762 INFO:tasks.workunit.client.1.smithi043.stdout:6/353: creat de/d23/d34/d66/d70/f76 x:0 0 0 2024-09-16T23:48:50.763 INFO:tasks.workunit.client.1.smithi043.stdout:6/354: write de/d1a/f42 [2812768,43085] 0 2024-09-16T23:48:50.763 INFO:tasks.workunit.client.1.smithi043.stdout:2/255: dwrite d8/d11/d16/f49 [0,4194304] 0 2024-09-16T23:48:50.764 INFO:tasks.workunit.client.0.smithi042.stdout:7/117: dread d3/d15/d16/f1c [0,4194304] 0 2024-09-16T23:48:50.765 INFO:tasks.workunit.client.1.smithi043.stdout:0/334: rename d0/de/f58 to d0/d2/f6d 0 2024-09-16T23:48:50.767 INFO:tasks.workunit.client.0.smithi042.stdout:2/143: creat d0/da/f32 x:0 0 0 2024-09-16T23:48:50.769 INFO:tasks.workunit.client.0.smithi042.stdout:1/179: creat d0/d17/f36 x:0 0 0 2024-09-16T23:48:50.771 INFO:tasks.workunit.client.1.smithi043.stdout:4/328: mknod d8/c5c 0 2024-09-16T23:48:50.771 INFO:tasks.workunit.client.0.smithi042.stdout:3/119: rename d2/d1b/c20 to d2/d1b/d1d/c27 0 2024-09-16T23:48:50.775 INFO:tasks.workunit.client.1.smithi043.stdout:2/256: mkdir d8/d4b/d5f/d63 0 2024-09-16T23:48:50.776 INFO:tasks.workunit.client.0.smithi042.stdout:2/144: dwrite d0/f2b [0,4194304] 0 2024-09-16T23:48:50.777 INFO:tasks.workunit.client.0.smithi042.stdout:2/145: stat d0/f1a 0 2024-09-16T23:48:50.780 INFO:tasks.workunit.client.0.smithi042.stdout:9/88: write d4/fa [50090,80028] 0 2024-09-16T23:48:50.780 INFO:tasks.workunit.client.0.smithi042.stdout:9/89: readlink d4/ld 0 2024-09-16T23:48:50.782 INFO:tasks.workunit.client.1.smithi043.stdout:0/335: unlink d0/d2/dd/c5c 0 2024-09-16T23:48:50.783 INFO:tasks.workunit.client.0.smithi042.stdout:3/120: creat d2/f28 x:0 0 0 2024-09-16T23:48:50.783 INFO:tasks.workunit.client.1.smithi043.stdout:0/336: chown d0/d2/df/d1a/d49 2537935 1 2024-09-16T23:48:50.785 INFO:tasks.workunit.client.1.smithi043.stdout:0/337: write d0/d2/f33 [299730,75052] 0 2024-09-16T23:48:50.785 INFO:tasks.workunit.client.0.smithi042.stdout:1/180: rmdir d0/d2e 0 2024-09-16T23:48:50.785 INFO:tasks.workunit.client.0.smithi042.stdout:9/90: dread d4/fb [0,4194304] 0 2024-09-16T23:48:50.786 INFO:tasks.workunit.client.0.smithi042.stdout:6/124: truncate d8/d9/fa 3238655 0 2024-09-16T23:48:50.787 INFO:tasks.workunit.client.0.smithi042.stdout:9/91: chown d4/d10/l11 55527 1 2024-09-16T23:48:50.789 INFO:tasks.workunit.client.0.smithi042.stdout:7/118: write d3/f18 [396229,62910] 0 2024-09-16T23:48:50.789 INFO:tasks.workunit.client.0.smithi042.stdout:9/92: chown d4/le 241953729 1 2024-09-16T23:48:50.790 INFO:tasks.workunit.client.0.smithi042.stdout:2/146: mkdir d0/da/d17/d33 0 2024-09-16T23:48:50.792 INFO:tasks.workunit.client.0.smithi042.stdout:2/147: chown d0/da 130 1 2024-09-16T23:48:50.792 INFO:tasks.workunit.client.0.smithi042.stdout:7/119: creat d3/d15/d16/f25 x:0 0 0 2024-09-16T23:48:50.794 INFO:tasks.workunit.client.0.smithi042.stdout:7/120: write d3/f18 [771222,50899] 0 2024-09-16T23:48:50.795 INFO:tasks.workunit.client.1.smithi043.stdout:9/306: rename d4/c9 to d4/d1c/d4f/c59 0 2024-09-16T23:48:50.795 INFO:tasks.workunit.client.0.smithi042.stdout:2/148: read d0/da/f1f [253039,75069] 0 2024-09-16T23:48:50.796 INFO:tasks.workunit.client.0.smithi042.stdout:6/125: rmdir d8/db 39 2024-09-16T23:48:50.797 INFO:tasks.workunit.client.0.smithi042.stdout:7/121: creat d3/d1f/f26 x:0 0 0 2024-09-16T23:48:50.798 INFO:tasks.workunit.client.0.smithi042.stdout:9/93: dwrite d4/d10/f1b [0,4194304] 0 2024-09-16T23:48:50.798 INFO:tasks.workunit.client.1.smithi043.stdout:7/332: creat d4/db/d50/f70 x:0 0 0 2024-09-16T23:48:50.800 INFO:tasks.workunit.client.0.smithi042.stdout:1/181: symlink d0/d9/d35/l37 0 2024-09-16T23:48:50.807 INFO:tasks.workunit.client.1.smithi043.stdout:2/257: creat d8/d4b/f64 x:0 0 0 2024-09-16T23:48:50.807 INFO:tasks.workunit.client.1.smithi043.stdout:2/258: readlink d8/d1f/l3e 0 2024-09-16T23:48:50.807 INFO:tasks.workunit.client.0.smithi042.stdout:7/122: unlink d3/l17 0 2024-09-16T23:48:50.807 INFO:tasks.workunit.client.0.smithi042.stdout:7/123: stat d3/f11 0 2024-09-16T23:48:50.808 INFO:tasks.workunit.client.1.smithi043.stdout:2/259: write d8/d4b/d5f/f61 [300515,52500] 0 2024-09-16T23:48:50.808 INFO:tasks.workunit.client.0.smithi042.stdout:2/149: mknod d0/da/d17/c34 0 2024-09-16T23:48:50.808 INFO:tasks.workunit.client.0.smithi042.stdout:7/124: symlink d3/l27 0 2024-09-16T23:48:50.809 INFO:tasks.workunit.client.1.smithi043.stdout:2/260: chown d8/d35/f37 0 1 2024-09-16T23:48:50.809 INFO:tasks.workunit.client.0.smithi042.stdout:7/125: chown d3/d15/d16 0 1 2024-09-16T23:48:50.810 INFO:tasks.workunit.client.1.smithi043.stdout:2/261: readlink d8/d1d/d20/d23/d38/l50 0 2024-09-16T23:48:50.810 INFO:tasks.workunit.client.0.smithi042.stdout:7/126: write d3/fa [2965101,68279] 0 2024-09-16T23:48:50.812 INFO:tasks.workunit.client.0.smithi042.stdout:9/94: creat d4/d12/f1e x:0 0 0 2024-09-16T23:48:50.813 INFO:tasks.workunit.client.0.smithi042.stdout:9/95: write d4/fa [1388167,52662] 0 2024-09-16T23:48:50.815 INFO:tasks.workunit.client.1.smithi043.stdout:9/307: dread d4/d1c/d4f/f52 [0,4194304] 0 2024-09-16T23:48:50.817 INFO:tasks.workunit.client.0.smithi042.stdout:1/182: symlink d0/d9/d35/l38 0 2024-09-16T23:48:50.817 INFO:tasks.workunit.client.1.smithi043.stdout:9/308: chown d4/d21/d2a/d2c 300692 1 2024-09-16T23:48:50.818 INFO:tasks.workunit.client.1.smithi043.stdout:9/309: chown d4/d21/d2a/l3f 4601842 1 2024-09-16T23:48:50.819 INFO:tasks.workunit.client.0.smithi042.stdout:9/96: read d4/fa [3480184,86592] 0 2024-09-16T23:48:50.819 INFO:tasks.workunit.client.0.smithi042.stdout:9/97: dread - d4/f19 zero size 2024-09-16T23:48:50.821 INFO:tasks.workunit.client.1.smithi043.stdout:0/338: rename d0/d2/df/d1a/d3b/d27 to d0/d2/df/d1a/d46/d39/d6e 0 2024-09-16T23:48:50.822 INFO:tasks.workunit.client.1.smithi043.stdout:0/339: dread - d0/d2/df/d1a/d3b/f5d zero size 2024-09-16T23:48:50.822 INFO:tasks.workunit.client.0.smithi042.stdout:7/127: getdents d3/d1f 0 2024-09-16T23:48:50.823 INFO:tasks.workunit.client.1.smithi043.stdout:7/333: symlink d4/db/d11/l71 0 2024-09-16T23:48:50.823 INFO:tasks.workunit.client.1.smithi043.stdout:0/340: write d0/f3c [1008629,74419] 0 2024-09-16T23:48:50.824 INFO:tasks.workunit.client.1.smithi043.stdout:7/334: chown d4/db/d11/d21 1374353 1 2024-09-16T23:48:50.824 INFO:tasks.workunit.client.1.smithi043.stdout:7/335: chown d4/d27/d41/f26 292 1 2024-09-16T23:48:50.825 INFO:tasks.workunit.client.0.smithi042.stdout:7/128: truncate d3/fc 1031697 0 2024-09-16T23:48:50.827 INFO:tasks.workunit.client.1.smithi043.stdout:6/355: creat de/d1a/d28/f77 x:0 0 0 2024-09-16T23:48:50.828 INFO:tasks.workunit.client.0.smithi042.stdout:7/129: mkdir d3/d15/d24/d28 0 2024-09-16T23:48:50.829 INFO:tasks.workunit.client.1.smithi043.stdout:3/345: getdents d4/de 0 2024-09-16T23:48:50.830 INFO:tasks.workunit.client.0.smithi042.stdout:7/130: mknod d3/d15/d24/d28/c29 0 2024-09-16T23:48:50.832 INFO:tasks.workunit.client.0.smithi042.stdout:7/131: rmdir d3/d15/d24 39 2024-09-16T23:48:50.834 INFO:tasks.workunit.client.0.smithi042.stdout:7/132: symlink d3/d1f/l2a 0 2024-09-16T23:48:50.835 INFO:tasks.workunit.client.1.smithi043.stdout:2/262: unlink d8/d1f/f30 0 2024-09-16T23:48:50.835 INFO:tasks.workunit.client.0.smithi042.stdout:7/133: readlink d3/l10 0 2024-09-16T23:48:50.835 INFO:tasks.workunit.client.0.smithi042.stdout:2/150: rmdir d0 39 2024-09-16T23:48:50.836 INFO:tasks.workunit.client.1.smithi043.stdout:8/341: dwrite de/d18/d27/d41/d5a/f1b [0,4194304] 0 2024-09-16T23:48:50.836 INFO:tasks.workunit.client.0.smithi042.stdout:7/134: write d3/d1f/f26 [815080,95623] 0 2024-09-16T23:48:50.838 INFO:tasks.workunit.client.0.smithi042.stdout:1/183: mknod d0/d15/c39 0 2024-09-16T23:48:50.842 INFO:tasks.workunit.client.0.smithi042.stdout:9/98: creat d4/d10/f1f x:0 0 0 2024-09-16T23:48:50.843 INFO:tasks.workunit.client.0.smithi042.stdout:9/99: chown d4/d10/f15 9478 1 2024-09-16T23:48:50.843 INFO:tasks.workunit.client.0.smithi042.stdout:7/135: dread d3/f11 [0,4194304] 0 2024-09-16T23:48:50.844 INFO:tasks.workunit.client.0.smithi042.stdout:9/100: dread - d4/d12/f1e zero size 2024-09-16T23:48:50.845 INFO:tasks.workunit.client.0.smithi042.stdout:9/101: stat d4/f5 0 2024-09-16T23:48:50.849 INFO:tasks.workunit.client.0.smithi042.stdout:6/126: creat d8/f29 x:0 0 0 2024-09-16T23:48:50.850 INFO:tasks.workunit.client.1.smithi043.stdout:4/329: creat d8/f5d x:0 0 0 2024-09-16T23:48:50.853 INFO:tasks.workunit.client.1.smithi043.stdout:7/336: symlink d4/db/d28/l72 0 2024-09-16T23:48:50.855 INFO:tasks.workunit.client.1.smithi043.stdout:6/356: creat de/d1a/d39/f78 x:0 0 0 2024-09-16T23:48:50.856 INFO:tasks.workunit.client.1.smithi043.stdout:5/474: sync 2024-09-16T23:48:50.856 INFO:tasks.workunit.client.1.smithi043.stdout:7/337: fdatasync d4/d27/d41/f40 0 2024-09-16T23:48:50.857 INFO:tasks.workunit.client.0.smithi042.stdout:7/136: write d3/fc [1679877,115081] 0 2024-09-16T23:48:50.858 INFO:tasks.workunit.client.0.smithi042.stdout:1/184: creat d0/d9/d35/f3a x:0 0 0 2024-09-16T23:48:50.860 INFO:tasks.workunit.client.1.smithi043.stdout:5/475: chown d3/d9/d6f/l74 90465814 1 2024-09-16T23:48:50.860 INFO:tasks.workunit.client.0.smithi042.stdout:1/185: dread - d0/d17/f36 zero size 2024-09-16T23:48:50.861 INFO:tasks.workunit.client.0.smithi042.stdout:7/137: mknod d3/d15/d16/c2b 0 2024-09-16T23:48:50.862 INFO:tasks.workunit.client.1.smithi043.stdout:4/330: dwrite d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:48:50.864 INFO:tasks.workunit.client.0.smithi042.stdout:9/102: mknod d4/d10/c20 0 2024-09-16T23:48:50.864 INFO:tasks.workunit.client.0.smithi042.stdout:7/138: mknod d3/d15/d16/c2c 0 2024-09-16T23:48:50.867 INFO:tasks.workunit.client.1.smithi043.stdout:2/263: rmdir d8/d4b/d5f 39 2024-09-16T23:48:50.869 INFO:tasks.workunit.client.1.smithi043.stdout:7/338: dwrite d4/d27/f6c [0,4194304] 0 2024-09-16T23:48:50.870 INFO:tasks.workunit.client.1.smithi043.stdout:8/342: mkdir de/d18/d1f/d25/d65 0 2024-09-16T23:48:50.872 INFO:tasks.workunit.client.0.smithi042.stdout:7/139: write d3/f18 [746704,124050] 0 2024-09-16T23:48:50.873 INFO:tasks.workunit.client.1.smithi043.stdout:2/264: read d8/d35/f37 [451331,33060] 0 2024-09-16T23:48:50.873 INFO:tasks.workunit.client.0.smithi042.stdout:6/127: write d8/db/dd/f22 [2704440,81535] 0 2024-09-16T23:48:50.874 INFO:tasks.workunit.client.0.smithi042.stdout:6/128: write d8/db/dd/f1c [101464,68153] 0 2024-09-16T23:48:50.877 INFO:tasks.workunit.client.0.smithi042.stdout:7/140: read d3/d15/d16/f1c [1975365,57281] 0 2024-09-16T23:48:50.878 INFO:tasks.workunit.client.0.smithi042.stdout:9/103: dwrite d4/f19 [0,4194304] 0 2024-09-16T23:48:50.882 INFO:tasks.workunit.client.0.smithi042.stdout:2/151: mknod d0/da/d17/d33/c35 0 2024-09-16T23:48:50.883 INFO:tasks.workunit.client.1.smithi043.stdout:4/331: mkdir d8/d15/d20/d2b/d54/d55/d5e 0 2024-09-16T23:48:50.884 INFO:tasks.workunit.client.1.smithi043.stdout:5/476: dread d3/f38 [0,4194304] 0 2024-09-16T23:48:50.885 INFO:tasks.workunit.client.0.smithi042.stdout:1/186: mkdir d0/d3b 0 2024-09-16T23:48:50.887 INFO:tasks.workunit.client.0.smithi042.stdout:1/187: truncate d0/d9/df/f2b 106767 0 2024-09-16T23:48:50.889 INFO:tasks.workunit.client.0.smithi042.stdout:7/141: link d3/d15/d16/d19/f21 d3/d15/d24/f2d 0 2024-09-16T23:48:50.892 INFO:tasks.workunit.client.0.smithi042.stdout:7/142: mkdir d3/d15/d16/d19/d2e 0 2024-09-16T23:48:50.894 INFO:tasks.workunit.client.0.smithi042.stdout:7/143: write d3/d1f/f23 [182885,85408] 0 2024-09-16T23:48:50.894 INFO:tasks.workunit.client.0.smithi042.stdout:2/152: symlink d0/da/l36 0 2024-09-16T23:48:50.897 INFO:tasks.workunit.client.0.smithi042.stdout:7/144: symlink d3/d15/d16/l2f 0 2024-09-16T23:48:50.900 INFO:tasks.workunit.client.0.smithi042.stdout:7/145: getdents d3/d1f 0 2024-09-16T23:48:50.901 INFO:tasks.workunit.client.1.smithi043.stdout:5/477: dread d3/db/d19/d3e/f35 [4194304,4194304] 0 2024-09-16T23:48:50.902 INFO:tasks.workunit.client.0.smithi042.stdout:7/146: creat d3/f30 x:0 0 0 2024-09-16T23:48:50.903 INFO:tasks.workunit.client.0.smithi042.stdout:9/104: symlink d4/l21 0 2024-09-16T23:48:50.903 INFO:tasks.workunit.client.0.smithi042.stdout:9/105: chown d4/ld 5 1 2024-09-16T23:48:50.904 INFO:tasks.workunit.client.0.smithi042.stdout:1/188: symlink d0/l3c 0 2024-09-16T23:48:50.905 INFO:tasks.workunit.client.0.smithi042.stdout:9/106: read d4/d10/f1b [1062196,125800] 0 2024-09-16T23:48:50.906 INFO:tasks.workunit.client.1.smithi043.stdout:3/346: dwrite d4/de/d40/f45 [4194304,4194304] 0 2024-09-16T23:48:50.908 INFO:tasks.workunit.client.0.smithi042.stdout:7/147: rmdir d3/d15/d16/d19/d2e 0 2024-09-16T23:48:50.912 INFO:tasks.workunit.client.0.smithi042.stdout:6/129: getdents d8 0 2024-09-16T23:48:50.914 INFO:tasks.workunit.client.0.smithi042.stdout:7/148: truncate d3/d15/f1b 1672342 0 2024-09-16T23:48:50.917 INFO:tasks.workunit.client.0.smithi042.stdout:7/149: write d3/d15/d16/f25 [747192,79266] 0 2024-09-16T23:48:50.918 INFO:tasks.workunit.client.0.smithi042.stdout:6/130: dwrite d8/db/f12 [0,4194304] 0 2024-09-16T23:48:50.922 INFO:tasks.workunit.client.1.smithi043.stdout:9/310: link d4/l17 d4/d28/d49/l5a 0 2024-09-16T23:48:50.923 INFO:tasks.workunit.client.0.smithi042.stdout:7/150: mknod d3/d15/d16/d19/c31 0 2024-09-16T23:48:50.925 INFO:tasks.workunit.client.1.smithi043.stdout:7/339: write d4/f2b [910722,24484] 0 2024-09-16T23:48:50.930 INFO:tasks.workunit.client.1.smithi043.stdout:9/311: dread d4/f31 [0,4194304] 0 2024-09-16T23:48:50.931 INFO:tasks.workunit.client.1.smithi043.stdout:6/357: truncate de/d1a/f38 1163000 0 2024-09-16T23:48:50.932 INFO:tasks.workunit.client.0.smithi042.stdout:7/151: dwrite d3/d1f/f26 [0,4194304] 0 2024-09-16T23:48:50.933 INFO:tasks.workunit.client.1.smithi043.stdout:6/358: readlink de/d23/d34/l3f 0 2024-09-16T23:48:50.934 INFO:tasks.workunit.client.0.smithi042.stdout:9/107: rename l3 to d4/d10/l22 0 2024-09-16T23:48:50.934 INFO:tasks.workunit.client.0.smithi042.stdout:9/108: chown d4/d10/f1b 752376 1 2024-09-16T23:48:50.935 INFO:tasks.workunit.client.1.smithi043.stdout:5/478: write d3/db/d19/d3e/d28/f42 [8810481,38570] 0 2024-09-16T23:48:50.935 INFO:tasks.workunit.client.0.smithi042.stdout:4/95: sync 2024-09-16T23:48:50.936 INFO:tasks.workunit.client.0.smithi042.stdout:8/224: sync 2024-09-16T23:48:50.936 INFO:tasks.workunit.client.1.smithi043.stdout:2/265: dwrite d8/d11/d16/f28 [0,4194304] 0 2024-09-16T23:48:50.936 INFO:tasks.workunit.client.0.smithi042.stdout:6/131: mknod d8/db/dd/d1b/c2a 0 2024-09-16T23:48:50.937 INFO:tasks.workunit.client.0.smithi042.stdout:4/96: fsync de/d12/f14 0 2024-09-16T23:48:50.937 INFO:tasks.workunit.client.1.smithi043.stdout:2/266: chown d8/d35/f36 14939 1 2024-09-16T23:48:50.938 INFO:tasks.workunit.client.0.smithi042.stdout:4/97: stat de/d1a/d1c/f1f 0 2024-09-16T23:48:50.941 INFO:tasks.workunit.client.0.smithi042.stdout:4/98: symlink de/l22 0 2024-09-16T23:48:50.941 INFO:tasks.workunit.client.1.smithi043.stdout:1/439: sync 2024-09-16T23:48:50.943 INFO:tasks.workunit.client.0.smithi042.stdout:9/109: symlink d4/d12/l23 0 2024-09-16T23:48:50.944 INFO:tasks.workunit.client.1.smithi043.stdout:1/440: chown d4/d9/d1c/d1e/d47/c8c 166264394 1 2024-09-16T23:48:50.945 INFO:tasks.workunit.client.1.smithi043.stdout:5/479: dwrite d3/db/f52 [0,4194304] 0 2024-09-16T23:48:50.946 INFO:tasks.workunit.client.0.smithi042.stdout:4/99: mknod de/d1a/c23 0 2024-09-16T23:48:50.947 INFO:tasks.workunit.client.0.smithi042.stdout:4/100: dread de/d1a/d1c/f1f [0,4194304] 0 2024-09-16T23:48:50.949 INFO:tasks.workunit.client.0.smithi042.stdout:4/101: symlink de/d1a/l24 0 2024-09-16T23:48:50.951 INFO:tasks.workunit.client.1.smithi043.stdout:6/359: dread de/d1a/d4d/f73 [0,4194304] 0 2024-09-16T23:48:50.954 INFO:tasks.workunit.client.0.smithi042.stdout:4/102: dread de/f19 [0,4194304] 0 2024-09-16T23:48:50.995 INFO:tasks.workunit.client.0.smithi042.stdout:4/103: link de/f19 de/d12/f25 0 2024-09-16T23:48:50.996 INFO:tasks.workunit.client.1.smithi043.stdout:8/343: truncate de/d18/d1f/d25/f5f 3460068 0 2024-09-16T23:48:50.996 INFO:tasks.workunit.client.0.smithi042.stdout:4/104: chown c1 1288 1 2024-09-16T23:48:50.997 INFO:tasks.workunit.client.1.smithi043.stdout:3/347: symlink d4/de/d1f/d2c/l62 0 2024-09-16T23:48:50.999 INFO:tasks.workunit.client.0.smithi042.stdout:1/189: link d0/l3c d0/d9/l3d 0 2024-09-16T23:48:51.003 INFO:tasks.workunit.client.0.smithi042.stdout:9/110: mkdir d4/d10/d24 0 2024-09-16T23:48:51.003 INFO:tasks.workunit.client.1.smithi043.stdout:4/332: write d8/d15/d20/d2b/f26 [1620238,41142] 0 2024-09-16T23:48:51.007 INFO:tasks.workunit.client.0.smithi042.stdout:4/105: write de/f10 [1231783,11872] 0 2024-09-16T23:48:51.007 INFO:tasks.workunit.client.0.smithi042.stdout:1/190: dwrite d0/d9/df/f12 [0,4194304] 0 2024-09-16T23:48:51.007 INFO:tasks.workunit.client.0.smithi042.stdout:4/106: fsync fd 0 2024-09-16T23:48:51.008 INFO:tasks.workunit.client.1.smithi043.stdout:0/341: sync 2024-09-16T23:48:51.008 INFO:tasks.workunit.client.0.smithi042.stdout:4/107: chown de/d12/d1b 61010 1 2024-09-16T23:48:51.009 INFO:tasks.workunit.client.1.smithi043.stdout:0/342: write d0/d2/df/d1a/d46/d39/f62 [240790,57509] 0 2024-09-16T23:48:51.011 INFO:tasks.workunit.client.0.smithi042.stdout:4/108: symlink de/d12/d1b/l26 0 2024-09-16T23:48:51.013 INFO:tasks.workunit.client.0.smithi042.stdout:4/109: symlink de/d1a/d1c/l27 0 2024-09-16T23:48:51.013 INFO:tasks.workunit.client.0.smithi042.stdout:4/110: dread - fd zero size 2024-09-16T23:48:51.015 INFO:tasks.workunit.client.0.smithi042.stdout:4/111: creat de/d12/f28 x:0 0 0 2024-09-16T23:48:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:50 smithi042 ceph-mon[27914]: Deploying cephadm binary to smithi043 2024-09-16T23:48:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:50 smithi042 ceph-mon[27914]: pgmap v4: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:50 smithi042 ceph-mon[27914]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:48:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:50 smithi042 ceph-mon[27914]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:51.023 INFO:tasks.workunit.client.0.smithi042.stdout:4/112: write de/f19 [3406796,95636] 0 2024-09-16T23:48:51.025 INFO:tasks.workunit.client.0.smithi042.stdout:4/113: readlink de/d1a/l20 0 2024-09-16T23:48:51.026 INFO:tasks.workunit.client.0.smithi042.stdout:4/114: symlink de/d12/d1b/l29 0 2024-09-16T23:48:51.028 INFO:tasks.workunit.client.0.smithi042.stdout:4/115: symlink de/d1a/d1c/l2a 0 2024-09-16T23:48:51.032 INFO:tasks.workunit.client.0.smithi042.stdout:4/116: write de/d1a/d1c/f1f [205639,71235] 0 2024-09-16T23:48:51.033 INFO:tasks.workunit.client.0.smithi042.stdout:4/117: rmdir de 39 2024-09-16T23:48:51.034 INFO:tasks.workunit.client.0.smithi042.stdout:4/118: fsync de/d12/f15 0 2024-09-16T23:48:51.092 INFO:tasks.workunit.client.1.smithi043.stdout:9/312: creat d4/d1c/d4f/f5b x:0 0 0 2024-09-16T23:48:51.093 INFO:tasks.workunit.client.1.smithi043.stdout:9/313: write d4/d21/f4d [36267,81454] 0 2024-09-16T23:48:51.098 INFO:tasks.workunit.client.1.smithi043.stdout:9/314: dread d4/f48 [0,4194304] 0 2024-09-16T23:48:51.099 INFO:tasks.workunit.client.0.smithi042.stdout:1/191: rename d0/d9 to d0/d9/d3e 22 2024-09-16T23:48:51.100 INFO:tasks.workunit.client.0.smithi042.stdout:1/192: write d0/d17/f20 [35107,53498] 0 2024-09-16T23:48:51.101 INFO:tasks.workunit.client.0.smithi042.stdout:1/193: readlink d0/d17/l19 0 2024-09-16T23:48:51.102 INFO:tasks.workunit.client.0.smithi042.stdout:9/111: mkdir d4/d10/d24/d25 0 2024-09-16T23:48:51.102 INFO:tasks.workunit.client.0.smithi042.stdout:4/119: rename de/l11 to de/d1a/l2b 0 2024-09-16T23:48:51.103 INFO:tasks.workunit.client.1.smithi043.stdout:2/267: dread d8/d4b/d5f/f61 [0,4194304] 0 2024-09-16T23:48:51.104 INFO:tasks.workunit.client.0.smithi042.stdout:8/225: getdents da/d19/d2c 0 2024-09-16T23:48:51.104 INFO:tasks.workunit.client.0.smithi042.stdout:4/120: write de/f10 [4212782,14562] 0 2024-09-16T23:48:51.106 INFO:tasks.workunit.client.0.smithi042.stdout:1/194: mknod d0/d9/df/d32/d2f/c3f 0 2024-09-16T23:48:51.108 INFO:tasks.workunit.client.0.smithi042.stdout:9/112: dread d4/d12/d16/f17 [0,4194304] 0 2024-09-16T23:48:51.109 INFO:tasks.workunit.client.1.smithi043.stdout:1/441: rename d4/d9/d1c/l89 to d4/d9/d12/d2b/d3d/d83/l92 0 2024-09-16T23:48:51.110 INFO:tasks.workunit.client.0.smithi042.stdout:4/121: truncate de/d12/f14 517141 0 2024-09-16T23:48:51.111 INFO:tasks.workunit.client.1.smithi043.stdout:6/360: creat de/d1a/d4d/d50/f79 x:0 0 0 2024-09-16T23:48:51.112 INFO:tasks.workunit.client.1.smithi043.stdout:8/344: symlink de/d18/d27/d41/d5a/d37/l66 0 2024-09-16T23:48:51.112 INFO:tasks.workunit.client.0.smithi042.stdout:4/122: creat de/d1a/d1c/f2c x:0 0 0 2024-09-16T23:48:51.113 INFO:tasks.workunit.client.0.smithi042.stdout:4/123: chown de/l22 495492 1 2024-09-16T23:48:51.114 INFO:tasks.workunit.client.1.smithi043.stdout:3/348: symlink d4/d30/l63 0 2024-09-16T23:48:51.115 INFO:tasks.workunit.client.0.smithi042.stdout:4/124: creat de/d12/d1b/f2d x:0 0 0 2024-09-16T23:48:51.116 INFO:tasks.workunit.client.0.smithi042.stdout:4/125: write fa [4583326,126788] 0 2024-09-16T23:48:51.117 INFO:tasks.workunit.client.0.smithi042.stdout:9/113: truncate d4/fb 631249 0 2024-09-16T23:48:51.121 INFO:tasks.workunit.client.0.smithi042.stdout:4/126: chown de/d12/c13 1894072 1 2024-09-16T23:48:51.122 INFO:tasks.workunit.client.1.smithi043.stdout:4/333: creat d8/d3d/f5f x:0 0 0 2024-09-16T23:48:51.124 INFO:tasks.workunit.client.0.smithi042.stdout:9/114: read d4/d10/f1b [1290062,104853] 0 2024-09-16T23:48:51.125 INFO:tasks.workunit.client.0.smithi042.stdout:4/127: dwrite f7 [0,4194304] 0 2024-09-16T23:48:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:50 smithi043 ceph-mon[35191]: Deploying cephadm binary to smithi043 2024-09-16T23:48:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:50 smithi043 ceph-mon[35191]: pgmap v4: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:50 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:48:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:50 smithi043 ceph-mon[35191]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:48:51.128 INFO:tasks.workunit.client.0.smithi042.stdout:4/128: fdatasync de/d12/f14 0 2024-09-16T23:48:51.131 INFO:tasks.workunit.client.0.smithi042.stdout:8/226: rename da/d19/d46 to da/d4b 0 2024-09-16T23:48:51.135 INFO:tasks.workunit.client.0.smithi042.stdout:4/129: dread de/d1a/d1c/f1f [0,4194304] 0 2024-09-16T23:48:51.137 INFO:tasks.workunit.client.0.smithi042.stdout:4/130: mkdir de/d2e 0 2024-09-16T23:48:51.138 INFO:tasks.workunit.client.0.smithi042.stdout:3/121: sync 2024-09-16T23:48:51.138 INFO:tasks.workunit.client.0.smithi042.stdout:5/169: sync 2024-09-16T23:48:51.138 INFO:tasks.workunit.client.0.smithi042.stdout:0/96: sync 2024-09-16T23:48:51.139 INFO:tasks.workunit.client.1.smithi043.stdout:9/315: dread d4/f31 [0,4194304] 0 2024-09-16T23:48:51.141 INFO:tasks.workunit.client.0.smithi042.stdout:8/227: stat da/d19/d1d/d30/d3f/c40 0 2024-09-16T23:48:51.142 INFO:tasks.workunit.client.0.smithi042.stdout:4/131: creat de/f2f x:0 0 0 2024-09-16T23:48:51.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/170: write d9/db/d11/f32 [613693,108544] 0 2024-09-16T23:48:51.142 INFO:tasks.workunit.client.0.smithi042.stdout:3/122: fdatasync f0 0 2024-09-16T23:48:51.144 INFO:tasks.workunit.client.0.smithi042.stdout:1/195: dwrite d0/d9/df/f13 [0,4194304] 0 2024-09-16T23:48:51.145 INFO:tasks.workunit.client.1.smithi043.stdout:6/361: rmdir de/d23/d34/d66 39 2024-09-16T23:48:51.146 INFO:tasks.workunit.client.0.smithi042.stdout:4/132: dread - de/d1a/d1c/f2c zero size 2024-09-16T23:48:51.146 INFO:tasks.workunit.client.1.smithi043.stdout:2/268: dwrite d8/d1d/d20/d23/d39/f59 [0,4194304] 0 2024-09-16T23:48:51.147 INFO:tasks.workunit.client.0.smithi042.stdout:1/196: readlink d0/d9/d35/l38 0 2024-09-16T23:48:51.148 INFO:tasks.workunit.client.0.smithi042.stdout:5/171: write d9/db/de/d10/f22 [1020313,104553] 0 2024-09-16T23:48:51.150 INFO:tasks.workunit.client.0.smithi042.stdout:1/197: truncate d0/d17/f36 394336 0 2024-09-16T23:48:51.150 INFO:tasks.workunit.client.0.smithi042.stdout:3/123: mknod d2/d9/dd/c29 0 2024-09-16T23:48:51.153 INFO:tasks.workunit.client.1.smithi043.stdout:3/349: symlink d4/d30/d53/l64 0 2024-09-16T23:48:51.153 INFO:tasks.workunit.client.0.smithi042.stdout:9/115: write d4/d12/d16/f17 [2193185,21469] 0 2024-09-16T23:48:51.156 INFO:tasks.workunit.client.0.smithi042.stdout:4/133: rename de/d1a/d1c/l27 to de/d1a/d1c/l30 0 2024-09-16T23:48:51.156 INFO:tasks.workunit.client.0.smithi042.stdout:8/228: write da/d19/d1d/d30/f31 [979013,5762] 0 2024-09-16T23:48:51.157 INFO:tasks.workunit.client.0.smithi042.stdout:3/124: creat d2/d1b/f2a x:0 0 0 2024-09-16T23:48:51.158 INFO:tasks.workunit.client.1.smithi043.stdout:7/340: creat d4/db/f73 x:0 0 0 2024-09-16T23:48:51.159 INFO:tasks.workunit.client.0.smithi042.stdout:3/125: fdatasync d2/f1f 0 2024-09-16T23:48:51.159 INFO:tasks.workunit.client.0.smithi042.stdout:1/198: mknod d0/d17/d18/c40 0 2024-09-16T23:48:51.160 INFO:tasks.workunit.client.0.smithi042.stdout:4/134: creat de/f31 x:0 0 0 2024-09-16T23:48:51.162 INFO:tasks.workunit.client.1.smithi043.stdout:4/334: creat d8/d25/d39/d41/f60 x:0 0 0 2024-09-16T23:48:51.163 INFO:tasks.workunit.client.0.smithi042.stdout:0/97: dwrite d0/f7 [0,4194304] 0 2024-09-16T23:48:51.164 INFO:tasks.workunit.client.0.smithi042.stdout:4/135: dread - fd zero size 2024-09-16T23:48:51.164 INFO:tasks.workunit.client.0.smithi042.stdout:8/229: chown da/f18 7993915 1 2024-09-16T23:48:51.168 INFO:tasks.workunit.client.0.smithi042.stdout:4/136: dread - de/d1a/d1c/f21 zero size 2024-09-16T23:48:51.169 INFO:tasks.workunit.client.0.smithi042.stdout:5/172: link d9/db/d19/l27 d9/db/l35 0 2024-09-16T23:48:51.169 INFO:tasks.workunit.client.1.smithi043.stdout:5/480: creat d3/f96 x:0 0 0 2024-09-16T23:48:51.170 INFO:tasks.workunit.client.0.smithi042.stdout:5/173: chown d9/db/d19 50529632 1 2024-09-16T23:48:51.170 INFO:tasks.workunit.client.0.smithi042.stdout:3/126: dread d2/d7/f8 [0,4194304] 0 2024-09-16T23:48:51.171 INFO:tasks.workunit.client.0.smithi042.stdout:4/137: dread de/d1a/d1c/f1f [0,4194304] 0 2024-09-16T23:48:51.172 INFO:tasks.workunit.client.0.smithi042.stdout:0/98: dread d0/df/f14 [0,4194304] 0 2024-09-16T23:48:51.172 INFO:tasks.workunit.client.1.smithi043.stdout:1/442: link d4/d34/f3a d4/d9/d1c/d1e/d42/d5d/f93 0 2024-09-16T23:48:51.172 INFO:tasks.workunit.client.0.smithi042.stdout:4/138: dread - de/d12/f28 zero size 2024-09-16T23:48:51.174 INFO:tasks.workunit.client.0.smithi042.stdout:3/127: mkdir d2/d9/dd/d2b 0 2024-09-16T23:48:51.175 INFO:tasks.workunit.client.0.smithi042.stdout:1/199: creat d0/d9/f41 x:0 0 0 2024-09-16T23:48:51.176 INFO:tasks.workunit.client.0.smithi042.stdout:5/174: creat d9/db/de/d10/d2f/f36 x:0 0 0 2024-09-16T23:48:51.177 INFO:tasks.workunit.client.0.smithi042.stdout:0/99: truncate d0/df/f10 1374738 0 2024-09-16T23:48:51.178 INFO:tasks.workunit.client.1.smithi043.stdout:3/350: mknod d4/de/d40/d4d/c65 0 2024-09-16T23:48:51.179 INFO:tasks.workunit.client.1.smithi043.stdout:3/351: chown d4/de/d35/f1d 30 1 2024-09-16T23:48:51.179 INFO:tasks.workunit.client.1.smithi043.stdout:7/341: symlink d4/db/d11/l74 0 2024-09-16T23:48:51.179 INFO:tasks.workunit.client.0.smithi042.stdout:4/139: dread de/d12/f14 [0,4194304] 0 2024-09-16T23:48:51.181 INFO:tasks.workunit.client.1.smithi043.stdout:4/335: truncate d8/d15/f3c 3072348 0 2024-09-16T23:48:51.181 INFO:tasks.workunit.client.0.smithi042.stdout:0/100: mknod d0/df/c1a 0 2024-09-16T23:48:51.184 INFO:tasks.workunit.client.0.smithi042.stdout:1/200: rename d0/d17/d18/l29 to d0/d9/df/d32/d2f/l42 0 2024-09-16T23:48:51.185 INFO:tasks.workunit.client.0.smithi042.stdout:4/140: mknod de/d2e/c32 0 2024-09-16T23:48:51.187 INFO:tasks.workunit.client.1.smithi043.stdout:1/443: rmdir d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a 39 2024-09-16T23:48:51.188 INFO:tasks.workunit.client.0.smithi042.stdout:0/101: creat d0/f1b x:0 0 0 2024-09-16T23:48:51.189 INFO:tasks.workunit.client.1.smithi043.stdout:1/444: dread - d4/d9/d72/f73 zero size 2024-09-16T23:48:51.189 INFO:tasks.workunit.client.0.smithi042.stdout:3/128: dwrite d2/f1f [0,4194304] 0 2024-09-16T23:48:51.190 INFO:tasks.workunit.client.0.smithi042.stdout:0/102: read d0/f9 [1599850,44310] 0 2024-09-16T23:48:51.190 INFO:tasks.workunit.client.1.smithi043.stdout:1/445: readlink d4/d9/d12/d2b/l7d 0 2024-09-16T23:48:51.191 INFO:tasks.workunit.client.1.smithi043.stdout:2/269: dread d8/fc [0,4194304] 0 2024-09-16T23:48:51.191 INFO:tasks.workunit.client.0.smithi042.stdout:0/103: fsync d0/f7 0 2024-09-16T23:48:51.191 INFO:tasks.workunit.client.0.smithi042.stdout:5/175: dwrite d9/db/de/d10/f1a [0,4194304] 0 2024-09-16T23:48:51.192 INFO:tasks.workunit.client.1.smithi043.stdout:2/270: write d8/d1d/f2a [2726125,17650] 0 2024-09-16T23:48:51.194 INFO:tasks.workunit.client.1.smithi043.stdout:3/352: dread d4/de/d1f/f29 [0,4194304] 0 2024-09-16T23:48:51.196 INFO:tasks.workunit.client.1.smithi043.stdout:8/345: symlink de/d18/d1f/d2f/l67 0 2024-09-16T23:48:51.197 INFO:tasks.workunit.client.0.smithi042.stdout:1/201: mkdir d0/d9/d35/d43 0 2024-09-16T23:48:51.201 INFO:tasks.workunit.client.1.smithi043.stdout:7/342: mkdir d4/d27/d41/d1b/d75 0 2024-09-16T23:48:51.201 INFO:tasks.workunit.client.0.smithi042.stdout:4/141: creat de/d1a/f33 x:0 0 0 2024-09-16T23:48:51.202 INFO:tasks.workunit.client.0.smithi042.stdout:8/230: write da/d19/f1f [679576,53352] 0 2024-09-16T23:48:51.203 INFO:tasks.workunit.client.1.smithi043.stdout:6/362: dwrite fd [0,4194304] 0 2024-09-16T23:48:51.203 INFO:tasks.workunit.client.1.smithi043.stdout:4/336: mknod d8/d15/d20/d2b/d22/c61 0 2024-09-16T23:48:51.204 INFO:tasks.workunit.client.1.smithi043.stdout:6/363: readlink de/l10 0 2024-09-16T23:48:51.204 INFO:tasks.workunit.client.1.smithi043.stdout:8/346: read de/f45 [3780184,60376] 0 2024-09-16T23:48:51.205 INFO:tasks.workunit.client.1.smithi043.stdout:0/343: link d0/l11 d0/de/l6f 0 2024-09-16T23:48:51.208 INFO:tasks.workunit.client.0.smithi042.stdout:4/142: write de/f10 [626619,97856] 0 2024-09-16T23:48:51.211 INFO:tasks.workunit.client.0.smithi042.stdout:5/176: dwrite d9/db/de/d10/d2f/f36 [0,4194304] 0 2024-09-16T23:48:51.211 INFO:tasks.workunit.client.1.smithi043.stdout:9/316: link d4/d1c/d25/d2b/d4c/d50/l40 d4/d1c/d4f/d45/l5c 0 2024-09-16T23:48:51.214 INFO:tasks.workunit.client.0.smithi042.stdout:5/177: fdatasync d9/db/d19/d21/f2b 0 2024-09-16T23:48:51.215 INFO:tasks.workunit.client.0.smithi042.stdout:8/231: mknod da/d19/d2c/c4c 0 2024-09-16T23:48:51.215 INFO:tasks.workunit.client.0.smithi042.stdout:4/143: mknod de/d2e/c34 0 2024-09-16T23:48:51.218 INFO:tasks.workunit.client.0.smithi042.stdout:0/104: truncate d0/df/f19 1586589 0 2024-09-16T23:48:51.220 INFO:tasks.workunit.client.1.smithi043.stdout:1/446: unlink d4/d9/f4e 0 2024-09-16T23:48:51.220 INFO:tasks.workunit.client.0.smithi042.stdout:3/129: dwrite d2/d7/f8 [0,4194304] 0 2024-09-16T23:48:51.221 INFO:tasks.workunit.client.0.smithi042.stdout:4/144: creat de/d12/d1b/f35 x:0 0 0 2024-09-16T23:48:51.222 INFO:tasks.workunit.client.0.smithi042.stdout:5/178: dread d9/db/d11/f1c [0,4194304] 0 2024-09-16T23:48:51.223 INFO:tasks.workunit.client.1.smithi043.stdout:8/347: dwrite de/d18/d27/d34/d3d/f62 [0,4194304] 0 2024-09-16T23:48:51.224 INFO:tasks.workunit.client.0.smithi042.stdout:1/202: rename d0/d9/f41 to d0/d3b/f44 0 2024-09-16T23:48:51.224 INFO:tasks.workunit.client.0.smithi042.stdout:1/203: chown d0 431988865 1 2024-09-16T23:48:51.225 INFO:tasks.workunit.client.1.smithi043.stdout:0/344: dwrite d0/d2/dd/f60 [0,4194304] 0 2024-09-16T23:48:51.227 INFO:tasks.workunit.client.0.smithi042.stdout:8/232: write da/f47 [623917,11828] 0 2024-09-16T23:48:51.228 INFO:tasks.workunit.client.0.smithi042.stdout:0/105: truncate d0/f9 4413957 0 2024-09-16T23:48:51.228 INFO:tasks.workunit.client.0.smithi042.stdout:3/130: mkdir d2/d9/dd/d2c 0 2024-09-16T23:48:51.229 INFO:tasks.workunit.client.1.smithi043.stdout:2/271: mknod d8/d4b/d4f/c65 0 2024-09-16T23:48:51.230 INFO:tasks.workunit.client.0.smithi042.stdout:4/145: creat de/d12/d1b/f36 x:0 0 0 2024-09-16T23:48:51.231 INFO:tasks.workunit.client.1.smithi043.stdout:5/481: dwrite d3/f38 [0,4194304] 0 2024-09-16T23:48:51.231 INFO:tasks.workunit.client.0.smithi042.stdout:8/233: readlink da/lb 0 2024-09-16T23:48:51.234 INFO:tasks.workunit.client.1.smithi043.stdout:9/317: dread d4/d1c/d4f/f52 [0,4194304] 0 2024-09-16T23:48:51.234 INFO:tasks.workunit.client.0.smithi042.stdout:1/204: symlink d0/l45 0 2024-09-16T23:48:51.235 INFO:tasks.workunit.client.0.smithi042.stdout:0/106: creat d0/f1c x:0 0 0 2024-09-16T23:48:51.235 INFO:tasks.workunit.client.0.smithi042.stdout:3/131: symlink d2/d9/l2d 0 2024-09-16T23:48:51.236 INFO:tasks.workunit.client.1.smithi043.stdout:2/272: stat d8/d4b/d4f 0 2024-09-16T23:48:51.236 INFO:tasks.workunit.client.1.smithi043.stdout:7/343: unlink d4/c2c 0 2024-09-16T23:48:51.236 INFO:tasks.workunit.client.1.smithi043.stdout:5/482: write d3/db/d15/f3c [980292,70537] 0 2024-09-16T23:48:51.237 INFO:tasks.workunit.client.0.smithi042.stdout:4/146: write de/d1a/d1c/f1f [1051316,114366] 0 2024-09-16T23:48:51.238 INFO:tasks.workunit.client.1.smithi043.stdout:7/344: readlink d4/d27/d41/l1d 0 2024-09-16T23:48:51.238 INFO:tasks.workunit.client.1.smithi043.stdout:6/364: fdatasync de/d1a/f41 0 2024-09-16T23:48:51.238 INFO:tasks.workunit.client.1.smithi043.stdout:4/337: unlink d8/d15/d20/d2b/d22/c61 0 2024-09-16T23:48:51.239 INFO:tasks.workunit.client.0.smithi042.stdout:8/234: chown l3 370848708 1 2024-09-16T23:48:51.239 INFO:tasks.workunit.client.1.smithi043.stdout:5/483: stat d3/d9/d22/d8e/c92 0 2024-09-16T23:48:51.240 INFO:tasks.workunit.client.0.smithi042.stdout:0/107: dread d0/f7 [0,4194304] 0 2024-09-16T23:48:51.243 INFO:tasks.workunit.client.0.smithi042.stdout:4/147: mkdir de/d2e/d37 0 2024-09-16T23:48:51.243 INFO:tasks.workunit.client.1.smithi043.stdout:5/484: stat d3/db/d15/d4d/d6b/d73 0 2024-09-16T23:48:51.244 INFO:tasks.workunit.client.0.smithi042.stdout:1/205: rename d0/d9/df to d0/d9/d35/d46 0 2024-09-16T23:48:51.244 INFO:tasks.workunit.client.0.smithi042.stdout:1/206: chown d0/d17/l1b 1345 1 2024-09-16T23:48:51.245 INFO:tasks.workunit.client.0.smithi042.stdout:1/207: chown d0/d17/l19 12704799 1 2024-09-16T23:48:51.245 INFO:tasks.workunit.client.1.smithi043.stdout:5/485: stat d3/d9/d22/d4a/d4c/f72 0 2024-09-16T23:48:51.246 INFO:tasks.workunit.client.0.smithi042.stdout:5/179: getdents d9/db/d17 0 2024-09-16T23:48:51.250 INFO:tasks.workunit.client.0.smithi042.stdout:5/180: fsync d9/db/de/d10/d2f/f34 0 2024-09-16T23:48:51.250 INFO:tasks.workunit.client.0.smithi042.stdout:4/148: mkdir de/d38 0 2024-09-16T23:48:51.251 INFO:tasks.workunit.client.1.smithi043.stdout:0/345: creat d0/d2/df/d5a/f70 x:0 0 0 2024-09-16T23:48:51.252 INFO:tasks.workunit.client.0.smithi042.stdout:4/149: creat de/d1a/f39 x:0 0 0 2024-09-16T23:48:51.253 INFO:tasks.workunit.client.0.smithi042.stdout:3/132: getdents d2/d1b/d1d 0 2024-09-16T23:48:51.254 INFO:tasks.workunit.client.0.smithi042.stdout:5/181: symlink d9/db/d17/l37 0 2024-09-16T23:48:51.257 INFO:tasks.workunit.client.0.smithi042.stdout:3/133: creat d2/d9/dd/d17/d21/f2e x:0 0 0 2024-09-16T23:48:51.257 INFO:tasks.workunit.client.0.smithi042.stdout:0/108: dread d0/df/f10 [0,4194304] 0 2024-09-16T23:48:51.258 INFO:tasks.workunit.client.0.smithi042.stdout:0/109: readlink d0/l6 0 2024-09-16T23:48:51.261 INFO:tasks.workunit.client.1.smithi043.stdout:0/346: dwrite d0/d2/df/d1a/d46/f5e [0,4194304] 0 2024-09-16T23:48:51.261 INFO:tasks.workunit.client.1.smithi043.stdout:1/447: fsync d4/d9/d1c/d1e/d42/d5d/f93 0 2024-09-16T23:48:51.262 INFO:tasks.workunit.client.0.smithi042.stdout:1/208: link d0/d9/d35/d46/d32/d2f/l31 d0/d9/d35/d46/d32/l47 0 2024-09-16T23:48:51.263 INFO:tasks.workunit.client.1.smithi043.stdout:0/347: fdatasync d0/d2/df/f17 0 2024-09-16T23:48:51.264 INFO:tasks.workunit.client.0.smithi042.stdout:5/182: symlink d9/db/d1b/l38 0 2024-09-16T23:48:51.265 INFO:tasks.workunit.client.1.smithi043.stdout:0/348: write d0/d2/df/f17 [794534,3576] 0 2024-09-16T23:48:51.268 INFO:tasks.workunit.client.1.smithi043.stdout:9/318: creat d4/d21/d35/f5d x:0 0 0 2024-09-16T23:48:51.269 INFO:tasks.workunit.client.0.smithi042.stdout:1/209: mknod d0/d17/d18/c48 0 2024-09-16T23:48:51.271 INFO:tasks.workunit.client.1.smithi043.stdout:2/273: creat d8/d1d/d20/d23/d38/f66 x:0 0 0 2024-09-16T23:48:51.272 INFO:tasks.workunit.client.0.smithi042.stdout:8/235: getdents da 0 2024-09-16T23:48:51.272 INFO:tasks.workunit.client.0.smithi042.stdout:0/110: mknod d0/df/d13/c1d 0 2024-09-16T23:48:51.273 INFO:tasks.workunit.client.1.smithi043.stdout:0/349: dread d0/d2/df/d1a/d46/d39/f6a [0,4194304] 0 2024-09-16T23:48:51.274 INFO:tasks.workunit.client.0.smithi042.stdout:5/183: link d9/db/f28 d9/db/d19/f39 0 2024-09-16T23:48:51.275 INFO:tasks.workunit.client.1.smithi043.stdout:7/345: creat d4/db/d11/d15/d29/f76 x:0 0 0 2024-09-16T23:48:51.276 INFO:tasks.workunit.client.0.smithi042.stdout:4/150: rmdir de/d38 0 2024-09-16T23:48:51.277 INFO:tasks.workunit.client.0.smithi042.stdout:3/134: rmdir d2/d9/dd/d2b 0 2024-09-16T23:48:51.277 INFO:tasks.workunit.client.0.smithi042.stdout:0/111: dread d0/df/f14 [0,4194304] 0 2024-09-16T23:48:51.278 INFO:tasks.workunit.client.0.smithi042.stdout:5/184: mknod d9/db/de/c3a 0 2024-09-16T23:48:51.279 INFO:tasks.workunit.client.0.smithi042.stdout:4/151: symlink de/d12/l3a 0 2024-09-16T23:48:51.281 INFO:tasks.workunit.client.0.smithi042.stdout:5/185: symlink d9/db/de/d10/l3b 0 2024-09-16T23:48:51.281 INFO:tasks.workunit.client.0.smithi042.stdout:3/135: write d2/d9/dd/d17/f19 [587603,68617] 0 2024-09-16T23:48:51.282 INFO:tasks.workunit.client.0.smithi042.stdout:0/112: truncate d0/f7 1611484 0 2024-09-16T23:48:51.283 INFO:tasks.workunit.client.0.smithi042.stdout:3/136: chown d2/d7/f8 51826 1 2024-09-16T23:48:51.283 INFO:tasks.workunit.client.0.smithi042.stdout:1/210: rename d0/d9/d35/d46/d32/d2f/l31 to d0/d9/d35/d43/l49 0 2024-09-16T23:48:51.286 INFO:tasks.workunit.client.0.smithi042.stdout:5/186: creat d9/f3c x:0 0 0 2024-09-16T23:48:51.286 INFO:tasks.workunit.client.0.smithi042.stdout:2/153: sync 2024-09-16T23:48:51.286 INFO:tasks.workunit.client.0.smithi042.stdout:7/152: sync 2024-09-16T23:48:51.286 INFO:tasks.workunit.client.1.smithi043.stdout:3/353: link d4/de/d35/c18 d4/de/d5d/c66 0 2024-09-16T23:48:51.287 INFO:tasks.workunit.client.0.smithi042.stdout:3/137: fsync d2/d1b/d1d/f15 0 2024-09-16T23:48:51.289 INFO:tasks.workunit.client.0.smithi042.stdout:1/211: dread - d0/d9/d35/d46/f30 zero size 2024-09-16T23:48:51.290 INFO:tasks.workunit.client.1.smithi043.stdout:1/448: rmdir d4/d9/d1c/d1e/d42/d5d/d71 39 2024-09-16T23:48:51.290 INFO:tasks.workunit.client.0.smithi042.stdout:8/236: creat da/d19/f4d x:0 0 0 2024-09-16T23:48:51.291 INFO:tasks.workunit.client.0.smithi042.stdout:4/152: dwrite de/f10 [4194304,4194304] 0 2024-09-16T23:48:51.292 INFO:tasks.workunit.client.0.smithi042.stdout:5/187: truncate d9/db/d11/f32 1484137 0 2024-09-16T23:48:51.292 INFO:tasks.workunit.client.0.smithi042.stdout:4/153: dread - de/d1a/f39 zero size 2024-09-16T23:48:51.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/153: fdatasync d3/d15/d16/f1c 0 2024-09-16T23:48:51.293 INFO:tasks.workunit.client.0.smithi042.stdout:5/188: fsync d9/db/f25 0 2024-09-16T23:48:51.293 INFO:tasks.workunit.client.0.smithi042.stdout:2/154: creat d0/da/d17/f37 x:0 0 0 2024-09-16T23:48:51.294 INFO:tasks.workunit.client.1.smithi043.stdout:9/319: mknod d4/d21/d2a/c5e 0 2024-09-16T23:48:51.295 INFO:tasks.workunit.client.0.smithi042.stdout:0/113: truncate d0/df/f14 366876 0 2024-09-16T23:48:51.297 INFO:tasks.workunit.client.1.smithi043.stdout:7/346: creat d4/db/d11/d15/d29/f77 x:0 0 0 2024-09-16T23:48:51.298 INFO:tasks.workunit.client.0.smithi042.stdout:3/138: truncate f0 3214845 0 2024-09-16T23:48:51.306 INFO:tasks.workunit.client.1.smithi043.stdout:2/274: mknod d8/d4b/d5f/d63/c67 0 2024-09-16T23:48:51.306 INFO:tasks.workunit.client.1.smithi043.stdout:6/365: rename de/d1a/d39/f16 to de/f7a 0 2024-09-16T23:48:51.307 INFO:tasks.workunit.client.1.smithi043.stdout:4/338: link d8/l53 d8/d15/d20/d2b/d54/l62 0 2024-09-16T23:48:51.309 INFO:tasks.workunit.client.1.smithi043.stdout:2/275: write d8/d11/d16/f28 [4541942,51375] 0 2024-09-16T23:48:51.312 INFO:tasks.workunit.client.1.smithi043.stdout:3/354: dwrite d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:48:51.312 INFO:tasks.workunit.client.1.smithi043.stdout:5/486: getdents d3/d9/d22/d4a 0 2024-09-16T23:48:51.314 INFO:tasks.workunit.client.1.smithi043.stdout:5/487: write d3/db/d19/d3e/f8b [725023,56904] 0 2024-09-16T23:48:51.316 INFO:tasks.workunit.client.1.smithi043.stdout:7/347: rmdir d4/db/d11/d21/d68 0 2024-09-16T23:48:51.319 INFO:tasks.workunit.client.1.smithi043.stdout:3/355: dwrite d4/f23 [0,4194304] 0 2024-09-16T23:48:51.320 INFO:tasks.workunit.client.1.smithi043.stdout:4/339: symlink d8/d25/d39/l63 0 2024-09-16T23:48:51.322 INFO:tasks.workunit.client.0.smithi042.stdout:0/114: rmdir d0/df 39 2024-09-16T23:48:51.325 INFO:tasks.workunit.client.0.smithi042.stdout:7/154: mkdir d3/d15/d32 0 2024-09-16T23:48:51.329 INFO:tasks.workunit.client.0.smithi042.stdout:4/154: dwrite de/d12/f14 [0,4194304] 0 2024-09-16T23:48:51.329 INFO:tasks.workunit.client.0.smithi042.stdout:2/155: link d0/da/fd d0/d30/f38 0 2024-09-16T23:48:51.330 INFO:tasks.workunit.client.1.smithi043.stdout:1/449: dwrite d4/d9/d12/d2b/f52 [4194304,4194304] 0 2024-09-16T23:48:51.330 INFO:tasks.workunit.client.0.smithi042.stdout:7/155: readlink d3/d15/d16/l2f 0 2024-09-16T23:48:51.332 INFO:tasks.workunit.client.0.smithi042.stdout:1/212: creat d0/f4a x:0 0 0 2024-09-16T23:48:51.336 INFO:tasks.workunit.client.0.smithi042.stdout:5/189: dwrite d9/db/d19/f39 [0,4194304] 0 2024-09-16T23:48:51.337 INFO:tasks.workunit.client.1.smithi043.stdout:5/488: creat d3/d1b/f97 x:0 0 0 2024-09-16T23:48:51.337 INFO:tasks.workunit.client.0.smithi042.stdout:0/115: dread d0/df/f19 [0,4194304] 0 2024-09-16T23:48:51.338 INFO:tasks.workunit.client.1.smithi043.stdout:2/276: unlink d8/ff 0 2024-09-16T23:48:51.339 INFO:tasks.workunit.client.0.smithi042.stdout:7/156: rmdir d3/d15/d16/d19 39 2024-09-16T23:48:51.339 INFO:tasks.workunit.client.1.smithi043.stdout:5/489: fsync d3/db/d19/d3e/f8b 0 2024-09-16T23:48:51.340 INFO:tasks.workunit.client.0.smithi042.stdout:7/157: chown d3/d15/d16/c2c 198067 1 2024-09-16T23:48:51.341 INFO:tasks.workunit.client.0.smithi042.stdout:4/155: creat de/d2e/d37/f3b x:0 0 0 2024-09-16T23:48:51.343 INFO:tasks.workunit.client.0.smithi042.stdout:1/213: dread d0/d9/d35/d46/f12 [0,4194304] 0 2024-09-16T23:48:51.344 INFO:tasks.workunit.client.1.smithi043.stdout:8/348: rename de/d18/d1f/l48 to de/d18/d1f/d2f/l68 0 2024-09-16T23:48:51.345 INFO:tasks.workunit.client.1.smithi043.stdout:7/348: dread d4/f62 [0,4194304] 0 2024-09-16T23:48:51.345 INFO:tasks.workunit.client.0.smithi042.stdout:1/214: readlink d0/d17/l1b 0 2024-09-16T23:48:51.346 INFO:tasks.workunit.client.0.smithi042.stdout:0/116: symlink d0/l1e 0 2024-09-16T23:48:51.348 INFO:tasks.workunit.client.0.smithi042.stdout:0/117: write d0/f4 [1415487,18467] 0 2024-09-16T23:48:51.348 INFO:tasks.workunit.client.0.smithi042.stdout:2/156: rename d0/c18 to d0/da/c39 0 2024-09-16T23:48:51.349 INFO:tasks.workunit.client.1.smithi043.stdout:6/366: truncate de/f13 943755 0 2024-09-16T23:48:51.349 INFO:tasks.workunit.client.0.smithi042.stdout:4/156: mkdir de/d2e/d37/d3c 0 2024-09-16T23:48:51.351 INFO:tasks.workunit.client.0.smithi042.stdout:1/215: mknod d0/d17/c4b 0 2024-09-16T23:48:51.352 INFO:tasks.workunit.client.1.smithi043.stdout:6/367: read f2 [1918483,15321] 0 2024-09-16T23:48:51.352 INFO:tasks.workunit.client.1.smithi043.stdout:1/450: fdatasync d4/d34/f3a 0 2024-09-16T23:48:51.353 INFO:tasks.workunit.client.0.smithi042.stdout:5/190: unlink d9/db/d19/d21/f26 0 2024-09-16T23:48:51.353 INFO:tasks.workunit.client.1.smithi043.stdout:4/340: unlink d8/l53 0 2024-09-16T23:48:51.354 INFO:tasks.workunit.client.1.smithi043.stdout:6/368: write de/d23/d63/f6f [3809939,20293] 0 2024-09-16T23:48:51.356 INFO:tasks.workunit.client.1.smithi043.stdout:5/490: creat d3/d9/d6f/d85/f98 x:0 0 0 2024-09-16T23:48:51.358 INFO:tasks.workunit.client.1.smithi043.stdout:5/491: truncate d3/db/d26/f33 4852815 0 2024-09-16T23:48:51.358 INFO:tasks.workunit.client.1.smithi043.stdout:5/492: stat d3/d79/d91 0 2024-09-16T23:48:51.362 INFO:tasks.workunit.client.0.smithi042.stdout:5/191: write d9/db/f28 [2664729,103907] 0 2024-09-16T23:48:51.364 INFO:tasks.workunit.client.1.smithi043.stdout:7/349: dread d4/db/d28/f2d [0,4194304] 0 2024-09-16T23:48:51.366 INFO:tasks.workunit.client.0.smithi042.stdout:0/118: dwrite d0/f1c [0,4194304] 0 2024-09-16T23:48:51.367 INFO:tasks.workunit.client.0.smithi042.stdout:7/158: dwrite d3/d15/d16/f1c [0,4194304] 0 2024-09-16T23:48:51.368 INFO:tasks.workunit.client.0.smithi042.stdout:0/119: readlink d0/ld 0 2024-09-16T23:48:51.368 INFO:tasks.workunit.client.1.smithi043.stdout:4/341: mkdir d8/d25/d64 0 2024-09-16T23:48:51.369 INFO:tasks.workunit.client.0.smithi042.stdout:5/192: stat d9/db/d19/d21/f2b 0 2024-09-16T23:48:51.370 INFO:tasks.workunit.client.0.smithi042.stdout:4/157: dwrite de/d1a/f33 [0,4194304] 0 2024-09-16T23:48:51.372 INFO:tasks.workunit.client.1.smithi043.stdout:4/342: read d8/d15/d20/d2b/d54/d55/f2a [799336,126183] 0 2024-09-16T23:48:51.372 INFO:tasks.workunit.client.1.smithi043.stdout:3/356: link d4/f54 d4/f67 0 2024-09-16T23:48:51.378 INFO:tasks.workunit.client.0.smithi042.stdout:5/193: write d9/db/d19/f39 [5098560,82670] 0 2024-09-16T23:48:51.381 INFO:tasks.workunit.client.1.smithi043.stdout:6/369: rmdir de/d2a 39 2024-09-16T23:48:51.382 INFO:tasks.workunit.client.1.smithi043.stdout:6/370: write de/d1a/d39/f78 [1006092,65072] 0 2024-09-16T23:48:51.383 INFO:tasks.workunit.client.1.smithi043.stdout:2/277: mknod d8/d4b/d4f/d56/c68 0 2024-09-16T23:48:51.384 INFO:tasks.workunit.client.1.smithi043.stdout:6/371: write de/d1a/f3b [601124,118891] 0 2024-09-16T23:48:51.386 INFO:tasks.workunit.client.0.smithi042.stdout:7/159: dread d3/fc [0,4194304] 0 2024-09-16T23:48:51.387 INFO:tasks.workunit.client.1.smithi043.stdout:5/493: unlink d3/l83 0 2024-09-16T23:48:51.392 INFO:tasks.workunit.client.1.smithi043.stdout:7/350: creat d4/db/d11/d21/d36/f78 x:0 0 0 2024-09-16T23:48:51.392 INFO:tasks.workunit.client.1.smithi043.stdout:6/372: dread de/d1a/f3b [0,4194304] 0 2024-09-16T23:48:51.393 INFO:tasks.workunit.client.1.smithi043.stdout:7/351: fsync d4/d27/d41/f40 0 2024-09-16T23:48:51.393 INFO:tasks.workunit.client.1.smithi043.stdout:7/352: stat d4/db/c10 0 2024-09-16T23:48:51.394 INFO:tasks.workunit.client.1.smithi043.stdout:3/357: symlink d4/de/d40/l68 0 2024-09-16T23:48:51.395 INFO:tasks.workunit.client.0.smithi042.stdout:1/216: getdents d0/d9/d35/d46/d32/d2f 0 2024-09-16T23:48:51.396 INFO:tasks.workunit.client.0.smithi042.stdout:6/132: sync 2024-09-16T23:48:51.396 INFO:tasks.workunit.client.0.smithi042.stdout:9/116: sync 2024-09-16T23:48:51.396 INFO:tasks.workunit.client.1.smithi043.stdout:3/358: chown d4/de/d1f/d34/c3f 62195 1 2024-09-16T23:48:51.397 INFO:tasks.workunit.client.1.smithi043.stdout:4/343: dread d8/d25/f2c [0,4194304] 0 2024-09-16T23:48:51.397 INFO:tasks.workunit.client.0.smithi042.stdout:3/139: dwrite f0 [0,4194304] 0 2024-09-16T23:48:51.397 INFO:tasks.workunit.client.1.smithi043.stdout:4/344: fsync d8/d3d/f5f 0 2024-09-16T23:48:51.400 INFO:tasks.workunit.client.0.smithi042.stdout:4/158: dwrite de/d12/f25 [4194304,4194304] 0 2024-09-16T23:48:51.403 INFO:tasks.workunit.client.0.smithi042.stdout:2/157: write d0/da/f32 [197475,20030] 0 2024-09-16T23:48:51.408 INFO:tasks.workunit.client.0.smithi042.stdout:7/160: mknod d3/d15/d24/c33 0 2024-09-16T23:48:51.408 INFO:tasks.workunit.client.0.smithi042.stdout:6/133: fdatasync d8/db/dd/f11 0 2024-09-16T23:48:51.410 INFO:tasks.workunit.client.1.smithi043.stdout:1/451: creat d4/f94 x:0 0 0 2024-09-16T23:48:51.411 INFO:tasks.workunit.client.0.smithi042.stdout:5/194: dwrite d9/db/d19/d21/f2e [0,4194304] 0 2024-09-16T23:48:51.411 INFO:tasks.workunit.client.0.smithi042.stdout:0/120: dwrite d0/df/f14 [0,4194304] 0 2024-09-16T23:48:51.412 INFO:tasks.workunit.client.1.smithi043.stdout:6/373: write de/d2a/f33 [1495697,110296] 0 2024-09-16T23:48:51.413 INFO:tasks.workunit.client.1.smithi043.stdout:6/374: fdatasync de/d23/d49/f6a 0 2024-09-16T23:48:51.416 INFO:tasks.workunit.client.0.smithi042.stdout:0/121: readlink d0/l1e 0 2024-09-16T23:48:51.417 INFO:tasks.workunit.client.1.smithi043.stdout:1/452: dwrite d4/d9/d12/d2b/f6b [0,4194304] 0 2024-09-16T23:48:51.419 INFO:tasks.workunit.client.0.smithi042.stdout:4/159: symlink de/d1a/d1c/l3d 0 2024-09-16T23:48:51.419 INFO:tasks.workunit.client.0.smithi042.stdout:3/140: mknod d2/d9/dd/d17/d21/c2f 0 2024-09-16T23:48:51.420 INFO:tasks.workunit.client.0.smithi042.stdout:4/160: dread - fd zero size 2024-09-16T23:48:51.422 INFO:tasks.workunit.client.0.smithi042.stdout:6/134: mknod d8/db/dd/d1b/c2b 0 2024-09-16T23:48:51.423 INFO:tasks.workunit.client.0.smithi042.stdout:1/217: dwrite d0/d9/d35/d46/f30 [0,4194304] 0 2024-09-16T23:48:51.423 INFO:tasks.workunit.client.1.smithi043.stdout:7/353: creat d4/db/d43/f79 x:0 0 0 2024-09-16T23:48:51.424 INFO:tasks.workunit.client.1.smithi043.stdout:7/354: chown d4/db/d11/d15/d29/f76 806384 1 2024-09-16T23:48:51.425 INFO:tasks.workunit.client.0.smithi042.stdout:9/117: write d4/f5 [393787,52941] 0 2024-09-16T23:48:51.427 INFO:tasks.workunit.client.0.smithi042.stdout:1/218: write d0/d9/d35/d46/f30 [3179398,91665] 0 2024-09-16T23:48:51.427 INFO:tasks.workunit.client.0.smithi042.stdout:3/141: mkdir d2/d1b/d1d/d30 0 2024-09-16T23:48:51.428 INFO:tasks.workunit.client.0.smithi042.stdout:4/161: mkdir de/d2e/d37/d3e 0 2024-09-16T23:48:51.430 INFO:tasks.workunit.client.0.smithi042.stdout:5/195: symlink d9/db/d1b/d31/l3d 0 2024-09-16T23:48:51.430 INFO:tasks.workunit.client.0.smithi042.stdout:0/122: link d0/f1b d0/d15/f1f 0 2024-09-16T23:48:51.431 INFO:tasks.workunit.client.1.smithi043.stdout:8/349: getdents de/d18/d27/d41/d5a/d37 0 2024-09-16T23:48:51.433 INFO:tasks.workunit.client.0.smithi042.stdout:7/161: rename d3/d1f/c22 to d3/d15/c34 0 2024-09-16T23:48:51.434 INFO:tasks.workunit.client.0.smithi042.stdout:1/219: symlink d0/d17/l4c 0 2024-09-16T23:48:51.434 INFO:tasks.workunit.client.0.smithi042.stdout:3/142: dread d2/d9/dd/f1e [0,4194304] 0 2024-09-16T23:48:51.435 INFO:tasks.workunit.client.1.smithi043.stdout:7/355: dwrite d4/db/d11/d21/d31/d33/f55 [0,4194304] 0 2024-09-16T23:48:51.437 INFO:tasks.workunit.client.0.smithi042.stdout:9/118: symlink d4/d10/d24/d25/l26 0 2024-09-16T23:48:51.438 INFO:tasks.workunit.client.0.smithi042.stdout:4/162: write de/d1a/d1c/f1e [2964824,128882] 0 2024-09-16T23:48:51.439 INFO:tasks.workunit.client.1.smithi043.stdout:7/356: chown d4/db/d11/d21/d31/d65 719004641 1 2024-09-16T23:48:51.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/158: creat d0/f3a x:0 0 0 2024-09-16T23:48:51.441 INFO:tasks.workunit.client.0.smithi042.stdout:7/162: dread d3/fa [0,4194304] 0 2024-09-16T23:48:51.443 INFO:tasks.workunit.client.0.smithi042.stdout:5/196: rmdir d9/db/d19 39 2024-09-16T23:48:51.444 INFO:tasks.workunit.client.1.smithi043.stdout:3/359: dwrite d4/f67 [0,4194304] 0 2024-09-16T23:48:51.446 INFO:tasks.workunit.client.0.smithi042.stdout:4/163: dread de/d1a/f33 [0,4194304] 0 2024-09-16T23:48:51.447 INFO:tasks.workunit.client.1.smithi043.stdout:3/360: chown d4/f54 1594 1 2024-09-16T23:48:51.449 INFO:tasks.workunit.client.0.smithi042.stdout:6/135: rename d8/d9 to d8/db/dd/d2c 0 2024-09-16T23:48:51.449 INFO:tasks.workunit.client.0.smithi042.stdout:3/143: dread d2/d9/dd/d17/f19 [0,4194304] 0 2024-09-16T23:48:51.453 INFO:tasks.workunit.client.0.smithi042.stdout:2/159: fsync d0/da/d17/f2d 0 2024-09-16T23:48:51.453 INFO:tasks.workunit.client.1.smithi043.stdout:5/494: link d3/db/d26/l57 d3/d9/d6f/d85/l99 0 2024-09-16T23:48:51.453 INFO:tasks.workunit.client.0.smithi042.stdout:8/237: sync 2024-09-16T23:48:51.454 INFO:tasks.workunit.client.1.smithi043.stdout:3/361: dwrite d4/f23 [0,4194304] 0 2024-09-16T23:48:51.456 INFO:tasks.workunit.client.0.smithi042.stdout:8/238: stat da/d19/d1d/f2e 0 2024-09-16T23:48:51.456 INFO:tasks.workunit.client.1.smithi043.stdout:7/357: mkdir d4/db/d11/d15/d29/d7a 0 2024-09-16T23:48:51.458 INFO:tasks.workunit.client.0.smithi042.stdout:5/197: write d9/db/d19/d21/f2b [5221153,42813] 0 2024-09-16T23:48:51.459 INFO:tasks.workunit.client.0.smithi042.stdout:5/198: fdatasync d9/db/de/d10/f22 0 2024-09-16T23:48:51.464 INFO:tasks.workunit.client.0.smithi042.stdout:2/160: mkdir d0/da/d1b/d3b 0 2024-09-16T23:48:51.464 INFO:tasks.workunit.client.0.smithi042.stdout:1/220: getdents d0/d3b 0 2024-09-16T23:48:51.465 INFO:tasks.workunit.client.1.smithi043.stdout:7/358: dwrite d4/db/d11/d21/d31/d33/f5b [0,4194304] 0 2024-09-16T23:48:51.466 INFO:tasks.workunit.client.0.smithi042.stdout:7/163: symlink d3/d15/d16/l35 0 2024-09-16T23:48:51.470 INFO:tasks.workunit.client.1.smithi043.stdout:6/375: creat de/f7b x:0 0 0 2024-09-16T23:48:51.471 INFO:tasks.workunit.client.0.smithi042.stdout:9/119: write d4/d10/f1b [812618,76480] 0 2024-09-16T23:48:51.471 INFO:tasks.workunit.client.0.smithi042.stdout:4/164: creat de/d2e/d37/d3c/f3f x:0 0 0 2024-09-16T23:48:51.472 INFO:tasks.workunit.client.0.smithi042.stdout:9/120: stat d4/d10/d24/d25 0 2024-09-16T23:48:51.473 INFO:tasks.workunit.client.1.smithi043.stdout:5/495: mknod d3/db/d19/d76/c9a 0 2024-09-16T23:48:51.473 INFO:tasks.workunit.client.0.smithi042.stdout:9/121: chown d4/d12/d16/f17 749 1 2024-09-16T23:48:51.474 INFO:tasks.workunit.client.0.smithi042.stdout:4/165: truncate de/d1a/d1c/f2c 861322 0 2024-09-16T23:48:51.474 INFO:tasks.workunit.client.0.smithi042.stdout:7/164: fdatasync d3/f8 0 2024-09-16T23:48:51.475 INFO:tasks.workunit.client.1.smithi043.stdout:2/278: dwrite d8/d1d/d20/d23/d39/d3a/f57 [0,4194304] 0 2024-09-16T23:48:51.476 INFO:tasks.workunit.client.0.smithi042.stdout:1/221: creat d0/d17/d18/f4d x:0 0 0 2024-09-16T23:48:51.477 INFO:tasks.workunit.client.1.smithi043.stdout:4/345: truncate d8/fc 4580096 0 2024-09-16T23:48:51.478 INFO:tasks.workunit.client.1.smithi043.stdout:3/362: creat d4/de/d40/f69 x:0 0 0 2024-09-16T23:48:51.481 INFO:tasks.workunit.client.0.smithi042.stdout:5/199: creat d9/db/f3e x:0 0 0 2024-09-16T23:48:51.484 INFO:tasks.workunit.client.0.smithi042.stdout:2/161: unlink d0/f20 0 2024-09-16T23:48:51.485 INFO:tasks.workunit.client.1.smithi043.stdout:1/453: dwrite d4/d34/f41 [0,4194304] 0 2024-09-16T23:48:51.487 INFO:tasks.workunit.client.0.smithi042.stdout:7/165: mknod d3/d15/c36 0 2024-09-16T23:48:51.487 INFO:tasks.workunit.client.0.smithi042.stdout:5/200: fdatasync d9/db/d19/f2d 0 2024-09-16T23:48:51.487 INFO:tasks.workunit.client.1.smithi043.stdout:7/359: mknod d4/db/d11/d15/c7b 0 2024-09-16T23:48:51.488 INFO:tasks.workunit.client.0.smithi042.stdout:0/123: dwrite d0/f9 [4194304,4194304] 0 2024-09-16T23:48:51.488 INFO:tasks.workunit.client.0.smithi042.stdout:4/166: creat de/d2e/d37/d3e/f40 x:0 0 0 2024-09-16T23:48:51.489 INFO:tasks.workunit.client.1.smithi043.stdout:7/360: dread - d4/db/d43/f79 zero size 2024-09-16T23:48:51.490 INFO:tasks.workunit.client.0.smithi042.stdout:9/122: dwrite d4/d10/f1b [4194304,4194304] 0 2024-09-16T23:48:51.491 INFO:tasks.workunit.client.1.smithi043.stdout:6/376: dread de/d23/d49/f4c [0,4194304] 0 2024-09-16T23:48:51.495 INFO:tasks.workunit.client.0.smithi042.stdout:4/167: write de/d1a/d1c/f21 [377177,34844] 0 2024-09-16T23:48:51.499 INFO:tasks.workunit.client.0.smithi042.stdout:2/162: write d0/ff [533348,58397] 0 2024-09-16T23:48:51.499 INFO:tasks.workunit.client.0.smithi042.stdout:1/222: dread d0/d9/d35/d46/f13 [0,4194304] 0 2024-09-16T23:48:51.501 INFO:tasks.workunit.client.0.smithi042.stdout:5/201: symlink d9/db/l3f 0 2024-09-16T23:48:51.501 INFO:tasks.workunit.client.0.smithi042.stdout:7/166: dread d3/f8 [0,4194304] 0 2024-09-16T23:48:51.502 INFO:tasks.workunit.client.0.smithi042.stdout:6/136: mkdir d8/db/dd/d1b/d2d 0 2024-09-16T23:48:51.502 INFO:tasks.workunit.client.1.smithi043.stdout:6/377: dwrite de/d1a/d39/f78 [0,4194304] 0 2024-09-16T23:48:51.505 INFO:tasks.workunit.client.0.smithi042.stdout:0/124: mknod d0/d15/c20 0 2024-09-16T23:48:51.506 INFO:tasks.workunit.client.1.smithi043.stdout:3/363: creat d4/d30/d53/f6a x:0 0 0 2024-09-16T23:48:51.506 INFO:tasks.workunit.client.0.smithi042.stdout:4/168: creat de/f41 x:0 0 0 2024-09-16T23:48:51.507 INFO:tasks.workunit.client.0.smithi042.stdout:5/202: dread d9/db/d19/d21/f2e [0,4194304] 0 2024-09-16T23:48:51.507 INFO:tasks.workunit.client.0.smithi042.stdout:0/125: dread - d0/df/f18 zero size 2024-09-16T23:48:51.509 INFO:tasks.workunit.client.0.smithi042.stdout:4/169: dread - de/d2e/d37/f3b zero size 2024-09-16T23:48:51.510 INFO:tasks.workunit.client.0.smithi042.stdout:9/123: mknod d4/c27 0 2024-09-16T23:48:51.510 INFO:tasks.workunit.client.1.smithi043.stdout:2/279: dwrite d8/d11/f32 [0,4194304] 0 2024-09-16T23:48:51.512 INFO:tasks.workunit.client.0.smithi042.stdout:0/126: creat d0/df/f21 x:0 0 0 2024-09-16T23:48:51.512 INFO:tasks.workunit.client.1.smithi043.stdout:2/280: fdatasync d8/d1d/d20/d23/d39/d3a/f57 0 2024-09-16T23:48:51.513 INFO:tasks.workunit.client.0.smithi042.stdout:1/223: mkdir d0/d9/d4e 0 2024-09-16T23:48:51.513 INFO:tasks.workunit.client.0.smithi042.stdout:0/127: stat d0/le 0 2024-09-16T23:48:51.515 INFO:tasks.workunit.client.1.smithi043.stdout:6/378: creat de/d23/d49/f7c x:0 0 0 2024-09-16T23:48:51.516 INFO:tasks.workunit.client.0.smithi042.stdout:0/128: chown d0/l6 3361 1 2024-09-16T23:48:51.516 INFO:tasks.workunit.client.0.smithi042.stdout:7/167: symlink d3/d15/d32/l37 0 2024-09-16T23:48:51.516 INFO:tasks.workunit.client.0.smithi042.stdout:5/203: creat d9/db/d11/f40 x:0 0 0 2024-09-16T23:48:51.516 INFO:tasks.workunit.client.0.smithi042.stdout:4/170: mkdir de/d42 0 2024-09-16T23:48:51.517 INFO:tasks.workunit.client.0.smithi042.stdout:2/163: mknod d0/c3c 0 2024-09-16T23:48:51.519 INFO:tasks.workunit.client.0.smithi042.stdout:0/129: chown d0/df/d13 988 1 2024-09-16T23:48:51.519 INFO:tasks.workunit.client.0.smithi042.stdout:4/171: chown de/c18 1 1 2024-09-16T23:48:51.521 INFO:tasks.workunit.client.0.smithi042.stdout:5/204: fsync d9/db/d11/f32 0 2024-09-16T23:48:51.524 INFO:tasks.workunit.client.0.smithi042.stdout:4/172: chown de/d12/d1b/l29 18 1 2024-09-16T23:48:51.524 INFO:tasks.workunit.client.0.smithi042.stdout:9/124: dwrite d4/d12/d16/f17 [0,4194304] 0 2024-09-16T23:48:51.525 INFO:tasks.workunit.client.0.smithi042.stdout:5/205: chown l5 1875176 1 2024-09-16T23:48:51.525 INFO:tasks.workunit.client.0.smithi042.stdout:1/224: creat d0/d17/f4f x:0 0 0 2024-09-16T23:48:51.527 INFO:tasks.workunit.client.1.smithi043.stdout:7/361: mkdir d4/d7c 0 2024-09-16T23:48:51.528 INFO:tasks.workunit.client.1.smithi043.stdout:3/364: dwrite d4/f19 [0,4194304] 0 2024-09-16T23:48:51.532 INFO:tasks.workunit.client.0.smithi042.stdout:0/130: creat d0/df/d13/f22 x:0 0 0 2024-09-16T23:48:51.533 INFO:tasks.workunit.client.1.smithi043.stdout:2/281: readlink d8/d11/d16/l53 0 2024-09-16T23:48:51.537 INFO:tasks.workunit.client.1.smithi043.stdout:5/496: creat d3/f9b x:0 0 0 2024-09-16T23:48:51.541 INFO:tasks.workunit.client.0.smithi042.stdout:9/125: stat d4/fb 0 2024-09-16T23:48:51.541 INFO:tasks.workunit.client.0.smithi042.stdout:5/206: dwrite d9/db/f28 [4194304,4194304] 0 2024-09-16T23:48:51.543 INFO:tasks.workunit.client.1.smithi043.stdout:2/282: dwrite d8/d1d/f52 [0,4194304] 0 2024-09-16T23:48:51.544 INFO:tasks.workunit.client.1.smithi043.stdout:1/454: readlink d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/l77 0 2024-09-16T23:48:51.549 INFO:tasks.workunit.client.0.smithi042.stdout:2/164: write d0/da/f1f [1065400,49183] 0 2024-09-16T23:48:51.551 INFO:tasks.workunit.client.0.smithi042.stdout:4/173: unlink de/d2e/d37/d3c/f3f 0 2024-09-16T23:48:51.552 INFO:tasks.workunit.client.1.smithi043.stdout:5/497: rmdir d3/db/d19/d3e/d28 39 2024-09-16T23:48:51.553 INFO:tasks.workunit.client.0.smithi042.stdout:1/225: dwrite d0/d9/d35/d46/d32/d2f/f34 [0,4194304] 0 2024-09-16T23:48:51.554 INFO:tasks.workunit.client.1.smithi043.stdout:1/455: dwrite d4/d9/d72/f73 [0,4194304] 0 2024-09-16T23:48:51.555 INFO:tasks.workunit.client.1.smithi043.stdout:5/498: truncate d3/db/d15/f69 606066 0 2024-09-16T23:48:51.556 INFO:tasks.workunit.client.1.smithi043.stdout:5/499: write d3/f17 [7694813,40460] 0 2024-09-16T23:48:51.558 INFO:tasks.workunit.client.0.smithi042.stdout:1/226: fdatasync d0/f3 0 2024-09-16T23:48:51.559 INFO:tasks.workunit.client.1.smithi043.stdout:5/500: read d3/db/d19/d3e/f45 [1988666,4015] 0 2024-09-16T23:48:51.561 INFO:tasks.workunit.client.1.smithi043.stdout:3/365: dread d4/de/d1f/d34/f3a [0,4194304] 0 2024-09-16T23:48:51.563 INFO:tasks.workunit.client.1.smithi043.stdout:6/379: creat de/d1a/d28/f7d x:0 0 0 2024-09-16T23:48:51.565 INFO:tasks.workunit.client.0.smithi042.stdout:2/165: truncate d0/f1a 1237693 0 2024-09-16T23:48:51.567 INFO:tasks.workunit.client.0.smithi042.stdout:5/207: dwrite d9/db/de/d10/d2f/f34 [0,4194304] 0 2024-09-16T23:48:51.568 INFO:tasks.workunit.client.0.smithi042.stdout:7/168: dwrite d3/d15/f1b [0,4194304] 0 2024-09-16T23:48:51.569 INFO:tasks.workunit.client.0.smithi042.stdout:0/131: rmdir d0/df/d13 39 2024-09-16T23:48:51.569 INFO:tasks.workunit.client.1.smithi043.stdout:1/456: mkdir d4/d3e/d95 0 2024-09-16T23:48:51.571 INFO:tasks.workunit.client.0.smithi042.stdout:7/169: stat d3/d15 0 2024-09-16T23:48:51.571 INFO:tasks.workunit.client.0.smithi042.stdout:7/170: chown d3/d15/d32 554099 1 2024-09-16T23:48:51.573 INFO:tasks.workunit.client.0.smithi042.stdout:0/132: truncate d0/df/f21 339529 0 2024-09-16T23:48:51.574 INFO:tasks.workunit.client.0.smithi042.stdout:6/137: dread f5 [0,4194304] 0 2024-09-16T23:48:51.574 INFO:tasks.workunit.client.1.smithi043.stdout:7/362: rmdir d4/d7c 0 2024-09-16T23:48:51.575 INFO:tasks.workunit.client.0.smithi042.stdout:4/174: rename de/d12/d1b/l26 to de/d1a/l43 0 2024-09-16T23:48:51.575 INFO:tasks.workunit.client.0.smithi042.stdout:9/126: write d4/fb [718923,51771] 0 2024-09-16T23:48:51.576 INFO:tasks.workunit.client.0.smithi042.stdout:5/208: mkdir d9/db/d19/d21/d41 0 2024-09-16T23:48:51.576 INFO:tasks.workunit.client.0.smithi042.stdout:6/138: write d8/db/f12 [2518438,96585] 0 2024-09-16T23:48:51.577 INFO:tasks.workunit.client.1.smithi043.stdout:5/501: symlink d3/d9/d22/d8e/l9c 0 2024-09-16T23:48:51.581 INFO:tasks.workunit.client.1.smithi043.stdout:6/380: truncate de/d1a/f3b 1084972 0 2024-09-16T23:48:51.582 INFO:tasks.workunit.client.0.smithi042.stdout:1/227: creat d0/d9/d35/d46/d32/d2f/f50 x:0 0 0 2024-09-16T23:48:51.582 INFO:tasks.workunit.client.0.smithi042.stdout:2/166: mknod d0/da/d1b/c3d 0 2024-09-16T23:48:51.586 INFO:tasks.workunit.client.1.smithi043.stdout:2/283: link d8/d35/f37 d8/d1d/d20/d23/d38/f69 0 2024-09-16T23:48:51.586 INFO:tasks.workunit.client.0.smithi042.stdout:0/133: mkdir d0/df/d23 0 2024-09-16T23:48:51.588 INFO:tasks.workunit.client.0.smithi042.stdout:9/127: mkdir d4/d10/d24/d25/d28 0 2024-09-16T23:48:51.590 INFO:tasks.workunit.client.1.smithi043.stdout:5/502: mkdir d3/d9/d22/d60/d9d 0 2024-09-16T23:48:51.594 INFO:tasks.workunit.client.0.smithi042.stdout:4/175: dwrite de/d12/f15 [0,4194304] 0 2024-09-16T23:48:51.595 INFO:tasks.workunit.client.1.smithi043.stdout:0/350: sync 2024-09-16T23:48:51.595 INFO:tasks.workunit.client.1.smithi043.stdout:3/366: symlink d4/l6b 0 2024-09-16T23:48:51.598 INFO:tasks.workunit.client.0.smithi042.stdout:2/167: mkdir d0/da/d17/d3e 0 2024-09-16T23:48:51.599 INFO:tasks.workunit.client.0.smithi042.stdout:7/171: dwrite d3/f11 [0,4194304] 0 2024-09-16T23:48:51.599 INFO:tasks.workunit.client.0.smithi042.stdout:4/176: chown de/d2e/d37/d3e 470829 1 2024-09-16T23:48:51.601 INFO:tasks.workunit.client.0.smithi042.stdout:6/139: link f6 d8/db/dd/d2c/d26/f2e 0 2024-09-16T23:48:51.603 INFO:tasks.workunit.client.0.smithi042.stdout:7/172: truncate d3/f30 1019207 0 2024-09-16T23:48:51.604 INFO:tasks.workunit.client.1.smithi043.stdout:1/457: creat d4/df/d26/d79/f96 x:0 0 0 2024-09-16T23:48:51.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/134: dwrite d0/df/f14 [0,4194304] 0 2024-09-16T23:48:51.609 INFO:tasks.workunit.client.0.smithi042.stdout:9/128: symlink d4/d12/l29 0 2024-09-16T23:48:51.609 INFO:tasks.workunit.client.1.smithi043.stdout:2/284: creat d8/d1d/d20/d23/f6a x:0 0 0 2024-09-16T23:48:51.611 INFO:tasks.workunit.client.0.smithi042.stdout:4/177: chown de/d1a/l43 1934 1 2024-09-16T23:48:51.614 INFO:tasks.workunit.client.0.smithi042.stdout:2/168: mknod d0/da/d1b/d3b/c3f 0 2024-09-16T23:48:51.615 INFO:tasks.workunit.client.0.smithi042.stdout:7/173: link d3/cf d3/c38 0 2024-09-16T23:48:51.615 INFO:tasks.workunit.client.0.smithi042.stdout:9/129: write d4/d10/f1b [683580,835] 0 2024-09-16T23:48:51.615 INFO:tasks.workunit.client.1.smithi043.stdout:1/458: mkdir d4/d9/d1c/d1e/d47/d97 0 2024-09-16T23:48:51.618 INFO:tasks.workunit.client.1.smithi043.stdout:5/503: symlink d3/d9/d6f/d7c/d80/l9e 0 2024-09-16T23:48:51.618 INFO:tasks.workunit.client.0.smithi042.stdout:1/228: dwrite d0/f1d [0,4194304] 0 2024-09-16T23:48:51.619 INFO:tasks.workunit.client.0.smithi042.stdout:2/169: write d0/da/f23 [131998,127585] 0 2024-09-16T23:48:51.620 INFO:tasks.workunit.client.1.smithi043.stdout:0/351: mkdir d0/d2/df/d1a/d49/d71 0 2024-09-16T23:48:51.621 INFO:tasks.workunit.client.1.smithi043.stdout:0/352: dread - d0/f4d zero size 2024-09-16T23:48:51.621 INFO:tasks.workunit.client.0.smithi042.stdout:7/174: rmdir d3/d1f 39 2024-09-16T23:48:51.623 INFO:tasks.workunit.client.0.smithi042.stdout:0/135: dread d0/f4 [0,4194304] 0 2024-09-16T23:48:51.623 INFO:tasks.workunit.client.1.smithi043.stdout:2/285: mkdir d8/d1f/d6b 0 2024-09-16T23:48:51.624 INFO:tasks.workunit.client.1.smithi043.stdout:2/286: readlink d8/d1d/d20/d23/d38/l4d 0 2024-09-16T23:48:51.625 INFO:tasks.workunit.client.1.smithi043.stdout:2/287: write d8/d1d/d20/f48 [753661,93889] 0 2024-09-16T23:48:51.626 INFO:tasks.workunit.client.1.smithi043.stdout:2/288: chown d8/d1f/f5d 156701 1 2024-09-16T23:48:51.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/175: write d3/f18 [1546796,58118] 0 2024-09-16T23:48:51.629 INFO:tasks.workunit.client.0.smithi042.stdout:0/136: chown d0/f1c 10 1 2024-09-16T23:48:51.631 INFO:tasks.workunit.client.0.smithi042.stdout:0/137: chown d0/df/c1a 482221137 1 2024-09-16T23:48:51.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/229: dwrite d0/d17/f4f [0,4194304] 0 2024-09-16T23:48:51.633 INFO:tasks.workunit.client.0.smithi042.stdout:6/140: truncate d8/db/dd/f15 1903404 0 2024-09-16T23:48:51.633 INFO:tasks.workunit.client.0.smithi042.stdout:6/141: chown d8 52 1 2024-09-16T23:48:51.634 INFO:tasks.workunit.client.1.smithi043.stdout:1/459: mkdir d4/d9/d12/d2b/d3d/d83/d98 0 2024-09-16T23:48:51.635 INFO:tasks.workunit.client.0.smithi042.stdout:9/130: symlink d4/d10/d24/d25/l2a 0 2024-09-16T23:48:51.637 INFO:tasks.workunit.client.0.smithi042.stdout:7/176: dread d3/f18 [0,4194304] 0 2024-09-16T23:48:51.638 INFO:tasks.workunit.client.0.smithi042.stdout:7/177: chown d3 84 1 2024-09-16T23:48:51.640 INFO:tasks.workunit.client.1.smithi043.stdout:5/504: dread d3/f17 [4194304,4194304] 0 2024-09-16T23:48:51.641 INFO:tasks.workunit.client.0.smithi042.stdout:0/138: symlink d0/df/l24 0 2024-09-16T23:48:51.645 INFO:tasks.workunit.client.0.smithi042.stdout:7/178: dread d3/d15/d16/f1c [0,4194304] 0 2024-09-16T23:48:51.646 INFO:tasks.workunit.client.0.smithi042.stdout:1/230: rmdir d0/d17 39 2024-09-16T23:48:51.647 INFO:tasks.workunit.client.0.smithi042.stdout:1/231: fdatasync d0/d9/d35/d46/f30 0 2024-09-16T23:48:51.647 INFO:tasks.workunit.client.0.smithi042.stdout:0/139: rename d0/l1e to d0/df/d23/l25 0 2024-09-16T23:48:51.647 INFO:tasks.workunit.client.1.smithi043.stdout:3/367: truncate d4/de/f58 1787261 0 2024-09-16T23:48:51.650 INFO:tasks.workunit.client.0.smithi042.stdout:7/179: dread d3/fc [0,4194304] 0 2024-09-16T23:48:51.650 INFO:tasks.workunit.client.0.smithi042.stdout:7/180: chown d3/lb 21619728 1 2024-09-16T23:48:51.652 INFO:tasks.workunit.client.0.smithi042.stdout:0/140: write d0/df/d13/f22 [377931,115834] 0 2024-09-16T23:48:51.654 INFO:tasks.workunit.client.0.smithi042.stdout:6/142: dread d8/db/dd/d2c/f10 [0,4194304] 0 2024-09-16T23:48:51.655 INFO:tasks.workunit.client.1.smithi043.stdout:1/460: creat d4/d34/f99 x:0 0 0 2024-09-16T23:48:51.656 INFO:tasks.workunit.client.1.smithi043.stdout:1/461: chown d4/df/d26/d79/f96 5854 1 2024-09-16T23:48:51.659 INFO:tasks.workunit.client.0.smithi042.stdout:1/232: rmdir d0/d15 39 2024-09-16T23:48:51.661 INFO:tasks.workunit.client.0.smithi042.stdout:0/141: dwrite d0/f9 [4194304,4194304] 0 2024-09-16T23:48:51.662 INFO:tasks.workunit.client.1.smithi043.stdout:5/505: unlink d3/d9/d22/d4a/d4c/f72 0 2024-09-16T23:48:51.662 INFO:tasks.workunit.client.0.smithi042.stdout:0/142: stat d0/d15/c20 0 2024-09-16T23:48:51.665 INFO:tasks.workunit.client.0.smithi042.stdout:0/143: rmdir d0/df 39 2024-09-16T23:48:51.666 INFO:tasks.workunit.client.0.smithi042.stdout:0/144: chown d0/df/d13/f22 1151 1 2024-09-16T23:48:51.668 INFO:tasks.workunit.client.0.smithi042.stdout:0/145: dread d0/df/f21 [0,4194304] 0 2024-09-16T23:48:51.668 INFO:tasks.workunit.client.0.smithi042.stdout:6/143: chown d8/c1e 84803217 1 2024-09-16T23:48:51.670 INFO:tasks.workunit.client.1.smithi043.stdout:5/506: dwrite d3/db/d15/f3c [0,4194304] 0 2024-09-16T23:48:51.671 INFO:tasks.workunit.client.1.smithi043.stdout:3/368: rename d4/f33 to d4/de/d5d/f6c 0 2024-09-16T23:48:51.672 INFO:tasks.workunit.client.1.smithi043.stdout:6/381: getdents de/d23/d34/d66 0 2024-09-16T23:48:51.674 INFO:tasks.workunit.client.0.smithi042.stdout:1/233: symlink d0/d9/d35/d43/l51 0 2024-09-16T23:48:51.678 INFO:tasks.workunit.client.0.smithi042.stdout:9/131: link d4/d12/c1a d4/d10/d24/d25/c2b 0 2024-09-16T23:48:51.679 INFO:tasks.workunit.client.0.smithi042.stdout:6/144: rmdir d8/db/dd 39 2024-09-16T23:48:51.681 INFO:tasks.workunit.client.1.smithi043.stdout:0/353: creat d0/f72 x:0 0 0 2024-09-16T23:48:51.681 INFO:tasks.workunit.client.0.smithi042.stdout:2/170: link d0/l9 d0/d30/l40 0 2024-09-16T23:48:51.683 INFO:tasks.workunit.client.0.smithi042.stdout:0/146: dwrite d0/f7 [0,4194304] 0 2024-09-16T23:48:51.684 INFO:tasks.workunit.client.0.smithi042.stdout:9/132: unlink d4/fa 0 2024-09-16T23:48:51.685 INFO:tasks.workunit.client.0.smithi042.stdout:2/171: dread d0/da/f32 [0,4194304] 0 2024-09-16T23:48:51.685 INFO:tasks.workunit.client.1.smithi043.stdout:2/289: creat d8/d4b/f6c x:0 0 0 2024-09-16T23:48:51.686 INFO:tasks.workunit.client.1.smithi043.stdout:2/290: chown d8/d4b/f5b 0 1 2024-09-16T23:48:51.687 INFO:tasks.workunit.client.0.smithi042.stdout:2/172: dread d0/da/f32 [0,4194304] 0 2024-09-16T23:48:51.687 INFO:tasks.workunit.client.0.smithi042.stdout:3/144: sync 2024-09-16T23:48:51.687 INFO:tasks.workunit.client.1.smithi043.stdout:5/507: mkdir d3/d9/d22/d9f 0 2024-09-16T23:48:51.689 INFO:tasks.workunit.client.0.smithi042.stdout:9/133: readlink d4/d10/l22 0 2024-09-16T23:48:51.691 INFO:tasks.workunit.client.1.smithi043.stdout:6/382: symlink de/d23/d34/d66/d70/l7e 0 2024-09-16T23:48:51.691 INFO:tasks.workunit.client.1.smithi043.stdout:5/508: rename d3/d9/d6f/d7c/d80 to d3/d9/d6f/d7c/d80/da0 22 2024-09-16T23:48:51.691 INFO:tasks.workunit.client.0.smithi042.stdout:0/147: creat d0/df/f26 x:0 0 0 2024-09-16T23:48:51.692 INFO:tasks.workunit.client.0.smithi042.stdout:6/145: dwrite d8/db/dd/f22 [0,4194304] 0 2024-09-16T23:48:51.694 INFO:tasks.workunit.client.1.smithi043.stdout:0/354: symlink d0/d2/df/d1a/d46/l73 0 2024-09-16T23:48:51.695 INFO:tasks.workunit.client.1.smithi043.stdout:2/291: dwrite d8/fd [0,4194304] 0 2024-09-16T23:48:51.696 INFO:tasks.workunit.client.0.smithi042.stdout:9/134: symlink d4/d10/d24/l2c 0 2024-09-16T23:48:51.697 INFO:tasks.workunit.client.0.smithi042.stdout:0/148: creat d0/df/d23/f27 x:0 0 0 2024-09-16T23:48:51.699 INFO:tasks.workunit.client.0.smithi042.stdout:0/149: readlink d0/ld 0 2024-09-16T23:48:51.699 INFO:tasks.workunit.client.1.smithi043.stdout:6/383: mkdir de/d23/d63/d7f 0 2024-09-16T23:48:51.700 INFO:tasks.workunit.client.1.smithi043.stdout:5/509: dread d3/db/d15/f69 [0,4194304] 0 2024-09-16T23:48:51.701 INFO:tasks.workunit.client.0.smithi042.stdout:9/135: rmdir d4/d10/d24/d25 39 2024-09-16T23:48:51.701 INFO:tasks.workunit.client.0.smithi042.stdout:6/146: read d8/db/dd/d2c/fa [1141860,125465] 0 2024-09-16T23:48:51.701 INFO:tasks.workunit.client.1.smithi043.stdout:3/369: link d4/de/d1f/d2c/l62 d4/d30/l6d 0 2024-09-16T23:48:51.702 INFO:tasks.workunit.client.0.smithi042.stdout:3/145: link d2/d9/f22 d2/d9/dd/d17/d21/f31 0 2024-09-16T23:48:51.703 INFO:tasks.workunit.client.1.smithi043.stdout:1/462: getdents d4/d9/d1c/d1e 0 2024-09-16T23:48:51.704 INFO:tasks.workunit.client.0.smithi042.stdout:2/173: dwrite d0/f1a [0,4194304] 0 2024-09-16T23:48:51.706 INFO:tasks.workunit.client.1.smithi043.stdout:2/292: rename d8/d1d/d20/d23/d38/l4d to d8/d1d/l6d 0 2024-09-16T23:48:51.708 INFO:tasks.workunit.client.0.smithi042.stdout:3/146: stat d2/d1b/d1d/f15 0 2024-09-16T23:48:51.710 INFO:tasks.workunit.client.0.smithi042.stdout:0/150: dread d0/f1c [0,4194304] 0 2024-09-16T23:48:51.711 INFO:tasks.workunit.client.0.smithi042.stdout:6/147: dread d8/db/dd/f1f [0,4194304] 0 2024-09-16T23:48:51.711 INFO:tasks.workunit.client.0.smithi042.stdout:3/147: chown f0 0 1 2024-09-16T23:48:51.714 INFO:tasks.workunit.client.0.smithi042.stdout:9/136: dread d4/d10/f1b [0,4194304] 0 2024-09-16T23:48:51.715 INFO:tasks.workunit.client.1.smithi043.stdout:5/510: rmdir d3/d9/d22/d4a/d4c 39 2024-09-16T23:48:51.716 INFO:tasks.workunit.client.1.smithi043.stdout:8/350: sync 2024-09-16T23:48:51.716 INFO:tasks.workunit.client.1.smithi043.stdout:9/320: sync 2024-09-16T23:48:51.716 INFO:tasks.workunit.client.1.smithi043.stdout:4/346: sync 2024-09-16T23:48:51.717 INFO:tasks.workunit.client.0.smithi042.stdout:9/137: fdatasync d4/f5 0 2024-09-16T23:48:51.720 INFO:tasks.workunit.client.0.smithi042.stdout:9/138: read d4/d12/d16/f17 [1610918,76615] 0 2024-09-16T23:48:51.722 INFO:tasks.workunit.client.0.smithi042.stdout:6/148: mknod d8/db/dd/d2c/d26/c2f 0 2024-09-16T23:48:51.723 INFO:tasks.workunit.client.0.smithi042.stdout:0/151: dwrite d0/f4 [0,4194304] 0 2024-09-16T23:48:51.724 INFO:tasks.workunit.client.0.smithi042.stdout:3/148: dread d2/d9/dd/d17/d21/f31 [0,4194304] 0 2024-09-16T23:48:51.725 INFO:tasks.workunit.client.0.smithi042.stdout:0/152: chown d0/df/l16 129757780 1 2024-09-16T23:48:51.728 INFO:tasks.workunit.client.0.smithi042.stdout:3/149: creat d2/d9/dd/d2c/f32 x:0 0 0 2024-09-16T23:48:51.728 INFO:tasks.workunit.client.1.smithi043.stdout:6/384: mkdir de/d1a/d80 0 2024-09-16T23:48:51.729 INFO:tasks.workunit.client.1.smithi043.stdout:6/385: dread - de/f71 zero size 2024-09-16T23:48:51.729 INFO:tasks.workunit.client.1.smithi043.stdout:6/386: fsync de/d1a/d39/f78 0 2024-09-16T23:48:51.730 INFO:tasks.workunit.client.0.smithi042.stdout:3/150: write d2/d9/dd/d17/d21/f23 [703715,8556] 0 2024-09-16T23:48:51.731 INFO:tasks.workunit.client.0.smithi042.stdout:3/151: chown f0 0 1 2024-09-16T23:48:51.733 INFO:tasks.workunit.client.0.smithi042.stdout:0/153: link d0/c3 d0/c28 0 2024-09-16T23:48:51.733 INFO:tasks.workunit.client.1.smithi043.stdout:8/351: dread de/d18/d27/d41/d5a/f26 [0,4194304] 0 2024-09-16T23:48:51.735 INFO:tasks.workunit.client.0.smithi042.stdout:0/154: symlink d0/df/d13/l29 0 2024-09-16T23:48:51.735 INFO:tasks.workunit.client.0.smithi042.stdout:9/139: mkdir d4/d12/d2d 0 2024-09-16T23:48:51.738 INFO:tasks.workunit.client.0.smithi042.stdout:6/149: mknod d8/db/dd/c30 0 2024-09-16T23:48:51.738 INFO:tasks.workunit.client.0.smithi042.stdout:3/152: link f0 d2/d7/f33 0 2024-09-16T23:48:51.738 INFO:tasks.workunit.client.0.smithi042.stdout:3/153: read - d2/f24 zero size 2024-09-16T23:48:51.739 INFO:tasks.workunit.client.0.smithi042.stdout:0/155: symlink d0/l2a 0 2024-09-16T23:48:51.740 INFO:tasks.workunit.client.0.smithi042.stdout:3/154: mkdir d2/d1b/d1d/d34 0 2024-09-16T23:48:51.743 INFO:tasks.workunit.client.0.smithi042.stdout:3/155: creat d2/d9/f35 x:0 0 0 2024-09-16T23:48:51.744 INFO:tasks.workunit.client.1.smithi043.stdout:0/355: getdents d0 0 2024-09-16T23:48:51.745 INFO:tasks.workunit.client.0.smithi042.stdout:9/140: mknod d4/d12/d16/c2e 0 2024-09-16T23:48:51.745 INFO:tasks.workunit.client.0.smithi042.stdout:3/156: dread d2/d9/dd/d17/d21/f31 [0,4194304] 0 2024-09-16T23:48:51.748 INFO:tasks.workunit.client.0.smithi042.stdout:3/157: creat d2/d7/f36 x:0 0 0 2024-09-16T23:48:51.749 INFO:tasks.workunit.client.0.smithi042.stdout:6/150: rename f5 to d8/db/dd/d2c/f31 0 2024-09-16T23:48:51.750 INFO:tasks.workunit.client.1.smithi043.stdout:9/321: mkdir d4/d28/d5f 0 2024-09-16T23:48:51.752 INFO:tasks.workunit.client.0.smithi042.stdout:2/174: link d0/da/d1b/l1d d0/da/d17/d33/l41 0 2024-09-16T23:48:51.755 INFO:tasks.workunit.client.0.smithi042.stdout:3/158: truncate d2/d1b/d1d/f1a 615414 0 2024-09-16T23:48:51.755 INFO:tasks.workunit.client.0.smithi042.stdout:2/175: fsync d0/da/d17/f28 0 2024-09-16T23:48:51.759 INFO:tasks.workunit.client.0.smithi042.stdout:9/141: dwrite d4/f5 [8388608,4194304] 0 2024-09-16T23:48:51.760 INFO:tasks.workunit.client.1.smithi043.stdout:1/463: rename d4/d34/f99 to d4/d9/d1c/d1e/d42/d5d/f9a 0 2024-09-16T23:48:51.762 INFO:tasks.workunit.client.0.smithi042.stdout:3/159: mknod d2/d1b/c37 0 2024-09-16T23:48:51.763 INFO:tasks.workunit.client.0.smithi042.stdout:0/156: dwrite d0/df/f19 [0,4194304] 0 2024-09-16T23:48:51.763 INFO:tasks.workunit.client.0.smithi042.stdout:9/142: write d4/f5 [7672879,105370] 0 2024-09-16T23:48:51.766 INFO:tasks.workunit.client.0.smithi042.stdout:2/176: creat d0/da/d17/f42 x:0 0 0 2024-09-16T23:48:51.766 INFO:tasks.workunit.client.1.smithi043.stdout:3/370: dwrite d4/de/f55 [0,4194304] 0 2024-09-16T23:48:51.767 INFO:tasks.workunit.client.0.smithi042.stdout:6/151: dwrite d8/db/fc [0,4194304] 0 2024-09-16T23:48:51.768 INFO:tasks.workunit.client.0.smithi042.stdout:3/160: fsync d2/d9/f22 0 2024-09-16T23:48:51.771 INFO:tasks.workunit.client.0.smithi042.stdout:9/143: chown d4/d10/d24/d25/l2a 1115790 1 2024-09-16T23:48:51.774 INFO:tasks.workunit.client.0.smithi042.stdout:9/144: dread d4/fb [0,4194304] 0 2024-09-16T23:48:51.774 INFO:tasks.workunit.client.0.smithi042.stdout:0/157: unlink d0/f1b 0 2024-09-16T23:48:51.775 INFO:tasks.workunit.client.0.smithi042.stdout:2/177: mkdir d0/d30/d43 0 2024-09-16T23:48:51.775 INFO:tasks.workunit.client.1.smithi043.stdout:3/371: dwrite d4/de/d40/f69 [0,4194304] 0 2024-09-16T23:48:51.776 INFO:tasks.workunit.client.0.smithi042.stdout:3/161: creat d2/d1b/d1d/f38 x:0 0 0 2024-09-16T23:48:51.778 INFO:tasks.workunit.client.1.smithi043.stdout:3/372: readlink d4/l50 0 2024-09-16T23:48:51.779 INFO:tasks.workunit.client.0.smithi042.stdout:6/152: getdents d8/db/dd/d1b/d2d 0 2024-09-16T23:48:51.779 INFO:tasks.workunit.client.0.smithi042.stdout:6/153: fsync d8/db/f23 0 2024-09-16T23:48:51.781 INFO:tasks.workunit.client.0.smithi042.stdout:3/162: creat d2/d9/f39 x:0 0 0 2024-09-16T23:48:51.782 INFO:tasks.workunit.client.1.smithi043.stdout:5/511: truncate d3/db/d19/d3e/f45 3461773 0 2024-09-16T23:48:51.784 INFO:tasks.workunit.client.1.smithi043.stdout:5/512: read d3/db/d15/f3c [504308,95743] 0 2024-09-16T23:48:51.786 INFO:tasks.workunit.client.0.smithi042.stdout:0/158: symlink d0/df/d13/l2b 0 2024-09-16T23:48:51.790 INFO:tasks.workunit.client.0.smithi042.stdout:2/178: dwrite d0/da/d1b/f29 [0,4194304] 0 2024-09-16T23:48:51.793 INFO:tasks.workunit.client.0.smithi042.stdout:0/159: chown d0/df/f21 1 1 2024-09-16T23:48:51.794 INFO:tasks.workunit.client.0.smithi042.stdout:2/179: read d0/f1a [2748975,121786] 0 2024-09-16T23:48:51.794 INFO:tasks.workunit.client.1.smithi043.stdout:2/293: rename d8/d4b/f6c to d8/d1f/f6e 0 2024-09-16T23:48:51.795 INFO:tasks.workunit.client.0.smithi042.stdout:6/154: dwrite d8/db/f12 [0,4194304] 0 2024-09-16T23:48:51.795 INFO:tasks.workunit.client.1.smithi043.stdout:2/294: dread - d8/d1f/f2e zero size 2024-09-16T23:48:51.796 INFO:tasks.workunit.client.0.smithi042.stdout:3/163: rmdir d2/d7 39 2024-09-16T23:48:51.796 INFO:tasks.workunit.client.1.smithi043.stdout:9/322: unlink d4/c7 0 2024-09-16T23:48:51.800 INFO:tasks.workunit.client.1.smithi043.stdout:9/323: dread d4/d1c/d4f/f52 [0,4194304] 0 2024-09-16T23:48:51.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/160: symlink d0/df/d23/l2c 0 2024-09-16T23:48:51.801 INFO:tasks.workunit.client.1.smithi043.stdout:9/324: fdatasync d4/d21/d33/f34 0 2024-09-16T23:48:51.803 INFO:tasks.workunit.client.0.smithi042.stdout:3/164: chown d2/d1b/f2a 8 1 2024-09-16T23:48:51.805 INFO:tasks.workunit.client.1.smithi043.stdout:5/513: fsync d3/db/d15/f5f 0 2024-09-16T23:48:51.805 INFO:tasks.workunit.client.1.smithi043.stdout:5/514: fdatasync d3/d9/d6f/d85/f98 0 2024-09-16T23:48:51.809 INFO:tasks.workunit.client.0.smithi042.stdout:6/155: creat d8/db/dd/d2c/d26/f32 x:0 0 0 2024-09-16T23:48:51.810 INFO:tasks.workunit.client.0.smithi042.stdout:9/145: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:51.811 INFO:tasks.workunit.client.1.smithi043.stdout:1/464: symlink d4/d9/d1c/d1e/d42/d5d/d71/d76/l9b 0 2024-09-16T23:48:51.815 INFO:tasks.workunit.client.1.smithi043.stdout:9/325: creat d4/d1c/d25/d2b/d4c/d50/f60 x:0 0 0 2024-09-16T23:48:51.817 INFO:tasks.workunit.client.1.smithi043.stdout:9/326: truncate d4/d21/d33/f55 607762 0 2024-09-16T23:48:51.818 INFO:tasks.workunit.client.1.smithi043.stdout:1/465: dwrite d4/d9/d1c/d1e/d47/f86 [4194304,4194304] 0 2024-09-16T23:48:51.818 INFO:tasks.workunit.client.0.smithi042.stdout:0/161: dwrite d0/df/f21 [0,4194304] 0 2024-09-16T23:48:51.820 INFO:tasks.workunit.client.0.smithi042.stdout:3/165: link d2/d9/dd/d17/f19 d2/d9/dd/d17/d21/f3a 0 2024-09-16T23:48:51.820 INFO:tasks.workunit.client.1.smithi043.stdout:1/466: fdatasync d4/d9/d1c/d1e/f23 0 2024-09-16T23:48:51.821 INFO:tasks.workunit.client.1.smithi043.stdout:8/352: getdents de/d18/d1f/d25 0 2024-09-16T23:48:51.822 INFO:tasks.workunit.client.0.smithi042.stdout:3/166: write d2/d9/dd/f12 [1942106,41326] 0 2024-09-16T23:48:51.823 INFO:tasks.workunit.client.0.smithi042.stdout:0/162: mkdir d0/df/d13/d2d 0 2024-09-16T23:48:51.825 INFO:tasks.workunit.client.1.smithi043.stdout:5/515: write d3/d9/d22/f4e [1014352,107125] 0 2024-09-16T23:48:51.828 INFO:tasks.workunit.client.1.smithi043.stdout:9/327: dwrite d4/d21/d33/f4e [0,4194304] 0 2024-09-16T23:48:51.828 INFO:tasks.workunit.client.0.smithi042.stdout:6/156: symlink d8/db/dd/d1b/l33 0 2024-09-16T23:48:51.829 INFO:tasks.workunit.client.0.smithi042.stdout:6/157: chown d8/db/dd/f1f 2946555 1 2024-09-16T23:48:51.830 INFO:tasks.workunit.client.1.smithi043.stdout:9/328: write d4/f1a [2088879,46677] 0 2024-09-16T23:48:51.831 INFO:tasks.workunit.client.1.smithi043.stdout:6/387: rename de/l69 to de/d1a/d80/l81 0 2024-09-16T23:48:51.832 INFO:tasks.workunit.client.0.smithi042.stdout:9/146: rename d4/d10/l11 to d4/d12/l2f 0 2024-09-16T23:48:51.832 INFO:tasks.workunit.client.0.smithi042.stdout:0/163: read - d0/d15/f1f zero size 2024-09-16T23:48:51.834 INFO:tasks.workunit.client.0.smithi042.stdout:9/147: write d4/f5 [4585714,55566] 0 2024-09-16T23:48:51.837 INFO:tasks.workunit.client.0.smithi042.stdout:3/167: rename d2/d9/f39 to d2/d1b/f3b 0 2024-09-16T23:48:51.837 INFO:tasks.workunit.client.0.smithi042.stdout:0/164: dread - d0/df/f26 zero size 2024-09-16T23:48:51.838 INFO:tasks.workunit.client.0.smithi042.stdout:6/158: mkdir d8/db/dd/d1b/d34 0 2024-09-16T23:48:51.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/239: sync 2024-09-16T23:48:51.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/240: chown da/d19/d2c 897302329 1 2024-09-16T23:48:51.842 INFO:tasks.workunit.client.0.smithi042.stdout:0/165: write d0/df/f21 [1149278,22471] 0 2024-09-16T23:48:51.842 INFO:tasks.workunit.client.0.smithi042.stdout:3/168: getdents d2/d1b/d1d/d30 0 2024-09-16T23:48:51.844 INFO:tasks.workunit.client.0.smithi042.stdout:9/148: dwrite d4/f5 [4194304,4194304] 0 2024-09-16T23:48:51.845 INFO:tasks.workunit.client.0.smithi042.stdout:2/180: write d0/f2a [43852,125377] 0 2024-09-16T23:48:51.849 INFO:tasks.workunit.client.0.smithi042.stdout:0/166: write d0/f1c [1974374,40248] 0 2024-09-16T23:48:51.850 INFO:tasks.workunit.client.1.smithi043.stdout:7/363: sync 2024-09-16T23:48:51.851 INFO:tasks.workunit.client.0.smithi042.stdout:3/169: creat d2/d9/dd/d17/d21/f3c x:0 0 0 2024-09-16T23:48:51.852 INFO:tasks.workunit.client.0.smithi042.stdout:6/159: dread d8/db/dd/f22 [0,4194304] 0 2024-09-16T23:48:51.852 INFO:tasks.workunit.client.1.smithi043.stdout:6/388: symlink de/d1a/d39/l82 0 2024-09-16T23:48:51.853 INFO:tasks.workunit.client.0.smithi042.stdout:0/167: rmdir d0/df/d13 39 2024-09-16T23:48:51.854 INFO:tasks.workunit.client.1.smithi043.stdout:0/356: rename d0/d2/df/d1a/d46/l73 to d0/dc/l74 0 2024-09-16T23:48:51.854 INFO:tasks.workunit.client.0.smithi042.stdout:6/160: fdatasync d8/db/f1d 0 2024-09-16T23:48:51.855 INFO:tasks.workunit.client.0.smithi042.stdout:9/149: chown d4/d10/d24/d25/c2b 866 1 2024-09-16T23:48:51.856 INFO:tasks.workunit.client.0.smithi042.stdout:9/150: chown d4/d12/f1e 24290328 1 2024-09-16T23:48:51.858 INFO:tasks.workunit.client.0.smithi042.stdout:2/181: rename d0/da/d17/d33 to d0/da/d1b/d3b/d44 0 2024-09-16T23:48:51.858 INFO:tasks.workunit.client.1.smithi043.stdout:2/295: creat d8/d11/f6f x:0 0 0 2024-09-16T23:48:51.859 INFO:tasks.workunit.client.1.smithi043.stdout:3/373: link d4/c26 d4/d30/d53/c6e 0 2024-09-16T23:48:51.860 INFO:tasks.workunit.client.1.smithi043.stdout:2/296: chown d8/d1d/f2a 32459158 1 2024-09-16T23:48:51.860 INFO:tasks.workunit.client.0.smithi042.stdout:0/168: dread d0/f9 [0,4194304] 0 2024-09-16T23:48:51.861 INFO:tasks.workunit.client.1.smithi043.stdout:3/374: fsync d4/de/d1f/f56 0 2024-09-16T23:48:51.861 INFO:tasks.workunit.client.0.smithi042.stdout:6/161: dread d8/db/dd/d2c/f13 [0,4194304] 0 2024-09-16T23:48:51.862 INFO:tasks.workunit.client.0.smithi042.stdout:3/170: write d2/d7/f33 [297523,120471] 0 2024-09-16T23:48:51.863 INFO:tasks.workunit.client.1.smithi043.stdout:0/357: dwrite d0/d2/df/d1a/d3b/f6c [0,4194304] 0 2024-09-16T23:48:51.863 INFO:tasks.workunit.client.1.smithi043.stdout:1/467: fdatasync d4/d9/f17 0 2024-09-16T23:48:51.865 INFO:tasks.workunit.client.0.smithi042.stdout:9/151: creat d4/d10/f30 x:0 0 0 2024-09-16T23:48:51.865 INFO:tasks.workunit.client.0.smithi042.stdout:6/162: write d8/db/dd/d2c/f10 [889746,91278] 0 2024-09-16T23:48:51.865 INFO:tasks.workunit.client.0.smithi042.stdout:3/171: chown d2/d1b/d1d 1043074595 1 2024-09-16T23:48:51.866 INFO:tasks.workunit.client.1.smithi043.stdout:3/375: truncate d4/de/f2a 4967092 0 2024-09-16T23:48:51.866 INFO:tasks.workunit.client.0.smithi042.stdout:9/152: readlink d4/ld 0 2024-09-16T23:48:51.867 INFO:tasks.workunit.client.1.smithi043.stdout:3/376: chown d4/d30/l39 5 1 2024-09-16T23:48:51.867 INFO:tasks.workunit.client.0.smithi042.stdout:2/182: mknod d0/da/d1b/c45 0 2024-09-16T23:48:51.870 INFO:tasks.workunit.client.0.smithi042.stdout:0/169: rename d0/d15/c20 to d0/df/c2e 0 2024-09-16T23:48:51.871 INFO:tasks.workunit.client.0.smithi042.stdout:0/170: stat d0/df/c17 0 2024-09-16T23:48:51.872 INFO:tasks.workunit.client.0.smithi042.stdout:6/163: dread f6 [0,4194304] 0 2024-09-16T23:48:51.873 INFO:tasks.workunit.client.0.smithi042.stdout:8/241: getdents da/d19/d2c 0 2024-09-16T23:48:51.874 INFO:tasks.workunit.client.0.smithi042.stdout:6/164: stat d8/db/f1d 0 2024-09-16T23:48:51.875 INFO:tasks.workunit.client.1.smithi043.stdout:3/377: dwrite d4/f54 [0,4194304] 0 2024-09-16T23:48:51.875 INFO:tasks.workunit.client.0.smithi042.stdout:3/172: dread d2/d9/dd/f1e [0,4194304] 0 2024-09-16T23:48:51.876 INFO:tasks.workunit.client.1.smithi043.stdout:7/364: rename c2 to d4/db/d11/d21/d36/c7d 0 2024-09-16T23:48:51.876 INFO:tasks.workunit.client.1.smithi043.stdout:4/347: sync 2024-09-16T23:48:51.882 INFO:tasks.workunit.client.0.smithi042.stdout:3/173: dread d2/d9/dd/d17/d21/f3a [0,4194304] 0 2024-09-16T23:48:51.884 INFO:tasks.workunit.client.0.smithi042.stdout:3/174: dread - d2/d1b/f3b zero size 2024-09-16T23:48:51.885 INFO:tasks.workunit.client.1.smithi043.stdout:2/297: truncate d8/d1d/d20/d23/d39/d3a/f62 77175 0 2024-09-16T23:48:51.887 INFO:tasks.workunit.client.0.smithi042.stdout:0/171: creat d0/d15/f2f x:0 0 0 2024-09-16T23:48:51.887 INFO:tasks.workunit.client.0.smithi042.stdout:3/175: creat d2/d1b/f3d x:0 0 0 2024-09-16T23:48:51.887 INFO:tasks.workunit.client.0.smithi042.stdout:0/172: write d0/df/f19 [1139490,2461] 0 2024-09-16T23:48:51.889 INFO:tasks.workunit.client.0.smithi042.stdout:0/173: chown d0/df/c17 2 1 2024-09-16T23:48:51.890 INFO:tasks.workunit.client.1.smithi043.stdout:0/358: mknod d0/d2/dd/c75 0 2024-09-16T23:48:51.890 INFO:tasks.workunit.client.1.smithi043.stdout:0/359: stat d0 0 2024-09-16T23:48:51.891 INFO:tasks.workunit.client.0.smithi042.stdout:2/183: mkdir d0/d46 0 2024-09-16T23:48:51.893 INFO:tasks.workunit.client.0.smithi042.stdout:3/176: getdents d2/d1b/d1d 0 2024-09-16T23:48:51.893 INFO:tasks.workunit.client.0.smithi042.stdout:9/153: creat d4/d10/d24/d25/d28/f31 x:0 0 0 2024-09-16T23:48:51.895 INFO:tasks.workunit.client.0.smithi042.stdout:9/154: fdatasync d4/d10/d24/d25/d28/f31 0 2024-09-16T23:48:51.895 INFO:tasks.workunit.client.1.smithi043.stdout:5/516: mkdir d3/d9/d22/d4a/d64/d8f/da1 0 2024-09-16T23:48:51.895 INFO:tasks.workunit.client.0.smithi042.stdout:0/174: link d0/l2a d0/d15/l30 0 2024-09-16T23:48:51.896 INFO:tasks.workunit.client.1.smithi043.stdout:5/517: chown d3/db/d19/d3e/l30 18 1 2024-09-16T23:48:51.897 INFO:tasks.workunit.client.0.smithi042.stdout:9/155: dread - d4/d10/f30 zero size 2024-09-16T23:48:51.897 INFO:tasks.workunit.client.0.smithi042.stdout:3/177: creat d2/d1b/d1d/d34/f3e x:0 0 0 2024-09-16T23:48:51.897 INFO:tasks.workunit.client.0.smithi042.stdout:0/175: fsync d0/df/d23/f27 0 2024-09-16T23:48:51.898 INFO:tasks.workunit.client.1.smithi043.stdout:9/329: creat d4/d28/f61 x:0 0 0 2024-09-16T23:48:51.900 INFO:tasks.workunit.client.0.smithi042.stdout:0/176: rename d0/df/f26 to d0/df/d23/f31 0 2024-09-16T23:48:51.901 INFO:tasks.workunit.client.0.smithi042.stdout:3/178: dread d2/d7/f33 [0,4194304] 0 2024-09-16T23:48:51.902 INFO:tasks.workunit.client.0.smithi042.stdout:0/177: chown d0/df/f21 141633 1 2024-09-16T23:48:51.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/179: chown f0 0 1 2024-09-16T23:48:51.903 INFO:tasks.workunit.client.0.smithi042.stdout:3/180: fdatasync d2/d7/ff 0 2024-09-16T23:48:51.903 INFO:tasks.workunit.client.0.smithi042.stdout:0/178: dread - d0/df/d23/f27 zero size 2024-09-16T23:48:51.904 INFO:tasks.workunit.client.1.smithi043.stdout:3/378: rmdir d4 39 2024-09-16T23:48:51.906 INFO:tasks.workunit.client.1.smithi043.stdout:7/365: rmdir d4/db/d11/d15 39 2024-09-16T23:48:51.906 INFO:tasks.workunit.client.0.smithi042.stdout:3/181: rename d2/d9/dd/d17/f19 to d2/d9/dd/f3f 0 2024-09-16T23:48:51.908 INFO:tasks.workunit.client.1.smithi043.stdout:2/298: symlink d8/d4b/d4f/d56/l70 0 2024-09-16T23:48:51.909 INFO:tasks.workunit.client.0.smithi042.stdout:8/242: getdents da/d19/d1d/d30 0 2024-09-16T23:48:51.909 INFO:tasks.workunit.client.0.smithi042.stdout:3/182: creat d2/d1b/d1d/f40 x:0 0 0 2024-09-16T23:48:51.911 INFO:tasks.workunit.client.1.smithi043.stdout:8/353: truncate de/d18/d1f/f2d 464999 0 2024-09-16T23:48:51.911 INFO:tasks.workunit.client.0.smithi042.stdout:8/243: write da/d2f/f38 [547083,9130] 0 2024-09-16T23:48:51.911 INFO:tasks.workunit.client.0.smithi042.stdout:0/179: rename d0/df/d13/l2b to d0/d15/l32 0 2024-09-16T23:48:51.912 INFO:tasks.workunit.client.0.smithi042.stdout:9/156: creat d4/d12/d2d/f32 x:0 0 0 2024-09-16T23:48:51.914 INFO:tasks.workunit.client.0.smithi042.stdout:4/178: sync 2024-09-16T23:48:51.915 INFO:tasks.workunit.client.0.smithi042.stdout:5/209: sync 2024-09-16T23:48:51.915 INFO:tasks.workunit.client.0.smithi042.stdout:9/157: fsync d4/d10/f1f 0 2024-09-16T23:48:51.915 INFO:tasks.workunit.client.0.smithi042.stdout:3/183: creat d2/f41 x:0 0 0 2024-09-16T23:48:51.917 INFO:tasks.workunit.client.0.smithi042.stdout:0/180: mkdir d0/df/d23/d33 0 2024-09-16T23:48:51.917 INFO:tasks.workunit.client.0.smithi042.stdout:6/165: chown f7 1 1 2024-09-16T23:48:51.917 INFO:tasks.workunit.client.0.smithi042.stdout:2/184: dwrite d0/f12 [0,4194304] 0 2024-09-16T23:48:51.918 INFO:tasks.workunit.client.0.smithi042.stdout:4/179: chown de/d12/f15 726281 1 2024-09-16T23:48:51.920 INFO:tasks.workunit.client.0.smithi042.stdout:5/210: dread - d9/db/f3e zero size 2024-09-16T23:48:51.920 INFO:tasks.workunit.client.0.smithi042.stdout:0/181: fsync d0/df/f14 0 2024-09-16T23:48:51.921 INFO:tasks.workunit.client.1.smithi043.stdout:3/379: readlink d4/de/d1f/d34/l5b 0 2024-09-16T23:48:51.922 INFO:tasks.workunit.client.0.smithi042.stdout:2/185: write d0/da/f1f [1511950,38712] 0 2024-09-16T23:48:51.924 INFO:tasks.workunit.client.0.smithi042.stdout:8/244: dread da/f16 [0,4194304] 0 2024-09-16T23:48:51.925 INFO:tasks.workunit.client.1.smithi043.stdout:2/299: symlink d8/d1d/d20/d23/l71 0 2024-09-16T23:48:51.925 INFO:tasks.workunit.client.0.smithi042.stdout:9/158: read d4/d10/f15 [897460,78728] 0 2024-09-16T23:48:51.926 INFO:tasks.workunit.client.0.smithi042.stdout:9/159: fsync d4/fb 0 2024-09-16T23:48:51.928 INFO:tasks.workunit.client.0.smithi042.stdout:8/245: chown da/d2f/f38 549260 1 2024-09-16T23:48:51.929 INFO:tasks.workunit.client.0.smithi042.stdout:5/211: mknod d9/db/d1b/c42 0 2024-09-16T23:48:51.930 INFO:tasks.workunit.client.1.smithi043.stdout:6/389: write de/d1a/f3b [1080299,12116] 0 2024-09-16T23:48:51.931 INFO:tasks.workunit.client.1.smithi043.stdout:7/366: dwrite d4/db/d11/d15/d29/f77 [0,4194304] 0 2024-09-16T23:48:51.931 INFO:tasks.workunit.client.0.smithi042.stdout:4/180: dread f7 [4194304,4194304] 0 2024-09-16T23:48:51.936 INFO:tasks.workunit.client.0.smithi042.stdout:3/184: rename d2/d7/fe to d2/d9/dd/d17/d21/f42 0 2024-09-16T23:48:51.936 INFO:tasks.workunit.client.0.smithi042.stdout:2/186: dread d0/ff [0,4194304] 0 2024-09-16T23:48:51.936 INFO:tasks.workunit.client.1.smithi043.stdout:5/518: symlink d3/d9/d22/d4a/d64/d8f/da1/la2 0 2024-09-16T23:48:51.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/212: mkdir d9/db/d17/d43 0 2024-09-16T23:48:51.937 INFO:tasks.workunit.client.1.smithi043.stdout:5/519: readlink d3/d1b/l1d 0 2024-09-16T23:48:51.939 INFO:tasks.workunit.client.1.smithi043.stdout:9/330: link d4/d21/d33/f55 d4/d1c/d25/f62 0 2024-09-16T23:48:51.941 INFO:tasks.workunit.client.1.smithi043.stdout:3/380: symlink d4/de/d40/d4d/l6f 0 2024-09-16T23:48:51.942 INFO:tasks.workunit.client.0.smithi042.stdout:3/185: creat d2/d1b/f43 x:0 0 0 2024-09-16T23:48:51.942 INFO:tasks.workunit.client.0.smithi042.stdout:9/160: mknod d4/d10/c33 0 2024-09-16T23:48:51.943 INFO:tasks.workunit.client.0.smithi042.stdout:4/181: creat de/d42/f44 x:0 0 0 2024-09-16T23:48:51.943 INFO:tasks.workunit.client.1.smithi043.stdout:0/360: dwrite d0/d2/df/d1a/f30 [0,4194304] 0 2024-09-16T23:48:51.944 INFO:tasks.workunit.client.0.smithi042.stdout:3/186: chown d2/d9/dd/c29 140724 1 2024-09-16T23:48:51.944 INFO:tasks.workunit.client.1.smithi043.stdout:9/331: truncate d4/d5/d16/f54 870012 0 2024-09-16T23:48:51.945 INFO:tasks.workunit.client.0.smithi042.stdout:0/182: getdents d0/df 0 2024-09-16T23:48:51.946 INFO:tasks.workunit.client.1.smithi043.stdout:2/300: unlink d8/d11/c18 0 2024-09-16T23:48:51.946 INFO:tasks.workunit.client.0.smithi042.stdout:8/246: creat da/d19/d2c/f4e x:0 0 0 2024-09-16T23:48:51.947 INFO:tasks.workunit.client.1.smithi043.stdout:5/520: dread d3/db/f52 [0,4194304] 0 2024-09-16T23:48:51.947 INFO:tasks.workunit.client.1.smithi043.stdout:1/468: link d4/d9/c87 d4/d9/d12/d2b/d3d/d83/d98/c9c 0 2024-09-16T23:48:51.947 INFO:tasks.workunit.client.0.smithi042.stdout:5/213: mkdir d9/db/d17/d44 0 2024-09-16T23:48:51.948 INFO:tasks.workunit.client.0.smithi042.stdout:8/247: chown da/d19/d1d/f2e 2673096 1 2024-09-16T23:48:51.952 INFO:tasks.workunit.client.0.smithi042.stdout:2/187: creat d0/d30/f47 x:0 0 0 2024-09-16T23:48:51.952 INFO:tasks.workunit.client.0.smithi042.stdout:4/182: rmdir de/d12/d1b 39 2024-09-16T23:48:51.953 INFO:tasks.workunit.client.0.smithi042.stdout:0/183: dread d0/df/f19 [0,4194304] 0 2024-09-16T23:48:51.955 INFO:tasks.workunit.client.1.smithi043.stdout:1/469: dwrite d4/d9/d12/f82 [0,4194304] 0 2024-09-16T23:48:51.956 INFO:tasks.workunit.client.0.smithi042.stdout:4/183: write de/d12/f15 [4082328,41447] 0 2024-09-16T23:48:51.957 INFO:tasks.workunit.client.1.smithi043.stdout:1/470: write d4/d9/d1c/d1e/d47/f53 [412575,12879] 0 2024-09-16T23:48:51.957 INFO:tasks.workunit.client.0.smithi042.stdout:6/166: truncate d8/db/fc 3773593 0 2024-09-16T23:48:51.958 INFO:tasks.workunit.client.1.smithi043.stdout:1/471: write d4/d34/f41 [2630377,32611] 0 2024-09-16T23:48:51.960 INFO:tasks.workunit.client.0.smithi042.stdout:0/184: dread d0/f4 [0,4194304] 0 2024-09-16T23:48:51.961 INFO:tasks.workunit.client.0.smithi042.stdout:2/188: mkdir d0/da/d1b/d48 0 2024-09-16T23:48:51.963 INFO:tasks.workunit.client.0.smithi042.stdout:7/181: sync 2024-09-16T23:48:51.963 INFO:tasks.workunit.client.0.smithi042.stdout:1/234: sync 2024-09-16T23:48:51.963 INFO:tasks.workunit.client.0.smithi042.stdout:3/187: link d2/d9/dd/f1e d2/d1b/d1d/f44 0 2024-09-16T23:48:51.964 INFO:tasks.workunit.client.1.smithi043.stdout:1/472: dread d4/d9/d12/f82 [0,4194304] 0 2024-09-16T23:48:51.964 INFO:tasks.workunit.client.1.smithi043.stdout:3/381: stat d4/c26 0 2024-09-16T23:48:51.965 INFO:tasks.workunit.client.0.smithi042.stdout:3/188: write d2/d9/dd/d17/d21/f23 [1143009,12557] 0 2024-09-16T23:48:51.967 INFO:tasks.workunit.client.1.smithi043.stdout:3/382: read d4/de/d1f/f46 [917972,128007] 0 2024-09-16T23:48:51.968 INFO:tasks.workunit.client.0.smithi042.stdout:9/161: truncate d4/d10/f15 697540 0 2024-09-16T23:48:51.968 INFO:tasks.workunit.client.0.smithi042.stdout:4/184: write de/d12/d1b/f36 [596210,18291] 0 2024-09-16T23:48:51.969 INFO:tasks.workunit.client.1.smithi043.stdout:9/332: fsync d4/f48 0 2024-09-16T23:48:51.969 INFO:tasks.workunit.client.0.smithi042.stdout:3/189: creat d2/d9/dd/d17/d21/f45 x:0 0 0 2024-09-16T23:48:51.970 INFO:tasks.workunit.client.1.smithi043.stdout:9/333: dread d4/d1c/d25/f62 [0,4194304] 0 2024-09-16T23:48:51.973 INFO:tasks.workunit.client.0.smithi042.stdout:8/248: rename da/d19/d1d/d30/d49/c4a to da/d19/d1d/d30/d3f/c4f 0 2024-09-16T23:48:51.974 INFO:tasks.workunit.client.1.smithi043.stdout:0/361: creat d0/d2/df/d1a/d49/f76 x:0 0 0 2024-09-16T23:48:51.975 INFO:tasks.workunit.client.1.smithi043.stdout:0/362: chown d0/d2/dd/c54 1 1 2024-09-16T23:48:51.975 INFO:tasks.workunit.client.0.smithi042.stdout:7/182: creat d3/d15/d16/f39 x:0 0 0 2024-09-16T23:48:51.976 INFO:tasks.workunit.client.1.smithi043.stdout:0/363: truncate d0/f3c 2048316 0 2024-09-16T23:48:51.978 INFO:tasks.workunit.client.0.smithi042.stdout:4/185: dwrite de/f41 [0,4194304] 0 2024-09-16T23:48:51.979 INFO:tasks.workunit.client.0.smithi042.stdout:5/214: dread d9/db/d19/d21/f2b [0,4194304] 0 2024-09-16T23:48:51.980 INFO:tasks.workunit.client.0.smithi042.stdout:4/186: stat de/d1a/l43 0 2024-09-16T23:48:51.981 INFO:tasks.workunit.client.0.smithi042.stdout:2/189: creat d0/da/f49 x:0 0 0 2024-09-16T23:48:51.982 INFO:tasks.workunit.client.1.smithi043.stdout:5/521: symlink d3/d9/d6f/d85/la3 0 2024-09-16T23:48:51.982 INFO:tasks.workunit.client.0.smithi042.stdout:7/183: readlink d3/d1f/l2a 0 2024-09-16T23:48:51.983 INFO:tasks.workunit.client.0.smithi042.stdout:8/249: rmdir da/d19/d2c 39 2024-09-16T23:48:51.984 INFO:tasks.workunit.client.0.smithi042.stdout:7/184: truncate d3/f18 1824354 0 2024-09-16T23:48:51.985 INFO:tasks.workunit.client.0.smithi042.stdout:5/215: link d9/db/l3f d9/db/de/d10/l45 0 2024-09-16T23:48:51.987 INFO:tasks.workunit.client.0.smithi042.stdout:5/216: write d9/db/de/d10/d2f/f34 [3028932,122193] 0 2024-09-16T23:48:51.989 INFO:tasks.workunit.client.0.smithi042.stdout:7/185: symlink d3/l3a 0 2024-09-16T23:48:51.990 INFO:tasks.workunit.client.0.smithi042.stdout:8/250: rmdir da/d19/d1d 39 2024-09-16T23:48:51.991 INFO:tasks.workunit.client.0.smithi042.stdout:4/187: getdents de/d2e 0 2024-09-16T23:48:51.991 INFO:tasks.workunit.client.1.smithi043.stdout:7/367: symlink d4/db/d11/d21/d31/d33/d64/l7e 0 2024-09-16T23:48:51.992 INFO:tasks.workunit.client.1.smithi043.stdout:2/301: write d8/d1d/d20/d26/f3f [3872041,63917] 0 2024-09-16T23:48:51.993 INFO:tasks.workunit.client.0.smithi042.stdout:7/186: symlink d3/d1f/l3b 0 2024-09-16T23:48:51.993 INFO:tasks.workunit.client.0.smithi042.stdout:5/217: getdents d9/db/d17/d43 0 2024-09-16T23:48:51.995 INFO:tasks.workunit.client.1.smithi043.stdout:8/354: write de/d18/d1f/f2d [285019,6681] 0 2024-09-16T23:48:51.995 INFO:tasks.workunit.client.0.smithi042.stdout:7/187: mknod d3/d1f/c3c 0 2024-09-16T23:48:51.996 INFO:tasks.workunit.client.1.smithi043.stdout:6/390: mknod de/d23/d34/c83 0 2024-09-16T23:48:51.997 INFO:tasks.workunit.client.1.smithi043.stdout:6/391: chown de/l59 853690333 1 2024-09-16T23:48:51.998 INFO:tasks.workunit.client.0.smithi042.stdout:4/188: link de/d12/f25 de/d12/d1b/f45 0 2024-09-16T23:48:51.999 INFO:tasks.workunit.client.0.smithi042.stdout:5/218: truncate d9/db/d19/d21/f2e 4433626 0 2024-09-16T23:48:52.002 INFO:tasks.workunit.client.0.smithi042.stdout:4/189: rename f7 to de/d1a/f46 0 2024-09-16T23:48:52.003 INFO:tasks.workunit.client.0.smithi042.stdout:2/190: mknod d0/d46/c4a 0 2024-09-16T23:48:52.004 INFO:tasks.workunit.client.0.smithi042.stdout:4/190: write de/d1a/d1c/f2c [1092093,4812] 0 2024-09-16T23:48:52.005 INFO:tasks.workunit.client.0.smithi042.stdout:4/191: write de/d1a/f39 [104456,23859] 0 2024-09-16T23:48:52.006 INFO:tasks.workunit.client.1.smithi043.stdout:1/473: fsync d4/d9/f17 0 2024-09-16T23:48:52.007 INFO:tasks.workunit.client.0.smithi042.stdout:5/219: getdents d9/db/d19 0 2024-09-16T23:48:52.008 INFO:tasks.workunit.client.0.smithi042.stdout:7/188: write d3/d15/d24/f2d [412664,128228] 0 2024-09-16T23:48:52.010 INFO:tasks.workunit.client.0.smithi042.stdout:5/220: read d9/db/de/d10/f22 [135975,125903] 0 2024-09-16T23:48:52.010 INFO:tasks.workunit.client.0.smithi042.stdout:7/189: write d3/d15/d16/f25 [1200766,24139] 0 2024-09-16T23:48:52.011 INFO:tasks.workunit.client.0.smithi042.stdout:8/251: rmdir da/d19/d1d/d30/d3f 39 2024-09-16T23:48:52.012 INFO:tasks.workunit.client.0.smithi042.stdout:5/221: chown d9/db/f3e 978 1 2024-09-16T23:48:52.013 INFO:tasks.workunit.client.0.smithi042.stdout:7/190: write d3/fc [2050186,17425] 0 2024-09-16T23:48:52.014 INFO:tasks.workunit.client.1.smithi043.stdout:3/383: rename d4/de/l4b to d4/d30/d53/l70 0 2024-09-16T23:48:52.015 INFO:tasks.workunit.client.0.smithi042.stdout:5/222: mkdir d9/db/d1b/d31/d46 0 2024-09-16T23:48:52.017 INFO:tasks.workunit.client.0.smithi042.stdout:5/223: write d9/db/de/d10/f22 [1646409,118214] 0 2024-09-16T23:48:52.018 INFO:tasks.workunit.client.0.smithi042.stdout:2/191: truncate d0/f22 838959 0 2024-09-16T23:48:52.019 INFO:tasks.workunit.client.0.smithi042.stdout:7/191: dread d3/d15/d16/f1e [0,4194304] 0 2024-09-16T23:48:52.021 INFO:tasks.workunit.client.1.smithi043.stdout:0/364: mkdir d0/d2/df/d1a/d49/d77 0 2024-09-16T23:48:52.021 INFO:tasks.workunit.client.0.smithi042.stdout:6/167: truncate d8/db/dd/f19 4086898 0 2024-09-16T23:48:52.022 INFO:tasks.workunit.client.0.smithi042.stdout:5/224: truncate d9/db/f3e 1023564 0 2024-09-16T23:48:52.022 INFO:tasks.workunit.client.0.smithi042.stdout:4/192: dread de/d1a/d1c/f1e [0,4194304] 0 2024-09-16T23:48:52.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:51 smithi042 ceph-mon[27914]: mgrmap e23: smithi043.xveyou(active, since 3s) 2024-09-16T23:48:52.024 INFO:tasks.workunit.client.1.smithi043.stdout:7/368: creat d4/db/d43/f7f x:0 0 0 2024-09-16T23:48:52.025 INFO:tasks.workunit.client.0.smithi042.stdout:2/192: symlink d0/da/d1b/d3b/d44/l4b 0 2024-09-16T23:48:52.025 INFO:tasks.workunit.client.0.smithi042.stdout:4/193: creat de/d1a/d1c/f47 x:0 0 0 2024-09-16T23:48:52.026 INFO:tasks.workunit.client.0.smithi042.stdout:5/225: symlink d9/db/de/d10/l47 0 2024-09-16T23:48:52.027 INFO:tasks.workunit.client.0.smithi042.stdout:7/192: dread d3/d15/f1b [0,4194304] 0 2024-09-16T23:48:52.029 INFO:tasks.workunit.client.1.smithi043.stdout:4/348: sync 2024-09-16T23:48:52.029 INFO:tasks.workunit.client.1.smithi043.stdout:9/334: dwrite d4/d5/f46 [0,4194304] 0 2024-09-16T23:48:52.030 INFO:tasks.workunit.client.0.smithi042.stdout:5/226: write d9/db/de/d10/f20 [849559,123102] 0 2024-09-16T23:48:52.030 INFO:tasks.workunit.client.0.smithi042.stdout:4/194: creat de/d12/f48 x:0 0 0 2024-09-16T23:48:52.031 INFO:tasks.workunit.client.1.smithi043.stdout:9/335: write d4/d1c/d4f/f5b [816541,20678] 0 2024-09-16T23:48:52.033 INFO:tasks.workunit.client.0.smithi042.stdout:7/193: creat d3/d1f/f3d x:0 0 0 2024-09-16T23:48:52.035 INFO:tasks.workunit.client.0.smithi042.stdout:5/227: symlink d9/db/d1b/d31/l48 0 2024-09-16T23:48:52.035 INFO:tasks.workunit.client.1.smithi043.stdout:2/302: write d8/d1d/d20/d26/f3c [664160,39680] 0 2024-09-16T23:48:52.036 INFO:tasks.workunit.client.0.smithi042.stdout:6/168: symlink d8/db/dd/d1b/d34/l35 0 2024-09-16T23:48:52.036 INFO:tasks.workunit.client.1.smithi043.stdout:8/355: write de/d18/d27/d41/d5a/f56 [3886259,67072] 0 2024-09-16T23:48:52.038 INFO:tasks.workunit.client.1.smithi043.stdout:2/303: chown d8/d1d/d20/d23/d39/f59 1 1 2024-09-16T23:48:52.043 INFO:tasks.workunit.client.1.smithi043.stdout:1/474: symlink d4/df/d26/l9d 0 2024-09-16T23:48:52.043 INFO:tasks.workunit.client.1.smithi043.stdout:3/384: unlink d4/de/d5d/c66 0 2024-09-16T23:48:52.044 INFO:tasks.workunit.client.0.smithi042.stdout:7/194: rename d3/d15/d16/f39 to d3/d15/d24/d28/f3e 0 2024-09-16T23:48:52.044 INFO:tasks.workunit.client.0.smithi042.stdout:4/195: dread de/d1a/f46 [4194304,4194304] 0 2024-09-16T23:48:52.046 INFO:tasks.workunit.client.1.smithi043.stdout:0/365: creat d0/de/f78 x:0 0 0 2024-09-16T23:48:52.047 INFO:tasks.workunit.client.1.smithi043.stdout:0/366: fsync d0/dc/f50 0 2024-09-16T23:48:52.049 INFO:tasks.workunit.client.1.smithi043.stdout:3/385: dwrite d4/f67 [0,4194304] 0 2024-09-16T23:48:52.053 INFO:tasks.workunit.client.1.smithi043.stdout:8/356: dread de/f23 [0,4194304] 0 2024-09-16T23:48:52.054 INFO:tasks.workunit.client.1.smithi043.stdout:2/304: chown d8/d1d/d20/d23/d38/f69 1147 1 2024-09-16T23:48:52.055 INFO:tasks.workunit.client.0.smithi042.stdout:6/169: dwrite d8/db/dd/d2c/d26/f32 [0,4194304] 0 2024-09-16T23:48:52.057 INFO:tasks.workunit.client.1.smithi043.stdout:9/336: link d4/f31 d4/d21/f63 0 2024-09-16T23:48:52.063 INFO:tasks.workunit.client.1.smithi043.stdout:0/367: dread d0/f41 [0,4194304] 0 2024-09-16T23:48:52.065 INFO:tasks.workunit.client.0.smithi042.stdout:7/195: getdents d3 0 2024-09-16T23:48:52.067 INFO:tasks.workunit.client.1.smithi043.stdout:6/392: creat de/d1a/d28/f84 x:0 0 0 2024-09-16T23:48:52.067 INFO:tasks.workunit.client.0.smithi042.stdout:5/228: dwrite d9/db/d11/f40 [0,4194304] 0 2024-09-16T23:48:52.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/229: write d9/f3c [719092,1453] 0 2024-09-16T23:48:52.069 INFO:tasks.workunit.client.1.smithi043.stdout:1/475: symlink d4/l9e 0 2024-09-16T23:48:52.070 INFO:tasks.workunit.client.1.smithi043.stdout:5/522: getdents d3/d1b 0 2024-09-16T23:48:52.070 INFO:tasks.workunit.client.1.smithi043.stdout:1/476: dread - d4/df/d26/d79/f96 zero size 2024-09-16T23:48:52.071 INFO:tasks.workunit.client.0.smithi042.stdout:6/170: mknod d8/db/dd/d2c/c36 0 2024-09-16T23:48:52.072 INFO:tasks.workunit.client.1.smithi043.stdout:1/477: read - d4/d9/f84 zero size 2024-09-16T23:48:52.072 INFO:tasks.workunit.client.0.smithi042.stdout:4/196: dread de/d1a/f33 [0,4194304] 0 2024-09-16T23:48:52.072 INFO:tasks.workunit.client.1.smithi043.stdout:2/305: readlink d8/d1d/d20/d23/d39/d3a/l3b 0 2024-09-16T23:48:52.074 INFO:tasks.workunit.client.0.smithi042.stdout:2/193: dwrite d0/f11 [0,4194304] 0 2024-09-16T23:48:52.076 INFO:tasks.workunit.client.0.smithi042.stdout:2/194: write d0/da/d17/f2d [836442,114468] 0 2024-09-16T23:48:52.076 INFO:tasks.workunit.client.1.smithi043.stdout:8/357: rename de/d18/f21 to de/d18/d27/d41/d5a/d37/f69 0 2024-09-16T23:48:52.078 INFO:tasks.workunit.client.0.smithi042.stdout:5/230: getdents d9/d2c 0 2024-09-16T23:48:52.078 INFO:tasks.workunit.client.0.smithi042.stdout:6/171: chown d8/db/dd/d2c/f13 5 1 2024-09-16T23:48:52.079 INFO:tasks.workunit.client.0.smithi042.stdout:7/196: dread d3/d15/f1b [0,4194304] 0 2024-09-16T23:48:52.080 INFO:tasks.workunit.client.1.smithi043.stdout:5/523: dwrite d3/db/d19/d3e/f70 [0,4194304] 0 2024-09-16T23:48:52.080 INFO:tasks.workunit.client.1.smithi043.stdout:0/368: rmdir d0/d2/df/d1a/d46/d39/d6e 39 2024-09-16T23:48:52.080 INFO:tasks.workunit.client.1.smithi043.stdout:5/524: readlink d3/d9/d6f/l95 0 2024-09-16T23:48:52.081 INFO:tasks.workunit.client.0.smithi042.stdout:2/195: write d0/da/f23 [543452,30221] 0 2024-09-16T23:48:52.082 INFO:tasks.workunit.client.1.smithi043.stdout:0/369: write d0/d2/df/d1a/d49/f76 [41816,1985] 0 2024-09-16T23:48:52.082 INFO:tasks.workunit.client.1.smithi043.stdout:3/386: link d4/f67 d4/de/d40/f71 0 2024-09-16T23:48:52.083 INFO:tasks.workunit.client.0.smithi042.stdout:6/172: chown d8/db/lf 3103 1 2024-09-16T23:48:52.084 INFO:tasks.workunit.client.0.smithi042.stdout:7/197: mkdir d3/d3f 0 2024-09-16T23:48:52.085 INFO:tasks.workunit.client.1.smithi043.stdout:3/387: write d4/de/f55 [2317843,102308] 0 2024-09-16T23:48:52.091 INFO:tasks.workunit.client.1.smithi043.stdout:1/478: mknod d4/d9/d12/d2b/c9f 0 2024-09-16T23:48:52.092 INFO:tasks.workunit.client.1.smithi043.stdout:6/393: dwrite de/d1a/d28/f46 [0,4194304] 0 2024-09-16T23:48:52.092 INFO:tasks.workunit.client.1.smithi043.stdout:4/349: getdents d8/d3d 0 2024-09-16T23:48:52.092 INFO:tasks.workunit.client.1.smithi043.stdout:8/358: mkdir de/d18/d27/d41/d5a/d37/d6a 0 2024-09-16T23:48:52.095 INFO:tasks.workunit.client.0.smithi042.stdout:2/196: dread d0/da/f23 [0,4194304] 0 2024-09-16T23:48:52.095 INFO:tasks.workunit.client.0.smithi042.stdout:6/173: readlink d8/db/l28 0 2024-09-16T23:48:52.096 INFO:tasks.workunit.client.1.smithi043.stdout:0/370: symlink d0/de/l79 0 2024-09-16T23:48:52.096 INFO:tasks.workunit.client.0.smithi042.stdout:5/231: dwrite d9/db/d19/f39 [0,4194304] 0 2024-09-16T23:48:52.097 INFO:tasks.workunit.client.1.smithi043.stdout:3/388: dwrite d4/de/d1f/d2c/f4a [0,4194304] 0 2024-09-16T23:48:52.099 INFO:tasks.workunit.client.1.smithi043.stdout:8/359: read de/d18/d27/d34/d3d/f62 [1449347,67882] 0 2024-09-16T23:48:52.100 INFO:tasks.workunit.client.1.smithi043.stdout:0/371: truncate d0/f4d 440731 0 2024-09-16T23:48:52.101 INFO:tasks.workunit.client.0.smithi042.stdout:4/197: dwrite de/d1a/d1c/f21 [0,4194304] 0 2024-09-16T23:48:52.101 INFO:tasks.workunit.client.1.smithi043.stdout:5/525: mkdir d3/da4 0 2024-09-16T23:48:52.105 INFO:tasks.workunit.client.1.smithi043.stdout:6/394: dwrite de/d2a/f33 [0,4194304] 0 2024-09-16T23:48:52.105 INFO:tasks.workunit.client.1.smithi043.stdout:2/306: rename d8/d1d/f52 to d8/d1d/d20/d23/d39/d3a/f72 0 2024-09-16T23:48:52.106 INFO:tasks.workunit.client.1.smithi043.stdout:2/307: fsync d8/f43 0 2024-09-16T23:48:52.106 INFO:tasks.workunit.client.1.smithi043.stdout:6/395: readlink de/d2a/l4b 0 2024-09-16T23:48:52.110 INFO:tasks.workunit.client.0.smithi042.stdout:4/198: write de/d12/f25 [879442,20387] 0 2024-09-16T23:48:52.110 INFO:tasks.workunit.client.0.smithi042.stdout:6/174: dread - d8/f29 zero size 2024-09-16T23:48:52.110 INFO:tasks.workunit.client.1.smithi043.stdout:6/396: truncate de/d23/d49/f7c 335356 0 2024-09-16T23:48:52.115 INFO:tasks.workunit.client.0.smithi042.stdout:4/199: stat c3 0 2024-09-16T23:48:52.116 INFO:tasks.workunit.client.0.smithi042.stdout:2/197: unlink d0/da/d17/l19 0 2024-09-16T23:48:52.121 INFO:tasks.workunit.client.0.smithi042.stdout:4/200: read de/d12/d1b/f36 [889,98329] 0 2024-09-16T23:48:52.122 INFO:tasks.workunit.client.0.smithi042.stdout:4/201: read de/d12/d1b/f36 [115979,46446] 0 2024-09-16T23:48:52.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:51 smithi043 ceph-mon[35191]: mgrmap e23: smithi043.xveyou(active, since 3s) 2024-09-16T23:48:52.130 INFO:tasks.workunit.client.0.smithi042.stdout:2/198: dwrite d0/f12 [0,4194304] 0 2024-09-16T23:48:52.133 INFO:tasks.workunit.client.0.smithi042.stdout:4/202: link de/d12/c13 de/d2e/d37/d3c/c49 0 2024-09-16T23:48:52.134 INFO:tasks.workunit.client.0.smithi042.stdout:6/175: creat d8/f37 x:0 0 0 2024-09-16T23:48:52.136 INFO:tasks.workunit.client.0.smithi042.stdout:4/203: creat de/d1a/d1c/f4a x:0 0 0 2024-09-16T23:48:52.137 INFO:tasks.workunit.client.0.smithi042.stdout:5/232: truncate d9/db/d19/f39 376269 0 2024-09-16T23:48:52.138 INFO:tasks.workunit.client.1.smithi043.stdout:6/397: truncate de/d23/d49/f4c 446467 0 2024-09-16T23:48:52.138 INFO:tasks.workunit.client.0.smithi042.stdout:7/198: chown d3/d15/d16/f25 1036500 1 2024-09-16T23:48:52.139 INFO:tasks.workunit.client.0.smithi042.stdout:4/204: creat de/d1a/f4b x:0 0 0 2024-09-16T23:48:52.144 INFO:tasks.workunit.client.1.smithi043.stdout:9/337: dwrite d4/d21/d35/f5d [0,4194304] 0 2024-09-16T23:48:52.144 INFO:tasks.workunit.client.0.smithi042.stdout:7/199: symlink d3/d15/d16/l40 0 2024-09-16T23:48:52.145 INFO:tasks.workunit.client.1.smithi043.stdout:8/360: write de/d18/d1f/f32 [223813,8313] 0 2024-09-16T23:48:52.148 INFO:tasks.workunit.client.1.smithi043.stdout:2/308: write d8/d1d/d20/d23/d38/f69 [483133,30916] 0 2024-09-16T23:48:52.150 INFO:tasks.workunit.client.0.smithi042.stdout:4/205: dwrite de/d1a/d1c/f2c [0,4194304] 0 2024-09-16T23:48:52.150 INFO:tasks.workunit.client.1.smithi043.stdout:2/309: stat d8/d4b/d4f/d56 0 2024-09-16T23:48:52.152 INFO:tasks.workunit.client.0.smithi042.stdout:4/206: dread - de/d12/f28 zero size 2024-09-16T23:48:52.152 INFO:tasks.workunit.client.0.smithi042.stdout:5/233: dread d9/db/d11/f40 [0,4194304] 0 2024-09-16T23:48:52.153 INFO:tasks.workunit.client.1.smithi043.stdout:2/310: write d8/f1a [102592,57249] 0 2024-09-16T23:48:52.154 INFO:tasks.workunit.client.1.smithi043.stdout:2/311: truncate d8/d11/d16/f49 5104336 0 2024-09-16T23:48:52.155 INFO:tasks.workunit.client.0.smithi042.stdout:6/176: mkdir d8/d38 0 2024-09-16T23:48:52.156 INFO:tasks.workunit.client.1.smithi043.stdout:3/389: rename d4/de/d1f/d2c/l62 to d4/de/d1f/d34/l72 0 2024-09-16T23:48:52.156 INFO:tasks.workunit.client.0.smithi042.stdout:2/199: write d0/da/f23 [432706,103824] 0 2024-09-16T23:48:52.156 INFO:tasks.workunit.client.0.smithi042.stdout:7/200: rmdir d3/d15/d24/d28 39 2024-09-16T23:48:52.159 INFO:tasks.workunit.client.0.smithi042.stdout:4/207: rmdir de/d2e/d37 39 2024-09-16T23:48:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:5/234: creat d9/db/d1b/d31/d46/f49 x:0 0 0 2024-09-16T23:48:52.163 INFO:tasks.workunit.client.0.smithi042.stdout:4/208: mkdir de/d2e/d37/d4c 0 2024-09-16T23:48:52.164 INFO:tasks.workunit.client.0.smithi042.stdout:5/235: fdatasync d9/db/d19/f2d 0 2024-09-16T23:48:52.164 INFO:tasks.workunit.client.1.smithi043.stdout:4/350: symlink d8/d15/d20/d2b/d54/d55/l65 0 2024-09-16T23:48:52.165 INFO:tasks.workunit.client.1.smithi043.stdout:9/338: readlink d4/ld 0 2024-09-16T23:48:52.165 INFO:tasks.workunit.client.1.smithi043.stdout:3/390: dread d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:48:52.168 INFO:tasks.workunit.client.1.smithi043.stdout:8/361: rename de/d18/d2b/f2e to de/d18/d1f/d25/f6b 0 2024-09-16T23:48:52.169 INFO:tasks.workunit.client.0.smithi042.stdout:2/200: creat d0/da/d17/f4c x:0 0 0 2024-09-16T23:48:52.170 INFO:tasks.workunit.client.1.smithi043.stdout:8/362: write de/d18/d27/d41/d5a/f56 [4404010,72989] 0 2024-09-16T23:48:52.171 INFO:tasks.workunit.client.0.smithi042.stdout:5/236: readlink d9/l15 0 2024-09-16T23:48:52.173 INFO:tasks.workunit.client.0.smithi042.stdout:5/237: mkdir d9/d4a 0 2024-09-16T23:48:52.174 INFO:tasks.workunit.client.1.smithi043.stdout:1/479: truncate d4/d9/d12/f82 3481242 0 2024-09-16T23:48:52.174 INFO:tasks.workunit.client.1.smithi043.stdout:5/526: truncate d3/db/d26/f33 1720846 0 2024-09-16T23:48:52.175 INFO:tasks.workunit.client.1.smithi043.stdout:6/398: fdatasync de/f13 0 2024-09-16T23:48:52.175 INFO:tasks.workunit.client.1.smithi043.stdout:0/372: link d0/d2/df/f10 d0/d2/df/d1a/d46/d39/f7a 0 2024-09-16T23:48:52.176 INFO:tasks.workunit.client.0.smithi042.stdout:5/238: chown d9/db/d17/d44 15840121 1 2024-09-16T23:48:52.176 INFO:tasks.workunit.client.0.smithi042.stdout:7/201: dwrite d3/d15/d24/d28/f3e [0,4194304] 0 2024-09-16T23:48:52.176 INFO:tasks.workunit.client.1.smithi043.stdout:1/480: write d4/d9/d1c/d1e/d47/f4f [1218706,69146] 0 2024-09-16T23:48:52.177 INFO:tasks.workunit.client.1.smithi043.stdout:0/373: chown d0/d2/df/d1a/d3b/c3f 39412 1 2024-09-16T23:48:52.177 INFO:tasks.workunit.client.0.smithi042.stdout:5/239: write d9/db/d11/f32 [678457,69096] 0 2024-09-16T23:48:52.178 INFO:tasks.workunit.client.1.smithi043.stdout:9/339: mkdir d4/d5/d64 0 2024-09-16T23:48:52.178 INFO:tasks.workunit.client.1.smithi043.stdout:3/391: fsync d4/de/d1f/f29 0 2024-09-16T23:48:52.179 INFO:tasks.workunit.client.1.smithi043.stdout:4/351: readlink d8/d15/d20/d2b/d54/d55/l51 0 2024-09-16T23:48:52.180 INFO:tasks.workunit.client.1.smithi043.stdout:1/481: write d4/d9/d12/f5c [3036860,26425] 0 2024-09-16T23:48:52.180 INFO:tasks.workunit.client.0.smithi042.stdout:5/240: write d9/db/de/d10/d2f/f36 [5226567,2848] 0 2024-09-16T23:48:52.181 INFO:tasks.workunit.client.1.smithi043.stdout:0/374: fsync d0/d2/df/d1a/d3b/f2a 0 2024-09-16T23:48:52.181 INFO:tasks.workunit.client.1.smithi043.stdout:8/363: truncate de/d18/d1f/d25/f5f 3667466 0 2024-09-16T23:48:52.181 INFO:tasks.workunit.client.0.smithi042.stdout:5/241: fsync d9/db/f25 0 2024-09-16T23:48:52.182 INFO:tasks.workunit.client.0.smithi042.stdout:5/242: read d9/db/d11/f40 [463188,64216] 0 2024-09-16T23:48:52.186 INFO:tasks.workunit.client.0.smithi042.stdout:7/202: mkdir d3/d15/d41 0 2024-09-16T23:48:52.186 INFO:tasks.workunit.client.1.smithi043.stdout:3/392: write d4/de/d35/f4c [206887,106164] 0 2024-09-16T23:48:52.186 INFO:tasks.workunit.client.1.smithi043.stdout:0/375: readlink d0/l4 0 2024-09-16T23:48:52.188 INFO:tasks.workunit.client.0.smithi042.stdout:5/243: mkdir d9/db/de/d10/d4b 0 2024-09-16T23:48:52.192 INFO:tasks.workunit.client.0.smithi042.stdout:4/209: dread de/d1a/f46 [0,4194304] 0 2024-09-16T23:48:52.192 INFO:tasks.workunit.client.0.smithi042.stdout:4/210: fdatasync de/f2f 0 2024-09-16T23:48:52.193 INFO:tasks.workunit.client.1.smithi043.stdout:6/399: dread f2 [0,4194304] 0 2024-09-16T23:48:52.194 INFO:tasks.workunit.client.0.smithi042.stdout:4/211: truncate de/f19 8537925 0 2024-09-16T23:48:52.194 INFO:tasks.workunit.client.1.smithi043.stdout:2/312: rename d8/d11/d16/l53 to d8/d4b/d5f/l73 0 2024-09-16T23:48:52.199 INFO:tasks.workunit.client.0.smithi042.stdout:7/203: write d3/fe [5054116,10181] 0 2024-09-16T23:48:52.201 INFO:tasks.workunit.client.0.smithi042.stdout:2/201: write d0/da/f32 [545044,88402] 0 2024-09-16T23:48:52.205 INFO:tasks.workunit.client.0.smithi042.stdout:7/204: mknod d3/d15/c42 0 2024-09-16T23:48:52.205 INFO:tasks.workunit.client.1.smithi043.stdout:7/369: sync 2024-09-16T23:48:52.207 INFO:tasks.workunit.client.0.smithi042.stdout:5/244: dwrite d9/db/d19/d21/f2e [0,4194304] 0 2024-09-16T23:48:52.210 INFO:tasks.workunit.client.1.smithi043.stdout:1/482: symlink d4/d9/d72/la0 0 2024-09-16T23:48:52.210 INFO:tasks.workunit.client.0.smithi042.stdout:4/212: dwrite de/d1a/f33 [0,4194304] 0 2024-09-16T23:48:52.212 INFO:tasks.workunit.client.0.smithi042.stdout:5/245: stat d9/db/d19/f39 0 2024-09-16T23:48:52.213 INFO:tasks.workunit.client.0.smithi042.stdout:7/205: mknod d3/d1f/c43 0 2024-09-16T23:48:52.215 INFO:tasks.workunit.client.0.smithi042.stdout:5/246: read d9/db/de/d10/d2f/f34 [2906199,93216] 0 2024-09-16T23:48:52.216 INFO:tasks.workunit.client.0.smithi042.stdout:4/213: creat de/d2e/d37/d3e/f4d x:0 0 0 2024-09-16T23:48:52.217 INFO:tasks.workunit.client.1.smithi043.stdout:5/527: truncate d3/d9/f4b 3550666 0 2024-09-16T23:48:52.218 INFO:tasks.workunit.client.0.smithi042.stdout:7/206: chown d3/f8 401082990 1 2024-09-16T23:48:52.219 INFO:tasks.workunit.client.1.smithi043.stdout:0/376: mknod d0/dc/c7b 0 2024-09-16T23:48:52.219 INFO:tasks.workunit.client.1.smithi043.stdout:5/528: fdatasync d3/db/d15/f5f 0 2024-09-16T23:48:52.220 INFO:tasks.workunit.client.0.smithi042.stdout:4/214: write de/d1a/d1c/f2c [1042086,111857] 0 2024-09-16T23:48:52.220 INFO:tasks.workunit.client.0.smithi042.stdout:5/247: mknod d9/db/d11/c4c 0 2024-09-16T23:48:52.222 INFO:tasks.workunit.client.0.smithi042.stdout:5/248: dread - d9/db/d19/f2d zero size 2024-09-16T23:48:52.222 INFO:tasks.workunit.client.1.smithi043.stdout:4/352: dwrite d8/d15/d20/d2b/f28 [8388608,4194304] 0 2024-09-16T23:48:52.222 INFO:tasks.workunit.client.1.smithi043.stdout:2/313: mknod d8/d1d/d25/c74 0 2024-09-16T23:48:52.223 INFO:tasks.workunit.client.1.smithi043.stdout:0/377: chown d0/d2/df/d1a/d46/l4c 0 1 2024-09-16T23:48:52.224 INFO:tasks.workunit.client.1.smithi043.stdout:5/529: write d3/db/d19/f54 [403233,89803] 0 2024-09-16T23:48:52.224 INFO:tasks.workunit.client.1.smithi043.stdout:2/314: fsync d8/d1d/d20/f40 0 2024-09-16T23:48:52.224 INFO:tasks.workunit.client.0.smithi042.stdout:2/202: dwrite d0/d26/f2f [0,4194304] 0 2024-09-16T23:48:52.226 INFO:tasks.workunit.client.1.smithi043.stdout:1/483: creat d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fa1 x:0 0 0 2024-09-16T23:48:52.230 INFO:tasks.workunit.client.0.smithi042.stdout:4/215: mknod de/d2e/d37/c4e 0 2024-09-16T23:48:52.233 INFO:tasks.workunit.client.0.smithi042.stdout:7/207: mknod d3/d15/d16/c44 0 2024-09-16T23:48:52.233 INFO:tasks.workunit.client.0.smithi042.stdout:5/249: symlink d9/db/d17/d44/l4d 0 2024-09-16T23:48:52.234 INFO:tasks.workunit.client.0.smithi042.stdout:2/203: unlink d0/ff 0 2024-09-16T23:48:52.235 INFO:tasks.workunit.client.0.smithi042.stdout:2/204: fdatasync d0/da/d17/f28 0 2024-09-16T23:48:52.236 INFO:tasks.workunit.client.1.smithi043.stdout:4/353: symlink d8/d25/l66 0 2024-09-16T23:48:52.238 INFO:tasks.workunit.client.1.smithi043.stdout:5/530: symlink d3/d9/d22/la5 0 2024-09-16T23:48:52.239 INFO:tasks.workunit.client.1.smithi043.stdout:4/354: write d8/d15/f21 [6611672,97173] 0 2024-09-16T23:48:52.240 INFO:tasks.workunit.client.0.smithi042.stdout:2/205: unlink d0/da/d1b/c3d 0 2024-09-16T23:48:52.245 INFO:tasks.workunit.client.0.smithi042.stdout:4/216: getdents de/d42 0 2024-09-16T23:48:52.249 INFO:tasks.workunit.client.0.smithi042.stdout:4/217: fdatasync de/d1a/d1c/f21 0 2024-09-16T23:48:52.255 INFO:tasks.workunit.client.0.smithi042.stdout:5/250: dwrite d9/db/f28 [0,4194304] 0 2024-09-16T23:48:52.256 INFO:tasks.workunit.client.1.smithi043.stdout:3/393: link d4/de/l61 d4/de/d40/d4d/l73 0 2024-09-16T23:48:52.258 INFO:tasks.workunit.client.0.smithi042.stdout:2/206: dwrite d0/da/d17/f4c [0,4194304] 0 2024-09-16T23:48:52.259 INFO:tasks.workunit.client.1.smithi043.stdout:7/370: dwrite d4/d27/d41/f26 [0,4194304] 0 2024-09-16T23:48:52.261 INFO:tasks.workunit.client.1.smithi043.stdout:9/340: getdents d4/d28 0 2024-09-16T23:48:52.263 INFO:tasks.workunit.client.1.smithi043.stdout:6/400: creat de/f85 x:0 0 0 2024-09-16T23:48:52.263 INFO:tasks.workunit.client.0.smithi042.stdout:5/251: creat d9/db/d17/f4e x:0 0 0 2024-09-16T23:48:52.265 INFO:tasks.workunit.client.0.smithi042.stdout:4/218: getdents de/d2e/d37/d4c 0 2024-09-16T23:48:52.266 INFO:tasks.workunit.client.1.smithi043.stdout:0/378: creat d0/d2/df/d1a/d46/d39/d6e/f7c x:0 0 0 2024-09-16T23:48:52.267 INFO:tasks.workunit.client.1.smithi043.stdout:5/531: creat d3/d1b/fa6 x:0 0 0 2024-09-16T23:48:52.269 INFO:tasks.workunit.client.1.smithi043.stdout:2/315: symlink d8/d1d/d20/d23/d39/d3a/l75 0 2024-09-16T23:48:52.275 INFO:tasks.workunit.client.0.smithi042.stdout:4/219: dwrite fa [4194304,4194304] 0 2024-09-16T23:48:52.279 INFO:tasks.workunit.client.0.smithi042.stdout:5/252: dwrite d9/db/de/d10/d2f/f34 [0,4194304] 0 2024-09-16T23:48:52.279 INFO:tasks.workunit.client.1.smithi043.stdout:6/401: dread de/d1a/f42 [0,4194304] 0 2024-09-16T23:48:52.282 INFO:tasks.workunit.client.0.smithi042.stdout:5/253: mknod d9/db/d11/c4f 0 2024-09-16T23:48:52.282 INFO:tasks.workunit.client.0.smithi042.stdout:7/208: dread d3/fa [0,4194304] 0 2024-09-16T23:48:52.283 INFO:tasks.workunit.client.0.smithi042.stdout:4/220: dread de/d1a/d1c/f1e [0,4194304] 0 2024-09-16T23:48:52.288 INFO:tasks.workunit.client.0.smithi042.stdout:4/221: rmdir de/d1a 39 2024-09-16T23:48:52.291 INFO:tasks.workunit.client.1.smithi043.stdout:1/484: symlink d4/d9/d8f/la2 0 2024-09-16T23:48:52.294 INFO:tasks.workunit.client.0.smithi042.stdout:4/222: chown de/d12/l3a 2691 1 2024-09-16T23:48:52.294 INFO:tasks.workunit.client.0.smithi042.stdout:2/207: link d0/f22 d0/da/d17/f4d 0 2024-09-16T23:48:52.295 INFO:tasks.workunit.client.1.smithi043.stdout:3/394: rename d4/de/f2a to d4/de/d1f/d34/f74 0 2024-09-16T23:48:52.295 INFO:tasks.workunit.client.0.smithi042.stdout:2/208: fsync d0/da/f23 0 2024-09-16T23:48:52.298 INFO:tasks.workunit.client.0.smithi042.stdout:4/223: mkdir de/d2e/d37/d4c/d4f 0 2024-09-16T23:48:52.299 INFO:tasks.workunit.client.1.smithi043.stdout:5/532: fdatasync d3/db/f2a 0 2024-09-16T23:48:52.300 INFO:tasks.workunit.client.0.smithi042.stdout:4/224: stat de/d2e/d37/d4c/d4f 0 2024-09-16T23:48:52.301 INFO:tasks.workunit.client.0.smithi042.stdout:7/209: dwrite d3/f18 [0,4194304] 0 2024-09-16T23:48:52.304 INFO:tasks.workunit.client.0.smithi042.stdout:7/210: rmdir d3/d15/d24/d28 39 2024-09-16T23:48:52.308 INFO:tasks.workunit.client.0.smithi042.stdout:4/225: rename de/d1a/d1c/f1f to de/d42/f50 0 2024-09-16T23:48:52.309 INFO:tasks.workunit.client.1.smithi043.stdout:0/379: unlink d0/d2/df/d1a/d46/d39/d6e/f7c 0 2024-09-16T23:48:52.310 INFO:tasks.workunit.client.0.smithi042.stdout:7/211: dread d3/fa [0,4194304] 0 2024-09-16T23:48:52.310 INFO:tasks.workunit.client.0.smithi042.stdout:5/254: dread d9/db/de/d10/f22 [0,4194304] 0 2024-09-16T23:48:52.310 INFO:tasks.workunit.client.1.smithi043.stdout:2/316: creat d8/d4b/d5f/d63/f76 x:0 0 0 2024-09-16T23:48:52.311 INFO:tasks.workunit.client.1.smithi043.stdout:8/364: creat de/d18/d1f/d2f/d39/f6c x:0 0 0 2024-09-16T23:48:52.312 INFO:tasks.workunit.client.1.smithi043.stdout:1/485: dread d4/d9/d1c/d1e/f2a [0,4194304] 0 2024-09-16T23:48:52.312 INFO:tasks.workunit.client.1.smithi043.stdout:8/365: fsync de/f15 0 2024-09-16T23:48:52.313 INFO:tasks.workunit.client.1.smithi043.stdout:2/317: chown d8/d11/d16/f28 708709 1 2024-09-16T23:48:52.315 INFO:tasks.workunit.client.0.smithi042.stdout:7/212: fdatasync d3/d15/d16/d19/f21 0 2024-09-16T23:48:52.315 INFO:tasks.workunit.client.0.smithi042.stdout:2/209: creat d0/d26/f4e x:0 0 0 2024-09-16T23:48:52.316 INFO:tasks.workunit.client.1.smithi043.stdout:6/402: symlink de/d23/d34/l86 0 2024-09-16T23:48:52.316 INFO:tasks.workunit.client.1.smithi043.stdout:2/318: dread - d8/d1d/d20/f40 zero size 2024-09-16T23:48:52.316 INFO:tasks.workunit.client.1.smithi043.stdout:0/380: creat d0/d2/df/d1a/d3b/f7d x:0 0 0 2024-09-16T23:48:52.316 INFO:tasks.workunit.client.1.smithi043.stdout:4/355: getdents d8/d15 0 2024-09-16T23:48:52.317 INFO:tasks.workunit.client.1.smithi043.stdout:2/319: read - d8/d11/f46 zero size 2024-09-16T23:48:52.317 INFO:tasks.workunit.client.1.smithi043.stdout:8/366: rmdir de/d18/d1f/d25 39 2024-09-16T23:48:52.318 INFO:tasks.workunit.client.1.smithi043.stdout:4/356: read d8/d25/f2c [7370862,41485] 0 2024-09-16T23:48:52.318 INFO:tasks.workunit.client.1.smithi043.stdout:2/320: stat d8/d11/d16/c29 0 2024-09-16T23:48:52.319 INFO:tasks.workunit.client.1.smithi043.stdout:2/321: fsync d8/d35/f37 0 2024-09-16T23:48:52.320 INFO:tasks.workunit.client.0.smithi042.stdout:4/226: dread de/d1a/f46 [0,4194304] 0 2024-09-16T23:48:52.320 INFO:tasks.workunit.client.1.smithi043.stdout:7/371: rename d4/db/d28/l56 to d4/db/l80 0 2024-09-16T23:48:52.321 INFO:tasks.workunit.client.0.smithi042.stdout:7/213: read d3/f11 [22339,10212] 0 2024-09-16T23:48:52.325 INFO:tasks.workunit.client.0.smithi042.stdout:5/255: dwrite d9/db/d11/f32 [0,4194304] 0 2024-09-16T23:48:52.325 INFO:tasks.workunit.client.0.smithi042.stdout:7/214: chown d3/f30 0 1 2024-09-16T23:48:52.326 INFO:tasks.workunit.client.0.smithi042.stdout:5/256: truncate d9/db/d17/f24 979007 0 2024-09-16T23:48:52.328 INFO:tasks.workunit.client.0.smithi042.stdout:2/210: getdents d0/da/d1b/d48 0 2024-09-16T23:48:52.332 INFO:tasks.workunit.client.0.smithi042.stdout:4/227: getdents de/d2e/d37/d4c/d4f 0 2024-09-16T23:48:52.336 INFO:tasks.workunit.client.0.smithi042.stdout:4/228: readlink de/d1a/d1c/l3d 0 2024-09-16T23:48:52.337 INFO:tasks.workunit.client.1.smithi043.stdout:4/357: dread d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:48:52.339 INFO:tasks.workunit.client.0.smithi042.stdout:7/215: dwrite d3/d15/f1b [4194304,4194304] 0 2024-09-16T23:48:52.341 INFO:tasks.workunit.client.0.smithi042.stdout:2/211: mkdir d0/d30/d4f 0 2024-09-16T23:48:52.343 INFO:tasks.workunit.client.1.smithi043.stdout:4/358: dwrite d8/d15/d20/d2b/d23/f38 [0,4194304] 0 2024-09-16T23:48:52.344 INFO:tasks.workunit.client.0.smithi042.stdout:4/229: link de/c18 de/d1a/c51 0 2024-09-16T23:48:52.348 INFO:tasks.workunit.client.0.smithi042.stdout:7/216: creat d3/d15/d41/f45 x:0 0 0 2024-09-16T23:48:52.351 INFO:tasks.workunit.client.0.smithi042.stdout:5/257: dread d9/db/de/d10/f1a [0,4194304] 0 2024-09-16T23:48:52.352 INFO:tasks.workunit.client.0.smithi042.stdout:7/217: symlink d3/d15/d16/d19/l46 0 2024-09-16T23:48:52.353 INFO:tasks.workunit.client.1.smithi043.stdout:1/486: truncate d4/f69 3068799 0 2024-09-16T23:48:52.354 INFO:tasks.workunit.client.1.smithi043.stdout:1/487: read - d4/d9/f8e zero size 2024-09-16T23:48:52.355 INFO:tasks.workunit.client.1.smithi043.stdout:1/488: chown d4/d9/d1c/d1e/d47/c8c 1579244 1 2024-09-16T23:48:52.357 INFO:tasks.workunit.client.0.smithi042.stdout:2/212: creat d0/d26/f50 x:0 0 0 2024-09-16T23:48:52.357 INFO:tasks.workunit.client.0.smithi042.stdout:7/218: creat d3/d15/d41/f47 x:0 0 0 2024-09-16T23:48:52.357 INFO:tasks.workunit.client.0.smithi042.stdout:4/230: truncate de/d1a/d1c/f1e 399551 0 2024-09-16T23:48:52.359 INFO:tasks.workunit.client.0.smithi042.stdout:2/213: chown d0/f3a 15265932 1 2024-09-16T23:48:52.359 INFO:tasks.workunit.client.1.smithi043.stdout:9/341: rename d4/d21/d33/f34 to d4/d5/d64/f65 0 2024-09-16T23:48:52.361 INFO:tasks.workunit.client.1.smithi043.stdout:9/342: read d4/d1c/d4f/f5b [63713,47195] 0 2024-09-16T23:48:52.362 INFO:tasks.workunit.client.0.smithi042.stdout:4/231: mknod de/d2e/c52 0 2024-09-16T23:48:52.362 INFO:tasks.workunit.client.0.smithi042.stdout:5/258: dread d9/db/f28 [0,4194304] 0 2024-09-16T23:48:52.363 INFO:tasks.workunit.client.0.smithi042.stdout:7/219: creat d3/d15/f48 x:0 0 0 2024-09-16T23:48:52.364 INFO:tasks.workunit.client.1.smithi043.stdout:0/381: creat d0/d2/df/d1a/d49/d71/f7e x:0 0 0 2024-09-16T23:48:52.365 INFO:tasks.workunit.client.1.smithi043.stdout:8/367: symlink de/d18/d27/l6d 0 2024-09-16T23:48:52.366 INFO:tasks.workunit.client.0.smithi042.stdout:7/220: mkdir d3/d1f/d49 0 2024-09-16T23:48:52.367 INFO:tasks.workunit.client.0.smithi042.stdout:5/259: rename d9/db/d11/c4c to d9/d4a/c50 0 2024-09-16T23:48:52.369 INFO:tasks.workunit.client.1.smithi043.stdout:1/489: unlink d4/d9/d12/d2b/d3d/f61 0 2024-09-16T23:48:52.369 INFO:tasks.workunit.client.0.smithi042.stdout:7/221: creat d3/d15/d32/f4a x:0 0 0 2024-09-16T23:48:52.371 INFO:tasks.workunit.client.0.smithi042.stdout:5/260: unlink d9/db/d19/f2d 0 2024-09-16T23:48:52.372 INFO:tasks.workunit.client.0.smithi042.stdout:5/261: chown d9/db/de/d10/c18 2 1 2024-09-16T23:48:52.373 INFO:tasks.workunit.client.0.smithi042.stdout:7/222: symlink d3/d15/d16/d19/l4b 0 2024-09-16T23:48:52.374 INFO:tasks.workunit.client.0.smithi042.stdout:4/232: getdents de/d2e/d37/d3c 0 2024-09-16T23:48:52.375 INFO:tasks.workunit.client.1.smithi043.stdout:2/322: mkdir d8/d1d/d20/d23/d39/d3a/d77 0 2024-09-16T23:48:52.375 INFO:tasks.workunit.client.0.smithi042.stdout:4/233: dread - de/d42/f44 zero size 2024-09-16T23:48:52.376 INFO:tasks.workunit.client.1.smithi043.stdout:4/359: dread d8/fc [0,4194304] 0 2024-09-16T23:48:52.376 INFO:tasks.workunit.client.0.smithi042.stdout:5/262: mknod d9/db/d17/d43/c51 0 2024-09-16T23:48:52.377 INFO:tasks.workunit.client.0.smithi042.stdout:7/223: symlink d3/d15/d24/l4c 0 2024-09-16T23:48:52.379 INFO:tasks.workunit.client.0.smithi042.stdout:4/234: mknod de/d2e/c53 0 2024-09-16T23:48:52.379 INFO:tasks.workunit.client.0.smithi042.stdout:5/263: write d9/db/de/d10/d2f/f36 [4433795,42590] 0 2024-09-16T23:48:52.381 INFO:tasks.workunit.client.0.smithi042.stdout:5/264: write d9/db/f25 [1037652,26513] 0 2024-09-16T23:48:52.385 INFO:tasks.workunit.client.0.smithi042.stdout:5/265: creat d9/db/d19/f52 x:0 0 0 2024-09-16T23:48:52.386 INFO:tasks.workunit.client.1.smithi043.stdout:6/403: getdents de/d23/d34 0 2024-09-16T23:48:52.387 INFO:tasks.workunit.client.0.smithi042.stdout:5/266: chown d9/db/de/d10 690065504 1 2024-09-16T23:48:52.388 INFO:tasks.workunit.client.1.smithi043.stdout:7/372: creat d4/db/f81 x:0 0 0 2024-09-16T23:48:52.388 INFO:tasks.workunit.client.0.smithi042.stdout:7/224: dwrite d3/d15/d16/f1c [0,4194304] 0 2024-09-16T23:48:52.389 INFO:tasks.workunit.client.1.smithi043.stdout:8/368: dwrite de/d18/d27/d41/d5a/f26 [0,4194304] 0 2024-09-16T23:48:52.390 INFO:tasks.workunit.client.1.smithi043.stdout:6/404: dread f2 [0,4194304] 0 2024-09-16T23:48:52.392 INFO:tasks.workunit.client.0.smithi042.stdout:4/235: dwrite de/d1a/d1c/f2c [0,4194304] 0 2024-09-16T23:48:52.394 INFO:tasks.workunit.client.0.smithi042.stdout:4/236: write de/d1a/f39 [875724,53132] 0 2024-09-16T23:48:52.395 INFO:tasks.workunit.client.0.smithi042.stdout:7/225: dread d3/d15/d16/f1e [0,4194304] 0 2024-09-16T23:48:52.399 INFO:tasks.workunit.client.0.smithi042.stdout:5/267: dread d9/db/d19/f39 [0,4194304] 0 2024-09-16T23:48:52.400 INFO:tasks.workunit.client.0.smithi042.stdout:7/226: fdatasync d3/f8 0 2024-09-16T23:48:52.400 INFO:tasks.workunit.client.1.smithi043.stdout:3/395: rename d4/de/d35/c18 to d4/de/d5d/c75 0 2024-09-16T23:48:52.400 INFO:tasks.workunit.client.1.smithi043.stdout:9/343: dwrite d4/d21/f63 [0,4194304] 0 2024-09-16T23:48:52.401 INFO:tasks.workunit.client.0.smithi042.stdout:4/237: chown de/d12/c13 6 1 2024-09-16T23:48:52.403 INFO:tasks.workunit.client.1.smithi043.stdout:4/360: mknod d8/d3d/c67 0 2024-09-16T23:48:52.404 INFO:tasks.workunit.client.0.smithi042.stdout:5/268: mknod d9/db/d17/d43/c53 0 2024-09-16T23:48:52.405 INFO:tasks.workunit.client.0.smithi042.stdout:7/227: dread d3/d15/d16/f1e [0,4194304] 0 2024-09-16T23:48:52.408 INFO:tasks.workunit.client.1.smithi043.stdout:1/490: mknod d4/d9/ca3 0 2024-09-16T23:48:52.408 INFO:tasks.workunit.client.1.smithi043.stdout:4/361: chown d8/d15/d20/d2b/d54/d55/d49/l4e 384 1 2024-09-16T23:48:52.413 INFO:tasks.workunit.client.1.smithi043.stdout:5/533: rename d3/db/d15/d4d/c62 to d3/db/d19/d3e/d28/ca7 0 2024-09-16T23:48:52.413 INFO:tasks.workunit.client.0.smithi042.stdout:4/238: rename de/d1a/d1c/f2c to de/d12/f54 0 2024-09-16T23:48:52.414 INFO:tasks.workunit.client.0.smithi042.stdout:7/228: dwrite d3/d1f/f3d [0,4194304] 0 2024-09-16T23:48:52.414 INFO:tasks.workunit.client.1.smithi043.stdout:2/323: truncate d8/d1d/d20/d23/d39/d3a/f57 2914267 0 2024-09-16T23:48:52.415 INFO:tasks.workunit.client.1.smithi043.stdout:0/382: link d0/dc/c7b d0/d2/df/d1a/d3b/c7f 0 2024-09-16T23:48:52.418 INFO:tasks.workunit.client.0.smithi042.stdout:4/239: creat de/d1a/f55 x:0 0 0 2024-09-16T23:48:52.419 INFO:tasks.workunit.client.1.smithi043.stdout:7/373: unlink d4/d27/d41/d1b/f25 0 2024-09-16T23:48:52.421 INFO:tasks.workunit.client.1.smithi043.stdout:9/344: creat d4/d1c/d25/d2b/d4c/d50/f66 x:0 0 0 2024-09-16T23:48:52.421 INFO:tasks.workunit.client.0.smithi042.stdout:7/229: mkdir d3/d15/d24/d4d 0 2024-09-16T23:48:52.422 INFO:tasks.workunit.client.0.smithi042.stdout:7/230: chown d3/d1f/f26 768816804 1 2024-09-16T23:48:52.424 INFO:tasks.workunit.client.0.smithi042.stdout:7/231: stat d3/f11 0 2024-09-16T23:48:52.424 INFO:tasks.workunit.client.1.smithi043.stdout:8/369: creat de/d18/d1f/d25/f6e x:0 0 0 2024-09-16T23:48:52.425 INFO:tasks.workunit.client.0.smithi042.stdout:4/240: rmdir de/d1a/d1c 39 2024-09-16T23:48:52.427 INFO:tasks.workunit.client.0.smithi042.stdout:5/269: unlink d9/d4a/c50 0 2024-09-16T23:48:52.429 INFO:tasks.workunit.client.0.smithi042.stdout:4/241: write de/d1a/f4b [87606,34706] 0 2024-09-16T23:48:52.431 INFO:tasks.workunit.client.0.smithi042.stdout:4/242: read fa [5527765,67069] 0 2024-09-16T23:48:52.431 INFO:tasks.workunit.client.0.smithi042.stdout:5/270: symlink d9/db/d17/d43/l54 0 2024-09-16T23:48:52.432 INFO:tasks.workunit.client.1.smithi043.stdout:2/324: mknod d8/d1d/d20/d26/c78 0 2024-09-16T23:48:52.433 INFO:tasks.workunit.client.0.smithi042.stdout:5/271: dread - d9/db/d1b/d31/d46/f49 zero size 2024-09-16T23:48:52.433 INFO:tasks.workunit.client.1.smithi043.stdout:2/325: write d8/d4b/f5b [987303,54336] 0 2024-09-16T23:48:52.434 INFO:tasks.workunit.client.1.smithi043.stdout:0/383: mkdir d0/d2/df/d1a/d49/d80 0 2024-09-16T23:48:52.436 INFO:tasks.workunit.client.1.smithi043.stdout:7/374: creat d4/db/d11/d21/d31/f82 x:0 0 0 2024-09-16T23:48:52.437 INFO:tasks.workunit.client.1.smithi043.stdout:9/345: mkdir d4/d5/d30/d67 0 2024-09-16T23:48:52.442 INFO:tasks.workunit.client.0.smithi042.stdout:4/243: link de/d1a/f4b de/d1a/d1c/f56 0 2024-09-16T23:48:52.442 INFO:tasks.workunit.client.1.smithi043.stdout:0/384: dwrite d0/d2/df/d1a/d3b/f2a [0,4194304] 0 2024-09-16T23:48:52.443 INFO:tasks.workunit.client.0.smithi042.stdout:7/232: truncate d3/fe 2110789 0 2024-09-16T23:48:52.444 INFO:tasks.workunit.client.1.smithi043.stdout:2/326: unlink d8/d1d/d20/f40 0 2024-09-16T23:48:52.446 INFO:tasks.workunit.client.1.smithi043.stdout:6/405: getdents de/d23/d34/d66/d70 0 2024-09-16T23:48:52.446 INFO:tasks.workunit.client.0.smithi042.stdout:4/244: rmdir de/d2e/d37/d3c 39 2024-09-16T23:48:52.447 INFO:tasks.workunit.client.0.smithi042.stdout:5/272: getdents d9/db/d19 0 2024-09-16T23:48:52.448 INFO:tasks.workunit.client.1.smithi043.stdout:7/375: mkdir d4/db/d43/d83 0 2024-09-16T23:48:52.452 INFO:tasks.workunit.client.0.smithi042.stdout:4/245: unlink de/d2e/c53 0 2024-09-16T23:48:52.455 INFO:tasks.workunit.client.0.smithi042.stdout:5/273: mknod d9/c55 0 2024-09-16T23:48:52.455 INFO:tasks.workunit.client.0.smithi042.stdout:7/233: mknod d3/d15/c4e 0 2024-09-16T23:48:52.455 INFO:tasks.workunit.client.0.smithi042.stdout:5/274: chown d9/db/d1b/d31/d46/f49 1359146658 1 2024-09-16T23:48:52.457 INFO:tasks.workunit.client.1.smithi043.stdout:4/362: creat d8/f68 x:0 0 0 2024-09-16T23:48:52.457 INFO:tasks.workunit.client.1.smithi043.stdout:1/491: truncate d4/f62 952762 0 2024-09-16T23:48:52.458 INFO:tasks.workunit.client.1.smithi043.stdout:1/492: chown d4/d9/d12/d2b 0 1 2024-09-16T23:48:52.458 INFO:tasks.workunit.client.0.smithi042.stdout:7/234: write d3/d15/d16/f1c [2292151,100625] 0 2024-09-16T23:48:52.458 INFO:tasks.workunit.client.0.smithi042.stdout:4/246: dread - de/d2e/d37/d3e/f4d zero size 2024-09-16T23:48:52.458 INFO:tasks.workunit.client.0.smithi042.stdout:5/275: write d9/db/f28 [3181636,130638] 0 2024-09-16T23:48:52.459 INFO:tasks.workunit.client.0.smithi042.stdout:4/247: dread - de/d2e/d37/f3b zero size 2024-09-16T23:48:52.461 INFO:tasks.workunit.client.1.smithi043.stdout:5/534: creat d3/d9/d22/d4a/fa8 x:0 0 0 2024-09-16T23:48:52.461 INFO:tasks.workunit.client.0.smithi042.stdout:5/276: creat d9/db/d1b/f56 x:0 0 0 2024-09-16T23:48:52.462 INFO:tasks.workunit.client.1.smithi043.stdout:5/535: chown d3/d9/d22/c46 5072 1 2024-09-16T23:48:52.463 INFO:tasks.workunit.client.1.smithi043.stdout:0/385: mknod d0/d2/df/d1a/d3b/c81 0 2024-09-16T23:48:52.463 INFO:tasks.workunit.client.0.smithi042.stdout:4/248: mknod de/d2e/d37/d3e/c57 0 2024-09-16T23:48:52.465 INFO:tasks.workunit.client.0.smithi042.stdout:4/249: fdatasync de/d2e/d37/d3e/f4d 0 2024-09-16T23:48:52.467 INFO:tasks.workunit.client.0.smithi042.stdout:5/277: rename d9/db/d17/f24 to d9/db/d1b/d31/d46/f57 0 2024-09-16T23:48:52.468 INFO:tasks.workunit.client.1.smithi043.stdout:6/406: symlink de/d1a/d80/l87 0 2024-09-16T23:48:52.471 INFO:tasks.workunit.client.0.smithi042.stdout:5/278: chown d9/db/de/d10/f20 15819873 1 2024-09-16T23:48:52.471 INFO:tasks.workunit.client.0.smithi042.stdout:5/279: write d9/db/d11/f1c [326030,71314] 0 2024-09-16T23:48:52.471 INFO:tasks.workunit.client.1.smithi043.stdout:8/370: rename de/d18/d27/d41/d5a/d37/l5b to de/d18/d27/d41/l6f 0 2024-09-16T23:48:52.472 INFO:tasks.workunit.client.0.smithi042.stdout:5/280: stat d9/db/d1b/d31/l3d 0 2024-09-16T23:48:52.473 INFO:tasks.workunit.client.1.smithi043.stdout:4/363: truncate d8/d15/d20/d2b/d54/d55/f2a 4425200 0 2024-09-16T23:48:52.473 INFO:tasks.workunit.client.1.smithi043.stdout:8/371: chown de/d18/d27/d41/d5a/d37/l66 1616 1 2024-09-16T23:48:52.473 INFO:tasks.workunit.client.0.smithi042.stdout:4/250: creat de/d2e/d37/d4c/d4f/f58 x:0 0 0 2024-09-16T23:48:52.474 INFO:tasks.workunit.client.1.smithi043.stdout:8/372: write de/f15 [1298013,49530] 0 2024-09-16T23:48:52.474 INFO:tasks.workunit.client.1.smithi043.stdout:4/364: readlink d8/d15/d20/d2b/d54/d55/l65 0 2024-09-16T23:48:52.476 INFO:tasks.workunit.client.1.smithi043.stdout:0/386: dread d0/d2/dd/f60 [0,4194304] 0 2024-09-16T23:48:52.478 INFO:tasks.workunit.client.1.smithi043.stdout:7/376: link d4/db/d11/d15/f3d d4/db/d11/d15/d29/d7a/f84 0 2024-09-16T23:48:52.479 INFO:tasks.workunit.client.0.smithi042.stdout:4/251: creat de/d12/d1b/f59 x:0 0 0 2024-09-16T23:48:52.479 INFO:tasks.workunit.client.1.smithi043.stdout:9/346: rename d4/d5/d16/l1e to d4/d1c/l68 0 2024-09-16T23:48:52.481 INFO:tasks.workunit.client.0.smithi042.stdout:7/235: truncate d3/d15/d16/f1e 160189 0 2024-09-16T23:48:52.481 INFO:tasks.workunit.client.1.smithi043.stdout:7/377: truncate d4/d27/d41/f40 898729 0 2024-09-16T23:48:52.483 INFO:tasks.workunit.client.0.smithi042.stdout:7/236: mkdir d3/d1f/d4f 0 2024-09-16T23:48:52.484 INFO:tasks.workunit.client.0.smithi042.stdout:4/252: rmdir de/d2e/d37/d3c 39 2024-09-16T23:48:52.485 INFO:tasks.workunit.client.1.smithi043.stdout:1/493: rename d4/d9/d12/d2b/f6b to d4/df/d13/fa4 0 2024-09-16T23:48:52.486 INFO:tasks.workunit.client.1.smithi043.stdout:1/494: chown d4/d34/f4d 80286568 1 2024-09-16T23:48:52.487 INFO:tasks.workunit.client.0.smithi042.stdout:4/253: readlink de/d1a/l2b 0 2024-09-16T23:48:52.491 INFO:tasks.workunit.client.1.smithi043.stdout:5/536: dread d3/d9/d22/d60/f63 [0,4194304] 0 2024-09-16T23:48:52.491 INFO:tasks.workunit.client.1.smithi043.stdout:0/387: symlink d0/d2/df/d1a/l82 0 2024-09-16T23:48:52.492 INFO:tasks.workunit.client.1.smithi043.stdout:0/388: dread - d0/de/f78 zero size 2024-09-16T23:48:52.492 INFO:tasks.workunit.client.0.smithi042.stdout:7/237: dwrite d3/d1f/f23 [0,4194304] 0 2024-09-16T23:48:52.497 INFO:tasks.workunit.client.0.smithi042.stdout:7/238: stat d3/lb 0 2024-09-16T23:48:52.500 INFO:tasks.workunit.client.0.smithi042.stdout:4/254: dwrite de/d12/d1b/f2d [0,4194304] 0 2024-09-16T23:48:52.501 INFO:tasks.workunit.client.0.smithi042.stdout:4/255: read de/d12/d1b/f2d [756682,10108] 0 2024-09-16T23:48:52.502 INFO:tasks.workunit.client.0.smithi042.stdout:7/239: symlink d3/d1f/l50 0 2024-09-16T23:48:52.507 INFO:tasks.workunit.client.1.smithi043.stdout:0/389: mkdir d0/d2/df/d1a/d46/d39/d83 0 2024-09-16T23:48:52.511 INFO:tasks.workunit.client.1.smithi043.stdout:4/365: dread d8/d15/d20/d2b/d54/d55/f2a [0,4194304] 0 2024-09-16T23:48:52.512 INFO:tasks.workunit.client.0.smithi042.stdout:7/240: mkdir d3/d15/d24/d28/d51 0 2024-09-16T23:48:52.513 INFO:tasks.workunit.client.1.smithi043.stdout:6/407: rename de/d1a/f3b to de/d1a/d4d/f88 0 2024-09-16T23:48:52.514 INFO:tasks.workunit.client.0.smithi042.stdout:4/256: creat de/f5a x:0 0 0 2024-09-16T23:48:52.515 INFO:tasks.workunit.client.1.smithi043.stdout:1/495: link d4/d9/d72/l8d d4/d9/d1c/d1e/d47/d97/la5 0 2024-09-16T23:48:52.516 INFO:tasks.workunit.client.1.smithi043.stdout:5/537: link d3/d9/c2b d3/d9/d6f/d7c/d80/ca9 0 2024-09-16T23:48:52.519 INFO:tasks.workunit.client.0.smithi042.stdout:4/257: rmdir de/d12 39 2024-09-16T23:48:52.519 INFO:tasks.workunit.client.0.smithi042.stdout:4/258: readlink de/d1a/d1c/l2a 0 2024-09-16T23:48:52.520 INFO:tasks.workunit.client.1.smithi043.stdout:4/366: creat d8/d15/f69 x:0 0 0 2024-09-16T23:48:52.520 INFO:tasks.workunit.client.1.smithi043.stdout:4/367: stat d8/c12 0 2024-09-16T23:48:52.521 INFO:tasks.workunit.client.1.smithi043.stdout:6/408: symlink de/d1a/d4d/d50/l89 0 2024-09-16T23:48:52.522 INFO:tasks.workunit.client.0.smithi042.stdout:7/241: truncate d3/f11 3940216 0 2024-09-16T23:48:52.524 INFO:tasks.workunit.client.0.smithi042.stdout:4/259: mknod de/d2e/d37/c5b 0 2024-09-16T23:48:52.524 INFO:tasks.workunit.client.1.smithi043.stdout:1/496: write d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f25 [716893,62133] 0 2024-09-16T23:48:52.524 INFO:tasks.workunit.client.0.smithi042.stdout:7/242: rename d3/d15/d24/d28/d51 to d3/d15/d24/d28/d51/d52 22 2024-09-16T23:48:52.525 INFO:tasks.workunit.client.1.smithi043.stdout:0/390: dwrite d0/dc/f42 [0,4194304] 0 2024-09-16T23:48:52.526 INFO:tasks.workunit.client.0.smithi042.stdout:4/260: read - de/d1a/f55 zero size 2024-09-16T23:48:52.527 INFO:tasks.workunit.client.0.smithi042.stdout:4/261: stat fa 0 2024-09-16T23:48:52.528 INFO:tasks.workunit.client.1.smithi043.stdout:0/391: truncate d0/d2/df/d5a/f63 841935 0 2024-09-16T23:48:52.529 INFO:tasks.workunit.client.1.smithi043.stdout:8/373: rename de/d18/d27/d34 to de/d70 0 2024-09-16T23:48:52.529 INFO:tasks.workunit.client.1.smithi043.stdout:5/538: mknod d3/d9/d22/d4a/d4c/caa 0 2024-09-16T23:48:52.530 INFO:tasks.workunit.client.0.smithi042.stdout:4/262: creat de/d2e/d37/d3e/f5c x:0 0 0 2024-09-16T23:48:52.531 INFO:tasks.workunit.client.0.smithi042.stdout:4/263: dread - de/d1a/d1c/f4a zero size 2024-09-16T23:48:52.533 INFO:tasks.workunit.client.0.smithi042.stdout:7/243: write d3/d15/d16/f1e [707653,26061] 0 2024-09-16T23:48:52.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/264: mkdir de/d42/d5d 0 2024-09-16T23:48:52.537 INFO:tasks.workunit.client.0.smithi042.stdout:7/244: readlink d3/l27 0 2024-09-16T23:48:52.539 INFO:tasks.workunit.client.1.smithi043.stdout:5/539: dwrite d3/d9/d22/d4a/d64/f7b [0,4194304] 0 2024-09-16T23:48:52.541 INFO:tasks.workunit.client.0.smithi042.stdout:4/265: symlink de/d2e/d37/l5e 0 2024-09-16T23:48:52.543 INFO:tasks.workunit.client.0.smithi042.stdout:4/266: truncate de/f31 874337 0 2024-09-16T23:48:52.543 INFO:tasks.workunit.client.0.smithi042.stdout:7/245: mknod d3/d15/d24/c53 0 2024-09-16T23:48:52.545 INFO:tasks.workunit.client.1.smithi043.stdout:7/378: rename d4/db/d11/d21/d36 to d4/db/d43/d83/d85 0 2024-09-16T23:48:52.548 INFO:tasks.workunit.client.0.smithi042.stdout:4/267: creat de/d2e/d37/d4c/f5f x:0 0 0 2024-09-16T23:48:52.552 INFO:tasks.workunit.client.1.smithi043.stdout:4/368: write d8/d15/f21 [5916416,30161] 0 2024-09-16T23:48:52.552 INFO:tasks.workunit.client.1.smithi043.stdout:0/392: dwrite d0/d2/f6d [0,4194304] 0 2024-09-16T23:48:52.553 INFO:tasks.workunit.client.1.smithi043.stdout:1/497: mknod d4/d9/d1c/d1e/d47/d97/ca6 0 2024-09-16T23:48:52.556 INFO:tasks.workunit.client.0.smithi042.stdout:4/268: dwrite de/d12/f15 [0,4194304] 0 2024-09-16T23:48:52.560 INFO:tasks.workunit.client.1.smithi043.stdout:9/347: rename d4/d5/f46 to d4/d21/d2a/d2c/f69 0 2024-09-16T23:48:52.562 INFO:tasks.workunit.client.1.smithi043.stdout:0/393: dwrite d0/d2/df/d1a/f47 [0,4194304] 0 2024-09-16T23:48:52.563 INFO:tasks.workunit.client.0.smithi042.stdout:4/269: mknod de/d1a/d1c/c60 0 2024-09-16T23:48:52.565 INFO:tasks.workunit.client.1.smithi043.stdout:4/369: creat d8/d25/f6a x:0 0 0 2024-09-16T23:48:52.568 INFO:tasks.workunit.client.0.smithi042.stdout:7/246: dwrite d3/fe [0,4194304] 0 2024-09-16T23:48:52.568 INFO:tasks.workunit.client.1.smithi043.stdout:1/498: mknod d4/d9/d1c/d1e/d42/ca7 0 2024-09-16T23:48:52.569 INFO:tasks.workunit.client.0.smithi042.stdout:4/270: creat de/d2e/d37/d3e/f61 x:0 0 0 2024-09-16T23:48:52.570 INFO:tasks.workunit.client.1.smithi043.stdout:8/374: link de/d18/d27/d41/d5a/d37/f60 de/f71 0 2024-09-16T23:48:52.570 INFO:tasks.workunit.client.0.smithi042.stdout:4/271: chown fa 1 1 2024-09-16T23:48:52.576 INFO:tasks.workunit.client.1.smithi043.stdout:6/409: getdents de/d23/d63 0 2024-09-16T23:48:52.580 INFO:tasks.workunit.client.1.smithi043.stdout:7/379: dread d4/db/d11/d15/f47 [0,4194304] 0 2024-09-16T23:48:52.581 INFO:tasks.workunit.client.1.smithi043.stdout:5/540: rename c1 to d3/d79/cab 0 2024-09-16T23:48:52.582 INFO:tasks.workunit.client.1.smithi043.stdout:1/499: creat d4/d9/d12/d2b/d3d/d83/fa8 x:0 0 0 2024-09-16T23:48:52.584 INFO:tasks.workunit.client.0.smithi042.stdout:4/272: dwrite de/f5a [0,4194304] 0 2024-09-16T23:48:52.584 INFO:tasks.workunit.client.0.smithi042.stdout:4/273: readlink de/l22 0 2024-09-16T23:48:52.585 INFO:tasks.workunit.client.1.smithi043.stdout:7/380: read d4/d27/d41/f26 [6274180,88147] 0 2024-09-16T23:48:52.586 INFO:tasks.workunit.client.0.smithi042.stdout:0/185: sync 2024-09-16T23:48:52.586 INFO:tasks.workunit.client.1.smithi043.stdout:8/375: read de/d18/d1f/d25/f5f [74423,10321] 0 2024-09-16T23:48:52.588 INFO:tasks.workunit.client.1.smithi043.stdout:9/348: write d4/d21/d33/f55 [1399457,130319] 0 2024-09-16T23:48:52.590 INFO:tasks.workunit.client.1.smithi043.stdout:9/349: chown d4/d5/d64/f65 15 1 2024-09-16T23:48:52.592 INFO:tasks.workunit.client.0.smithi042.stdout:0/186: creat d0/df/d13/f34 x:0 0 0 2024-09-16T23:48:52.592 INFO:tasks.workunit.client.1.smithi043.stdout:0/394: symlink d0/d2/df/l84 0 2024-09-16T23:48:52.594 INFO:tasks.workunit.client.1.smithi043.stdout:0/395: chown d0/de/f69 25 1 2024-09-16T23:48:52.594 INFO:tasks.workunit.client.1.smithi043.stdout:4/370: creat d8/d15/d20/d2b/f6b x:0 0 0 2024-09-16T23:48:52.594 INFO:tasks.workunit.client.0.smithi042.stdout:4/274: write de/d1a/d1c/f56 [557667,33976] 0 2024-09-16T23:48:52.599 INFO:tasks.workunit.client.1.smithi043.stdout:1/500: rename d4/d9/d1c/d1e/d42/l5e to d4/d9/d1c/d1e/d42/d5d/d7a/la9 0 2024-09-16T23:48:52.600 INFO:tasks.workunit.client.1.smithi043.stdout:7/381: fdatasync d4/db/d28/f2d 0 2024-09-16T23:48:52.600 INFO:tasks.workunit.client.0.smithi042.stdout:7/247: dread d3/f8 [0,4194304] 0 2024-09-16T23:48:52.603 INFO:tasks.workunit.client.1.smithi043.stdout:6/410: mknod de/c8a 0 2024-09-16T23:48:52.604 INFO:tasks.workunit.client.1.smithi043.stdout:4/371: symlink d8/d15/d20/d2b/d22/l6c 0 2024-09-16T23:48:52.606 INFO:tasks.workunit.client.1.smithi043.stdout:4/372: readlink d8/d15/d20/d2b/d54/d55/l1e 0 2024-09-16T23:48:52.606 INFO:tasks.workunit.client.0.smithi042.stdout:4/275: truncate de/f5a 2346066 0 2024-09-16T23:48:52.607 INFO:tasks.workunit.client.1.smithi043.stdout:5/541: mkdir d3/d9/d22/d4a/dac 0 2024-09-16T23:48:52.608 INFO:tasks.workunit.client.0.smithi042.stdout:4/276: fdatasync de/d2e/d37/d4c/d4f/f58 0 2024-09-16T23:48:52.608 INFO:tasks.workunit.client.0.smithi042.stdout:7/248: link d3/d1f/f26 d3/d15/d24/d28/f54 0 2024-09-16T23:48:52.609 INFO:tasks.workunit.client.0.smithi042.stdout:7/249: chown d3/d15/d32 245 1 2024-09-16T23:48:52.611 INFO:tasks.workunit.client.0.smithi042.stdout:7/250: write d3/d15/d41/f47 [730480,92172] 0 2024-09-16T23:48:52.611 INFO:tasks.workunit.client.0.smithi042.stdout:4/277: fsync de/f2f 0 2024-09-16T23:48:52.611 INFO:tasks.workunit.client.1.smithi043.stdout:1/501: rmdir d4/d9/d72 39 2024-09-16T23:48:52.613 INFO:tasks.workunit.client.0.smithi042.stdout:4/278: dread - de/d2e/d37/d4c/d4f/f58 zero size 2024-09-16T23:48:52.614 INFO:tasks.workunit.client.0.smithi042.stdout:7/251: mkdir d3/d15/d24/d4d/d55 0 2024-09-16T23:48:52.615 INFO:tasks.workunit.client.0.smithi042.stdout:7/252: fdatasync d3/f18 0 2024-09-16T23:48:52.615 INFO:tasks.workunit.client.1.smithi043.stdout:9/350: dread d4/d5/d16/f1f [0,4194304] 0 2024-09-16T23:48:52.617 INFO:tasks.workunit.client.0.smithi042.stdout:4/279: mknod de/d2e/d37/d3c/c62 0 2024-09-16T23:48:52.619 INFO:tasks.workunit.client.1.smithi043.stdout:8/376: mknod de/d18/d27/d41/d5a/d37/d6a/c72 0 2024-09-16T23:48:52.620 INFO:tasks.workunit.client.0.smithi042.stdout:4/280: mknod de/d12/d1b/c63 0 2024-09-16T23:48:52.620 INFO:tasks.workunit.client.1.smithi043.stdout:8/377: chown de/d18/l4e 979780624 1 2024-09-16T23:48:52.621 INFO:tasks.workunit.client.1.smithi043.stdout:8/378: write de/f17 [892915,75570] 0 2024-09-16T23:48:52.621 INFO:tasks.workunit.client.0.smithi042.stdout:4/281: fdatasync de/d12/f48 0 2024-09-16T23:48:52.622 INFO:tasks.workunit.client.1.smithi043.stdout:5/542: read d3/db/f52 [3587880,107430] 0 2024-09-16T23:48:52.624 INFO:tasks.workunit.client.0.smithi042.stdout:7/253: link d3/d15/d16/f25 d3/d1f/d49/f56 0 2024-09-16T23:48:52.625 INFO:tasks.workunit.client.0.smithi042.stdout:7/254: chown d3/d15/d24/c53 39 1 2024-09-16T23:48:52.625 INFO:tasks.workunit.client.1.smithi043.stdout:0/396: rename d0/d2/df/d1a/d3b/l1d to d0/d2/df/d1a/d46/l85 0 2024-09-16T23:48:52.627 INFO:tasks.workunit.client.1.smithi043.stdout:1/502: creat d4/d9/d12/faa x:0 0 0 2024-09-16T23:48:52.627 INFO:tasks.workunit.client.1.smithi043.stdout:0/397: write d0/d2/df/d1a/d49/d71/f7e [392671,106116] 0 2024-09-16T23:48:52.629 INFO:tasks.workunit.client.0.smithi042.stdout:7/255: unlink d3/d1f/c3c 0 2024-09-16T23:48:52.630 INFO:tasks.workunit.client.1.smithi043.stdout:6/411: mkdir de/d1a/d4d/d8b 0 2024-09-16T23:48:52.630 INFO:tasks.workunit.client.1.smithi043.stdout:2/327: sync 2024-09-16T23:48:52.630 INFO:tasks.workunit.client.1.smithi043.stdout:3/396: sync 2024-09-16T23:48:52.631 INFO:tasks.workunit.client.1.smithi043.stdout:4/373: mknod d8/d25/d64/c6d 0 2024-09-16T23:48:52.631 INFO:tasks.workunit.client.1.smithi043.stdout:2/328: chown d8/d35/f36 42 1 2024-09-16T23:48:52.631 INFO:tasks.workunit.client.1.smithi043.stdout:6/412: dread - de/d1a/d28/f77 zero size 2024-09-16T23:48:52.633 INFO:tasks.workunit.client.1.smithi043.stdout:8/379: creat de/d18/d27/d41/d5a/d37/d6a/f73 x:0 0 0 2024-09-16T23:48:52.635 INFO:tasks.workunit.client.0.smithi042.stdout:4/282: truncate de/d1a/f33 1724966 0 2024-09-16T23:48:52.635 INFO:tasks.workunit.client.1.smithi043.stdout:5/543: mknod d3/d9/d6f/d85/cad 0 2024-09-16T23:48:52.638 INFO:tasks.workunit.client.1.smithi043.stdout:9/351: creat d4/d1c/d4f/d45/f6a x:0 0 0 2024-09-16T23:48:52.638 INFO:tasks.workunit.client.0.smithi042.stdout:7/256: rmdir d3/d15/d24/d28/d51 0 2024-09-16T23:48:52.639 INFO:tasks.workunit.client.0.smithi042.stdout:4/283: creat de/d2e/d37/d4c/d4f/f64 x:0 0 0 2024-09-16T23:48:52.639 INFO:tasks.workunit.client.1.smithi043.stdout:4/374: dwrite d8/d15/f21 [0,4194304] 0 2024-09-16T23:48:52.640 INFO:tasks.workunit.client.1.smithi043.stdout:3/397: symlink d4/d30/d53/l76 0 2024-09-16T23:48:52.640 INFO:tasks.workunit.client.0.smithi042.stdout:4/284: dread - de/d42/f44 zero size 2024-09-16T23:48:52.641 INFO:tasks.workunit.client.0.smithi042.stdout:4/285: write de/d1a/f39 [822625,85783] 0 2024-09-16T23:48:52.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/257: dread d3/fa [0,4194304] 0 2024-09-16T23:48:52.648 INFO:tasks.workunit.client.0.smithi042.stdout:4/286: creat de/d2e/d37/f65 x:0 0 0 2024-09-16T23:48:52.649 INFO:tasks.workunit.client.0.smithi042.stdout:4/287: chown de/d1a/l2b 28450305 1 2024-09-16T23:48:52.649 INFO:tasks.workunit.client.1.smithi043.stdout:7/382: dwrite d4/db/d43/d83/d85/f57 [0,4194304] 0 2024-09-16T23:48:52.649 INFO:tasks.workunit.client.1.smithi043.stdout:8/380: fdatasync de/d70/d3d/f62 0 2024-09-16T23:48:52.650 INFO:tasks.workunit.client.0.smithi042.stdout:7/258: creat d3/f57 x:0 0 0 2024-09-16T23:48:52.651 INFO:tasks.workunit.client.1.smithi043.stdout:5/544: rename d3/db/d15/c41 to d3/d9/d22/d4a/d64/cae 0 2024-09-16T23:48:52.652 INFO:tasks.workunit.client.0.smithi042.stdout:4/288: mknod de/d42/c66 0 2024-09-16T23:48:52.653 INFO:tasks.workunit.client.0.smithi042.stdout:4/289: dread - de/d2e/d37/d3e/f4d zero size 2024-09-16T23:48:52.655 INFO:tasks.workunit.client.1.smithi043.stdout:3/398: truncate d4/de/d1f/f3e 1910085 0 2024-09-16T23:48:52.659 INFO:tasks.workunit.client.1.smithi043.stdout:9/352: read d4/f10 [4501711,101096] 0 2024-09-16T23:48:52.659 INFO:tasks.workunit.client.1.smithi043.stdout:7/383: dread d4/db/d43/d83/d85/f57 [0,4194304] 0 2024-09-16T23:48:52.660 INFO:tasks.workunit.client.0.smithi042.stdout:4/290: mknod de/d12/d1b/c67 0 2024-09-16T23:48:52.663 INFO:tasks.workunit.client.1.smithi043.stdout:2/329: dwrite d8/d11/f4a [0,4194304] 0 2024-09-16T23:48:52.665 INFO:tasks.workunit.client.0.smithi042.stdout:7/259: dwrite d3/fe [0,4194304] 0 2024-09-16T23:48:52.667 INFO:tasks.workunit.client.0.smithi042.stdout:4/291: dread de/d1a/d1c/f21 [0,4194304] 0 2024-09-16T23:48:52.667 INFO:tasks.workunit.client.0.smithi042.stdout:4/292: stat de/d1a/d1c/l3d 0 2024-09-16T23:48:52.668 INFO:tasks.workunit.client.1.smithi043.stdout:9/353: read d4/f48 [187251,112733] 0 2024-09-16T23:48:52.668 INFO:tasks.workunit.client.0.smithi042.stdout:4/293: stat de/f2f 0 2024-09-16T23:48:52.669 INFO:tasks.workunit.client.1.smithi043.stdout:2/330: write d8/d4b/f5b [1033855,58615] 0 2024-09-16T23:48:52.669 INFO:tasks.workunit.client.0.smithi042.stdout:4/294: chown de/d12/l3a 615 1 2024-09-16T23:48:52.672 INFO:tasks.workunit.client.1.smithi043.stdout:3/399: write d4/de/d1f/d34/f74 [5094464,39447] 0 2024-09-16T23:48:52.674 INFO:tasks.workunit.client.0.smithi042.stdout:7/260: mknod d3/d15/d24/d28/c58 0 2024-09-16T23:48:52.675 INFO:tasks.workunit.client.1.smithi043.stdout:1/503: truncate d4/d9/d1c/d1e/f66 3831341 0 2024-09-16T23:48:52.676 INFO:tasks.workunit.client.1.smithi043.stdout:7/384: dwrite d4/db/f73 [0,4194304] 0 2024-09-16T23:48:52.676 INFO:tasks.workunit.client.0.smithi042.stdout:4/295: dread de/d1a/d1c/f56 [0,4194304] 0 2024-09-16T23:48:52.678 INFO:tasks.workunit.client.0.smithi042.stdout:4/296: write de/d12/f14 [1740295,126216] 0 2024-09-16T23:48:52.680 INFO:tasks.workunit.client.1.smithi043.stdout:6/413: getdents de/d1a/d80 0 2024-09-16T23:48:52.685 INFO:tasks.workunit.client.0.smithi042.stdout:4/297: mkdir de/d42/d68 0 2024-09-16T23:48:52.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/261: creat d3/d15/d24/f59 x:0 0 0 2024-09-16T23:48:52.688 INFO:tasks.workunit.client.0.smithi042.stdout:7/262: chown d3/d15/c4e 25638558 1 2024-09-16T23:48:52.690 INFO:tasks.workunit.client.1.smithi043.stdout:5/545: mkdir d3/d79/d91/daf 0 2024-09-16T23:48:52.691 INFO:tasks.workunit.client.1.smithi043.stdout:4/375: rename d8/d3d to d8/d15/d6e 0 2024-09-16T23:48:52.692 INFO:tasks.workunit.client.1.smithi043.stdout:2/331: dwrite d8/d11/d16/f19 [0,4194304] 0 2024-09-16T23:48:52.696 INFO:tasks.workunit.client.0.smithi042.stdout:4/298: creat de/d42/d68/f69 x:0 0 0 2024-09-16T23:48:52.697 INFO:tasks.workunit.client.1.smithi043.stdout:2/332: chown d8/d1f/f5c 5214795 1 2024-09-16T23:48:52.700 INFO:tasks.workunit.client.0.smithi042.stdout:7/263: link d3/d15/d16/d19/l46 d3/d15/d24/d4d/l5a 0 2024-09-16T23:48:52.701 INFO:tasks.workunit.client.1.smithi043.stdout:6/414: dread - de/d1a/d28/f5f zero size 2024-09-16T23:48:52.704 INFO:tasks.workunit.client.0.smithi042.stdout:7/264: mkdir d3/d15/d24/d4d/d55/d5b 0 2024-09-16T23:48:52.704 INFO:tasks.workunit.client.1.smithi043.stdout:9/354: rename d4/d5/d30 to d4/d1c/d6b 0 2024-09-16T23:48:52.707 INFO:tasks.workunit.client.1.smithi043.stdout:5/546: unlink d3/db/d19/l68 0 2024-09-16T23:48:52.710 INFO:tasks.workunit.client.1.smithi043.stdout:2/333: dwrite d8/fd [4194304,4194304] 0 2024-09-16T23:48:52.710 INFO:tasks.workunit.client.1.smithi043.stdout:3/400: dread d4/de/d1f/f29 [0,4194304] 0 2024-09-16T23:48:52.712 INFO:tasks.workunit.client.1.smithi043.stdout:7/385: rename d4/f45 to d4/db/d11/d15/d29/d7a/f86 0 2024-09-16T23:48:52.713 INFO:tasks.workunit.client.1.smithi043.stdout:6/415: symlink de/d1a/d4d/d5a/l8c 0 2024-09-16T23:48:52.715 INFO:tasks.workunit.client.1.smithi043.stdout:5/547: chown d3/d9/d22/d4a/d64/d8f/da1 39617 1 2024-09-16T23:48:52.715 INFO:tasks.workunit.client.1.smithi043.stdout:4/376: mknod d8/d15/d20/d2b/d23/d27/c6f 0 2024-09-16T23:48:52.719 INFO:tasks.workunit.client.0.smithi042.stdout:7/265: rename d3/d15/d24/d4d/d55 to d3/d5c 0 2024-09-16T23:48:52.720 INFO:tasks.workunit.client.1.smithi043.stdout:4/377: chown d8/d15/d20/d2b/d54/d55/d49/d4c 1940863 1 2024-09-16T23:48:52.723 INFO:tasks.workunit.client.1.smithi043.stdout:2/334: dread d8/d4b/d5f/f61 [0,4194304] 0 2024-09-16T23:48:52.724 INFO:tasks.workunit.client.1.smithi043.stdout:9/355: dwrite d4/d21/d33/f4e [0,4194304] 0 2024-09-16T23:48:52.726 INFO:tasks.workunit.client.1.smithi043.stdout:1/504: dread d4/d9/d12/d15/f31 [0,4194304] 0 2024-09-16T23:48:52.727 INFO:tasks.workunit.client.0.smithi042.stdout:7/266: dwrite d3/fe [4194304,4194304] 0 2024-09-16T23:48:52.733 INFO:tasks.workunit.client.0.smithi042.stdout:7/267: readlink d3/l13 0 2024-09-16T23:48:52.736 INFO:tasks.workunit.client.1.smithi043.stdout:9/356: dread d4/d1c/d25/f62 [0,4194304] 0 2024-09-16T23:48:52.739 INFO:tasks.workunit.client.0.smithi042.stdout:7/268: creat d3/d1f/d4f/f5d x:0 0 0 2024-09-16T23:48:52.740 INFO:tasks.workunit.client.1.smithi043.stdout:3/401: dread d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:48:52.741 INFO:tasks.workunit.client.1.smithi043.stdout:3/402: fsync d4/de/f55 0 2024-09-16T23:48:52.743 INFO:tasks.workunit.client.0.smithi042.stdout:7/269: getdents d3/d3f 0 2024-09-16T23:48:52.744 INFO:tasks.workunit.client.1.smithi043.stdout:2/335: dread d8/d1d/f2a [0,4194304] 0 2024-09-16T23:48:52.748 INFO:tasks.workunit.client.0.smithi042.stdout:7/270: dread d3/d1f/f23 [0,4194304] 0 2024-09-16T23:48:52.750 INFO:tasks.workunit.client.0.smithi042.stdout:7/271: creat d3/d3f/f5e x:0 0 0 2024-09-16T23:48:52.760 INFO:tasks.workunit.client.0.smithi042.stdout:7/272: dwrite d3/f11 [0,4194304] 0 2024-09-16T23:48:52.774 INFO:tasks.workunit.client.0.smithi042.stdout:1/235: sync 2024-09-16T23:48:52.780 INFO:tasks.workunit.client.0.smithi042.stdout:7/273: dread d3/f1a [0,4194304] 0 2024-09-16T23:48:52.788 INFO:tasks.workunit.client.0.smithi042.stdout:7/274: rename d3/d15/f48 to d3/f5f 0 2024-09-16T23:48:52.789 INFO:tasks.workunit.client.0.smithi042.stdout:7/275: write d3/d15/d24/f2d [1328441,123315] 0 2024-09-16T23:48:52.791 INFO:tasks.workunit.client.0.smithi042.stdout:7/276: fdatasync d3/d15/d24/f59 0 2024-09-16T23:48:52.792 INFO:tasks.workunit.client.0.smithi042.stdout:7/277: write d3/f11 [94509,40302] 0 2024-09-16T23:48:52.795 INFO:tasks.workunit.client.0.smithi042.stdout:1/236: dread d0/d9/d35/d46/d32/d2f/f34 [0,4194304] 0 2024-09-16T23:48:52.795 INFO:tasks.workunit.client.0.smithi042.stdout:7/278: unlink d3/d15/d16/c2c 0 2024-09-16T23:48:52.800 INFO:tasks.workunit.client.0.smithi042.stdout:7/279: rename d3/f57 to d3/d5c/f60 0 2024-09-16T23:48:52.801 INFO:tasks.workunit.client.0.smithi042.stdout:7/280: fsync d3/d15/d41/f47 0 2024-09-16T23:48:52.803 INFO:tasks.workunit.client.0.smithi042.stdout:7/281: creat d3/d1f/f61 x:0 0 0 2024-09-16T23:48:52.804 INFO:tasks.workunit.client.1.smithi043.stdout:6/416: write de/d1a/f41 [1302383,88936] 0 2024-09-16T23:48:52.804 INFO:tasks.workunit.client.1.smithi043.stdout:5/548: write d3/d9/d22/d60/f63 [4424860,48708] 0 2024-09-16T23:48:52.805 INFO:tasks.workunit.client.0.smithi042.stdout:7/282: write d3/d15/d16/d19/f21 [286534,43939] 0 2024-09-16T23:48:52.807 INFO:tasks.workunit.client.1.smithi043.stdout:9/357: rmdir d4/d1c/d25/d2b/d4c/d50 39 2024-09-16T23:48:52.810 INFO:tasks.workunit.client.1.smithi043.stdout:7/386: truncate d4/d27/d41/f40 236465 0 2024-09-16T23:48:52.810 INFO:tasks.workunit.client.1.smithi043.stdout:9/358: read d4/d1c/d4f/f52 [4429221,1697] 0 2024-09-16T23:48:52.810 INFO:tasks.workunit.client.1.smithi043.stdout:4/378: truncate d8/d15/d20/d2b/d22/f36 3756203 0 2024-09-16T23:48:52.810 INFO:tasks.workunit.client.1.smithi043.stdout:9/359: write d4/d1c/f27 [4647210,62505] 0 2024-09-16T23:48:52.812 INFO:tasks.workunit.client.0.smithi042.stdout:1/237: mknod d0/d17/c52 0 2024-09-16T23:48:52.812 INFO:tasks.workunit.client.1.smithi043.stdout:7/387: chown d4/db/d11/d21/d31/d33/f5b 6 1 2024-09-16T23:48:52.813 INFO:tasks.workunit.client.0.smithi042.stdout:7/283: dwrite d3/d1f/f3d [0,4194304] 0 2024-09-16T23:48:52.816 INFO:tasks.workunit.client.0.smithi042.stdout:1/238: truncate d0/d9/d35/d46/d32/f33 285396 0 2024-09-16T23:48:52.818 INFO:tasks.workunit.client.1.smithi043.stdout:2/336: truncate d8/d4b/d5f/f61 722558 0 2024-09-16T23:48:52.818 INFO:tasks.workunit.client.1.smithi043.stdout:3/403: unlink d4/de/l5a 0 2024-09-16T23:48:52.818 INFO:tasks.workunit.client.0.smithi042.stdout:7/284: creat d3/d15/d41/f62 x:0 0 0 2024-09-16T23:48:52.821 INFO:tasks.workunit.client.0.smithi042.stdout:7/285: write d3/f1a [107009,20312] 0 2024-09-16T23:48:52.830 INFO:tasks.workunit.client.0.smithi042.stdout:7/286: dwrite d3/d15/f1b [0,4194304] 0 2024-09-16T23:48:52.831 INFO:tasks.workunit.client.0.smithi042.stdout:1/239: rmdir d0/d9 39 2024-09-16T23:48:52.834 INFO:tasks.workunit.client.1.smithi043.stdout:4/379: mkdir d8/d25/d39/d58/d70 0 2024-09-16T23:48:52.834 INFO:tasks.workunit.client.0.smithi042.stdout:7/287: mkdir d3/d15/d16/d19/d63 0 2024-09-16T23:48:52.836 INFO:tasks.workunit.client.1.smithi043.stdout:9/360: chown d4/d5/l38 389648 1 2024-09-16T23:48:52.836 INFO:tasks.workunit.client.0.smithi042.stdout:7/288: chown d3/c4 207754 1 2024-09-16T23:48:52.838 INFO:tasks.workunit.client.1.smithi043.stdout:1/505: dwrite d4/d9/d12/f82 [0,4194304] 0 2024-09-16T23:48:52.841 INFO:tasks.workunit.client.1.smithi043.stdout:7/388: write d4/f62 [2121227,129763] 0 2024-09-16T23:48:52.843 INFO:tasks.workunit.client.1.smithi043.stdout:3/404: symlink d4/de/d1f/d34/l77 0 2024-09-16T23:48:52.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/240: mknod d0/d9/d35/c53 0 2024-09-16T23:48:52.847 INFO:tasks.workunit.client.1.smithi043.stdout:2/337: rename d8/d11/f4a to d8/d1d/d20/d23/f79 0 2024-09-16T23:48:52.848 INFO:tasks.workunit.client.0.smithi042.stdout:7/289: write d3/f30 [1350813,74760] 0 2024-09-16T23:48:52.850 INFO:tasks.workunit.client.1.smithi043.stdout:2/338: write d8/d1d/d20/d23/d38/f47 [1797656,123556] 0 2024-09-16T23:48:52.859 INFO:tasks.workunit.client.0.smithi042.stdout:1/241: unlink d0/d9/d35/d46/c10 0 2024-09-16T23:48:52.859 INFO:tasks.workunit.client.1.smithi043.stdout:9/361: truncate d4/d1c/d4f/f5b 990451 0 2024-09-16T23:48:52.859 INFO:tasks.workunit.client.1.smithi043.stdout:1/506: creat d4/d9/d1c/d1e/d47/fab x:0 0 0 2024-09-16T23:48:52.860 INFO:tasks.workunit.client.0.smithi042.stdout:7/290: truncate d3/d1f/f23 3426258 0 2024-09-16T23:48:52.861 INFO:tasks.workunit.client.0.smithi042.stdout:1/242: mknod d0/d9/d35/d46/d32/c54 0 2024-09-16T23:48:52.861 INFO:tasks.workunit.client.1.smithi043.stdout:1/507: read d4/d9/d12/f5c [3033954,56860] 0 2024-09-16T23:48:52.862 INFO:tasks.workunit.client.0.smithi042.stdout:7/291: mkdir d3/d5c/d64 0 2024-09-16T23:48:52.863 INFO:tasks.workunit.client.0.smithi042.stdout:7/292: fdatasync d3/d15/d24/d28/f3e 0 2024-09-16T23:48:52.866 INFO:tasks.workunit.client.1.smithi043.stdout:3/405: dwrite d4/de/d1f/d34/f3a [4194304,4194304] 0 2024-09-16T23:48:52.867 INFO:tasks.workunit.client.1.smithi043.stdout:5/549: write d3/db/d19/d3e/f45 [2731434,61596] 0 2024-09-16T23:48:52.869 INFO:tasks.workunit.client.1.smithi043.stdout:6/417: truncate de/d23/d34/d66/f6e 2881434 0 2024-09-16T23:48:52.870 INFO:tasks.workunit.client.0.smithi042.stdout:1/243: symlink d0/l55 0 2024-09-16T23:48:52.870 INFO:tasks.workunit.client.0.smithi042.stdout:7/293: mknod d3/d5c/d64/c65 0 2024-09-16T23:48:52.870 INFO:tasks.workunit.client.1.smithi043.stdout:3/406: write d4/de/d1f/d34/f3a [694552,93383] 0 2024-09-16T23:48:52.872 INFO:tasks.workunit.client.0.smithi042.stdout:7/294: chown d3/d3f/f5e 162309 1 2024-09-16T23:48:52.875 INFO:tasks.workunit.client.1.smithi043.stdout:2/339: mknod d8/d1d/d20/d26/c7a 0 2024-09-16T23:48:52.876 INFO:tasks.workunit.client.0.smithi042.stdout:7/295: stat d3/cd 0 2024-09-16T23:48:52.878 INFO:tasks.workunit.client.1.smithi043.stdout:5/550: dwrite d3/f50 [0,4194304] 0 2024-09-16T23:48:52.882 INFO:tasks.workunit.client.1.smithi043.stdout:7/389: getdents d4/db/d11/d21/d31/d65 0 2024-09-16T23:48:52.884 INFO:tasks.workunit.client.1.smithi043.stdout:1/508: mkdir d4/d9/d12/d15/dac 0 2024-09-16T23:48:52.885 INFO:tasks.workunit.client.0.smithi042.stdout:7/296: dwrite d3/d1f/f61 [0,4194304] 0 2024-09-16T23:48:52.885 INFO:tasks.workunit.client.1.smithi043.stdout:1/509: read - d4/f94 zero size 2024-09-16T23:48:52.890 INFO:tasks.workunit.client.0.smithi042.stdout:7/297: mkdir d3/d15/d24/d28/d66 0 2024-09-16T23:48:52.891 INFO:tasks.workunit.client.1.smithi043.stdout:3/407: truncate d4/de/d1f/d2c/f32 1088247 0 2024-09-16T23:48:52.892 INFO:tasks.workunit.client.0.smithi042.stdout:7/298: write d3/d15/d41/f47 [743033,123333] 0 2024-09-16T23:48:52.892 INFO:tasks.workunit.client.1.smithi043.stdout:5/551: dread d3/db/d19/d3e/d28/f42 [4194304,4194304] 0 2024-09-16T23:48:52.893 INFO:tasks.workunit.client.1.smithi043.stdout:3/408: fdatasync d4/de/d1f/d34/f74 0 2024-09-16T23:48:52.894 INFO:tasks.workunit.client.1.smithi043.stdout:5/552: stat d3/db/d15/f3c 0 2024-09-16T23:48:52.898 INFO:tasks.workunit.client.0.smithi042.stdout:7/299: mknod d3/c67 0 2024-09-16T23:48:52.899 INFO:tasks.workunit.client.1.smithi043.stdout:0/398: sync 2024-09-16T23:48:52.899 INFO:tasks.workunit.client.1.smithi043.stdout:7/390: creat d4/db/d50/f87 x:0 0 0 2024-09-16T23:48:52.902 INFO:tasks.workunit.client.1.smithi043.stdout:4/380: rename d8/d15/d20/l45 to d8/d15/d20/d2b/d54/l71 0 2024-09-16T23:48:52.903 INFO:tasks.workunit.client.1.smithi043.stdout:4/381: stat d8/d15/d20/d2b/d22/f24 0 2024-09-16T23:48:52.906 INFO:tasks.workunit.client.0.smithi042.stdout:7/300: link d3/c67 d3/d15/d24/c68 0 2024-09-16T23:48:52.906 INFO:tasks.workunit.client.0.smithi042.stdout:7/301: fsync d3/f30 0 2024-09-16T23:48:52.907 INFO:tasks.workunit.client.1.smithi043.stdout:3/409: creat d4/de/d40/d4d/f78 x:0 0 0 2024-09-16T23:48:52.907 INFO:tasks.workunit.client.1.smithi043.stdout:7/391: dread d4/db/d11/d15/d29/d7a/f84 [0,4194304] 0 2024-09-16T23:48:52.909 INFO:tasks.workunit.client.0.smithi042.stdout:7/302: write d3/fc [1280043,112696] 0 2024-09-16T23:48:52.910 INFO:tasks.workunit.client.0.smithi042.stdout:7/303: stat d3/d15/d16/f1e 0 2024-09-16T23:48:52.911 INFO:tasks.workunit.client.0.smithi042.stdout:1/244: getdents d0/d15 0 2024-09-16T23:48:52.912 INFO:tasks.workunit.client.0.smithi042.stdout:7/304: chown d3/d1f/l2a 1407508 1 2024-09-16T23:48:52.915 INFO:tasks.workunit.client.1.smithi043.stdout:6/418: link de/d1a/d28/f5f de/d23/d34/d66/f8d 0 2024-09-16T23:48:52.915 INFO:tasks.workunit.client.0.smithi042.stdout:7/305: unlink d3/d15/d24/d28/f54 0 2024-09-16T23:48:52.919 INFO:tasks.workunit.client.0.smithi042.stdout:1/245: dwrite d0/f1d [0,4194304] 0 2024-09-16T23:48:52.920 INFO:tasks.workunit.client.0.smithi042.stdout:1/246: readlink d0/d9/d35/d43/l51 0 2024-09-16T23:48:52.921 INFO:tasks.workunit.client.0.smithi042.stdout:1/247: fdatasync d0/d17/f21 0 2024-09-16T23:48:52.921 INFO:tasks.workunit.client.1.smithi043.stdout:3/410: dread d4/f23 [0,4194304] 0 2024-09-16T23:48:52.923 INFO:tasks.workunit.client.1.smithi043.stdout:9/362: write d4/d1c/d6b/f39 [1479667,1340] 0 2024-09-16T23:48:52.926 INFO:tasks.workunit.client.1.smithi043.stdout:3/411: dread d4/de/d1f/d34/f3a [4194304,4194304] 0 2024-09-16T23:48:52.927 INFO:tasks.workunit.client.1.smithi043.stdout:3/412: stat d4/de/d40/d4d/c65 0 2024-09-16T23:48:52.927 INFO:tasks.workunit.client.1.smithi043.stdout:5/553: symlink d3/da4/lb0 0 2024-09-16T23:48:52.928 INFO:tasks.workunit.client.0.smithi042.stdout:7/306: truncate d3/d15/d24/f2d 270457 0 2024-09-16T23:48:52.929 INFO:tasks.workunit.client.0.smithi042.stdout:1/248: fsync d0/d9/d35/d46/f13 0 2024-09-16T23:48:52.929 INFO:tasks.workunit.client.0.smithi042.stdout:7/307: stat d3/d15/d24/d28/c29 0 2024-09-16T23:48:52.930 INFO:tasks.workunit.client.1.smithi043.stdout:4/382: symlink d8/d15/d20/d2b/d54/d55/d49/d4c/l72 0 2024-09-16T23:48:52.932 INFO:tasks.workunit.client.0.smithi042.stdout:1/249: write d0/d9/d35/d46/f13 [4608418,129347] 0 2024-09-16T23:48:52.932 INFO:tasks.workunit.client.0.smithi042.stdout:1/250: fdatasync d0/d9/d35/f3a 0 2024-09-16T23:48:52.933 INFO:tasks.workunit.client.1.smithi043.stdout:4/383: write d8/d25/f6a [991542,24533] 0 2024-09-16T23:48:52.934 INFO:tasks.workunit.client.0.smithi042.stdout:1/251: write d0/d9/d35/d46/f13 [4593140,66778] 0 2024-09-16T23:48:52.935 INFO:tasks.workunit.client.0.smithi042.stdout:7/308: creat d3/d15/d16/d19/d63/f69 x:0 0 0 2024-09-16T23:48:52.936 INFO:tasks.workunit.client.1.smithi043.stdout:7/392: fdatasync d4/db/d11/d15/f3d 0 2024-09-16T23:48:52.936 INFO:tasks.workunit.client.1.smithi043.stdout:5/554: dwrite d3/f9b [0,4194304] 0 2024-09-16T23:48:52.938 INFO:tasks.workunit.client.1.smithi043.stdout:7/393: write d4/db/d11/d21/d31/d33/f5b [3477525,60793] 0 2024-09-16T23:48:52.944 INFO:tasks.workunit.client.0.smithi042.stdout:7/309: rename d3/f18 to d3/d15/d24/f6a 0 2024-09-16T23:48:52.944 INFO:tasks.workunit.client.1.smithi043.stdout:0/399: symlink d0/d2/df/d1a/d49/d77/l86 0 2024-09-16T23:48:52.946 INFO:tasks.workunit.client.1.smithi043.stdout:7/394: dwrite d4/d27/d41/d1b/f4e [0,4194304] 0 2024-09-16T23:48:52.948 INFO:tasks.workunit.client.0.smithi042.stdout:1/252: unlink d0/d9/d35/d46/d32/d2f/f34 0 2024-09-16T23:48:52.949 INFO:tasks.workunit.client.1.smithi043.stdout:9/363: chown d4/d21/d2a/d2c/l58 0 1 2024-09-16T23:48:52.950 INFO:tasks.workunit.client.0.smithi042.stdout:1/253: read d0/d17/f36 [19382,100304] 0 2024-09-16T23:48:52.952 INFO:tasks.workunit.client.1.smithi043.stdout:7/395: dwrite d4/db/d11/f13 [4194304,4194304] 0 2024-09-16T23:48:52.954 INFO:tasks.workunit.client.0.smithi042.stdout:7/310: dread d3/fc [0,4194304] 0 2024-09-16T23:48:52.957 INFO:tasks.workunit.client.0.smithi042.stdout:9/162: sync 2024-09-16T23:48:52.960 INFO:tasks.workunit.client.1.smithi043.stdout:4/384: mkdir d8/d15/d20/d2b/d54/d55/d73 0 2024-09-16T23:48:52.961 INFO:tasks.workunit.client.1.smithi043.stdout:8/381: sync 2024-09-16T23:48:52.963 INFO:tasks.workunit.client.1.smithi043.stdout:5/555: fdatasync d3/db/d15/f3c 0 2024-09-16T23:48:52.963 INFO:tasks.workunit.client.1.smithi043.stdout:6/419: write de/d1a/f42 [1606422,1116] 0 2024-09-16T23:48:52.966 INFO:tasks.workunit.client.0.smithi042.stdout:1/254: rename d0/f5 to d0/d9/f56 0 2024-09-16T23:48:52.966 INFO:tasks.workunit.client.1.smithi043.stdout:2/340: link d8/c2d d8/d11/c7b 0 2024-09-16T23:48:52.967 INFO:tasks.workunit.client.1.smithi043.stdout:4/385: dwrite d8/d25/f5b [0,4194304] 0 2024-09-16T23:48:52.968 INFO:tasks.workunit.client.1.smithi043.stdout:2/341: readlink d8/d1d/d20/d23/d38/l4e 0 2024-09-16T23:48:52.968 INFO:tasks.workunit.client.0.smithi042.stdout:9/163: creat d4/d10/d24/d25/d28/f34 x:0 0 0 2024-09-16T23:48:52.969 INFO:tasks.workunit.client.1.smithi043.stdout:2/342: fdatasync d8/d11/f32 0 2024-09-16T23:48:52.969 INFO:tasks.workunit.client.0.smithi042.stdout:9/164: dread - d4/d12/d2d/f32 zero size 2024-09-16T23:48:52.969 INFO:tasks.workunit.client.0.smithi042.stdout:7/311: rename d3/d5c to d3/d15/d32/d6b 0 2024-09-16T23:48:52.977 INFO:tasks.workunit.client.1.smithi043.stdout:4/386: dwrite d8/d25/f4f [0,4194304] 0 2024-09-16T23:48:52.980 INFO:tasks.workunit.client.0.smithi042.stdout:7/312: dread d3/d15/d16/f25 [0,4194304] 0 2024-09-16T23:48:52.981 INFO:tasks.workunit.client.0.smithi042.stdout:9/165: unlink d4/le 0 2024-09-16T23:48:52.982 INFO:tasks.workunit.client.0.smithi042.stdout:9/166: write d4/f19 [4586830,17974] 0 2024-09-16T23:48:52.983 INFO:tasks.workunit.client.0.smithi042.stdout:9/167: chown d4/d12/d16/f17 116 1 2024-09-16T23:48:52.986 INFO:tasks.workunit.client.1.smithi043.stdout:1/510: link d4/d9/d12/d2b/d3d/d83/l92 d4/d34/lad 0 2024-09-16T23:48:52.988 INFO:tasks.workunit.client.0.smithi042.stdout:1/255: symlink d0/d15/l57 0 2024-09-16T23:48:52.990 INFO:tasks.workunit.client.1.smithi043.stdout:9/364: write d4/d21/f4d [421933,75045] 0 2024-09-16T23:48:52.991 INFO:tasks.workunit.client.1.smithi043.stdout:9/365: fdatasync d4/d1c/d6b/f57 0 2024-09-16T23:48:52.991 INFO:tasks.workunit.client.0.smithi042.stdout:1/256: creat d0/d17/f58 x:0 0 0 2024-09-16T23:48:52.994 INFO:tasks.workunit.client.0.smithi042.stdout:5/281: sync 2024-09-16T23:48:52.994 INFO:tasks.workunit.client.0.smithi042.stdout:2/214: sync 2024-09-16T23:48:52.994 INFO:tasks.workunit.client.0.smithi042.stdout:3/190: sync 2024-09-16T23:48:52.995 INFO:tasks.workunit.client.0.smithi042.stdout:0/187: sync 2024-09-16T23:48:52.995 INFO:tasks.workunit.client.0.smithi042.stdout:8/252: sync 2024-09-16T23:48:52.995 INFO:tasks.workunit.client.0.smithi042.stdout:4/299: sync 2024-09-16T23:48:52.995 INFO:tasks.workunit.client.0.smithi042.stdout:6/177: sync 2024-09-16T23:48:52.997 INFO:tasks.workunit.client.0.smithi042.stdout:9/168: creat d4/d10/d24/f35 x:0 0 0 2024-09-16T23:48:52.997 INFO:tasks.workunit.client.1.smithi043.stdout:8/382: rmdir de/d18/d1f/d2f 39 2024-09-16T23:48:52.998 INFO:tasks.workunit.client.1.smithi043.stdout:7/396: dwrite d4/db/d11/d15/f47 [0,4194304] 0 2024-09-16T23:48:52.998 INFO:tasks.workunit.client.0.smithi042.stdout:4/300: chown de/d2e/c32 33309 1 2024-09-16T23:48:53.000 INFO:tasks.workunit.client.1.smithi043.stdout:6/420: mknod de/d23/d34/d66/d70/c8e 0 2024-09-16T23:48:53.001 INFO:tasks.workunit.client.0.smithi042.stdout:7/313: dwrite d3/d15/d16/d19/f21 [0,4194304] 0 2024-09-16T23:48:53.001 INFO:tasks.workunit.client.0.smithi042.stdout:1/257: mkdir d0/d9/d4e/d59 0 2024-09-16T23:48:53.001 INFO:tasks.workunit.client.0.smithi042.stdout:3/191: write d2/d1b/f3d [365617,74227] 0 2024-09-16T23:48:53.005 INFO:tasks.workunit.client.0.smithi042.stdout:3/192: write d2/d9/dd/d2c/f32 [134074,39206] 0 2024-09-16T23:48:53.005 INFO:tasks.workunit.client.0.smithi042.stdout:5/282: write d9/db/de/d10/f20 [1475605,14683] 0 2024-09-16T23:48:53.008 INFO:tasks.workunit.client.0.smithi042.stdout:8/253: creat da/d19/d1d/f50 x:0 0 0 2024-09-16T23:48:53.010 INFO:tasks.workunit.client.0.smithi042.stdout:4/301: write fd [140832,102637] 0 2024-09-16T23:48:53.011 INFO:tasks.workunit.client.0.smithi042.stdout:5/283: rmdir d9/db/d17/d43 39 2024-09-16T23:48:53.011 INFO:tasks.workunit.client.0.smithi042.stdout:8/254: dread - da/d19/d1d/f3c zero size 2024-09-16T23:48:53.015 INFO:tasks.workunit.client.0.smithi042.stdout:0/188: dread d0/df/f10 [0,4194304] 0 2024-09-16T23:48:53.016 INFO:tasks.workunit.client.1.smithi043.stdout:0/400: mknod d0/d2/c87 0 2024-09-16T23:48:53.017 INFO:tasks.workunit.client.1.smithi043.stdout:0/401: stat d0/d2/dd/f60 0 2024-09-16T23:48:53.017 INFO:tasks.workunit.client.0.smithi042.stdout:7/314: creat d3/d15/f6c x:0 0 0 2024-09-16T23:48:53.017 INFO:tasks.workunit.client.0.smithi042.stdout:0/189: chown d0/df 3453 1 2024-09-16T23:48:53.018 INFO:tasks.workunit.client.1.smithi043.stdout:0/402: read d0/dc/f42 [3877536,25935] 0 2024-09-16T23:48:53.018 INFO:tasks.workunit.client.1.smithi043.stdout:0/403: readlink d0/d2/dd/l64 0 2024-09-16T23:48:53.018 INFO:tasks.workunit.client.1.smithi043.stdout:1/511: fsync d4/d9/d1c/d1e/d42/d5d/f9a 0 2024-09-16T23:48:53.019 INFO:tasks.workunit.client.0.smithi042.stdout:4/302: rmdir de/d12/d1b 39 2024-09-16T23:48:53.020 INFO:tasks.workunit.client.0.smithi042.stdout:5/284: mkdir d9/db/d1b/d31/d46/d58 0 2024-09-16T23:48:53.020 INFO:tasks.workunit.client.0.smithi042.stdout:2/215: dread d0/da/f21 [0,4194304] 0 2024-09-16T23:48:53.021 INFO:tasks.workunit.client.0.smithi042.stdout:0/190: readlink d0/ld 0 2024-09-16T23:48:53.021 INFO:tasks.workunit.client.0.smithi042.stdout:3/193: dread d2/d9/dd/d17/d21/f31 [0,4194304] 0 2024-09-16T23:48:53.022 INFO:tasks.workunit.client.0.smithi042.stdout:8/255: fsync da/d2f/f38 0 2024-09-16T23:48:53.022 INFO:tasks.workunit.client.0.smithi042.stdout:7/315: rmdir d3/d15/d32 39 2024-09-16T23:48:53.023 INFO:tasks.workunit.client.0.smithi042.stdout:0/191: read d0/f9 [3906644,73972] 0 2024-09-16T23:48:53.024 INFO:tasks.workunit.client.0.smithi042.stdout:4/303: chown de/d1a/d1c/f21 126994 1 2024-09-16T23:48:53.025 INFO:tasks.workunit.client.0.smithi042.stdout:9/169: rename d4/d10/d24/d25/d28/f34 to d4/d12/f36 0 2024-09-16T23:48:53.025 INFO:tasks.workunit.client.1.smithi043.stdout:3/413: rename d4/de/d1f/d34/f44 to d4/de/d1f/f79 0 2024-09-16T23:48:53.025 INFO:tasks.workunit.client.1.smithi043.stdout:3/414: chown d4/de/d1f/f56 1200595 1 2024-09-16T23:48:53.026 INFO:tasks.workunit.client.0.smithi042.stdout:8/256: dread - da/d19/d1d/f50 zero size 2024-09-16T23:48:53.026 INFO:tasks.workunit.client.0.smithi042.stdout:3/194: chown d2/d1b/f43 1495476303 1 2024-09-16T23:48:53.026 INFO:tasks.workunit.client.0.smithi042.stdout:2/216: write d0/da/f49 [331675,27318] 0 2024-09-16T23:48:53.027 INFO:tasks.workunit.client.0.smithi042.stdout:1/258: mknod d0/c5a 0 2024-09-16T23:48:53.029 INFO:tasks.workunit.client.1.smithi043.stdout:8/383: mkdir de/d70/d74 0 2024-09-16T23:48:53.029 INFO:tasks.workunit.client.0.smithi042.stdout:5/285: creat d9/db/de/d10/d2f/f59 x:0 0 0 2024-09-16T23:48:53.031 INFO:tasks.workunit.client.0.smithi042.stdout:9/170: dread - d4/d10/d24/f35 zero size 2024-09-16T23:48:53.032 INFO:tasks.workunit.client.0.smithi042.stdout:0/192: truncate d0/d15/f2f 929268 0 2024-09-16T23:48:53.032 INFO:tasks.workunit.client.1.smithi043.stdout:7/397: creat d4/db/d11/d21/d31/f88 x:0 0 0 2024-09-16T23:48:53.032 INFO:tasks.workunit.client.1.smithi043.stdout:8/384: dread de/d18/d1f/f59 [0,4194304] 0 2024-09-16T23:48:53.035 INFO:tasks.workunit.client.0.smithi042.stdout:3/195: mknod d2/d1b/d1d/c46 0 2024-09-16T23:48:53.035 INFO:tasks.workunit.client.0.smithi042.stdout:6/178: dwrite d8/db/dd/d2c/f10 [0,4194304] 0 2024-09-16T23:48:53.037 INFO:tasks.workunit.client.0.smithi042.stdout:4/304: link de/c18 de/d1a/d1c/c6a 0 2024-09-16T23:48:53.039 INFO:tasks.workunit.client.1.smithi043.stdout:4/387: rename d8/d15/d20/l29 to d8/d25/d39/l74 0 2024-09-16T23:48:53.039 INFO:tasks.workunit.client.0.smithi042.stdout:0/193: dread - d0/df/d23/f27 zero size 2024-09-16T23:48:53.041 INFO:tasks.workunit.client.1.smithi043.stdout:3/415: dread d4/de/d1f/d2c/f32 [0,4194304] 0 2024-09-16T23:48:53.042 INFO:tasks.workunit.client.0.smithi042.stdout:5/286: rename d9/db/de/c3a to d9/db/d19/d21/d41/c5a 0 2024-09-16T23:48:53.042 INFO:tasks.workunit.client.0.smithi042.stdout:2/217: chown d0/da/d1b/d3b/d44/l41 6 1 2024-09-16T23:48:53.042 INFO:tasks.workunit.client.1.smithi043.stdout:6/421: mknod de/d1a/d28/c8f 0 2024-09-16T23:48:53.043 INFO:tasks.workunit.client.1.smithi043.stdout:6/422: write de/d1a/d28/f3e [377665,54536] 0 2024-09-16T23:48:53.045 INFO:tasks.workunit.client.0.smithi042.stdout:3/196: truncate d2/d7/f8 4879177 0 2024-09-16T23:48:53.048 INFO:tasks.workunit.client.0.smithi042.stdout:3/197: truncate d2/d1b/f2a 385424 0 2024-09-16T23:48:53.048 INFO:tasks.workunit.client.0.smithi042.stdout:6/179: mkdir d8/db/dd/d1b/d34/d39 0 2024-09-16T23:48:53.049 INFO:tasks.workunit.client.0.smithi042.stdout:8/257: dwrite da/d19/d1d/d30/f45 [0,4194304] 0 2024-09-16T23:48:53.049 INFO:tasks.workunit.client.0.smithi042.stdout:5/287: mknod d9/db/d1b/c5b 0 2024-09-16T23:48:53.051 INFO:tasks.workunit.client.0.smithi042.stdout:4/305: rename de/d12/d1b/f45 to de/d2e/d37/d4c/d4f/f6b 0 2024-09-16T23:48:53.053 INFO:tasks.workunit.client.0.smithi042.stdout:3/198: dread - d2/d1b/d1d/f38 zero size 2024-09-16T23:48:53.054 INFO:tasks.workunit.client.0.smithi042.stdout:2/218: chown d0/da/ce 232324842 1 2024-09-16T23:48:53.055 INFO:tasks.workunit.client.0.smithi042.stdout:6/180: fdatasync d8/db/dd/d2c/d26/f32 0 2024-09-16T23:48:53.055 INFO:tasks.workunit.client.0.smithi042.stdout:4/306: chown de/d1a/l20 829058 1 2024-09-16T23:48:53.055 INFO:tasks.workunit.client.1.smithi043.stdout:0/404: mkdir d0/d2/df/d1a/d49/d80/d88 0 2024-09-16T23:48:53.056 INFO:tasks.workunit.client.0.smithi042.stdout:2/219: read - d0/f3a zero size 2024-09-16T23:48:53.057 INFO:tasks.workunit.client.0.smithi042.stdout:0/194: write d0/df/f10 [1838375,46775] 0 2024-09-16T23:48:53.058 INFO:tasks.workunit.client.1.smithi043.stdout:1/512: symlink d4/d9/d1c/d1e/d42/d5d/d71/d76/lae 0 2024-09-16T23:48:53.061 INFO:tasks.workunit.client.0.smithi042.stdout:3/199: write d2/d9/dd/d17/d21/f2e [270830,97972] 0 2024-09-16T23:48:53.062 INFO:tasks.workunit.client.0.smithi042.stdout:0/195: write d0/d15/f2f [1162019,30665] 0 2024-09-16T23:48:53.062 INFO:tasks.workunit.client.0.smithi042.stdout:1/259: dread d0/d17/f21 [0,4194304] 0 2024-09-16T23:48:53.065 INFO:tasks.workunit.client.0.smithi042.stdout:4/307: symlink de/d42/l6c 0 2024-09-16T23:48:53.067 INFO:tasks.workunit.client.1.smithi043.stdout:4/388: write d8/d25/d39/d58/f4a [685666,87293] 0 2024-09-16T23:48:53.069 INFO:tasks.workunit.client.0.smithi042.stdout:0/196: chown d0/d15/l32 3639 1 2024-09-16T23:48:53.070 INFO:tasks.workunit.client.0.smithi042.stdout:4/308: mknod de/d2e/d37/d3c/c6d 0 2024-09-16T23:48:53.071 INFO:tasks.workunit.client.0.smithi042.stdout:4/309: fdatasync de/d12/f28 0 2024-09-16T23:48:53.072 INFO:tasks.workunit.client.0.smithi042.stdout:3/200: creat d2/d9/dd/d17/d21/f47 x:0 0 0 2024-09-16T23:48:53.073 INFO:tasks.workunit.client.0.smithi042.stdout:0/197: rmdir d0/d15 39 2024-09-16T23:48:53.075 INFO:tasks.workunit.client.0.smithi042.stdout:4/310: rename de/d42/c66 to de/d42/d68/c6e 0 2024-09-16T23:48:53.076 INFO:tasks.workunit.client.0.smithi042.stdout:5/288: truncate d9/db/de/d10/f1a 2093457 0 2024-09-16T23:48:53.079 INFO:tasks.workunit.client.1.smithi043.stdout:2/343: getdents d8/d4b 0 2024-09-16T23:48:53.079 INFO:tasks.workunit.client.0.smithi042.stdout:0/198: mkdir d0/df/d13/d2d/d35 0 2024-09-16T23:48:53.080 INFO:tasks.workunit.client.0.smithi042.stdout:5/289: write d9/db/d17/f4e [1047487,2133] 0 2024-09-16T23:48:53.082 INFO:tasks.workunit.client.0.smithi042.stdout:5/290: fdatasync d9/db/f25 0 2024-09-16T23:48:53.083 INFO:tasks.workunit.client.1.smithi043.stdout:6/423: dread de/d23/d34/d66/f6e [0,4194304] 0 2024-09-16T23:48:53.083 INFO:tasks.workunit.client.1.smithi043.stdout:7/398: truncate d4/f62 4946705 0 2024-09-16T23:48:53.083 INFO:tasks.workunit.client.1.smithi043.stdout:6/424: fsync de/d1a/f41 0 2024-09-16T23:48:53.084 INFO:tasks.workunit.client.1.smithi043.stdout:1/513: creat d4/df/d13/faf x:0 0 0 2024-09-16T23:48:53.084 INFO:tasks.workunit.client.1.smithi043.stdout:7/399: truncate d4/d27/f6c 5072537 0 2024-09-16T23:48:53.085 INFO:tasks.workunit.client.0.smithi042.stdout:0/199: symlink d0/df/d13/d2d/d35/l36 0 2024-09-16T23:48:53.085 INFO:tasks.workunit.client.0.smithi042.stdout:4/311: dwrite de/d1a/f39 [0,4194304] 0 2024-09-16T23:48:53.085 INFO:tasks.workunit.client.1.smithi043.stdout:9/366: link d4/d1c/l68 d4/d28/d5f/l6c 0 2024-09-16T23:48:53.086 INFO:tasks.workunit.client.0.smithi042.stdout:5/291: write d9/db/d19/f39 [273309,4336] 0 2024-09-16T23:48:53.086 INFO:tasks.workunit.client.0.smithi042.stdout:9/171: getdents d4/d10/d24/d25/d28 0 2024-09-16T23:48:53.087 INFO:tasks.workunit.client.0.smithi042.stdout:6/181: mknod d8/db/dd/d1b/d2d/c3a 0 2024-09-16T23:48:53.087 INFO:tasks.workunit.client.1.smithi043.stdout:4/389: fdatasync d8/d25/f2c 0 2024-09-16T23:48:53.088 INFO:tasks.workunit.client.0.smithi042.stdout:5/292: stat d9/db/de/d10/f22 0 2024-09-16T23:48:53.089 INFO:tasks.workunit.client.0.smithi042.stdout:0/200: fsync d0/df/d13/f34 0 2024-09-16T23:48:53.089 INFO:tasks.workunit.client.1.smithi043.stdout:4/390: fsync d8/d15/d20/d2b/f28 0 2024-09-16T23:48:53.090 INFO:tasks.workunit.client.0.smithi042.stdout:5/293: dread - d9/db/d19/f52 zero size 2024-09-16T23:48:53.091 INFO:tasks.workunit.client.1.smithi043.stdout:8/385: creat de/d18/d27/f75 x:0 0 0 2024-09-16T23:48:53.093 INFO:tasks.workunit.client.0.smithi042.stdout:1/260: mknod d0/d9/d4e/d59/c5b 0 2024-09-16T23:48:53.093 INFO:tasks.workunit.client.1.smithi043.stdout:7/400: dwrite d4/db/f51 [0,4194304] 0 2024-09-16T23:48:53.093 INFO:tasks.workunit.client.1.smithi043.stdout:2/344: rmdir d8/d1d/d20/d23/d39/d3a 39 2024-09-16T23:48:53.094 INFO:tasks.workunit.client.0.smithi042.stdout:2/220: link d0/c3c d0/da/d17/d3e/c51 0 2024-09-16T23:48:53.096 INFO:tasks.workunit.client.1.smithi043.stdout:7/401: chown d4/db/d43/f79 1661101 1 2024-09-16T23:48:53.097 INFO:tasks.workunit.client.0.smithi042.stdout:0/201: creat d0/f37 x:0 0 0 2024-09-16T23:48:53.098 INFO:tasks.workunit.client.1.smithi043.stdout:1/514: truncate d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f30 2245109 0 2024-09-16T23:48:53.098 INFO:tasks.workunit.client.1.smithi043.stdout:4/391: dread d8/d15/d20/d2b/d23/f38 [0,4194304] 0 2024-09-16T23:48:53.099 INFO:tasks.workunit.client.0.smithi042.stdout:8/258: getdents da/d2f 0 2024-09-16T23:48:53.100 INFO:tasks.workunit.client.0.smithi042.stdout:5/294: link d9/db/f28 d9/db/d1b/d31/f5c 0 2024-09-16T23:48:53.100 INFO:tasks.workunit.client.0.smithi042.stdout:1/261: mknod d0/d9/d35/d46/c5c 0 2024-09-16T23:48:53.106 INFO:tasks.workunit.client.0.smithi042.stdout:0/202: creat d0/df/d23/f38 x:0 0 0 2024-09-16T23:48:53.108 INFO:tasks.workunit.client.1.smithi043.stdout:2/345: mkdir d8/d4b/d4f/d7c 0 2024-09-16T23:48:53.109 INFO:tasks.workunit.client.0.smithi042.stdout:4/312: truncate de/d12/f54 2126818 0 2024-09-16T23:48:53.111 INFO:tasks.workunit.client.0.smithi042.stdout:2/221: dwrite d0/da/d17/f42 [0,4194304] 0 2024-09-16T23:48:53.112 INFO:tasks.workunit.client.1.smithi043.stdout:1/515: dread d4/d34/f7b [0,4194304] 0 2024-09-16T23:48:53.113 INFO:tasks.workunit.client.1.smithi043.stdout:9/367: rmdir d4/d1c/d25/d2b/d4c/d50 39 2024-09-16T23:48:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:4/313: rmdir de/d12/d1b 39 2024-09-16T23:48:53.117 INFO:tasks.workunit.client.0.smithi042.stdout:8/259: symlink da/d19/l51 0 2024-09-16T23:48:53.118 INFO:tasks.workunit.client.0.smithi042.stdout:5/295: creat d9/db/d17/f5d x:0 0 0 2024-09-16T23:48:53.119 INFO:tasks.workunit.client.0.smithi042.stdout:5/296: dread - d9/db/d17/f5d zero size 2024-09-16T23:48:53.120 INFO:tasks.workunit.client.1.smithi043.stdout:4/392: symlink d8/d15/d20/d2b/d22/l75 0 2024-09-16T23:48:53.121 INFO:tasks.workunit.client.0.smithi042.stdout:8/260: write da/f14 [5233987,51415] 0 2024-09-16T23:48:53.121 INFO:tasks.workunit.client.0.smithi042.stdout:4/314: chown de/c18 1 1 2024-09-16T23:48:53.122 INFO:tasks.workunit.client.1.smithi043.stdout:7/402: dread d4/db/d11/f13 [0,4194304] 0 2024-09-16T23:48:53.124 INFO:tasks.workunit.client.1.smithi043.stdout:7/403: write d4/d27/d41/d1b/f44 [791818,84642] 0 2024-09-16T23:48:53.124 INFO:tasks.workunit.client.0.smithi042.stdout:5/297: write d9/db/d19/f52 [492823,124979] 0 2024-09-16T23:48:53.125 INFO:tasks.workunit.client.0.smithi042.stdout:1/262: dwrite d0/d9/d35/d46/f30 [0,4194304] 0 2024-09-16T23:48:53.126 INFO:tasks.workunit.client.1.smithi043.stdout:7/404: stat d4/db/d11/d21/d31/f6b 0 2024-09-16T23:48:53.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:52 smithi043 ceph-mon[35191]: pgmap v5: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:52 smithi043 ceph-mon[35191]: [16/Sep/2024:23:48:51] ENGINE Bus STARTING 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:52 smithi043 ceph-mon[35191]: [16/Sep/2024:23:48:51] ENGINE Serving on https://172.21.15.43:7150 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:52 smithi043 ceph-mon[35191]: [16/Sep/2024:23:48:51] ENGINE Error in HTTPServer.serve 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Traceback (most recent call last): 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._run(expiration_interval) 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T23:48:53.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: s = self.context.wrap_socket( 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: return self.sslsocket_class._create( 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self.do_handshake() 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._sslobj.do_handshake() 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:52 smithi043 ceph-mon[35191]: [16/Sep/2024:23:48:52] ENGINE Serving on http://172.21.15.43:8765 2024-09-16T23:48:53.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:52 smithi043 ceph-mon[35191]: [16/Sep/2024:23:48:52] ENGINE Bus STARTED 2024-09-16T23:48:53.130 INFO:tasks.workunit.client.0.smithi042.stdout:1/263: chown d0/d9/d35 9026789 1 2024-09-16T23:48:53.130 INFO:tasks.workunit.client.1.smithi043.stdout:8/386: rename de/d18/d1f/d2f/d39/f6c to de/d70/f76 0 2024-09-16T23:48:53.130 INFO:tasks.workunit.client.1.smithi043.stdout:7/405: dread d4/db/f51 [0,4194304] 0 2024-09-16T23:48:53.132 INFO:tasks.workunit.client.1.smithi043.stdout:8/387: write de/d18/d27/f75 [197591,113465] 0 2024-09-16T23:48:53.133 INFO:tasks.workunit.client.0.smithi042.stdout:1/264: fdatasync d0/d17/f20 0 2024-09-16T23:48:53.134 INFO:tasks.workunit.client.1.smithi043.stdout:2/346: rmdir d8/d1f 39 2024-09-16T23:48:53.136 INFO:tasks.workunit.client.0.smithi042.stdout:4/315: rename de/l22 to de/d2e/d37/l6f 0 2024-09-16T23:48:53.136 INFO:tasks.workunit.client.0.smithi042.stdout:6/182: dwrite f7 [0,4194304] 0 2024-09-16T23:48:53.141 INFO:tasks.workunit.client.0.smithi042.stdout:4/316: dread - de/d12/d1b/f59 zero size 2024-09-16T23:48:53.142 INFO:tasks.workunit.client.1.smithi043.stdout:2/347: dwrite d8/f1a [0,4194304] 0 2024-09-16T23:48:53.142 INFO:tasks.workunit.client.0.smithi042.stdout:4/317: chown de/f19 373335 1 2024-09-16T23:48:53.143 INFO:tasks.workunit.client.1.smithi043.stdout:2/348: write d8/d11/f46 [320746,79445] 0 2024-09-16T23:48:53.144 INFO:tasks.workunit.client.0.smithi042.stdout:4/318: write de/d1a/f55 [866803,106668] 0 2024-09-16T23:48:53.146 INFO:tasks.workunit.client.0.smithi042.stdout:5/298: rename d9/db/d17/d44 to d9/db/de/d5e 0 2024-09-16T23:48:53.148 INFO:tasks.workunit.client.1.smithi043.stdout:4/393: unlink d8/d25/f2c 0 2024-09-16T23:48:53.149 INFO:tasks.workunit.client.0.smithi042.stdout:4/319: mkdir de/d1a/d70 0 2024-09-16T23:48:53.150 INFO:tasks.workunit.client.0.smithi042.stdout:2/222: rmdir d0/d30/d4f 0 2024-09-16T23:48:53.151 INFO:tasks.workunit.client.0.smithi042.stdout:8/261: write da/d19/f23 [843287,127246] 0 2024-09-16T23:48:53.153 INFO:tasks.workunit.client.0.smithi042.stdout:4/320: truncate de/d1a/d1c/f47 454820 0 2024-09-16T23:48:53.153 INFO:tasks.workunit.client.0.smithi042.stdout:1/265: symlink d0/d9/d4e/d59/l5d 0 2024-09-16T23:48:53.156 INFO:tasks.workunit.client.0.smithi042.stdout:4/321: chown de/d1a/d1c/f21 28 1 2024-09-16T23:48:53.157 INFO:tasks.workunit.client.0.smithi042.stdout:5/299: rename d9/db/d1b/d31/f5c to d9/db/d17/f5f 0 2024-09-16T23:48:53.158 INFO:tasks.workunit.client.1.smithi043.stdout:1/516: rename d4/df/d13/faf to d4/d9/d12/d2b/fb0 0 2024-09-16T23:48:53.160 INFO:tasks.workunit.client.0.smithi042.stdout:8/262: mknod da/d19/d1d/c52 0 2024-09-16T23:48:53.160 INFO:tasks.workunit.client.0.smithi042.stdout:1/266: dread d0/d9/f24 [0,4194304] 0 2024-09-16T23:48:53.164 INFO:tasks.workunit.client.0.smithi042.stdout:2/223: creat d0/da/d17/f52 x:0 0 0 2024-09-16T23:48:53.165 INFO:tasks.workunit.client.1.smithi043.stdout:8/388: creat de/d18/d2b/f77 x:0 0 0 2024-09-16T23:48:53.166 INFO:tasks.workunit.client.1.smithi043.stdout:8/389: write de/f54 [3901389,113617] 0 2024-09-16T23:48:53.166 INFO:tasks.workunit.client.0.smithi042.stdout:6/183: mknod d8/db/dd/d1b/d34/d39/c3b 0 2024-09-16T23:48:53.172 INFO:tasks.workunit.client.1.smithi043.stdout:2/349: fsync d8/d1d/f2a 0 2024-09-16T23:48:53.172 INFO:tasks.workunit.client.0.smithi042.stdout:4/322: dwrite de/d2e/d37/d4c/d4f/f58 [0,4194304] 0 2024-09-16T23:48:53.173 INFO:tasks.workunit.client.1.smithi043.stdout:2/350: dread - d8/d1d/d20/d23/d38/f66 zero size 2024-09-16T23:48:53.175 INFO:tasks.workunit.client.1.smithi043.stdout:2/351: truncate d8/d1d/d20/d26/f60 1546756 0 2024-09-16T23:48:53.175 INFO:tasks.workunit.client.1.smithi043.stdout:2/352: chown d8/d1d/d20/d26 114534669 1 2024-09-16T23:48:53.176 INFO:tasks.workunit.client.0.smithi042.stdout:8/263: dread da/d19/d1d/f3e [0,4194304] 0 2024-09-16T23:48:53.179 INFO:tasks.workunit.client.1.smithi043.stdout:2/353: dread d8/d35/f37 [0,4194304] 0 2024-09-16T23:48:53.180 INFO:tasks.workunit.client.0.smithi042.stdout:5/300: dwrite d9/db/de/d10/f20 [0,4194304] 0 2024-09-16T23:48:53.181 INFO:tasks.workunit.client.0.smithi042.stdout:8/264: chown da/c3a 10 1 2024-09-16T23:48:53.182 INFO:tasks.workunit.client.0.smithi042.stdout:1/267: rename d0/d15 to d0/d17/d5e 0 2024-09-16T23:48:53.187 INFO:tasks.workunit.client.0.smithi042.stdout:2/224: dwrite d0/da/d17/f42 [0,4194304] 0 2024-09-16T23:48:53.187 INFO:tasks.workunit.client.1.smithi043.stdout:1/517: dread - d4/d34/f6e zero size 2024-09-16T23:48:53.188 INFO:tasks.workunit.client.0.smithi042.stdout:4/323: creat de/d1a/d70/f71 x:0 0 0 2024-09-16T23:48:53.188 INFO:tasks.workunit.client.1.smithi043.stdout:1/518: dread - d4/f94 zero size 2024-09-16T23:48:53.189 INFO:tasks.workunit.client.0.smithi042.stdout:5/301: getdents d9/d2c 0 2024-09-16T23:48:53.191 INFO:tasks.workunit.client.1.smithi043.stdout:4/394: mknod d8/d15/d20/d2b/d23/d27/d4b/c76 0 2024-09-16T23:48:53.192 INFO:tasks.workunit.client.0.smithi042.stdout:4/324: creat de/d42/d68/f72 x:0 0 0 2024-09-16T23:48:53.192 INFO:tasks.workunit.client.0.smithi042.stdout:6/184: dread d8/db/dd/d2c/fa [0,4194304] 0 2024-09-16T23:48:53.194 INFO:tasks.workunit.client.0.smithi042.stdout:4/325: write de/d2e/d37/d3e/f5c [667944,13673] 0 2024-09-16T23:48:53.195 INFO:tasks.workunit.client.0.smithi042.stdout:1/268: rename d0/d9/d35/d43/l51 to d0/d17/l5f 0 2024-09-16T23:48:53.196 INFO:tasks.workunit.client.0.smithi042.stdout:5/302: fsync d9/db/de/d10/f1a 0 2024-09-16T23:48:53.202 INFO:tasks.workunit.client.0.smithi042.stdout:5/303: fsync d9/db/d19/d21/f2e 0 2024-09-16T23:48:53.202 INFO:tasks.workunit.client.0.smithi042.stdout:2/225: creat d0/d26/f53 x:0 0 0 2024-09-16T23:48:53.203 INFO:tasks.workunit.client.1.smithi043.stdout:1/519: chown d4/d9/d12/f5c 858 1 2024-09-16T23:48:53.204 INFO:tasks.workunit.client.1.smithi043.stdout:1/520: chown d4/d9/d12/d15/l22 5004 1 2024-09-16T23:48:53.205 INFO:tasks.workunit.client.1.smithi043.stdout:1/521: dread - d4/d9/d12/d2b/d3d/d83/fa8 zero size 2024-09-16T23:48:53.205 INFO:tasks.workunit.client.1.smithi043.stdout:8/390: symlink de/d18/d1f/d2f/d39/d4d/l78 0 2024-09-16T23:48:53.207 INFO:tasks.workunit.client.0.smithi042.stdout:5/304: mkdir d9/db/d19/d60 0 2024-09-16T23:48:53.208 INFO:tasks.workunit.client.1.smithi043.stdout:4/395: read d8/d15/d20/d2b/d22/f36 [2926787,121494] 0 2024-09-16T23:48:53.208 INFO:tasks.workunit.client.0.smithi042.stdout:1/269: dwrite d0/d17/f21 [0,4194304] 0 2024-09-16T23:48:53.209 INFO:tasks.workunit.client.1.smithi043.stdout:7/406: link d4/l8 d4/d27/d41/d1b/l89 0 2024-09-16T23:48:53.210 INFO:tasks.workunit.client.1.smithi043.stdout:7/407: truncate d4/db/d11/d15/d29/f76 260550 0 2024-09-16T23:48:53.210 INFO:tasks.workunit.client.0.smithi042.stdout:6/185: dread d8/f18 [0,4194304] 0 2024-09-16T23:48:53.211 INFO:tasks.workunit.client.1.smithi043.stdout:7/408: read d4/db/d43/d83/d85/f57 [3157374,14328] 0 2024-09-16T23:48:53.212 INFO:tasks.workunit.client.0.smithi042.stdout:1/270: write d0/f4a [239388,299] 0 2024-09-16T23:48:53.213 INFO:tasks.workunit.client.1.smithi043.stdout:7/409: write d4/db/f73 [2197431,104239] 0 2024-09-16T23:48:53.215 INFO:tasks.workunit.client.0.smithi042.stdout:8/265: link da/d2f/f38 da/d19/d1d/d30/d3f/f53 0 2024-09-16T23:48:53.215 INFO:tasks.workunit.client.0.smithi042.stdout:4/326: dwrite de/d42/f50 [0,4194304] 0 2024-09-16T23:48:53.217 INFO:tasks.workunit.client.1.smithi043.stdout:7/410: write d4/d27/d41/d1b/f6f [824214,84948] 0 2024-09-16T23:48:53.217 INFO:tasks.workunit.client.1.smithi043.stdout:2/354: dwrite d8/d1d/d20/d23/d39/d3a/f72 [0,4194304] 0 2024-09-16T23:48:53.218 INFO:tasks.workunit.client.1.smithi043.stdout:9/368: rename d4/d1c/d25/f26 to d4/d21/f6d 0 2024-09-16T23:48:53.218 INFO:tasks.workunit.client.0.smithi042.stdout:5/305: rename d9/db/d17 to d9/db/de/d10/d4b/d61 0 2024-09-16T23:48:53.220 INFO:tasks.workunit.client.1.smithi043.stdout:1/522: creat d4/d9/d1c/d1e/d42/d5d/d71/fb1 x:0 0 0 2024-09-16T23:48:53.221 INFO:tasks.workunit.client.0.smithi042.stdout:4/327: chown de/d1a/f33 77082 1 2024-09-16T23:48:53.225 INFO:tasks.workunit.client.0.smithi042.stdout:5/306: symlink d9/db/d11/l62 0 2024-09-16T23:48:53.230 INFO:tasks.workunit.client.0.smithi042.stdout:2/226: dwrite d0/f2b [0,4194304] 0 2024-09-16T23:48:53.231 INFO:tasks.workunit.client.0.smithi042.stdout:5/307: write d9/db/d1b/d31/d46/f49 [371039,113583] 0 2024-09-16T23:48:53.231 INFO:tasks.workunit.client.0.smithi042.stdout:4/328: mkdir de/d1a/d1c/d73 0 2024-09-16T23:48:53.231 INFO:tasks.workunit.client.0.smithi042.stdout:1/271: mkdir d0/d17/d18/d60 0 2024-09-16T23:48:53.240 INFO:tasks.workunit.client.0.smithi042.stdout:6/186: dwrite d8/db/dd/d2c/f10 [0,4194304] 0 2024-09-16T23:48:53.240 INFO:tasks.workunit.client.0.smithi042.stdout:2/227: fsync d0/d30/f38 0 2024-09-16T23:48:53.247 INFO:tasks.workunit.client.0.smithi042.stdout:2/228: mkdir d0/da/d1b/d3b/d44/d54 0 2024-09-16T23:48:53.250 INFO:tasks.workunit.client.0.smithi042.stdout:1/272: link d0/d17/d18/f4d d0/d9/d35/d46/f61 0 2024-09-16T23:48:53.251 INFO:tasks.workunit.client.0.smithi042.stdout:2/229: readlink d0/l27 0 2024-09-16T23:48:53.254 INFO:tasks.workunit.client.0.smithi042.stdout:1/273: mknod d0/d3b/c62 0 2024-09-16T23:48:53.257 INFO:tasks.workunit.client.0.smithi042.stdout:6/187: mkdir d8/db/d3c 0 2024-09-16T23:48:53.258 INFO:tasks.workunit.client.0.smithi042.stdout:6/188: chown d8/db/dd/f22 6075 1 2024-09-16T23:48:53.260 INFO:tasks.workunit.client.0.smithi042.stdout:2/230: creat d0/d30/f55 x:0 0 0 2024-09-16T23:48:53.265 INFO:tasks.workunit.client.0.smithi042.stdout:6/189: getdents d8/db/d3c 0 2024-09-16T23:48:53.267 INFO:tasks.workunit.client.0.smithi042.stdout:6/190: truncate d8/db/dd/d2c/f31 3341404 0 2024-09-16T23:48:53.269 INFO:tasks.workunit.client.0.smithi042.stdout:6/191: read d8/db/dd/f11 [447780,57378] 0 2024-09-16T23:48:53.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:52 smithi042 ceph-mon[27914]: pgmap v5: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:53.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:52 smithi042 ceph-mon[27914]: [16/Sep/2024:23:48:51] ENGINE Bus STARTING 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:52 smithi042 ceph-mon[27914]: [16/Sep/2024:23:48:51] ENGINE Serving on https://172.21.15.43:7150 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:52 smithi042 ceph-mon[27914]: [16/Sep/2024:23:48:51] ENGINE Error in HTTPServer.serve 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Traceback (most recent call last): 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._connections.run(self.expiration_interval) 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._run(expiration_interval) 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T23:48:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: s = self.context.wrap_socket( 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: return self.sslsocket_class._create( 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self.do_handshake() 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._sslobj.do_handshake() 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:52 smithi042 ceph-mon[27914]: [16/Sep/2024:23:48:52] ENGINE Serving on http://172.21.15.43:8765 2024-09-16T23:48:53.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:52 smithi042 ceph-mon[27914]: [16/Sep/2024:23:48:52] ENGINE Bus STARTED 2024-09-16T23:48:53.274 INFO:tasks.workunit.client.0.smithi042.stdout:6/192: creat d8/db/dd/d1b/d34/f3d x:0 0 0 2024-09-16T23:48:53.275 INFO:tasks.workunit.client.0.smithi042.stdout:6/193: creat d8/db/dd/d1b/d2d/f3e x:0 0 0 2024-09-16T23:48:53.276 INFO:tasks.workunit.client.0.smithi042.stdout:6/194: getdents d8/db/d3c 0 2024-09-16T23:48:53.278 INFO:tasks.workunit.client.0.smithi042.stdout:6/195: rename d8/db/dd/f25 to d8/db/dd/d1b/f3f 0 2024-09-16T23:48:53.279 INFO:tasks.workunit.client.0.smithi042.stdout:6/196: chown d8/db/dd/f1c 347 1 2024-09-16T23:48:53.282 INFO:tasks.workunit.client.0.smithi042.stdout:6/197: truncate d8/db/dd/d2c/fa 1905108 0 2024-09-16T23:48:53.282 INFO:tasks.workunit.client.0.smithi042.stdout:6/198: dread - d8/db/f23 zero size 2024-09-16T23:48:53.282 INFO:tasks.workunit.client.0.smithi042.stdout:6/199: chown d8/f18 0 1 2024-09-16T23:48:53.283 INFO:tasks.workunit.client.0.smithi042.stdout:6/200: truncate d8/db/f20 933722 0 2024-09-16T23:48:53.284 INFO:tasks.workunit.client.0.smithi042.stdout:6/201: chown d8/db/l28 135 1 2024-09-16T23:48:53.288 INFO:tasks.workunit.client.0.smithi042.stdout:6/202: dread d8/db/dd/f11 [0,4194304] 0 2024-09-16T23:48:53.290 INFO:tasks.workunit.client.0.smithi042.stdout:6/203: mknod d8/db/dd/d2c/d26/c40 0 2024-09-16T23:48:53.297 INFO:tasks.workunit.client.0.smithi042.stdout:4/329: creat de/d42/d5d/f74 x:0 0 0 2024-09-16T23:48:53.298 INFO:tasks.workunit.client.0.smithi042.stdout:6/204: getdents d8/db/dd/d1b 0 2024-09-16T23:48:53.299 INFO:tasks.workunit.client.0.smithi042.stdout:4/330: chown de/d2e 29025816 1 2024-09-16T23:48:53.300 INFO:tasks.workunit.client.0.smithi042.stdout:4/331: dread - de/d2e/d37/d3e/f61 zero size 2024-09-16T23:48:53.300 INFO:tasks.workunit.client.1.smithi043.stdout:8/391: mknod de/d70/d3d/c79 0 2024-09-16T23:48:53.301 INFO:tasks.workunit.client.1.smithi043.stdout:7/411: mkdir d4/d27/d8a 0 2024-09-16T23:48:53.302 INFO:tasks.workunit.client.1.smithi043.stdout:4/396: rename d8/d25/d39/l47 to d8/d25/d39/d41/l77 0 2024-09-16T23:48:53.302 INFO:tasks.workunit.client.0.smithi042.stdout:5/308: write d9/db/d19/f39 [1864723,81693] 0 2024-09-16T23:48:53.304 INFO:tasks.workunit.client.1.smithi043.stdout:1/523: write d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f25 [8773285,120951] 0 2024-09-16T23:48:53.306 INFO:tasks.workunit.client.0.smithi042.stdout:5/309: mkdir d9/d63 0 2024-09-16T23:48:53.307 INFO:tasks.workunit.client.0.smithi042.stdout:4/332: rmdir de/d1a/d1c/d73 0 2024-09-16T23:48:53.308 INFO:tasks.workunit.client.0.smithi042.stdout:5/310: chown d9/db/de/d10/d2f/f36 19867406 1 2024-09-16T23:48:53.311 INFO:tasks.workunit.client.0.smithi042.stdout:4/333: rmdir de/d2e/d37 39 2024-09-16T23:48:53.312 INFO:tasks.workunit.client.0.smithi042.stdout:5/311: mkdir d9/db/d11/d64 0 2024-09-16T23:48:53.314 INFO:tasks.workunit.client.0.smithi042.stdout:4/334: symlink de/d1a/d70/l75 0 2024-09-16T23:48:53.316 INFO:tasks.workunit.client.0.smithi042.stdout:5/312: creat d9/db/d19/d60/f65 x:0 0 0 2024-09-16T23:48:53.317 INFO:tasks.workunit.client.0.smithi042.stdout:1/274: dwrite d0/d9/f24 [0,4194304] 0 2024-09-16T23:48:53.319 INFO:tasks.workunit.client.0.smithi042.stdout:5/313: fsync d9/db/de/d10/f1a 0 2024-09-16T23:48:53.323 INFO:tasks.workunit.client.0.smithi042.stdout:4/335: rename de/l16 to de/d2e/d37/d3c/l76 0 2024-09-16T23:48:53.323 INFO:tasks.workunit.client.0.smithi042.stdout:6/205: rmdir d8 39 2024-09-16T23:48:53.323 INFO:tasks.workunit.client.0.smithi042.stdout:5/314: creat d9/db/d11/f66 x:0 0 0 2024-09-16T23:48:53.325 INFO:tasks.workunit.client.0.smithi042.stdout:4/336: write de/f5a [3186883,57409] 0 2024-09-16T23:48:53.327 INFO:tasks.workunit.client.0.smithi042.stdout:5/315: creat d9/db/de/d5e/f67 x:0 0 0 2024-09-16T23:48:53.327 INFO:tasks.workunit.client.0.smithi042.stdout:4/337: dread de/f31 [0,4194304] 0 2024-09-16T23:48:53.330 INFO:tasks.workunit.client.0.smithi042.stdout:4/338: write de/d12/d1b/f36 [838602,12771] 0 2024-09-16T23:48:53.333 INFO:tasks.workunit.client.0.smithi042.stdout:6/206: fdatasync d8/db/dd/d1b/d2d/f3e 0 2024-09-16T23:48:53.334 INFO:tasks.workunit.client.0.smithi042.stdout:6/207: readlink d8/db/lf 0 2024-09-16T23:48:53.336 INFO:tasks.workunit.client.0.smithi042.stdout:4/339: creat de/d2e/f77 x:0 0 0 2024-09-16T23:48:53.336 INFO:tasks.workunit.client.0.smithi042.stdout:5/316: link d9/db/l35 d9/db/d19/d60/l68 0 2024-09-16T23:48:53.337 INFO:tasks.workunit.client.0.smithi042.stdout:1/275: truncate d0/d17/f4f 1854314 0 2024-09-16T23:48:53.340 INFO:tasks.workunit.client.0.smithi042.stdout:1/276: chown d0/d9/f11 965082334 1 2024-09-16T23:48:53.341 INFO:tasks.workunit.client.0.smithi042.stdout:4/340: rename de/f19 to de/d12/d1b/f78 0 2024-09-16T23:48:53.343 INFO:tasks.workunit.client.0.smithi042.stdout:4/341: write de/d2e/d37/d4c/d4f/f58 [1627717,115615] 0 2024-09-16T23:48:53.344 INFO:tasks.workunit.client.0.smithi042.stdout:5/317: dread d9/db/f28 [0,4194304] 0 2024-09-16T23:48:53.350 INFO:tasks.workunit.client.0.smithi042.stdout:4/342: write de/d1a/f4b [198354,31137] 0 2024-09-16T23:48:53.350 INFO:tasks.workunit.client.0.smithi042.stdout:2/231: unlink d0/da/d17/c34 0 2024-09-16T23:48:53.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/208: dread d8/db/f12 [0,4194304] 0 2024-09-16T23:48:53.353 INFO:tasks.workunit.client.0.smithi042.stdout:1/277: getdents d0/d9/d26 0 2024-09-16T23:48:53.356 INFO:tasks.workunit.client.0.smithi042.stdout:6/209: creat d8/db/dd/d1b/d2d/f41 x:0 0 0 2024-09-16T23:48:53.358 INFO:tasks.workunit.client.0.smithi042.stdout:1/278: mkdir d0/d9/d35/d46/d63 0 2024-09-16T23:48:53.359 INFO:tasks.workunit.client.0.smithi042.stdout:4/343: dwrite de/d2e/d37/d3e/f40 [0,4194304] 0 2024-09-16T23:48:53.361 INFO:tasks.workunit.client.0.smithi042.stdout:4/344: dread - de/d12/d1b/f59 zero size 2024-09-16T23:48:53.361 INFO:tasks.workunit.client.1.smithi043.stdout:2/355: dread d8/d1d/d20/d23/f79 [0,4194304] 0 2024-09-16T23:48:53.362 INFO:tasks.workunit.client.0.smithi042.stdout:2/232: creat d0/da/d1b/d3b/f56 x:0 0 0 2024-09-16T23:48:53.362 INFO:tasks.workunit.client.0.smithi042.stdout:6/210: mkdir d8/d38/d42 0 2024-09-16T23:48:53.362 INFO:tasks.workunit.client.1.smithi043.stdout:4/397: chown d8/d15/d20/d2b/d23/d27/c34 1221357 1 2024-09-16T23:48:53.362 INFO:tasks.workunit.client.1.smithi043.stdout:9/369: dwrite d4/d1c/d6b/f3c [0,4194304] 0 2024-09-16T23:48:53.363 INFO:tasks.workunit.client.0.smithi042.stdout:4/345: write de/d2e/d37/f3b [462590,68201] 0 2024-09-16T23:48:53.363 INFO:tasks.workunit.client.1.smithi043.stdout:4/398: fdatasync d8/f68 0 2024-09-16T23:48:53.364 INFO:tasks.workunit.client.1.smithi043.stdout:7/412: mkdir d4/db/d43/d83/d85/d8b 0 2024-09-16T23:48:53.365 INFO:tasks.workunit.client.0.smithi042.stdout:6/211: dread - d8/db/dd/d1b/d34/f3d zero size 2024-09-16T23:48:53.365 INFO:tasks.workunit.client.0.smithi042.stdout:1/279: symlink d0/d3b/l64 0 2024-09-16T23:48:53.367 INFO:tasks.workunit.client.1.smithi043.stdout:8/392: link l6 de/d18/d27/d41/d5a/d37/l7a 0 2024-09-16T23:48:53.368 INFO:tasks.workunit.client.1.smithi043.stdout:2/356: creat d8/d1d/d20/d26/f7d x:0 0 0 2024-09-16T23:48:53.369 INFO:tasks.workunit.client.0.smithi042.stdout:2/233: creat d0/da/d1b/d3b/d44/d54/f57 x:0 0 0 2024-09-16T23:48:53.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/346: mknod de/d12/d1b/c79 0 2024-09-16T23:48:53.372 INFO:tasks.workunit.client.0.smithi042.stdout:1/280: rmdir d0/d3b 39 2024-09-16T23:48:53.372 INFO:tasks.workunit.client.0.smithi042.stdout:4/347: chown de/d12/d1b/f35 9 1 2024-09-16T23:48:53.373 INFO:tasks.workunit.client.0.smithi042.stdout:4/348: write de/f10 [918995,42527] 0 2024-09-16T23:48:53.373 INFO:tasks.workunit.client.1.smithi043.stdout:1/524: dwrite d4/d9/d12/d2b/d3d/d83/fa8 [0,4194304] 0 2024-09-16T23:48:53.375 INFO:tasks.workunit.client.1.smithi043.stdout:1/525: readlink d4/d34/lad 0 2024-09-16T23:48:53.376 INFO:tasks.workunit.client.0.smithi042.stdout:2/234: rename d0/da/f49 to d0/da/d1b/d48/f58 0 2024-09-16T23:48:53.378 INFO:tasks.workunit.client.1.smithi043.stdout:7/413: dwrite d4/db/d11/d21/d31/f88 [0,4194304] 0 2024-09-16T23:48:53.379 INFO:tasks.workunit.client.0.smithi042.stdout:1/281: write d0/d9/d26/f27 [6105468,86583] 0 2024-09-16T23:48:53.381 INFO:tasks.workunit.client.1.smithi043.stdout:2/357: fsync d8/d1d/d20/d23/f79 0 2024-09-16T23:48:53.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/349: dread de/d1a/f39 [0,4194304] 0 2024-09-16T23:48:53.386 INFO:tasks.workunit.client.0.smithi042.stdout:4/350: creat de/d2e/d37/d4c/d4f/f7a x:0 0 0 2024-09-16T23:48:53.387 INFO:tasks.workunit.client.0.smithi042.stdout:4/351: dread - de/d2e/d37/f65 zero size 2024-09-16T23:48:53.389 INFO:tasks.workunit.client.1.smithi043.stdout:9/370: truncate d4/d1c/d4f/f5b 1831413 0 2024-09-16T23:48:53.395 INFO:tasks.workunit.client.1.smithi043.stdout:1/526: mknod d4/d9/d1c/d1e/d47/d97/cb2 0 2024-09-16T23:48:53.395 INFO:tasks.workunit.client.0.smithi042.stdout:4/352: dwrite de/d1a/d1c/f21 [0,4194304] 0 2024-09-16T23:48:53.396 INFO:tasks.workunit.client.1.smithi043.stdout:1/527: chown d4/d34 490173 1 2024-09-16T23:48:53.398 INFO:tasks.workunit.client.0.smithi042.stdout:2/235: link d0/da/d17/f37 d0/da/d17/f59 0 2024-09-16T23:48:53.399 INFO:tasks.workunit.client.0.smithi042.stdout:4/353: mknod de/d2e/d37/d4c/c7b 0 2024-09-16T23:48:53.401 INFO:tasks.workunit.client.1.smithi043.stdout:7/414: rename d4/db/d11/d21/d31/d33/f5b to d4/d27/d41/d1b/d66/f8c 0 2024-09-16T23:48:53.406 INFO:tasks.workunit.client.1.smithi043.stdout:7/415: dread d4/db/d11/f13 [0,4194304] 0 2024-09-16T23:48:53.406 INFO:tasks.workunit.client.1.smithi043.stdout:8/393: mkdir de/d18/d1f/d2f/d39/d7b 0 2024-09-16T23:48:53.408 INFO:tasks.workunit.client.0.smithi042.stdout:2/236: dwrite d0/da/f2c [0,4194304] 0 2024-09-16T23:48:53.411 INFO:tasks.workunit.client.1.smithi043.stdout:4/399: link d8/d25/d39/c3e d8/d15/d20/d2b/c78 0 2024-09-16T23:48:53.411 INFO:tasks.workunit.client.0.smithi042.stdout:4/354: dwrite de/d1a/f55 [0,4194304] 0 2024-09-16T23:48:53.413 INFO:tasks.workunit.client.1.smithi043.stdout:4/400: read d8/d25/f6a [977047,69687] 0 2024-09-16T23:48:53.414 INFO:tasks.workunit.client.1.smithi043.stdout:9/371: fdatasync d4/d21/f4d 0 2024-09-16T23:48:53.421 INFO:tasks.workunit.client.0.smithi042.stdout:1/282: write d0/d17/f4f [1619288,72682] 0 2024-09-16T23:48:53.422 INFO:tasks.workunit.client.0.smithi042.stdout:1/283: stat d0/d9/d35/l38 0 2024-09-16T23:48:53.423 INFO:tasks.workunit.client.0.smithi042.stdout:4/355: rename de/d12/d1b/f35 to de/d2e/d37/f7c 0 2024-09-16T23:48:53.424 INFO:tasks.workunit.client.0.smithi042.stdout:4/356: fdatasync de/d1a/d1c/f47 0 2024-09-16T23:48:53.425 INFO:tasks.workunit.client.1.smithi043.stdout:1/528: dwrite d4/d9/d12/d2b/fb0 [0,4194304] 0 2024-09-16T23:48:53.426 INFO:tasks.workunit.client.0.smithi042.stdout:2/237: link d0/da/d1b/c45 d0/da/d1b/d48/c5a 0 2024-09-16T23:48:53.426 INFO:tasks.workunit.client.0.smithi042.stdout:4/357: fdatasync de/d2e/d37/d4c/d4f/f64 0 2024-09-16T23:48:53.428 INFO:tasks.workunit.client.1.smithi043.stdout:2/358: rename d8/c14 to d8/d4b/d5f/d63/c7e 0 2024-09-16T23:48:53.430 INFO:tasks.workunit.client.0.smithi042.stdout:4/358: mkdir de/d12/d7d 0 2024-09-16T23:48:53.431 INFO:tasks.workunit.client.0.smithi042.stdout:2/238: dread - d0/f3a zero size 2024-09-16T23:48:53.432 INFO:tasks.workunit.client.0.smithi042.stdout:4/359: fsync de/d1a/d70/f71 0 2024-09-16T23:48:53.436 INFO:tasks.workunit.client.1.smithi043.stdout:8/394: dread de/d18/d1f/d25/f5f [0,4194304] 0 2024-09-16T23:48:53.437 INFO:tasks.workunit.client.1.smithi043.stdout:1/529: creat d4/d9/d12/d2b/d3d/fb3 x:0 0 0 2024-09-16T23:48:53.438 INFO:tasks.workunit.client.0.smithi042.stdout:4/360: unlink de/d1a/l2b 0 2024-09-16T23:48:53.438 INFO:tasks.workunit.client.1.smithi043.stdout:1/530: stat d4/d9/d1c/d1e/d47 0 2024-09-16T23:48:53.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/239: dwrite d0/da/d17/f52 [0,4194304] 0 2024-09-16T23:48:53.441 INFO:tasks.workunit.client.0.smithi042.stdout:4/361: fdatasync de/d2e/d37/f3b 0 2024-09-16T23:48:53.442 INFO:tasks.workunit.client.1.smithi043.stdout:9/372: symlink d4/d1c/d25/d2b/d4c/d50/l6e 0 2024-09-16T23:48:53.443 INFO:tasks.workunit.client.0.smithi042.stdout:2/240: stat d0/da/d1b/d3b/d44/l4b 0 2024-09-16T23:48:53.443 INFO:tasks.workunit.client.0.smithi042.stdout:1/284: dread d0/d17/f4f [0,4194304] 0 2024-09-16T23:48:53.445 INFO:tasks.workunit.client.1.smithi043.stdout:2/359: symlink d8/l7f 0 2024-09-16T23:48:53.445 INFO:tasks.workunit.client.0.smithi042.stdout:2/241: write d0/da/d1b/d3b/f56 [788676,46280] 0 2024-09-16T23:48:53.445 INFO:tasks.workunit.client.0.smithi042.stdout:4/362: symlink de/l7e 0 2024-09-16T23:48:53.447 INFO:tasks.workunit.client.1.smithi043.stdout:9/373: stat d4/lc 0 2024-09-16T23:48:53.447 INFO:tasks.workunit.client.1.smithi043.stdout:9/374: stat c0 0 2024-09-16T23:48:53.447 INFO:tasks.workunit.client.0.smithi042.stdout:1/285: fsync d0/d9/f56 0 2024-09-16T23:48:53.448 INFO:tasks.workunit.client.1.smithi043.stdout:9/375: fsync d4/d1c/d25/d2b/d4c/d50/f66 0 2024-09-16T23:48:53.451 INFO:tasks.workunit.client.0.smithi042.stdout:4/363: rename de/d12/d1b/f2d to de/d2e/d37/d3e/f7f 0 2024-09-16T23:48:53.453 INFO:tasks.workunit.client.1.smithi043.stdout:2/360: chown d8/d1f/f2e 589 1 2024-09-16T23:48:53.453 INFO:tasks.workunit.client.1.smithi043.stdout:9/376: dread d4/d21/f63 [0,4194304] 0 2024-09-16T23:48:53.454 INFO:tasks.workunit.client.0.smithi042.stdout:4/364: symlink de/d42/d5d/l80 0 2024-09-16T23:48:53.454 INFO:tasks.workunit.client.1.smithi043.stdout:2/361: write d8/d1d/d20/d26/f3c [38792,123723] 0 2024-09-16T23:48:53.455 INFO:tasks.workunit.client.1.smithi043.stdout:4/401: write d8/d15/d20/d2b/d22/f36 [350684,85697] 0 2024-09-16T23:48:53.456 INFO:tasks.workunit.client.0.smithi042.stdout:1/286: creat d0/d9/d4e/d59/f65 x:0 0 0 2024-09-16T23:48:53.456 INFO:tasks.workunit.client.0.smithi042.stdout:1/287: fdatasync d0/d9/f56 0 2024-09-16T23:48:53.460 INFO:tasks.workunit.client.0.smithi042.stdout:1/288: write d0/d9/d35/d46/f13 [3859696,82052] 0 2024-09-16T23:48:53.460 INFO:tasks.workunit.client.1.smithi043.stdout:2/362: creat d8/d1d/d20/f80 x:0 0 0 2024-09-16T23:48:53.461 INFO:tasks.workunit.client.1.smithi043.stdout:8/395: getdents de/d18/d1f 0 2024-09-16T23:48:53.462 INFO:tasks.workunit.client.1.smithi043.stdout:8/396: chown de/d18/d1f/f2d 82565 1 2024-09-16T23:48:53.465 INFO:tasks.workunit.client.0.smithi042.stdout:4/365: dwrite de/d1a/d1c/f56 [0,4194304] 0 2024-09-16T23:48:53.468 INFO:tasks.workunit.client.0.smithi042.stdout:2/242: getdents d0/d46 0 2024-09-16T23:48:53.469 INFO:tasks.workunit.client.1.smithi043.stdout:4/402: mknod d8/d15/d20/d2b/d54/d55/d73/c79 0 2024-09-16T23:48:53.470 INFO:tasks.workunit.client.0.smithi042.stdout:2/243: chown d0/da/d17/f1c 834202 1 2024-09-16T23:48:53.474 INFO:tasks.workunit.client.0.smithi042.stdout:4/366: truncate de/d12/f54 3166318 0 2024-09-16T23:48:53.474 INFO:tasks.workunit.client.0.smithi042.stdout:2/244: dread d0/f22 [0,4194304] 0 2024-09-16T23:48:53.476 INFO:tasks.workunit.client.1.smithi043.stdout:4/403: dwrite d8/d15/d20/d2b/f28 [8388608,4194304] 0 2024-09-16T23:48:53.480 INFO:tasks.workunit.client.1.smithi043.stdout:4/404: creat d8/d15/d20/d2b/d54/d55/f7a x:0 0 0 2024-09-16T23:48:53.481 INFO:tasks.workunit.client.0.smithi042.stdout:1/289: dread d0/f6 [0,4194304] 0 2024-09-16T23:48:53.481 INFO:tasks.workunit.client.0.smithi042.stdout:4/367: getdents de/d1a/d70 0 2024-09-16T23:48:53.482 INFO:tasks.workunit.client.1.smithi043.stdout:8/397: dread de/d18/d27/d41/d5a/f1b [0,4194304] 0 2024-09-16T23:48:53.483 INFO:tasks.workunit.client.0.smithi042.stdout:2/245: dread d0/da/d17/f42 [0,4194304] 0 2024-09-16T23:48:53.485 INFO:tasks.workunit.client.1.smithi043.stdout:4/405: rmdir d8/d15/d20/d2b/d22 39 2024-09-16T23:48:53.485 INFO:tasks.workunit.client.0.smithi042.stdout:4/368: truncate de/d12/d1b/f59 752068 0 2024-09-16T23:48:53.487 INFO:tasks.workunit.client.0.smithi042.stdout:2/246: write d0/da/d17/f1c [5147168,100822] 0 2024-09-16T23:48:53.489 INFO:tasks.workunit.client.1.smithi043.stdout:8/398: dwrite de/f17 [0,4194304] 0 2024-09-16T23:48:53.489 INFO:tasks.workunit.client.1.smithi043.stdout:4/406: mknod d8/d15/d20/c7b 0 2024-09-16T23:48:53.495 INFO:tasks.workunit.client.1.smithi043.stdout:4/407: unlink d8/d25/l66 0 2024-09-16T23:48:53.497 INFO:tasks.workunit.client.0.smithi042.stdout:4/369: dread de/f5a [0,4194304] 0 2024-09-16T23:48:53.498 INFO:tasks.workunit.client.0.smithi042.stdout:2/247: dwrite d0/da/f1f [0,4194304] 0 2024-09-16T23:48:53.499 INFO:tasks.workunit.client.0.smithi042.stdout:4/370: read de/d1a/f4b [2700522,103048] 0 2024-09-16T23:48:53.503 INFO:tasks.workunit.client.0.smithi042.stdout:4/371: symlink de/d42/d68/l81 0 2024-09-16T23:48:53.508 INFO:tasks.workunit.client.0.smithi042.stdout:4/372: mkdir de/d42/d82 0 2024-09-16T23:48:53.509 INFO:tasks.workunit.client.0.smithi042.stdout:2/248: dwrite d0/da/d1b/d3b/d44/d54/f57 [0,4194304] 0 2024-09-16T23:48:53.512 INFO:tasks.workunit.client.1.smithi043.stdout:8/399: dread de/d18/f36 [0,4194304] 0 2024-09-16T23:48:53.512 INFO:tasks.workunit.client.1.smithi043.stdout:8/400: chown de/d18/d1f/d2f/d39/d4d/l5e 6407717 1 2024-09-16T23:48:53.513 INFO:tasks.workunit.client.0.smithi042.stdout:1/290: truncate d0/d9/d26/f27 1431042 0 2024-09-16T23:48:53.523 INFO:tasks.workunit.client.0.smithi042.stdout:1/291: dwrite d0/d17/f20 [0,4194304] 0 2024-09-16T23:48:53.523 INFO:tasks.workunit.client.0.smithi042.stdout:2/249: unlink d0/da/fd 0 2024-09-16T23:48:53.524 INFO:tasks.workunit.client.1.smithi043.stdout:4/408: dwrite d8/d15/d20/d2b/d54/d55/f2a [0,4194304] 0 2024-09-16T23:48:53.526 INFO:tasks.workunit.client.0.smithi042.stdout:2/250: dread - d0/d30/f47 zero size 2024-09-16T23:48:53.527 INFO:tasks.workunit.client.0.smithi042.stdout:4/373: dwrite de/d2e/d37/d4c/f5f [0,4194304] 0 2024-09-16T23:48:53.528 INFO:tasks.workunit.client.0.smithi042.stdout:2/251: dread - d0/d30/f47 zero size 2024-09-16T23:48:53.528 INFO:tasks.workunit.client.1.smithi043.stdout:8/401: dread de/d18/d27/d41/d5a/d37/f69 [4194304,4194304] 0 2024-09-16T23:48:53.529 INFO:tasks.workunit.client.1.smithi043.stdout:4/409: mknod d8/d15/d20/d2b/d23/c7c 0 2024-09-16T23:48:53.533 INFO:tasks.workunit.client.1.smithi043.stdout:4/410: fdatasync d8/f1d 0 2024-09-16T23:48:53.535 INFO:tasks.workunit.client.1.smithi043.stdout:4/411: write d8/f5d [603249,61122] 0 2024-09-16T23:48:53.537 INFO:tasks.workunit.client.0.smithi042.stdout:4/374: link de/d12/d1b/f59 de/d12/d7d/f83 0 2024-09-16T23:48:53.538 INFO:tasks.workunit.client.1.smithi043.stdout:4/412: creat d8/d25/d64/f7d x:0 0 0 2024-09-16T23:48:53.538 INFO:tasks.workunit.client.0.smithi042.stdout:4/375: write de/d42/f44 [104435,80518] 0 2024-09-16T23:48:53.540 INFO:tasks.workunit.client.0.smithi042.stdout:2/252: rename d0/d46 to d0/da/d5b 0 2024-09-16T23:48:53.541 INFO:tasks.workunit.client.1.smithi043.stdout:3/416: sync 2024-09-16T23:48:53.541 INFO:tasks.workunit.client.1.smithi043.stdout:5/556: sync 2024-09-16T23:48:53.542 INFO:tasks.workunit.client.1.smithi043.stdout:3/417: fsync d4/de/d35/f1d 0 2024-09-16T23:48:53.546 INFO:tasks.workunit.client.1.smithi043.stdout:4/413: read d8/d15/d20/d2b/d22/f24 [5575416,115435] 0 2024-09-16T23:48:53.548 INFO:tasks.workunit.client.1.smithi043.stdout:3/418: fdatasync d4/f67 0 2024-09-16T23:48:53.549 INFO:tasks.workunit.client.0.smithi042.stdout:4/376: dwrite de/d2e/d37/d3e/f7f [0,4194304] 0 2024-09-16T23:48:53.550 INFO:tasks.workunit.client.1.smithi043.stdout:8/402: dread de/f23 [0,4194304] 0 2024-09-16T23:48:53.550 INFO:tasks.workunit.client.1.smithi043.stdout:5/557: dwrite d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:53.556 INFO:tasks.workunit.client.0.smithi042.stdout:4/377: rename de/d1a/d1c/c60 to de/d2e/d37/d3e/c84 0 2024-09-16T23:48:53.560 INFO:tasks.workunit.client.0.smithi042.stdout:4/378: dread de/d2e/d37/f3b [0,4194304] 0 2024-09-16T23:48:53.561 INFO:tasks.workunit.client.0.smithi042.stdout:2/253: chown d0/da/f23 250 1 2024-09-16T23:48:53.561 INFO:tasks.workunit.client.0.smithi042.stdout:4/379: fdatasync de/d2e/d37/f65 0 2024-09-16T23:48:53.563 INFO:tasks.workunit.client.0.smithi042.stdout:4/380: write de/d42/f44 [521094,122573] 0 2024-09-16T23:48:53.564 INFO:tasks.workunit.client.0.smithi042.stdout:2/254: dread d0/da/f21 [0,4194304] 0 2024-09-16T23:48:53.564 INFO:tasks.workunit.client.1.smithi043.stdout:4/414: rename d8/d15/d20/d2b/d23/l44 to d8/d15/d20/d2b/d54/d55/l7e 0 2024-09-16T23:48:53.564 INFO:tasks.workunit.client.1.smithi043.stdout:4/415: dread - d8/d25/d64/f7d zero size 2024-09-16T23:48:53.565 INFO:tasks.workunit.client.0.smithi042.stdout:4/381: chown de/f31 1282681549 1 2024-09-16T23:48:53.566 INFO:tasks.workunit.client.0.smithi042.stdout:4/382: stat de/d1a/f4b 0 2024-09-16T23:48:53.566 INFO:tasks.workunit.client.1.smithi043.stdout:8/403: dread de/d18/d1f/d25/f6b [0,4194304] 0 2024-09-16T23:48:53.566 INFO:tasks.workunit.client.0.smithi042.stdout:2/255: readlink d0/l9 0 2024-09-16T23:48:53.569 INFO:tasks.workunit.client.0.smithi042.stdout:4/383: unlink de/d12/d1b/f59 0 2024-09-16T23:48:53.570 INFO:tasks.workunit.client.0.smithi042.stdout:4/384: dread - de/d42/d5d/f74 zero size 2024-09-16T23:48:53.570 INFO:tasks.workunit.client.1.smithi043.stdout:4/416: creat d8/d25/d64/f7f x:0 0 0 2024-09-16T23:48:53.571 INFO:tasks.workunit.client.0.smithi042.stdout:4/385: stat de/d1a/f4b 0 2024-09-16T23:48:53.571 INFO:tasks.workunit.client.1.smithi043.stdout:4/417: fsync d8/d15/d20/d2b/f26 0 2024-09-16T23:48:53.572 INFO:tasks.workunit.client.0.smithi042.stdout:2/256: mkdir d0/da/d17/d5c 0 2024-09-16T23:48:53.572 INFO:tasks.workunit.client.1.smithi043.stdout:3/419: write d4/de/d1f/f3e [1042463,42707] 0 2024-09-16T23:48:53.576 INFO:tasks.workunit.client.1.smithi043.stdout:4/418: symlink d8/d25/l80 0 2024-09-16T23:48:53.577 INFO:tasks.workunit.client.1.smithi043.stdout:8/404: symlink de/l7c 0 2024-09-16T23:48:53.577 INFO:tasks.workunit.client.1.smithi043.stdout:5/558: getdents d3/d9/d6f/d7c 0 2024-09-16T23:48:53.578 INFO:tasks.workunit.client.0.smithi042.stdout:2/257: mkdir d0/d5d 0 2024-09-16T23:48:53.589 INFO:tasks.workunit.client.1.smithi043.stdout:4/419: dread f7 [0,4194304] 0 2024-09-16T23:48:53.591 INFO:tasks.workunit.client.1.smithi043.stdout:8/405: dwrite de/f71 [0,4194304] 0 2024-09-16T23:48:53.592 INFO:tasks.workunit.client.1.smithi043.stdout:5/559: dwrite d3/db/d15/f3c [0,4194304] 0 2024-09-16T23:48:53.594 INFO:tasks.workunit.client.1.smithi043.stdout:5/560: dread - d3/d9/d6f/d85/f98 zero size 2024-09-16T23:48:53.596 INFO:tasks.workunit.client.1.smithi043.stdout:5/561: stat d3/db/d19/d3e/c93 0 2024-09-16T23:48:53.596 INFO:tasks.workunit.client.1.smithi043.stdout:8/406: symlink de/d18/d1f/d2f/d39/d7b/l7d 0 2024-09-16T23:48:53.597 INFO:tasks.workunit.client.0.smithi042.stdout:2/258: dread d0/f11 [0,4194304] 0 2024-09-16T23:48:53.599 INFO:tasks.workunit.client.1.smithi043.stdout:5/562: fdatasync d3/db/d26/f33 0 2024-09-16T23:48:53.602 INFO:tasks.workunit.client.1.smithi043.stdout:8/407: creat de/d70/d74/f7e x:0 0 0 2024-09-16T23:48:53.610 INFO:tasks.workunit.client.1.smithi043.stdout:8/408: mkdir de/d18/d1f/d25/d65/d7f 0 2024-09-16T23:48:53.612 INFO:tasks.workunit.client.1.smithi043.stdout:5/563: dread d3/f38 [0,4194304] 0 2024-09-16T23:48:53.612 INFO:tasks.workunit.client.1.smithi043.stdout:8/409: chown de/d18/d27/c47 7 1 2024-09-16T23:48:53.613 INFO:tasks.workunit.client.0.smithi042.stdout:2/259: truncate d0/da/d17/f1c 5814419 0 2024-09-16T23:48:53.614 INFO:tasks.workunit.client.0.smithi042.stdout:2/260: chown d0/da/d1b/l25 6375331 1 2024-09-16T23:48:53.615 INFO:tasks.workunit.client.1.smithi043.stdout:8/410: creat de/d70/d74/f80 x:0 0 0 2024-09-16T23:48:53.616 INFO:tasks.workunit.client.0.smithi042.stdout:2/261: creat d0/da/d1b/d3b/d44/d54/f5e x:0 0 0 2024-09-16T23:48:53.618 INFO:tasks.workunit.client.1.smithi043.stdout:5/564: creat d3/d9/d22/d4a/fb1 x:0 0 0 2024-09-16T23:48:53.621 INFO:tasks.workunit.client.1.smithi043.stdout:8/411: creat de/d18/d1f/d2f/d39/f81 x:0 0 0 2024-09-16T23:48:53.622 INFO:tasks.workunit.client.0.smithi042.stdout:2/262: fdatasync d0/d30/f38 0 2024-09-16T23:48:53.623 INFO:tasks.workunit.client.0.smithi042.stdout:2/263: chown d0/da/d1b/d3b/d44 82 1 2024-09-16T23:48:53.625 INFO:tasks.workunit.client.0.smithi042.stdout:2/264: chown d0/da/d1b/d48/f58 228595044 1 2024-09-16T23:48:53.627 INFO:tasks.workunit.client.1.smithi043.stdout:8/412: dread de/d18/d1f/d25/f3b [0,4194304] 0 2024-09-16T23:48:53.629 INFO:tasks.workunit.client.0.smithi042.stdout:2/265: chown d0/c8 656 1 2024-09-16T23:48:53.630 INFO:tasks.workunit.client.1.smithi043.stdout:8/413: mkdir de/d18/d1f/d25/d82 0 2024-09-16T23:48:53.632 INFO:tasks.workunit.client.1.smithi043.stdout:5/565: dread d3/db/d19/d3e/f2d [0,4194304] 0 2024-09-16T23:48:53.633 INFO:tasks.workunit.client.1.smithi043.stdout:5/566: rename d3/d9 to d3/d9/d6f/d7c/db2 22 2024-09-16T23:48:53.634 INFO:tasks.workunit.client.1.smithi043.stdout:8/414: mkdir de/d18/d1f/d2f/d39/d7b/d83 0 2024-09-16T23:48:53.635 INFO:tasks.workunit.client.1.smithi043.stdout:5/567: symlink d3/d79/lb3 0 2024-09-16T23:48:53.638 INFO:tasks.workunit.client.0.smithi042.stdout:2/266: dwrite d0/d26/f4e [0,4194304] 0 2024-09-16T23:48:53.639 INFO:tasks.workunit.client.1.smithi043.stdout:8/415: creat de/d18/d1f/d2f/d39/d7b/d83/f84 x:0 0 0 2024-09-16T23:48:53.642 INFO:tasks.workunit.client.0.smithi042.stdout:2/267: stat d0/l9 0 2024-09-16T23:48:53.646 INFO:tasks.workunit.client.1.smithi043.stdout:8/416: dwrite de/d18/d27/d41/f50 [0,4194304] 0 2024-09-16T23:48:53.647 INFO:tasks.workunit.client.1.smithi043.stdout:8/417: write de/d18/d1f/f32 [941262,102371] 0 2024-09-16T23:48:53.648 INFO:tasks.workunit.client.0.smithi042.stdout:2/268: getdents d0/da/d1b 0 2024-09-16T23:48:53.651 INFO:tasks.workunit.client.1.smithi043.stdout:5/568: write d3/db/d19/d3e/f35 [4312231,79179] 0 2024-09-16T23:48:53.655 INFO:tasks.workunit.client.1.smithi043.stdout:5/569: getdents d3/d79/d91/daf 0 2024-09-16T23:48:53.657 INFO:tasks.workunit.client.1.smithi043.stdout:8/418: creat de/d18/d27/f85 x:0 0 0 2024-09-16T23:48:53.659 INFO:tasks.workunit.client.1.smithi043.stdout:5/570: dread d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:53.660 INFO:tasks.workunit.client.1.smithi043.stdout:8/419: mknod de/d18/d1f/d25/c86 0 2024-09-16T23:48:53.660 INFO:tasks.workunit.client.0.smithi042.stdout:2/269: dwrite d0/d30/f38 [0,4194304] 0 2024-09-16T23:48:53.661 INFO:tasks.workunit.client.1.smithi043.stdout:8/420: dread de/d18/d1f/d25/f6b [0,4194304] 0 2024-09-16T23:48:53.662 INFO:tasks.workunit.client.1.smithi043.stdout:5/571: symlink d3/db/d15/lb4 0 2024-09-16T23:48:53.664 INFO:tasks.workunit.client.0.smithi042.stdout:7/316: sync 2024-09-16T23:48:53.665 INFO:tasks.workunit.client.0.smithi042.stdout:2/270: creat d0/da/d1b/d3b/d44/d54/f5f x:0 0 0 2024-09-16T23:48:53.668 INFO:tasks.workunit.client.1.smithi043.stdout:5/572: unlink d3/db/f52 0 2024-09-16T23:48:53.668 INFO:tasks.workunit.client.0.smithi042.stdout:7/317: mknod d3/d15/d32/d6b/d5b/c6d 0 2024-09-16T23:48:53.670 INFO:tasks.workunit.client.1.smithi043.stdout:8/421: getdents de/d18/d1f/d2f/d39/d7b/d83 0 2024-09-16T23:48:53.671 INFO:tasks.workunit.client.1.smithi043.stdout:5/573: symlink d3/db/d19/d76/lb5 0 2024-09-16T23:48:53.671 INFO:tasks.workunit.client.1.smithi043.stdout:5/574: stat d3/f96 0 2024-09-16T23:48:53.673 INFO:tasks.workunit.client.1.smithi043.stdout:8/422: mknod de/d18/d1f/d2f/d39/d7b/c87 0 2024-09-16T23:48:53.674 INFO:tasks.workunit.client.1.smithi043.stdout:5/575: mkdir d3/db/d84/db6 0 2024-09-16T23:48:53.677 INFO:tasks.workunit.client.1.smithi043.stdout:6/425: sync 2024-09-16T23:48:53.677 INFO:tasks.workunit.client.1.smithi043.stdout:0/405: sync 2024-09-16T23:48:53.678 INFO:tasks.workunit.client.0.smithi042.stdout:7/318: write d3/d1f/f26 [12144,118708] 0 2024-09-16T23:48:53.679 INFO:tasks.workunit.client.1.smithi043.stdout:8/423: dwrite de/d18/d1f/d2f/d39/d4d/f5d [0,4194304] 0 2024-09-16T23:48:53.681 INFO:tasks.workunit.client.1.smithi043.stdout:5/576: unlink d3/d9/d22/d60/f63 0 2024-09-16T23:48:53.682 INFO:tasks.workunit.client.1.smithi043.stdout:8/424: dread - de/d18/d1f/d25/f6e zero size 2024-09-16T23:48:53.683 INFO:tasks.workunit.client.0.smithi042.stdout:7/319: dread d3/d15/f1b [0,4194304] 0 2024-09-16T23:48:53.684 INFO:tasks.workunit.client.1.smithi043.stdout:6/426: write de/d1a/d4d/f88 [1947642,73221] 0 2024-09-16T23:48:53.685 INFO:tasks.workunit.client.0.smithi042.stdout:7/320: write d3/d15/d16/d19/f21 [4686858,35777] 0 2024-09-16T23:48:53.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/321: write d3/d1f/f61 [2270824,55112] 0 2024-09-16T23:48:53.689 INFO:tasks.workunit.client.1.smithi043.stdout:6/427: write de/d1a/d4d/f88 [2688548,66628] 0 2024-09-16T23:48:53.691 INFO:tasks.workunit.client.1.smithi043.stdout:0/406: dwrite d0/d2/df/d1a/d3b/f2a [0,4194304] 0 2024-09-16T23:48:53.691 INFO:tasks.workunit.client.1.smithi043.stdout:6/428: readlink de/d23/d34/l3f 0 2024-09-16T23:48:53.694 INFO:tasks.workunit.client.1.smithi043.stdout:5/577: creat d3/db/d15/d43/fb7 x:0 0 0 2024-09-16T23:48:53.695 INFO:tasks.workunit.client.1.smithi043.stdout:6/429: dread - de/d1a/d28/f77 zero size 2024-09-16T23:48:53.696 INFO:tasks.workunit.client.1.smithi043.stdout:0/407: rmdir d0/d2/df/d1a/d49 39 2024-09-16T23:48:53.697 INFO:tasks.workunit.client.0.smithi042.stdout:7/322: creat d3/d15/d24/f6e x:0 0 0 2024-09-16T23:48:53.698 INFO:tasks.workunit.client.1.smithi043.stdout:0/408: dread - d0/d2/df/d1a/d3b/f5d zero size 2024-09-16T23:48:53.699 INFO:tasks.workunit.client.1.smithi043.stdout:8/425: dwrite de/d18/d27/d41/d5a/f1b [0,4194304] 0 2024-09-16T23:48:53.701 INFO:tasks.workunit.client.1.smithi043.stdout:8/426: fdatasync de/f45 0 2024-09-16T23:48:53.703 INFO:tasks.workunit.client.1.smithi043.stdout:6/430: dwrite de/f6c [0,4194304] 0 2024-09-16T23:48:53.703 INFO:tasks.workunit.client.1.smithi043.stdout:8/427: readlink de/d18/d27/d41/d5a/l38 0 2024-09-16T23:48:53.703 INFO:tasks.workunit.client.1.smithi043.stdout:5/578: rename d3/d1b/f97 to d3/da4/fb8 0 2024-09-16T23:48:53.708 INFO:tasks.workunit.client.0.smithi042.stdout:7/323: dwrite d3/d15/d16/f1e [0,4194304] 0 2024-09-16T23:48:53.708 INFO:tasks.workunit.client.1.smithi043.stdout:0/409: dwrite d0/de/f69 [0,4194304] 0 2024-09-16T23:48:53.713 INFO:tasks.workunit.client.1.smithi043.stdout:6/431: dread de/f13 [0,4194304] 0 2024-09-16T23:48:53.715 INFO:tasks.workunit.client.1.smithi043.stdout:5/579: read d3/d9/f4b [290495,122779] 0 2024-09-16T23:48:53.715 INFO:tasks.workunit.client.1.smithi043.stdout:8/428: unlink l7 0 2024-09-16T23:48:53.716 INFO:tasks.workunit.client.1.smithi043.stdout:0/410: stat d0/d2/df/f10 0 2024-09-16T23:48:53.716 INFO:tasks.workunit.client.1.smithi043.stdout:8/429: chown de/d18/d1f/d25/d65 3789 1 2024-09-16T23:48:53.717 INFO:tasks.workunit.client.0.smithi042.stdout:7/324: rename d3/d15/d16/d19/c31 to d3/d15/d32/c6f 0 2024-09-16T23:48:53.717 INFO:tasks.workunit.client.1.smithi043.stdout:5/580: chown d3/db/d19/d76/c9a 3666430 1 2024-09-16T23:48:53.718 INFO:tasks.workunit.client.0.smithi042.stdout:7/325: chown d3/d15/d16/c2b 13014835 1 2024-09-16T23:48:53.726 INFO:tasks.workunit.client.1.smithi043.stdout:0/411: unlink d0/d2/df/d1a/d3b/f4e 0 2024-09-16T23:48:53.728 INFO:tasks.workunit.client.0.smithi042.stdout:7/326: dwrite d3/d15/f1b [4194304,4194304] 0 2024-09-16T23:48:53.730 INFO:tasks.workunit.client.0.smithi042.stdout:7/327: write d3/d15/d41/f62 [91895,106202] 0 2024-09-16T23:48:53.733 INFO:tasks.workunit.client.1.smithi043.stdout:6/432: truncate de/f6c 1087173 0 2024-09-16T23:48:53.734 INFO:tasks.workunit.client.0.smithi042.stdout:7/328: creat d3/d15/d16/d19/d63/f70 x:0 0 0 2024-09-16T23:48:53.737 INFO:tasks.workunit.client.1.smithi043.stdout:5/581: symlink d3/d9/d22/d4a/d4c/d8c/lb9 0 2024-09-16T23:48:53.737 INFO:tasks.workunit.client.0.smithi042.stdout:7/329: creat d3/d15/d24/d4d/f71 x:0 0 0 2024-09-16T23:48:53.740 INFO:tasks.workunit.client.0.smithi042.stdout:7/330: symlink d3/l72 0 2024-09-16T23:48:53.741 INFO:tasks.workunit.client.1.smithi043.stdout:0/412: unlink d0/de/f69 0 2024-09-16T23:48:53.743 INFO:tasks.workunit.client.0.smithi042.stdout:7/331: mknod d3/d15/d16/d19/c73 0 2024-09-16T23:48:53.743 INFO:tasks.workunit.client.1.smithi043.stdout:0/413: write d0/d2/df/d1a/d3b/f7d [511018,97056] 0 2024-09-16T23:48:53.746 INFO:tasks.workunit.client.1.smithi043.stdout:6/433: dread de/f7a [0,4194304] 0 2024-09-16T23:48:53.746 INFO:tasks.workunit.client.1.smithi043.stdout:6/434: fdatasync de/f85 0 2024-09-16T23:48:53.747 INFO:tasks.workunit.client.0.smithi042.stdout:7/332: unlink d3/d15/d16/f1e 0 2024-09-16T23:48:53.748 INFO:tasks.workunit.client.1.smithi043.stdout:0/414: rename d0/de/c5f to d0/dc/c89 0 2024-09-16T23:48:53.748 INFO:tasks.workunit.client.0.smithi042.stdout:7/333: mknod d3/d1f/c74 0 2024-09-16T23:48:53.751 INFO:tasks.workunit.client.0.smithi042.stdout:7/334: creat d3/d15/d16/d19/d63/f75 x:0 0 0 2024-09-16T23:48:53.752 INFO:tasks.workunit.client.0.smithi042.stdout:7/335: dread - d3/d15/d32/d6b/f60 zero size 2024-09-16T23:48:53.753 INFO:tasks.workunit.client.1.smithi043.stdout:5/582: dread d3/d9/d22/d4a/d64/f7b [0,4194304] 0 2024-09-16T23:48:53.754 INFO:tasks.workunit.client.0.smithi042.stdout:7/336: write d3/d15/d16/d19/d63/f70 [917814,12330] 0 2024-09-16T23:48:53.754 INFO:tasks.workunit.client.1.smithi043.stdout:6/435: link de/d1a/d28/f3e de/d1a/d4d/d5a/f90 0 2024-09-16T23:48:53.755 INFO:tasks.workunit.client.0.smithi042.stdout:7/337: chown d3/d15/d16/d19/d63 6131342 1 2024-09-16T23:48:53.755 INFO:tasks.workunit.client.1.smithi043.stdout:0/415: mknod d0/d2/df/d1a/d49/d71/c8a 0 2024-09-16T23:48:53.757 INFO:tasks.workunit.client.1.smithi043.stdout:5/583: mkdir d3/db/d19/d76/dba 0 2024-09-16T23:48:53.763 INFO:tasks.workunit.client.1.smithi043.stdout:6/436: symlink de/d1a/d28/l91 0 2024-09-16T23:48:53.764 INFO:tasks.workunit.client.1.smithi043.stdout:5/584: write d3/f17 [7871260,106753] 0 2024-09-16T23:48:53.766 INFO:tasks.workunit.client.0.smithi042.stdout:7/338: rename d3/d15/d16 to d3/d1f/d49/d76 0 2024-09-16T23:48:53.768 INFO:tasks.workunit.client.1.smithi043.stdout:0/416: dwrite d0/d2/df/d1a/d49/d71/f7e [0,4194304] 0 2024-09-16T23:48:53.770 INFO:tasks.workunit.client.0.smithi042.stdout:3/201: sync 2024-09-16T23:48:53.771 INFO:tasks.workunit.client.1.smithi043.stdout:6/437: dwrite de/f47 [0,4194304] 0 2024-09-16T23:48:53.771 INFO:tasks.workunit.client.0.smithi042.stdout:3/202: write d2/d9/dd/d2c/f32 [559439,52695] 0 2024-09-16T23:48:53.772 INFO:tasks.workunit.client.0.smithi042.stdout:7/339: rename d3/d1f/d49/d76/l35 to d3/d15/l77 0 2024-09-16T23:48:53.774 INFO:tasks.workunit.client.0.smithi042.stdout:7/340: read - d3/d15/d32/d6b/f60 zero size 2024-09-16T23:48:53.774 INFO:tasks.workunit.client.1.smithi043.stdout:5/585: creat d3/d9/d22/d8e/fbb x:0 0 0 2024-09-16T23:48:53.776 INFO:tasks.workunit.client.0.smithi042.stdout:3/203: symlink d2/d9/dd/d17/d21/l48 0 2024-09-16T23:48:53.776 INFO:tasks.workunit.client.0.smithi042.stdout:7/341: fdatasync d3/d15/d24/d4d/f71 0 2024-09-16T23:48:53.778 INFO:tasks.workunit.client.1.smithi043.stdout:0/417: mknod d0/d2/df/d1a/d49/d80/c8b 0 2024-09-16T23:48:53.779 INFO:tasks.workunit.client.0.smithi042.stdout:7/342: truncate d3/d1f/d49/d76/d19/d63/f70 1375688 0 2024-09-16T23:48:53.780 INFO:tasks.workunit.client.1.smithi043.stdout:5/586: creat d3/d79/d91/daf/fbc x:0 0 0 2024-09-16T23:48:53.780 INFO:tasks.workunit.client.1.smithi043.stdout:5/587: read - d3/f96 zero size 2024-09-16T23:48:53.781 INFO:tasks.workunit.client.1.smithi043.stdout:0/418: mknod d0/de/c8c 0 2024-09-16T23:48:53.783 INFO:tasks.workunit.client.0.smithi042.stdout:7/343: mkdir d3/d1f/d49/d78 0 2024-09-16T23:48:53.788 INFO:tasks.workunit.client.1.smithi043.stdout:0/419: write d0/d2/df/d1a/d46/d39/d6e/f55 [65322,36680] 0 2024-09-16T23:48:53.788 INFO:tasks.workunit.client.0.smithi042.stdout:3/204: rmdir d2/d9/dd/d17/d21 39 2024-09-16T23:48:53.788 INFO:tasks.workunit.client.0.smithi042.stdout:3/205: chown d2 215 1 2024-09-16T23:48:53.790 INFO:tasks.workunit.client.1.smithi043.stdout:5/588: dwrite d3/d9/d22/d8e/fbb [0,4194304] 0 2024-09-16T23:48:53.790 INFO:tasks.workunit.client.0.smithi042.stdout:3/206: write d2/d9/dd/d17/d21/f2e [1142589,100123] 0 2024-09-16T23:48:53.792 INFO:tasks.workunit.client.1.smithi043.stdout:0/420: symlink d0/l8d 0 2024-09-16T23:48:53.794 INFO:tasks.workunit.client.0.smithi042.stdout:3/207: symlink d2/d9/dd/d2c/l49 0 2024-09-16T23:48:53.800 INFO:tasks.workunit.client.0.smithi042.stdout:3/208: rename d2/d7/ff to d2/f4a 0 2024-09-16T23:48:53.801 INFO:tasks.workunit.client.0.smithi042.stdout:3/209: readlink d2/d9/dd/d2c/l49 0 2024-09-16T23:48:53.807 INFO:tasks.workunit.client.0.smithi042.stdout:3/210: write f0 [3106730,76416] 0 2024-09-16T23:48:53.809 INFO:tasks.workunit.client.0.smithi042.stdout:3/211: symlink d2/d9/dd/l4b 0 2024-09-16T23:48:53.815 INFO:tasks.workunit.client.1.smithi043.stdout:5/589: dwrite d3/db/d19/d3e/d28/f37 [0,4194304] 0 2024-09-16T23:48:53.817 INFO:tasks.workunit.client.1.smithi043.stdout:0/421: dwrite d0/d2/f6d [0,4194304] 0 2024-09-16T23:48:53.826 INFO:tasks.workunit.client.1.smithi043.stdout:5/590: dwrite d3/db/d19/d3e/d28/f37 [0,4194304] 0 2024-09-16T23:48:53.828 INFO:tasks.workunit.client.1.smithi043.stdout:0/422: dwrite d0/d2/df/f59 [0,4194304] 0 2024-09-16T23:48:53.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/212: dread d2/d7/f8 [0,4194304] 0 2024-09-16T23:48:53.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/213: dread - d2/d9/f35 zero size 2024-09-16T23:48:53.830 INFO:tasks.workunit.client.0.smithi042.stdout:3/214: chown d2/f4a 176210825 1 2024-09-16T23:48:53.831 INFO:tasks.workunit.client.0.smithi042.stdout:3/215: stat d2/d9/dd/d17 0 2024-09-16T23:48:53.833 INFO:tasks.workunit.client.1.smithi043.stdout:5/591: rename d3/d1b/l66 to d3/d9/d22/d60/lbd 0 2024-09-16T23:48:53.835 INFO:tasks.workunit.client.0.smithi042.stdout:3/216: write d2/f4a [1201485,21878] 0 2024-09-16T23:48:53.844 INFO:tasks.workunit.client.0.smithi042.stdout:3/217: dwrite d2/d9/dd/f1e [0,4194304] 0 2024-09-16T23:48:53.844 INFO:tasks.workunit.client.1.smithi043.stdout:0/423: getdents d0/d2/df/d5a 0 2024-09-16T23:48:53.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/218: dread - d2/d1b/d1d/d34/f3e zero size 2024-09-16T23:48:53.848 INFO:tasks.workunit.client.1.smithi043.stdout:5/592: unlink d3/d9/c89 0 2024-09-16T23:48:53.849 INFO:tasks.workunit.client.0.smithi042.stdout:3/219: chown d2/d1b/d1d/f38 1635 1 2024-09-16T23:48:53.849 INFO:tasks.workunit.client.0.smithi042.stdout:3/220: dread - d2/d9/f35 zero size 2024-09-16T23:48:53.851 INFO:tasks.workunit.client.1.smithi043.stdout:0/424: rename d0/d2/dd/l64 to d0/d2/df/d1a/d46/l8e 0 2024-09-16T23:48:53.852 INFO:tasks.workunit.client.0.smithi042.stdout:3/221: chown d2/c3 0 1 2024-09-16T23:48:53.853 INFO:tasks.workunit.client.0.smithi042.stdout:3/222: readlink d2/d9/dd/d2c/l49 0 2024-09-16T23:48:53.854 INFO:tasks.workunit.client.1.smithi043.stdout:5/593: write d3/d9/d22/d4a/d64/f7b [1570753,73519] 0 2024-09-16T23:48:53.854 INFO:tasks.workunit.client.0.smithi042.stdout:3/223: readlink d2/d9/dd/d17/d21/l48 0 2024-09-16T23:48:53.857 INFO:tasks.workunit.client.0.smithi042.stdout:3/224: mknod d2/d9/dd/c4c 0 2024-09-16T23:48:53.858 INFO:tasks.workunit.client.0.smithi042.stdout:3/225: chown d2/d9/dd/d17/d21/f42 1 1 2024-09-16T23:48:53.860 INFO:tasks.workunit.client.1.smithi043.stdout:5/594: dread d3/db/d19/f54 [0,4194304] 0 2024-09-16T23:48:53.861 INFO:tasks.workunit.client.1.smithi043.stdout:5/595: chown d3/d1b/l6e 3 1 2024-09-16T23:48:53.865 INFO:tasks.workunit.client.1.smithi043.stdout:5/596: symlink d3/db/d15/lbe 0 2024-09-16T23:48:53.868 INFO:tasks.workunit.client.1.smithi043.stdout:0/425: dread d0/d2/df/f10 [4194304,4194304] 0 2024-09-16T23:48:53.869 INFO:tasks.workunit.client.1.smithi043.stdout:0/426: chown d0/d2/df/l19 474125 1 2024-09-16T23:48:53.869 INFO:tasks.workunit.client.1.smithi043.stdout:5/597: rename d3/da4/lb0 to d3/db/d15/d4d/d6b/lbf 0 2024-09-16T23:48:53.870 INFO:tasks.workunit.client.1.smithi043.stdout:0/427: fdatasync d0/d2/df/d1a/d3b/f2a 0 2024-09-16T23:48:53.872 INFO:tasks.workunit.client.1.smithi043.stdout:0/428: rmdir d0/d2 39 2024-09-16T23:48:53.875 INFO:tasks.workunit.client.1.smithi043.stdout:5/598: link d3/db/d15/f3c d3/d9/d22/d4a/fc0 0 2024-09-16T23:48:53.876 INFO:tasks.workunit.client.0.smithi042.stdout:3/226: dread d2/d1b/d1d/f15 [0,4194304] 0 2024-09-16T23:48:53.878 INFO:tasks.workunit.client.1.smithi043.stdout:5/599: creat d3/d9/d22/d9f/fc1 x:0 0 0 2024-09-16T23:48:53.878 INFO:tasks.workunit.client.1.smithi043.stdout:5/600: readlink d3/d9/d22/d8e/l9c 0 2024-09-16T23:48:53.879 INFO:tasks.workunit.client.1.smithi043.stdout:0/429: rename d0/d2/df/d1a/f36 to d0/d2/df/d1a/d49/d71/f8f 0 2024-09-16T23:48:53.880 INFO:tasks.workunit.client.0.smithi042.stdout:3/227: mknod d2/d9/dd/d17/c4d 0 2024-09-16T23:48:53.880 INFO:tasks.workunit.client.1.smithi043.stdout:5/601: creat d3/d79/d91/daf/fc2 x:0 0 0 2024-09-16T23:48:53.880 INFO:tasks.workunit.client.1.smithi043.stdout:0/430: chown d0/d2/df/d1a/d46/d39/f61 2081644 1 2024-09-16T23:48:53.881 INFO:tasks.workunit.client.1.smithi043.stdout:0/431: chown d0/d2/df/d1a/l82 1 1 2024-09-16T23:48:53.884 INFO:tasks.workunit.client.1.smithi043.stdout:0/432: mknod d0/d2/df/d1a/d49/d77/c90 0 2024-09-16T23:48:53.884 INFO:tasks.workunit.client.0.smithi042.stdout:3/228: dread d2/d9/dd/d17/d21/f2e [0,4194304] 0 2024-09-16T23:48:53.887 INFO:tasks.workunit.client.1.smithi043.stdout:5/602: dwrite d3/db/d19/f77 [0,4194304] 0 2024-09-16T23:48:53.887 INFO:tasks.workunit.client.0.smithi042.stdout:3/229: creat d2/f4e x:0 0 0 2024-09-16T23:48:53.889 INFO:tasks.workunit.client.0.smithi042.stdout:3/230: fdatasync d2/d1b/d1d/f1a 0 2024-09-16T23:48:53.893 INFO:tasks.workunit.client.0.smithi042.stdout:3/231: dread d2/d1b/d1d/f44 [0,4194304] 0 2024-09-16T23:48:53.894 INFO:tasks.workunit.client.1.smithi043.stdout:1/531: sync 2024-09-16T23:48:53.894 INFO:tasks.workunit.client.1.smithi043.stdout:7/416: sync 2024-09-16T23:48:53.894 INFO:tasks.workunit.client.1.smithi043.stdout:7/417: fsync d4/db/d43/f7f 0 2024-09-16T23:48:53.895 INFO:tasks.workunit.client.1.smithi043.stdout:5/603: rename d3/f17 to d3/d9/fc3 0 2024-09-16T23:48:53.898 INFO:tasks.workunit.client.0.smithi042.stdout:3/232: getdents d2/d9 0 2024-09-16T23:48:53.899 INFO:tasks.workunit.client.1.smithi043.stdout:5/604: write d3/db/d19/d3e/f2d [5227704,104441] 0 2024-09-16T23:48:53.906 INFO:tasks.workunit.client.0.smithi042.stdout:3/233: dwrite d2/d1b/d1d/f1a [0,4194304] 0 2024-09-16T23:48:53.907 INFO:tasks.workunit.client.1.smithi043.stdout:7/418: link d4/db/d43/d83/d85/f42 d4/d27/d41/f8d 0 2024-09-16T23:48:53.910 INFO:tasks.workunit.client.0.smithi042.stdout:3/234: getdents d2/d1b/d1d/d34 0 2024-09-16T23:48:53.912 INFO:tasks.workunit.client.1.smithi043.stdout:7/419: rename d4/db/d11/l71 to d4/d27/d41/d1b/d75/l8e 0 2024-09-16T23:48:53.916 INFO:tasks.workunit.client.1.smithi043.stdout:0/433: write d0/dc/f42 [2359846,34159] 0 2024-09-16T23:48:53.916 INFO:tasks.workunit.client.0.smithi042.stdout:3/235: truncate d2/d7/f8 3083314 0 2024-09-16T23:48:53.917 INFO:tasks.workunit.client.1.smithi043.stdout:7/420: unlink d4/db/d50/c52 0 2024-09-16T23:48:53.918 INFO:tasks.workunit.client.0.smithi042.stdout:3/236: truncate d2/d7/f36 500327 0 2024-09-16T23:48:53.926 INFO:tasks.workunit.client.0.smithi042.stdout:3/237: dwrite d2/d1b/f3b [0,4194304] 0 2024-09-16T23:48:53.928 INFO:tasks.workunit.client.0.smithi042.stdout:3/238: write d2/d1b/d1d/f25 [1115486,16556] 0 2024-09-16T23:48:53.931 INFO:tasks.workunit.client.0.smithi042.stdout:3/239: mknod d2/d9/c4f 0 2024-09-16T23:48:53.932 INFO:tasks.workunit.client.0.smithi042.stdout:3/240: truncate d2/f28 193239 0 2024-09-16T23:48:53.935 INFO:tasks.workunit.client.0.smithi042.stdout:3/241: unlink d2/d9/dd/f1e 0 2024-09-16T23:48:53.939 INFO:tasks.workunit.client.0.smithi042.stdout:3/242: rename d2/d9/dd/d17 to d2/d1b/d1d/d34/d50 0 2024-09-16T23:48:53.942 INFO:tasks.workunit.client.0.smithi042.stdout:3/243: creat d2/d9/dd/f51 x:0 0 0 2024-09-16T23:48:53.944 INFO:tasks.workunit.client.0.smithi042.stdout:3/244: mkdir d2/d1b/d52 0 2024-09-16T23:48:53.947 INFO:tasks.workunit.client.0.smithi042.stdout:3/245: symlink d2/d1b/d1d/d34/d50/d21/l53 0 2024-09-16T23:48:53.948 INFO:tasks.workunit.client.0.smithi042.stdout:3/246: dread - d2/f41 zero size 2024-09-16T23:48:53.949 INFO:tasks.workunit.client.0.smithi042.stdout:3/247: fsync d2/d1b/d1d/f15 0 2024-09-16T23:48:53.987 INFO:tasks.workunit.client.1.smithi043.stdout:0/434: creat d0/de/f91 x:0 0 0 2024-09-16T23:48:53.990 INFO:tasks.workunit.client.1.smithi043.stdout:1/532: dwrite d4/d9/d1c/d1e/f55 [0,4194304] 0 2024-09-16T23:48:53.992 INFO:tasks.workunit.client.1.smithi043.stdout:1/533: write d4/d9/d1c/d1e/d47/f53 [864127,45419] 0 2024-09-16T23:48:54.007 INFO:tasks.workunit.client.1.smithi043.stdout:9/377: sync 2024-09-16T23:48:54.007 INFO:tasks.workunit.client.1.smithi043.stdout:2/363: sync 2024-09-16T23:48:54.008 INFO:tasks.workunit.client.1.smithi043.stdout:2/364: truncate d8/d1f/f2e 1048072 0 2024-09-16T23:48:54.038 INFO:tasks.workunit.client.1.smithi043.stdout:1/534: rmdir d4/d9/d12/d2b/d3d 39 2024-09-16T23:48:54.040 INFO:tasks.workunit.client.1.smithi043.stdout:1/535: symlink d4/df/d26/d79/lb4 0 2024-09-16T23:48:54.041 INFO:tasks.workunit.client.1.smithi043.stdout:0/435: mkdir d0/d2/df/d1a/d46/d92 0 2024-09-16T23:48:54.043 INFO:tasks.workunit.client.1.smithi043.stdout:0/436: truncate d0/d2/df/d1a/d3b/f5d 665089 0 2024-09-16T23:48:54.046 INFO:tasks.workunit.client.1.smithi043.stdout:0/437: mknod d0/d2/df/d1a/d46/c93 0 2024-09-16T23:48:54.047 INFO:tasks.workunit.client.1.smithi043.stdout:0/438: truncate d0/de/f91 235817 0 2024-09-16T23:48:54.049 INFO:tasks.workunit.client.1.smithi043.stdout:7/421: mkdir d4/d27/d41/d8f 0 2024-09-16T23:48:54.051 INFO:tasks.workunit.client.1.smithi043.stdout:0/439: creat d0/d2/df/d1a/d46/f94 x:0 0 0 2024-09-16T23:48:54.057 INFO:tasks.workunit.client.1.smithi043.stdout:0/440: dwrite d0/d2/df/d1a/d46/f5e [4194304,4194304] 0 2024-09-16T23:48:54.059 INFO:tasks.workunit.client.1.smithi043.stdout:2/365: write d8/fc [2024193,33735] 0 2024-09-16T23:48:54.061 INFO:tasks.workunit.client.1.smithi043.stdout:1/536: mkdir d4/db5 0 2024-09-16T23:48:54.064 INFO:tasks.workunit.client.1.smithi043.stdout:2/366: write d8/d1f/f6e [403063,111383] 0 2024-09-16T23:48:54.064 INFO:tasks.workunit.client.1.smithi043.stdout:0/441: dread d0/dc/f50 [0,4194304] 0 2024-09-16T23:48:54.064 INFO:tasks.workunit.client.1.smithi043.stdout:1/537: write d4/d9/d1c/d1e/d47/f86 [8486893,77525] 0 2024-09-16T23:48:54.067 INFO:tasks.workunit.client.1.smithi043.stdout:9/378: creat d4/d1c/f6f x:0 0 0 2024-09-16T23:48:54.068 INFO:tasks.workunit.client.0.smithi042.stdout:9/172: sync 2024-09-16T23:48:54.070 INFO:tasks.workunit.client.1.smithi043.stdout:0/442: mknod d0/d2/df/d1a/d49/d80/d88/c95 0 2024-09-16T23:48:54.071 INFO:tasks.workunit.client.1.smithi043.stdout:2/367: dwrite d8/d1d/d20/f58 [0,4194304] 0 2024-09-16T23:48:54.072 INFO:tasks.workunit.client.1.smithi043.stdout:9/379: chown d4/c8 1255540 1 2024-09-16T23:48:54.081 INFO:tasks.workunit.client.1.smithi043.stdout:2/368: creat d8/d11/d16/f81 x:0 0 0 2024-09-16T23:48:54.087 INFO:tasks.workunit.client.1.smithi043.stdout:7/422: write d4/f32 [692508,3695] 0 2024-09-16T23:48:54.087 INFO:tasks.workunit.client.1.smithi043.stdout:3/420: sync 2024-09-16T23:48:54.088 INFO:tasks.workunit.client.1.smithi043.stdout:2/369: symlink d8/d4b/d4f/d7c/l82 0 2024-09-16T23:48:54.090 INFO:tasks.workunit.client.1.smithi043.stdout:7/423: creat d4/d27/d41/d1b/f90 x:0 0 0 2024-09-16T23:48:54.090 INFO:tasks.workunit.client.1.smithi043.stdout:7/424: chown d4/db/f81 170502 1 2024-09-16T23:48:54.092 INFO:tasks.workunit.client.0.smithi042.stdout:9/173: dwrite d4/d10/f1b [4194304,4194304] 0 2024-09-16T23:48:54.093 INFO:tasks.workunit.client.1.smithi043.stdout:2/370: rename d8/d1d/d20/d23/d39 to d8/d5e/d83 0 2024-09-16T23:48:54.096 INFO:tasks.workunit.client.1.smithi043.stdout:7/425: dwrite d4/db/d11/d15/d29/f76 [0,4194304] 0 2024-09-16T23:48:54.100 INFO:tasks.workunit.client.0.smithi042.stdout:9/174: rename d4/d10/d24/d25 to d4/d12/d2d/d37 0 2024-09-16T23:48:54.103 INFO:tasks.workunit.client.1.smithi043.stdout:1/538: write d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f64 [4337804,38976] 0 2024-09-16T23:48:54.108 INFO:tasks.workunit.client.1.smithi043.stdout:3/421: dread d4/de/f21 [0,4194304] 0 2024-09-16T23:48:54.108 INFO:tasks.workunit.client.1.smithi043.stdout:0/443: dwrite d0/de/f2d [0,4194304] 0 2024-09-16T23:48:54.109 INFO:tasks.workunit.client.0.smithi042.stdout:9/175: link d4/d12/d2d/d37/l2a d4/d10/d24/l38 0 2024-09-16T23:48:54.110 INFO:tasks.workunit.client.1.smithi043.stdout:3/422: chown d4/de/d1f/d34/f74 577532 1 2024-09-16T23:48:54.113 INFO:tasks.workunit.client.0.smithi042.stdout:9/176: creat d4/d12/f39 x:0 0 0 2024-09-16T23:48:54.115 INFO:tasks.workunit.client.1.smithi043.stdout:0/444: dwrite d0/d2/df/d1a/d3b/f2a [4194304,4194304] 0 2024-09-16T23:48:54.118 INFO:tasks.workunit.client.1.smithi043.stdout:0/445: write d0/d2/df/d1a/d46/f94 [565361,61777] 0 2024-09-16T23:48:54.120 INFO:tasks.workunit.client.1.smithi043.stdout:0/446: write d0/de/f78 [601646,89805] 0 2024-09-16T23:48:54.121 INFO:tasks.workunit.client.0.smithi042.stdout:9/177: dwrite d4/d10/f1b [0,4194304] 0 2024-09-16T23:48:54.122 INFO:tasks.workunit.client.0.smithi042.stdout:9/178: read - d4/d10/f30 zero size 2024-09-16T23:48:54.125 INFO:tasks.workunit.client.0.smithi042.stdout:9/179: mkdir d4/d12/d2d/d37/d3a 0 2024-09-16T23:48:54.125 INFO:tasks.workunit.client.0.smithi042.stdout:9/180: readlink d4/d10/l22 0 2024-09-16T23:48:54.126 INFO:tasks.workunit.client.1.smithi043.stdout:3/423: link d4/de/d40/l5f d4/d30/l7a 0 2024-09-16T23:48:54.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:53 smithi043 ceph-mon[35191]: mgrmap e24: smithi043.xveyou(active, since 5s) 2024-09-16T23:48:54.127 INFO:tasks.workunit.client.1.smithi043.stdout:0/447: dread - d0/d2/df/d1a/d49/d71/f8f zero size 2024-09-16T23:48:54.128 INFO:tasks.workunit.client.1.smithi043.stdout:0/448: chown d0/d2/df/d1a/d46/d39/d6e 858464948 1 2024-09-16T23:48:54.129 INFO:tasks.workunit.client.0.smithi042.stdout:8/266: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:0/203: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:6/212: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:3/248: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:7/344: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:4/386: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:1/292: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:5/318: sync 2024-09-16T23:48:54.130 INFO:tasks.workunit.client.0.smithi042.stdout:2/271: sync 2024-09-16T23:48:54.132 INFO:tasks.workunit.client.1.smithi043.stdout:3/424: creat d4/d30/f7b x:0 0 0 2024-09-16T23:48:54.135 INFO:tasks.workunit.client.1.smithi043.stdout:1/539: dread d4/d9/d12/f1f [0,4194304] 0 2024-09-16T23:48:54.136 INFO:tasks.workunit.client.0.smithi042.stdout:2/272: write d0/da/d1b/d3b/d44/d54/f5e [655851,49924] 0 2024-09-16T23:48:54.136 INFO:tasks.workunit.client.0.smithi042.stdout:4/387: mkdir de/d42/d85 0 2024-09-16T23:48:54.137 INFO:tasks.workunit.client.1.smithi043.stdout:0/449: symlink d0/d2/df/d1a/d3b/l96 0 2024-09-16T23:48:54.137 INFO:tasks.workunit.client.0.smithi042.stdout:7/345: write d3/d15/d41/f62 [77387,120501] 0 2024-09-16T23:48:54.141 INFO:tasks.workunit.client.0.smithi042.stdout:9/181: dwrite d4/d12/f1e [0,4194304] 0 2024-09-16T23:48:54.141 INFO:tasks.workunit.client.0.smithi042.stdout:2/273: write d0/d26/f2f [2610261,32527] 0 2024-09-16T23:48:54.142 INFO:tasks.workunit.client.0.smithi042.stdout:7/346: dread - d3/d15/d32/f4a zero size 2024-09-16T23:48:54.144 INFO:tasks.workunit.client.0.smithi042.stdout:5/319: truncate d9/db/d1b/d31/d46/f49 1443977 0 2024-09-16T23:48:54.144 INFO:tasks.workunit.client.1.smithi043.stdout:1/540: mkdir d4/d9/d8f/db6 0 2024-09-16T23:48:54.145 INFO:tasks.workunit.client.0.smithi042.stdout:1/293: mkdir d0/d9/d35/d66 0 2024-09-16T23:48:54.147 INFO:tasks.workunit.client.1.smithi043.stdout:3/425: dread d4/de/f27 [0,4194304] 0 2024-09-16T23:48:54.147 INFO:tasks.workunit.client.0.smithi042.stdout:4/388: mknod de/d1a/d1c/c86 0 2024-09-16T23:48:54.148 INFO:tasks.workunit.client.0.smithi042.stdout:2/274: chown d0/da/d1b/l25 165795064 1 2024-09-16T23:48:54.150 INFO:tasks.workunit.client.0.smithi042.stdout:3/249: dread d2/f4a [0,4194304] 0 2024-09-16T23:48:54.152 INFO:tasks.workunit.client.0.smithi042.stdout:4/389: rmdir de/d12 39 2024-09-16T23:48:54.153 INFO:tasks.workunit.client.1.smithi043.stdout:1/541: mknod d4/d3e/d95/cb7 0 2024-09-16T23:48:54.155 INFO:tasks.workunit.client.0.smithi042.stdout:4/390: dread de/d1a/f46 [8388608,4194304] 0 2024-09-16T23:48:54.155 INFO:tasks.workunit.client.0.smithi042.stdout:3/250: mknod d2/d1b/d52/c54 0 2024-09-16T23:48:54.156 INFO:tasks.workunit.client.1.smithi043.stdout:0/450: dwrite d0/d2/df/d1a/d46/d39/d6e/f65 [0,4194304] 0 2024-09-16T23:48:54.156 INFO:tasks.workunit.client.0.smithi042.stdout:8/267: mknod da/d19/d2c/c54 0 2024-09-16T23:48:54.156 INFO:tasks.workunit.client.0.smithi042.stdout:7/347: getdents d3/d15/d24 0 2024-09-16T23:48:54.159 INFO:tasks.workunit.client.0.smithi042.stdout:7/348: readlink d3/d1f/l50 0 2024-09-16T23:48:54.160 INFO:tasks.workunit.client.0.smithi042.stdout:4/391: write de/d2e/d37/d3e/f61 [553587,46493] 0 2024-09-16T23:48:54.161 INFO:tasks.workunit.client.1.smithi043.stdout:3/426: dread d4/de/f58 [0,4194304] 0 2024-09-16T23:48:54.164 INFO:tasks.workunit.client.0.smithi042.stdout:6/213: write d8/db/fc [3643709,113274] 0 2024-09-16T23:48:54.166 INFO:tasks.workunit.client.0.smithi042.stdout:3/251: creat d2/d1b/d1d/d34/d50/f55 x:0 0 0 2024-09-16T23:48:54.166 INFO:tasks.workunit.client.1.smithi043.stdout:3/427: mknod d4/de/d40/c7c 0 2024-09-16T23:48:54.169 INFO:tasks.workunit.client.0.smithi042.stdout:7/349: rmdir d3/d15/d24/d28/d66 0 2024-09-16T23:48:54.169 INFO:tasks.workunit.client.1.smithi043.stdout:1/542: creat d4/d9/d12/d2b/d3d/d83/d98/fb8 x:0 0 0 2024-09-16T23:48:54.171 INFO:tasks.workunit.client.0.smithi042.stdout:3/252: chown d2/d1b/d1d/f38 4187315 1 2024-09-16T23:48:54.173 INFO:tasks.workunit.client.0.smithi042.stdout:9/182: creat d4/d10/d24/f3b x:0 0 0 2024-09-16T23:48:54.174 INFO:tasks.workunit.client.1.smithi043.stdout:3/428: truncate d4/f54 2556372 0 2024-09-16T23:48:54.174 INFO:tasks.workunit.client.0.smithi042.stdout:2/275: dwrite d0/da/d17/f59 [0,4194304] 0 2024-09-16T23:48:54.177 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:53 smithi042 ceph-mon[27914]: mgrmap e24: smithi043.xveyou(active, since 5s) 2024-09-16T23:48:54.178 INFO:tasks.workunit.client.1.smithi043.stdout:1/543: creat d4/df/d26/d79/fb9 x:0 0 0 2024-09-16T23:48:54.179 INFO:tasks.workunit.client.0.smithi042.stdout:1/294: truncate d0/f6 622133 0 2024-09-16T23:48:54.186 INFO:tasks.workunit.client.0.smithi042.stdout:3/253: dread d2/d1b/f3d [0,4194304] 0 2024-09-16T23:48:54.186 INFO:tasks.workunit.client.0.smithi042.stdout:3/254: dread d2/f28 [0,4194304] 0 2024-09-16T23:48:54.186 INFO:tasks.workunit.client.1.smithi043.stdout:1/544: symlink d4/d9/d12/d2b/d3d/d83/d98/lba 0 2024-09-16T23:48:54.188 INFO:tasks.workunit.client.1.smithi043.stdout:0/451: link d0/d2/df/c67 d0/d2/c97 0 2024-09-16T23:48:54.192 INFO:tasks.workunit.client.1.smithi043.stdout:0/452: write d0/dc/f50 [1768547,67936] 0 2024-09-16T23:48:54.193 INFO:tasks.workunit.client.0.smithi042.stdout:9/183: creat d4/d12/d2d/d37/f3c x:0 0 0 2024-09-16T23:48:54.193 INFO:tasks.workunit.client.0.smithi042.stdout:4/392: dread de/d12/d1b/f78 [0,4194304] 0 2024-09-16T23:48:54.193 INFO:tasks.workunit.client.0.smithi042.stdout:3/255: fdatasync d2/d7/f8 0 2024-09-16T23:48:54.193 INFO:tasks.workunit.client.0.smithi042.stdout:9/184: read - d4/d12/d2d/d37/d28/f31 zero size 2024-09-16T23:48:54.193 INFO:tasks.workunit.client.0.smithi042.stdout:2/276: creat d0/da/d1b/f60 x:0 0 0 2024-09-16T23:48:54.195 INFO:tasks.workunit.client.0.smithi042.stdout:9/185: chown d4/d12/d2d/d37/d28/f31 2 1 2024-09-16T23:48:54.195 INFO:tasks.workunit.client.0.smithi042.stdout:6/214: dread d8/db/dd/d2c/f31 [0,4194304] 0 2024-09-16T23:48:54.196 INFO:tasks.workunit.client.1.smithi043.stdout:1/545: dread d4/df/d26/f39 [0,4194304] 0 2024-09-16T23:48:54.197 INFO:tasks.workunit.client.0.smithi042.stdout:7/350: truncate d3/d1f/d49/d76/f1c 1547379 0 2024-09-16T23:48:54.198 INFO:tasks.workunit.client.1.smithi043.stdout:0/453: chown d0/l4b 2206160 1 2024-09-16T23:48:54.198 INFO:tasks.workunit.client.0.smithi042.stdout:4/393: symlink de/d2e/l87 0 2024-09-16T23:48:54.199 INFO:tasks.workunit.client.0.smithi042.stdout:1/295: symlink d0/d9/d26/l67 0 2024-09-16T23:48:54.199 INFO:tasks.workunit.client.0.smithi042.stdout:3/256: symlink d2/l56 0 2024-09-16T23:48:54.201 INFO:tasks.workunit.client.0.smithi042.stdout:8/268: truncate da/d19/d1d/d30/f45 1672511 0 2024-09-16T23:48:54.202 INFO:tasks.workunit.client.0.smithi042.stdout:7/351: unlink d3/f11 0 2024-09-16T23:48:54.202 INFO:tasks.workunit.client.0.smithi042.stdout:9/186: rmdir d4/d12/d2d/d37/d28 39 2024-09-16T23:48:54.203 INFO:tasks.workunit.client.0.smithi042.stdout:3/257: write d2/d1b/d1d/f25 [3868331,13854] 0 2024-09-16T23:48:54.204 INFO:tasks.workunit.client.0.smithi042.stdout:9/187: write d4/f19 [553454,83176] 0 2024-09-16T23:48:54.207 INFO:tasks.workunit.client.1.smithi043.stdout:1/546: rename d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f25 to d4/df/d13/fbb 0 2024-09-16T23:48:54.210 INFO:tasks.workunit.client.1.smithi043.stdout:3/429: getdents d4/de 0 2024-09-16T23:48:54.211 INFO:tasks.workunit.client.1.smithi043.stdout:0/454: dread d0/d2/df/d1a/d46/d39/d6e/f32 [0,4194304] 0 2024-09-16T23:48:54.212 INFO:tasks.workunit.client.0.smithi042.stdout:4/394: truncate de/f5a 368917 0 2024-09-16T23:48:54.213 INFO:tasks.workunit.client.0.smithi042.stdout:1/296: creat d0/d9/d35/d46/d32/d2f/f68 x:0 0 0 2024-09-16T23:48:54.213 INFO:tasks.workunit.client.1.smithi043.stdout:0/455: write d0/d2/df/d1a/d46/d39/f62 [587877,39905] 0 2024-09-16T23:48:54.215 INFO:tasks.workunit.client.0.smithi042.stdout:3/258: dwrite d2/d1b/d1d/f40 [0,4194304] 0 2024-09-16T23:48:54.218 INFO:tasks.workunit.client.1.smithi043.stdout:1/547: rename d4/d9/d12/d2b/d3d/fb3 to d4/d9/d8f/db6/fbc 0 2024-09-16T23:48:54.218 INFO:tasks.workunit.client.0.smithi042.stdout:2/277: dread d0/da/d17/f4c [0,4194304] 0 2024-09-16T23:48:54.221 INFO:tasks.workunit.client.0.smithi042.stdout:9/188: dwrite d4/d10/d24/f35 [0,4194304] 0 2024-09-16T23:48:54.223 INFO:tasks.workunit.client.0.smithi042.stdout:3/259: fsync d2/d1b/d1d/d34/d50/d21/f3c 0 2024-09-16T23:48:54.226 INFO:tasks.workunit.client.0.smithi042.stdout:6/215: truncate d8/db/dd/d2c/f10 3853487 0 2024-09-16T23:48:54.227 INFO:tasks.workunit.client.0.smithi042.stdout:1/297: dread d0/d17/f21 [0,4194304] 0 2024-09-16T23:48:54.227 INFO:tasks.workunit.client.0.smithi042.stdout:6/216: fdatasync d8/db/dd/f1c 0 2024-09-16T23:48:54.229 INFO:tasks.workunit.client.1.smithi043.stdout:3/430: unlink d4/de/d35/l2e 0 2024-09-16T23:48:54.229 INFO:tasks.workunit.client.0.smithi042.stdout:4/395: link de/d2e/d37/d3e/f7f de/d12/d7d/f88 0 2024-09-16T23:48:54.231 INFO:tasks.workunit.client.0.smithi042.stdout:3/260: mknod d2/d1b/d1d/d30/c57 0 2024-09-16T23:48:54.233 INFO:tasks.workunit.client.0.smithi042.stdout:3/261: dread - d2/d1b/d1d/f38 zero size 2024-09-16T23:48:54.235 INFO:tasks.workunit.client.0.smithi042.stdout:3/262: stat f0 0 2024-09-16T23:48:54.236 INFO:tasks.workunit.client.0.smithi042.stdout:4/396: write de/d2e/d37/d4c/d4f/f6b [7760197,108360] 0 2024-09-16T23:48:54.238 INFO:tasks.workunit.client.0.smithi042.stdout:1/298: dwrite d0/f3 [4194304,4194304] 0 2024-09-16T23:48:54.239 INFO:tasks.workunit.client.0.smithi042.stdout:2/278: write d0/f3a [533897,21128] 0 2024-09-16T23:48:54.242 INFO:tasks.workunit.client.0.smithi042.stdout:8/269: mknod da/d19/d1d/d30/c55 0 2024-09-16T23:48:54.243 INFO:tasks.workunit.client.1.smithi043.stdout:1/548: dwrite d4/df/d13/fa4 [4194304,4194304] 0 2024-09-16T23:48:54.243 INFO:tasks.workunit.client.1.smithi043.stdout:0/456: mknod d0/d2/df/c98 0 2024-09-16T23:48:54.244 INFO:tasks.workunit.client.0.smithi042.stdout:8/270: chown da 46315 1 2024-09-16T23:48:54.249 INFO:tasks.workunit.client.0.smithi042.stdout:4/397: dread de/d1a/f55 [0,4194304] 0 2024-09-16T23:48:54.253 INFO:tasks.workunit.client.0.smithi042.stdout:2/279: dwrite d0/d26/f50 [0,4194304] 0 2024-09-16T23:48:54.254 INFO:tasks.workunit.client.0.smithi042.stdout:6/217: mkdir d8/db/dd/d1b/d34/d39/d43 0 2024-09-16T23:48:54.255 INFO:tasks.workunit.client.0.smithi042.stdout:4/398: unlink de/d1a/d70/f71 0 2024-09-16T23:48:54.257 INFO:tasks.workunit.client.0.smithi042.stdout:1/299: readlink d0/l3c 0 2024-09-16T23:48:54.261 INFO:tasks.workunit.client.0.smithi042.stdout:4/399: write de/d12/d1b/f78 [3156693,114318] 0 2024-09-16T23:48:54.261 INFO:tasks.workunit.client.0.smithi042.stdout:3/263: dwrite d2/f28 [0,4194304] 0 2024-09-16T23:48:54.262 INFO:tasks.workunit.client.0.smithi042.stdout:9/189: creat d4/d12/d2d/d37/d3a/f3d x:0 0 0 2024-09-16T23:48:54.262 INFO:tasks.workunit.client.0.smithi042.stdout:4/400: chown de/d2e 3572949 1 2024-09-16T23:48:54.264 INFO:tasks.workunit.client.0.smithi042.stdout:2/280: symlink d0/da/d1b/d3b/l61 0 2024-09-16T23:48:54.270 INFO:tasks.workunit.client.0.smithi042.stdout:1/300: getdents d0/d17/d5e 0 2024-09-16T23:48:54.271 INFO:tasks.workunit.client.0.smithi042.stdout:4/401: truncate de/d12/f15 4939720 0 2024-09-16T23:48:54.272 INFO:tasks.workunit.client.0.smithi042.stdout:3/264: rename d2/d9/f35 to d2/d1b/d1d/d34/f58 0 2024-09-16T23:48:54.272 INFO:tasks.workunit.client.0.smithi042.stdout:2/281: write d0/da/f2c [3135799,121211] 0 2024-09-16T23:48:54.273 INFO:tasks.workunit.client.0.smithi042.stdout:6/218: dread d8/db/dd/f1f [0,4194304] 0 2024-09-16T23:48:54.277 INFO:tasks.workunit.client.0.smithi042.stdout:9/190: dwrite d4/d10/f1f [0,4194304] 0 2024-09-16T23:48:54.278 INFO:tasks.workunit.client.0.smithi042.stdout:1/301: write d0/f4a [1139625,100655] 0 2024-09-16T23:48:54.281 INFO:tasks.workunit.client.0.smithi042.stdout:3/265: chown d2/d1b/d1d/d34/d50/d21/f2e 15778 1 2024-09-16T23:48:54.283 INFO:tasks.workunit.client.0.smithi042.stdout:6/219: creat d8/db/dd/d1b/d2d/f44 x:0 0 0 2024-09-16T23:48:54.283 INFO:tasks.workunit.client.1.smithi043.stdout:0/457: creat d0/d2/df/d1a/d3b/f99 x:0 0 0 2024-09-16T23:48:54.284 INFO:tasks.workunit.client.1.smithi043.stdout:0/458: chown d0/l8d 199274 1 2024-09-16T23:48:54.287 INFO:tasks.workunit.client.0.smithi042.stdout:2/282: creat d0/d30/d43/f62 x:0 0 0 2024-09-16T23:48:54.289 INFO:tasks.workunit.client.0.smithi042.stdout:4/402: dwrite de/d2e/f77 [0,4194304] 0 2024-09-16T23:48:54.289 INFO:tasks.workunit.client.0.smithi042.stdout:3/266: mknod d2/d1b/d1d/d34/d50/d21/c59 0 2024-09-16T23:48:54.290 INFO:tasks.workunit.client.0.smithi042.stdout:1/302: rename d0/d17/d5e/l57 to d0/d9/d35/d46/d32/l69 0 2024-09-16T23:48:54.291 INFO:tasks.workunit.client.1.smithi043.stdout:0/459: mkdir d0/d2/df/d1a/d9a 0 2024-09-16T23:48:54.294 INFO:tasks.workunit.client.0.smithi042.stdout:9/191: dread d4/d10/f1b [4194304,4194304] 0 2024-09-16T23:48:54.294 INFO:tasks.workunit.client.0.smithi042.stdout:3/267: symlink d2/d1b/d1d/d34/d50/d21/l5a 0 2024-09-16T23:48:54.296 INFO:tasks.workunit.client.0.smithi042.stdout:6/220: rename d8/d38/d42 to d8/db/dd/d1b/d2d/d45 0 2024-09-16T23:48:54.298 INFO:tasks.workunit.client.0.smithi042.stdout:3/268: unlink d2/d9/dd/f51 0 2024-09-16T23:48:54.301 INFO:tasks.workunit.client.0.smithi042.stdout:3/269: creat d2/d9/dd/f5b x:0 0 0 2024-09-16T23:48:54.301 INFO:tasks.workunit.client.0.smithi042.stdout:9/192: symlink d4/d12/d16/l3e 0 2024-09-16T23:48:54.302 INFO:tasks.workunit.client.0.smithi042.stdout:1/303: dread d0/d9/f56 [0,4194304] 0 2024-09-16T23:48:54.303 INFO:tasks.workunit.client.1.smithi043.stdout:0/460: dread d0/d2/df/d1a/d3b/f6c [0,4194304] 0 2024-09-16T23:48:54.304 INFO:tasks.workunit.client.0.smithi042.stdout:1/304: chown d0/c5a 469737 1 2024-09-16T23:48:54.304 INFO:tasks.workunit.client.0.smithi042.stdout:1/305: fdatasync d0/f4a 0 2024-09-16T23:48:54.305 INFO:tasks.workunit.client.0.smithi042.stdout:1/306: stat d0/d9/d35/d46/d32/c54 0 2024-09-16T23:48:54.308 INFO:tasks.workunit.client.0.smithi042.stdout:2/283: link d0/da/d17/f1c d0/d30/d43/f63 0 2024-09-16T23:48:54.310 INFO:tasks.workunit.client.0.smithi042.stdout:6/221: symlink d8/db/dd/d1b/d34/d39/d43/l46 0 2024-09-16T23:48:54.314 INFO:tasks.workunit.client.0.smithi042.stdout:2/284: dwrite d0/f3a [0,4194304] 0 2024-09-16T23:48:54.314 INFO:tasks.workunit.client.0.smithi042.stdout:9/193: link d4/d12/d16/f17 d4/d10/f3f 0 2024-09-16T23:48:54.315 INFO:tasks.workunit.client.1.smithi043.stdout:0/461: link d0/f24 d0/dc/f9b 0 2024-09-16T23:48:54.316 INFO:tasks.workunit.client.0.smithi042.stdout:6/222: fsync f3 0 2024-09-16T23:48:54.318 INFO:tasks.workunit.client.0.smithi042.stdout:6/223: fsync d8/db/dd/d1b/d2d/f3e 0 2024-09-16T23:48:54.318 INFO:tasks.workunit.client.0.smithi042.stdout:2/285: creat d0/d5d/f64 x:0 0 0 2024-09-16T23:48:54.320 INFO:tasks.workunit.client.0.smithi042.stdout:6/224: truncate d8/db/f23 639751 0 2024-09-16T23:48:54.320 INFO:tasks.workunit.client.0.smithi042.stdout:9/194: symlink d4/l40 0 2024-09-16T23:48:54.324 INFO:tasks.workunit.client.0.smithi042.stdout:9/195: stat d4/d10/f1b 0 2024-09-16T23:48:54.324 INFO:tasks.workunit.client.0.smithi042.stdout:6/225: dread - d8/db/dd/d1b/d2d/f44 zero size 2024-09-16T23:48:54.324 INFO:tasks.workunit.client.1.smithi043.stdout:0/462: rename d0/d2/df/d1a/d49/d77 to d0/d2/df/d5a/d9c 0 2024-09-16T23:48:54.325 INFO:tasks.workunit.client.1.smithi043.stdout:0/463: truncate d0/d2/df/d1a/d46/f94 1442794 0 2024-09-16T23:48:54.330 INFO:tasks.workunit.client.0.smithi042.stdout:6/226: creat d8/db/d3c/f47 x:0 0 0 2024-09-16T23:48:54.331 INFO:tasks.workunit.client.0.smithi042.stdout:9/196: mkdir d4/d12/d2d/d37/d41 0 2024-09-16T23:48:54.333 INFO:tasks.workunit.client.1.smithi043.stdout:0/464: mknod d0/d2/df/c9d 0 2024-09-16T23:48:54.335 INFO:tasks.workunit.client.1.smithi043.stdout:0/465: dread d0/d2/df/d1a/d3b/f7d [0,4194304] 0 2024-09-16T23:48:54.336 INFO:tasks.workunit.client.0.smithi042.stdout:6/227: mknod d8/db/c48 0 2024-09-16T23:48:54.336 INFO:tasks.workunit.client.0.smithi042.stdout:2/286: write d0/da/d17/f4d [1143274,62170] 0 2024-09-16T23:48:54.337 INFO:tasks.workunit.client.1.smithi043.stdout:0/466: truncate d0/d2/df/d1a/d46/f5e 9122849 0 2024-09-16T23:48:54.339 INFO:tasks.workunit.client.0.smithi042.stdout:9/197: symlink d4/d12/d2d/d37/d28/l42 0 2024-09-16T23:48:54.342 INFO:tasks.workunit.client.0.smithi042.stdout:6/228: rename d8/db/dd/d1b/l33 to d8/db/dd/d1b/d34/d39/l49 0 2024-09-16T23:48:54.344 INFO:tasks.workunit.client.0.smithi042.stdout:6/229: readlink d8/db/dd/d1b/d34/d39/d43/l46 0 2024-09-16T23:48:54.345 INFO:tasks.workunit.client.0.smithi042.stdout:2/287: getdents d0/d30 0 2024-09-16T23:48:54.346 INFO:tasks.workunit.client.0.smithi042.stdout:9/198: dwrite d4/d12/f1e [0,4194304] 0 2024-09-16T23:48:54.347 INFO:tasks.workunit.client.0.smithi042.stdout:9/199: truncate d4/fb 4487658 0 2024-09-16T23:48:54.347 INFO:tasks.workunit.client.0.smithi042.stdout:6/230: rmdir d8 39 2024-09-16T23:48:54.350 INFO:tasks.workunit.client.0.smithi042.stdout:2/288: mkdir d0/da/d1b/d3b/d44/d65 0 2024-09-16T23:48:54.351 INFO:tasks.workunit.client.0.smithi042.stdout:2/289: chown d0/da/d17/f2d 2264740 1 2024-09-16T23:48:54.353 INFO:tasks.workunit.client.0.smithi042.stdout:9/200: mkdir d4/d12/d16/d43 0 2024-09-16T23:48:54.354 INFO:tasks.workunit.client.0.smithi042.stdout:9/201: write d4/f19 [1777080,122790] 0 2024-09-16T23:48:54.355 INFO:tasks.workunit.client.0.smithi042.stdout:2/290: mkdir d0/d30/d66 0 2024-09-16T23:48:54.358 INFO:tasks.workunit.client.0.smithi042.stdout:6/231: mknod d8/db/dd/d1b/d34/c4a 0 2024-09-16T23:48:54.360 INFO:tasks.workunit.client.0.smithi042.stdout:2/291: dread d0/da/d1b/d3b/f56 [0,4194304] 0 2024-09-16T23:48:54.360 INFO:tasks.workunit.client.1.smithi043.stdout:0/467: truncate d0/d2/df/f17 523561 0 2024-09-16T23:48:54.361 INFO:tasks.workunit.client.1.smithi043.stdout:0/468: fdatasync d0/d2/df/d1a/d3b/f5d 0 2024-09-16T23:48:54.363 INFO:tasks.workunit.client.1.smithi043.stdout:0/469: write d0/d2/df/d1a/d49/d71/f7e [1715581,110478] 0 2024-09-16T23:48:54.363 INFO:tasks.workunit.client.0.smithi042.stdout:9/202: dwrite d4/d10/f1f [0,4194304] 0 2024-09-16T23:48:54.364 INFO:tasks.workunit.client.0.smithi042.stdout:6/232: creat d8/db/dd/d1b/f4b x:0 0 0 2024-09-16T23:48:54.364 INFO:tasks.workunit.client.0.smithi042.stdout:9/203: read - d4/d12/f39 zero size 2024-09-16T23:48:54.368 INFO:tasks.workunit.client.0.smithi042.stdout:2/292: symlink d0/da/d1b/d3b/d44/d54/l67 0 2024-09-16T23:48:54.374 INFO:tasks.workunit.client.0.smithi042.stdout:6/233: mkdir d8/db/d3c/d4c 0 2024-09-16T23:48:54.376 INFO:tasks.workunit.client.0.smithi042.stdout:9/204: rename d4/d12/d16/d43 to d4/d12/d44 0 2024-09-16T23:48:54.378 INFO:tasks.workunit.client.0.smithi042.stdout:2/293: mknod d0/d26/c68 0 2024-09-16T23:48:54.380 INFO:tasks.workunit.client.0.smithi042.stdout:9/205: dread d4/d10/d24/f35 [0,4194304] 0 2024-09-16T23:48:54.381 INFO:tasks.workunit.client.1.smithi043.stdout:0/470: rename d0/dc/f42 to d0/d2/df/d1a/f9e 0 2024-09-16T23:48:54.383 INFO:tasks.workunit.client.0.smithi042.stdout:6/234: write d8/db/dd/f22 [2302571,80250] 0 2024-09-16T23:48:54.384 INFO:tasks.workunit.client.0.smithi042.stdout:6/235: fdatasync d8/db/dd/f1c 0 2024-09-16T23:48:54.389 INFO:tasks.workunit.client.0.smithi042.stdout:9/206: chown d4/d12/d2d/d37/l2a 0 1 2024-09-16T23:48:54.392 INFO:tasks.workunit.client.0.smithi042.stdout:2/294: dread d0/d26/f2f [0,4194304] 0 2024-09-16T23:48:54.398 INFO:tasks.workunit.client.1.smithi043.stdout:0/471: truncate d0/d2/df/d1a/d46/d39/f45 2752457 0 2024-09-16T23:48:54.401 INFO:tasks.workunit.client.0.smithi042.stdout:6/236: symlink d8/l4d 0 2024-09-16T23:48:54.403 INFO:tasks.workunit.client.0.smithi042.stdout:2/295: creat d0/da/d17/d3e/f69 x:0 0 0 2024-09-16T23:48:54.405 INFO:tasks.workunit.client.0.smithi042.stdout:6/237: dread d8/db/dd/d2c/d26/f2e [0,4194304] 0 2024-09-16T23:48:54.409 INFO:tasks.workunit.client.0.smithi042.stdout:2/296: dread d0/da/d17/f52 [0,4194304] 0 2024-09-16T23:48:54.409 INFO:tasks.workunit.client.1.smithi043.stdout:0/472: dwrite d0/d2/df/d1a/f9e [4194304,4194304] 0 2024-09-16T23:48:54.410 INFO:tasks.workunit.client.0.smithi042.stdout:6/238: mkdir d8/db/d3c/d4c/d4e 0 2024-09-16T23:48:54.414 INFO:tasks.workunit.client.0.smithi042.stdout:2/297: mknod d0/d5d/c6a 0 2024-09-16T23:48:54.416 INFO:tasks.workunit.client.1.smithi043.stdout:0/473: symlink d0/dc/l9f 0 2024-09-16T23:48:54.417 INFO:tasks.workunit.client.0.smithi042.stdout:9/207: dread d4/d10/f3f [0,4194304] 0 2024-09-16T23:48:54.420 INFO:tasks.workunit.client.0.smithi042.stdout:2/298: creat d0/da/d5b/f6b x:0 0 0 2024-09-16T23:48:54.422 INFO:tasks.workunit.client.1.smithi043.stdout:4/420: sync 2024-09-16T23:48:54.423 INFO:tasks.workunit.client.1.smithi043.stdout:8/430: sync 2024-09-16T23:48:54.425 INFO:tasks.workunit.client.0.smithi042.stdout:2/299: chown d0/l24 0 1 2024-09-16T23:48:54.426 INFO:tasks.workunit.client.1.smithi043.stdout:4/421: dread d8/f1d [0,4194304] 0 2024-09-16T23:48:54.426 INFO:tasks.workunit.client.0.smithi042.stdout:9/208: symlink d4/d10/d24/l45 0 2024-09-16T23:48:54.427 INFO:tasks.workunit.client.0.smithi042.stdout:9/209: fdatasync d4/d12/f1e 0 2024-09-16T23:48:54.428 INFO:tasks.workunit.client.1.smithi043.stdout:8/431: dwrite de/d18/d27/f75 [0,4194304] 0 2024-09-16T23:48:54.429 INFO:tasks.workunit.client.1.smithi043.stdout:0/474: write d0/d2/df/d1a/f2c [3566855,8607] 0 2024-09-16T23:48:54.430 INFO:tasks.workunit.client.0.smithi042.stdout:2/300: dread d0/da/d1b/d3b/d44/d54/f57 [0,4194304] 0 2024-09-16T23:48:54.437 INFO:tasks.workunit.client.0.smithi042.stdout:2/301: mknod d0/da/d1b/d3b/d44/d54/c6c 0 2024-09-16T23:48:54.439 INFO:tasks.workunit.client.1.smithi043.stdout:4/422: truncate d8/d15/d20/d2b/d22/f36 251412 0 2024-09-16T23:48:54.440 INFO:tasks.workunit.client.1.smithi043.stdout:4/423: truncate d8/d25/d64/f7d 960855 0 2024-09-16T23:48:54.441 INFO:tasks.workunit.client.0.smithi042.stdout:9/210: mknod d4/c46 0 2024-09-16T23:48:54.442 INFO:tasks.workunit.client.0.smithi042.stdout:2/302: mknod d0/da/d1b/d3b/d44/d54/c6d 0 2024-09-16T23:48:54.444 INFO:tasks.workunit.client.1.smithi043.stdout:8/432: getdents de/d18/d1f 0 2024-09-16T23:48:54.445 INFO:tasks.workunit.client.0.smithi042.stdout:2/303: mkdir d0/da/d17/d3e/d6e 0 2024-09-16T23:48:54.447 INFO:tasks.workunit.client.1.smithi043.stdout:4/424: mknod d8/d15/d20/d2b/d54/d55/c81 0 2024-09-16T23:48:54.448 INFO:tasks.workunit.client.0.smithi042.stdout:2/304: write d0/f2a [2544666,108149] 0 2024-09-16T23:48:54.458 INFO:tasks.workunit.client.1.smithi043.stdout:8/433: write de/d18/d27/d41/d5a/f4a [885515,120346] 0 2024-09-16T23:48:54.460 INFO:tasks.workunit.client.0.smithi042.stdout:6/239: rmdir d8/db/dd/d1b/d2d/d45 0 2024-09-16T23:48:54.461 INFO:tasks.workunit.client.0.smithi042.stdout:9/211: unlink d4/d10/c33 0 2024-09-16T23:48:54.463 INFO:tasks.workunit.client.0.smithi042.stdout:2/305: link d0/c6 d0/d30/d66/c6f 0 2024-09-16T23:48:54.463 INFO:tasks.workunit.client.1.smithi043.stdout:4/425: dread d8/d15/f3c [0,4194304] 0 2024-09-16T23:48:54.464 INFO:tasks.workunit.client.1.smithi043.stdout:8/434: dwrite de/d18/d1f/f32 [0,4194304] 0 2024-09-16T23:48:54.464 INFO:tasks.workunit.client.0.smithi042.stdout:9/212: mknod d4/d12/d2d/c47 0 2024-09-16T23:48:54.464 INFO:tasks.workunit.client.1.smithi043.stdout:4/426: write d8/d15/d6e/f5f [309439,122452] 0 2024-09-16T23:48:54.465 INFO:tasks.workunit.client.0.smithi042.stdout:6/240: mkdir d8/db/d3c/d4c/d4e/d4f 0 2024-09-16T23:48:54.465 INFO:tasks.workunit.client.1.smithi043.stdout:8/435: chown de/d70/d74/f7e 228 1 2024-09-16T23:48:54.465 INFO:tasks.workunit.client.1.smithi043.stdout:4/427: chown d8/d15/d20/d2b/f5a 11061881 1 2024-09-16T23:48:54.466 INFO:tasks.workunit.client.0.smithi042.stdout:9/213: truncate d4/d12/d2d/f32 850992 0 2024-09-16T23:48:54.467 INFO:tasks.workunit.client.0.smithi042.stdout:2/306: unlink d0/f2a 0 2024-09-16T23:48:54.470 INFO:tasks.workunit.client.1.smithi043.stdout:8/436: creat de/d70/d3d/f88 x:0 0 0 2024-09-16T23:48:54.471 INFO:tasks.workunit.client.0.smithi042.stdout:6/241: mkdir d8/db/dd/d1b/d50 0 2024-09-16T23:48:54.473 INFO:tasks.workunit.client.1.smithi043.stdout:4/428: creat d8/d15/d20/d2b/d54/d55/f82 x:0 0 0 2024-09-16T23:48:54.473 INFO:tasks.workunit.client.0.smithi042.stdout:9/214: dread d4/d10/f1b [4194304,4194304] 0 2024-09-16T23:48:54.475 INFO:tasks.workunit.client.0.smithi042.stdout:6/242: creat d8/db/d3c/d4c/d4e/f51 x:0 0 0 2024-09-16T23:48:54.476 INFO:tasks.workunit.client.1.smithi043.stdout:8/437: symlink de/d18/d27/l89 0 2024-09-16T23:48:54.477 INFO:tasks.workunit.client.0.smithi042.stdout:9/215: creat d4/d12/d2d/d37/d3a/f48 x:0 0 0 2024-09-16T23:48:54.479 INFO:tasks.workunit.client.0.smithi042.stdout:2/307: creat d0/d30/f70 x:0 0 0 2024-09-16T23:48:54.479 INFO:tasks.workunit.client.1.smithi043.stdout:4/429: dwrite d8/d25/d64/f7f [0,4194304] 0 2024-09-16T23:48:54.482 INFO:tasks.workunit.client.0.smithi042.stdout:2/308: chown d0/f22 40 1 2024-09-16T23:48:54.483 INFO:tasks.workunit.client.0.smithi042.stdout:6/243: dwrite d8/db/dd/f19 [0,4194304] 0 2024-09-16T23:48:54.484 INFO:tasks.workunit.client.1.smithi043.stdout:4/430: unlink d8/d15/d20/d2b/d23/c7c 0 2024-09-16T23:48:54.484 INFO:tasks.workunit.client.1.smithi043.stdout:8/438: dwrite de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:48:54.486 INFO:tasks.workunit.client.0.smithi042.stdout:9/216: dread d4/d10/f3f [0,4194304] 0 2024-09-16T23:48:54.491 INFO:tasks.workunit.client.0.smithi042.stdout:2/309: symlink d0/da/d1b/l71 0 2024-09-16T23:48:54.493 INFO:tasks.workunit.client.1.smithi043.stdout:8/439: dwrite de/d70/d3d/f51 [0,4194304] 0 2024-09-16T23:48:54.494 INFO:tasks.workunit.client.0.smithi042.stdout:9/217: dwrite d4/d12/d2d/f32 [0,4194304] 0 2024-09-16T23:48:54.506 INFO:tasks.workunit.client.1.smithi043.stdout:4/431: getdents d8/d15 0 2024-09-16T23:48:54.511 INFO:tasks.workunit.client.0.smithi042.stdout:2/310: dwrite d0/da/d1b/d3b/d44/d54/f57 [0,4194304] 0 2024-09-16T23:48:54.512 INFO:tasks.workunit.client.1.smithi043.stdout:8/440: truncate de/d18/f4c 720310 0 2024-09-16T23:48:54.515 INFO:tasks.workunit.client.0.smithi042.stdout:6/244: rename d8/db/dd/d1b/f3f to d8/db/f52 0 2024-09-16T23:48:54.519 INFO:tasks.workunit.client.0.smithi042.stdout:2/311: write d0/f12 [984139,74638] 0 2024-09-16T23:48:54.520 INFO:tasks.workunit.client.1.smithi043.stdout:8/441: creat de/d18/d1f/d25/d65/f8a x:0 0 0 2024-09-16T23:48:54.526 INFO:tasks.workunit.client.0.smithi042.stdout:6/245: mknod d8/c53 0 2024-09-16T23:48:54.528 INFO:tasks.workunit.client.0.smithi042.stdout:2/312: mknod d0/c72 0 2024-09-16T23:48:54.533 INFO:tasks.workunit.client.0.smithi042.stdout:6/246: mknod d8/db/dd/d1b/d50/c54 0 2024-09-16T23:48:54.534 INFO:tasks.workunit.client.1.smithi043.stdout:4/432: dwrite d8/d15/d20/d2b/d22/f24 [0,4194304] 0 2024-09-16T23:48:54.536 INFO:tasks.workunit.client.0.smithi042.stdout:6/247: creat d8/db/dd/d1b/d34/f55 x:0 0 0 2024-09-16T23:48:54.539 INFO:tasks.workunit.client.1.smithi043.stdout:4/433: mknod d8/d25/d39/d41/c83 0 2024-09-16T23:48:54.539 INFO:tasks.workunit.client.0.smithi042.stdout:6/248: readlink l4 0 2024-09-16T23:48:54.540 INFO:tasks.workunit.client.0.smithi042.stdout:2/313: dread d0/da/d17/f2d [0,4194304] 0 2024-09-16T23:48:54.544 INFO:tasks.workunit.client.0.smithi042.stdout:2/314: write d0/da/d1b/d3b/d44/d54/f5f [351966,23702] 0 2024-09-16T23:48:54.544 INFO:tasks.workunit.client.0.smithi042.stdout:6/249: mknod d8/db/dd/d1b/d34/d39/d43/c56 0 2024-09-16T23:48:54.546 INFO:tasks.workunit.client.0.smithi042.stdout:6/250: write d8/db/dd/f19 [1649798,104318] 0 2024-09-16T23:48:54.547 INFO:tasks.workunit.client.0.smithi042.stdout:2/315: dread - d0/da/d5b/f6b zero size 2024-09-16T23:48:54.549 INFO:tasks.workunit.client.0.smithi042.stdout:2/316: chown d0/da/d1b/l31 87429 1 2024-09-16T23:48:54.550 INFO:tasks.workunit.client.1.smithi043.stdout:4/434: dread d8/d25/f42 [0,4194304] 0 2024-09-16T23:48:54.551 INFO:tasks.workunit.client.0.smithi042.stdout:2/317: symlink d0/d5d/l73 0 2024-09-16T23:48:54.553 INFO:tasks.workunit.client.1.smithi043.stdout:6/438: sync 2024-09-16T23:48:54.554 INFO:tasks.workunit.client.0.smithi042.stdout:2/318: symlink d0/da/d17/d5c/l74 0 2024-09-16T23:48:54.555 INFO:tasks.workunit.client.1.smithi043.stdout:4/435: dwrite d8/d25/f4f [0,4194304] 0 2024-09-16T23:48:54.557 INFO:tasks.workunit.client.0.smithi042.stdout:2/319: creat d0/da/d5b/f75 x:0 0 0 2024-09-16T23:48:54.557 INFO:tasks.workunit.client.1.smithi043.stdout:6/439: mknod de/d1a/d80/c92 0 2024-09-16T23:48:54.560 INFO:tasks.workunit.client.1.smithi043.stdout:6/440: creat de/d23/d34/d66/f93 x:0 0 0 2024-09-16T23:48:54.564 INFO:tasks.workunit.client.0.smithi042.stdout:6/251: dread d8/db/dd/f22 [0,4194304] 0 2024-09-16T23:48:54.564 INFO:tasks.workunit.client.0.smithi042.stdout:2/320: unlink d0/da/d1b/d3b/d44/d54/f57 0 2024-09-16T23:48:54.566 INFO:tasks.workunit.client.1.smithi043.stdout:6/441: rename c7 to de/d23/d34/d66/c94 0 2024-09-16T23:48:54.566 INFO:tasks.workunit.client.1.smithi043.stdout:4/436: link d8/d15/d20/d2b/d23/d27/d4b/c76 d8/d25/d64/c84 0 2024-09-16T23:48:54.567 INFO:tasks.workunit.client.1.smithi043.stdout:6/442: dread de/f40 [0,4194304] 0 2024-09-16T23:48:54.568 INFO:tasks.workunit.client.1.smithi043.stdout:6/443: dread de/f40 [0,4194304] 0 2024-09-16T23:48:54.569 INFO:tasks.workunit.client.1.smithi043.stdout:4/437: read d8/d15/d20/d2b/d23/f38 [1092168,104355] 0 2024-09-16T23:48:54.570 INFO:tasks.workunit.client.0.smithi042.stdout:6/252: rename d8/db/dd/d1b/d34/d39/d43 to d8/db/dd/d2c/d57 0 2024-09-16T23:48:54.570 INFO:tasks.workunit.client.1.smithi043.stdout:6/444: fdatasync de/d1a/d4d/d5a/f90 0 2024-09-16T23:48:54.573 INFO:tasks.workunit.client.0.smithi042.stdout:6/253: rename d8/db/dd/c30 to d8/db/dd/c58 0 2024-09-16T23:48:54.573 INFO:tasks.workunit.client.0.smithi042.stdout:2/321: rmdir d0/da/d17 39 2024-09-16T23:48:54.576 INFO:tasks.workunit.client.1.smithi043.stdout:6/445: dwrite de/d1a/f41 [0,4194304] 0 2024-09-16T23:48:54.577 INFO:tasks.workunit.client.0.smithi042.stdout:6/254: symlink d8/db/dd/d1b/d34/l59 0 2024-09-16T23:48:54.579 INFO:tasks.workunit.client.1.smithi043.stdout:4/438: getdents d8/d15 0 2024-09-16T23:48:54.580 INFO:tasks.workunit.client.1.smithi043.stdout:4/439: chown d8/f68 2673839 1 2024-09-16T23:48:54.581 INFO:tasks.workunit.client.1.smithi043.stdout:4/440: chown d8/d15/d20/d2b/d23/f38 7288 1 2024-09-16T23:48:54.582 INFO:tasks.workunit.client.0.smithi042.stdout:6/255: mkdir d8/db/d3c/d5a 0 2024-09-16T23:48:54.587 INFO:tasks.workunit.client.1.smithi043.stdout:4/441: dwrite d8/d15/d20/d2b/f26 [0,4194304] 0 2024-09-16T23:48:54.587 INFO:tasks.workunit.client.1.smithi043.stdout:6/446: mknod de/d2a/c95 0 2024-09-16T23:48:54.588 INFO:tasks.workunit.client.1.smithi043.stdout:4/442: write d8/d15/f21 [8928868,7730] 0 2024-09-16T23:48:54.589 INFO:tasks.workunit.client.0.smithi042.stdout:2/322: creat d0/da/d17/d3e/f76 x:0 0 0 2024-09-16T23:48:54.589 INFO:tasks.workunit.client.0.smithi042.stdout:6/256: write d8/db/dd/f19 [1708908,115257] 0 2024-09-16T23:48:54.591 INFO:tasks.workunit.client.0.smithi042.stdout:6/257: write d8/db/dd/d1b/f4b [602673,43241] 0 2024-09-16T23:48:54.591 INFO:tasks.workunit.client.1.smithi043.stdout:6/447: rmdir de/d23/d63 39 2024-09-16T23:48:54.598 INFO:tasks.workunit.client.1.smithi043.stdout:4/443: mkdir d8/d25/d39/d41/d57/d85 0 2024-09-16T23:48:54.598 INFO:tasks.workunit.client.0.smithi042.stdout:2/323: dread d0/da/d1b/d3b/f56 [0,4194304] 0 2024-09-16T23:48:54.599 INFO:tasks.workunit.client.0.smithi042.stdout:2/324: write d0/d26/f4e [4217669,44353] 0 2024-09-16T23:48:54.600 INFO:tasks.workunit.client.1.smithi043.stdout:6/448: creat de/d23/d63/f96 x:0 0 0 2024-09-16T23:48:54.604 INFO:tasks.workunit.client.1.smithi043.stdout:4/444: creat d8/d25/d39/d58/d70/f86 x:0 0 0 2024-09-16T23:48:54.604 INFO:tasks.workunit.client.1.smithi043.stdout:6/449: rmdir de/d1a/d80 39 2024-09-16T23:48:54.610 INFO:tasks.workunit.client.1.smithi043.stdout:6/450: symlink de/d23/d63/d7f/l97 0 2024-09-16T23:48:54.611 INFO:tasks.workunit.client.0.smithi042.stdout:6/258: truncate d8/db/dd/d1b/f4b 282292 0 2024-09-16T23:48:54.613 INFO:tasks.workunit.client.0.smithi042.stdout:2/325: mkdir d0/d30/d43/d77 0 2024-09-16T23:48:54.617 INFO:tasks.workunit.client.0.smithi042.stdout:6/259: rename d8/db/dd/f1c to d8/db/d3c/d4c/d4e/d4f/f5b 0 2024-09-16T23:48:54.618 INFO:tasks.workunit.client.1.smithi043.stdout:4/445: link d8/d25/c56 d8/d15/d20/d2b/c87 0 2024-09-16T23:48:54.618 INFO:tasks.workunit.client.1.smithi043.stdout:4/446: read d8/f1d [227005,12592] 0 2024-09-16T23:48:54.622 INFO:tasks.workunit.client.0.smithi042.stdout:6/260: mknod d8/db/d3c/d4c/c5c 0 2024-09-16T23:48:54.622 INFO:tasks.workunit.client.0.smithi042.stdout:6/261: stat d8/db/dd/d2c/c36 0 2024-09-16T23:48:54.625 INFO:tasks.workunit.client.0.smithi042.stdout:2/326: mkdir d0/d78 0 2024-09-16T23:48:54.626 INFO:tasks.workunit.client.1.smithi043.stdout:6/451: getdents de/d1a/d28 0 2024-09-16T23:48:54.630 INFO:tasks.workunit.client.0.smithi042.stdout:2/327: fsync d0/d30/f47 0 2024-09-16T23:48:54.633 INFO:tasks.workunit.client.1.smithi043.stdout:4/447: truncate d8/d25/f42 2338123 0 2024-09-16T23:48:54.635 INFO:tasks.workunit.client.0.smithi042.stdout:2/328: rename d0/da/d1b/f60 to d0/d5d/f79 0 2024-09-16T23:48:54.638 INFO:tasks.workunit.client.1.smithi043.stdout:4/448: creat d8/d25/d39/d58/d70/f88 x:0 0 0 2024-09-16T23:48:54.639 INFO:tasks.workunit.client.1.smithi043.stdout:6/452: creat de/d1a/d4d/f98 x:0 0 0 2024-09-16T23:48:54.640 INFO:tasks.workunit.client.0.smithi042.stdout:2/329: mknod d0/da/d1b/d3b/d44/d54/c7a 0 2024-09-16T23:48:54.640 INFO:tasks.workunit.client.1.smithi043.stdout:5/605: sync 2024-09-16T23:48:54.640 INFO:tasks.workunit.client.1.smithi043.stdout:9/380: sync 2024-09-16T23:48:54.640 INFO:tasks.workunit.client.1.smithi043.stdout:2/371: sync 2024-09-16T23:48:54.640 INFO:tasks.workunit.client.1.smithi043.stdout:3/431: sync 2024-09-16T23:48:54.640 INFO:tasks.workunit.client.1.smithi043.stdout:1/549: sync 2024-09-16T23:48:54.640 INFO:tasks.workunit.client.1.smithi043.stdout:7/426: sync 2024-09-16T23:48:54.643 INFO:tasks.workunit.client.1.smithi043.stdout:3/432: fsync d4/de/f51 0 2024-09-16T23:48:54.643 INFO:tasks.workunit.client.0.smithi042.stdout:2/330: rmdir d0/da/d17/d5c 39 2024-09-16T23:48:54.644 INFO:tasks.workunit.client.1.smithi043.stdout:1/550: chown d4/d9/d8f/la2 427 1 2024-09-16T23:48:54.645 INFO:tasks.workunit.client.1.smithi043.stdout:3/433: stat d4/de/f27 0 2024-09-16T23:48:54.646 INFO:tasks.workunit.client.1.smithi043.stdout:4/449: mknod d8/d15/d20/d2b/d54/c89 0 2024-09-16T23:48:54.647 INFO:tasks.workunit.client.0.smithi042.stdout:2/331: mknod d0/d30/d43/c7b 0 2024-09-16T23:48:54.648 INFO:tasks.workunit.client.1.smithi043.stdout:6/453: dwrite de/f40 [0,4194304] 0 2024-09-16T23:48:54.648 INFO:tasks.workunit.client.1.smithi043.stdout:5/606: creat d3/db/fc4 x:0 0 0 2024-09-16T23:48:54.649 INFO:tasks.workunit.client.0.smithi042.stdout:6/262: write d8/db/dd/d1b/f4b [518070,2690] 0 2024-09-16T23:48:54.650 INFO:tasks.workunit.client.1.smithi043.stdout:5/607: write d3/db/d19/d3e/f35 [4565506,13304] 0 2024-09-16T23:48:54.651 INFO:tasks.workunit.client.1.smithi043.stdout:2/372: symlink d8/d4b/d4f/d56/l84 0 2024-09-16T23:48:54.653 INFO:tasks.workunit.client.0.smithi042.stdout:2/332: creat d0/d26/f7c x:0 0 0 2024-09-16T23:48:54.655 INFO:tasks.workunit.client.1.smithi043.stdout:3/434: rmdir d4 39 2024-09-16T23:48:54.657 INFO:tasks.workunit.client.1.smithi043.stdout:9/381: mknod d4/d21/d2a/c70 0 2024-09-16T23:48:54.657 INFO:tasks.workunit.client.0.smithi042.stdout:6/263: dwrite d8/db/dd/d2c/f13 [0,4194304] 0 2024-09-16T23:48:54.660 INFO:tasks.workunit.client.1.smithi043.stdout:7/427: rename d4/db/d28/l72 to d4/db/d43/d83/d85/d8b/l91 0 2024-09-16T23:48:54.661 INFO:tasks.workunit.client.1.smithi043.stdout:4/450: symlink d8/d25/d39/d58/d70/l8a 0 2024-09-16T23:48:54.664 INFO:tasks.workunit.client.0.smithi042.stdout:2/333: symlink d0/da/l7d 0 2024-09-16T23:48:54.668 INFO:tasks.workunit.client.0.smithi042.stdout:6/264: creat d8/db/dd/d2c/d57/f5d x:0 0 0 2024-09-16T23:48:54.670 INFO:tasks.workunit.client.0.smithi042.stdout:6/265: write d8/db/dd/d1b/f4b [648542,107586] 0 2024-09-16T23:48:54.670 INFO:tasks.workunit.client.1.smithi043.stdout:5/608: creat d3/d79/d91/daf/fc5 x:0 0 0 2024-09-16T23:48:54.671 INFO:tasks.workunit.client.1.smithi043.stdout:5/609: fsync d3/d9/d22/d4a/fb1 0 2024-09-16T23:48:54.675 INFO:tasks.workunit.client.1.smithi043.stdout:9/382: fdatasync d4/f48 0 2024-09-16T23:48:54.677 INFO:tasks.workunit.client.0.smithi042.stdout:6/266: dwrite d8/db/fc [4194304,4194304] 0 2024-09-16T23:48:54.679 INFO:tasks.workunit.client.1.smithi043.stdout:1/551: rename d4/d9/d1c/d1e/d47/c8c to d4/df/d26/d79/cbd 0 2024-09-16T23:48:54.681 INFO:tasks.workunit.client.1.smithi043.stdout:7/428: creat d4/db/d43/d83/d85/f92 x:0 0 0 2024-09-16T23:48:54.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/267: dread f6 [0,4194304] 0 2024-09-16T23:48:54.681 INFO:tasks.workunit.client.0.smithi042.stdout:2/334: link d0/c72 d0/da/d1b/c7e 0 2024-09-16T23:48:54.682 INFO:tasks.workunit.client.1.smithi043.stdout:8/442: sync 2024-09-16T23:48:54.682 INFO:tasks.workunit.client.1.smithi043.stdout:0/475: sync 2024-09-16T23:48:54.683 INFO:tasks.workunit.client.1.smithi043.stdout:4/451: mkdir d8/d25/d39/d58/d70/d8b 0 2024-09-16T23:48:54.683 INFO:tasks.workunit.client.0.smithi042.stdout:6/268: chown d8/db/dd/d1b/d2d/f41 252623052 1 2024-09-16T23:48:54.684 INFO:tasks.workunit.client.1.smithi043.stdout:0/476: read d0/de/f2d [2684718,59500] 0 2024-09-16T23:48:54.685 INFO:tasks.workunit.client.1.smithi043.stdout:1/552: dread d4/d9/d12/f1f [0,4194304] 0 2024-09-16T23:48:54.685 INFO:tasks.workunit.client.1.smithi043.stdout:5/610: creat d3/d9/d22/d9f/fc6 x:0 0 0 2024-09-16T23:48:54.685 INFO:tasks.workunit.client.1.smithi043.stdout:4/452: chown d8/f5d 31 1 2024-09-16T23:48:54.687 INFO:tasks.workunit.client.1.smithi043.stdout:1/553: readlink d4/d34/lad 0 2024-09-16T23:48:54.687 INFO:tasks.workunit.client.1.smithi043.stdout:0/477: chown d0/l4b 726504 1 2024-09-16T23:48:54.687 INFO:tasks.workunit.client.1.smithi043.stdout:8/443: read de/d70/d3d/f51 [2553876,72321] 0 2024-09-16T23:48:54.688 INFO:tasks.workunit.client.1.smithi043.stdout:8/444: fsync de/f15 0 2024-09-16T23:48:54.689 INFO:tasks.workunit.client.0.smithi042.stdout:6/269: dwrite d8/db/dd/d2c/f13 [0,4194304] 0 2024-09-16T23:48:54.691 INFO:tasks.workunit.client.0.smithi042.stdout:2/335: symlink d0/da/d17/d3e/d6e/l7f 0 2024-09-16T23:48:54.691 INFO:tasks.workunit.client.1.smithi043.stdout:7/429: rmdir d4/d27/d41/d1b/d75 39 2024-09-16T23:48:54.692 INFO:tasks.workunit.client.1.smithi043.stdout:0/478: dread d0/d2/df/d1a/d46/d39/d6e/f55 [0,4194304] 0 2024-09-16T23:48:54.697 INFO:tasks.workunit.client.0.smithi042.stdout:6/270: rename d8/db/d3c to d8/db/dd/d1b/d34/d39/d5e 0 2024-09-16T23:48:54.698 INFO:tasks.workunit.client.1.smithi043.stdout:5/611: dwrite d3/d79/d91/daf/fc5 [0,4194304] 0 2024-09-16T23:48:54.699 INFO:tasks.workunit.client.0.smithi042.stdout:6/271: dread d8/db/f23 [0,4194304] 0 2024-09-16T23:48:54.700 INFO:tasks.workunit.client.1.smithi043.stdout:5/612: chown d3/d9/d6f/d7c/d80/l9e 6011295 1 2024-09-16T23:48:54.700 INFO:tasks.workunit.client.0.smithi042.stdout:6/272: stat d8/db/dd/d2c/d26/c2f 0 2024-09-16T23:48:54.702 INFO:tasks.workunit.client.1.smithi043.stdout:8/445: dwrite de/d18/d1f/d2f/d39/d4d/f5d [0,4194304] 0 2024-09-16T23:48:54.710 INFO:tasks.workunit.client.0.smithi042.stdout:6/273: dwrite d8/db/dd/d2c/f13 [0,4194304] 0 2024-09-16T23:48:54.711 INFO:tasks.workunit.client.1.smithi043.stdout:1/554: creat d4/d34/fbe x:0 0 0 2024-09-16T23:48:54.712 INFO:tasks.workunit.client.1.smithi043.stdout:8/446: dwrite de/d18/d1f/d2f/d39/d4d/f5d [0,4194304] 0 2024-09-16T23:48:54.713 INFO:tasks.workunit.client.1.smithi043.stdout:3/435: truncate d4/de/d1f/d2c/f4a 1526289 0 2024-09-16T23:48:54.719 INFO:tasks.workunit.client.1.smithi043.stdout:6/454: rename de/d23/d34/d66/c94 to de/d23/d63/c99 0 2024-09-16T23:48:54.720 INFO:tasks.workunit.client.0.smithi042.stdout:6/274: dwrite d8/db/fc [0,4194304] 0 2024-09-16T23:48:54.724 INFO:tasks.workunit.client.0.smithi042.stdout:2/336: rmdir d0/da/d1b/d3b/d44/d65 0 2024-09-16T23:48:54.725 INFO:tasks.workunit.client.0.smithi042.stdout:2/337: write d0/d5d/f64 [353935,31178] 0 2024-09-16T23:48:54.731 INFO:tasks.workunit.client.0.smithi042.stdout:6/275: rename d8/db/dd/d2c/fa to d8/db/dd/f5f 0 2024-09-16T23:48:54.731 INFO:tasks.workunit.client.0.smithi042.stdout:6/276: chown d8/db/c48 1274 1 2024-09-16T23:48:54.732 INFO:tasks.workunit.client.1.smithi043.stdout:8/447: symlink de/d18/d1f/d2f/d39/d7b/l8b 0 2024-09-16T23:48:54.736 INFO:tasks.workunit.client.1.smithi043.stdout:1/555: symlink d4/d3e/lbf 0 2024-09-16T23:48:54.739 INFO:tasks.workunit.client.0.smithi042.stdout:2/338: rename d0/da/f21 to d0/da/d17/d3e/d6e/f80 0 2024-09-16T23:48:54.740 INFO:tasks.workunit.client.0.smithi042.stdout:2/339: chown d0/l24 4734270 1 2024-09-16T23:48:54.741 INFO:tasks.workunit.client.1.smithi043.stdout:6/455: unlink de/d23/d34/l64 0 2024-09-16T23:48:54.741 INFO:tasks.workunit.client.0.smithi042.stdout:0/204: sync 2024-09-16T23:48:54.741 INFO:tasks.workunit.client.0.smithi042.stdout:5/320: sync 2024-09-16T23:48:54.743 INFO:tasks.workunit.client.0.smithi042.stdout:5/321: fdatasync d9/db/de/d10/d4b/d61/f4e 0 2024-09-16T23:48:54.747 INFO:tasks.workunit.client.0.smithi042.stdout:2/340: unlink d0/f11 0 2024-09-16T23:48:54.748 INFO:tasks.workunit.client.1.smithi043.stdout:5/613: mknod d3/db/d19/d3e/cc7 0 2024-09-16T23:48:54.749 INFO:tasks.workunit.client.0.smithi042.stdout:0/205: rename d0/le to d0/df/d23/d33/l39 0 2024-09-16T23:48:54.750 INFO:tasks.workunit.client.0.smithi042.stdout:6/277: dread d8/db/dd/f11 [0,4194304] 0 2024-09-16T23:48:54.751 INFO:tasks.workunit.client.1.smithi043.stdout:4/453: creat d8/d15/d20/d2b/f8c x:0 0 0 2024-09-16T23:48:54.752 INFO:tasks.workunit.client.1.smithi043.stdout:3/436: truncate d4/d30/f41 554390 0 2024-09-16T23:48:54.753 INFO:tasks.workunit.client.0.smithi042.stdout:5/322: dwrite d9/db/f3e [0,4194304] 0 2024-09-16T23:48:54.754 INFO:tasks.workunit.client.1.smithi043.stdout:3/437: chown d4/de/d1f/c4f 62 1 2024-09-16T23:48:54.755 INFO:tasks.workunit.client.1.smithi043.stdout:5/614: dwrite d3/db/fc4 [0,4194304] 0 2024-09-16T23:48:54.757 INFO:tasks.workunit.client.0.smithi042.stdout:6/278: truncate d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f51 442073 0 2024-09-16T23:48:54.758 INFO:tasks.workunit.client.1.smithi043.stdout:1/556: creat d4/d34/fc0 x:0 0 0 2024-09-16T23:48:54.759 INFO:tasks.workunit.client.0.smithi042.stdout:5/323: symlink d9/db/de/d5e/l69 0 2024-09-16T23:48:54.762 INFO:tasks.workunit.client.1.smithi043.stdout:2/373: rename d8/d1d/l51 to d8/d1f/d6b/l85 0 2024-09-16T23:48:54.762 INFO:tasks.workunit.client.0.smithi042.stdout:0/206: getdents d0/df/d23 0 2024-09-16T23:48:54.764 INFO:tasks.workunit.client.0.smithi042.stdout:0/207: dread - d0/df/d23/f38 zero size 2024-09-16T23:48:54.768 INFO:tasks.workunit.client.0.smithi042.stdout:2/341: rename d0/da/d17/d3e/d6e to d0/da/d1b/d48/d81 0 2024-09-16T23:48:54.769 INFO:tasks.workunit.client.0.smithi042.stdout:2/342: stat d0/da/d1b/d3b/d44 0 2024-09-16T23:48:54.771 INFO:tasks.workunit.client.1.smithi043.stdout:2/374: dread - d8/d11/d16/f81 zero size 2024-09-16T23:48:54.773 INFO:tasks.workunit.client.0.smithi042.stdout:0/208: link d0/l2a d0/df/d23/l3a 0 2024-09-16T23:48:54.773 INFO:tasks.workunit.client.1.smithi043.stdout:5/615: dwrite d3/db/fc4 [0,4194304] 0 2024-09-16T23:48:54.774 INFO:tasks.workunit.client.1.smithi043.stdout:7/430: truncate d4/db/d43/d83/d85/f57 1569822 0 2024-09-16T23:48:54.774 INFO:tasks.workunit.client.1.smithi043.stdout:2/375: chown d8/d11/f6f 1 1 2024-09-16T23:48:54.775 INFO:tasks.workunit.client.1.smithi043.stdout:4/454: read d8/d25/f6a [162627,68511] 0 2024-09-16T23:48:54.775 INFO:tasks.workunit.client.1.smithi043.stdout:8/448: creat de/d18/d1f/d25/d65/d7f/f8c x:0 0 0 2024-09-16T23:48:54.778 INFO:tasks.workunit.client.0.smithi042.stdout:0/209: mknod d0/df/d13/c3b 0 2024-09-16T23:48:54.779 INFO:tasks.workunit.client.0.smithi042.stdout:6/279: link d8/f29 d8/db/dd/d2c/d57/f60 0 2024-09-16T23:48:54.779 INFO:tasks.workunit.client.1.smithi043.stdout:1/557: dwrite d4/d9/d1c/d1e/f55 [4194304,4194304] 0 2024-09-16T23:48:54.781 INFO:tasks.workunit.client.1.smithi043.stdout:3/438: symlink d4/d30/d53/l7d 0 2024-09-16T23:48:54.782 INFO:tasks.workunit.client.0.smithi042.stdout:2/343: dwrite d0/d30/d43/f62 [0,4194304] 0 2024-09-16T23:48:54.784 INFO:tasks.workunit.client.1.smithi043.stdout:2/376: dread d8/d4b/f5b [0,4194304] 0 2024-09-16T23:48:54.787 INFO:tasks.workunit.client.0.smithi042.stdout:6/280: write d8/db/dd/d1b/d34/f3d [388624,121586] 0 2024-09-16T23:48:54.788 INFO:tasks.workunit.client.0.smithi042.stdout:0/210: dread d0/df/d13/f22 [0,4194304] 0 2024-09-16T23:48:54.789 INFO:tasks.workunit.client.1.smithi043.stdout:8/449: dwrite de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:48:54.789 INFO:tasks.workunit.client.0.smithi042.stdout:0/211: chown d0/df/f14 437675 1 2024-09-16T23:48:54.789 INFO:tasks.workunit.client.0.smithi042.stdout:6/281: read d8/f18 [185071,12053] 0 2024-09-16T23:48:54.790 INFO:tasks.workunit.client.1.smithi043.stdout:8/450: dread de/d18/d1f/d25/f6b [0,4194304] 0 2024-09-16T23:48:54.797 INFO:tasks.workunit.client.0.smithi042.stdout:5/324: dread d9/db/d19/d21/f2b [0,4194304] 0 2024-09-16T23:48:54.804 INFO:tasks.workunit.client.0.smithi042.stdout:5/325: getdents d9/db/de/d10/d4b 0 2024-09-16T23:48:54.805 INFO:tasks.workunit.client.0.smithi042.stdout:0/212: dread d0/d15/f2f [0,4194304] 0 2024-09-16T23:48:54.805 INFO:tasks.workunit.client.0.smithi042.stdout:0/213: write d0/df/f21 [3215455,87640] 0 2024-09-16T23:48:54.809 INFO:tasks.workunit.client.0.smithi042.stdout:5/326: dread d9/db/d1b/d31/d46/f49 [0,4194304] 0 2024-09-16T23:48:54.809 INFO:tasks.workunit.client.0.smithi042.stdout:6/282: unlink d8/db/dd/d1b/d2d/f44 0 2024-09-16T23:48:54.810 INFO:tasks.workunit.client.0.smithi042.stdout:2/344: write d0/f3a [4666796,43802] 0 2024-09-16T23:48:54.813 INFO:tasks.workunit.client.0.smithi042.stdout:5/327: symlink d9/db/d19/l6a 0 2024-09-16T23:48:54.815 INFO:tasks.workunit.client.0.smithi042.stdout:0/214: dwrite d0/df/d13/f22 [0,4194304] 0 2024-09-16T23:48:54.817 INFO:tasks.workunit.client.0.smithi042.stdout:6/283: mknod d8/c61 0 2024-09-16T23:48:54.822 INFO:tasks.workunit.client.0.smithi042.stdout:0/215: creat d0/d15/f3c x:0 0 0 2024-09-16T23:48:54.824 INFO:tasks.workunit.client.0.smithi042.stdout:5/328: dwrite d9/db/d11/f1c [0,4194304] 0 2024-09-16T23:48:54.826 INFO:tasks.workunit.client.0.smithi042.stdout:0/216: write d0/df/f21 [3344360,44871] 0 2024-09-16T23:48:54.831 INFO:tasks.workunit.client.0.smithi042.stdout:6/284: dwrite d8/db/fc [4194304,4194304] 0 2024-09-16T23:48:54.832 INFO:tasks.workunit.client.0.smithi042.stdout:5/329: write d9/db/de/d10/d2f/f34 [3140417,6342] 0 2024-09-16T23:48:54.835 INFO:tasks.workunit.client.0.smithi042.stdout:0/217: symlink d0/df/d13/l3d 0 2024-09-16T23:48:54.839 INFO:tasks.workunit.client.1.smithi043.stdout:6/456: link de/d23/d49/f7c de/d1a/d28/f9a 0 2024-09-16T23:48:54.839 INFO:tasks.workunit.client.0.smithi042.stdout:0/218: truncate d0/df/d23/f38 405278 0 2024-09-16T23:48:54.841 INFO:tasks.workunit.client.1.smithi043.stdout:6/457: write de/d1a/d39/f78 [1509241,77881] 0 2024-09-16T23:48:54.841 INFO:tasks.workunit.client.0.smithi042.stdout:2/345: truncate d0/d26/f4e 2819935 0 2024-09-16T23:48:54.841 INFO:tasks.workunit.client.0.smithi042.stdout:5/330: link d9/db/l3f d9/db/d11/l6b 0 2024-09-16T23:48:54.843 INFO:tasks.workunit.client.0.smithi042.stdout:6/285: dwrite d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f51 [0,4194304] 0 2024-09-16T23:48:54.844 INFO:tasks.workunit.client.0.smithi042.stdout:0/219: mkdir d0/df/d23/d3e 0 2024-09-16T23:48:54.845 INFO:tasks.workunit.client.1.smithi043.stdout:4/455: creat d8/d15/d20/f8d x:0 0 0 2024-09-16T23:48:54.848 INFO:tasks.workunit.client.1.smithi043.stdout:6/458: dwrite de/d1a/d39/f78 [0,4194304] 0 2024-09-16T23:48:54.849 INFO:tasks.workunit.client.0.smithi042.stdout:5/331: dread d9/db/d19/d21/f2b [0,4194304] 0 2024-09-16T23:48:54.850 INFO:tasks.workunit.client.0.smithi042.stdout:6/286: fsync d8/db/dd/f1f 0 2024-09-16T23:48:54.851 INFO:tasks.workunit.client.1.smithi043.stdout:1/558: creat d4/df/d13/fc1 x:0 0 0 2024-09-16T23:48:54.851 INFO:tasks.workunit.client.0.smithi042.stdout:0/220: rename d0/df/d13/c3b to d0/c3f 0 2024-09-16T23:48:54.853 INFO:tasks.workunit.client.0.smithi042.stdout:5/332: mkdir d9/db/d1b/d31/d6c 0 2024-09-16T23:48:54.853 INFO:tasks.workunit.client.0.smithi042.stdout:0/221: dread - d0/d15/f3c zero size 2024-09-16T23:48:54.853 INFO:tasks.workunit.client.0.smithi042.stdout:2/346: unlink d0/c6 0 2024-09-16T23:48:54.855 INFO:tasks.workunit.client.0.smithi042.stdout:2/347: readlink d0/da/d1b/d48/d81/l7f 0 2024-09-16T23:48:54.857 INFO:tasks.workunit.client.0.smithi042.stdout:5/333: creat d9/db/d1b/d31/d46/f6d x:0 0 0 2024-09-16T23:48:54.857 INFO:tasks.workunit.client.0.smithi042.stdout:6/287: creat d8/db/dd/d1b/d2d/f62 x:0 0 0 2024-09-16T23:48:54.859 INFO:tasks.workunit.client.0.smithi042.stdout:6/288: chown d8/db/fc 364858838 1 2024-09-16T23:48:54.860 INFO:tasks.workunit.client.0.smithi042.stdout:5/334: read d9/db/de/d10/d4b/d61/f4e [687441,86220] 0 2024-09-16T23:48:54.860 INFO:tasks.workunit.client.1.smithi043.stdout:2/377: symlink d8/d1f/l86 0 2024-09-16T23:48:54.862 INFO:tasks.workunit.client.0.smithi042.stdout:2/348: creat d0/d5d/f82 x:0 0 0 2024-09-16T23:48:54.862 INFO:tasks.workunit.client.0.smithi042.stdout:5/335: fdatasync d9/db/d19/d21/f2e 0 2024-09-16T23:48:54.863 INFO:tasks.workunit.client.1.smithi043.stdout:9/383: rename d4/d1c/d4f/c59 to d4/d5/d16/c71 0 2024-09-16T23:48:54.863 INFO:tasks.workunit.client.0.smithi042.stdout:2/349: chown d0/da/d1b/d48/f58 5 1 2024-09-16T23:48:54.864 INFO:tasks.workunit.client.0.smithi042.stdout:2/350: truncate d0/da/d17/f28 364621 0 2024-09-16T23:48:54.866 INFO:tasks.workunit.client.0.smithi042.stdout:5/336: symlink d9/db/l6e 0 2024-09-16T23:48:54.867 INFO:tasks.workunit.client.1.smithi043.stdout:0/479: sync 2024-09-16T23:48:54.867 INFO:tasks.workunit.client.0.smithi042.stdout:6/289: rename d8/db/dd/d2c/d26/c2f to d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/c63 0 2024-09-16T23:48:54.867 INFO:tasks.workunit.client.0.smithi042.stdout:5/337: dread - d9/db/de/d10/d2f/f59 zero size 2024-09-16T23:48:54.869 INFO:tasks.workunit.client.0.smithi042.stdout:2/351: symlink d0/da/d1b/l83 0 2024-09-16T23:48:54.870 INFO:tasks.workunit.client.0.smithi042.stdout:2/352: chown d0/d26/f53 1473899494 1 2024-09-16T23:48:54.871 INFO:tasks.workunit.client.0.smithi042.stdout:0/222: getdents d0/df/d23 0 2024-09-16T23:48:54.873 INFO:tasks.workunit.client.0.smithi042.stdout:5/338: dread d9/db/d19/d21/f2b [0,4194304] 0 2024-09-16T23:48:54.875 INFO:tasks.workunit.client.1.smithi043.stdout:4/456: symlink d8/d15/d20/d2b/d54/d55/d49/d4c/l8e 0 2024-09-16T23:48:54.876 INFO:tasks.workunit.client.1.smithi043.stdout:4/457: dread - d8/d15/d20/d2b/f5a zero size 2024-09-16T23:48:54.880 INFO:tasks.workunit.client.1.smithi043.stdout:4/458: dread d8/d25/d64/f7f [0,4194304] 0 2024-09-16T23:48:54.881 INFO:tasks.workunit.client.0.smithi042.stdout:5/339: creat d9/db/de/f6f x:0 0 0 2024-09-16T23:48:54.886 INFO:tasks.workunit.client.1.smithi043.stdout:4/459: dread d8/d25/d64/f7f [0,4194304] 0 2024-09-16T23:48:54.888 INFO:tasks.workunit.client.0.smithi042.stdout:0/223: dwrite d0/df/f18 [0,4194304] 0 2024-09-16T23:48:54.889 INFO:tasks.workunit.client.1.smithi043.stdout:1/559: unlink d4/d9/f8e 0 2024-09-16T23:48:54.889 INFO:tasks.workunit.client.0.smithi042.stdout:5/340: getdents d9 0 2024-09-16T23:48:54.893 INFO:tasks.workunit.client.0.smithi042.stdout:2/353: dread d0/f12 [0,4194304] 0 2024-09-16T23:48:54.894 INFO:tasks.workunit.client.1.smithi043.stdout:3/439: dwrite d4/de/f58 [0,4194304] 0 2024-09-16T23:48:54.895 INFO:tasks.workunit.client.0.smithi042.stdout:5/341: creat d9/d4a/f70 x:0 0 0 2024-09-16T23:48:54.895 INFO:tasks.workunit.client.1.smithi043.stdout:3/440: fdatasync d4/d30/d53/f6a 0 2024-09-16T23:48:54.899 INFO:tasks.workunit.client.0.smithi042.stdout:0/224: mknod d0/df/d23/d3e/c40 0 2024-09-16T23:48:54.899 INFO:tasks.workunit.client.1.smithi043.stdout:8/451: write de/d18/d27/d41/d5a/f13 [1320348,8233] 0 2024-09-16T23:48:54.901 INFO:tasks.workunit.client.1.smithi043.stdout:8/452: read de/f54 [2760884,66325] 0 2024-09-16T23:48:54.902 INFO:tasks.workunit.client.0.smithi042.stdout:0/225: write d0/df/d23/f27 [459916,21386] 0 2024-09-16T23:48:54.902 INFO:tasks.workunit.client.0.smithi042.stdout:2/354: dread d0/da/d17/f59 [0,4194304] 0 2024-09-16T23:48:54.903 INFO:tasks.workunit.client.0.smithi042.stdout:0/226: fsync d0/df/d23/f38 0 2024-09-16T23:48:54.904 INFO:tasks.workunit.client.0.smithi042.stdout:5/342: link d9/db/d19/l1d d9/d4a/l71 0 2024-09-16T23:48:54.906 INFO:tasks.workunit.client.0.smithi042.stdout:0/227: symlink d0/df/l41 0 2024-09-16T23:48:54.906 INFO:tasks.workunit.client.1.smithi043.stdout:0/480: rename d0/d2/df/d1a/d46/c93 to d0/dc/ca0 0 2024-09-16T23:48:54.909 INFO:tasks.workunit.client.0.smithi042.stdout:0/228: unlink d0/c28 0 2024-09-16T23:48:54.910 INFO:tasks.workunit.client.0.smithi042.stdout:0/229: dread - d0/f37 zero size 2024-09-16T23:48:54.912 INFO:tasks.workunit.client.1.smithi043.stdout:5/616: truncate d3/d9/fc3 4346048 0 2024-09-16T23:48:54.913 INFO:tasks.workunit.client.0.smithi042.stdout:0/230: creat d0/df/f42 x:0 0 0 2024-09-16T23:48:54.914 INFO:tasks.workunit.client.0.smithi042.stdout:5/343: dwrite d9/db/d19/f39 [0,4194304] 0 2024-09-16T23:48:54.914 INFO:tasks.workunit.client.0.smithi042.stdout:0/231: fdatasync d0/d15/f3c 0 2024-09-16T23:48:54.917 INFO:tasks.workunit.client.0.smithi042.stdout:5/344: creat d9/db/d19/d60/f72 x:0 0 0 2024-09-16T23:48:54.918 INFO:tasks.workunit.client.0.smithi042.stdout:5/345: chown d9/c14 71783 1 2024-09-16T23:48:54.919 INFO:tasks.workunit.client.1.smithi043.stdout:5/617: dwrite d3/db/d15/f5f [0,4194304] 0 2024-09-16T23:48:54.919 INFO:tasks.workunit.client.0.smithi042.stdout:5/346: stat d9/db/d19/d21/d41 0 2024-09-16T23:48:54.922 INFO:tasks.workunit.client.0.smithi042.stdout:0/232: link d0/l6 d0/df/d13/d2d/l43 0 2024-09-16T23:48:54.922 INFO:tasks.workunit.client.1.smithi043.stdout:3/441: unlink d4/l6b 0 2024-09-16T23:48:54.924 INFO:tasks.workunit.client.0.smithi042.stdout:5/347: mkdir d9/db/d73 0 2024-09-16T23:48:54.924 INFO:tasks.workunit.client.1.smithi043.stdout:0/481: truncate d0/f3c 2822036 0 2024-09-16T23:48:54.927 INFO:tasks.workunit.client.0.smithi042.stdout:0/233: creat d0/df/d13/d2d/f44 x:0 0 0 2024-09-16T23:48:54.928 INFO:tasks.workunit.client.0.smithi042.stdout:5/348: unlink d9/db/d19/l6a 0 2024-09-16T23:48:54.930 INFO:tasks.workunit.client.0.smithi042.stdout:5/349: chown d9/db/d1b/c5b 1817 1 2024-09-16T23:48:54.930 INFO:tasks.workunit.client.0.smithi042.stdout:5/350: chown d9/d2c 430 1 2024-09-16T23:48:54.934 INFO:tasks.workunit.client.1.smithi043.stdout:7/431: sync 2024-09-16T23:48:54.935 INFO:tasks.workunit.client.1.smithi043.stdout:2/378: dwrite d8/d35/f36 [0,4194304] 0 2024-09-16T23:48:54.935 INFO:tasks.workunit.client.0.smithi042.stdout:2/355: dread d0/d26/f4e [0,4194304] 0 2024-09-16T23:48:54.937 INFO:tasks.workunit.client.1.smithi043.stdout:1/560: dread d4/df/d13/f57 [0,4194304] 0 2024-09-16T23:48:54.939 INFO:tasks.workunit.client.1.smithi043.stdout:9/384: creat d4/d1c/d25/d2b/d4c/f72 x:0 0 0 2024-09-16T23:48:54.944 INFO:tasks.workunit.client.0.smithi042.stdout:2/356: creat d0/d26/f84 x:0 0 0 2024-09-16T23:48:54.948 INFO:tasks.workunit.client.0.smithi042.stdout:0/234: write d0/d15/f1f [97814,46385] 0 2024-09-16T23:48:54.950 INFO:tasks.workunit.client.0.smithi042.stdout:5/351: dread d9/db/de/d10/d2f/f36 [0,4194304] 0 2024-09-16T23:48:54.952 INFO:tasks.workunit.client.0.smithi042.stdout:2/357: symlink d0/d78/l85 0 2024-09-16T23:48:54.952 INFO:tasks.workunit.client.0.smithi042.stdout:5/352: fdatasync d9/db/de/f6f 0 2024-09-16T23:48:54.954 INFO:tasks.workunit.client.0.smithi042.stdout:2/358: read d0/da/d17/f4d [1083965,107798] 0 2024-09-16T23:48:54.954 INFO:tasks.workunit.client.0.smithi042.stdout:5/353: creat d9/f74 x:0 0 0 2024-09-16T23:48:54.958 INFO:tasks.workunit.client.0.smithi042.stdout:5/354: unlink l8 0 2024-09-16T23:48:54.961 INFO:tasks.workunit.client.0.smithi042.stdout:5/355: symlink d9/db/de/d10/d4b/l75 0 2024-09-16T23:48:54.961 INFO:tasks.workunit.client.1.smithi043.stdout:6/459: rename de/d1a/d4d to de/d23/d34/d9b 0 2024-09-16T23:48:54.963 INFO:tasks.workunit.client.1.smithi043.stdout:4/460: getdents d8/d25 0 2024-09-16T23:48:54.964 INFO:tasks.workunit.client.0.smithi042.stdout:5/356: unlink d9/db/de/d10/d2f/f59 0 2024-09-16T23:48:54.964 INFO:tasks.workunit.client.0.smithi042.stdout:2/359: unlink d0/d5d/f64 0 2024-09-16T23:48:54.965 INFO:tasks.workunit.client.1.smithi043.stdout:7/432: dread d4/db/f14 [0,4194304] 0 2024-09-16T23:48:54.965 INFO:tasks.workunit.client.0.smithi042.stdout:5/357: read - d9/db/de/f6f zero size 2024-09-16T23:48:54.966 INFO:tasks.workunit.client.1.smithi043.stdout:7/433: chown d4/db/d28/c3b 216885327 1 2024-09-16T23:48:54.966 INFO:tasks.workunit.client.0.smithi042.stdout:0/235: dread d0/f9 [4194304,4194304] 0 2024-09-16T23:48:54.968 INFO:tasks.workunit.client.1.smithi043.stdout:9/385: creat d4/d28/d49/f73 x:0 0 0 2024-09-16T23:48:54.970 INFO:tasks.workunit.client.1.smithi043.stdout:7/434: chown d4/db/d11/d21/d31 224757492 1 2024-09-16T23:48:54.970 INFO:tasks.workunit.client.1.smithi043.stdout:3/442: creat d4/f7e x:0 0 0 2024-09-16T23:48:54.970 INFO:tasks.workunit.client.0.smithi042.stdout:5/358: rename d9/db/d19/l1d to d9/db/de/d10/d4b/l76 0 2024-09-16T23:48:54.972 INFO:tasks.workunit.client.0.smithi042.stdout:0/236: dwrite d0/f1c [0,4194304] 0 2024-09-16T23:48:54.973 INFO:tasks.workunit.client.1.smithi043.stdout:0/482: dwrite d0/d2/df/d1a/d46/d39/f45 [0,4194304] 0 2024-09-16T23:48:54.974 INFO:tasks.workunit.client.1.smithi043.stdout:7/435: chown d4/db/d11/d21/f2f 1 1 2024-09-16T23:48:54.974 INFO:tasks.workunit.client.1.smithi043.stdout:4/461: dwrite d8/d25/d64/f7d [0,4194304] 0 2024-09-16T23:48:54.977 INFO:tasks.workunit.client.1.smithi043.stdout:6/460: symlink de/d23/d34/d9b/d8b/l9c 0 2024-09-16T23:48:54.980 INFO:tasks.workunit.client.1.smithi043.stdout:4/462: write d8/f5d [323956,102376] 0 2024-09-16T23:48:54.984 INFO:tasks.workunit.client.0.smithi042.stdout:0/237: unlink d0/l2a 0 2024-09-16T23:48:54.988 INFO:tasks.workunit.client.1.smithi043.stdout:2/379: dwrite d8/d5e/d83/d3a/f57 [0,4194304] 0 2024-09-16T23:48:54.990 INFO:tasks.workunit.client.1.smithi043.stdout:8/453: rename de/d18/d1f/c3a to de/c8d 0 2024-09-16T23:48:54.993 INFO:tasks.workunit.client.1.smithi043.stdout:9/386: dwrite d4/d21/d2a/d2c/f69 [4194304,4194304] 0 2024-09-16T23:48:54.994 INFO:tasks.workunit.client.1.smithi043.stdout:5/618: rename d3/d9/d22/d4a/d4c to d3/d9/d22/d4a/d4c/dc8 22 2024-09-16T23:48:54.995 INFO:tasks.workunit.client.0.smithi042.stdout:5/359: dread d9/db/de/d10/f20 [0,4194304] 0 2024-09-16T23:48:54.996 INFO:tasks.workunit.client.1.smithi043.stdout:4/463: getdents d8/d25/d39/d41/d57/d85 0 2024-09-16T23:48:54.998 INFO:tasks.workunit.client.0.smithi042.stdout:0/238: rename d0/f1c to d0/d15/f45 0 2024-09-16T23:48:54.999 INFO:tasks.workunit.client.1.smithi043.stdout:3/443: getdents d4/d30 0 2024-09-16T23:48:55.004 INFO:tasks.workunit.client.1.smithi043.stdout:1/561: rename d4/d9/d1c/f37 to d4/d9/d12/fc2 0 2024-09-16T23:48:55.004 INFO:tasks.workunit.client.1.smithi043.stdout:9/387: mknod d4/d21/c74 0 2024-09-16T23:48:55.004 INFO:tasks.workunit.client.0.smithi042.stdout:0/239: truncate d0/df/f19 3330860 0 2024-09-16T23:48:55.005 INFO:tasks.workunit.client.1.smithi043.stdout:2/380: dread d8/d1f/f6e [0,4194304] 0 2024-09-16T23:48:55.005 INFO:tasks.workunit.client.1.smithi043.stdout:0/483: getdents d0/d2/df/d1a/d49/d80/d88 0 2024-09-16T23:48:55.007 INFO:tasks.workunit.client.1.smithi043.stdout:6/461: write de/d23/d49/f7c [911954,28402] 0 2024-09-16T23:48:55.009 INFO:tasks.workunit.client.1.smithi043.stdout:3/444: creat d4/de/f7f x:0 0 0 2024-09-16T23:48:55.009 INFO:tasks.workunit.client.1.smithi043.stdout:9/388: truncate d4/f10 4322233 0 2024-09-16T23:48:55.011 INFO:tasks.workunit.client.1.smithi043.stdout:4/464: unlink d8/d15/d20/d2b/d22/f36 0 2024-09-16T23:48:55.011 INFO:tasks.workunit.client.0.smithi042.stdout:0/240: unlink d0/df/f42 0 2024-09-16T23:48:55.014 INFO:tasks.workunit.client.0.smithi042.stdout:5/360: link d9/db/d11/c4f d9/d2c/c77 0 2024-09-16T23:48:55.015 INFO:tasks.workunit.client.1.smithi043.stdout:8/454: rename de/d70/d74 to de/d18/d27/d8e 0 2024-09-16T23:48:55.015 INFO:tasks.workunit.client.0.smithi042.stdout:5/361: fdatasync d9/db/d1b/f56 0 2024-09-16T23:48:55.016 INFO:tasks.workunit.client.1.smithi043.stdout:3/445: dread d4/d30/f41 [0,4194304] 0 2024-09-16T23:48:55.017 INFO:tasks.workunit.client.0.smithi042.stdout:5/362: dread - d9/db/d1b/f56 zero size 2024-09-16T23:48:55.017 INFO:tasks.workunit.client.0.smithi042.stdout:5/363: dread - d9/db/de/d5e/f67 zero size 2024-09-16T23:48:55.017 INFO:tasks.workunit.client.1.smithi043.stdout:6/462: stat de/d1a/d28/f77 0 2024-09-16T23:48:55.017 INFO:tasks.workunit.client.1.smithi043.stdout:9/389: mknod d4/d21/c75 0 2024-09-16T23:48:55.017 INFO:tasks.workunit.client.1.smithi043.stdout:3/446: chown d4/de 307 1 2024-09-16T23:48:55.018 INFO:tasks.workunit.client.0.smithi042.stdout:5/364: stat d9/d4a/f70 0 2024-09-16T23:48:55.018 INFO:tasks.workunit.client.1.smithi043.stdout:0/484: creat d0/fa1 x:0 0 0 2024-09-16T23:48:55.020 INFO:tasks.workunit.client.1.smithi043.stdout:9/390: truncate d4/d28/d49/f73 486577 0 2024-09-16T23:48:55.020 INFO:tasks.workunit.client.0.smithi042.stdout:0/241: rmdir d0/df/d13/d2d 39 2024-09-16T23:48:55.021 INFO:tasks.workunit.client.1.smithi043.stdout:1/562: rename d4/d9/d1c/d1e/d47/d97/cb2 to d4/d9/d1c/d1e/d42/cc3 0 2024-09-16T23:48:55.021 INFO:tasks.workunit.client.0.smithi042.stdout:0/242: write d0/d15/f1f [351655,119346] 0 2024-09-16T23:48:55.021 INFO:tasks.workunit.client.1.smithi043.stdout:9/391: fdatasync d4/d1c/f6f 0 2024-09-16T23:48:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:54 smithi042 ceph-mon[27914]: pgmap v6: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:55.023 INFO:tasks.workunit.client.0.smithi042.stdout:5/365: chown d9/db/d19/d21/d41/c5a 1020 1 2024-09-16T23:48:55.023 INFO:tasks.workunit.client.1.smithi043.stdout:1/563: fdatasync d4/d9/d12/d2b/f52 0 2024-09-16T23:48:55.024 INFO:tasks.workunit.client.0.smithi042.stdout:5/366: write d9/db/de/f6f [673067,129890] 0 2024-09-16T23:48:55.025 INFO:tasks.workunit.client.1.smithi043.stdout:2/381: dread d8/d1d/d20/d26/f27 [0,4194304] 0 2024-09-16T23:48:55.026 INFO:tasks.workunit.client.1.smithi043.stdout:4/465: creat d8/d25/d39/d58/d70/d8b/f8f x:0 0 0 2024-09-16T23:48:55.029 INFO:tasks.workunit.client.1.smithi043.stdout:0/485: rmdir d0/d2/df/d1a/d49 39 2024-09-16T23:48:55.030 INFO:tasks.workunit.client.1.smithi043.stdout:3/447: dread d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:48:55.033 INFO:tasks.workunit.client.1.smithi043.stdout:6/463: mkdir de/d23/d34/d66/d9d 0 2024-09-16T23:48:55.033 INFO:tasks.workunit.client.0.smithi042.stdout:5/367: link d9/db/l3f d9/db/d19/l78 0 2024-09-16T23:48:55.034 INFO:tasks.workunit.client.1.smithi043.stdout:6/464: read de/f24 [2606672,113181] 0 2024-09-16T23:48:55.035 INFO:tasks.workunit.client.1.smithi043.stdout:8/455: creat de/d18/d27/d41/f8f x:0 0 0 2024-09-16T23:48:55.036 INFO:tasks.workunit.client.0.smithi042.stdout:0/243: read d0/df/f19 [1374775,61391] 0 2024-09-16T23:48:55.036 INFO:tasks.workunit.client.1.smithi043.stdout:8/456: fdatasync de/d18/d1f/f32 0 2024-09-16T23:48:55.036 INFO:tasks.workunit.client.1.smithi043.stdout:6/465: dread - de/d23/d34/d66/d70/f76 zero size 2024-09-16T23:48:55.037 INFO:tasks.workunit.client.1.smithi043.stdout:3/448: creat d4/de/d40/f80 x:0 0 0 2024-09-16T23:48:55.037 INFO:tasks.workunit.client.1.smithi043.stdout:9/392: truncate d4/d1c/d4f/f5b 1566249 0 2024-09-16T23:48:55.039 INFO:tasks.workunit.client.1.smithi043.stdout:1/564: link d4/df/f91 d4/d9/d12/d2b/d3d/d83/fc4 0 2024-09-16T23:48:55.039 INFO:tasks.workunit.client.0.smithi042.stdout:0/244: write d0/f9 [5611665,108177] 0 2024-09-16T23:48:55.040 INFO:tasks.workunit.client.0.smithi042.stdout:0/245: truncate d0/d15/f3c 816639 0 2024-09-16T23:48:55.040 INFO:tasks.workunit.client.1.smithi043.stdout:9/393: write d4/d1c/f27 [3674743,28229] 0 2024-09-16T23:48:55.040 INFO:tasks.workunit.client.1.smithi043.stdout:0/486: symlink d0/la2 0 2024-09-16T23:48:55.041 INFO:tasks.workunit.client.0.smithi042.stdout:5/368: truncate d9/f3c 556332 0 2024-09-16T23:48:55.043 INFO:tasks.workunit.client.1.smithi043.stdout:6/466: creat de/d2a/f9e x:0 0 0 2024-09-16T23:48:55.043 INFO:tasks.workunit.client.1.smithi043.stdout:2/382: dread d8/d11/d16/f28 [0,4194304] 0 2024-09-16T23:48:55.044 INFO:tasks.workunit.client.0.smithi042.stdout:5/369: mkdir d9/db/d1b/d79 0 2024-09-16T23:48:55.044 INFO:tasks.workunit.client.1.smithi043.stdout:1/565: symlink d4/d9/d1c/d1e/lc5 0 2024-09-16T23:48:55.045 INFO:tasks.workunit.client.0.smithi042.stdout:0/246: write d0/df/d23/f38 [543180,27847] 0 2024-09-16T23:48:55.045 INFO:tasks.workunit.client.1.smithi043.stdout:2/383: write d8/fc [3597671,54393] 0 2024-09-16T23:48:55.046 INFO:tasks.workunit.client.1.smithi043.stdout:3/449: rename d4/de/d1f/c28 to d4/d30/c81 0 2024-09-16T23:48:55.047 INFO:tasks.workunit.client.0.smithi042.stdout:5/370: creat d9/db/d1b/d31/f7a x:0 0 0 2024-09-16T23:48:55.047 INFO:tasks.workunit.client.1.smithi043.stdout:9/394: creat d4/d28/d49/f76 x:0 0 0 2024-09-16T23:48:55.049 INFO:tasks.workunit.client.1.smithi043.stdout:4/466: getdents d8/d25 0 2024-09-16T23:48:55.050 INFO:tasks.workunit.client.0.smithi042.stdout:0/247: rename d0/df/l24 to d0/df/d23/l46 0 2024-09-16T23:48:55.052 INFO:tasks.workunit.client.1.smithi043.stdout:4/467: stat d8/d15/d20/d2b/d54/d55/cf 0 2024-09-16T23:48:55.053 INFO:tasks.workunit.client.1.smithi043.stdout:8/457: dwrite de/d18/d1f/d2f/d39/d7b/d83/f84 [0,4194304] 0 2024-09-16T23:48:55.053 INFO:tasks.workunit.client.1.smithi043.stdout:1/566: fdatasync d4/df/d26/f39 0 2024-09-16T23:48:55.054 INFO:tasks.workunit.client.0.smithi042.stdout:0/248: mknod d0/df/d23/c47 0 2024-09-16T23:48:55.057 INFO:tasks.workunit.client.1.smithi043.stdout:1/567: write d4/d34/f41 [5016478,128984] 0 2024-09-16T23:48:55.057 INFO:tasks.workunit.client.1.smithi043.stdout:3/450: creat d4/de/d40/f82 x:0 0 0 2024-09-16T23:48:55.058 INFO:tasks.workunit.client.0.smithi042.stdout:5/371: truncate d9/db/de/d10/d2f/f36 368887 0 2024-09-16T23:48:55.059 INFO:tasks.workunit.client.1.smithi043.stdout:3/451: write d4/d30/d53/f6a [1045460,62272] 0 2024-09-16T23:48:55.061 INFO:tasks.workunit.client.0.smithi042.stdout:0/249: truncate d0/df/f14 3591743 0 2024-09-16T23:48:55.062 INFO:tasks.workunit.client.1.smithi043.stdout:9/395: creat d4/d1c/d4f/f77 x:0 0 0 2024-09-16T23:48:55.063 INFO:tasks.workunit.client.1.smithi043.stdout:6/467: truncate de/d1a/f38 1058114 0 2024-09-16T23:48:55.065 INFO:tasks.workunit.client.1.smithi043.stdout:6/468: write de/d23/d34/d66/f93 [218774,34107] 0 2024-09-16T23:48:55.066 INFO:tasks.workunit.client.1.smithi043.stdout:4/468: fsync d8/f1d 0 2024-09-16T23:48:55.066 INFO:tasks.workunit.client.0.smithi042.stdout:5/372: mkdir d9/d2c/d7b 0 2024-09-16T23:48:55.068 INFO:tasks.workunit.client.1.smithi043.stdout:6/469: write de/d23/d49/f7c [1473039,35601] 0 2024-09-16T23:48:55.069 INFO:tasks.workunit.client.1.smithi043.stdout:0/487: creat d0/fa3 x:0 0 0 2024-09-16T23:48:55.070 INFO:tasks.workunit.client.0.smithi042.stdout:0/250: readlink d0/d15/l30 0 2024-09-16T23:48:55.071 INFO:tasks.workunit.client.1.smithi043.stdout:0/488: read d0/de/f91 [90388,27088] 0 2024-09-16T23:48:55.073 INFO:tasks.workunit.client.1.smithi043.stdout:1/568: fsync d4/df/d26/f39 0 2024-09-16T23:48:55.076 INFO:tasks.workunit.client.1.smithi043.stdout:2/384: link d8/d5e/d83/d3a/f57 d8/d5e/f87 0 2024-09-16T23:48:55.078 INFO:tasks.workunit.client.1.smithi043.stdout:4/469: dwrite d8/d15/f69 [0,4194304] 0 2024-09-16T23:48:55.078 INFO:tasks.workunit.client.1.smithi043.stdout:3/452: creat d4/de/d35/f83 x:0 0 0 2024-09-16T23:48:55.079 INFO:tasks.workunit.client.0.smithi042.stdout:0/251: mkdir d0/d48 0 2024-09-16T23:48:55.079 INFO:tasks.workunit.client.1.smithi043.stdout:2/385: chown d8/d5e/d83/d3a/f72 0 1 2024-09-16T23:48:55.080 INFO:tasks.workunit.client.1.smithi043.stdout:3/453: chown d4/f3b 27637 1 2024-09-16T23:48:55.081 INFO:tasks.workunit.client.1.smithi043.stdout:9/396: write d4/d5/d16/f1f [287707,55132] 0 2024-09-16T23:48:55.081 INFO:tasks.workunit.client.0.smithi042.stdout:5/373: creat d9/db/de/d10/d4b/d61/f7c x:0 0 0 2024-09-16T23:48:55.082 INFO:tasks.workunit.client.1.smithi043.stdout:4/470: dread - d8/d25/d39/d58/d70/f88 zero size 2024-09-16T23:48:55.083 INFO:tasks.workunit.client.0.smithi042.stdout:5/374: truncate d9/db/d1b/d31/d46/f6d 917470 0 2024-09-16T23:48:55.084 INFO:tasks.workunit.client.1.smithi043.stdout:9/397: fsync d4/d21/d2a/d2c/f69 0 2024-09-16T23:48:55.084 INFO:tasks.workunit.client.0.smithi042.stdout:5/375: fsync d9/db/d19/f52 0 2024-09-16T23:48:55.086 INFO:tasks.workunit.client.1.smithi043.stdout:6/470: dread de/d23/d34/d66/f6e [0,4194304] 0 2024-09-16T23:48:55.087 INFO:tasks.workunit.client.0.smithi042.stdout:5/376: mkdir d9/db/d19/d21/d41/d7d 0 2024-09-16T23:48:55.088 INFO:tasks.workunit.client.0.smithi042.stdout:5/377: fdatasync d9/db/d1b/d31/d46/f6d 0 2024-09-16T23:48:55.091 INFO:tasks.workunit.client.1.smithi043.stdout:9/398: dwrite d4/d1c/d4f/d45/f6a [0,4194304] 0 2024-09-16T23:48:55.092 INFO:tasks.workunit.client.1.smithi043.stdout:9/399: chown d4/d21 188481629 1 2024-09-16T23:48:55.093 INFO:tasks.workunit.client.1.smithi043.stdout:1/569: rmdir d4/d3e 39 2024-09-16T23:48:55.096 INFO:tasks.workunit.client.1.smithi043.stdout:2/386: write d8/d5e/d83/d3a/f57 [4312978,528] 0 2024-09-16T23:48:55.097 INFO:tasks.workunit.client.1.smithi043.stdout:3/454: mknod d4/de/d35/c84 0 2024-09-16T23:48:55.098 INFO:tasks.workunit.client.0.smithi042.stdout:5/378: dwrite d9/db/d19/d60/f72 [0,4194304] 0 2024-09-16T23:48:55.101 INFO:tasks.workunit.client.0.smithi042.stdout:5/379: chown d9/db/d11/c4f 21690 1 2024-09-16T23:48:55.101 INFO:tasks.workunit.client.1.smithi043.stdout:8/458: creat de/f90 x:0 0 0 2024-09-16T23:48:55.102 INFO:tasks.workunit.client.0.smithi042.stdout:5/380: chown d9/db/d11/l62 832390881 1 2024-09-16T23:48:55.102 INFO:tasks.workunit.client.1.smithi043.stdout:3/455: chown d4/de/d1f/d34/f3a 13840954 1 2024-09-16T23:48:55.106 INFO:tasks.workunit.client.1.smithi043.stdout:9/400: dwrite d4/d5/d16/f1f [0,4194304] 0 2024-09-16T23:48:55.106 INFO:tasks.workunit.client.1.smithi043.stdout:8/459: dread - de/d18/d27/f85 zero size 2024-09-16T23:48:55.110 INFO:tasks.workunit.client.1.smithi043.stdout:1/570: dwrite d4/df/d26/d79/f96 [0,4194304] 0 2024-09-16T23:48:55.110 INFO:tasks.workunit.client.1.smithi043.stdout:0/489: creat d0/d2/df/d1a/d49/d80/d88/fa4 x:0 0 0 2024-09-16T23:48:55.113 INFO:tasks.workunit.client.0.smithi042.stdout:5/381: truncate d9/db/d11/f40 2181097 0 2024-09-16T23:48:55.114 INFO:tasks.workunit.client.1.smithi043.stdout:2/387: creat d8/d4b/d4f/d7c/f88 x:0 0 0 2024-09-16T23:48:55.116 INFO:tasks.workunit.client.1.smithi043.stdout:2/388: write d8/d1d/d20/d26/f60 [392367,66146] 0 2024-09-16T23:48:55.119 INFO:tasks.workunit.client.0.smithi042.stdout:5/382: dread d9/db/d19/f39 [0,4194304] 0 2024-09-16T23:48:55.121 INFO:tasks.workunit.client.1.smithi043.stdout:8/460: mknod de/d18/d1f/d25/c91 0 2024-09-16T23:48:55.122 INFO:tasks.workunit.client.1.smithi043.stdout:0/490: rmdir d0/d2 39 2024-09-16T23:48:55.122 INFO:tasks.workunit.client.1.smithi043.stdout:8/461: readlink de/d18/d27/l6d 0 2024-09-16T23:48:55.123 INFO:tasks.workunit.client.1.smithi043.stdout:8/462: dread - de/f90 zero size 2024-09-16T23:48:55.125 INFO:tasks.workunit.client.1.smithi043.stdout:8/463: write de/d18/d27/d41/d5a/f1b [3591101,23708] 0 2024-09-16T23:48:55.126 INFO:tasks.workunit.client.0.smithi042.stdout:5/383: dread d9/db/f25 [0,4194304] 0 2024-09-16T23:48:55.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:54 smithi043 ceph-mon[35191]: pgmap v6: 65 pgs: 65 active+clean; 951 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail 2024-09-16T23:48:55.128 INFO:tasks.workunit.client.1.smithi043.stdout:2/389: fsync d8/d4b/f5b 0 2024-09-16T23:48:55.130 INFO:tasks.workunit.client.0.smithi042.stdout:5/384: fdatasync d9/db/de/d10/d2f/f36 0 2024-09-16T23:48:55.131 INFO:tasks.workunit.client.1.smithi043.stdout:4/471: dwrite d8/d25/f42 [0,4194304] 0 2024-09-16T23:48:55.131 INFO:tasks.workunit.client.1.smithi043.stdout:2/390: fsync d8/d1d/d20/d23/d38/f69 0 2024-09-16T23:48:55.134 INFO:tasks.workunit.client.1.smithi043.stdout:8/464: mknod de/d18/d27/d41/d5a/c92 0 2024-09-16T23:48:55.135 INFO:tasks.workunit.client.1.smithi043.stdout:8/465: dread - de/d18/d27/d41/f8f zero size 2024-09-16T23:48:55.136 INFO:tasks.workunit.client.1.smithi043.stdout:8/466: readlink de/d18/d27/l89 0 2024-09-16T23:48:55.136 INFO:tasks.workunit.client.1.smithi043.stdout:6/471: truncate de/f47 3039914 0 2024-09-16T23:48:55.137 INFO:tasks.workunit.client.1.smithi043.stdout:8/467: readlink de/d18/d27/l6d 0 2024-09-16T23:48:55.138 INFO:tasks.workunit.client.0.smithi042.stdout:5/385: dwrite d9/db/de/d10/d4b/d61/f4e [0,4194304] 0 2024-09-16T23:48:55.139 INFO:tasks.workunit.client.1.smithi043.stdout:0/491: dread d0/d2/df/f16 [0,4194304] 0 2024-09-16T23:48:55.141 INFO:tasks.workunit.client.1.smithi043.stdout:9/401: rename d4/d1c/d4f/d45/l5c to d4/d1c/d25/d2b/d4c/l78 0 2024-09-16T23:48:55.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/386: creat d9/db/d19/d21/d41/d7d/f7e x:0 0 0 2024-09-16T23:48:55.143 INFO:tasks.workunit.client.0.smithi042.stdout:5/387: read d9/db/f3e [3553537,65017] 0 2024-09-16T23:48:55.143 INFO:tasks.workunit.client.1.smithi043.stdout:6/472: fsync de/d1a/d28/f7d 0 2024-09-16T23:48:55.145 INFO:tasks.workunit.client.1.smithi043.stdout:2/391: symlink d8/d5e/d83/d3a/d77/l89 0 2024-09-16T23:48:55.149 INFO:tasks.workunit.client.1.smithi043.stdout:2/392: stat d8/d1d/d25/f2b 0 2024-09-16T23:48:55.152 INFO:tasks.workunit.client.1.smithi043.stdout:1/571: rename d4/d9/d1c/d1e/d42/d5d/d7a to d4/d9/d8f/db6/dc6 0 2024-09-16T23:48:55.152 INFO:tasks.workunit.client.1.smithi043.stdout:2/393: readlink d8/l7f 0 2024-09-16T23:48:55.155 INFO:tasks.workunit.client.1.smithi043.stdout:4/472: truncate d8/d25/f5b 3119668 0 2024-09-16T23:48:55.155 INFO:tasks.workunit.client.0.smithi042.stdout:5/388: write d9/db/de/d10/d2f/f36 [1360640,26356] 0 2024-09-16T23:48:55.156 INFO:tasks.workunit.client.0.smithi042.stdout:5/389: chown d9/db/l6e 81195290 1 2024-09-16T23:48:55.156 INFO:tasks.workunit.client.1.smithi043.stdout:1/572: rmdir d4/d9/d12/d2b 39 2024-09-16T23:48:55.156 INFO:tasks.workunit.client.1.smithi043.stdout:8/468: dwrite de/d18/d27/d8e/f80 [0,4194304] 0 2024-09-16T23:48:55.158 INFO:tasks.workunit.client.1.smithi043.stdout:3/456: dwrite d4/f54 [0,4194304] 0 2024-09-16T23:48:55.158 INFO:tasks.workunit.client.0.smithi042.stdout:5/390: read d9/f3c [543008,39826] 0 2024-09-16T23:48:55.160 INFO:tasks.workunit.client.1.smithi043.stdout:1/573: stat d4/d9/d8f/db6/dc6 0 2024-09-16T23:48:55.160 INFO:tasks.workunit.client.1.smithi043.stdout:8/469: write de/f71 [3398652,115697] 0 2024-09-16T23:48:55.161 INFO:tasks.workunit.client.0.smithi042.stdout:5/391: readlink d9/db/de/d10/d4b/d61/d43/l54 0 2024-09-16T23:48:55.161 INFO:tasks.workunit.client.0.smithi042.stdout:5/392: fdatasync d9/db/de/f6f 0 2024-09-16T23:48:55.165 INFO:tasks.workunit.client.1.smithi043.stdout:6/473: link de/d23/d34/d66/d70/l7e de/d23/l9f 0 2024-09-16T23:48:55.170 INFO:tasks.workunit.client.1.smithi043.stdout:6/474: unlink de/d23/d49/c6b 0 2024-09-16T23:48:55.171 INFO:tasks.workunit.client.1.smithi043.stdout:0/492: dread d0/d2/f33 [0,4194304] 0 2024-09-16T23:48:55.171 INFO:tasks.workunit.client.1.smithi043.stdout:2/394: getdents d8/d4b 0 2024-09-16T23:48:55.173 INFO:tasks.workunit.client.1.smithi043.stdout:2/395: read - d8/d11/f6f zero size 2024-09-16T23:48:55.175 INFO:tasks.workunit.client.1.smithi043.stdout:6/475: creat de/d1a/d39/fa0 x:0 0 0 2024-09-16T23:48:55.176 INFO:tasks.workunit.client.1.smithi043.stdout:2/396: read d8/d1d/d20/d26/f60 [1395014,74904] 0 2024-09-16T23:48:55.177 INFO:tasks.workunit.client.1.smithi043.stdout:1/574: getdents d4/d9/d12/d15 0 2024-09-16T23:48:55.179 INFO:tasks.workunit.client.1.smithi043.stdout:4/473: dread d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:48:55.179 INFO:tasks.workunit.client.1.smithi043.stdout:2/397: read - d8/d1d/d20/d26/f7d zero size 2024-09-16T23:48:55.181 INFO:tasks.workunit.client.1.smithi043.stdout:0/493: dread d0/f3c [0,4194304] 0 2024-09-16T23:48:55.182 INFO:tasks.workunit.client.1.smithi043.stdout:6/476: rename de/d23/d63/l65 to de/d1a/d39/la1 0 2024-09-16T23:48:55.183 INFO:tasks.workunit.client.1.smithi043.stdout:9/402: truncate d4/d21/d35/f5d 2568760 0 2024-09-16T23:48:55.185 INFO:tasks.workunit.client.1.smithi043.stdout:3/457: write d4/de/d1f/f46 [32319,82103] 0 2024-09-16T23:48:55.185 INFO:tasks.workunit.client.1.smithi043.stdout:6/477: read de/d1a/d28/f9a [1358236,36799] 0 2024-09-16T23:48:55.186 INFO:tasks.workunit.client.1.smithi043.stdout:2/398: mknod d8/d4b/d4f/d7c/c8a 0 2024-09-16T23:48:55.186 INFO:tasks.workunit.client.1.smithi043.stdout:8/470: write de/d70/d3d/f51 [2587310,74830] 0 2024-09-16T23:48:55.186 INFO:tasks.workunit.client.1.smithi043.stdout:4/474: dread d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:48:55.187 INFO:tasks.workunit.client.1.smithi043.stdout:0/494: rename d0/d2/df/d1a/d46/d39/d6e/c3e to d0/d2/dd/ca5 0 2024-09-16T23:48:55.188 INFO:tasks.workunit.client.1.smithi043.stdout:2/399: write d8/f1a [2485000,31216] 0 2024-09-16T23:48:55.189 INFO:tasks.workunit.client.1.smithi043.stdout:6/478: chown de/d23/d34/d66/f93 1692 1 2024-09-16T23:48:55.190 INFO:tasks.workunit.client.1.smithi043.stdout:4/475: dread d8/f1d [0,4194304] 0 2024-09-16T23:48:55.197 INFO:tasks.workunit.client.1.smithi043.stdout:0/495: mknod d0/d2/df/d1a/d46/ca6 0 2024-09-16T23:48:55.198 INFO:tasks.workunit.client.1.smithi043.stdout:9/403: creat d4/d21/d33/f79 x:0 0 0 2024-09-16T23:48:55.199 INFO:tasks.workunit.client.1.smithi043.stdout:0/496: chown d0/d2/c68 88703785 1 2024-09-16T23:48:55.199 INFO:tasks.workunit.client.1.smithi043.stdout:9/404: chown d4/ld 72768 1 2024-09-16T23:48:55.201 INFO:tasks.workunit.client.1.smithi043.stdout:9/405: chown d4/d28/d49/f73 522476 1 2024-09-16T23:48:55.202 INFO:tasks.workunit.client.1.smithi043.stdout:9/406: readlink d4/ld 0 2024-09-16T23:48:55.203 INFO:tasks.workunit.client.1.smithi043.stdout:2/400: mknod d8/d11/c8b 0 2024-09-16T23:48:55.203 INFO:tasks.workunit.client.1.smithi043.stdout:1/575: dwrite d4/d9/d12/fc2 [0,4194304] 0 2024-09-16T23:48:55.203 INFO:tasks.workunit.client.1.smithi043.stdout:4/476: dread d8/d15/d20/d2b/f28 [0,4194304] 0 2024-09-16T23:48:55.204 INFO:tasks.workunit.client.1.smithi043.stdout:8/471: write de/d70/d3d/f42 [5079933,111293] 0 2024-09-16T23:48:55.206 INFO:tasks.workunit.client.1.smithi043.stdout:3/458: creat d4/de/d1f/f85 x:0 0 0 2024-09-16T23:48:55.208 INFO:tasks.workunit.client.1.smithi043.stdout:6/479: creat de/d23/fa2 x:0 0 0 2024-09-16T23:48:55.208 INFO:tasks.workunit.client.1.smithi043.stdout:8/472: truncate de/f17 4978724 0 2024-09-16T23:48:55.212 INFO:tasks.workunit.client.1.smithi043.stdout:9/407: creat d4/d21/d2a/f7a x:0 0 0 2024-09-16T23:48:55.215 INFO:tasks.workunit.client.1.smithi043.stdout:4/477: mknod d8/d25/d64/c90 0 2024-09-16T23:48:55.217 INFO:tasks.workunit.client.1.smithi043.stdout:1/576: creat d4/d3e/fc7 x:0 0 0 2024-09-16T23:48:55.219 INFO:tasks.workunit.client.1.smithi043.stdout:4/478: write d8/d15/d20/d2b/d22/f24 [5350644,51649] 0 2024-09-16T23:48:55.219 INFO:tasks.workunit.client.1.smithi043.stdout:2/401: dwrite d8/d1d/d20/d23/f6a [0,4194304] 0 2024-09-16T23:48:55.220 INFO:tasks.workunit.client.1.smithi043.stdout:6/480: mkdir de/d23/d34/d9b/d5a/da3 0 2024-09-16T23:48:55.223 INFO:tasks.workunit.client.1.smithi043.stdout:4/479: write d8/d15/d20/f8d [1044103,39184] 0 2024-09-16T23:48:55.223 INFO:tasks.workunit.client.1.smithi043.stdout:0/497: dread d0/d2/df/d1a/f47 [0,4194304] 0 2024-09-16T23:48:55.224 INFO:tasks.workunit.client.1.smithi043.stdout:6/481: write de/f40 [4198889,6476] 0 2024-09-16T23:48:55.225 INFO:tasks.workunit.client.1.smithi043.stdout:9/408: write d4/f48 [211576,119792] 0 2024-09-16T23:48:55.227 INFO:tasks.workunit.client.1.smithi043.stdout:4/480: readlink d8/d15/d20/l33 0 2024-09-16T23:48:55.229 INFO:tasks.workunit.client.1.smithi043.stdout:9/409: chown d4/d1c/d4f/l42 147061 1 2024-09-16T23:48:55.229 INFO:tasks.workunit.client.1.smithi043.stdout:9/410: stat c0 0 2024-09-16T23:48:55.230 INFO:tasks.workunit.client.1.smithi043.stdout:2/402: rename d8/d4b/f64 to d8/d1f/d6b/f8c 0 2024-09-16T23:48:55.232 INFO:tasks.workunit.client.1.smithi043.stdout:8/473: truncate de/d18/d27/f75 2372712 0 2024-09-16T23:48:55.240 INFO:tasks.workunit.client.1.smithi043.stdout:0/498: getdents d0/dc 0 2024-09-16T23:48:55.240 INFO:tasks.workunit.client.1.smithi043.stdout:9/411: mkdir d4/d21/d2a/d2c/d7b 0 2024-09-16T23:48:55.241 INFO:tasks.workunit.client.1.smithi043.stdout:3/459: dwrite d4/de/d1f/d2c/f32 [0,4194304] 0 2024-09-16T23:48:55.241 INFO:tasks.workunit.client.1.smithi043.stdout:8/474: mknod de/c93 0 2024-09-16T23:48:55.243 INFO:tasks.workunit.client.1.smithi043.stdout:0/499: chown d0/d2 1 1 2024-09-16T23:48:55.244 INFO:tasks.workunit.client.1.smithi043.stdout:2/403: write d8/d5e/d83/f55 [163508,50078] 0 2024-09-16T23:48:55.245 INFO:tasks.workunit.client.1.smithi043.stdout:6/482: dwrite de/d1a/d39/fa0 [0,4194304] 0 2024-09-16T23:48:55.247 INFO:tasks.workunit.client.1.smithi043.stdout:1/577: dread d4/d9/d1c/d1e/f55 [0,4194304] 0 2024-09-16T23:48:55.249 INFO:tasks.workunit.client.1.smithi043.stdout:3/460: mkdir d4/de/d40/d86 0 2024-09-16T23:48:55.251 INFO:tasks.workunit.client.1.smithi043.stdout:4/481: dwrite d8/d25/d39/d58/d70/d8b/f8f [0,4194304] 0 2024-09-16T23:48:55.251 INFO:tasks.workunit.client.1.smithi043.stdout:0/500: unlink d0/l8d 0 2024-09-16T23:48:55.252 INFO:tasks.workunit.client.1.smithi043.stdout:9/412: mknod d4/d21/d2a/d2c/c7c 0 2024-09-16T23:48:55.257 INFO:tasks.workunit.client.1.smithi043.stdout:3/461: rmdir d4/de/d40 39 2024-09-16T23:48:55.263 INFO:tasks.workunit.client.1.smithi043.stdout:7/436: sync 2024-09-16T23:48:55.264 INFO:tasks.workunit.client.1.smithi043.stdout:5/619: sync 2024-09-16T23:48:55.265 INFO:tasks.workunit.client.1.smithi043.stdout:8/475: rename de/f15 to de/d18/d27/f94 0 2024-09-16T23:48:55.270 INFO:tasks.workunit.client.1.smithi043.stdout:4/482: link d8/d25/d39/d58/d70/d8b/f8f d8/d15/d20/d2b/d54/f91 0 2024-09-16T23:48:55.270 INFO:tasks.workunit.client.1.smithi043.stdout:7/437: mkdir d4/d27/d8a/d93 0 2024-09-16T23:48:55.270 INFO:tasks.workunit.client.1.smithi043.stdout:9/413: getdents d4/d5/d16 0 2024-09-16T23:48:55.270 INFO:tasks.workunit.client.1.smithi043.stdout:1/578: rename d4/d9/d1c/d1e/d47/d97 to d4/d9/d12/d2b/d3d/dc8 0 2024-09-16T23:48:55.272 INFO:tasks.workunit.client.1.smithi043.stdout:9/414: readlink d4/d21/d33/l37 0 2024-09-16T23:48:55.273 INFO:tasks.workunit.client.1.smithi043.stdout:5/620: creat d3/d9/d22/d4a/d64/d8f/fc9 x:0 0 0 2024-09-16T23:48:55.273 INFO:tasks.workunit.client.1.smithi043.stdout:8/476: mkdir de/d18/d1f/d25/d65/d7f/d95 0 2024-09-16T23:48:55.274 INFO:tasks.workunit.client.1.smithi043.stdout:0/501: truncate d0/d2/df/d1a/d46/f94 583142 0 2024-09-16T23:48:55.276 INFO:tasks.workunit.client.1.smithi043.stdout:4/483: write d8/d15/d20/d2b/f26 [2007756,18164] 0 2024-09-16T23:48:55.276 INFO:tasks.workunit.client.1.smithi043.stdout:6/483: dwrite de/d23/d49/f7c [0,4194304] 0 2024-09-16T23:48:55.276 INFO:tasks.workunit.client.1.smithi043.stdout:7/438: mkdir d4/db/d11/d21/d94 0 2024-09-16T23:48:55.278 INFO:tasks.workunit.client.1.smithi043.stdout:9/415: dread - d4/d21/d2a/f7a zero size 2024-09-16T23:48:55.284 INFO:tasks.workunit.client.1.smithi043.stdout:3/462: rename d4/de/d5d/c60 to d4/c87 0 2024-09-16T23:48:55.285 INFO:tasks.workunit.client.1.smithi043.stdout:0/502: mkdir d0/d2/df/da7 0 2024-09-16T23:48:55.285 INFO:tasks.workunit.client.1.smithi043.stdout:8/477: getdents de/d18/d1f/d25/d82 0 2024-09-16T23:48:55.286 INFO:tasks.workunit.client.1.smithi043.stdout:8/478: chown de/d18/d27 3389715 1 2024-09-16T23:48:55.288 INFO:tasks.workunit.client.1.smithi043.stdout:5/621: mknod d3/d9/d22/d60/d9d/cca 0 2024-09-16T23:48:55.289 INFO:tasks.workunit.client.1.smithi043.stdout:7/439: mkdir d4/d27/d41/d1b/d95 0 2024-09-16T23:48:55.291 INFO:tasks.workunit.client.1.smithi043.stdout:4/484: mknod d8/d15/d20/d2b/c92 0 2024-09-16T23:48:55.291 INFO:tasks.workunit.client.1.smithi043.stdout:5/622: write d3/db/d19/d3e/f2d [2901834,106585] 0 2024-09-16T23:48:55.295 INFO:tasks.workunit.client.1.smithi043.stdout:4/485: read - d8/d15/d20/d2b/d54/d55/f82 zero size 2024-09-16T23:48:55.295 INFO:tasks.workunit.client.1.smithi043.stdout:8/479: symlink de/d18/d1f/d25/l96 0 2024-09-16T23:48:55.296 INFO:tasks.workunit.client.1.smithi043.stdout:3/463: read d4/de/d40/f45 [2475942,125141] 0 2024-09-16T23:48:55.296 INFO:tasks.workunit.client.1.smithi043.stdout:6/484: truncate de/d23/d49/f7c 3191385 0 2024-09-16T23:48:55.297 INFO:tasks.workunit.client.1.smithi043.stdout:1/579: dwrite d4/d9/d1c/d1e/d42/f81 [0,4194304] 0 2024-09-16T23:48:55.298 INFO:tasks.workunit.client.1.smithi043.stdout:7/440: rmdir d4/d27/d41/d1b/d66 39 2024-09-16T23:48:55.298 INFO:tasks.workunit.client.1.smithi043.stdout:6/485: write de/d23/fa2 [747879,7069] 0 2024-09-16T23:48:55.299 INFO:tasks.workunit.client.1.smithi043.stdout:9/416: rename d4/d1c/c3a to d4/d21/d2a/d2c/d7b/c7d 0 2024-09-16T23:48:55.302 INFO:tasks.workunit.client.1.smithi043.stdout:5/623: mkdir d3/db/d19/d3e/d28/dcb 0 2024-09-16T23:48:55.302 INFO:tasks.workunit.client.1.smithi043.stdout:0/503: dread d0/f3c [0,4194304] 0 2024-09-16T23:48:55.303 INFO:tasks.workunit.client.1.smithi043.stdout:9/417: write d4/d1c/d25/d2b/d4c/d50/f66 [55112,41557] 0 2024-09-16T23:48:55.306 INFO:tasks.workunit.client.1.smithi043.stdout:6/486: dread de/f40 [0,4194304] 0 2024-09-16T23:48:55.307 INFO:tasks.workunit.client.1.smithi043.stdout:4/486: creat d8/d25/d39/d58/d70/d8b/f93 x:0 0 0 2024-09-16T23:48:55.307 INFO:tasks.workunit.client.1.smithi043.stdout:6/487: fdatasync de/f85 0 2024-09-16T23:48:55.308 INFO:tasks.workunit.client.1.smithi043.stdout:6/488: readlink de/d23/d34/d9b/d5a/l8c 0 2024-09-16T23:48:55.309 INFO:tasks.workunit.client.1.smithi043.stdout:3/464: unlink d4/de/d1f/f85 0 2024-09-16T23:48:55.309 INFO:tasks.workunit.client.1.smithi043.stdout:4/487: chown d8/d15/d20/d2b/d22/f24 70 1 2024-09-16T23:48:55.309 INFO:tasks.workunit.client.1.smithi043.stdout:6/489: chown de/d1a/c60 63 1 2024-09-16T23:48:55.310 INFO:tasks.workunit.client.1.smithi043.stdout:4/488: dread - d8/d15/d20/d2b/f5a zero size 2024-09-16T23:48:55.312 INFO:tasks.workunit.client.1.smithi043.stdout:7/441: creat d4/db/d43/d83/d85/f96 x:0 0 0 2024-09-16T23:48:55.313 INFO:tasks.workunit.client.0.smithi042.stdout:9/218: sync 2024-09-16T23:48:55.313 INFO:tasks.workunit.client.0.smithi042.stdout:8/271: sync 2024-09-16T23:48:55.313 INFO:tasks.workunit.client.0.smithi042.stdout:1/307: sync 2024-09-16T23:48:55.313 INFO:tasks.workunit.client.0.smithi042.stdout:4/403: sync 2024-09-16T23:48:55.313 INFO:tasks.workunit.client.0.smithi042.stdout:3/270: sync 2024-09-16T23:48:55.313 INFO:tasks.workunit.client.0.smithi042.stdout:7/352: sync 2024-09-16T23:48:55.314 INFO:tasks.workunit.client.1.smithi043.stdout:5/624: mkdir d3/d79/dcc 0 2024-09-16T23:48:55.315 INFO:tasks.workunit.client.1.smithi043.stdout:5/625: readlink d3/d9/d22/d8e/l9c 0 2024-09-16T23:48:55.316 INFO:tasks.workunit.client.1.smithi043.stdout:0/504: symlink d0/d2/df/d1a/d46/d39/d6e/la8 0 2024-09-16T23:48:55.317 INFO:tasks.workunit.client.0.smithi042.stdout:9/219: unlink d4/d12/l29 0 2024-09-16T23:48:55.318 INFO:tasks.workunit.client.0.smithi042.stdout:7/353: creat d3/d1f/d49/d76/d19/d63/f79 x:0 0 0 2024-09-16T23:48:55.318 INFO:tasks.workunit.client.0.smithi042.stdout:1/308: symlink d0/d17/l6a 0 2024-09-16T23:48:55.319 INFO:tasks.workunit.client.0.smithi042.stdout:1/309: readlink d0/l55 0 2024-09-16T23:48:55.319 INFO:tasks.workunit.client.1.smithi043.stdout:1/580: mkdir d4/d9/d12/d2b/dc9 0 2024-09-16T23:48:55.320 INFO:tasks.workunit.client.0.smithi042.stdout:3/271: rename d2/f4e to d2/d1b/d1d/d34/d50/d21/f5c 0 2024-09-16T23:48:55.320 INFO:tasks.workunit.client.1.smithi043.stdout:1/581: chown d4/d9/d12/d2b/d3d/d83 27 1 2024-09-16T23:48:55.321 INFO:tasks.workunit.client.0.smithi042.stdout:1/310: stat d0/d9/d4e/d59/c5b 0 2024-09-16T23:48:55.321 INFO:tasks.workunit.client.1.smithi043.stdout:3/465: mkdir d4/d30/d53/d88 0 2024-09-16T23:48:55.321 INFO:tasks.workunit.client.0.smithi042.stdout:3/272: fsync d2/d1b/d1d/d34/f3e 0 2024-09-16T23:48:55.323 INFO:tasks.workunit.client.0.smithi042.stdout:4/404: creat de/d2e/d37/f89 x:0 0 0 2024-09-16T23:48:55.323 INFO:tasks.workunit.client.0.smithi042.stdout:1/311: dread - d0/d17/d18/f4d zero size 2024-09-16T23:48:55.325 INFO:tasks.workunit.client.0.smithi042.stdout:3/273: truncate d2/f24 565857 0 2024-09-16T23:48:55.326 INFO:tasks.workunit.client.0.smithi042.stdout:4/405: chown de/d2e/d37/d3c/c49 5242 1 2024-09-16T23:48:55.327 INFO:tasks.workunit.client.1.smithi043.stdout:8/480: rename de/d18/d27/d41/d5a/d37/l7a to de/d18/d1f/d25/d65/d7f/l97 0 2024-09-16T23:48:55.328 INFO:tasks.workunit.client.0.smithi042.stdout:7/354: dwrite d3/d1f/f3d [4194304,4194304] 0 2024-09-16T23:48:55.329 INFO:tasks.workunit.client.0.smithi042.stdout:3/274: write d2/d9/dd/f5b [186560,124595] 0 2024-09-16T23:48:55.330 INFO:tasks.workunit.client.0.smithi042.stdout:4/406: write de/d42/d68/f69 [433059,37263] 0 2024-09-16T23:48:55.335 INFO:tasks.workunit.client.0.smithi042.stdout:3/275: creat d2/d1b/d1d/d34/f5d x:0 0 0 2024-09-16T23:48:55.336 INFO:tasks.workunit.client.1.smithi043.stdout:9/418: truncate d4/f10 3894287 0 2024-09-16T23:48:55.336 INFO:tasks.workunit.client.0.smithi042.stdout:7/355: chown d3/fc 0 1 2024-09-16T23:48:55.338 INFO:tasks.workunit.client.1.smithi043.stdout:9/419: write d4/d1c/d4f/f77 [495225,5408] 0 2024-09-16T23:48:55.338 INFO:tasks.workunit.client.1.smithi043.stdout:7/442: dwrite d4/db/d11/d15/d29/d7a/f86 [4194304,4194304] 0 2024-09-16T23:48:55.339 INFO:tasks.workunit.client.0.smithi042.stdout:1/312: dwrite d0/d9/d35/d46/f30 [0,4194304] 0 2024-09-16T23:48:55.339 INFO:tasks.workunit.client.1.smithi043.stdout:5/626: truncate d3/db/d15/f75 143522 0 2024-09-16T23:48:55.340 INFO:tasks.workunit.client.0.smithi042.stdout:3/276: symlink d2/d1b/d1d/l5e 0 2024-09-16T23:48:55.341 INFO:tasks.workunit.client.1.smithi043.stdout:5/627: dread - d3/f96 zero size 2024-09-16T23:48:55.341 INFO:tasks.workunit.client.0.smithi042.stdout:3/277: chown d2/d1b/c37 28735 1 2024-09-16T23:48:55.344 INFO:tasks.workunit.client.1.smithi043.stdout:1/582: mkdir d4/d3e/d5b/dca 0 2024-09-16T23:48:55.344 INFO:tasks.workunit.client.0.smithi042.stdout:7/356: creat d3/d15/d32/d6b/d5b/f7a x:0 0 0 2024-09-16T23:48:55.345 INFO:tasks.workunit.client.0.smithi042.stdout:8/272: getdents da/d19/d1d/d30 0 2024-09-16T23:48:55.346 INFO:tasks.workunit.client.0.smithi042.stdout:3/278: creat d2/d1b/d1d/d30/f5f x:0 0 0 2024-09-16T23:48:55.347 INFO:tasks.workunit.client.0.smithi042.stdout:3/279: fdatasync d2/d9/dd/f12 0 2024-09-16T23:48:55.349 INFO:tasks.workunit.client.0.smithi042.stdout:7/357: mknod d3/d1f/d49/d76/d19/c7b 0 2024-09-16T23:48:55.350 INFO:tasks.workunit.client.0.smithi042.stdout:7/358: fdatasync d3/fe 0 2024-09-16T23:48:55.352 INFO:tasks.workunit.client.1.smithi043.stdout:4/489: rename d8/d25/d39/f3a to d8/d15/d20/d2b/d54/d55/f94 0 2024-09-16T23:48:55.355 INFO:tasks.workunit.client.0.smithi042.stdout:3/280: rename d2/d1b/d1d/d34/d50/d21 to d2/d1b/d52/d60 0 2024-09-16T23:48:55.355 INFO:tasks.workunit.client.1.smithi043.stdout:7/443: truncate d4/db/f14 450826 0 2024-09-16T23:48:55.356 INFO:tasks.workunit.client.1.smithi043.stdout:0/505: symlink d0/d2/df/d1a/d46/d92/la9 0 2024-09-16T23:48:55.357 INFO:tasks.workunit.client.0.smithi042.stdout:4/407: truncate de/d1a/f39 372344 0 2024-09-16T23:48:55.358 INFO:tasks.workunit.client.1.smithi043.stdout:5/628: symlink d3/db/d15/d4d/d6b/d73/lcd 0 2024-09-16T23:48:55.359 INFO:tasks.workunit.client.0.smithi042.stdout:7/359: dwrite d3/d3f/f5e [0,4194304] 0 2024-09-16T23:48:55.359 INFO:tasks.workunit.client.1.smithi043.stdout:6/490: dwrite de/d23/d34/d9b/f73 [0,4194304] 0 2024-09-16T23:48:55.360 INFO:tasks.workunit.client.1.smithi043.stdout:5/629: write d3/d9/d22/d8e/fbb [3503109,8485] 0 2024-09-16T23:48:55.363 INFO:tasks.workunit.client.1.smithi043.stdout:4/490: creat d8/d15/d20/d2b/d54/d55/f95 x:0 0 0 2024-09-16T23:48:55.364 INFO:tasks.workunit.client.1.smithi043.stdout:8/481: mknod de/d18/d27/c98 0 2024-09-16T23:48:55.364 INFO:tasks.workunit.client.1.smithi043.stdout:4/491: fsync d8/d15/d6e/f5f 0 2024-09-16T23:48:55.366 INFO:tasks.workunit.client.1.smithi043.stdout:0/506: unlink d0/d2/df/d1a/d46/l4c 0 2024-09-16T23:48:55.367 INFO:tasks.workunit.client.1.smithi043.stdout:1/583: dwrite d4/df/d13/fbb [4194304,4194304] 0 2024-09-16T23:48:55.369 INFO:tasks.workunit.client.1.smithi043.stdout:4/492: truncate d8/d25/d39/d58/d70/f86 151007 0 2024-09-16T23:48:55.369 INFO:tasks.workunit.client.0.smithi042.stdout:7/360: creat d3/d1f/d49/d78/f7c x:0 0 0 2024-09-16T23:48:55.371 INFO:tasks.workunit.client.1.smithi043.stdout:5/630: unlink d3/db/d19/d76/lb5 0 2024-09-16T23:48:55.371 INFO:tasks.workunit.client.1.smithi043.stdout:1/584: dread - d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fa1 zero size 2024-09-16T23:48:55.371 INFO:tasks.workunit.client.0.smithi042.stdout:3/281: dwrite d2/d1b/d1d/d34/f3e [0,4194304] 0 2024-09-16T23:48:55.371 INFO:tasks.workunit.client.1.smithi043.stdout:5/631: readlink d3/db/d15/lb4 0 2024-09-16T23:48:55.372 INFO:tasks.workunit.client.1.smithi043.stdout:1/585: chown d4/d9/d12/d2b/d3d/dc8/ca6 927207 1 2024-09-16T23:48:55.373 INFO:tasks.workunit.client.0.smithi042.stdout:9/220: dwrite d4/d10/f15 [0,4194304] 0 2024-09-16T23:48:55.375 INFO:tasks.workunit.client.1.smithi043.stdout:7/444: link d4/db/d11/d15/f3d d4/db/d28/f97 0 2024-09-16T23:48:55.377 INFO:tasks.workunit.client.0.smithi042.stdout:3/282: unlink d2/d1b/f3d 0 2024-09-16T23:48:55.381 INFO:tasks.workunit.client.0.smithi042.stdout:3/283: write d2/d1b/d1d/f44 [2419845,63052] 0 2024-09-16T23:48:55.383 INFO:tasks.workunit.client.0.smithi042.stdout:3/284: fdatasync d2/f41 0 2024-09-16T23:48:55.383 INFO:tasks.workunit.client.0.smithi042.stdout:7/361: dwrite d3/d1f/f26 [0,4194304] 0 2024-09-16T23:48:55.386 INFO:tasks.workunit.client.0.smithi042.stdout:7/362: write d3/d15/f6c [407622,99375] 0 2024-09-16T23:48:55.387 INFO:tasks.workunit.client.0.smithi042.stdout:7/363: readlink l2 0 2024-09-16T23:48:55.391 INFO:tasks.workunit.client.1.smithi043.stdout:3/466: getdents d4/de/d1f/d34 0 2024-09-16T23:48:55.392 INFO:tasks.workunit.client.0.smithi042.stdout:7/364: creat d3/d15/d24/d4d/f7d x:0 0 0 2024-09-16T23:48:55.394 INFO:tasks.workunit.client.1.smithi043.stdout:4/493: symlink d8/d15/d20/d2b/d23/l96 0 2024-09-16T23:48:55.396 INFO:tasks.workunit.client.0.smithi042.stdout:4/408: dread de/d1a/d1c/f21 [0,4194304] 0 2024-09-16T23:48:55.399 INFO:tasks.workunit.client.1.smithi043.stdout:8/482: dwrite de/d18/d1f/d25/f3b [0,4194304] 0 2024-09-16T23:48:55.399 INFO:tasks.workunit.client.0.smithi042.stdout:9/221: getdents d4 0 2024-09-16T23:48:55.400 INFO:tasks.workunit.client.1.smithi043.stdout:8/483: fsync de/d70/d3d/f62 0 2024-09-16T23:48:55.402 INFO:tasks.workunit.client.1.smithi043.stdout:0/507: mkdir d0/d2/df/d1a/d46/d39/daa 0 2024-09-16T23:48:55.405 INFO:tasks.workunit.client.0.smithi042.stdout:9/222: creat d4/d12/d2d/d37/d41/f49 x:0 0 0 2024-09-16T23:48:55.406 INFO:tasks.workunit.client.1.smithi043.stdout:5/632: mkdir d3/dce 0 2024-09-16T23:48:55.407 INFO:tasks.workunit.client.1.smithi043.stdout:5/633: fdatasync d3/d9/d22/f4e 0 2024-09-16T23:48:55.409 INFO:tasks.workunit.client.1.smithi043.stdout:7/445: symlink d4/db/d11/d21/d31/d65/l98 0 2024-09-16T23:48:55.411 INFO:tasks.workunit.client.1.smithi043.stdout:8/484: unlink de/d70/d3d/c79 0 2024-09-16T23:48:55.412 INFO:tasks.workunit.client.1.smithi043.stdout:0/508: mknod d0/de/cab 0 2024-09-16T23:48:55.416 INFO:tasks.workunit.client.1.smithi043.stdout:3/467: link d4/f3b d4/de/d40/d86/f89 0 2024-09-16T23:48:55.416 INFO:tasks.workunit.client.1.smithi043.stdout:5/634: mknod d3/db/ccf 0 2024-09-16T23:48:55.419 INFO:tasks.workunit.client.1.smithi043.stdout:4/494: rename d8/d25/d64/c84 to d8/d15/d20/d2b/d23/d27/c97 0 2024-09-16T23:48:55.419 INFO:tasks.workunit.client.1.smithi043.stdout:7/446: mknod d4/db/d43/d83/c99 0 2024-09-16T23:48:55.420 INFO:tasks.workunit.client.1.smithi043.stdout:8/485: creat de/d18/d27/d41/d5a/d37/f99 x:0 0 0 2024-09-16T23:48:55.422 INFO:tasks.workunit.client.1.smithi043.stdout:7/447: chown d4/db/d11/f13 230775339 1 2024-09-16T23:48:55.424 INFO:tasks.workunit.client.1.smithi043.stdout:4/495: rename d8/d15/d6e/f5f to d8/d15/f98 0 2024-09-16T23:48:55.424 INFO:tasks.workunit.client.1.smithi043.stdout:7/448: read d4/f60 [2521447,67187] 0 2024-09-16T23:48:55.426 INFO:tasks.workunit.client.1.smithi043.stdout:5/635: dwrite d3/db/d19/d3e/f2d [4194304,4194304] 0 2024-09-16T23:48:55.426 INFO:tasks.workunit.client.1.smithi043.stdout:3/468: dread d4/de/d35/f1d [0,4194304] 0 2024-09-16T23:48:55.428 INFO:tasks.workunit.client.1.smithi043.stdout:5/636: chown d3/d9/d6f/d85/cad 336526058 1 2024-09-16T23:48:55.430 INFO:tasks.workunit.client.1.smithi043.stdout:5/637: chown d3/db/d15/d4d/d6b/d73/l7d 145636 1 2024-09-16T23:48:55.433 INFO:tasks.workunit.client.1.smithi043.stdout:7/449: dwrite d4/db/d11/d15/d29/d7a/f86 [0,4194304] 0 2024-09-16T23:48:55.434 INFO:tasks.workunit.client.1.smithi043.stdout:7/450: write d4/db/d43/f79 [776229,22051] 0 2024-09-16T23:48:55.441 INFO:tasks.workunit.client.1.smithi043.stdout:3/469: dread d4/de/d40/f69 [0,4194304] 0 2024-09-16T23:48:55.442 INFO:tasks.workunit.client.1.smithi043.stdout:3/470: stat d4/de/f21 0 2024-09-16T23:48:55.450 INFO:tasks.workunit.client.0.smithi042.stdout:6/290: sync 2024-09-16T23:48:55.452 INFO:tasks.workunit.client.1.smithi043.stdout:5/638: dread d3/d9/d22/d4a/d64/f7b [0,4194304] 0 2024-09-16T23:48:55.455 INFO:tasks.workunit.client.0.smithi042.stdout:9/223: dwrite d4/f19 [4194304,4194304] 0 2024-09-16T23:48:55.455 INFO:tasks.workunit.client.0.smithi042.stdout:1/313: fdatasync d0/d9/d35/d46/f30 0 2024-09-16T23:48:55.457 INFO:tasks.workunit.client.0.smithi042.stdout:4/409: dread de/d12/d1b/f36 [0,4194304] 0 2024-09-16T23:48:55.460 INFO:tasks.workunit.client.1.smithi043.stdout:8/486: mknod de/d18/d1f/d25/c9a 0 2024-09-16T23:48:55.460 INFO:tasks.workunit.client.1.smithi043.stdout:4/496: unlink d8/d15/d20/d2b/d54/d55/l51 0 2024-09-16T23:48:55.461 INFO:tasks.workunit.client.1.smithi043.stdout:4/497: chown d8/d25/d39/d41/d57 0 1 2024-09-16T23:48:55.462 INFO:tasks.workunit.client.1.smithi043.stdout:8/487: write de/d18/d27/d41/d5a/f4a [9029572,11659] 0 2024-09-16T23:48:55.465 INFO:tasks.workunit.client.1.smithi043.stdout:4/498: write d8/d25/d39/d41/f60 [311315,87999] 0 2024-09-16T23:48:55.466 INFO:tasks.workunit.client.0.smithi042.stdout:3/285: rmdir d2/d1b/d1d/d30 39 2024-09-16T23:48:55.468 INFO:tasks.workunit.client.1.smithi043.stdout:7/451: creat d4/d27/d41/d1b/d95/f9a x:0 0 0 2024-09-16T23:48:55.468 INFO:tasks.workunit.client.0.smithi042.stdout:3/286: chown d2/d9/c18 219 1 2024-09-16T23:48:55.469 INFO:tasks.workunit.client.0.smithi042.stdout:6/291: dwrite d8/db/dd/d2c/d26/f32 [0,4194304] 0 2024-09-16T23:48:55.471 INFO:tasks.workunit.client.1.smithi043.stdout:3/471: symlink d4/l8a 0 2024-09-16T23:48:55.472 INFO:tasks.workunit.client.1.smithi043.stdout:3/472: fdatasync d4/f54 0 2024-09-16T23:48:55.474 INFO:tasks.workunit.client.1.smithi043.stdout:3/473: read d4/d30/d53/f5c [3883208,83793] 0 2024-09-16T23:48:55.474 INFO:tasks.workunit.client.0.smithi042.stdout:1/314: dread d0/d9/f11 [0,4194304] 0 2024-09-16T23:48:55.476 INFO:tasks.workunit.client.0.smithi042.stdout:3/287: dread - d2/d1b/d52/d60/f45 zero size 2024-09-16T23:48:55.480 INFO:tasks.workunit.client.0.smithi042.stdout:4/410: dwrite de/d12/d1b/f36 [0,4194304] 0 2024-09-16T23:48:55.480 INFO:tasks.workunit.client.0.smithi042.stdout:3/288: truncate d2/d1b/f43 963390 0 2024-09-16T23:48:55.482 INFO:tasks.workunit.client.0.smithi042.stdout:6/292: rename d8/db/f52 to d8/db/dd/d1b/d34/f64 0 2024-09-16T23:48:55.483 INFO:tasks.workunit.client.1.smithi043.stdout:4/499: fdatasync d8/d25/d39/d58/f4a 0 2024-09-16T23:48:55.483 INFO:tasks.workunit.client.0.smithi042.stdout:1/315: mkdir d0/d9/d35/d6b 0 2024-09-16T23:48:55.485 INFO:tasks.workunit.client.0.smithi042.stdout:1/316: stat d0/d9/d35/d46/f12 0 2024-09-16T23:48:55.485 INFO:tasks.workunit.client.0.smithi042.stdout:4/411: read de/d2e/d37/d4c/d4f/f58 [358813,58731] 0 2024-09-16T23:48:55.486 INFO:tasks.workunit.client.0.smithi042.stdout:3/289: mkdir d2/d7/d61 0 2024-09-16T23:48:55.488 INFO:tasks.workunit.client.0.smithi042.stdout:1/317: write d0/d9/d4e/d59/f65 [896448,38148] 0 2024-09-16T23:48:55.489 INFO:tasks.workunit.client.0.smithi042.stdout:4/412: mkdir de/d12/d1b/d8a 0 2024-09-16T23:48:55.491 INFO:tasks.workunit.client.1.smithi043.stdout:3/474: unlink d4/de/d1f/d34/f3a 0 2024-09-16T23:48:55.491 INFO:tasks.workunit.client.0.smithi042.stdout:3/290: mkdir d2/d1b/d1d/d34/d62 0 2024-09-16T23:48:55.493 INFO:tasks.workunit.client.0.smithi042.stdout:4/413: mknod de/d1a/d1c/c8b 0 2024-09-16T23:48:55.493 INFO:tasks.workunit.client.0.smithi042.stdout:6/293: chown d8/db/dd/d1b/d34/d39/l49 133189680 1 2024-09-16T23:48:55.494 INFO:tasks.workunit.client.1.smithi043.stdout:5/639: link d3/d9/l3f d3/d79/dcc/ld0 0 2024-09-16T23:48:55.494 INFO:tasks.workunit.client.1.smithi043.stdout:1/586: dread d4/d9/d1c/d1e/f66 [0,4194304] 0 2024-09-16T23:48:55.494 INFO:tasks.workunit.client.0.smithi042.stdout:9/224: rmdir d4/d12/d2d/d37 39 2024-09-16T23:48:55.495 INFO:tasks.workunit.client.0.smithi042.stdout:3/291: symlink d2/d9/l63 0 2024-09-16T23:48:55.495 INFO:tasks.workunit.client.0.smithi042.stdout:4/414: stat de/d1a/d1c/l30 0 2024-09-16T23:48:55.496 INFO:tasks.workunit.client.0.smithi042.stdout:4/415: stat fa 0 2024-09-16T23:48:55.498 INFO:tasks.workunit.client.0.smithi042.stdout:4/416: chown de/d1a/f33 44 1 2024-09-16T23:48:55.499 INFO:tasks.workunit.client.0.smithi042.stdout:3/292: unlink d2/d9/dd/c4c 0 2024-09-16T23:48:55.500 INFO:tasks.workunit.client.0.smithi042.stdout:6/294: mknod d8/db/dd/d1b/d2d/c65 0 2024-09-16T23:48:55.501 INFO:tasks.workunit.client.1.smithi043.stdout:5/640: dwrite d3/f50 [4194304,4194304] 0 2024-09-16T23:48:55.501 INFO:tasks.workunit.client.1.smithi043.stdout:7/452: dread d4/d27/d41/d1b/f4e [0,4194304] 0 2024-09-16T23:48:55.502 INFO:tasks.workunit.client.0.smithi042.stdout:3/293: dread d2/d1b/f2a [0,4194304] 0 2024-09-16T23:48:55.503 INFO:tasks.workunit.client.0.smithi042.stdout:6/295: write d8/db/f1d [3460872,129446] 0 2024-09-16T23:48:55.504 INFO:tasks.workunit.client.0.smithi042.stdout:4/417: creat de/d42/d5d/f8c x:0 0 0 2024-09-16T23:48:55.504 INFO:tasks.workunit.client.0.smithi042.stdout:9/225: dwrite d4/d12/d2d/f32 [4194304,4194304] 0 2024-09-16T23:48:55.504 INFO:tasks.workunit.client.1.smithi043.stdout:4/500: symlink d8/d15/d20/d2b/d54/d55/d49/l99 0 2024-09-16T23:48:55.508 INFO:tasks.workunit.client.1.smithi043.stdout:7/453: dread d4/d27/d41/d1b/f44 [0,4194304] 0 2024-09-16T23:48:55.510 INFO:tasks.workunit.client.0.smithi042.stdout:3/294: rename d2/d1b/d1d/f15 to d2/d9/dd/d2c/f64 0 2024-09-16T23:48:55.510 INFO:tasks.workunit.client.0.smithi042.stdout:6/296: mknod d8/db/dd/d2c/c66 0 2024-09-16T23:48:55.512 INFO:tasks.workunit.client.1.smithi043.stdout:3/475: symlink d4/de/d1f/d2c/l8b 0 2024-09-16T23:48:55.512 INFO:tasks.workunit.client.0.smithi042.stdout:3/295: symlink d2/d1b/d52/d60/l65 0 2024-09-16T23:48:55.513 INFO:tasks.workunit.client.0.smithi042.stdout:3/296: chown d2/d9/l63 30 1 2024-09-16T23:48:55.514 INFO:tasks.workunit.client.0.smithi042.stdout:9/226: chown d4/d12/d2d/d37/d28/l42 238486 1 2024-09-16T23:48:55.516 INFO:tasks.workunit.client.0.smithi042.stdout:6/297: unlink d8/db/dd/d1b/d34/f3d 0 2024-09-16T23:48:55.518 INFO:tasks.workunit.client.0.smithi042.stdout:9/227: chown d4/ld 173915384 1 2024-09-16T23:48:55.519 INFO:tasks.workunit.client.1.smithi043.stdout:8/488: link de/d18/l4e de/d18/d1f/d25/d65/d7f/d95/l9b 0 2024-09-16T23:48:55.519 INFO:tasks.workunit.client.0.smithi042.stdout:9/228: read d4/d12/f1e [655262,20878] 0 2024-09-16T23:48:55.520 INFO:tasks.workunit.client.1.smithi043.stdout:8/489: readlink de/d18/d27/d41/l44 0 2024-09-16T23:48:55.521 INFO:tasks.workunit.client.0.smithi042.stdout:3/297: getdents d2/d9/dd 0 2024-09-16T23:48:55.523 INFO:tasks.workunit.client.0.smithi042.stdout:6/298: rmdir d8/db 39 2024-09-16T23:48:55.527 INFO:tasks.workunit.client.0.smithi042.stdout:9/229: rename d4/d12/d2d/d37/f3c to d4/d10/f4a 0 2024-09-16T23:48:55.527 INFO:tasks.workunit.client.0.smithi042.stdout:4/418: dwrite fa [8388608,4194304] 0 2024-09-16T23:48:55.528 INFO:tasks.workunit.client.0.smithi042.stdout:9/230: stat d4/d10/l22 0 2024-09-16T23:48:55.529 INFO:tasks.workunit.client.1.smithi043.stdout:4/501: creat d8/d25/d39/d58/f9a x:0 0 0 2024-09-16T23:48:55.529 INFO:tasks.workunit.client.0.smithi042.stdout:9/231: readlink d4/d12/l1d 0 2024-09-16T23:48:55.530 INFO:tasks.workunit.client.0.smithi042.stdout:3/298: truncate d2/d1b/d52/d60/f2e 977437 0 2024-09-16T23:48:55.532 INFO:tasks.workunit.client.0.smithi042.stdout:4/419: rmdir de/d1a 39 2024-09-16T23:48:55.532 INFO:tasks.workunit.client.1.smithi043.stdout:7/454: mknod d4/db/d11/d15/c9b 0 2024-09-16T23:48:55.533 INFO:tasks.workunit.client.0.smithi042.stdout:3/299: creat d2/d9/f66 x:0 0 0 2024-09-16T23:48:55.537 INFO:tasks.workunit.client.0.smithi042.stdout:9/232: mkdir d4/d10/d4b 0 2024-09-16T23:48:55.541 INFO:tasks.workunit.client.0.smithi042.stdout:3/300: symlink d2/d1b/d1d/d34/d62/l67 0 2024-09-16T23:48:55.541 INFO:tasks.workunit.client.0.smithi042.stdout:6/299: dwrite d8/db/dd/f19 [0,4194304] 0 2024-09-16T23:48:55.544 INFO:tasks.workunit.client.0.smithi042.stdout:3/301: mknod d2/d7/c68 0 2024-09-16T23:48:55.545 INFO:tasks.workunit.client.0.smithi042.stdout:3/302: fsync d2/f41 0 2024-09-16T23:48:55.546 INFO:tasks.workunit.client.1.smithi043.stdout:5/641: creat d3/db/d19/d3e/d28/dcb/fd1 x:0 0 0 2024-09-16T23:48:55.546 INFO:tasks.workunit.client.0.smithi042.stdout:4/420: truncate de/d12/f14 1999491 0 2024-09-16T23:48:55.547 INFO:tasks.workunit.client.0.smithi042.stdout:1/318: write d0/d17/d18/f4d [715883,12254] 0 2024-09-16T23:48:55.548 INFO:tasks.workunit.client.0.smithi042.stdout:1/319: chown d0/d17/l4c 213525884 1 2024-09-16T23:48:55.548 INFO:tasks.workunit.client.0.smithi042.stdout:6/300: mkdir d8/db/dd/d1b/d67 0 2024-09-16T23:48:55.550 INFO:tasks.workunit.client.0.smithi042.stdout:4/421: creat de/d42/d68/f8d x:0 0 0 2024-09-16T23:48:55.553 INFO:tasks.workunit.client.0.smithi042.stdout:4/422: getdents de/d42/d85 0 2024-09-16T23:48:55.554 INFO:tasks.workunit.client.0.smithi042.stdout:4/423: write de/f2f [122778,24466] 0 2024-09-16T23:48:55.554 INFO:tasks.workunit.client.1.smithi043.stdout:4/502: rename d8/d25/d39/d58/d70 to d8/d15/d9b 0 2024-09-16T23:48:55.555 INFO:tasks.workunit.client.0.smithi042.stdout:4/424: readlink de/d42/l6c 0 2024-09-16T23:48:55.556 INFO:tasks.workunit.client.0.smithi042.stdout:3/303: dread d2/d9/f22 [0,4194304] 0 2024-09-16T23:48:55.556 INFO:tasks.workunit.client.0.smithi042.stdout:9/233: symlink d4/d12/d2d/d37/l4c 0 2024-09-16T23:48:55.558 INFO:tasks.workunit.client.0.smithi042.stdout:4/425: creat de/d42/d82/f8e x:0 0 0 2024-09-16T23:48:55.558 INFO:tasks.workunit.client.1.smithi043.stdout:3/476: creat d4/de/d40/d86/f8c x:0 0 0 2024-09-16T23:48:55.560 INFO:tasks.workunit.client.1.smithi043.stdout:7/455: fsync d4/d27/d41/d1b/f4e 0 2024-09-16T23:48:55.561 INFO:tasks.workunit.client.0.smithi042.stdout:4/426: mkdir de/d42/d5d/d8f 0 2024-09-16T23:48:55.562 INFO:tasks.workunit.client.0.smithi042.stdout:6/301: creat d8/db/dd/d1b/d34/d39/d5e/d4c/f68 x:0 0 0 2024-09-16T23:48:55.563 INFO:tasks.workunit.client.1.smithi043.stdout:4/503: dwrite d8/d25/d39/d58/f9a [0,4194304] 0 2024-09-16T23:48:55.564 INFO:tasks.workunit.client.0.smithi042.stdout:9/234: fsync d4/d10/f1b 0 2024-09-16T23:48:55.566 INFO:tasks.workunit.client.1.smithi043.stdout:8/490: fdatasync de/d18/d1f/d2f/f43 0 2024-09-16T23:48:55.568 INFO:tasks.workunit.client.0.smithi042.stdout:4/427: symlink de/d1a/l90 0 2024-09-16T23:48:55.569 INFO:tasks.workunit.client.1.smithi043.stdout:4/504: truncate d8/d15/d20/d2b/f8c 56667 0 2024-09-16T23:48:55.569 INFO:tasks.workunit.client.1.smithi043.stdout:3/477: dwrite d4/f67 [0,4194304] 0 2024-09-16T23:48:55.570 INFO:tasks.workunit.client.1.smithi043.stdout:4/505: stat d8/d15/d20/d2b/f8c 0 2024-09-16T23:48:55.570 INFO:tasks.workunit.client.0.smithi042.stdout:4/428: mknod de/d1a/d1c/c91 0 2024-09-16T23:48:55.571 INFO:tasks.workunit.client.0.smithi042.stdout:6/302: symlink d8/db/dd/d2c/l69 0 2024-09-16T23:48:55.571 INFO:tasks.workunit.client.0.smithi042.stdout:3/304: dread d2/d1b/f3b [0,4194304] 0 2024-09-16T23:48:55.574 INFO:tasks.workunit.client.0.smithi042.stdout:6/303: write d8/f37 [234987,52764] 0 2024-09-16T23:48:55.575 INFO:tasks.workunit.client.0.smithi042.stdout:3/305: write d2/d1b/d52/d60/f45 [311394,125746] 0 2024-09-16T23:48:55.578 INFO:tasks.workunit.client.1.smithi043.stdout:1/587: getdents d4/d3e 0 2024-09-16T23:48:55.581 INFO:tasks.workunit.client.1.smithi043.stdout:3/478: dwrite d4/de/d35/f83 [0,4194304] 0 2024-09-16T23:48:55.581 INFO:tasks.workunit.client.1.smithi043.stdout:1/588: truncate d4/d3e/fc7 606112 0 2024-09-16T23:48:55.582 INFO:tasks.workunit.client.1.smithi043.stdout:4/506: dwrite d8/d25/d64/f7d [4194304,4194304] 0 2024-09-16T23:48:55.583 INFO:tasks.workunit.client.1.smithi043.stdout:3/479: readlink d4/de/d1f/d34/l77 0 2024-09-16T23:48:55.588 INFO:tasks.workunit.client.1.smithi043.stdout:4/507: dread d8/d15/d20/d2b/f28 [4194304,4194304] 0 2024-09-16T23:48:55.588 INFO:tasks.workunit.client.0.smithi042.stdout:1/320: getdents d0/d9/d35/d46 0 2024-09-16T23:48:55.591 INFO:tasks.workunit.client.0.smithi042.stdout:4/429: truncate de/d12/f25 8240508 0 2024-09-16T23:48:55.591 INFO:tasks.workunit.client.0.smithi042.stdout:3/306: write d2/d9/dd/f3f [37183,103204] 0 2024-09-16T23:48:55.595 INFO:tasks.workunit.client.0.smithi042.stdout:3/307: dread - d2/d1b/d1d/f38 zero size 2024-09-16T23:48:55.596 INFO:tasks.workunit.client.0.smithi042.stdout:3/308: mknod d2/d9/c69 0 2024-09-16T23:48:55.597 INFO:tasks.workunit.client.1.smithi043.stdout:7/456: creat d4/db/d11/d21/d94/f9c x:0 0 0 2024-09-16T23:48:55.599 INFO:tasks.workunit.client.0.smithi042.stdout:1/321: rename d0/l45 to d0/d9/d35/d43/l6c 0 2024-09-16T23:48:55.601 INFO:tasks.workunit.client.1.smithi043.stdout:1/589: mknod d4/d9/d8f/db6/dc6/ccb 0 2024-09-16T23:48:55.601 INFO:tasks.workunit.client.1.smithi043.stdout:2/404: sync 2024-09-16T23:48:55.602 INFO:tasks.workunit.client.1.smithi043.stdout:2/405: chown d8/d1f/d6b/f8c 10209317 1 2024-09-16T23:48:55.603 INFO:tasks.workunit.client.0.smithi042.stdout:6/304: truncate d8/db/fc 4163073 0 2024-09-16T23:48:55.604 INFO:tasks.workunit.client.0.smithi042.stdout:9/235: getdents d4/d12/d2d/d37/d3a 0 2024-09-16T23:48:55.605 INFO:tasks.workunit.client.1.smithi043.stdout:3/480: rmdir d4/d30/d53 39 2024-09-16T23:48:55.606 INFO:tasks.workunit.client.1.smithi043.stdout:3/481: chown d4/de/d5d 23394065 1 2024-09-16T23:48:55.608 INFO:tasks.workunit.client.0.smithi042.stdout:1/322: fsync d0/d17/f36 0 2024-09-16T23:48:55.608 INFO:tasks.workunit.client.0.smithi042.stdout:6/305: readlink d8/db/dd/d2c/d57/l46 0 2024-09-16T23:48:55.609 INFO:tasks.workunit.client.0.smithi042.stdout:9/236: write d4/d10/d24/f3b [882582,59229] 0 2024-09-16T23:48:55.609 INFO:tasks.workunit.client.0.smithi042.stdout:3/309: dwrite d2/d1b/d1d/d34/f5d [0,4194304] 0 2024-09-16T23:48:55.610 INFO:tasks.workunit.client.1.smithi043.stdout:5/642: rename d3/d9/d22/c46 to d3/d9/cd2 0 2024-09-16T23:48:55.610 INFO:tasks.workunit.client.1.smithi043.stdout:3/482: dread d4/d30/f41 [0,4194304] 0 2024-09-16T23:48:55.610 INFO:tasks.workunit.client.1.smithi043.stdout:2/406: dwrite d8/d5e/d83/f55 [0,4194304] 0 2024-09-16T23:48:55.611 INFO:tasks.workunit.client.0.smithi042.stdout:4/430: dwrite de/f31 [0,4194304] 0 2024-09-16T23:48:55.612 INFO:tasks.workunit.client.1.smithi043.stdout:4/508: creat d8/d15/d20/d2b/d54/d55/d49/d4c/f9c x:0 0 0 2024-09-16T23:48:55.613 INFO:tasks.workunit.client.0.smithi042.stdout:3/310: fsync d2/d1b/d1d/f40 0 2024-09-16T23:48:55.614 INFO:tasks.workunit.client.0.smithi042.stdout:1/323: fdatasync d0/d9/d35/d46/d32/d2f/f68 0 2024-09-16T23:48:55.615 INFO:tasks.workunit.client.0.smithi042.stdout:4/431: chown de/d42/d5d/d8f 7201 1 2024-09-16T23:48:55.617 INFO:tasks.workunit.client.0.smithi042.stdout:4/432: fdatasync de/d2e/d37/d4c/f5f 0 2024-09-16T23:48:55.621 INFO:tasks.workunit.client.1.smithi043.stdout:1/590: fsync d4/df/f38 0 2024-09-16T23:48:55.622 INFO:tasks.workunit.client.0.smithi042.stdout:4/433: mknod de/d2e/d37/d4c/d4f/c92 0 2024-09-16T23:48:55.624 INFO:tasks.workunit.client.0.smithi042.stdout:4/434: fsync de/d42/d68/f72 0 2024-09-16T23:48:55.625 INFO:tasks.workunit.client.1.smithi043.stdout:4/509: dread d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:48:55.625 INFO:tasks.workunit.client.1.smithi043.stdout:3/483: dwrite d4/f1b [0,4194304] 0 2024-09-16T23:48:55.627 INFO:tasks.workunit.client.0.smithi042.stdout:4/435: symlink de/d2e/d37/d3c/l93 0 2024-09-16T23:48:55.628 INFO:tasks.workunit.client.0.smithi042.stdout:4/436: readlink de/d42/l6c 0 2024-09-16T23:48:55.631 INFO:tasks.workunit.client.0.smithi042.stdout:4/437: mkdir de/d42/d82/d94 0 2024-09-16T23:48:55.635 INFO:tasks.workunit.client.0.smithi042.stdout:4/438: mknod de/d2e/d37/c95 0 2024-09-16T23:48:55.637 INFO:tasks.workunit.client.0.smithi042.stdout:4/439: creat de/d42/d68/f96 x:0 0 0 2024-09-16T23:48:55.637 INFO:tasks.workunit.client.1.smithi043.stdout:7/457: dread d4/f60 [0,4194304] 0 2024-09-16T23:48:55.638 INFO:tasks.workunit.client.1.smithi043.stdout:2/407: dread d8/d1f/f5c [0,4194304] 0 2024-09-16T23:48:55.639 INFO:tasks.workunit.client.1.smithi043.stdout:2/408: read d8/d5e/f87 [3270949,15525] 0 2024-09-16T23:48:55.641 INFO:tasks.workunit.client.0.smithi042.stdout:4/440: creat de/d12/f97 x:0 0 0 2024-09-16T23:48:55.642 INFO:tasks.workunit.client.0.smithi042.stdout:3/311: dread d2/d1b/d1d/f40 [0,4194304] 0 2024-09-16T23:48:55.644 INFO:tasks.workunit.client.1.smithi043.stdout:6/491: sync 2024-09-16T23:48:55.644 INFO:tasks.workunit.client.1.smithi043.stdout:9/420: sync 2024-09-16T23:48:55.644 INFO:tasks.workunit.client.1.smithi043.stdout:0/509: sync 2024-09-16T23:48:55.645 INFO:tasks.workunit.client.0.smithi042.stdout:2/360: sync 2024-09-16T23:48:55.650 INFO:tasks.workunit.client.0.smithi042.stdout:3/312: dwrite d2/f4a [0,4194304] 0 2024-09-16T23:48:55.656 INFO:tasks.workunit.client.0.smithi042.stdout:3/313: dwrite d2/d9/dd/f12 [0,4194304] 0 2024-09-16T23:48:55.694 INFO:tasks.workunit.client.0.smithi042.stdout:6/306: rename d8/l1a to d8/db/dd/d2c/l6a 0 2024-09-16T23:48:55.695 INFO:tasks.workunit.client.0.smithi042.stdout:6/307: write d8/f37 [1222542,26366] 0 2024-09-16T23:48:55.697 INFO:tasks.workunit.client.0.smithi042.stdout:4/441: rename de/d2e/d37/d3e/f61 to de/d12/d1b/f98 0 2024-09-16T23:48:55.697 INFO:tasks.workunit.client.0.smithi042.stdout:2/361: read d0/da/d17/f1c [5633951,27878] 0 2024-09-16T23:48:55.698 INFO:tasks.workunit.client.0.smithi042.stdout:4/442: rmdir de/d1a 39 2024-09-16T23:48:55.699 INFO:tasks.workunit.client.0.smithi042.stdout:4/443: mkdir de/d42/d99 0 2024-09-16T23:48:55.701 INFO:tasks.workunit.client.0.smithi042.stdout:4/444: truncate de/d2e/d37/d4c/d4f/f64 207079 0 2024-09-16T23:48:55.701 INFO:tasks.workunit.client.0.smithi042.stdout:4/445: write de/d12/f15 [3787816,24210] 0 2024-09-16T23:48:55.704 INFO:tasks.workunit.client.0.smithi042.stdout:4/446: creat de/d2e/d37/d3c/f9a x:0 0 0 2024-09-16T23:48:55.705 INFO:tasks.workunit.client.0.smithi042.stdout:4/447: creat de/d2e/d37/d4c/f9b x:0 0 0 2024-09-16T23:48:55.707 INFO:tasks.workunit.client.0.smithi042.stdout:4/448: write de/d1a/d1c/f47 [1376216,65842] 0 2024-09-16T23:48:55.710 INFO:tasks.workunit.client.0.smithi042.stdout:4/449: getdents de 0 2024-09-16T23:48:55.712 INFO:tasks.workunit.client.0.smithi042.stdout:4/450: symlink de/d42/d68/l9c 0 2024-09-16T23:48:55.712 INFO:tasks.workunit.client.0.smithi042.stdout:4/451: stat de/d42/d68 0 2024-09-16T23:48:55.713 INFO:tasks.workunit.client.0.smithi042.stdout:4/452: dread de/d2e/d37/d4c/d4f/f64 [0,4194304] 0 2024-09-16T23:48:55.723 INFO:tasks.workunit.client.0.smithi042.stdout:4/453: dread fa [4194304,4194304] 0 2024-09-16T23:48:55.727 INFO:tasks.workunit.client.0.smithi042.stdout:4/454: write de/d12/d7d/f88 [1386834,106363] 0 2024-09-16T23:48:55.733 INFO:tasks.workunit.client.0.smithi042.stdout:4/455: dwrite de/d12/d1b/f98 [0,4194304] 0 2024-09-16T23:48:55.735 INFO:tasks.workunit.client.0.smithi042.stdout:4/456: creat de/d2e/d37/d4c/f9d x:0 0 0 2024-09-16T23:48:55.737 INFO:tasks.workunit.client.0.smithi042.stdout:4/457: mkdir de/d42/d99/d9e 0 2024-09-16T23:48:55.740 INFO:tasks.workunit.client.0.smithi042.stdout:4/458: mkdir de/d42/d5d/d8f/d9f 0 2024-09-16T23:48:55.741 INFO:tasks.workunit.client.0.smithi042.stdout:4/459: dread - de/d2e/d37/d4c/f9b zero size 2024-09-16T23:48:55.742 INFO:tasks.workunit.client.0.smithi042.stdout:4/460: write de/d42/d68/f72 [253457,64731] 0 2024-09-16T23:48:55.745 INFO:tasks.workunit.client.0.smithi042.stdout:4/461: creat de/d42/d5d/d8f/fa0 x:0 0 0 2024-09-16T23:48:55.748 INFO:tasks.workunit.client.0.smithi042.stdout:4/462: creat de/d12/d7d/fa1 x:0 0 0 2024-09-16T23:48:55.749 INFO:tasks.workunit.client.0.smithi042.stdout:4/463: write de/d2e/d37/d4c/f9b [440897,124555] 0 2024-09-16T23:48:55.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/464: stat lb 0 2024-09-16T23:48:55.751 INFO:tasks.workunit.client.0.smithi042.stdout:4/465: dread - de/d2e/d37/d3c/f9a zero size 2024-09-16T23:48:55.759 INFO:tasks.workunit.client.0.smithi042.stdout:4/466: link de/d12/d7d/f88 de/d1a/d70/fa2 0 2024-09-16T23:48:55.760 INFO:tasks.workunit.client.0.smithi042.stdout:4/467: readlink de/d1a/d1c/l3d 0 2024-09-16T23:48:55.781 INFO:tasks.workunit.client.0.smithi042.stdout:1/324: write d0/d17/f21 [3902560,113768] 0 2024-09-16T23:48:55.782 INFO:tasks.workunit.client.1.smithi043.stdout:5/643: symlink d3/db/ld3 0 2024-09-16T23:48:55.783 INFO:tasks.workunit.client.0.smithi042.stdout:1/325: write d0/d9/d35/d46/f61 [831314,31407] 0 2024-09-16T23:48:55.786 INFO:tasks.workunit.client.0.smithi042.stdout:2/362: rmdir d0/da 39 2024-09-16T23:48:55.786 INFO:tasks.workunit.client.1.smithi043.stdout:1/591: rmdir d4/d34 39 2024-09-16T23:48:55.788 INFO:tasks.workunit.client.1.smithi043.stdout:5/644: dwrite d3/db/d19/d3e/f2d [0,4194304] 0 2024-09-16T23:48:55.790 INFO:tasks.workunit.client.0.smithi042.stdout:1/326: mknod d0/d9/d35/d46/d32/d2f/c6d 0 2024-09-16T23:48:55.791 INFO:tasks.workunit.client.1.smithi043.stdout:4/510: symlink d8/d15/d6e/l9d 0 2024-09-16T23:48:55.793 INFO:tasks.workunit.client.1.smithi043.stdout:3/484: dwrite d4/de/d1f/f79 [0,4194304] 0 2024-09-16T23:48:55.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/308: write d8/db/dd/d2c/f10 [1547802,86704] 0 2024-09-16T23:48:55.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/309: readlink d8/db/lf 0 2024-09-16T23:48:55.796 INFO:tasks.workunit.client.0.smithi042.stdout:1/327: readlink d0/d9/d35/d46/d32/l69 0 2024-09-16T23:48:55.797 INFO:tasks.workunit.client.1.smithi043.stdout:2/409: mknod d8/d4b/d4f/d56/c8d 0 2024-09-16T23:48:55.798 INFO:tasks.workunit.client.1.smithi043.stdout:2/410: fdatasync d8/d1d/d20/f48 0 2024-09-16T23:48:55.799 INFO:tasks.workunit.client.1.smithi043.stdout:8/491: getdents de/d18/d1f/d2f/d39 0 2024-09-16T23:48:55.800 INFO:tasks.workunit.client.1.smithi043.stdout:6/492: chown de/d1a/f32 31926 1 2024-09-16T23:48:55.801 INFO:tasks.workunit.client.0.smithi042.stdout:2/363: dwrite d0/d26/f4e [0,4194304] 0 2024-09-16T23:48:55.808 INFO:tasks.workunit.client.0.smithi042.stdout:1/328: dwrite d0/d9/f11 [0,4194304] 0 2024-09-16T23:48:55.809 INFO:tasks.workunit.client.0.smithi042.stdout:2/364: unlink d0/d30/f70 0 2024-09-16T23:48:55.810 INFO:tasks.workunit.client.0.smithi042.stdout:2/365: fsync d0/d26/f53 0 2024-09-16T23:48:55.812 INFO:tasks.workunit.client.1.smithi043.stdout:7/458: dwrite d4/db/d43/d83/d85/f42 [0,4194304] 0 2024-09-16T23:48:55.821 INFO:tasks.workunit.client.0.smithi042.stdout:2/366: fdatasync d0/da/d17/f4d 0 2024-09-16T23:48:55.824 INFO:tasks.workunit.client.1.smithi043.stdout:1/592: truncate d4/df/d13/f57 3244481 0 2024-09-16T23:48:55.825 INFO:tasks.workunit.client.1.smithi043.stdout:5/645: unlink d3/f38 0 2024-09-16T23:48:55.826 INFO:tasks.workunit.client.1.smithi043.stdout:2/411: dread d8/d5e/d83/f59 [0,4194304] 0 2024-09-16T23:48:55.827 INFO:tasks.workunit.client.1.smithi043.stdout:1/593: truncate d4/d9/d12/d2b/d3d/d83/d98/fb8 441151 0 2024-09-16T23:48:55.828 INFO:tasks.workunit.client.1.smithi043.stdout:4/511: creat d8/d25/d39/d41/f9e x:0 0 0 2024-09-16T23:48:55.828 INFO:tasks.workunit.client.1.smithi043.stdout:1/594: truncate d4/d9/f84 487056 0 2024-09-16T23:48:55.828 INFO:tasks.workunit.client.1.smithi043.stdout:4/512: chown d8/d15/f69 7 1 2024-09-16T23:48:55.830 INFO:tasks.workunit.client.1.smithi043.stdout:3/485: creat d4/de/d35/f8d x:0 0 0 2024-09-16T23:48:55.837 INFO:tasks.workunit.client.1.smithi043.stdout:2/412: rename d8/d1f/d6b to d8/d4b/d5f/d8e 0 2024-09-16T23:48:55.838 INFO:tasks.workunit.client.1.smithi043.stdout:1/595: mkdir d4/d9/d8f/db6/dcc 0 2024-09-16T23:48:55.838 INFO:tasks.workunit.client.0.smithi042.stdout:1/329: truncate d0/d17/f21 3178037 0 2024-09-16T23:48:55.839 INFO:tasks.workunit.client.1.smithi043.stdout:4/513: symlink d8/d15/d20/d2b/d23/d27/l9f 0 2024-09-16T23:48:55.840 INFO:tasks.workunit.client.1.smithi043.stdout:5/646: dwrite d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:55.841 INFO:tasks.workunit.client.0.smithi042.stdout:1/330: write d0/d17/f4f [1884946,8127] 0 2024-09-16T23:48:55.843 INFO:tasks.workunit.client.0.smithi042.stdout:1/331: write d0/d9/d35/d46/f30 [960314,47027] 0 2024-09-16T23:48:55.843 INFO:tasks.workunit.client.1.smithi043.stdout:6/493: readlink de/d1a/d80/l81 0 2024-09-16T23:48:55.844 INFO:tasks.workunit.client.1.smithi043.stdout:4/514: dread d8/d15/f98 [0,4194304] 0 2024-09-16T23:48:55.847 INFO:tasks.workunit.client.0.smithi042.stdout:0/252: sync 2024-09-16T23:48:55.848 INFO:tasks.workunit.client.0.smithi042.stdout:1/332: symlink d0/d9/d4e/d59/l6e 0 2024-09-16T23:48:55.850 INFO:tasks.workunit.client.0.smithi042.stdout:2/367: write d0/da/f2c [5106372,11506] 0 2024-09-16T23:48:55.851 INFO:tasks.workunit.client.1.smithi043.stdout:5/647: mkdir d3/d9/d6f/d85/dd4 0 2024-09-16T23:48:55.853 INFO:tasks.workunit.client.0.smithi042.stdout:0/253: getdents d0/df 0 2024-09-16T23:48:55.853 INFO:tasks.workunit.client.1.smithi043.stdout:4/515: dwrite d8/d15/d20/d2b/d54/d55/f82 [0,4194304] 0 2024-09-16T23:48:55.854 INFO:tasks.workunit.client.1.smithi043.stdout:3/486: rmdir d4/de/d35 39 2024-09-16T23:48:55.854 INFO:tasks.workunit.client.1.smithi043.stdout:4/516: stat d8/f5d 0 2024-09-16T23:48:55.855 INFO:tasks.workunit.client.0.smithi042.stdout:0/254: read d0/d15/f45 [1232395,43108] 0 2024-09-16T23:48:55.859 INFO:tasks.workunit.client.0.smithi042.stdout:2/368: dwrite d0/d26/f53 [0,4194304] 0 2024-09-16T23:48:55.860 INFO:tasks.workunit.client.0.smithi042.stdout:0/255: write d0/d15/f45 [4428221,59838] 0 2024-09-16T23:48:55.861 INFO:tasks.workunit.client.1.smithi043.stdout:6/494: dread de/d1a/f38 [0,4194304] 0 2024-09-16T23:48:55.863 INFO:tasks.workunit.client.1.smithi043.stdout:4/517: dwrite d8/d15/d20/d2b/f8c [0,4194304] 0 2024-09-16T23:48:55.864 INFO:tasks.workunit.client.0.smithi042.stdout:0/256: mkdir d0/df/d13/d49 0 2024-09-16T23:48:55.867 INFO:tasks.workunit.client.0.smithi042.stdout:0/257: dread - d0/df/d13/d2d/f44 zero size 2024-09-16T23:48:55.868 INFO:tasks.workunit.client.1.smithi043.stdout:1/596: unlink d4/df/d26/c7c 0 2024-09-16T23:48:55.870 INFO:tasks.workunit.client.1.smithi043.stdout:1/597: chown d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f2f 5447585 1 2024-09-16T23:48:55.871 INFO:tasks.workunit.client.0.smithi042.stdout:1/333: link d0/d17/c4b d0/d9/d35/d66/c6f 0 2024-09-16T23:48:55.871 INFO:tasks.workunit.client.0.smithi042.stdout:0/258: unlink d0/df/d13/l29 0 2024-09-16T23:48:55.872 INFO:tasks.workunit.client.0.smithi042.stdout:1/334: truncate d0/d17/f58 1034709 0 2024-09-16T23:48:55.878 INFO:tasks.workunit.client.0.smithi042.stdout:1/335: creat d0/d9/d26/f70 x:0 0 0 2024-09-16T23:48:55.880 INFO:tasks.workunit.client.1.smithi043.stdout:3/487: unlink d4/de/f24 0 2024-09-16T23:48:55.882 INFO:tasks.workunit.client.1.smithi043.stdout:8/492: getdents de/d18/d1f/d25 0 2024-09-16T23:48:55.888 INFO:tasks.workunit.client.1.smithi043.stdout:8/493: dwrite de/f90 [0,4194304] 0 2024-09-16T23:48:55.889 INFO:tasks.workunit.client.0.smithi042.stdout:1/336: rename d0/d9/d35/d66/c6f to d0/d9/d35/d46/d32/c71 0 2024-09-16T23:48:55.890 INFO:tasks.workunit.client.1.smithi043.stdout:6/495: dread fd [0,4194304] 0 2024-09-16T23:48:55.891 INFO:tasks.workunit.client.1.smithi043.stdout:9/421: sync 2024-09-16T23:48:55.895 INFO:tasks.workunit.client.0.smithi042.stdout:2/369: dwrite d0/da/d1b/f29 [4194304,4194304] 0 2024-09-16T23:48:55.896 INFO:tasks.workunit.client.0.smithi042.stdout:2/370: chown d0/da/d1b/l25 146967 1 2024-09-16T23:48:55.903 INFO:tasks.workunit.client.1.smithi043.stdout:3/488: rename d4/f54 to d4/de/d40/d86/f8e 0 2024-09-16T23:48:55.908 INFO:tasks.workunit.client.0.smithi042.stdout:1/337: rename d0/d9/d4e/d59 to d0/d72 0 2024-09-16T23:48:55.909 INFO:tasks.workunit.client.1.smithi043.stdout:4/518: write d8/d25/d64/f7f [1770103,36685] 0 2024-09-16T23:48:55.910 INFO:tasks.workunit.client.1.smithi043.stdout:2/413: getdents d8/d1d/d20 0 2024-09-16T23:48:55.911 INFO:tasks.workunit.client.1.smithi043.stdout:8/494: mknod de/d18/d1f/d25/c9c 0 2024-09-16T23:48:55.911 INFO:tasks.workunit.client.0.smithi042.stdout:1/338: read d0/d9/d35/d46/f13 [3773361,88713] 0 2024-09-16T23:48:55.912 INFO:tasks.workunit.client.1.smithi043.stdout:8/495: readlink de/d18/d27/d41/d5a/d37/l66 0 2024-09-16T23:48:55.913 INFO:tasks.workunit.client.1.smithi043.stdout:8/496: write de/d70/d3d/f51 [2497309,34406] 0 2024-09-16T23:48:55.914 INFO:tasks.workunit.client.1.smithi043.stdout:6/496: mkdir de/d23/d34/d9b/d5a/da4 0 2024-09-16T23:48:55.917 INFO:tasks.workunit.client.1.smithi043.stdout:9/422: write d4/d5/d64/f65 [4773070,32625] 0 2024-09-16T23:48:55.918 INFO:tasks.workunit.client.0.smithi042.stdout:2/371: rename d0/d30/d66 to d0/da/d17/d3e/d86 0 2024-09-16T23:48:55.919 INFO:tasks.workunit.client.1.smithi043.stdout:9/423: chown d4/d21/f63 0 1 2024-09-16T23:48:55.920 INFO:tasks.workunit.client.1.smithi043.stdout:1/598: symlink d4/db5/lcd 0 2024-09-16T23:48:55.920 INFO:tasks.workunit.client.1.smithi043.stdout:0/510: sync 2024-09-16T23:48:55.923 INFO:tasks.workunit.client.1.smithi043.stdout:6/497: dwrite de/d23/d34/d66/d70/f76 [0,4194304] 0 2024-09-16T23:48:55.923 INFO:tasks.workunit.client.1.smithi043.stdout:0/511: fsync d0/d2/df/d1a/f30 0 2024-09-16T23:48:55.925 INFO:tasks.workunit.client.1.smithi043.stdout:4/519: creat d8/d25/d64/fa0 x:0 0 0 2024-09-16T23:48:55.928 INFO:tasks.workunit.client.1.smithi043.stdout:8/497: dread de/d18/d27/f94 [0,4194304] 0 2024-09-16T23:48:55.930 INFO:tasks.workunit.client.1.smithi043.stdout:5/648: getdents d3/db/d15/d4d 0 2024-09-16T23:48:55.931 INFO:tasks.workunit.client.1.smithi043.stdout:5/649: read - d3/db/d15/d4d/f6c zero size 2024-09-16T23:48:55.934 INFO:tasks.workunit.client.1.smithi043.stdout:0/512: creat d0/d2/df/d1a/d46/d92/fac x:0 0 0 2024-09-16T23:48:55.934 INFO:tasks.workunit.client.1.smithi043.stdout:3/489: write d4/de/d1f/f29 [1130084,9201] 0 2024-09-16T23:48:55.935 INFO:tasks.workunit.client.1.smithi043.stdout:3/490: fsync d4/f7e 0 2024-09-16T23:48:55.938 INFO:tasks.workunit.client.0.smithi042.stdout:1/339: dwrite d0/d9/f16 [0,4194304] 0 2024-09-16T23:48:55.938 INFO:tasks.workunit.client.1.smithi043.stdout:4/520: creat d8/d25/d39/d41/d57/fa1 x:0 0 0 2024-09-16T23:48:55.939 INFO:tasks.workunit.client.1.smithi043.stdout:4/521: stat d8/d25/d39/d58 0 2024-09-16T23:48:55.939 INFO:tasks.workunit.client.0.smithi042.stdout:2/372: dwrite d0/da/d17/f1c [4194304,4194304] 0 2024-09-16T23:48:55.940 INFO:tasks.workunit.client.1.smithi043.stdout:4/522: truncate d8/d25/d39/d41/f9e 871997 0 2024-09-16T23:48:55.940 INFO:tasks.workunit.client.1.smithi043.stdout:2/414: symlink d8/d5e/d83/l8f 0 2024-09-16T23:48:55.941 INFO:tasks.workunit.client.0.smithi042.stdout:1/340: read d0/d17/f4f [871423,79966] 0 2024-09-16T23:48:55.941 INFO:tasks.workunit.client.1.smithi043.stdout:4/523: dread - d8/d15/d20/d2b/d54/d55/d49/d4c/f9c zero size 2024-09-16T23:48:55.946 INFO:tasks.workunit.client.1.smithi043.stdout:2/415: dwrite d8/d4b/d4f/d7c/f88 [0,4194304] 0 2024-09-16T23:48:55.947 INFO:tasks.workunit.client.1.smithi043.stdout:8/498: mknod de/d18/d1f/d2f/d39/c9d 0 2024-09-16T23:48:55.948 INFO:tasks.workunit.client.1.smithi043.stdout:2/416: read d8/fd [5273201,41558] 0 2024-09-16T23:48:55.950 INFO:tasks.workunit.client.0.smithi042.stdout:2/373: dwrite d0/da/d1b/d3b/d44/d54/f5f [0,4194304] 0 2024-09-16T23:48:55.952 INFO:tasks.workunit.client.1.smithi043.stdout:5/650: chown d3/d79/cab 476 1 2024-09-16T23:48:55.953 INFO:tasks.workunit.client.1.smithi043.stdout:5/651: dread - d3/db/d15/d4d/f6c zero size 2024-09-16T23:48:55.956 INFO:tasks.workunit.client.1.smithi043.stdout:9/424: write d4/d1c/d4f/f5b [1064222,66200] 0 2024-09-16T23:48:55.957 INFO:tasks.workunit.client.0.smithi042.stdout:1/341: dread d0/d17/f4f [0,4194304] 0 2024-09-16T23:48:55.958 INFO:tasks.workunit.client.0.smithi042.stdout:1/342: dread d0/d17/f36 [0,4194304] 0 2024-09-16T23:48:55.960 INFO:tasks.workunit.client.1.smithi043.stdout:8/499: rmdir de/d70/d3d 39 2024-09-16T23:48:55.962 INFO:tasks.workunit.client.1.smithi043.stdout:1/599: creat d4/d9/d12/fce x:0 0 0 2024-09-16T23:48:55.963 INFO:tasks.workunit.client.0.smithi042.stdout:2/374: creat d0/f87 x:0 0 0 2024-09-16T23:48:55.963 INFO:tasks.workunit.client.1.smithi043.stdout:1/600: read d4/d9/d1c/d1e/d47/f53 [437161,23658] 0 2024-09-16T23:48:55.964 INFO:tasks.workunit.client.1.smithi043.stdout:6/498: creat de/d1a/d28/fa5 x:0 0 0 2024-09-16T23:48:55.965 INFO:tasks.workunit.client.0.smithi042.stdout:1/343: creat d0/d17/d18/f73 x:0 0 0 2024-09-16T23:48:55.969 INFO:tasks.workunit.client.0.smithi042.stdout:2/375: mkdir d0/da/d88 0 2024-09-16T23:48:55.970 INFO:tasks.workunit.client.0.smithi042.stdout:2/376: chown d0/f12 12299304 1 2024-09-16T23:48:55.970 INFO:tasks.workunit.client.1.smithi043.stdout:2/417: dread d8/d11/f32 [0,4194304] 0 2024-09-16T23:48:55.970 INFO:tasks.workunit.client.1.smithi043.stdout:5/652: fdatasync d3/db/f2a 0 2024-09-16T23:48:55.974 INFO:tasks.workunit.client.1.smithi043.stdout:9/425: rename d4/d21/d33/f55 to d4/d21/d2a/d2c/d7b/f7e 0 2024-09-16T23:48:55.978 INFO:tasks.workunit.client.0.smithi042.stdout:2/377: creat d0/d30/d43/f89 x:0 0 0 2024-09-16T23:48:55.980 INFO:tasks.workunit.client.1.smithi043.stdout:1/601: chown d4/d34/f6e 3142 1 2024-09-16T23:48:55.981 INFO:tasks.workunit.client.1.smithi043.stdout:8/500: dwrite de/d70/d3d/f51 [0,4194304] 0 2024-09-16T23:48:55.981 INFO:tasks.workunit.client.1.smithi043.stdout:6/499: rmdir de/d23 39 2024-09-16T23:48:55.982 INFO:tasks.workunit.client.1.smithi043.stdout:8/501: chown de/d18/d27/l58 4785483 1 2024-09-16T23:48:55.983 INFO:tasks.workunit.client.1.smithi043.stdout:0/513: creat d0/d2/df/fad x:0 0 0 2024-09-16T23:48:55.985 INFO:tasks.workunit.client.0.smithi042.stdout:2/378: rename d0/da/d1b/d3b/d44/c35 to d0/da/d1b/d48/c8a 0 2024-09-16T23:48:55.992 INFO:tasks.workunit.client.0.smithi042.stdout:2/379: dwrite d0/d30/d43/f63 [4194304,4194304] 0 2024-09-16T23:48:55.992 INFO:tasks.workunit.client.1.smithi043.stdout:2/418: mknod d8/d4b/d4f/c90 0 2024-09-16T23:48:55.993 INFO:tasks.workunit.client.1.smithi043.stdout:5/653: symlink d3/db/d15/d4d/d6b/ld5 0 2024-09-16T23:48:55.994 INFO:tasks.workunit.client.1.smithi043.stdout:7/459: sync 2024-09-16T23:48:55.995 INFO:tasks.workunit.client.1.smithi043.stdout:8/502: dread de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:48:55.995 INFO:tasks.workunit.client.1.smithi043.stdout:9/426: symlink d4/d28/d49/l7f 0 2024-09-16T23:48:56.001 INFO:tasks.workunit.client.1.smithi043.stdout:2/419: dwrite d8/d11/d16/f49 [0,4194304] 0 2024-09-16T23:48:56.006 INFO:tasks.workunit.client.1.smithi043.stdout:3/491: link d4/l14 d4/d30/l8f 0 2024-09-16T23:48:56.006 INFO:tasks.workunit.client.1.smithi043.stdout:4/524: getdents d8/d15/d20/d2b/d23 0 2024-09-16T23:48:56.008 INFO:tasks.workunit.client.1.smithi043.stdout:7/460: rmdir d4/db/d43/d83/d85/d8b 39 2024-09-16T23:48:56.010 INFO:tasks.workunit.client.1.smithi043.stdout:8/503: symlink de/d18/d27/d41/d5a/d37/d6a/l9e 0 2024-09-16T23:48:56.014 INFO:tasks.workunit.client.1.smithi043.stdout:3/492: dwrite d4/de/d40/f82 [0,4194304] 0 2024-09-16T23:48:56.017 INFO:tasks.workunit.client.0.smithi042.stdout:2/380: dwrite d0/da/d17/f37 [4194304,4194304] 0 2024-09-16T23:48:56.017 INFO:tasks.workunit.client.1.smithi043.stdout:9/427: creat d4/d21/d35/d4b/f80 x:0 0 0 2024-09-16T23:48:56.021 INFO:tasks.workunit.client.1.smithi043.stdout:4/525: mknod d8/d15/d20/d2b/d22/ca2 0 2024-09-16T23:48:56.024 INFO:tasks.workunit.client.0.smithi042.stdout:2/381: dwrite d0/da/d17/f59 [0,4194304] 0 2024-09-16T23:48:56.028 INFO:tasks.workunit.client.1.smithi043.stdout:0/514: write d0/f24 [5373754,29094] 0 2024-09-16T23:48:56.028 INFO:tasks.workunit.client.1.smithi043.stdout:7/461: dwrite d4/db/d43/d83/d85/f42 [0,4194304] 0 2024-09-16T23:48:56.028 INFO:tasks.workunit.client.1.smithi043.stdout:1/602: dwrite d4/d34/f6e [0,4194304] 0 2024-09-16T23:48:56.028 INFO:tasks.workunit.client.1.smithi043.stdout:2/420: fdatasync d8/d4b/d5f/f61 0 2024-09-16T23:48:56.029 INFO:tasks.workunit.client.0.smithi042.stdout:2/382: chown d0/d30/f47 439 1 2024-09-16T23:48:56.029 INFO:tasks.workunit.client.1.smithi043.stdout:8/504: symlink de/d18/d2b/l9f 0 2024-09-16T23:48:56.029 INFO:tasks.workunit.client.1.smithi043.stdout:7/462: chown d4/db/d43/c46 9881 1 2024-09-16T23:48:56.031 INFO:tasks.workunit.client.1.smithi043.stdout:2/421: readlink d8/d1d/d20/d23/d38/l50 0 2024-09-16T23:48:56.033 INFO:tasks.workunit.client.0.smithi042.stdout:2/383: mkdir d0/d78/d8b 0 2024-09-16T23:48:56.035 INFO:tasks.workunit.client.1.smithi043.stdout:5/654: rename d3/d9/d22/d60/c65 to d3/cd6 0 2024-09-16T23:48:56.039 INFO:tasks.workunit.client.1.smithi043.stdout:8/505: truncate de/d18/d1f/d25/d65/f8a 545714 0 2024-09-16T23:48:56.040 INFO:tasks.workunit.client.1.smithi043.stdout:9/428: creat d4/d1c/d6b/f81 x:0 0 0 2024-09-16T23:48:56.043 INFO:tasks.workunit.client.1.smithi043.stdout:4/526: mknod d8/d25/d39/d41/d57/ca3 0 2024-09-16T23:48:56.043 INFO:tasks.workunit.client.1.smithi043.stdout:8/506: readlink de/l52 0 2024-09-16T23:48:56.045 INFO:tasks.workunit.client.0.smithi042.stdout:2/384: link d0/da/d1b/f29 d0/d5d/f8c 0 2024-09-16T23:48:56.051 INFO:tasks.workunit.client.1.smithi043.stdout:1/603: dwrite d4/d9/d12/d2b/d3d/d83/d98/fb8 [0,4194304] 0 2024-09-16T23:48:56.051 INFO:tasks.workunit.client.1.smithi043.stdout:3/493: dread d4/de/d1f/d2c/f4a [0,4194304] 0 2024-09-16T23:48:56.053 INFO:tasks.workunit.client.1.smithi043.stdout:2/422: dwrite d8/d11/f46 [0,4194304] 0 2024-09-16T23:48:56.054 INFO:tasks.workunit.client.1.smithi043.stdout:7/463: dwrite d4/d27/d41/d1b/f6f [0,4194304] 0 2024-09-16T23:48:56.055 INFO:tasks.workunit.client.1.smithi043.stdout:5/655: rename d3/db/d26/f33 to d3/d9/d22/d60/d9d/fd7 0 2024-09-16T23:48:56.059 INFO:tasks.workunit.client.1.smithi043.stdout:6/500: getdents de/d23/d34/d9b/d50 0 2024-09-16T23:48:56.059 INFO:tasks.workunit.client.1.smithi043.stdout:8/507: dread de/d70/d3d/f51 [0,4194304] 0 2024-09-16T23:48:56.061 INFO:tasks.workunit.client.1.smithi043.stdout:2/423: rename d8/d4b/d4f/d56 to d8/d4b/d4f/d56/d91 22 2024-09-16T23:48:56.062 INFO:tasks.workunit.client.1.smithi043.stdout:8/508: read - de/d70/d3d/f88 zero size 2024-09-16T23:48:56.063 INFO:tasks.workunit.client.0.smithi042.stdout:2/385: dread d0/f1a [0,4194304] 0 2024-09-16T23:48:56.064 INFO:tasks.workunit.client.1.smithi043.stdout:4/527: creat d8/d15/d20/d2b/d23/d27/d4b/fa4 x:0 0 0 2024-09-16T23:48:56.064 INFO:tasks.workunit.client.1.smithi043.stdout:8/509: chown l2 210316812 1 2024-09-16T23:48:56.064 INFO:tasks.workunit.client.1.smithi043.stdout:1/604: unlink d4/d9/d8f/la2 0 2024-09-16T23:48:56.065 INFO:tasks.workunit.client.1.smithi043.stdout:9/429: dwrite d4/d1c/d4f/f52 [0,4194304] 0 2024-09-16T23:48:56.066 INFO:tasks.workunit.client.1.smithi043.stdout:3/494: creat d4/de/d1f/d2c/f90 x:0 0 0 2024-09-16T23:48:56.067 INFO:tasks.workunit.client.1.smithi043.stdout:9/430: chown d4/d21/d2a/l3f 1436138 1 2024-09-16T23:48:56.068 INFO:tasks.workunit.client.1.smithi043.stdout:9/431: dread - d4/d28/d49/f76 zero size 2024-09-16T23:48:56.069 INFO:tasks.workunit.client.0.smithi042.stdout:2/386: fsync d0/da/d17/f4c 0 2024-09-16T23:48:56.069 INFO:tasks.workunit.client.0.smithi042.stdout:5/393: sync 2024-09-16T23:48:56.069 INFO:tasks.workunit.client.0.smithi042.stdout:8/273: sync 2024-09-16T23:48:56.070 INFO:tasks.workunit.client.1.smithi043.stdout:5/656: read d3/d9/d22/f61 [331752,10539] 0 2024-09-16T23:48:56.070 INFO:tasks.workunit.client.0.smithi042.stdout:8/274: chown da/l15 11331 1 2024-09-16T23:48:56.070 INFO:tasks.workunit.client.0.smithi042.stdout:2/387: write d0/d30/d43/f62 [2650366,128488] 0 2024-09-16T23:48:56.072 INFO:tasks.workunit.client.0.smithi042.stdout:5/394: creat d9/db/de/f7f x:0 0 0 2024-09-16T23:48:56.084 INFO:tasks.workunit.client.0.smithi042.stdout:5/395: mknod d9/db/d1b/d31/c80 0 2024-09-16T23:48:56.086 INFO:tasks.workunit.client.1.smithi043.stdout:4/528: unlink d8/d25/d64/c90 0 2024-09-16T23:48:56.086 INFO:tasks.workunit.client.1.smithi043.stdout:0/515: write d0/d2/df/f17 [1217486,100437] 0 2024-09-16T23:48:56.087 INFO:tasks.workunit.client.0.smithi042.stdout:2/388: dwrite d0/da/d17/f1c [8388608,4194304] 0 2024-09-16T23:48:56.088 INFO:tasks.workunit.client.1.smithi043.stdout:4/529: stat d8/d15/d20/d2b/d22 0 2024-09-16T23:48:56.089 INFO:tasks.workunit.client.0.smithi042.stdout:2/389: dread - d0/d5d/f79 zero size 2024-09-16T23:48:56.090 INFO:tasks.workunit.client.1.smithi043.stdout:1/605: mknod d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/ccf 0 2024-09-16T23:48:56.091 INFO:tasks.workunit.client.0.smithi042.stdout:2/390: write d0/d30/d43/f89 [25873,22999] 0 2024-09-16T23:48:56.094 INFO:tasks.workunit.client.1.smithi043.stdout:3/495: mknod d4/de/d5d/c91 0 2024-09-16T23:48:56.094 INFO:tasks.workunit.client.0.smithi042.stdout:5/396: rename d9/db/de/d10/d4b/d61/d43/c51 to d9/db/de/c81 0 2024-09-16T23:48:56.095 INFO:tasks.workunit.client.0.smithi042.stdout:5/397: chown d9/db/de/d10/f22 1 1 2024-09-16T23:48:56.095 INFO:tasks.workunit.client.0.smithi042.stdout:8/275: write da/d19/f1e [2203931,8723] 0 2024-09-16T23:48:56.096 INFO:tasks.workunit.client.1.smithi043.stdout:9/432: readlink d4/d28/d49/l5a 0 2024-09-16T23:48:56.096 INFO:tasks.workunit.client.1.smithi043.stdout:2/424: dread d8/fd [0,4194304] 0 2024-09-16T23:48:56.097 INFO:tasks.workunit.client.0.smithi042.stdout:5/398: chown d9/db/d1b/d31/f7a 8484537 1 2024-09-16T23:48:56.098 INFO:tasks.workunit.client.1.smithi043.stdout:8/510: truncate de/d18/f4c 1304356 0 2024-09-16T23:48:56.100 INFO:tasks.workunit.client.0.smithi042.stdout:5/399: mknod d9/db/d1b/d31/d6c/c82 0 2024-09-16T23:48:56.102 INFO:tasks.workunit.client.1.smithi043.stdout:6/501: dwrite de/d23/d34/d66/f6e [0,4194304] 0 2024-09-16T23:48:56.103 INFO:tasks.workunit.client.0.smithi042.stdout:8/276: mkdir da/d19/d2c/d56 0 2024-09-16T23:48:56.104 INFO:tasks.workunit.client.0.smithi042.stdout:8/277: stat da/d19/d2c/f2d 0 2024-09-16T23:48:56.105 INFO:tasks.workunit.client.0.smithi042.stdout:2/391: mkdir d0/d8d 0 2024-09-16T23:48:56.105 INFO:tasks.workunit.client.0.smithi042.stdout:8/278: chown da/c3a 60055 1 2024-09-16T23:48:56.107 INFO:tasks.workunit.client.0.smithi042.stdout:8/279: chown da/d19/d1d/d30/l48 1306 1 2024-09-16T23:48:56.109 INFO:tasks.workunit.client.1.smithi043.stdout:2/425: dwrite d8/d11/d16/f49 [0,4194304] 0 2024-09-16T23:48:56.111 INFO:tasks.workunit.client.1.smithi043.stdout:2/426: truncate d8/d1f/f5c 4462240 0 2024-09-16T23:48:56.112 INFO:tasks.workunit.client.0.smithi042.stdout:2/392: write d0/d30/f55 [403276,82702] 0 2024-09-16T23:48:56.113 INFO:tasks.workunit.client.1.smithi043.stdout:1/606: dread d4/d9/d1c/d1e/d42/f81 [0,4194304] 0 2024-09-16T23:48:56.113 INFO:tasks.workunit.client.1.smithi043.stdout:2/427: write d8/f43 [568482,127847] 0 2024-09-16T23:48:56.114 INFO:tasks.workunit.client.0.smithi042.stdout:5/400: link d9/db/d19/l78 d9/db/d1b/d31/d46/d58/l83 0 2024-09-16T23:48:56.117 INFO:tasks.workunit.client.1.smithi043.stdout:0/516: rename d0/f72 to d0/d2/df/d1a/d9a/fae 0 2024-09-16T23:48:56.118 INFO:tasks.workunit.client.0.smithi042.stdout:5/401: symlink d9/d2c/l84 0 2024-09-16T23:48:56.119 INFO:tasks.workunit.client.0.smithi042.stdout:8/280: write da/f18 [800845,6695] 0 2024-09-16T23:48:56.123 INFO:tasks.workunit.client.0.smithi042.stdout:8/281: stat da/d19/d1d/d30/d3f/f53 0 2024-09-16T23:48:56.124 INFO:tasks.workunit.client.0.smithi042.stdout:2/393: mkdir d0/da/d5b/d8e 0 2024-09-16T23:48:56.127 INFO:tasks.workunit.client.0.smithi042.stdout:5/402: dwrite d9/db/d19/d21/d41/d7d/f7e [0,4194304] 0 2024-09-16T23:48:56.127 INFO:tasks.workunit.client.1.smithi043.stdout:9/433: dread d4/d21/d33/f4e [0,4194304] 0 2024-09-16T23:48:56.128 INFO:tasks.workunit.client.1.smithi043.stdout:9/434: stat d4/d5/d16/f1f 0 2024-09-16T23:48:56.129 INFO:tasks.workunit.client.1.smithi043.stdout:3/496: dread d4/d30/d53/f5c [0,4194304] 0 2024-09-16T23:48:56.132 INFO:tasks.workunit.client.1.smithi043.stdout:9/435: dread d4/d1c/d25/d2b/d4c/d50/f66 [0,4194304] 0 2024-09-16T23:48:56.132 INFO:tasks.workunit.client.1.smithi043.stdout:4/530: truncate d8/d25/d39/d41/f9e 88159 0 2024-09-16T23:48:56.134 INFO:tasks.workunit.client.1.smithi043.stdout:2/428: mknod d8/d4b/d5f/c92 0 2024-09-16T23:48:56.135 INFO:tasks.workunit.client.0.smithi042.stdout:5/403: dwrite d9/db/d19/d21/d41/d7d/f7e [0,4194304] 0 2024-09-16T23:48:56.135 INFO:tasks.workunit.client.1.smithi043.stdout:2/429: read - d8/d1d/f5a zero size 2024-09-16T23:48:56.137 INFO:tasks.workunit.client.1.smithi043.stdout:0/517: readlink d0/l3a 0 2024-09-16T23:48:56.138 INFO:tasks.workunit.client.1.smithi043.stdout:4/531: dread d8/d25/d64/f7d [4194304,4194304] 0 2024-09-16T23:48:56.139 INFO:tasks.workunit.client.1.smithi043.stdout:0/518: write d0/f24 [3863358,96589] 0 2024-09-16T23:48:56.139 INFO:tasks.workunit.client.0.smithi042.stdout:2/394: symlink d0/da/d88/l8f 0 2024-09-16T23:48:56.141 INFO:tasks.workunit.client.0.smithi042.stdout:5/404: mkdir d9/db/d19/d21/d85 0 2024-09-16T23:48:56.141 INFO:tasks.workunit.client.1.smithi043.stdout:3/497: rename d4/de/d40/l68 to d4/de/d1f/d34/l92 0 2024-09-16T23:48:56.142 INFO:tasks.workunit.client.0.smithi042.stdout:2/395: write d0/d30/d43/f63 [8849640,53807] 0 2024-09-16T23:48:56.142 INFO:tasks.workunit.client.1.smithi043.stdout:9/436: symlink d4/d21/d2a/l82 0 2024-09-16T23:48:56.145 INFO:tasks.workunit.client.1.smithi043.stdout:1/607: symlink d4/d9/d8f/db6/dcc/ld0 0 2024-09-16T23:48:56.146 INFO:tasks.workunit.client.0.smithi042.stdout:7/365: sync 2024-09-16T23:48:56.146 INFO:tasks.workunit.client.0.smithi042.stdout:9/237: sync 2024-09-16T23:48:56.147 INFO:tasks.workunit.client.1.smithi043.stdout:2/430: creat d8/d4b/d4f/f93 x:0 0 0 2024-09-16T23:48:56.148 INFO:tasks.workunit.client.0.smithi042.stdout:5/405: unlink d9/db/de/d10/d4b/l75 0 2024-09-16T23:48:56.148 INFO:tasks.workunit.client.0.smithi042.stdout:7/366: dread - d3/f5f zero size 2024-09-16T23:48:56.148 INFO:tasks.workunit.client.0.smithi042.stdout:9/238: chown d4/d12/d2d 226 1 2024-09-16T23:48:56.149 INFO:tasks.workunit.client.0.smithi042.stdout:8/282: dwrite da/d19/d1d/f33 [0,4194304] 0 2024-09-16T23:48:56.150 INFO:tasks.workunit.client.1.smithi043.stdout:0/519: dwrite d0/fa3 [0,4194304] 0 2024-09-16T23:48:56.152 INFO:tasks.workunit.client.1.smithi043.stdout:0/520: chown d0/d2/df/d5a/d9c/l86 1797551 1 2024-09-16T23:48:56.152 INFO:tasks.workunit.client.0.smithi042.stdout:2/396: creat d0/da/d1b/d3b/d44/f90 x:0 0 0 2024-09-16T23:48:56.152 INFO:tasks.workunit.client.1.smithi043.stdout:4/532: creat d8/d25/d39/d58/fa5 x:0 0 0 2024-09-16T23:48:56.157 INFO:tasks.workunit.client.1.smithi043.stdout:3/498: dread d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:48:56.159 INFO:tasks.workunit.client.0.smithi042.stdout:7/367: dwrite d3/f1a [0,4194304] 0 2024-09-16T23:48:56.160 INFO:tasks.workunit.client.0.smithi042.stdout:8/283: symlink da/d4b/l57 0 2024-09-16T23:48:56.163 INFO:tasks.workunit.client.0.smithi042.stdout:9/239: mknod d4/c4d 0 2024-09-16T23:48:56.164 INFO:tasks.workunit.client.1.smithi043.stdout:9/437: dwrite d4/d21/d2a/d2c/d7b/f7e [0,4194304] 0 2024-09-16T23:48:56.165 INFO:tasks.workunit.client.0.smithi042.stdout:2/397: creat d0/da/d17/d3e/f91 x:0 0 0 2024-09-16T23:48:56.167 INFO:tasks.workunit.client.0.smithi042.stdout:7/368: rmdir d3 39 2024-09-16T23:48:56.168 INFO:tasks.workunit.client.1.smithi043.stdout:8/511: getdents de/d18/d27/d41 0 2024-09-16T23:48:56.170 INFO:tasks.workunit.client.1.smithi043.stdout:8/512: write de/d18/d1f/f32 [4961297,76604] 0 2024-09-16T23:48:56.171 INFO:tasks.workunit.client.1.smithi043.stdout:2/431: mkdir d8/d1d/d25/d94 0 2024-09-16T23:48:56.174 INFO:tasks.workunit.client.1.smithi043.stdout:2/432: write d8/d4b/d5f/d63/f76 [291562,127346] 0 2024-09-16T23:48:56.177 INFO:tasks.workunit.client.1.smithi043.stdout:9/438: dwrite d4/d28/d49/f76 [0,4194304] 0 2024-09-16T23:48:56.178 INFO:tasks.workunit.client.0.smithi042.stdout:7/369: mknod d3/d1f/d49/c7e 0 2024-09-16T23:48:56.178 INFO:tasks.workunit.client.1.smithi043.stdout:9/439: write d4/d21/f4d [713834,48863] 0 2024-09-16T23:48:56.179 INFO:tasks.workunit.client.0.smithi042.stdout:9/240: dwrite d4/f19 [8388608,4194304] 0 2024-09-16T23:48:56.179 INFO:tasks.workunit.client.1.smithi043.stdout:0/521: mknod d0/d2/dd/caf 0 2024-09-16T23:48:56.180 INFO:tasks.workunit.client.1.smithi043.stdout:0/522: chown d0/d2/df/d1a/d49/d71/f7e 1038 1 2024-09-16T23:48:56.181 INFO:tasks.workunit.client.1.smithi043.stdout:0/523: write d0/f24 [5532344,14423] 0 2024-09-16T23:48:56.182 INFO:tasks.workunit.client.0.smithi042.stdout:8/284: dread da/d19/d1d/d30/f41 [0,4194304] 0 2024-09-16T23:48:56.184 INFO:tasks.workunit.client.1.smithi043.stdout:0/524: dread d0/d2/df/d1a/d3b/f5d [0,4194304] 0 2024-09-16T23:48:56.184 INFO:tasks.workunit.client.0.smithi042.stdout:7/370: creat d3/d15/d41/f7f x:0 0 0 2024-09-16T23:48:56.186 INFO:tasks.workunit.client.1.smithi043.stdout:4/533: rename d8/d15/d20/d2b/d54/c89 to d8/d25/d39/ca6 0 2024-09-16T23:48:56.191 INFO:tasks.workunit.client.0.smithi042.stdout:8/285: dread da/d19/f1f [0,4194304] 0 2024-09-16T23:48:56.195 INFO:tasks.workunit.client.1.smithi043.stdout:1/608: fdatasync d4/df/d13/f57 0 2024-09-16T23:48:56.196 INFO:tasks.workunit.client.0.smithi042.stdout:7/371: dwrite d3/d1f/d49/d76/d19/d63/f75 [0,4194304] 0 2024-09-16T23:48:56.199 INFO:tasks.workunit.client.0.smithi042.stdout:9/241: dread d4/d10/d24/f3b [0,4194304] 0 2024-09-16T23:48:56.200 INFO:tasks.workunit.client.0.smithi042.stdout:7/372: creat d3/d15/d24/d4d/f80 x:0 0 0 2024-09-16T23:48:56.203 INFO:tasks.workunit.client.1.smithi043.stdout:2/433: creat d8/d4b/d4f/d56/f95 x:0 0 0 2024-09-16T23:48:56.203 INFO:tasks.workunit.client.0.smithi042.stdout:7/373: dread d3/d15/d41/f62 [0,4194304] 0 2024-09-16T23:48:56.203 INFO:tasks.workunit.client.0.smithi042.stdout:2/398: truncate d0/da/d1b/d3b/d44/d54/f5f 3300229 0 2024-09-16T23:48:56.204 INFO:tasks.workunit.client.0.smithi042.stdout:7/374: chown d3/d3f 965926049 1 2024-09-16T23:48:56.204 INFO:tasks.workunit.client.1.smithi043.stdout:2/434: chown d8/d4b/d4f/d7c/f88 4688 1 2024-09-16T23:48:56.212 INFO:tasks.workunit.client.0.smithi042.stdout:7/375: dwrite d3/d15/d41/f45 [0,4194304] 0 2024-09-16T23:48:56.213 INFO:tasks.workunit.client.0.smithi042.stdout:9/242: write d4/d12/d2d/d37/d28/f31 [7497,90751] 0 2024-09-16T23:48:56.215 INFO:tasks.workunit.client.0.smithi042.stdout:8/286: mkdir da/d19/d1d/d30/d49/d58 0 2024-09-16T23:48:56.216 INFO:tasks.workunit.client.0.smithi042.stdout:8/287: chown da/d19/f23 393 1 2024-09-16T23:48:56.217 INFO:tasks.workunit.client.0.smithi042.stdout:7/376: truncate d3/d1f/d49/d76/d19/d63/f69 139268 0 2024-09-16T23:48:56.218 INFO:tasks.workunit.client.1.smithi043.stdout:4/534: creat d8/d15/d9b/fa7 x:0 0 0 2024-09-16T23:48:56.220 INFO:tasks.workunit.client.0.smithi042.stdout:7/377: mknod d3/d15/d32/c81 0 2024-09-16T23:48:56.220 INFO:tasks.workunit.client.1.smithi043.stdout:3/499: creat d4/d30/d53/d88/f93 x:0 0 0 2024-09-16T23:48:56.222 INFO:tasks.workunit.client.0.smithi042.stdout:7/378: mknod d3/d15/d32/d6b/d5b/c82 0 2024-09-16T23:48:56.223 INFO:tasks.workunit.client.0.smithi042.stdout:9/243: creat d4/d12/d2d/d37/d28/f4e x:0 0 0 2024-09-16T23:48:56.223 INFO:tasks.workunit.client.1.smithi043.stdout:9/440: dwrite d4/f11 [0,4194304] 0 2024-09-16T23:48:56.224 INFO:tasks.workunit.client.0.smithi042.stdout:9/244: chown d4/d10/d4b 21908 1 2024-09-16T23:48:56.225 INFO:tasks.workunit.client.1.smithi043.stdout:1/609: creat d4/df/fd1 x:0 0 0 2024-09-16T23:48:56.226 INFO:tasks.workunit.client.0.smithi042.stdout:9/245: chown d4/d10/d24/l2c 4 1 2024-09-16T23:48:56.228 INFO:tasks.workunit.client.1.smithi043.stdout:2/435: mknod d8/d4b/d5f/d8e/c96 0 2024-09-16T23:48:56.230 INFO:tasks.workunit.client.0.smithi042.stdout:2/399: creat d0/f92 x:0 0 0 2024-09-16T23:48:56.230 INFO:tasks.workunit.client.0.smithi042.stdout:7/379: dread d3/f30 [0,4194304] 0 2024-09-16T23:48:56.236 INFO:tasks.workunit.client.1.smithi043.stdout:9/441: mknod d4/d1c/d25/d2b/c83 0 2024-09-16T23:48:56.236 INFO:tasks.workunit.client.0.smithi042.stdout:9/246: dwrite d4/d10/f15 [0,4194304] 0 2024-09-16T23:48:56.238 INFO:tasks.workunit.client.1.smithi043.stdout:1/610: creat d4/d34/fd2 x:0 0 0 2024-09-16T23:48:56.239 INFO:tasks.workunit.client.1.smithi043.stdout:8/513: creat de/d70/fa0 x:0 0 0 2024-09-16T23:48:56.241 INFO:tasks.workunit.client.0.smithi042.stdout:2/400: rename d0/c3c to d0/d30/d43/d77/c93 0 2024-09-16T23:48:56.243 INFO:tasks.workunit.client.1.smithi043.stdout:8/514: write de/d18/d1f/f32 [5366219,70999] 0 2024-09-16T23:48:56.244 INFO:tasks.workunit.client.0.smithi042.stdout:7/380: truncate d3/d1f/d49/d76/f25 587332 0 2024-09-16T23:48:56.245 INFO:tasks.workunit.client.1.smithi043.stdout:2/436: rename d8/d1f/f5d to d8/d1d/d20/d26/f97 0 2024-09-16T23:48:56.245 INFO:tasks.workunit.client.1.smithi043.stdout:9/442: dwrite d4/d28/f61 [0,4194304] 0 2024-09-16T23:48:56.247 INFO:tasks.workunit.client.1.smithi043.stdout:4/535: link d8/d15/d20/d2b/d23/f38 d8/d15/d9b/d8b/fa8 0 2024-09-16T23:48:56.253 INFO:tasks.workunit.client.1.smithi043.stdout:8/515: dwrite de/f17 [0,4194304] 0 2024-09-16T23:48:56.253 INFO:tasks.workunit.client.0.smithi042.stdout:7/381: dwrite d3/d1f/d4f/f5d [0,4194304] 0 2024-09-16T23:48:56.253 INFO:tasks.workunit.client.0.smithi042.stdout:2/401: chown d0/da/f32 63536855 1 2024-09-16T23:48:56.254 INFO:tasks.workunit.client.1.smithi043.stdout:8/516: write de/d18/d1f/d25/f3b [1471527,59604] 0 2024-09-16T23:48:56.255 INFO:tasks.workunit.client.0.smithi042.stdout:7/382: truncate d3/d1f/d4f/f5d 5226694 0 2024-09-16T23:48:56.256 INFO:tasks.workunit.client.0.smithi042.stdout:9/247: rmdir d4/d12 39 2024-09-16T23:48:56.257 INFO:tasks.workunit.client.0.smithi042.stdout:7/383: chown d3/d15/d41 3172 1 2024-09-16T23:48:56.257 INFO:tasks.workunit.client.1.smithi043.stdout:1/611: mknod d4/d9/d12/d2b/d3d/dc8/cd3 0 2024-09-16T23:48:56.260 INFO:tasks.workunit.client.1.smithi043.stdout:8/517: dwrite de/d18/d27/d41/d5a/f1b [0,4194304] 0 2024-09-16T23:48:56.261 INFO:tasks.workunit.client.1.smithi043.stdout:2/437: rmdir d8/d4b/d4f 39 2024-09-16T23:48:56.265 INFO:tasks.workunit.client.1.smithi043.stdout:4/536: rename f7 to d8/d15/d6e/fa9 0 2024-09-16T23:48:56.267 INFO:tasks.workunit.client.0.smithi042.stdout:8/288: dwrite da/d19/f3b [4194304,4194304] 0 2024-09-16T23:48:56.267 INFO:tasks.workunit.client.1.smithi043.stdout:1/612: chown d4/f62 15 1 2024-09-16T23:48:56.269 INFO:tasks.workunit.client.1.smithi043.stdout:9/443: mkdir d4/d84 0 2024-09-16T23:48:56.271 INFO:tasks.workunit.client.1.smithi043.stdout:9/444: read d4/d1c/d4f/f52 [2561111,112963] 0 2024-09-16T23:48:56.273 INFO:tasks.workunit.client.0.smithi042.stdout:7/384: creat d3/d15/d32/d6b/f83 x:0 0 0 2024-09-16T23:48:56.275 INFO:tasks.workunit.client.0.smithi042.stdout:2/402: dwrite d0/da/d17/f59 [0,4194304] 0 2024-09-16T23:48:56.279 INFO:tasks.workunit.client.1.smithi043.stdout:1/613: symlink d4/df/d13/ld4 0 2024-09-16T23:48:56.283 INFO:tasks.workunit.client.0.smithi042.stdout:2/403: rename d0/da/d1b/d3b/c3f to d0/da/d17/d3e/c94 0 2024-09-16T23:48:56.286 INFO:tasks.workunit.client.1.smithi043.stdout:2/438: rmdir d8/d4b/d4f/d56 39 2024-09-16T23:48:56.287 INFO:tasks.workunit.client.0.smithi042.stdout:7/385: link d3/d15/d24/f59 d3/f84 0 2024-09-16T23:48:56.288 INFO:tasks.workunit.client.1.smithi043.stdout:1/614: mkdir d4/df/d26/dd5 0 2024-09-16T23:48:56.290 INFO:tasks.workunit.client.0.smithi042.stdout:7/386: symlink d3/d1f/d4f/l85 0 2024-09-16T23:48:56.290 INFO:tasks.workunit.client.1.smithi043.stdout:4/537: dread d8/d15/d20/d2b/f10 [0,4194304] 0 2024-09-16T23:48:56.291 INFO:tasks.workunit.client.0.smithi042.stdout:2/404: creat d0/f95 x:0 0 0 2024-09-16T23:48:56.293 INFO:tasks.workunit.client.1.smithi043.stdout:1/615: symlink d4/d9/d1c/d1e/d42/d5d/ld6 0 2024-09-16T23:48:56.295 INFO:tasks.workunit.client.1.smithi043.stdout:1/616: fdatasync d4/fc 0 2024-09-16T23:48:56.298 INFO:tasks.workunit.client.0.smithi042.stdout:7/387: link d3/c67 d3/d15/d32/d6b/c86 0 2024-09-16T23:48:56.299 INFO:tasks.workunit.client.1.smithi043.stdout:4/538: dwrite d8/d25/f42 [4194304,4194304] 0 2024-09-16T23:48:56.300 INFO:tasks.workunit.client.0.smithi042.stdout:9/248: dread d4/f5 [8388608,4194304] 0 2024-09-16T23:48:56.303 INFO:tasks.workunit.client.1.smithi043.stdout:1/617: link d4/df/d13/fa4 d4/df/d26/d79/fd7 0 2024-09-16T23:48:56.303 INFO:tasks.workunit.client.1.smithi043.stdout:2/439: dread d8/d11/d16/f19 [0,4194304] 0 2024-09-16T23:48:56.304 INFO:tasks.workunit.client.1.smithi043.stdout:1/618: write d4/d34/fc0 [690461,85524] 0 2024-09-16T23:48:56.305 INFO:tasks.workunit.client.0.smithi042.stdout:9/249: chown d4/d12/f39 1742 1 2024-09-16T23:48:56.306 INFO:tasks.workunit.client.1.smithi043.stdout:1/619: write d4/d9/d1c/d1e/d42/d5d/d71/fb1 [926613,121641] 0 2024-09-16T23:48:56.306 INFO:tasks.workunit.client.1.smithi043.stdout:2/440: symlink d8/d1f/l98 0 2024-09-16T23:48:56.313 INFO:tasks.workunit.client.1.smithi043.stdout:1/620: creat d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fd8 x:0 0 0 2024-09-16T23:48:56.315 INFO:tasks.workunit.client.0.smithi042.stdout:9/250: dwrite d4/fb [4194304,4194304] 0 2024-09-16T23:48:56.315 INFO:tasks.workunit.client.1.smithi043.stdout:1/621: truncate d4/d9/f17 124056 0 2024-09-16T23:48:56.316 INFO:tasks.workunit.client.0.smithi042.stdout:9/251: dread - d4/d12/d2d/d37/d28/f4e zero size 2024-09-16T23:48:56.317 INFO:tasks.workunit.client.1.smithi043.stdout:1/622: fdatasync d4/d9/d8f/db6/fbc 0 2024-09-16T23:48:56.318 INFO:tasks.workunit.client.0.smithi042.stdout:9/252: truncate d4/d12/d2d/d37/d3a/f3d 903027 0 2024-09-16T23:48:56.319 INFO:tasks.workunit.client.1.smithi043.stdout:1/623: unlink d4/d9/d8f/db6/fbc 0 2024-09-16T23:48:56.319 INFO:tasks.workunit.client.1.smithi043.stdout:2/441: dread d8/d1d/d20/d23/f6a [0,4194304] 0 2024-09-16T23:48:56.320 INFO:tasks.workunit.client.1.smithi043.stdout:1/624: write d4/d9/d1c/d1e/f23 [2674329,127969] 0 2024-09-16T23:48:56.321 INFO:tasks.workunit.client.0.smithi042.stdout:9/253: symlink d4/d12/d2d/l4f 0 2024-09-16T23:48:56.322 INFO:tasks.workunit.client.1.smithi043.stdout:1/625: fsync d4/d9/d1c/d1e/d42/d5d/f9a 0 2024-09-16T23:48:56.323 INFO:tasks.workunit.client.0.smithi042.stdout:9/254: mkdir d4/d12/d2d/d37/d3a/d50 0 2024-09-16T23:48:56.324 INFO:tasks.workunit.client.1.smithi043.stdout:2/442: dread d8/d11/d16/f49 [0,4194304] 0 2024-09-16T23:48:56.325 INFO:tasks.workunit.client.0.smithi042.stdout:6/310: sync 2024-09-16T23:48:56.327 INFO:tasks.workunit.client.0.smithi042.stdout:6/311: chown d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/d4f 37 1 2024-09-16T23:48:56.327 INFO:tasks.workunit.client.0.smithi042.stdout:6/312: chown d8/c61 434 1 2024-09-16T23:48:56.327 INFO:tasks.workunit.client.1.smithi043.stdout:1/626: dread d4/d9/d1c/d1e/d47/f4f [0,4194304] 0 2024-09-16T23:48:56.328 INFO:tasks.workunit.client.0.smithi042.stdout:6/313: fdatasync d8/db/f23 0 2024-09-16T23:48:56.330 INFO:tasks.workunit.client.1.smithi043.stdout:1/627: mkdir d4/d9/d12/d2b/d3d/dd9 0 2024-09-16T23:48:56.340 INFO:tasks.workunit.client.1.smithi043.stdout:4/539: dread d8/f5d [0,4194304] 0 2024-09-16T23:48:56.341 INFO:tasks.workunit.client.1.smithi043.stdout:4/540: chown d8/d15/d20/d2b/d54/f91 14638496 1 2024-09-16T23:48:56.343 INFO:tasks.workunit.client.0.smithi042.stdout:6/314: dread d8/db/dd/f22 [0,4194304] 0 2024-09-16T23:48:56.344 INFO:tasks.workunit.client.0.smithi042.stdout:6/315: readlink l4 0 2024-09-16T23:48:56.346 INFO:tasks.workunit.client.0.smithi042.stdout:7/388: dread d3/d1f/d49/f56 [0,4194304] 0 2024-09-16T23:48:56.348 INFO:tasks.workunit.client.0.smithi042.stdout:7/389: mknod d3/d15/c87 0 2024-09-16T23:48:56.349 INFO:tasks.workunit.client.0.smithi042.stdout:6/316: creat d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/f6b x:0 0 0 2024-09-16T23:48:56.351 INFO:tasks.workunit.client.1.smithi043.stdout:4/541: fsync d8/d15/d9b/d8b/fa8 0 2024-09-16T23:48:56.351 INFO:tasks.workunit.client.0.smithi042.stdout:7/390: symlink d3/d1f/d4f/l88 0 2024-09-16T23:48:56.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/317: dread - d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/f6b zero size 2024-09-16T23:48:56.351 INFO:tasks.workunit.client.1.smithi043.stdout:4/542: readlink d8/d25/d39/l63 0 2024-09-16T23:48:56.352 INFO:tasks.workunit.client.1.smithi043.stdout:4/543: fdatasync d8/d25/d64/fa0 0 2024-09-16T23:48:56.352 INFO:tasks.workunit.client.0.smithi042.stdout:7/391: truncate d3/d1f/d49/f56 654980 0 2024-09-16T23:48:56.357 INFO:tasks.workunit.client.0.smithi042.stdout:7/392: dread d3/d1f/f3d [4194304,4194304] 0 2024-09-16T23:48:56.358 INFO:tasks.workunit.client.0.smithi042.stdout:7/393: read d3/d1f/d4f/f5d [1060816,100516] 0 2024-09-16T23:48:56.361 INFO:tasks.workunit.client.0.smithi042.stdout:7/394: creat d3/d15/d24/d4d/f89 x:0 0 0 2024-09-16T23:48:56.362 INFO:tasks.workunit.client.0.smithi042.stdout:8/289: write da/d19/f39 [738781,23116] 0 2024-09-16T23:48:56.365 INFO:tasks.workunit.client.1.smithi043.stdout:9/445: write d4/d1c/d25/d2b/d4c/d50/f66 [103723,100684] 0 2024-09-16T23:48:56.365 INFO:tasks.workunit.client.0.smithi042.stdout:7/395: mkdir d3/d1f/d49/d78/d8a 0 2024-09-16T23:48:56.365 INFO:tasks.workunit.client.1.smithi043.stdout:8/518: dwrite de/d18/f36 [4194304,4194304] 0 2024-09-16T23:48:56.366 INFO:tasks.workunit.client.0.smithi042.stdout:7/396: chown d3/d1f/d49/d76/f25 23892988 1 2024-09-16T23:48:56.367 INFO:tasks.workunit.client.1.smithi043.stdout:9/446: truncate d4/d1c/d6b/f81 362719 0 2024-09-16T23:48:56.368 INFO:tasks.workunit.client.1.smithi043.stdout:7/464: sync 2024-09-16T23:48:56.370 INFO:tasks.workunit.client.1.smithi043.stdout:9/447: mkdir d4/d1c/d4f/d45/d85 0 2024-09-16T23:48:56.371 INFO:tasks.workunit.client.1.smithi043.stdout:9/448: write d4/d1c/f27 [5274896,106744] 0 2024-09-16T23:48:56.373 INFO:tasks.workunit.client.1.smithi043.stdout:8/519: write de/d18/d1f/d2f/d39/d7b/d83/f84 [1954113,37007] 0 2024-09-16T23:48:56.373 INFO:tasks.workunit.client.0.smithi042.stdout:7/397: dwrite d3/d15/d41/f45 [4194304,4194304] 0 2024-09-16T23:48:56.375 INFO:tasks.workunit.client.0.smithi042.stdout:2/405: truncate d0/da/d17/f37 4648235 0 2024-09-16T23:48:56.376 INFO:tasks.workunit.client.1.smithi043.stdout:1/628: rmdir d4/df/d13 39 2024-09-16T23:48:56.377 INFO:tasks.workunit.client.0.smithi042.stdout:7/398: truncate d3/d1f/d49/f56 1411726 0 2024-09-16T23:48:56.381 INFO:tasks.workunit.client.1.smithi043.stdout:8/520: mkdir de/d18/d2b/da1 0 2024-09-16T23:48:56.383 INFO:tasks.workunit.client.1.smithi043.stdout:8/521: dread - de/d18/d1f/d25/f6e zero size 2024-09-16T23:48:56.384 INFO:tasks.workunit.client.0.smithi042.stdout:2/406: mkdir d0/da/d17/d3e/d86/d96 0 2024-09-16T23:48:56.385 INFO:tasks.workunit.client.0.smithi042.stdout:7/399: dwrite d3/d15/d24/f6e [0,4194304] 0 2024-09-16T23:48:56.385 INFO:tasks.workunit.client.0.smithi042.stdout:2/407: readlink d0/d78/l85 0 2024-09-16T23:48:56.387 INFO:tasks.workunit.client.1.smithi043.stdout:9/449: mkdir d4/d28/d5f/d86 0 2024-09-16T23:48:56.387 INFO:tasks.workunit.client.0.smithi042.stdout:2/408: truncate d0/da/d17/f2d 1780950 0 2024-09-16T23:48:56.389 INFO:tasks.workunit.client.1.smithi043.stdout:1/629: mkdir d4/d9/d8f/db6/dcc/dda 0 2024-09-16T23:48:56.389 INFO:tasks.workunit.client.0.smithi042.stdout:2/409: chown d0/da/d1b/f29 31423 1 2024-09-16T23:48:56.390 INFO:tasks.workunit.client.0.smithi042.stdout:2/410: read - d0/da/d5b/f6b zero size 2024-09-16T23:48:56.392 INFO:tasks.workunit.client.1.smithi043.stdout:8/522: symlink de/d18/d1f/d2f/d39/d7b/la2 0 2024-09-16T23:48:56.394 INFO:tasks.workunit.client.0.smithi042.stdout:7/400: dwrite d3/d1f/f3d [0,4194304] 0 2024-09-16T23:48:56.394 INFO:tasks.workunit.client.0.smithi042.stdout:2/411: rmdir d0/da 39 2024-09-16T23:48:56.399 INFO:tasks.workunit.client.0.smithi042.stdout:7/401: chown d3/d15/d32/d6b/f83 490047439 1 2024-09-16T23:48:56.399 INFO:tasks.workunit.client.0.smithi042.stdout:9/255: write d4/d10/f3f [2970741,58294] 0 2024-09-16T23:48:56.400 INFO:tasks.workunit.client.1.smithi043.stdout:7/465: getdents d4/db/d11/d15/d29/d7a 0 2024-09-16T23:48:56.400 INFO:tasks.workunit.client.0.smithi042.stdout:7/402: creat d3/d15/d41/f8b x:0 0 0 2024-09-16T23:48:56.400 INFO:tasks.workunit.client.0.smithi042.stdout:4/468: sync 2024-09-16T23:48:56.400 INFO:tasks.workunit.client.0.smithi042.stdout:3/314: sync 2024-09-16T23:48:56.405 INFO:tasks.workunit.client.1.smithi043.stdout:2/443: dwrite d8/d5e/f87 [0,4194304] 0 2024-09-16T23:48:56.405 INFO:tasks.workunit.client.0.smithi042.stdout:3/315: mkdir d2/d1b/d1d/d34/d62/d6a 0 2024-09-16T23:48:56.406 INFO:tasks.workunit.client.0.smithi042.stdout:4/469: read de/d42/d68/f72 [10483,11852] 0 2024-09-16T23:48:56.407 INFO:tasks.workunit.client.1.smithi043.stdout:8/523: rename de/d70/d3d/f42 to de/d70/d3d/fa3 0 2024-09-16T23:48:56.410 INFO:tasks.workunit.client.1.smithi043.stdout:7/466: chown d4/db/l80 5 1 2024-09-16T23:48:56.411 INFO:tasks.workunit.client.0.smithi042.stdout:4/470: rmdir de/d1a/d70 39 2024-09-16T23:48:56.412 INFO:tasks.workunit.client.0.smithi042.stdout:7/403: dwrite d3/f5f [0,4194304] 0 2024-09-16T23:48:56.412 INFO:tasks.workunit.client.0.smithi042.stdout:9/256: mkdir d4/d12/d2d/d51 0 2024-09-16T23:48:56.414 INFO:tasks.workunit.client.0.smithi042.stdout:6/318: write d8/db/dd/f15 [2928269,51653] 0 2024-09-16T23:48:56.415 INFO:tasks.workunit.client.0.smithi042.stdout:3/316: truncate d2/d1b/d1d/d30/f5f 351047 0 2024-09-16T23:48:56.416 INFO:tasks.workunit.client.0.smithi042.stdout:2/412: fsync d0/da/d1b/d48/d81/f80 0 2024-09-16T23:48:56.419 INFO:tasks.workunit.client.1.smithi043.stdout:4/544: dwrite d8/d15/d20/d2b/d54/d55/f94 [0,4194304] 0 2024-09-16T23:48:56.420 INFO:tasks.workunit.client.1.smithi043.stdout:7/467: dwrite d4/db/d11/d21/d31/f88 [0,4194304] 0 2024-09-16T23:48:56.421 INFO:tasks.workunit.client.0.smithi042.stdout:2/413: write d0/da/d17/d3e/f91 [367906,7065] 0 2024-09-16T23:48:56.423 INFO:tasks.workunit.client.0.smithi042.stdout:7/404: unlink d3/f1a 0 2024-09-16T23:48:56.425 INFO:tasks.workunit.client.0.smithi042.stdout:8/290: dwrite da/d19/d1d/f2b [0,4194304] 0 2024-09-16T23:48:56.425 INFO:tasks.workunit.client.0.smithi042.stdout:3/317: creat d2/d1b/d1d/d34/f6b x:0 0 0 2024-09-16T23:48:56.426 INFO:tasks.workunit.client.0.smithi042.stdout:6/319: mknod d8/db/dd/d2c/d26/c6c 0 2024-09-16T23:48:56.427 INFO:tasks.workunit.client.0.smithi042.stdout:7/405: chown d3/d15/d32/c81 6620 1 2024-09-16T23:48:56.428 INFO:tasks.workunit.client.1.smithi043.stdout:7/468: dread - d4/db/d11/d21/d31/f6b zero size 2024-09-16T23:48:56.429 INFO:tasks.workunit.client.0.smithi042.stdout:8/291: write da/d19/f3b [6721076,45410] 0 2024-09-16T23:48:56.429 INFO:tasks.workunit.client.1.smithi043.stdout:7/469: readlink d4/db/d11/l74 0 2024-09-16T23:48:56.431 INFO:tasks.workunit.client.0.smithi042.stdout:2/414: symlink d0/da/d1b/l97 0 2024-09-16T23:48:56.431 INFO:tasks.workunit.client.1.smithi043.stdout:9/450: write d4/d21/f6d [2024309,114878] 0 2024-09-16T23:48:56.432 INFO:tasks.workunit.client.1.smithi043.stdout:7/470: fsync d4/db/d43/d83/d85/f96 0 2024-09-16T23:48:56.433 INFO:tasks.workunit.client.0.smithi042.stdout:7/406: creat d3/d15/d24/d4d/f8c x:0 0 0 2024-09-16T23:48:56.433 INFO:tasks.workunit.client.1.smithi043.stdout:9/451: read d4/d28/d49/f73 [382966,31313] 0 2024-09-16T23:48:56.434 INFO:tasks.workunit.client.1.smithi043.stdout:4/545: mknod d8/d15/d20/d2b/d54/d55/caa 0 2024-09-16T23:48:56.441 INFO:tasks.workunit.client.1.smithi043.stdout:1/630: dwrite d4/d9/d12/d15/f31 [0,4194304] 0 2024-09-16T23:48:56.442 INFO:tasks.workunit.client.0.smithi042.stdout:8/292: write da/d19/d2c/f4e [175384,41156] 0 2024-09-16T23:48:56.442 INFO:tasks.workunit.client.0.smithi042.stdout:7/407: chown d3/c4 13 1 2024-09-16T23:48:56.442 INFO:tasks.workunit.client.0.smithi042.stdout:4/471: dwrite de/d1a/f4b [4194304,4194304] 0 2024-09-16T23:48:56.442 INFO:tasks.workunit.client.0.smithi042.stdout:6/320: mknod d8/d38/c6d 0 2024-09-16T23:48:56.442 INFO:tasks.workunit.client.0.smithi042.stdout:8/293: chown da/f47 0 1 2024-09-16T23:48:56.444 INFO:tasks.workunit.client.0.smithi042.stdout:2/415: fdatasync d0/da/d1b/d48/f58 0 2024-09-16T23:48:56.447 INFO:tasks.workunit.client.1.smithi043.stdout:4/546: creat d8/d25/d39/d58/fab x:0 0 0 2024-09-16T23:48:56.450 INFO:tasks.workunit.client.1.smithi043.stdout:2/444: truncate d8/d5e/d83/d3a/f57 2704952 0 2024-09-16T23:48:56.451 INFO:tasks.workunit.client.0.smithi042.stdout:3/318: dwrite d2/d1b/d52/d60/f42 [0,4194304] 0 2024-09-16T23:48:56.451 INFO:tasks.workunit.client.0.smithi042.stdout:9/257: link d4/d12/d2d/c47 d4/d10/d4b/c52 0 2024-09-16T23:48:56.452 INFO:tasks.workunit.client.1.smithi043.stdout:9/452: symlink d4/d21/d2a/d2c/d7b/l87 0 2024-09-16T23:48:56.453 INFO:tasks.workunit.client.0.smithi042.stdout:4/472: creat de/d42/d82/d94/fa3 x:0 0 0 2024-09-16T23:48:56.455 INFO:tasks.workunit.client.0.smithi042.stdout:3/319: chown d2/d1b/d52/d60/l65 94013 1 2024-09-16T23:48:56.457 INFO:tasks.workunit.client.1.smithi043.stdout:7/471: fsync d4/d27/d41/f40 0 2024-09-16T23:48:56.458 INFO:tasks.workunit.client.0.smithi042.stdout:5/406: sync 2024-09-16T23:48:56.458 INFO:tasks.workunit.client.0.smithi042.stdout:0/259: sync 2024-09-16T23:48:56.458 INFO:tasks.workunit.client.0.smithi042.stdout:1/344: sync 2024-09-16T23:48:56.459 INFO:tasks.workunit.client.0.smithi042.stdout:8/294: write da/d19/d1d/f3e [20260,59603] 0 2024-09-16T23:48:56.460 INFO:tasks.workunit.client.1.smithi043.stdout:1/631: fdatasync d4/d9/d12/d2b/d3d/d83/fc4 0 2024-09-16T23:48:56.460 INFO:tasks.workunit.client.0.smithi042.stdout:4/473: creat de/d42/d99/d9e/fa4 x:0 0 0 2024-09-16T23:48:56.463 INFO:tasks.workunit.client.0.smithi042.stdout:7/408: dwrite d3/d1f/d4f/f5d [0,4194304] 0 2024-09-16T23:48:56.465 INFO:tasks.workunit.client.0.smithi042.stdout:5/407: fsync d9/db/de/d10/d4b/d61/f7c 0 2024-09-16T23:48:56.465 INFO:tasks.workunit.client.0.smithi042.stdout:0/260: readlink d0/df/d23/l2c 0 2024-09-16T23:48:56.465 INFO:tasks.workunit.client.0.smithi042.stdout:2/416: mkdir d0/d98 0 2024-09-16T23:48:56.468 INFO:tasks.workunit.client.1.smithi043.stdout:9/453: rename d4/d5/c3d to d4/d28/d5f/d86/c88 0 2024-09-16T23:48:56.468 INFO:tasks.workunit.client.0.smithi042.stdout:1/345: write d0/d72/f65 [382564,11084] 0 2024-09-16T23:48:56.469 INFO:tasks.workunit.client.1.smithi043.stdout:4/547: dwrite d8/d15/f98 [0,4194304] 0 2024-09-16T23:48:56.469 INFO:tasks.workunit.client.1.smithi043.stdout:2/445: dread d8/d1d/d20/f58 [0,4194304] 0 2024-09-16T23:48:56.471 INFO:tasks.workunit.client.1.smithi043.stdout:9/454: truncate d4/d1c/d25/d2b/d4c/f72 721942 0 2024-09-16T23:48:56.472 INFO:tasks.workunit.client.0.smithi042.stdout:1/346: fdatasync d0/f4a 0 2024-09-16T23:48:56.472 INFO:tasks.workunit.client.0.smithi042.stdout:4/474: creat de/d42/d82/fa5 x:0 0 0 2024-09-16T23:48:56.473 INFO:tasks.workunit.client.0.smithi042.stdout:3/320: creat d2/d1b/d1d/d34/d50/f6c x:0 0 0 2024-09-16T23:48:56.476 INFO:tasks.workunit.client.0.smithi042.stdout:4/475: write de/d12/f48 [61985,35228] 0 2024-09-16T23:48:56.476 INFO:tasks.workunit.client.0.smithi042.stdout:0/261: truncate d0/df/f14 3341876 0 2024-09-16T23:48:56.479 INFO:tasks.workunit.client.0.smithi042.stdout:3/321: fdatasync d2/d1b/d52/d60/f47 0 2024-09-16T23:48:56.479 INFO:tasks.workunit.client.1.smithi043.stdout:7/472: fdatasync d4/d27/d41/f26 0 2024-09-16T23:48:56.480 INFO:tasks.workunit.client.0.smithi042.stdout:2/417: dread d0/da/d1b/d48/f58 [0,4194304] 0 2024-09-16T23:48:56.481 INFO:tasks.workunit.client.0.smithi042.stdout:7/409: dwrite d3/d3f/f5e [0,4194304] 0 2024-09-16T23:48:56.484 INFO:tasks.workunit.client.0.smithi042.stdout:0/262: creat d0/df/f4a x:0 0 0 2024-09-16T23:48:56.485 INFO:tasks.workunit.client.0.smithi042.stdout:4/476: symlink de/d42/d82/d94/la6 0 2024-09-16T23:48:56.486 INFO:tasks.workunit.client.0.smithi042.stdout:8/295: fsync da/d19/f1a 0 2024-09-16T23:48:56.487 INFO:tasks.workunit.client.0.smithi042.stdout:9/258: mknod d4/d12/d2d/d37/d3a/d50/c53 0 2024-09-16T23:48:56.488 INFO:tasks.workunit.client.0.smithi042.stdout:3/322: chown d2/d1b/d1d/c27 1633411 1 2024-09-16T23:48:56.489 INFO:tasks.workunit.client.0.smithi042.stdout:4/477: fsync de/d2e/d37/d4c/f9d 0 2024-09-16T23:48:56.490 INFO:tasks.workunit.client.1.smithi043.stdout:2/446: unlink d8/d1f/l86 0 2024-09-16T23:48:56.491 INFO:tasks.workunit.client.0.smithi042.stdout:5/408: dread d9/db/d11/f40 [0,4194304] 0 2024-09-16T23:48:56.492 INFO:tasks.workunit.client.0.smithi042.stdout:2/418: write d0/da/d17/d3e/f76 [392045,44769] 0 2024-09-16T23:48:56.493 INFO:tasks.workunit.client.1.smithi043.stdout:4/548: creat d8/d25/d39/fac x:0 0 0 2024-09-16T23:48:56.494 INFO:tasks.workunit.client.0.smithi042.stdout:9/259: write d4/d12/d16/f17 [1570472,3985] 0 2024-09-16T23:48:56.494 INFO:tasks.workunit.client.0.smithi042.stdout:7/410: write d3/d15/d24/d4d/f80 [194818,79771] 0 2024-09-16T23:48:56.495 INFO:tasks.workunit.client.0.smithi042.stdout:0/263: creat d0/f4b x:0 0 0 2024-09-16T23:48:56.496 INFO:tasks.workunit.client.0.smithi042.stdout:8/296: dread da/d19/d2c/f4e [0,4194304] 0 2024-09-16T23:48:56.498 INFO:tasks.workunit.client.0.smithi042.stdout:6/321: truncate d8/db/dd/f15 2954489 0 2024-09-16T23:48:56.499 INFO:tasks.workunit.client.0.smithi042.stdout:4/478: rename de/d1a/d1c to de/d1a/da7 0 2024-09-16T23:48:56.501 INFO:tasks.workunit.client.0.smithi042.stdout:1/347: unlink d0/d9/d26/f27 0 2024-09-16T23:48:56.501 INFO:tasks.workunit.client.0.smithi042.stdout:9/260: chown d4/f19 487429 1 2024-09-16T23:48:56.502 INFO:tasks.workunit.client.0.smithi042.stdout:5/409: write d9/db/de/d10/d4b/d61/f4e [2061218,31840] 0 2024-09-16T23:48:56.505 INFO:tasks.workunit.client.1.smithi043.stdout:5/657: sync 2024-09-16T23:48:56.505 INFO:tasks.workunit.client.1.smithi043.stdout:5/658: readlink d3/d1b/l58 0 2024-09-16T23:48:56.505 INFO:tasks.workunit.client.0.smithi042.stdout:8/297: chown da/d19/d1d/f2e 1126651 1 2024-09-16T23:48:56.506 INFO:tasks.workunit.client.1.smithi043.stdout:5/659: fsync d3/db/fc4 0 2024-09-16T23:48:56.506 INFO:tasks.workunit.client.0.smithi042.stdout:3/323: dwrite d2/d1b/d1d/f40 [0,4194304] 0 2024-09-16T23:48:56.507 INFO:tasks.workunit.client.0.smithi042.stdout:7/411: creat d3/d1f/d49/f8d x:0 0 0 2024-09-16T23:48:56.508 INFO:tasks.workunit.client.0.smithi042.stdout:2/419: rmdir d0/d30/d43/d77 39 2024-09-16T23:48:56.510 INFO:tasks.workunit.client.0.smithi042.stdout:8/298: chown da/d19/d1d/d30/c43 47181397 1 2024-09-16T23:48:56.512 INFO:tasks.workunit.client.1.smithi043.stdout:1/632: link d4/d34/fc0 d4/d3e/fdb 0 2024-09-16T23:48:56.514 INFO:tasks.workunit.client.0.smithi042.stdout:4/479: write de/d12/f14 [868504,20128] 0 2024-09-16T23:48:56.517 INFO:tasks.workunit.client.0.smithi042.stdout:9/261: mknod d4/d12/d2d/d37/d41/c54 0 2024-09-16T23:48:56.518 INFO:tasks.workunit.client.0.smithi042.stdout:5/410: rename d9/d2c/c77 to d9/db/d19/d21/d41/d7d/c86 0 2024-09-16T23:48:56.521 INFO:tasks.workunit.client.0.smithi042.stdout:8/299: rename da/d2f to da/d2f/d59 22 2024-09-16T23:48:56.524 INFO:tasks.workunit.client.0.smithi042.stdout:0/264: dread d0/d15/f45 [0,4194304] 0 2024-09-16T23:48:56.525 INFO:tasks.workunit.client.1.smithi043.stdout:5/660: dread d3/d9/d22/f4e [0,4194304] 0 2024-09-16T23:48:56.525 INFO:tasks.workunit.client.0.smithi042.stdout:3/324: truncate d2/f28 3281908 0 2024-09-16T23:48:56.532 INFO:tasks.workunit.client.1.smithi043.stdout:1/633: rmdir d4/df 39 2024-09-16T23:48:56.533 INFO:tasks.workunit.client.0.smithi042.stdout:1/348: dwrite d0/d9/d35/f3a [0,4194304] 0 2024-09-16T23:48:56.534 INFO:tasks.workunit.client.1.smithi043.stdout:9/455: creat d4/d21/f89 x:0 0 0 2024-09-16T23:48:56.534 INFO:tasks.workunit.client.1.smithi043.stdout:2/447: dwrite d8/d4b/d5f/f61 [0,4194304] 0 2024-09-16T23:48:56.538 INFO:tasks.workunit.client.0.smithi042.stdout:7/412: dwrite d3/d15/d24/d4d/f89 [0,4194304] 0 2024-09-16T23:48:56.539 INFO:tasks.workunit.client.0.smithi042.stdout:9/262: symlink d4/d12/d2d/d37/d3a/d50/l55 0 2024-09-16T23:48:56.542 INFO:tasks.workunit.client.1.smithi043.stdout:7/473: rmdir d4/db/d43/d4b 0 2024-09-16T23:48:56.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/480: truncate de/d42/f44 126308 0 2024-09-16T23:48:56.545 INFO:tasks.workunit.client.0.smithi042.stdout:0/265: rename d0/ld to d0/df/d23/d3e/l4c 0 2024-09-16T23:48:56.546 INFO:tasks.workunit.client.1.smithi043.stdout:9/456: dwrite d4/d1c/d25/d2b/d4c/d50/f66 [0,4194304] 0 2024-09-16T23:48:56.547 INFO:tasks.workunit.client.1.smithi043.stdout:9/457: stat d4/d1c/d6b/d67 0 2024-09-16T23:48:56.548 INFO:tasks.workunit.client.0.smithi042.stdout:8/300: dwrite da/d19/f4d [0,4194304] 0 2024-09-16T23:48:56.549 INFO:tasks.workunit.client.1.smithi043.stdout:5/661: creat d3/d9/d22/d60/d9d/fd8 x:0 0 0 2024-09-16T23:48:56.554 INFO:tasks.workunit.client.1.smithi043.stdout:2/448: write d8/d11/d16/f28 [2593734,13347] 0 2024-09-16T23:48:56.554 INFO:tasks.workunit.client.1.smithi043.stdout:7/474: symlink d4/db/d50/l9d 0 2024-09-16T23:48:56.555 INFO:tasks.workunit.client.0.smithi042.stdout:7/413: mkdir d3/d1f/d8e 0 2024-09-16T23:48:56.556 INFO:tasks.workunit.client.1.smithi043.stdout:2/449: chown d8/d35 0 1 2024-09-16T23:48:56.556 INFO:tasks.workunit.client.0.smithi042.stdout:2/420: dwrite d0/da/f2c [0,4194304] 0 2024-09-16T23:48:56.556 INFO:tasks.workunit.client.0.smithi042.stdout:6/322: dread d8/db/dd/f15 [0,4194304] 0 2024-09-16T23:48:56.557 INFO:tasks.workunit.client.0.smithi042.stdout:4/481: dread - de/d2e/d37/f89 zero size 2024-09-16T23:48:56.558 INFO:tasks.workunit.client.1.smithi043.stdout:7/475: write d4/db/d50/f70 [211383,75481] 0 2024-09-16T23:48:56.563 INFO:tasks.workunit.client.0.smithi042.stdout:3/325: dread d2/d1b/d52/d60/f3a [0,4194304] 0 2024-09-16T23:48:56.564 INFO:tasks.workunit.client.0.smithi042.stdout:2/421: chown d0/da/d1b/l83 11800 1 2024-09-16T23:48:56.564 INFO:tasks.workunit.client.0.smithi042.stdout:0/266: mknod d0/df/d13/d2d/d35/c4d 0 2024-09-16T23:48:56.564 INFO:tasks.workunit.client.0.smithi042.stdout:3/326: chown d2/d7/d61 1397 1 2024-09-16T23:48:56.566 INFO:tasks.workunit.client.0.smithi042.stdout:5/411: rename d9/db/de/d10/d4b to d9/db/de/d5e/d87 0 2024-09-16T23:48:56.567 INFO:tasks.workunit.client.1.smithi043.stdout:2/450: rmdir d8/d1d/d20/d23 39 2024-09-16T23:48:56.568 INFO:tasks.workunit.client.1.smithi043.stdout:5/662: rmdir d3/d9 39 2024-09-16T23:48:56.573 INFO:tasks.workunit.client.0.smithi042.stdout:8/301: mkdir da/d19/d1d/d5a 0 2024-09-16T23:48:56.573 INFO:tasks.workunit.client.0.smithi042.stdout:2/422: dread d0/d30/d43/f89 [0,4194304] 0 2024-09-16T23:48:56.573 INFO:tasks.workunit.client.0.smithi042.stdout:7/414: creat d3/d15/d24/d4d/f8f x:0 0 0 2024-09-16T23:48:56.574 INFO:tasks.workunit.client.0.smithi042.stdout:3/327: write d2/d9/dd/f3f [1396421,15010] 0 2024-09-16T23:48:56.575 INFO:tasks.workunit.client.0.smithi042.stdout:5/412: creat d9/db/d73/f88 x:0 0 0 2024-09-16T23:48:56.576 INFO:tasks.workunit.client.0.smithi042.stdout:8/302: write da/d19/f1e [2995095,116569] 0 2024-09-16T23:48:56.576 INFO:tasks.workunit.client.1.smithi043.stdout:7/476: dwrite d4/db/d11/f13 [8388608,4194304] 0 2024-09-16T23:48:56.578 INFO:tasks.workunit.client.1.smithi043.stdout:5/663: unlink d3/db/d19/d3e/l30 0 2024-09-16T23:48:56.581 INFO:tasks.workunit.client.0.smithi042.stdout:9/263: dread d4/d10/f1f [0,4194304] 0 2024-09-16T23:48:56.583 INFO:tasks.workunit.client.0.smithi042.stdout:3/328: unlink d2/d9/c4f 0 2024-09-16T23:48:56.584 INFO:tasks.workunit.client.0.smithi042.stdout:6/323: rename d8/f29 to d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f6e 0 2024-09-16T23:48:56.585 INFO:tasks.workunit.client.1.smithi043.stdout:5/664: stat d3/d9/d22/d8e 0 2024-09-16T23:48:56.589 INFO:tasks.workunit.client.0.smithi042.stdout:3/329: creat d2/d1b/d1d/d34/d62/f6d x:0 0 0 2024-09-16T23:48:56.589 INFO:tasks.workunit.client.0.smithi042.stdout:7/415: dwrite d3/d1f/d49/d76/f25 [0,4194304] 0 2024-09-16T23:48:56.590 INFO:tasks.workunit.client.0.smithi042.stdout:7/416: fsync d3/d1f/f26 0 2024-09-16T23:48:56.591 INFO:tasks.workunit.client.0.smithi042.stdout:1/349: creat d0/d17/d5e/f74 x:0 0 0 2024-09-16T23:48:56.593 INFO:tasks.workunit.client.1.smithi043.stdout:9/458: dread d4/d5/d64/f65 [0,4194304] 0 2024-09-16T23:48:56.594 INFO:tasks.workunit.client.0.smithi042.stdout:3/330: rmdir d2/d9 39 2024-09-16T23:48:56.595 INFO:tasks.workunit.client.1.smithi043.stdout:5/665: symlink d3/d9/d22/d4a/d4c/ld9 0 2024-09-16T23:48:56.596 INFO:tasks.workunit.client.1.smithi043.stdout:7/477: link d4/db/d11/d21/d31/d33/d64/l7e d4/d27/d8a/d93/l9e 0 2024-09-16T23:48:56.596 INFO:tasks.workunit.client.0.smithi042.stdout:3/331: truncate d2/d1b/f3b 4825250 0 2024-09-16T23:48:56.597 INFO:tasks.workunit.client.0.smithi042.stdout:2/423: creat d0/d78/d8b/f99 x:0 0 0 2024-09-16T23:48:56.599 INFO:tasks.workunit.client.1.smithi043.stdout:9/459: creat d4/d1c/d25/d2b/f8a x:0 0 0 2024-09-16T23:48:56.599 INFO:tasks.workunit.client.0.smithi042.stdout:0/267: rename d0/d15/f1f to d0/df/d13/f4e 0 2024-09-16T23:48:56.600 INFO:tasks.workunit.client.0.smithi042.stdout:2/424: write d0/da/d5b/f6b [184951,46236] 0 2024-09-16T23:48:56.600 INFO:tasks.workunit.client.0.smithi042.stdout:1/350: fdatasync d0/d17/f2d 0 2024-09-16T23:48:56.601 INFO:tasks.workunit.client.1.smithi043.stdout:5/666: mknod d3/d79/d91/daf/cda 0 2024-09-16T23:48:56.601 INFO:tasks.workunit.client.1.smithi043.stdout:9/460: truncate d4/d1c/d25/d2b/d4c/d50/f60 753221 0 2024-09-16T23:48:56.602 INFO:tasks.workunit.client.0.smithi042.stdout:8/303: getdents da/d19/d2c/d56 0 2024-09-16T23:48:56.603 INFO:tasks.workunit.client.0.smithi042.stdout:3/332: mknod d2/d1b/d1d/d34/c6e 0 2024-09-16T23:48:56.603 INFO:tasks.workunit.client.0.smithi042.stdout:1/351: write d0/d9/d35/f3a [2568359,117522] 0 2024-09-16T23:48:56.603 INFO:tasks.workunit.client.0.smithi042.stdout:6/324: mknod d8/c6f 0 2024-09-16T23:48:56.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/268: write d0/f4 [614047,5400] 0 2024-09-16T23:48:56.605 INFO:tasks.workunit.client.1.smithi043.stdout:7/478: unlink d4/ld 0 2024-09-16T23:48:56.606 INFO:tasks.workunit.client.0.smithi042.stdout:5/413: rename d9/db/d19/d60 to d9/d4a/d89 0 2024-09-16T23:48:56.608 INFO:tasks.workunit.client.0.smithi042.stdout:1/352: dread d0/d17/f36 [0,4194304] 0 2024-09-16T23:48:56.608 INFO:tasks.workunit.client.0.smithi042.stdout:7/417: link d3/d1f/f23 d3/d3f/f90 0 2024-09-16T23:48:56.608 INFO:tasks.workunit.client.0.smithi042.stdout:3/333: symlink d2/d1b/d52/l6f 0 2024-09-16T23:48:56.611 INFO:tasks.workunit.client.0.smithi042.stdout:0/269: chown d0/df/d23/l3a 1798 1 2024-09-16T23:48:56.612 INFO:tasks.workunit.client.0.smithi042.stdout:2/425: mknod d0/da/d1b/c9a 0 2024-09-16T23:48:56.612 INFO:tasks.workunit.client.1.smithi043.stdout:9/461: rmdir d4/d21/d2a/d2c/d7b 39 2024-09-16T23:48:56.612 INFO:tasks.workunit.client.0.smithi042.stdout:7/418: mknod d3/d15/c91 0 2024-09-16T23:48:56.613 INFO:tasks.workunit.client.0.smithi042.stdout:5/414: creat d9/d4a/d89/f8a x:0 0 0 2024-09-16T23:48:56.615 INFO:tasks.workunit.client.0.smithi042.stdout:3/334: rename d2/d1b/c37 to d2/d1b/d1d/d30/c70 0 2024-09-16T23:48:56.615 INFO:tasks.workunit.client.0.smithi042.stdout:7/419: dread - d3/d15/d32/d6b/f83 zero size 2024-09-16T23:48:56.617 INFO:tasks.workunit.client.0.smithi042.stdout:0/270: symlink d0/df/d13/d49/l4f 0 2024-09-16T23:48:56.618 INFO:tasks.workunit.client.0.smithi042.stdout:7/420: unlink d3/c4 0 2024-09-16T23:48:56.618 INFO:tasks.workunit.client.1.smithi043.stdout:9/462: dwrite d4/d21/d33/f79 [0,4194304] 0 2024-09-16T23:48:56.619 INFO:tasks.workunit.client.0.smithi042.stdout:5/415: symlink d9/db/d1b/d79/l8b 0 2024-09-16T23:48:56.621 INFO:tasks.workunit.client.0.smithi042.stdout:3/335: mkdir d2/d1b/d1d/d71 0 2024-09-16T23:48:56.621 INFO:tasks.workunit.client.1.smithi043.stdout:7/479: creat d4/db/d43/d83/d85/d8b/f9f x:0 0 0 2024-09-16T23:48:56.622 INFO:tasks.workunit.client.0.smithi042.stdout:7/421: mkdir d3/d15/d32/d6b/d64/d92 0 2024-09-16T23:48:56.622 INFO:tasks.workunit.client.0.smithi042.stdout:8/304: dread da/d19/d1d/d30/f31 [0,4194304] 0 2024-09-16T23:48:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:0/271: mknod d0/d15/c50 0 2024-09-16T23:48:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:6/325: mkdir d8/d70 0 2024-09-16T23:48:56.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/305: stat da/d19/l51 0 2024-09-16T23:48:56.624 INFO:tasks.workunit.client.1.smithi043.stdout:7/480: write d4/d27/d41/f8d [4343612,25994] 0 2024-09-16T23:48:56.625 INFO:tasks.workunit.client.0.smithi042.stdout:5/416: creat d9/db/d1b/f8c x:0 0 0 2024-09-16T23:48:56.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/422: mknod d3/c93 0 2024-09-16T23:48:56.628 INFO:tasks.workunit.client.1.smithi043.stdout:5/667: dwrite d3/d9/d22/d60/f82 [0,4194304] 0 2024-09-16T23:48:56.631 INFO:tasks.workunit.client.0.smithi042.stdout:3/336: rename d2/d1b/d52/d60/l48 to d2/d9/dd/d2c/l72 0 2024-09-16T23:48:56.633 INFO:tasks.workunit.client.0.smithi042.stdout:3/337: chown d2/d9/dd/d2c/f32 23219 1 2024-09-16T23:48:56.634 INFO:tasks.workunit.client.0.smithi042.stdout:7/423: creat d3/d15/d32/d6b/d64/f94 x:0 0 0 2024-09-16T23:48:56.635 INFO:tasks.workunit.client.0.smithi042.stdout:8/306: fsync da/d19/d2c/f4e 0 2024-09-16T23:48:56.637 INFO:tasks.workunit.client.1.smithi043.stdout:5/668: symlink d3/db/d15/d4d/d6b/d73/ldb 0 2024-09-16T23:48:56.637 INFO:tasks.workunit.client.0.smithi042.stdout:3/338: mkdir d2/d7/d61/d73 0 2024-09-16T23:48:56.638 INFO:tasks.workunit.client.0.smithi042.stdout:7/424: symlink d3/d15/d32/d6b/d64/l95 0 2024-09-16T23:48:56.639 INFO:tasks.workunit.client.0.smithi042.stdout:2/426: dread d0/f2b [0,4194304] 0 2024-09-16T23:48:56.640 INFO:tasks.workunit.client.1.smithi043.stdout:7/481: creat d4/fa0 x:0 0 0 2024-09-16T23:48:56.641 INFO:tasks.workunit.client.0.smithi042.stdout:6/326: fdatasync d8/db/fc 0 2024-09-16T23:48:56.642 INFO:tasks.workunit.client.0.smithi042.stdout:3/339: link d2/d9/l63 d2/d1b/d52/d60/l74 0 2024-09-16T23:48:56.643 INFO:tasks.workunit.client.1.smithi043.stdout:5/669: symlink d3/db/d15/d4d/ldc 0 2024-09-16T23:48:56.644 INFO:tasks.workunit.client.0.smithi042.stdout:1/353: link d0/d9/fe d0/d9/d35/d46/f75 0 2024-09-16T23:48:56.645 INFO:tasks.workunit.client.0.smithi042.stdout:3/340: creat d2/d1b/f75 x:0 0 0 2024-09-16T23:48:56.647 INFO:tasks.workunit.client.0.smithi042.stdout:3/341: stat d2/d1b/d52/d60/f42 0 2024-09-16T23:48:56.648 INFO:tasks.workunit.client.0.smithi042.stdout:7/425: dwrite d3/d15/d24/d4d/f8c [0,4194304] 0 2024-09-16T23:48:56.650 INFO:tasks.workunit.client.0.smithi042.stdout:7/426: chown d3/d15/d32/d6b/d64/d92 0 1 2024-09-16T23:48:56.656 INFO:tasks.workunit.client.1.smithi043.stdout:7/482: creat d4/db/d43/d83/d85/fa1 x:0 0 0 2024-09-16T23:48:56.657 INFO:tasks.workunit.client.0.smithi042.stdout:3/342: rename d2/d9/l2d to d2/d7/d61/d73/l76 0 2024-09-16T23:48:56.657 INFO:tasks.workunit.client.1.smithi043.stdout:5/670: creat d3/d9/fdd x:0 0 0 2024-09-16T23:48:56.658 INFO:tasks.workunit.client.0.smithi042.stdout:7/427: creat d3/d15/d41/f96 x:0 0 0 2024-09-16T23:48:56.660 INFO:tasks.workunit.client.0.smithi042.stdout:2/427: write d0/f22 [1817601,83150] 0 2024-09-16T23:48:56.661 INFO:tasks.workunit.client.0.smithi042.stdout:1/354: creat d0/d9/d35/d46/f76 x:0 0 0 2024-09-16T23:48:56.662 INFO:tasks.workunit.client.1.smithi043.stdout:5/671: fsync d3/db/d19/f54 0 2024-09-16T23:48:56.662 INFO:tasks.workunit.client.0.smithi042.stdout:6/327: rmdir d8 39 2024-09-16T23:48:56.662 INFO:tasks.workunit.client.0.smithi042.stdout:5/417: dwrite d9/db/d11/f40 [0,4194304] 0 2024-09-16T23:48:56.665 INFO:tasks.workunit.client.0.smithi042.stdout:3/343: chown d2/d9/dd/f3f 50852 1 2024-09-16T23:48:56.667 INFO:tasks.workunit.client.1.smithi043.stdout:5/672: dwrite d3/db/fc4 [0,4194304] 0 2024-09-16T23:48:56.668 INFO:tasks.workunit.client.1.smithi043.stdout:0/525: sync 2024-09-16T23:48:56.668 INFO:tasks.workunit.client.1.smithi043.stdout:6/502: sync 2024-09-16T23:48:56.668 INFO:tasks.workunit.client.0.smithi042.stdout:1/355: write d0/d17/d18/f73 [502687,37269] 0 2024-09-16T23:48:56.668 INFO:tasks.workunit.client.0.smithi042.stdout:7/428: dread - d3/d15/d32/d6b/d64/f94 zero size 2024-09-16T23:48:56.669 INFO:tasks.workunit.client.1.smithi043.stdout:7/483: truncate d4/db/d11/f13 8414339 0 2024-09-16T23:48:56.671 INFO:tasks.workunit.client.0.smithi042.stdout:8/307: dwrite da/d19/f1f [0,4194304] 0 2024-09-16T23:48:56.672 INFO:tasks.workunit.client.1.smithi043.stdout:7/484: dread - d4/db/d43/d83/d85/d8b/f9f zero size 2024-09-16T23:48:56.673 INFO:tasks.workunit.client.0.smithi042.stdout:8/308: chown da/d2f 1548174 1 2024-09-16T23:48:56.674 INFO:tasks.workunit.client.0.smithi042.stdout:0/272: dwrite d0/df/f14 [0,4194304] 0 2024-09-16T23:48:56.675 INFO:tasks.workunit.client.1.smithi043.stdout:0/526: creat d0/d2/df/d1a/d46/fb0 x:0 0 0 2024-09-16T23:48:56.677 INFO:tasks.workunit.client.1.smithi043.stdout:6/503: mknod de/d1a/ca6 0 2024-09-16T23:48:56.677 INFO:tasks.workunit.client.1.smithi043.stdout:5/673: dwrite d3/f50 [0,4194304] 0 2024-09-16T23:48:56.677 INFO:tasks.workunit.client.0.smithi042.stdout:5/418: symlink d9/db/d1b/d31/d46/l8d 0 2024-09-16T23:48:56.679 INFO:tasks.workunit.client.1.smithi043.stdout:6/504: write de/d1a/d39/f1f [926635,79990] 0 2024-09-16T23:48:56.681 INFO:tasks.workunit.client.0.smithi042.stdout:6/328: write d8/db/dd/d1b/d34/f64 [390593,124530] 0 2024-09-16T23:48:56.682 INFO:tasks.workunit.client.0.smithi042.stdout:3/344: write d2/f28 [4131565,83518] 0 2024-09-16T23:48:56.682 INFO:tasks.workunit.client.1.smithi043.stdout:7/485: creat d4/db/d11/d15/d29/d7a/fa2 x:0 0 0 2024-09-16T23:48:56.684 INFO:tasks.workunit.client.0.smithi042.stdout:6/329: write d8/db/dd/f11 [866523,103895] 0 2024-09-16T23:48:56.684 INFO:tasks.workunit.client.0.smithi042.stdout:2/428: creat d0/da/d17/d3e/d86/d96/f9b x:0 0 0 2024-09-16T23:48:56.685 INFO:tasks.workunit.client.0.smithi042.stdout:5/419: symlink d9/db/d11/l8e 0 2024-09-16T23:48:56.688 INFO:tasks.workunit.client.0.smithi042.stdout:0/273: mkdir d0/df/d23/d33/d51 0 2024-09-16T23:48:56.689 INFO:tasks.workunit.client.1.smithi043.stdout:5/674: mkdir d3/d1b/dde 0 2024-09-16T23:48:56.692 INFO:tasks.workunit.client.0.smithi042.stdout:1/356: symlink d0/l77 0 2024-09-16T23:48:56.693 INFO:tasks.workunit.client.0.smithi042.stdout:3/345: mknod d2/d1b/d1d/d71/c77 0 2024-09-16T23:48:56.695 INFO:tasks.workunit.client.0.smithi042.stdout:5/420: dwrite d9/db/f25 [0,4194304] 0 2024-09-16T23:48:56.698 INFO:tasks.workunit.client.0.smithi042.stdout:0/274: creat d0/df/d23/d33/d51/f52 x:0 0 0 2024-09-16T23:48:56.698 INFO:tasks.workunit.client.0.smithi042.stdout:2/429: unlink d0/da/f32 0 2024-09-16T23:48:56.699 INFO:tasks.workunit.client.0.smithi042.stdout:3/346: creat d2/d1b/d1d/d34/d62/f78 x:0 0 0 2024-09-16T23:48:56.699 INFO:tasks.workunit.client.1.smithi043.stdout:5/675: dwrite d3/d9/d22/d4a/d64/d8f/fc9 [0,4194304] 0 2024-09-16T23:48:56.700 INFO:tasks.workunit.client.0.smithi042.stdout:6/330: symlink d8/db/dd/d1b/d50/l71 0 2024-09-16T23:48:56.701 INFO:tasks.workunit.client.0.smithi042.stdout:5/421: write d9/db/d73/f88 [603696,1563] 0 2024-09-16T23:48:56.703 INFO:tasks.workunit.client.0.smithi042.stdout:0/275: dread - d0/f4b zero size 2024-09-16T23:48:56.703 INFO:tasks.workunit.client.0.smithi042.stdout:8/309: creat da/d19/d1d/d30/d49/d58/f5b x:0 0 0 2024-09-16T23:48:56.703 INFO:tasks.workunit.client.1.smithi043.stdout:0/527: rename d0/d2/df/d1a/d46/d39/l51 to d0/d2/df/d1a/d46/d39/daa/lb1 0 2024-09-16T23:48:56.708 INFO:tasks.workunit.client.1.smithi043.stdout:6/505: link de/d23/d34/d9b/f98 de/d23/d34/d9b/d5a/fa7 0 2024-09-16T23:48:56.712 INFO:tasks.workunit.client.0.smithi042.stdout:0/276: creat d0/d48/f53 x:0 0 0 2024-09-16T23:48:56.713 INFO:tasks.workunit.client.0.smithi042.stdout:2/430: creat d0/da/d17/d3e/d86/d96/f9c x:0 0 0 2024-09-16T23:48:56.713 INFO:tasks.workunit.client.0.smithi042.stdout:1/357: dwrite d0/d9/d26/f70 [0,4194304] 0 2024-09-16T23:48:56.714 INFO:tasks.workunit.client.1.smithi043.stdout:5/676: rename d3/db/ccf to d3/d1b/dde/cdf 0 2024-09-16T23:48:56.714 INFO:tasks.workunit.client.1.smithi043.stdout:6/506: creat de/d23/d34/d9b/d50/fa8 x:0 0 0 2024-09-16T23:48:56.715 INFO:tasks.workunit.client.1.smithi043.stdout:5/677: chown d3/db/d15/c2e 227426918 1 2024-09-16T23:48:56.717 INFO:tasks.workunit.client.0.smithi042.stdout:5/422: mknod d9/db/d19/d21/d85/c8f 0 2024-09-16T23:48:56.717 INFO:tasks.workunit.client.0.smithi042.stdout:0/277: dread - d0/df/d23/d33/d51/f52 zero size 2024-09-16T23:48:56.717 INFO:tasks.workunit.client.0.smithi042.stdout:3/347: creat d2/d1b/d1d/d30/f79 x:0 0 0 2024-09-16T23:48:56.718 INFO:tasks.workunit.client.0.smithi042.stdout:8/310: unlink da/d19/d1d/d30/c43 0 2024-09-16T23:48:56.719 INFO:tasks.workunit.client.0.smithi042.stdout:5/423: chown d9/db 63716 1 2024-09-16T23:48:56.719 INFO:tasks.workunit.client.0.smithi042.stdout:0/278: chown d0/df/d13/d49 8251576 1 2024-09-16T23:48:56.720 INFO:tasks.workunit.client.0.smithi042.stdout:8/311: chown da/c3a 23573 1 2024-09-16T23:48:56.720 INFO:tasks.workunit.client.0.smithi042.stdout:0/279: read d0/df/f19 [1398610,8117] 0 2024-09-16T23:48:56.720 INFO:tasks.workunit.client.1.smithi043.stdout:6/507: write de/d23/d34/d9b/d5a/fa7 [368569,84802] 0 2024-09-16T23:48:56.725 INFO:tasks.workunit.client.1.smithi043.stdout:5/678: symlink d3/db/d84/le0 0 2024-09-16T23:48:56.726 INFO:tasks.workunit.client.1.smithi043.stdout:5/679: write d3/db/d15/f5f [4169869,113564] 0 2024-09-16T23:48:56.727 INFO:tasks.workunit.client.0.smithi042.stdout:3/348: rmdir d2/d1b/d1d/d30 39 2024-09-16T23:48:56.727 INFO:tasks.workunit.client.0.smithi042.stdout:6/331: rename d8/db/dd/d2c/d26 to d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72 0 2024-09-16T23:48:56.729 INFO:tasks.workunit.client.0.smithi042.stdout:3/349: readlink d2/d1b/d52/l6f 0 2024-09-16T23:48:56.730 INFO:tasks.workunit.client.1.smithi043.stdout:6/508: truncate de/f67 832983 0 2024-09-16T23:48:56.730 INFO:tasks.workunit.client.1.smithi043.stdout:7/486: write d4/db/d28/f4d [1018323,115448] 0 2024-09-16T23:48:56.732 INFO:tasks.workunit.client.0.smithi042.stdout:0/280: rename d0/df/f21 to d0/df/d23/d3e/f54 0 2024-09-16T23:48:56.734 INFO:tasks.workunit.client.0.smithi042.stdout:3/350: symlink d2/d1b/d1d/d34/d50/l7a 0 2024-09-16T23:48:56.735 INFO:tasks.workunit.client.0.smithi042.stdout:0/281: chown d0/df/d13/d2d/d35/l36 109233 1 2024-09-16T23:48:56.735 INFO:tasks.workunit.client.1.smithi043.stdout:6/509: dwrite de/d23/d49/f6a [0,4194304] 0 2024-09-16T23:48:56.736 INFO:tasks.workunit.client.0.smithi042.stdout:8/312: symlink da/l5c 0 2024-09-16T23:48:56.737 INFO:tasks.workunit.client.0.smithi042.stdout:3/351: write d2/d1b/f43 [1880094,127786] 0 2024-09-16T23:48:56.741 INFO:tasks.workunit.client.0.smithi042.stdout:1/358: creat d0/d9/f78 x:0 0 0 2024-09-16T23:48:56.742 INFO:tasks.workunit.client.0.smithi042.stdout:5/424: dread d9/db/d11/f1c [0,4194304] 0 2024-09-16T23:48:56.742 INFO:tasks.workunit.client.0.smithi042.stdout:1/359: readlink d0/d9/l28 0 2024-09-16T23:48:56.743 INFO:tasks.workunit.client.0.smithi042.stdout:6/332: dwrite d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f51 [0,4194304] 0 2024-09-16T23:48:56.744 INFO:tasks.workunit.client.1.smithi043.stdout:7/487: dwrite d4/db/f51 [0,4194304] 0 2024-09-16T23:48:56.745 INFO:tasks.workunit.client.0.smithi042.stdout:0/282: write d0/df/d23/d3e/f54 [5114928,56615] 0 2024-09-16T23:48:56.746 INFO:tasks.workunit.client.0.smithi042.stdout:3/352: symlink d2/d1b/d1d/d34/d62/l7b 0 2024-09-16T23:48:56.747 INFO:tasks.workunit.client.1.smithi043.stdout:5/680: mknod d3/dce/ce1 0 2024-09-16T23:48:56.747 INFO:tasks.workunit.client.0.smithi042.stdout:5/425: mknod d9/db/d73/c90 0 2024-09-16T23:48:56.747 INFO:tasks.workunit.client.1.smithi043.stdout:5/681: stat d3/d1b/fa6 0 2024-09-16T23:48:56.750 INFO:tasks.workunit.client.0.smithi042.stdout:1/360: mkdir d0/d17/d79 0 2024-09-16T23:48:56.752 INFO:tasks.workunit.client.0.smithi042.stdout:5/426: rename d9/db/d19/c1e to d9/db/d19/d21/c91 0 2024-09-16T23:48:56.752 INFO:tasks.workunit.client.0.smithi042.stdout:0/283: fdatasync d0/df/d13/f34 0 2024-09-16T23:48:56.755 INFO:tasks.workunit.client.0.smithi042.stdout:3/353: creat d2/d1b/d1d/d30/f7c x:0 0 0 2024-09-16T23:48:56.757 INFO:tasks.workunit.client.1.smithi043.stdout:6/510: dwrite fd [0,4194304] 0 2024-09-16T23:48:56.758 INFO:tasks.workunit.client.0.smithi042.stdout:5/427: rmdir d9/db/de/d10/d2f 39 2024-09-16T23:48:56.764 INFO:tasks.workunit.client.0.smithi042.stdout:3/354: symlink d2/l7d 0 2024-09-16T23:48:56.765 INFO:tasks.workunit.client.0.smithi042.stdout:0/284: creat d0/df/d23/f55 x:0 0 0 2024-09-16T23:48:56.766 INFO:tasks.workunit.client.1.smithi043.stdout:6/511: dwrite de/d23/d34/d9b/f73 [0,4194304] 0 2024-09-16T23:48:56.767 INFO:tasks.workunit.client.0.smithi042.stdout:0/285: chown d0/df/d23/l3a 0 1 2024-09-16T23:48:56.769 INFO:tasks.workunit.client.1.smithi043.stdout:7/488: mkdir d4/d27/d41/d1b/d66/da3 0 2024-09-16T23:48:56.770 INFO:tasks.workunit.client.0.smithi042.stdout:5/428: dwrite d9/f74 [0,4194304] 0 2024-09-16T23:48:56.770 INFO:tasks.workunit.client.0.smithi042.stdout:3/355: fsync d2/d9/dd/d2c/f64 0 2024-09-16T23:48:56.774 INFO:tasks.workunit.client.0.smithi042.stdout:3/356: chown d2/d1b/d1d/d71/c77 167675138 1 2024-09-16T23:48:56.774 INFO:tasks.workunit.client.0.smithi042.stdout:6/333: link d8/db/dd/d1b/d2d/f62 d8/db/dd/d1b/d34/d39/d5e/d4c/f73 0 2024-09-16T23:48:56.776 INFO:tasks.workunit.client.1.smithi043.stdout:6/512: dread - de/d1a/d28/f84 zero size 2024-09-16T23:48:56.777 INFO:tasks.workunit.client.1.smithi043.stdout:7/489: creat d4/db/d11/d21/fa4 x:0 0 0 2024-09-16T23:48:56.779 INFO:tasks.workunit.client.0.smithi042.stdout:3/357: mknod d2/d7/d61/d73/c7e 0 2024-09-16T23:48:56.785 INFO:tasks.workunit.client.1.smithi043.stdout:7/490: dwrite d4/db/f73 [0,4194304] 0 2024-09-16T23:48:56.785 INFO:tasks.workunit.client.0.smithi042.stdout:8/313: dread da/d19/d1d/d30/f31 [0,4194304] 0 2024-09-16T23:48:56.785 INFO:tasks.workunit.client.0.smithi042.stdout:1/361: write d0/d9/fe [312015,25216] 0 2024-09-16T23:48:56.788 INFO:tasks.workunit.client.0.smithi042.stdout:8/314: chown da/l5c 129313 1 2024-09-16T23:48:56.789 INFO:tasks.workunit.client.0.smithi042.stdout:6/334: dread d8/db/dd/f22 [0,4194304] 0 2024-09-16T23:48:56.789 INFO:tasks.workunit.client.0.smithi042.stdout:5/429: dwrite d9/db/de/d5e/f67 [0,4194304] 0 2024-09-16T23:48:56.795 INFO:tasks.workunit.client.0.smithi042.stdout:5/430: mkdir d9/db/de/d92 0 2024-09-16T23:48:56.796 INFO:tasks.workunit.client.1.smithi043.stdout:5/682: link d3/cd6 d3/d9/ce2 0 2024-09-16T23:48:56.800 INFO:tasks.workunit.client.1.smithi043.stdout:6/513: link fd de/d23/d34/d9b/fa9 0 2024-09-16T23:48:56.801 INFO:tasks.workunit.client.0.smithi042.stdout:8/315: mknod da/d4b/c5d 0 2024-09-16T23:48:56.801 INFO:tasks.workunit.client.1.smithi043.stdout:5/683: creat d3/d9/d22/d4a/d4c/fe3 x:0 0 0 2024-09-16T23:48:56.802 INFO:tasks.workunit.client.0.smithi042.stdout:5/431: stat d9/d4a/f70 0 2024-09-16T23:48:56.804 INFO:tasks.workunit.client.1.smithi043.stdout:5/684: mkdir d3/d9/d22/d60/d9d/de4 0 2024-09-16T23:48:56.807 INFO:tasks.workunit.client.1.smithi043.stdout:5/685: mkdir d3/d9/d22/d60/d9d/de5 0 2024-09-16T23:48:56.808 INFO:tasks.workunit.client.0.smithi042.stdout:1/362: dwrite d0/d9/d35/d46/d32/d2f/f68 [0,4194304] 0 2024-09-16T23:48:56.809 INFO:tasks.workunit.client.0.smithi042.stdout:6/335: dwrite f3 [0,4194304] 0 2024-09-16T23:48:56.811 INFO:tasks.workunit.client.0.smithi042.stdout:5/432: creat d9/db/d1b/d31/d6c/f93 x:0 0 0 2024-09-16T23:48:56.811 INFO:tasks.workunit.client.0.smithi042.stdout:6/336: readlink d8/db/dd/l17 0 2024-09-16T23:48:56.813 INFO:tasks.workunit.client.0.smithi042.stdout:0/286: dread d0/f7 [0,4194304] 0 2024-09-16T23:48:56.815 INFO:tasks.workunit.client.0.smithi042.stdout:5/433: fdatasync d9/db/f3e 0 2024-09-16T23:48:56.815 INFO:tasks.workunit.client.0.smithi042.stdout:3/358: dread d2/d1b/d1d/d34/f5d [0,4194304] 0 2024-09-16T23:48:56.816 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:56 smithi042 ceph-mon[27914]: pgmap v7: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 50 MiB/s rd, 195 MiB/s wr, 374 op/s 2024-09-16T23:48:56.817 INFO:tasks.workunit.client.0.smithi042.stdout:3/359: stat d2/d1b/f3b 0 2024-09-16T23:48:56.820 INFO:tasks.workunit.client.0.smithi042.stdout:6/337: creat d8/d38/f74 x:0 0 0 2024-09-16T23:48:56.821 INFO:tasks.workunit.client.1.smithi043.stdout:7/491: truncate d4/d27/d41/d1b/f6f 502893 0 2024-09-16T23:48:56.822 INFO:tasks.workunit.client.0.smithi042.stdout:1/363: dread d0/d9/d35/d46/f1e [0,4194304] 0 2024-09-16T23:48:56.822 INFO:tasks.workunit.client.0.smithi042.stdout:3/360: rename d2/d1b/d52 to d2/d9/dd/d7f 0 2024-09-16T23:48:56.823 INFO:tasks.workunit.client.0.smithi042.stdout:6/338: symlink d8/db/dd/d1b/d2d/l75 0 2024-09-16T23:48:56.823 INFO:tasks.workunit.client.0.smithi042.stdout:3/361: readlink d2/d1b/d1d/l5e 0 2024-09-16T23:48:56.825 INFO:tasks.workunit.client.0.smithi042.stdout:5/434: dread d9/db/de/f6f [0,4194304] 0 2024-09-16T23:48:56.826 INFO:tasks.workunit.client.0.smithi042.stdout:3/362: read d2/d1b/d1d/d34/f5d [2187905,127635] 0 2024-09-16T23:48:56.826 INFO:tasks.workunit.client.0.smithi042.stdout:0/287: truncate d0/f9 7959552 0 2024-09-16T23:48:56.828 INFO:tasks.workunit.client.1.smithi043.stdout:7/492: unlink d4/db/d11/d21/l34 0 2024-09-16T23:48:56.828 INFO:tasks.workunit.client.0.smithi042.stdout:1/364: mknod d0/d9/d26/c7a 0 2024-09-16T23:48:56.829 INFO:tasks.workunit.client.1.smithi043.stdout:7/493: stat d4/d27/d41/d1b/l5c 0 2024-09-16T23:48:56.830 INFO:tasks.workunit.client.0.smithi042.stdout:0/288: mkdir d0/df/d23/d33/d56 0 2024-09-16T23:48:56.831 INFO:tasks.workunit.client.0.smithi042.stdout:5/435: creat d9/db/d73/f94 x:0 0 0 2024-09-16T23:48:56.832 INFO:tasks.workunit.client.0.smithi042.stdout:3/363: mkdir d2/d1b/d1d/d34/d62/d80 0 2024-09-16T23:48:56.833 INFO:tasks.workunit.client.0.smithi042.stdout:6/339: dwrite d8/db/dd/d2c/f13 [0,4194304] 0 2024-09-16T23:48:56.833 INFO:tasks.workunit.client.0.smithi042.stdout:1/365: unlink d0/d9/d35/d46/f13 0 2024-09-16T23:48:56.834 INFO:tasks.workunit.client.0.smithi042.stdout:0/289: symlink d0/df/d23/d33/l57 0 2024-09-16T23:48:56.835 INFO:tasks.workunit.client.1.smithi043.stdout:6/514: dwrite de/f40 [0,4194304] 0 2024-09-16T23:48:56.836 INFO:tasks.workunit.client.1.smithi043.stdout:7/494: rename d4/l5 to d4/db/d11/d15/la5 0 2024-09-16T23:48:56.836 INFO:tasks.workunit.client.1.smithi043.stdout:5/686: dwrite d3/d9/d22/d4a/d64/f7b [0,4194304] 0 2024-09-16T23:48:56.840 INFO:tasks.workunit.client.1.smithi043.stdout:7/495: mkdir d4/db/da6 0 2024-09-16T23:48:56.842 INFO:tasks.workunit.client.1.smithi043.stdout:5/687: symlink d3/db/d84/db6/le6 0 2024-09-16T23:48:56.842 INFO:tasks.workunit.client.0.smithi042.stdout:3/364: creat d2/d7/f81 x:0 0 0 2024-09-16T23:48:56.842 INFO:tasks.workunit.client.0.smithi042.stdout:0/290: truncate d0/df/d13/f34 491613 0 2024-09-16T23:48:56.844 INFO:tasks.workunit.client.0.smithi042.stdout:5/436: dwrite d9/d4a/f70 [0,4194304] 0 2024-09-16T23:48:56.844 INFO:tasks.workunit.client.0.smithi042.stdout:3/365: readlink d2/l6 0 2024-09-16T23:48:56.844 INFO:tasks.workunit.client.0.smithi042.stdout:1/366: symlink d0/d9/d35/d46/d32/d2f/l7b 0 2024-09-16T23:48:56.846 INFO:tasks.workunit.client.0.smithi042.stdout:0/291: write d0/f37 [627598,99214] 0 2024-09-16T23:48:56.852 INFO:tasks.workunit.client.0.smithi042.stdout:5/437: fdatasync d9/db/d1b/f8c 0 2024-09-16T23:48:56.853 INFO:tasks.workunit.client.0.smithi042.stdout:0/292: symlink d0/df/l58 0 2024-09-16T23:48:56.853 INFO:tasks.workunit.client.0.smithi042.stdout:1/367: creat d0/d17/d79/f7c x:0 0 0 2024-09-16T23:48:56.854 INFO:tasks.workunit.client.1.smithi043.stdout:7/496: getdents d4/d27/d41 0 2024-09-16T23:48:56.856 INFO:tasks.workunit.client.0.smithi042.stdout:0/293: write d0/f4b [397496,125643] 0 2024-09-16T23:48:56.857 INFO:tasks.workunit.client.0.smithi042.stdout:3/366: dwrite d2/d7/f33 [0,4194304] 0 2024-09-16T23:48:56.859 INFO:tasks.workunit.client.0.smithi042.stdout:5/438: link d9/db/d19/f39 d9/db/d1b/f95 0 2024-09-16T23:48:56.863 INFO:tasks.workunit.client.0.smithi042.stdout:3/367: chown d2/d9/c18 419970 1 2024-09-16T23:48:56.866 INFO:tasks.workunit.client.0.smithi042.stdout:1/368: symlink d0/d9/d35/d46/l7d 0 2024-09-16T23:48:56.867 INFO:tasks.workunit.client.0.smithi042.stdout:0/294: rename d0/d15/f45 to d0/df/d23/d33/d51/f59 0 2024-09-16T23:48:56.869 INFO:tasks.workunit.client.0.smithi042.stdout:3/368: creat d2/d9/dd/d7f/f82 x:0 0 0 2024-09-16T23:48:56.870 INFO:tasks.workunit.client.0.smithi042.stdout:0/295: dread - d0/df/d23/d33/d51/f52 zero size 2024-09-16T23:48:56.874 INFO:tasks.workunit.client.0.smithi042.stdout:1/369: creat d0/d9/d4e/f7e x:0 0 0 2024-09-16T23:48:56.878 INFO:tasks.workunit.client.1.smithi043.stdout:7/497: dread d4/db/d11/d21/f2f [0,4194304] 0 2024-09-16T23:48:56.880 INFO:tasks.workunit.client.1.smithi043.stdout:7/498: readlink d4/d27/d41/l1d 0 2024-09-16T23:48:56.880 INFO:tasks.workunit.client.0.smithi042.stdout:6/340: dread d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/f32 [0,4194304] 0 2024-09-16T23:48:56.881 INFO:tasks.workunit.client.1.smithi043.stdout:4/549: sync 2024-09-16T23:48:56.881 INFO:tasks.workunit.client.1.smithi043.stdout:8/524: sync 2024-09-16T23:48:56.881 INFO:tasks.workunit.client.1.smithi043.stdout:3/500: sync 2024-09-16T23:48:56.882 INFO:tasks.workunit.client.0.smithi042.stdout:5/439: dread d9/db/de/d10/f20 [0,4194304] 0 2024-09-16T23:48:56.882 INFO:tasks.workunit.client.1.smithi043.stdout:5/688: dwrite d3/db/d15/f3c [0,4194304] 0 2024-09-16T23:48:56.883 INFO:tasks.workunit.client.0.smithi042.stdout:3/369: truncate d2/f4 3947347 0 2024-09-16T23:48:56.883 INFO:tasks.workunit.client.1.smithi043.stdout:3/501: chown d4/de/c36 5856572 1 2024-09-16T23:48:56.884 INFO:tasks.workunit.client.0.smithi042.stdout:1/370: dwrite d0/f1d [4194304,4194304] 0 2024-09-16T23:48:56.884 INFO:tasks.workunit.client.1.smithi043.stdout:7/499: mkdir d4/db/d11/d15/d29/da7 0 2024-09-16T23:48:56.885 INFO:tasks.workunit.client.1.smithi043.stdout:3/502: stat d4/de/f7f 0 2024-09-16T23:48:56.885 INFO:tasks.workunit.client.1.smithi043.stdout:5/689: chown d3/d9/d22/d4a/d4c/d8c/lb9 32844 1 2024-09-16T23:48:56.886 INFO:tasks.workunit.client.1.smithi043.stdout:4/550: symlink d8/d15/d6e/lad 0 2024-09-16T23:48:56.886 INFO:tasks.workunit.client.0.smithi042.stdout:5/440: rename d9/db/d1b/c5b to d9/c96 0 2024-09-16T23:48:56.888 INFO:tasks.workunit.client.0.smithi042.stdout:3/370: stat d2/d1b/d1d/d34/f3e 0 2024-09-16T23:48:56.890 INFO:tasks.workunit.client.0.smithi042.stdout:3/371: symlink d2/d1b/d1d/d34/l83 0 2024-09-16T23:48:56.892 INFO:tasks.workunit.client.0.smithi042.stdout:1/371: creat d0/d9/d35/d66/f7f x:0 0 0 2024-09-16T23:48:56.895 INFO:tasks.workunit.client.0.smithi042.stdout:6/341: getdents d8 0 2024-09-16T23:48:56.895 INFO:tasks.workunit.client.0.smithi042.stdout:1/372: rename d0/d9 to d0/d9/d35/d6b/d80 22 2024-09-16T23:48:56.896 INFO:tasks.workunit.client.0.smithi042.stdout:6/342: stat d8/db/dd/l14 0 2024-09-16T23:48:56.897 INFO:tasks.workunit.client.0.smithi042.stdout:5/441: dwrite d9/d4a/f70 [4194304,4194304] 0 2024-09-16T23:48:56.897 INFO:tasks.workunit.client.0.smithi042.stdout:1/373: chown d0/f6 1175 1 2024-09-16T23:48:56.901 INFO:tasks.workunit.client.0.smithi042.stdout:1/374: mknod d0/d3b/c81 0 2024-09-16T23:48:56.903 INFO:tasks.workunit.client.1.smithi043.stdout:5/690: mknod d3/db/d26/ce7 0 2024-09-16T23:48:56.903 INFO:tasks.workunit.client.0.smithi042.stdout:6/343: dwrite d8/db/dd/d1b/f4b [0,4194304] 0 2024-09-16T23:48:56.903 INFO:tasks.workunit.client.1.smithi043.stdout:5/691: chown d3/d9/d6f/d85 1330 1 2024-09-16T23:48:56.904 INFO:tasks.workunit.client.1.smithi043.stdout:7/500: truncate d4/db/d28/f2d 621395 0 2024-09-16T23:48:56.904 INFO:tasks.workunit.client.1.smithi043.stdout:8/525: creat de/d18/d1f/d2f/d39/fa4 x:0 0 0 2024-09-16T23:48:56.904 INFO:tasks.workunit.client.0.smithi042.stdout:3/372: creat d2/d1b/d1d/f84 x:0 0 0 2024-09-16T23:48:56.905 INFO:tasks.workunit.client.0.smithi042.stdout:1/375: creat d0/d3b/f82 x:0 0 0 2024-09-16T23:48:56.906 INFO:tasks.workunit.client.0.smithi042.stdout:3/373: stat d2/d1b/d1d/d34/d50/c4d 0 2024-09-16T23:48:56.908 INFO:tasks.workunit.client.0.smithi042.stdout:5/442: creat d9/db/d1b/f97 x:0 0 0 2024-09-16T23:48:56.908 INFO:tasks.workunit.client.0.smithi042.stdout:6/344: creat d8/db/dd/d1b/d34/f76 x:0 0 0 2024-09-16T23:48:56.910 INFO:tasks.workunit.client.1.smithi043.stdout:3/503: rename d4/de/d35/f83 to d4/de/d1f/d2c/f94 0 2024-09-16T23:48:56.911 INFO:tasks.workunit.client.0.smithi042.stdout:1/376: mknod d0/d72/c83 0 2024-09-16T23:48:56.914 INFO:tasks.workunit.client.1.smithi043.stdout:4/551: link d8/d15/d20/d2b/d54/d55/f7a d8/d25/d39/d41/d57/fae 0 2024-09-16T23:48:56.914 INFO:tasks.workunit.client.1.smithi043.stdout:5/692: dread d3/d9/d22/d4a/d64/f7b [0,4194304] 0 2024-09-16T23:48:56.915 INFO:tasks.workunit.client.0.smithi042.stdout:3/374: creat d2/d1b/d1d/f85 x:0 0 0 2024-09-16T23:48:56.918 INFO:tasks.workunit.client.0.smithi042.stdout:5/443: dwrite d9/db/d19/f52 [0,4194304] 0 2024-09-16T23:48:56.919 INFO:tasks.workunit.client.0.smithi042.stdout:6/345: rename d8/db/dd/d1b/d2d/f62 to d8/db/dd/d2c/d57/f77 0 2024-09-16T23:48:56.919 INFO:tasks.workunit.client.1.smithi043.stdout:5/693: write d3/d9/fdd [283559,103419] 0 2024-09-16T23:48:56.920 INFO:tasks.workunit.client.0.smithi042.stdout:1/377: creat d0/d9/d26/f84 x:0 0 0 2024-09-16T23:48:56.921 INFO:tasks.workunit.client.1.smithi043.stdout:7/501: mknod d4/d27/d41/d1b/d66/da3/ca8 0 2024-09-16T23:48:56.924 INFO:tasks.workunit.client.1.smithi043.stdout:3/504: dwrite d4/de/d35/f4c [0,4194304] 0 2024-09-16T23:48:56.924 INFO:tasks.workunit.client.1.smithi043.stdout:8/526: dread de/d18/f4c [0,4194304] 0 2024-09-16T23:48:56.927 INFO:tasks.workunit.client.1.smithi043.stdout:5/694: mknod d3/d9/d22/d4a/d4c/ce8 0 2024-09-16T23:48:56.931 INFO:tasks.workunit.client.0.smithi042.stdout:6/346: dread d8/db/dd/d1b/d34/f64 [0,4194304] 0 2024-09-16T23:48:56.934 INFO:tasks.workunit.client.1.smithi043.stdout:4/552: dwrite d8/d15/d20/d2b/d54/d55/f7a [0,4194304] 0 2024-09-16T23:48:56.934 INFO:tasks.workunit.client.1.smithi043.stdout:5/695: readlink d3/db/d84/db6/le6 0 2024-09-16T23:48:56.935 INFO:tasks.workunit.client.0.smithi042.stdout:3/375: dwrite d2/d9/dd/f3f [0,4194304] 0 2024-09-16T23:48:56.935 INFO:tasks.workunit.client.1.smithi043.stdout:3/505: symlink d4/l95 0 2024-09-16T23:48:56.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/444: dwrite d9/db/de/d5e/d87/d61/f7c [0,4194304] 0 2024-09-16T23:48:56.940 INFO:tasks.workunit.client.0.smithi042.stdout:6/347: stat d8/db/dd/f11 0 2024-09-16T23:48:56.942 INFO:tasks.workunit.client.0.smithi042.stdout:1/378: dwrite d0/d3b/f82 [0,4194304] 0 2024-09-16T23:48:56.944 INFO:tasks.workunit.client.0.smithi042.stdout:5/445: dread - d9/db/d1b/f97 zero size 2024-09-16T23:48:56.946 INFO:tasks.workunit.client.1.smithi043.stdout:4/553: creat d8/d15/d20/d2b/d23/d27/faf x:0 0 0 2024-09-16T23:48:56.947 INFO:tasks.workunit.client.1.smithi043.stdout:7/502: truncate d4/d27/d41/d1b/f4e 2972809 0 2024-09-16T23:48:56.948 INFO:tasks.workunit.client.1.smithi043.stdout:4/554: write d8/d25/f4f [2219913,16862] 0 2024-09-16T23:48:56.949 INFO:tasks.workunit.client.1.smithi043.stdout:7/503: chown d4/db/da6 3 1 2024-09-16T23:48:56.950 INFO:tasks.workunit.client.0.smithi042.stdout:3/376: write d2/d1b/d1d/d34/f5d [3665176,19127] 0 2024-09-16T23:48:56.951 INFO:tasks.workunit.client.0.smithi042.stdout:5/446: stat d9/db/de/d92 0 2024-09-16T23:48:56.951 INFO:tasks.workunit.client.1.smithi043.stdout:3/506: dwrite d4/de/d40/f82 [0,4194304] 0 2024-09-16T23:48:56.951 INFO:tasks.workunit.client.1.smithi043.stdout:4/555: read d8/d25/d39/d41/f60 [117434,85142] 0 2024-09-16T23:48:56.953 INFO:tasks.workunit.client.1.smithi043.stdout:7/504: readlink d4/db/d11/d21/d31/d65/l98 0 2024-09-16T23:48:56.954 INFO:tasks.workunit.client.0.smithi042.stdout:3/377: chown d2/d1b/d1d/d34/f6b 56 1 2024-09-16T23:48:56.955 INFO:tasks.workunit.client.0.smithi042.stdout:6/348: creat d8/db/dd/d1b/d34/d39/d5e/d4c/f78 x:0 0 0 2024-09-16T23:48:56.956 INFO:tasks.workunit.client.0.smithi042.stdout:3/378: chown d2/d9/dd/d7f/d60/l5a 35306 1 2024-09-16T23:48:56.960 INFO:tasks.workunit.client.0.smithi042.stdout:3/379: write d2/d1b/d1d/d34/d50/f55 [209609,113423] 0 2024-09-16T23:48:56.961 INFO:tasks.workunit.client.1.smithi043.stdout:8/527: getdents de/d18/d27/d8e 0 2024-09-16T23:48:56.962 INFO:tasks.workunit.client.1.smithi043.stdout:8/528: chown de/f90 10856010 1 2024-09-16T23:48:56.963 INFO:tasks.workunit.client.0.smithi042.stdout:5/447: read d9/d4a/d89/f72 [3618121,45458] 0 2024-09-16T23:48:56.964 INFO:tasks.workunit.client.1.smithi043.stdout:7/505: dwrite d4/db/d11/d21/d31/f82 [0,4194304] 0 2024-09-16T23:48:56.964 INFO:tasks.workunit.client.1.smithi043.stdout:3/507: read d4/fb [1416289,87197] 0 2024-09-16T23:48:56.965 INFO:tasks.workunit.client.1.smithi043.stdout:4/556: fdatasync d8/f1d 0 2024-09-16T23:48:56.966 INFO:tasks.workunit.client.1.smithi043.stdout:7/506: chown d4/d27/d41/f26 10 1 2024-09-16T23:48:56.967 INFO:tasks.workunit.client.1.smithi043.stdout:5/696: link d3/db/d15/d4d/d6b/ld5 d3/d9/d6f/d85/le9 0 2024-09-16T23:48:56.968 INFO:tasks.workunit.client.1.smithi043.stdout:7/507: dread - d4/db/d43/d83/d85/d8b/f9f zero size 2024-09-16T23:48:56.969 INFO:tasks.workunit.client.1.smithi043.stdout:5/697: dread - d3/d9/d22/d4a/d4c/fe3 zero size 2024-09-16T23:48:56.970 INFO:tasks.workunit.client.0.smithi042.stdout:1/379: truncate d0/f4a 263912 0 2024-09-16T23:48:56.975 INFO:tasks.workunit.client.0.smithi042.stdout:3/380: dwrite d2/d1b/d1d/f84 [0,4194304] 0 2024-09-16T23:48:56.976 INFO:tasks.workunit.client.1.smithi043.stdout:5/698: dwrite d3/d9/d22/d9f/fc1 [0,4194304] 0 2024-09-16T23:48:56.977 INFO:tasks.workunit.client.1.smithi043.stdout:8/529: mkdir de/d18/d1f/d2f/d39/d7b/d83/da5 0 2024-09-16T23:48:56.978 INFO:tasks.workunit.client.0.smithi042.stdout:3/381: read d2/d7/f36 [73089,57437] 0 2024-09-16T23:48:56.981 INFO:tasks.workunit.client.1.smithi043.stdout:8/530: read de/f23 [970749,80253] 0 2024-09-16T23:48:56.981 INFO:tasks.workunit.client.0.smithi042.stdout:1/380: symlink d0/d17/d79/l85 0 2024-09-16T23:48:56.982 INFO:tasks.workunit.client.0.smithi042.stdout:5/448: dwrite d9/d4a/d89/f65 [0,4194304] 0 2024-09-16T23:48:56.985 INFO:tasks.workunit.client.0.smithi042.stdout:3/382: creat d2/d1b/f86 x:0 0 0 2024-09-16T23:48:56.987 INFO:tasks.workunit.client.1.smithi043.stdout:7/508: creat d4/db/d50/fa9 x:0 0 0 2024-09-16T23:48:56.987 INFO:tasks.workunit.client.0.smithi042.stdout:5/449: creat d9/db/d19/f98 x:0 0 0 2024-09-16T23:48:56.988 INFO:tasks.workunit.client.0.smithi042.stdout:6/349: dread d8/db/dd/f5f [0,4194304] 0 2024-09-16T23:48:56.991 INFO:tasks.workunit.client.0.smithi042.stdout:3/383: mknod d2/d1b/c87 0 2024-09-16T23:48:56.991 INFO:tasks.workunit.client.0.smithi042.stdout:1/381: dread d0/d9/d35/f3a [0,4194304] 0 2024-09-16T23:48:56.992 INFO:tasks.workunit.client.0.smithi042.stdout:6/350: dread - d8/db/dd/d1b/d34/d39/d5e/d4c/f78 zero size 2024-09-16T23:48:56.992 INFO:tasks.workunit.client.0.smithi042.stdout:5/450: readlink d9/d4a/d89/l68 0 2024-09-16T23:48:56.994 INFO:tasks.workunit.client.0.smithi042.stdout:3/384: rmdir d2/d9 39 2024-09-16T23:48:56.995 INFO:tasks.workunit.client.1.smithi043.stdout:8/531: dwrite de/d70/d3d/f88 [0,4194304] 0 2024-09-16T23:48:56.996 INFO:tasks.workunit.client.0.smithi042.stdout:5/451: creat d9/db/d73/f99 x:0 0 0 2024-09-16T23:48:56.998 INFO:tasks.workunit.client.1.smithi043.stdout:4/557: rename d8/d15/d20/d2b/d54/l62 to d8/d15/d20/d2b/d22/lb0 0 2024-09-16T23:48:56.999 INFO:tasks.workunit.client.1.smithi043.stdout:4/558: fdatasync d8/d15/d20/d2b/d54/d55/d49/d4c/f9c 0 2024-09-16T23:48:57.004 INFO:tasks.workunit.client.1.smithi043.stdout:5/699: dread d3/db/d19/d3e/f70 [0,4194304] 0 2024-09-16T23:48:57.008 INFO:tasks.workunit.client.0.smithi042.stdout:1/382: dwrite d0/d9/d35/d46/f76 [0,4194304] 0 2024-09-16T23:48:57.010 INFO:tasks.workunit.client.1.smithi043.stdout:4/559: rmdir d8/d15/d20/d2b/d54 39 2024-09-16T23:48:57.010 INFO:tasks.workunit.client.0.smithi042.stdout:6/351: dwrite d8/db/dd/d1b/d34/f55 [0,4194304] 0 2024-09-16T23:48:57.011 INFO:tasks.workunit.client.1.smithi043.stdout:3/508: getdents d4/d30 0 2024-09-16T23:48:57.011 INFO:tasks.workunit.client.0.smithi042.stdout:5/452: rename d9/d4a/d89/f65 to d9/db/d1b/d31/d46/d58/f9a 0 2024-09-16T23:48:57.013 INFO:tasks.workunit.client.1.smithi043.stdout:3/509: truncate d4/de/f51 746579 0 2024-09-16T23:48:57.015 INFO:tasks.workunit.client.0.smithi042.stdout:5/453: write d9/db/f25 [3674554,119402] 0 2024-09-16T23:48:57.015 INFO:tasks.workunit.client.1.smithi043.stdout:3/510: dread d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:48:57.016 INFO:tasks.workunit.client.0.smithi042.stdout:3/385: rename d2/d1b/d1d/d34/d50/f55 to d2/d1b/d1d/d34/d62/f88 0 2024-09-16T23:48:57.017 INFO:tasks.workunit.client.1.smithi043.stdout:3/511: chown d4/de/d40/d86/f8c 2380990 1 2024-09-16T23:48:57.019 INFO:tasks.workunit.client.1.smithi043.stdout:3/512: dread - d4/de/d40/d86/f8c zero size 2024-09-16T23:48:57.019 INFO:tasks.workunit.client.0.smithi042.stdout:3/386: readlink d2/d1b/d1d/d34/l83 0 2024-09-16T23:48:57.020 INFO:tasks.workunit.client.1.smithi043.stdout:3/513: readlink d4/d30/l39 0 2024-09-16T23:48:57.020 INFO:tasks.workunit.client.0.smithi042.stdout:5/454: creat d9/d4a/f9b x:0 0 0 2024-09-16T23:48:57.021 INFO:tasks.workunit.client.1.smithi043.stdout:3/514: chown d4/de/d1f/d34/f74 1009415 1 2024-09-16T23:48:57.021 INFO:tasks.workunit.client.0.smithi042.stdout:6/352: fdatasync d8/db/dd/d1b/d34/d39/d5e/d4c/f73 0 2024-09-16T23:48:57.023 INFO:tasks.workunit.client.0.smithi042.stdout:1/383: symlink d0/l86 0 2024-09-16T23:48:57.025 INFO:tasks.workunit.client.1.smithi043.stdout:5/700: creat d3/db/d19/d3e/fea x:0 0 0 2024-09-16T23:48:57.026 INFO:tasks.workunit.client.0.smithi042.stdout:5/455: symlink d9/db/d19/d21/d41/l9c 0 2024-09-16T23:48:57.026 INFO:tasks.workunit.client.1.smithi043.stdout:7/509: dread d4/f32 [0,4194304] 0 2024-09-16T23:48:57.028 INFO:tasks.workunit.client.0.smithi042.stdout:3/387: rename d2/d9/dd/d7f/d60 to d2/d1b/d89 0 2024-09-16T23:48:57.028 INFO:tasks.workunit.client.1.smithi043.stdout:5/701: truncate d3/db/d19/d3e/fea 984371 0 2024-09-16T23:48:57.029 INFO:tasks.workunit.client.0.smithi042.stdout:5/456: mkdir d9/db/d1b/d31/d46/d58/d9d 0 2024-09-16T23:48:57.033 INFO:tasks.workunit.client.0.smithi042.stdout:3/388: dread d2/d1b/d89/f45 [0,4194304] 0 2024-09-16T23:48:57.033 INFO:tasks.workunit.client.0.smithi042.stdout:1/384: unlink d0/d17/d18/c48 0 2024-09-16T23:48:57.034 INFO:tasks.workunit.client.0.smithi042.stdout:1/385: read - d0/d3b/f44 zero size 2024-09-16T23:48:57.035 INFO:tasks.workunit.client.1.smithi043.stdout:3/515: symlink d4/de/d40/l96 0 2024-09-16T23:48:57.035 INFO:tasks.workunit.client.0.smithi042.stdout:3/389: fdatasync d2/d1b/f3b 0 2024-09-16T23:48:57.038 INFO:tasks.workunit.client.0.smithi042.stdout:3/390: unlink d2/d1b/d1d/c27 0 2024-09-16T23:48:57.039 INFO:tasks.workunit.client.1.smithi043.stdout:5/702: chown d3/db/d15/c71 4149695 1 2024-09-16T23:48:57.040 INFO:tasks.workunit.client.0.smithi042.stdout:3/391: write d2/d1b/d89/f3c [495195,130169] 0 2024-09-16T23:48:57.042 INFO:tasks.workunit.client.0.smithi042.stdout:3/392: truncate d2/d7/f33 4677716 0 2024-09-16T23:48:57.043 INFO:tasks.workunit.client.1.smithi043.stdout:4/560: link d8/d25/d39/l63 d8/d15/lb1 0 2024-09-16T23:48:57.043 INFO:tasks.workunit.client.0.smithi042.stdout:5/457: dread d9/db/d1b/f95 [0,4194304] 0 2024-09-16T23:48:57.044 INFO:tasks.workunit.client.0.smithi042.stdout:5/458: chown d9/db/de/d5e/d87/d61/d43 1816206 1 2024-09-16T23:48:57.046 INFO:tasks.workunit.client.1.smithi043.stdout:3/516: creat d4/de/d1f/d34/f97 x:0 0 0 2024-09-16T23:48:57.047 INFO:tasks.workunit.client.0.smithi042.stdout:1/386: unlink d0/d9/d35/l37 0 2024-09-16T23:48:57.048 INFO:tasks.workunit.client.0.smithi042.stdout:1/387: rename d0/d9 to d0/d9/d4e/d87 22 2024-09-16T23:48:57.049 INFO:tasks.workunit.client.0.smithi042.stdout:5/459: link d9/db/de/d5e/f67 d9/d63/f9e 0 2024-09-16T23:48:57.051 INFO:tasks.workunit.client.1.smithi043.stdout:4/561: dwrite d8/d15/d9b/d8b/f93 [0,4194304] 0 2024-09-16T23:48:57.053 INFO:tasks.workunit.client.1.smithi043.stdout:3/517: creat d4/de/d40/f98 x:0 0 0 2024-09-16T23:48:57.053 INFO:tasks.workunit.client.0.smithi042.stdout:1/388: dread d0/d17/d18/f73 [0,4194304] 0 2024-09-16T23:48:57.054 INFO:tasks.workunit.client.0.smithi042.stdout:6/353: write d8/db/dd/f5f [33077,71317] 0 2024-09-16T23:48:57.054 INFO:tasks.workunit.client.1.smithi043.stdout:8/532: dwrite de/d18/f4c [0,4194304] 0 2024-09-16T23:48:57.055 INFO:tasks.workunit.client.0.smithi042.stdout:5/460: creat d9/d4a/d89/f9f x:0 0 0 2024-09-16T23:48:57.057 INFO:tasks.workunit.client.1.smithi043.stdout:5/703: rename d3/db/d26/l57 to d3/d9/d22/d4a/d4c/d8c/leb 0 2024-09-16T23:48:57.057 INFO:tasks.workunit.client.0.smithi042.stdout:5/461: write d9/db/d1b/d31/f7a [344683,59801] 0 2024-09-16T23:48:57.058 INFO:tasks.workunit.client.1.smithi043.stdout:5/704: chown d3/db/d19/d3e/f70 4 1 2024-09-16T23:48:57.060 INFO:tasks.workunit.client.1.smithi043.stdout:5/705: write d3/db/d15/f5f [3645607,98636] 0 2024-09-16T23:48:57.061 INFO:tasks.workunit.client.0.smithi042.stdout:3/393: dread d2/d9/dd/f12 [0,4194304] 0 2024-09-16T23:48:57.065 INFO:tasks.workunit.client.0.smithi042.stdout:3/394: write d2/d1b/d1d/f84 [3590981,47430] 0 2024-09-16T23:48:57.065 INFO:tasks.workunit.client.1.smithi043.stdout:4/562: mkdir d8/d15/d20/d2b/d22/db2 0 2024-09-16T23:48:57.066 INFO:tasks.workunit.client.1.smithi043.stdout:4/563: readlink d8/d15/d6e/l9d 0 2024-09-16T23:48:57.067 INFO:tasks.workunit.client.1.smithi043.stdout:7/510: write d4/db/d11/d15/f5f [349472,70562] 0 2024-09-16T23:48:57.069 INFO:tasks.workunit.client.0.smithi042.stdout:5/462: dwrite d9/db/d19/d21/f2b [0,4194304] 0 2024-09-16T23:48:57.072 INFO:tasks.workunit.client.0.smithi042.stdout:1/389: rename d0/d9/d35/d46 to d0/d9/d35/d6b/d88 0 2024-09-16T23:48:57.072 INFO:tasks.workunit.client.1.smithi043.stdout:3/518: symlink d4/de/d40/d4d/l99 0 2024-09-16T23:48:57.073 INFO:tasks.workunit.client.0.smithi042.stdout:5/463: rmdir d9/db/d73 39 2024-09-16T23:48:57.074 INFO:tasks.workunit.client.1.smithi043.stdout:1/634: sync 2024-09-16T23:48:57.075 INFO:tasks.workunit.client.1.smithi043.stdout:9/463: sync 2024-09-16T23:48:57.075 INFO:tasks.workunit.client.1.smithi043.stdout:2/451: sync 2024-09-16T23:48:57.075 INFO:tasks.workunit.client.1.smithi043.stdout:2/452: fdatasync d8/d1d/f2a 0 2024-09-16T23:48:57.076 INFO:tasks.workunit.client.1.smithi043.stdout:8/533: rename de/d18/c1e to de/d70/d3d/ca6 0 2024-09-16T23:48:57.077 INFO:tasks.workunit.client.1.smithi043.stdout:2/453: stat d8/d1d/d20/d26/f3c 0 2024-09-16T23:48:57.078 INFO:tasks.workunit.client.1.smithi043.stdout:2/454: readlink d8/d1f/l98 0 2024-09-16T23:48:57.078 INFO:tasks.workunit.client.0.smithi042.stdout:3/395: rename d2/d1b/c87 to d2/d1b/d1d/d34/d50/c8a 0 2024-09-16T23:48:57.078 INFO:tasks.workunit.client.0.smithi042.stdout:1/390: mknod d0/d72/c89 0 2024-09-16T23:48:57.082 INFO:tasks.workunit.client.1.smithi043.stdout:5/706: creat d3/db/d15/d43/fec x:0 0 0 2024-09-16T23:48:57.083 INFO:tasks.workunit.client.1.smithi043.stdout:4/564: chown d8/d15/d20/d2b/d54/d55/f7a 409522 1 2024-09-16T23:48:57.084 INFO:tasks.workunit.client.1.smithi043.stdout:4/565: chown d8/d15/d9b/f88 78 1 2024-09-16T23:48:57.084 INFO:tasks.workunit.client.1.smithi043.stdout:4/566: chown d8/d15/d20 664964 1 2024-09-16T23:48:57.088 INFO:tasks.workunit.client.1.smithi043.stdout:9/464: creat d4/d1c/d4f/f8b x:0 0 0 2024-09-16T23:48:57.092 INFO:tasks.workunit.client.1.smithi043.stdout:3/519: dwrite d4/de/d40/d86/f8e [0,4194304] 0 2024-09-16T23:48:57.092 INFO:tasks.workunit.client.0.smithi042.stdout:5/464: dwrite d9/f3c [0,4194304] 0 2024-09-16T23:48:57.092 INFO:tasks.workunit.client.1.smithi043.stdout:2/455: write d8/d1d/d20/f58 [2955825,7934] 0 2024-09-16T23:48:57.094 INFO:tasks.workunit.client.1.smithi043.stdout:5/707: mkdir d3/d9/d22/d4a/d4c/ded 0 2024-09-16T23:48:57.094 INFO:tasks.workunit.client.1.smithi043.stdout:2/456: write d8/d1d/d20/f48 [302461,44677] 0 2024-09-16T23:48:57.095 INFO:tasks.workunit.client.0.smithi042.stdout:1/391: dwrite d0/d17/d79/f7c [0,4194304] 0 2024-09-16T23:48:57.097 INFO:tasks.workunit.client.0.smithi042.stdout:3/396: truncate d2/d7/f8 1096424 0 2024-09-16T23:48:57.100 INFO:tasks.workunit.client.0.smithi042.stdout:3/397: symlink d2/d1b/d1d/d30/l8b 0 2024-09-16T23:48:57.102 INFO:tasks.workunit.client.1.smithi043.stdout:9/465: rename d4/d1c/d25/d2b/d4c/d50 to d4/d1c/d4f/d8c 0 2024-09-16T23:48:57.105 INFO:tasks.workunit.client.0.smithi042.stdout:1/392: readlink d0/d9/d35/d43/l6c 0 2024-09-16T23:48:57.108 INFO:tasks.workunit.client.1.smithi043.stdout:9/466: dread d4/d1c/d4f/d8c/f66 [0,4194304] 0 2024-09-16T23:48:57.108 INFO:tasks.workunit.client.1.smithi043.stdout:3/520: mknod d4/de/d1f/c9a 0 2024-09-16T23:48:57.109 INFO:tasks.workunit.client.1.smithi043.stdout:3/521: stat d4/de/f58 0 2024-09-16T23:48:57.110 INFO:tasks.workunit.client.1.smithi043.stdout:8/534: write de/d18/d1f/d25/f6b [1344707,90220] 0 2024-09-16T23:48:57.111 INFO:tasks.workunit.client.0.smithi042.stdout:3/398: dwrite d2/d1b/d1d/d34/f58 [0,4194304] 0 2024-09-16T23:48:57.111 INFO:tasks.workunit.client.1.smithi043.stdout:3/522: dread d4/de/f51 [0,4194304] 0 2024-09-16T23:48:57.113 INFO:tasks.workunit.client.1.smithi043.stdout:3/523: write d4/de/d1f/d2c/f90 [781257,36661] 0 2024-09-16T23:48:57.114 INFO:tasks.workunit.client.0.smithi042.stdout:1/393: symlink d0/d9/l8a 0 2024-09-16T23:48:57.115 INFO:tasks.workunit.client.0.smithi042.stdout:3/399: fsync d2/d1b/d89/f45 0 2024-09-16T23:48:57.118 INFO:tasks.workunit.client.0.smithi042.stdout:3/400: symlink d2/d9/dd/d7f/l8c 0 2024-09-16T23:48:57.123 INFO:tasks.workunit.client.0.smithi042.stdout:1/394: mkdir d0/d9/d26/d8b 0 2024-09-16T23:48:57.123 INFO:tasks.workunit.client.0.smithi042.stdout:3/401: getdents d2/d1b/d1d 0 2024-09-16T23:48:57.124 INFO:tasks.workunit.client.1.smithi043.stdout:8/535: mknod de/d70/d3d/ca7 0 2024-09-16T23:48:57.127 INFO:tasks.workunit.client.1.smithi043.stdout:5/708: link d3/d9/d22/d4a/d4c/d8c/lb9 d3/d79/dcc/lee 0 2024-09-16T23:48:57.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:56 smithi043 ceph-mon[35191]: pgmap v7: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 50 MiB/s rd, 195 MiB/s wr, 374 op/s 2024-09-16T23:48:57.128 INFO:tasks.workunit.client.1.smithi043.stdout:9/467: mknod d4/d1c/d6b/d67/c8d 0 2024-09-16T23:48:57.132 INFO:tasks.workunit.client.0.smithi042.stdout:3/402: dwrite d2/d1b/d1d/d34/d50/f6c [0,4194304] 0 2024-09-16T23:48:57.132 INFO:tasks.workunit.client.1.smithi043.stdout:2/457: getdents d8 0 2024-09-16T23:48:57.134 INFO:tasks.workunit.client.0.smithi042.stdout:1/395: mknod d0/d17/d5e/c8c 0 2024-09-16T23:48:57.135 INFO:tasks.workunit.client.0.smithi042.stdout:3/403: symlink d2/d1b/d1d/d34/d62/d6a/l8d 0 2024-09-16T23:48:57.138 INFO:tasks.workunit.client.0.smithi042.stdout:1/396: fsync d0/d17/f2d 0 2024-09-16T23:48:57.139 INFO:tasks.workunit.client.1.smithi043.stdout:8/536: mkdir de/d18/d1f/d2f/da8 0 2024-09-16T23:48:57.141 INFO:tasks.workunit.client.0.smithi042.stdout:3/404: rename d2/d9/dd to d2/d7/d61/d8e 0 2024-09-16T23:48:57.141 INFO:tasks.workunit.client.0.smithi042.stdout:1/397: write d0/d9/d35/f3a [4336240,128570] 0 2024-09-16T23:48:57.142 INFO:tasks.workunit.client.0.smithi042.stdout:3/405: fdatasync d2/d1b/f43 0 2024-09-16T23:48:57.143 INFO:tasks.workunit.client.0.smithi042.stdout:1/398: fsync d0/d9/d35/d6b/d88/d32/d2f/f68 0 2024-09-16T23:48:57.144 INFO:tasks.workunit.client.0.smithi042.stdout:3/406: write d2/d1b/d89/f45 [627861,92693] 0 2024-09-16T23:48:57.145 INFO:tasks.workunit.client.0.smithi042.stdout:1/399: write d0/d17/d18/f4d [1101615,93716] 0 2024-09-16T23:48:57.156 INFO:tasks.workunit.client.0.smithi042.stdout:1/400: dwrite d0/d17/d79/f7c [0,4194304] 0 2024-09-16T23:48:57.156 INFO:tasks.workunit.client.0.smithi042.stdout:3/407: dwrite d2/d1b/d1d/d34/f58 [0,4194304] 0 2024-09-16T23:48:57.158 INFO:tasks.workunit.client.0.smithi042.stdout:1/401: mkdir d0/d3b/d8d 0 2024-09-16T23:48:57.161 INFO:tasks.workunit.client.0.smithi042.stdout:3/408: creat d2/d7/d61/d73/f8f x:0 0 0 2024-09-16T23:48:57.161 INFO:tasks.workunit.client.0.smithi042.stdout:1/402: mkdir d0/d9/d35/d6b/d8e 0 2024-09-16T23:48:57.162 INFO:tasks.workunit.client.0.smithi042.stdout:1/403: chown d0/d17/f4f 31 1 2024-09-16T23:48:57.164 INFO:tasks.workunit.client.0.smithi042.stdout:3/409: creat d2/d7/d61/f90 x:0 0 0 2024-09-16T23:48:57.167 INFO:tasks.workunit.client.0.smithi042.stdout:3/410: rmdir d2/d1b/d1d/d71 39 2024-09-16T23:48:57.167 INFO:tasks.workunit.client.0.smithi042.stdout:1/404: dread d0/f6 [0,4194304] 0 2024-09-16T23:48:57.168 INFO:tasks.workunit.client.0.smithi042.stdout:3/411: chown d2/d1b/d89/f47 112946093 1 2024-09-16T23:48:57.170 INFO:tasks.workunit.client.0.smithi042.stdout:1/405: mkdir d0/d9/d35/d6b/d88/d8f 0 2024-09-16T23:48:57.170 INFO:tasks.workunit.client.0.smithi042.stdout:3/412: chown d2/d1b/d1d/d30/f7c 1418 1 2024-09-16T23:48:57.172 INFO:tasks.workunit.client.0.smithi042.stdout:3/413: mknod d2/d7/d61/d8e/d7f/c91 0 2024-09-16T23:48:57.175 INFO:tasks.workunit.client.0.smithi042.stdout:3/414: creat d2/d1b/d1d/d34/f92 x:0 0 0 2024-09-16T23:48:57.177 INFO:tasks.workunit.client.0.smithi042.stdout:1/406: rename d0/d9/d35 to d0/d9/d90 0 2024-09-16T23:48:57.178 INFO:tasks.workunit.client.0.smithi042.stdout:3/415: unlink d2/f24 0 2024-09-16T23:48:57.178 INFO:tasks.workunit.client.0.smithi042.stdout:1/407: dread d0/d9/d90/d6b/d88/f2b [0,4194304] 0 2024-09-16T23:48:57.179 INFO:tasks.workunit.client.0.smithi042.stdout:3/416: stat d2/d1b/d89/f3a 0 2024-09-16T23:48:57.180 INFO:tasks.workunit.client.0.smithi042.stdout:3/417: write d2/d7/d61/d8e/f5b [464550,94071] 0 2024-09-16T23:48:57.180 INFO:tasks.workunit.client.0.smithi042.stdout:1/408: mknod d0/d9/d90/d66/c91 0 2024-09-16T23:48:57.184 INFO:tasks.workunit.client.0.smithi042.stdout:3/418: mkdir d2/d1b/d1d/d34/d50/d93 0 2024-09-16T23:48:57.184 INFO:tasks.workunit.client.0.smithi042.stdout:1/409: rename d0/d9/d90/d6b/d88/d8f to d0/d17/d79/d92 0 2024-09-16T23:48:57.187 INFO:tasks.workunit.client.0.smithi042.stdout:1/410: rename d0/d17/c52 to d0/d9/d4e/c93 0 2024-09-16T23:48:57.194 INFO:tasks.workunit.client.0.smithi042.stdout:1/411: dread d0/d17/f4f [0,4194304] 0 2024-09-16T23:48:57.197 INFO:tasks.workunit.client.0.smithi042.stdout:3/419: dread d2/f4 [0,4194304] 0 2024-09-16T23:48:57.198 INFO:tasks.workunit.client.0.smithi042.stdout:1/412: dread d0/f4a [0,4194304] 0 2024-09-16T23:48:57.199 INFO:tasks.workunit.client.0.smithi042.stdout:3/420: creat d2/d1b/f94 x:0 0 0 2024-09-16T23:48:57.200 INFO:tasks.workunit.client.0.smithi042.stdout:1/413: dread d0/d17/d18/f73 [0,4194304] 0 2024-09-16T23:48:57.201 INFO:tasks.workunit.client.0.smithi042.stdout:1/414: write d0/d17/f58 [227527,90604] 0 2024-09-16T23:48:57.201 INFO:tasks.workunit.client.0.smithi042.stdout:3/421: truncate d2/d1b/d1d/d34/d50/f6c 4947699 0 2024-09-16T23:48:57.202 INFO:tasks.workunit.client.0.smithi042.stdout:1/415: stat d0/d3b/f44 0 2024-09-16T23:48:57.208 INFO:tasks.workunit.client.0.smithi042.stdout:3/422: dread d2/d1b/d1d/f44 [0,4194304] 0 2024-09-16T23:48:57.209 INFO:tasks.workunit.client.0.smithi042.stdout:1/416: dwrite d0/d17/f58 [0,4194304] 0 2024-09-16T23:48:57.209 INFO:tasks.workunit.client.1.smithi043.stdout:8/537: mkdir de/d18/d27/d41/d5a/da9 0 2024-09-16T23:48:57.209 INFO:tasks.workunit.client.1.smithi043.stdout:3/524: truncate d4/de/d40/f82 484914 0 2024-09-16T23:48:57.210 INFO:tasks.workunit.client.1.smithi043.stdout:9/468: getdents d4/d28/d49 0 2024-09-16T23:48:57.211 INFO:tasks.workunit.client.1.smithi043.stdout:3/525: chown d4/de/d1f/d34/f74 463787 1 2024-09-16T23:48:57.212 INFO:tasks.workunit.client.1.smithi043.stdout:2/458: dread d8/d11/f32 [0,4194304] 0 2024-09-16T23:48:57.214 INFO:tasks.workunit.client.0.smithi042.stdout:1/417: dread d0/d17/f4f [0,4194304] 0 2024-09-16T23:48:57.214 INFO:tasks.workunit.client.1.smithi043.stdout:8/538: unlink de/d18/d1f/d25/l96 0 2024-09-16T23:48:57.216 INFO:tasks.workunit.client.1.smithi043.stdout:5/709: dwrite d3/db/d15/d4d/f6c [0,4194304] 0 2024-09-16T23:48:57.217 INFO:tasks.workunit.client.0.smithi042.stdout:3/423: write d2/d1b/d1d/d34/f6b [344,90110] 0 2024-09-16T23:48:57.218 INFO:tasks.workunit.client.1.smithi043.stdout:2/459: creat d8/d4b/d5f/d63/f99 x:0 0 0 2024-09-16T23:48:57.221 INFO:tasks.workunit.client.0.smithi042.stdout:1/418: dwrite d0/f3 [4194304,4194304] 0 2024-09-16T23:48:57.222 INFO:tasks.workunit.client.1.smithi043.stdout:3/526: rename d4/de/d40/d4d/l73 to d4/de/d1f/d34/l9b 0 2024-09-16T23:48:57.222 INFO:tasks.workunit.client.0.smithi042.stdout:3/424: truncate d2/d7/d61/d8e/d2c/f32 1456897 0 2024-09-16T23:48:57.224 INFO:tasks.workunit.client.1.smithi043.stdout:9/469: getdents d4 0 2024-09-16T23:48:57.227 INFO:tasks.workunit.client.1.smithi043.stdout:2/460: readlink d8/d1d/d20/d23/d38/l50 0 2024-09-16T23:48:57.228 INFO:tasks.workunit.client.1.smithi043.stdout:5/710: dwrite d3/db/d19/d3e/f45 [0,4194304] 0 2024-09-16T23:48:57.230 INFO:tasks.workunit.client.0.smithi042.stdout:1/419: link d0/d17/f58 d0/d9/f94 0 2024-09-16T23:48:57.231 INFO:tasks.workunit.client.1.smithi043.stdout:5/711: write d3/f50 [1427917,12759] 0 2024-09-16T23:48:57.232 INFO:tasks.workunit.client.1.smithi043.stdout:8/539: rmdir de/d18/d1f/d2f/da8 0 2024-09-16T23:48:57.237 INFO:tasks.workunit.client.1.smithi043.stdout:8/540: chown de/d18/d27/c28 1429459105 1 2024-09-16T23:48:57.238 INFO:tasks.workunit.client.1.smithi043.stdout:3/527: dwrite d4/de/d5d/f6c [0,4194304] 0 2024-09-16T23:48:57.242 INFO:tasks.workunit.client.0.smithi042.stdout:3/425: dread d2/f4a [0,4194304] 0 2024-09-16T23:48:57.243 INFO:tasks.workunit.client.1.smithi043.stdout:2/461: mknod d8/d5e/d83/d3a/c9a 0 2024-09-16T23:48:57.243 INFO:tasks.workunit.client.1.smithi043.stdout:5/712: dwrite d3/d9/d22/d4a/d64/d8f/fc9 [0,4194304] 0 2024-09-16T23:48:57.244 INFO:tasks.workunit.client.0.smithi042.stdout:3/426: fsync d2/d9/f66 0 2024-09-16T23:48:57.244 INFO:tasks.workunit.client.1.smithi043.stdout:5/713: dread - d3/db/d15/d43/fb7 zero size 2024-09-16T23:48:57.245 INFO:tasks.workunit.client.0.smithi042.stdout:3/427: write d2/d7/d61/d8e/f3f [1277231,81760] 0 2024-09-16T23:48:57.246 INFO:tasks.workunit.client.1.smithi043.stdout:6/515: sync 2024-09-16T23:48:57.246 INFO:tasks.workunit.client.1.smithi043.stdout:0/528: sync 2024-09-16T23:48:57.251 INFO:tasks.workunit.client.0.smithi042.stdout:3/428: getdents d2/d7/d61/d8e/d2c 0 2024-09-16T23:48:57.252 INFO:tasks.workunit.client.1.smithi043.stdout:3/528: read d4/de/d40/f45 [7895646,27539] 0 2024-09-16T23:48:57.253 INFO:tasks.workunit.client.0.smithi042.stdout:3/429: dread - d2/d9/f66 zero size 2024-09-16T23:48:57.253 INFO:tasks.workunit.client.1.smithi043.stdout:3/529: fdatasync d4/d30/d53/d88/f93 0 2024-09-16T23:48:57.254 INFO:tasks.workunit.client.1.smithi043.stdout:9/470: dread d4/d1c/d6b/f57 [0,4194304] 0 2024-09-16T23:48:57.254 INFO:tasks.workunit.client.0.smithi042.stdout:1/420: creat d0/f95 x:0 0 0 2024-09-16T23:48:57.255 INFO:tasks.workunit.client.0.smithi042.stdout:3/430: mkdir d2/d95 0 2024-09-16T23:48:57.256 INFO:tasks.workunit.client.1.smithi043.stdout:2/462: write d8/d11/d16/f19 [4909983,94871] 0 2024-09-16T23:48:57.257 INFO:tasks.workunit.client.0.smithi042.stdout:1/421: dread d0/d9/d90/d6b/d88/d32/f33 [0,4194304] 0 2024-09-16T23:48:57.257 INFO:tasks.workunit.client.1.smithi043.stdout:2/463: read - d8/d1d/f5a zero size 2024-09-16T23:48:57.260 INFO:tasks.workunit.client.0.smithi042.stdout:3/431: mkdir d2/d1b/d1d/d34/d50/d93/d96 0 2024-09-16T23:48:57.261 INFO:tasks.workunit.client.1.smithi043.stdout:0/529: mkdir d0/d2/df/d1a/d49/db2 0 2024-09-16T23:48:57.262 INFO:tasks.workunit.client.1.smithi043.stdout:6/516: truncate de/d1a/d39/f75 421712 0 2024-09-16T23:48:57.264 INFO:tasks.workunit.client.1.smithi043.stdout:9/471: dwrite d4/d1c/d25/d2b/d4c/f72 [0,4194304] 0 2024-09-16T23:48:57.266 INFO:tasks.workunit.client.1.smithi043.stdout:8/541: rename de/d18/d1f/d2f/d39/d4d/c53 to de/d18/d27/d8e/caa 0 2024-09-16T23:48:57.267 INFO:tasks.workunit.client.1.smithi043.stdout:8/542: chown de/f45 155708 1 2024-09-16T23:48:57.267 INFO:tasks.workunit.client.1.smithi043.stdout:8/543: chown de/d18/d27/d41/d5a/f1b 31 1 2024-09-16T23:48:57.268 INFO:tasks.workunit.client.1.smithi043.stdout:8/544: stat de/d18/d1f/d25/c9a 0 2024-09-16T23:48:57.271 INFO:tasks.workunit.client.0.smithi042.stdout:3/432: dwrite d2/d1b/d1d/d34/f3e [4194304,4194304] 0 2024-09-16T23:48:57.271 INFO:tasks.workunit.client.1.smithi043.stdout:3/530: chown d4/d30/l7a 7863157 1 2024-09-16T23:48:57.275 INFO:tasks.workunit.client.1.smithi043.stdout:5/714: creat d3/d9/d22/d4a/dac/fef x:0 0 0 2024-09-16T23:48:57.276 INFO:tasks.workunit.client.0.smithi042.stdout:3/433: rename d2/d7/d61/d73/c7e to d2/d1b/d1d/d34/d50/d93/c97 0 2024-09-16T23:48:57.277 INFO:tasks.workunit.client.1.smithi043.stdout:2/464: creat d8/d5e/d83/d3a/d77/f9b x:0 0 0 2024-09-16T23:48:57.278 INFO:tasks.workunit.client.0.smithi042.stdout:3/434: write d2/d7/d61/d8e/f12 [5148566,11014] 0 2024-09-16T23:48:57.280 INFO:tasks.workunit.client.1.smithi043.stdout:6/517: rmdir de/d1a 39 2024-09-16T23:48:57.280 INFO:tasks.workunit.client.1.smithi043.stdout:5/715: dread d3/d9/d22/d60/f82 [0,4194304] 0 2024-09-16T23:48:57.281 INFO:tasks.workunit.client.1.smithi043.stdout:9/472: symlink d4/d21/d2a/l8e 0 2024-09-16T23:48:57.282 INFO:tasks.workunit.client.0.smithi042.stdout:1/422: write d0/d17/f2d [813121,40837] 0 2024-09-16T23:48:57.282 INFO:tasks.workunit.client.1.smithi043.stdout:3/531: mknod d4/de/d40/d86/c9c 0 2024-09-16T23:48:57.283 INFO:tasks.workunit.client.1.smithi043.stdout:3/532: readlink d4/de/d1f/d2c/l3d 0 2024-09-16T23:48:57.285 INFO:tasks.workunit.client.1.smithi043.stdout:0/530: mknod d0/d2/df/d1a/cb3 0 2024-09-16T23:48:57.286 INFO:tasks.workunit.client.1.smithi043.stdout:6/518: creat de/d23/d49/faa x:0 0 0 2024-09-16T23:48:57.287 INFO:tasks.workunit.client.1.smithi043.stdout:9/473: dwrite d4/d21/f6d [0,4194304] 0 2024-09-16T23:48:57.289 INFO:tasks.workunit.client.1.smithi043.stdout:5/716: fdatasync d3/db/d15/f75 0 2024-09-16T23:48:57.290 INFO:tasks.workunit.client.1.smithi043.stdout:9/474: rmdir d4/d1c/d25/d2b 39 2024-09-16T23:48:57.292 INFO:tasks.workunit.client.1.smithi043.stdout:2/465: rename d8/d1d/d25/d94 to d8/d5e/d83/d9c 0 2024-09-16T23:48:57.295 INFO:tasks.workunit.client.1.smithi043.stdout:5/717: write d3/d9/d22/d4a/d64/f7b [1935906,52791] 0 2024-09-16T23:48:57.297 INFO:tasks.workunit.client.1.smithi043.stdout:2/466: symlink d8/d1d/d20/d23/d38/l9d 0 2024-09-16T23:48:57.298 INFO:tasks.workunit.client.1.smithi043.stdout:3/533: getdents d4/de/d35 0 2024-09-16T23:48:57.299 INFO:tasks.workunit.client.1.smithi043.stdout:5/718: mkdir d3/d79/dcc/df0 0 2024-09-16T23:48:57.301 INFO:tasks.workunit.client.1.smithi043.stdout:2/467: mkdir d8/d5e/d83/d9e 0 2024-09-16T23:48:57.304 INFO:tasks.workunit.client.1.smithi043.stdout:5/719: creat d3/db/d19/d3e/ff1 x:0 0 0 2024-09-16T23:48:57.306 INFO:tasks.workunit.client.1.smithi043.stdout:3/534: dwrite d4/de/d1f/d34/f97 [0,4194304] 0 2024-09-16T23:48:57.308 INFO:tasks.workunit.client.1.smithi043.stdout:5/720: mknod d3/d9/d6f/d7c/d80/cf2 0 2024-09-16T23:48:57.309 INFO:tasks.workunit.client.1.smithi043.stdout:3/535: symlink d4/de/d40/d4d/l9d 0 2024-09-16T23:48:57.312 INFO:tasks.workunit.client.1.smithi043.stdout:3/536: write d4/f67 [676246,35778] 0 2024-09-16T23:48:57.313 INFO:tasks.workunit.client.1.smithi043.stdout:0/531: dread d0/fa3 [0,4194304] 0 2024-09-16T23:48:57.313 INFO:tasks.workunit.client.1.smithi043.stdout:5/721: unlink d3/d9/d22/d4a/d4c/d8c/lb9 0 2024-09-16T23:48:57.313 INFO:tasks.workunit.client.1.smithi043.stdout:3/537: write d4/de/d1f/d34/f74 [3630421,27911] 0 2024-09-16T23:48:57.314 INFO:tasks.workunit.client.1.smithi043.stdout:3/538: dread - d4/f7e zero size 2024-09-16T23:48:57.316 INFO:tasks.workunit.client.1.smithi043.stdout:5/722: creat d3/d9/d22/d4a/d64/d8f/da1/ff3 x:0 0 0 2024-09-16T23:48:57.319 INFO:tasks.workunit.client.1.smithi043.stdout:3/539: getdents d4 0 2024-09-16T23:48:57.321 INFO:tasks.workunit.client.1.smithi043.stdout:5/723: dread d3/db/d19/d3e/f8b [0,4194304] 0 2024-09-16T23:48:57.322 INFO:tasks.workunit.client.1.smithi043.stdout:5/724: write d3/db/d19/f54 [1256444,15072] 0 2024-09-16T23:48:57.342 INFO:tasks.workunit.client.0.smithi042.stdout:3/435: dread d2/d1b/d89/f31 [0,4194304] 0 2024-09-16T23:48:57.342 INFO:tasks.workunit.client.0.smithi042.stdout:3/436: dread - d2/d7/f81 zero size 2024-09-16T23:48:57.343 INFO:tasks.workunit.client.0.smithi042.stdout:3/437: read d2/d1b/d1d/f25 [417265,42993] 0 2024-09-16T23:48:57.347 INFO:tasks.workunit.client.0.smithi042.stdout:3/438: rename d2/d7/d61/d8e/d7f/c54 to d2/d1b/d1d/d30/c98 0 2024-09-16T23:48:57.350 INFO:tasks.workunit.client.0.smithi042.stdout:3/439: unlink d2/d7/f8 0 2024-09-16T23:48:57.366 INFO:tasks.workunit.client.0.smithi042.stdout:3/440: rename d2/d1b/d1d/d30/f5f to d2/d1b/d1d/d34/d50/d93/f99 0 2024-09-16T23:48:57.366 INFO:tasks.workunit.client.0.smithi042.stdout:3/441: dread - d2/f41 zero size 2024-09-16T23:48:57.369 INFO:tasks.workunit.client.0.smithi042.stdout:3/442: creat d2/d7/d61/d73/f9a x:0 0 0 2024-09-16T23:48:57.369 INFO:tasks.workunit.client.0.smithi042.stdout:3/443: chown d2/d7/f81 32689 1 2024-09-16T23:48:57.373 INFO:tasks.workunit.client.0.smithi042.stdout:3/444: rmdir d2/d1b/d1d/d71 39 2024-09-16T23:48:57.377 INFO:tasks.workunit.client.1.smithi043.stdout:0/532: dread d0/d2/df/d1a/d49/d71/f7e [0,4194304] 0 2024-09-16T23:48:57.381 INFO:tasks.workunit.client.1.smithi043.stdout:7/511: sync 2024-09-16T23:48:57.382 INFO:tasks.workunit.client.1.smithi043.stdout:7/512: write d4/db/d11/d15/d29/d7a/fa2 [101109,29620] 0 2024-09-16T23:48:57.390 INFO:tasks.workunit.client.0.smithi042.stdout:4/482: sync 2024-09-16T23:48:57.392 INFO:tasks.workunit.client.1.smithi043.stdout:3/540: rename d4/d30/f7b to d4/de/d40/d86/f9e 0 2024-09-16T23:48:57.393 INFO:tasks.workunit.client.0.smithi042.stdout:4/483: symlink de/d42/d5d/la8 0 2024-09-16T23:48:57.393 INFO:tasks.workunit.client.0.smithi042.stdout:4/484: readlink de/d12/l3a 0 2024-09-16T23:48:57.394 INFO:tasks.workunit.client.1.smithi043.stdout:5/725: mkdir d3/d79/d91/df4 0 2024-09-16T23:48:57.396 INFO:tasks.workunit.client.1.smithi043.stdout:0/533: mknod d0/d2/df/d1a/d46/d92/cb4 0 2024-09-16T23:48:57.397 INFO:tasks.workunit.client.1.smithi043.stdout:0/534: chown d0/d2/df/c9d 0 1 2024-09-16T23:48:57.397 INFO:tasks.workunit.client.1.smithi043.stdout:7/513: creat d4/d27/d8a/faa x:0 0 0 2024-09-16T23:48:57.399 INFO:tasks.workunit.client.0.smithi042.stdout:4/485: rename de/d1a/da7/l30 to de/d2e/d37/d4c/d4f/la9 0 2024-09-16T23:48:57.399 INFO:tasks.workunit.client.0.smithi042.stdout:4/486: fsync de/d2e/d37/f65 0 2024-09-16T23:48:57.400 INFO:tasks.workunit.client.1.smithi043.stdout:5/726: creat d3/db/d15/d4d/d6b/ff5 x:0 0 0 2024-09-16T23:48:57.400 INFO:tasks.workunit.client.0.smithi042.stdout:4/487: stat de/d2e/d37/d4c/d4f/f7a 0 2024-09-16T23:48:57.401 INFO:tasks.workunit.client.0.smithi042.stdout:3/445: mkdir d2/d9b 0 2024-09-16T23:48:57.401 INFO:tasks.workunit.client.0.smithi042.stdout:1/423: write d0/d9/d90/d6b/d88/f12 [4982210,44377] 0 2024-09-16T23:48:57.402 INFO:tasks.workunit.client.1.smithi043.stdout:0/535: write d0/f4d [248621,99882] 0 2024-09-16T23:48:57.402 INFO:tasks.workunit.client.0.smithi042.stdout:4/488: chown de/d1a/f46 7 1 2024-09-16T23:48:57.403 INFO:tasks.workunit.client.1.smithi043.stdout:0/536: readlink d0/la2 0 2024-09-16T23:48:57.405 INFO:tasks.workunit.client.0.smithi042.stdout:1/424: creat d0/d17/f96 x:0 0 0 2024-09-16T23:48:57.406 INFO:tasks.workunit.client.1.smithi043.stdout:3/541: dread d4/de/d1f/f79 [0,4194304] 0 2024-09-16T23:48:57.406 INFO:tasks.workunit.client.0.smithi042.stdout:4/489: mknod de/d1a/caa 0 2024-09-16T23:48:57.406 INFO:tasks.workunit.client.0.smithi042.stdout:4/490: fsync de/d42/d82/f8e 0 2024-09-16T23:48:57.407 INFO:tasks.workunit.client.0.smithi042.stdout:1/425: fsync d0/d9/d90/d6b/d88/d32/d2f/f68 0 2024-09-16T23:48:57.408 INFO:tasks.workunit.client.1.smithi043.stdout:0/537: symlink d0/d2/df/d1a/d46/d39/d6e/lb5 0 2024-09-16T23:48:57.408 INFO:tasks.workunit.client.1.smithi043.stdout:0/538: dread - d0/fa1 zero size 2024-09-16T23:48:57.409 INFO:tasks.workunit.client.0.smithi042.stdout:3/446: stat d2/d1b/d1d/d34/f3e 0 2024-09-16T23:48:57.410 INFO:tasks.workunit.client.0.smithi042.stdout:4/491: write de/d42/d68/f72 [1185938,23824] 0 2024-09-16T23:48:57.412 INFO:tasks.workunit.client.0.smithi042.stdout:3/447: write d2/d1b/d1d/d30/f79 [92312,67882] 0 2024-09-16T23:48:57.414 INFO:tasks.workunit.client.1.smithi043.stdout:0/539: creat d0/d2/df/d5a/d9c/fb6 x:0 0 0 2024-09-16T23:48:57.414 INFO:tasks.workunit.client.0.smithi042.stdout:4/492: truncate de/d12/d7d/fa1 528872 0 2024-09-16T23:48:57.417 INFO:tasks.workunit.client.0.smithi042.stdout:1/426: dwrite d0/d9/f16 [4194304,4194304] 0 2024-09-16T23:48:57.417 INFO:tasks.workunit.client.0.smithi042.stdout:3/448: mknod d2/d1b/d1d/d30/c9c 0 2024-09-16T23:48:57.417 INFO:tasks.workunit.client.1.smithi043.stdout:9/475: dwrite d4/d21/f63 [0,4194304] 0 2024-09-16T23:48:57.418 INFO:tasks.workunit.client.1.smithi043.stdout:2/468: truncate d8/d1f/f5c 2216370 0 2024-09-16T23:48:57.419 INFO:tasks.workunit.client.1.smithi043.stdout:3/542: dread d4/de/d40/f45 [4194304,4194304] 0 2024-09-16T23:48:57.419 INFO:tasks.workunit.client.1.smithi043.stdout:0/540: truncate d0/d2/df/d5a/f70 503709 0 2024-09-16T23:48:57.420 INFO:tasks.workunit.client.1.smithi043.stdout:2/469: rmdir d8/d4b/d5f 39 2024-09-16T23:48:57.421 INFO:tasks.workunit.client.0.smithi042.stdout:1/427: chown d0/d9/d90/d66/c91 1987 1 2024-09-16T23:48:57.423 INFO:tasks.workunit.client.1.smithi043.stdout:9/476: creat d4/d1c/d4f/d45/f8f x:0 0 0 2024-09-16T23:48:57.425 INFO:tasks.workunit.client.0.smithi042.stdout:4/493: dwrite de/d42/d68/f69 [0,4194304] 0 2024-09-16T23:48:57.426 INFO:tasks.workunit.client.1.smithi043.stdout:3/543: unlink d4/de/d1f/d2c/f94 0 2024-09-16T23:48:57.431 INFO:tasks.workunit.client.0.smithi042.stdout:3/449: dread d2/d1b/f3b [0,4194304] 0 2024-09-16T23:48:57.431 INFO:tasks.workunit.client.1.smithi043.stdout:2/470: mkdir d8/d35/d9f 0 2024-09-16T23:48:57.432 INFO:tasks.workunit.client.1.smithi043.stdout:1/635: sync 2024-09-16T23:48:57.432 INFO:tasks.workunit.client.1.smithi043.stdout:4/567: sync 2024-09-16T23:48:57.433 INFO:tasks.workunit.client.1.smithi043.stdout:1/636: chown d4/d3e 0 1 2024-09-16T23:48:57.434 INFO:tasks.workunit.client.0.smithi042.stdout:1/428: fdatasync d0/d17/f21 0 2024-09-16T23:48:57.435 INFO:tasks.workunit.client.0.smithi042.stdout:4/494: getdents de/d42/d5d 0 2024-09-16T23:48:57.436 INFO:tasks.workunit.client.1.smithi043.stdout:0/541: mkdir d0/d2/df/d1a/d46/d39/daa/db7 0 2024-09-16T23:48:57.438 INFO:tasks.workunit.client.0.smithi042.stdout:4/495: mknod de/d42/cab 0 2024-09-16T23:48:57.439 INFO:tasks.workunit.client.0.smithi042.stdout:3/450: mkdir d2/d7/d61/d8e/d9d 0 2024-09-16T23:48:57.440 INFO:tasks.workunit.client.1.smithi043.stdout:7/514: write d4/db/d43/d83/d85/f78 [326224,14451] 0 2024-09-16T23:48:57.440 INFO:tasks.workunit.client.0.smithi042.stdout:1/429: symlink d0/d17/d79/d92/l97 0 2024-09-16T23:48:57.442 INFO:tasks.workunit.client.0.smithi042.stdout:3/451: write d2/d1b/f43 [1588368,105844] 0 2024-09-16T23:48:57.445 INFO:tasks.workunit.client.1.smithi043.stdout:5/727: write d3/d9/d22/d4a/f7f [468087,108302] 0 2024-09-16T23:48:57.447 INFO:tasks.workunit.client.0.smithi042.stdout:3/452: rename d2/d1b/d89/f31 to d2/d1b/d1d/d34/d62/f9e 0 2024-09-16T23:48:57.447 INFO:tasks.workunit.client.0.smithi042.stdout:1/430: mkdir d0/d9/d90/d6b/d88/d32/d2f/d98 0 2024-09-16T23:48:57.448 INFO:tasks.workunit.client.1.smithi043.stdout:9/477: mknod d4/d21/d2a/d2c/d7b/c90 0 2024-09-16T23:48:57.451 INFO:tasks.workunit.client.0.smithi042.stdout:3/453: unlink d2/d1b/f3b 0 2024-09-16T23:48:57.452 INFO:tasks.workunit.client.0.smithi042.stdout:1/431: fdatasync d0/d17/f36 0 2024-09-16T23:48:57.453 INFO:tasks.workunit.client.0.smithi042.stdout:3/454: write d2/d7/d61/f90 [127990,46365] 0 2024-09-16T23:48:57.453 INFO:tasks.workunit.client.0.smithi042.stdout:1/432: stat d0/d9/d90/d66/f7f 0 2024-09-16T23:48:57.455 INFO:tasks.workunit.client.0.smithi042.stdout:3/455: mknod d2/d1b/d1d/d30/c9f 0 2024-09-16T23:48:57.456 INFO:tasks.workunit.client.0.smithi042.stdout:4/496: dread de/f41 [0,4194304] 0 2024-09-16T23:48:57.457 INFO:tasks.workunit.client.1.smithi043.stdout:8/545: sync 2024-09-16T23:48:57.457 INFO:tasks.workunit.client.1.smithi043.stdout:6/519: sync 2024-09-16T23:48:57.459 INFO:tasks.workunit.client.1.smithi043.stdout:1/637: rename d4/d9/d8f/db6/dc6/la9 to d4/d9/d1c/d1e/d42/ldc 0 2024-09-16T23:48:57.460 INFO:tasks.workunit.client.1.smithi043.stdout:1/638: chown d4/d34/fc0 269 1 2024-09-16T23:48:57.463 INFO:tasks.workunit.client.1.smithi043.stdout:9/478: truncate d4/d1c/d4f/f52 2519640 0 2024-09-16T23:48:57.465 INFO:tasks.workunit.client.1.smithi043.stdout:3/544: link d4/de/d35/c84 d4/de/d40/d4d/c9f 0 2024-09-16T23:48:57.469 INFO:tasks.workunit.client.0.smithi042.stdout:3/456: truncate d2/d1b/d1d/d30/f79 50770 0 2024-09-16T23:48:57.469 INFO:tasks.workunit.client.0.smithi042.stdout:1/433: creat d0/d9/d26/d8b/f99 x:0 0 0 2024-09-16T23:48:57.469 INFO:tasks.workunit.client.0.smithi042.stdout:3/457: creat d2/d1b/d1d/fa0 x:0 0 0 2024-09-16T23:48:57.469 INFO:tasks.workunit.client.0.smithi042.stdout:4/497: dread de/f31 [0,4194304] 0 2024-09-16T23:48:57.471 INFO:tasks.workunit.client.1.smithi043.stdout:2/471: creat d8/d5e/d83/fa0 x:0 0 0 2024-09-16T23:48:57.471 INFO:tasks.workunit.client.1.smithi043.stdout:8/546: rmdir de/d18/d27/d41/d5a 39 2024-09-16T23:48:57.472 INFO:tasks.workunit.client.0.smithi042.stdout:4/498: unlink de/d12/d1b/l29 0 2024-09-16T23:48:57.474 INFO:tasks.workunit.client.0.smithi042.stdout:3/458: link d2/d1b/d1d/d34/d62/l67 d2/d7/d61/d8e/la1 0 2024-09-16T23:48:57.476 INFO:tasks.workunit.client.0.smithi042.stdout:1/434: mkdir d0/d9a 0 2024-09-16T23:48:57.477 INFO:tasks.workunit.client.1.smithi043.stdout:8/547: dwrite de/f45 [0,4194304] 0 2024-09-16T23:48:57.479 INFO:tasks.workunit.client.1.smithi043.stdout:1/639: chown d4/df/l2d 33177223 1 2024-09-16T23:48:57.479 INFO:tasks.workunit.client.0.smithi042.stdout:4/499: dwrite de/d2e/d37/f65 [0,4194304] 0 2024-09-16T23:48:57.483 INFO:tasks.workunit.client.1.smithi043.stdout:9/479: mkdir d4/d28/d5f/d91 0 2024-09-16T23:48:57.484 INFO:tasks.workunit.client.1.smithi043.stdout:3/545: creat d4/fa0 x:0 0 0 2024-09-16T23:48:57.485 INFO:tasks.workunit.client.0.smithi042.stdout:4/500: read de/d12/d7d/f83 [704022,28812] 0 2024-09-16T23:48:57.489 INFO:tasks.workunit.client.1.smithi043.stdout:2/472: mknod d8/d5e/d83/d3a/d77/ca1 0 2024-09-16T23:48:57.493 INFO:tasks.workunit.client.0.smithi042.stdout:4/501: mkdir de/d1a/d70/dac 0 2024-09-16T23:48:57.494 INFO:tasks.workunit.client.1.smithi043.stdout:1/640: symlink d4/df/d26/d79/ldd 0 2024-09-16T23:48:57.494 INFO:tasks.workunit.client.0.smithi042.stdout:3/459: truncate d2/d9/f22 3247898 0 2024-09-16T23:48:57.496 INFO:tasks.workunit.client.1.smithi043.stdout:9/480: truncate d4/d1c/f20 2938786 0 2024-09-16T23:48:57.496 INFO:tasks.workunit.client.0.smithi042.stdout:3/460: write d2/d1b/d1d/d34/f5d [3666284,79193] 0 2024-09-16T23:48:57.497 INFO:tasks.workunit.client.0.smithi042.stdout:3/461: fsync d2/d1b/f43 0 2024-09-16T23:48:57.498 INFO:tasks.workunit.client.0.smithi042.stdout:4/502: mknod de/d42/d5d/d8f/d9f/cad 0 2024-09-16T23:48:57.499 INFO:tasks.workunit.client.1.smithi043.stdout:4/568: rename d8/d15/d20/d2b/d54/d55/d49 to d8/d15/d20/d2b/d54/db3 0 2024-09-16T23:48:57.502 INFO:tasks.workunit.client.0.smithi042.stdout:4/503: mkdir de/d2e/d37/d3e/dae 0 2024-09-16T23:48:57.502 INFO:tasks.workunit.client.0.smithi042.stdout:1/435: dwrite d0/d17/d18/f73 [0,4194304] 0 2024-09-16T23:48:57.505 INFO:tasks.workunit.client.0.smithi042.stdout:4/504: rmdir de/d1a 39 2024-09-16T23:48:57.508 INFO:tasks.workunit.client.1.smithi043.stdout:4/569: dwrite d8/d15/d20/d2b/d23/d27/d4b/fa4 [0,4194304] 0 2024-09-16T23:48:57.510 INFO:tasks.workunit.client.0.smithi042.stdout:4/505: stat de/d2e/d37/l5e 0 2024-09-16T23:48:57.512 INFO:tasks.workunit.client.0.smithi042.stdout:1/436: dread d0/d9/d90/d6b/d88/f75 [0,4194304] 0 2024-09-16T23:48:57.516 INFO:tasks.workunit.client.0.smithi042.stdout:4/506: chown de/d42/f44 33 1 2024-09-16T23:48:57.517 INFO:tasks.workunit.client.0.smithi042.stdout:3/462: dwrite d2/d1b/d89/f42 [0,4194304] 0 2024-09-16T23:48:57.521 INFO:tasks.workunit.client.0.smithi042.stdout:3/463: chown d2/d1b/d1d/f38 5 1 2024-09-16T23:48:57.527 INFO:tasks.workunit.client.0.smithi042.stdout:4/507: dread de/d2e/d37/f3b [0,4194304] 0 2024-09-16T23:48:57.527 INFO:tasks.workunit.client.1.smithi043.stdout:4/570: dread d8/d15/d20/d2b/f8c [0,4194304] 0 2024-09-16T23:48:57.529 INFO:tasks.workunit.client.1.smithi043.stdout:4/571: write d8/d15/d20/d2b/d22/f24 [890631,114842] 0 2024-09-16T23:48:57.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/508: dwrite de/d2e/d37/d3c/f9a [0,4194304] 0 2024-09-16T23:48:57.534 INFO:tasks.workunit.client.0.smithi042.stdout:4/509: readlink l5 0 2024-09-16T23:48:57.535 INFO:tasks.workunit.client.0.smithi042.stdout:3/464: fsync d2/d1b/d89/f42 0 2024-09-16T23:48:57.537 INFO:tasks.workunit.client.1.smithi043.stdout:4/572: dwrite d8/d15/d20/d2b/f8c [0,4194304] 0 2024-09-16T23:48:57.538 INFO:tasks.workunit.client.0.smithi042.stdout:3/465: mkdir d2/d1b/d1d/d30/da2 0 2024-09-16T23:48:57.539 INFO:tasks.workunit.client.0.smithi042.stdout:3/466: write d2/d1b/d1d/f38 [911411,87444] 0 2024-09-16T23:48:57.541 INFO:tasks.workunit.client.0.smithi042.stdout:4/510: getdents de/d42/d68 0 2024-09-16T23:48:57.543 INFO:tasks.workunit.client.0.smithi042.stdout:3/467: symlink d2/d7/d61/d8e/d7f/la3 0 2024-09-16T23:48:57.545 INFO:tasks.workunit.client.0.smithi042.stdout:4/511: mknod de/d42/d99/caf 0 2024-09-16T23:48:57.548 INFO:tasks.workunit.client.0.smithi042.stdout:3/468: link d2/d1b/d89/f5c d2/d7/fa4 0 2024-09-16T23:48:57.550 INFO:tasks.workunit.client.0.smithi042.stdout:3/469: creat d2/d1b/d1d/d34/d62/d6a/fa5 x:0 0 0 2024-09-16T23:48:57.552 INFO:tasks.workunit.client.0.smithi042.stdout:3/470: rmdir d2/d1b/d1d/d30 39 2024-09-16T23:48:57.553 INFO:tasks.workunit.client.0.smithi042.stdout:3/471: chown d2/d7/d61/d8e/d7f/la3 136837308 1 2024-09-16T23:48:57.556 INFO:tasks.workunit.client.0.smithi042.stdout:3/472: mknod d2/d7/d61/d8e/d9d/ca6 0 2024-09-16T23:48:57.556 INFO:tasks.workunit.client.0.smithi042.stdout:3/473: chown d2/d1b/f75 34413 1 2024-09-16T23:48:57.598 INFO:tasks.workunit.client.1.smithi043.stdout:5/728: getdents d3/d79/d91 0 2024-09-16T23:48:57.599 INFO:tasks.workunit.client.1.smithi043.stdout:5/729: chown d3/db/d19/d76/dba 4920 1 2024-09-16T23:48:57.599 INFO:tasks.workunit.client.1.smithi043.stdout:1/641: unlink d4/df/d26/l2c 0 2024-09-16T23:48:57.601 INFO:tasks.workunit.client.1.smithi043.stdout:2/473: write d8/d4b/f5b [1492111,26605] 0 2024-09-16T23:48:57.603 INFO:tasks.workunit.client.1.smithi043.stdout:3/546: read d4/de/d1f/f29 [1753548,117807] 0 2024-09-16T23:48:57.603 INFO:tasks.workunit.client.1.smithi043.stdout:9/481: creat d4/d5/f92 x:0 0 0 2024-09-16T23:48:57.606 INFO:tasks.workunit.client.1.smithi043.stdout:0/542: rename d0/d2/dd to d0/d2/df/da7/db8 0 2024-09-16T23:48:57.607 INFO:tasks.workunit.client.1.smithi043.stdout:0/543: read d0/d2/df/d1a/d46/d39/d6e/f32 [1668949,92328] 0 2024-09-16T23:48:57.608 INFO:tasks.workunit.client.0.smithi042.stdout:1/437: write d0/d17/f58 [2084828,127659] 0 2024-09-16T23:48:57.609 INFO:tasks.workunit.client.1.smithi043.stdout:0/544: stat d0/d2/df/d1a/cb3 0 2024-09-16T23:48:57.612 INFO:tasks.workunit.client.1.smithi043.stdout:4/573: creat d8/d25/d64/fb4 x:0 0 0 2024-09-16T23:48:57.613 INFO:tasks.workunit.client.0.smithi042.stdout:1/438: dread d0/d17/d79/f7c [0,4194304] 0 2024-09-16T23:48:57.614 INFO:tasks.workunit.client.0.smithi042.stdout:1/439: write d0/d17/f2d [1348223,10366] 0 2024-09-16T23:48:57.615 INFO:tasks.workunit.client.1.smithi043.stdout:5/730: rmdir d3/d9/d6f/d7c 39 2024-09-16T23:48:57.618 INFO:tasks.workunit.client.1.smithi043.stdout:1/642: creat d4/d3e/d95/fde x:0 0 0 2024-09-16T23:48:57.618 INFO:tasks.workunit.client.1.smithi043.stdout:1/643: stat d4/l9e 0 2024-09-16T23:48:57.618 INFO:tasks.workunit.client.1.smithi043.stdout:6/520: sync 2024-09-16T23:48:57.620 INFO:tasks.workunit.client.1.smithi043.stdout:6/521: write de/d23/d34/d9b/f88 [3787121,71525] 0 2024-09-16T23:48:57.620 INFO:tasks.workunit.client.0.smithi042.stdout:1/440: dwrite d0/d9/d90/d6b/d88/f12 [4194304,4194304] 0 2024-09-16T23:48:57.622 INFO:tasks.workunit.client.1.smithi043.stdout:2/474: mkdir d8/d1d/d20/d23/d38/da2 0 2024-09-16T23:48:57.625 INFO:tasks.workunit.client.1.smithi043.stdout:3/547: mknod d4/de/d5d/ca1 0 2024-09-16T23:48:57.628 INFO:tasks.workunit.client.0.smithi042.stdout:1/441: truncate d0/d9/f56 520954 0 2024-09-16T23:48:57.630 INFO:tasks.workunit.client.1.smithi043.stdout:9/482: unlink d4/d1c/d4f/d45/f8f 0 2024-09-16T23:48:57.631 INFO:tasks.workunit.client.1.smithi043.stdout:5/731: dread d3/db/d19/d3e/d28/f37 [0,4194304] 0 2024-09-16T23:48:57.632 INFO:tasks.workunit.client.1.smithi043.stdout:9/483: write d4/d21/d2a/f7a [924098,13959] 0 2024-09-16T23:48:57.632 INFO:tasks.workunit.client.0.smithi042.stdout:1/442: dread d0/d9/d90/d6b/d88/f2a [4194304,4194304] 0 2024-09-16T23:48:57.633 INFO:tasks.workunit.client.1.smithi043.stdout:5/732: write d3/d9/d22/f4e [2034958,36074] 0 2024-09-16T23:48:57.634 INFO:tasks.workunit.client.1.smithi043.stdout:0/545: symlink d0/d2/df/d1a/d3b/lb9 0 2024-09-16T23:48:57.636 INFO:tasks.workunit.client.1.smithi043.stdout:1/644: dread d4/d9/d12/d2b/f52 [4194304,4194304] 0 2024-09-16T23:48:57.636 INFO:tasks.workunit.client.1.smithi043.stdout:0/546: chown d0/d2/df/d1a/d46/d39/daa 34 1 2024-09-16T23:48:57.638 INFO:tasks.workunit.client.1.smithi043.stdout:1/645: write d4/d9/d12/d2b/d3d/d83/d98/fb8 [2304053,79511] 0 2024-09-16T23:48:57.639 INFO:tasks.workunit.client.1.smithi043.stdout:1/646: dread - d4/d9/d1c/d1e/d42/d5d/f9a zero size 2024-09-16T23:48:57.639 INFO:tasks.workunit.client.1.smithi043.stdout:4/574: mknod d8/d15/d20/cb5 0 2024-09-16T23:48:57.655 INFO:tasks.workunit.client.1.smithi043.stdout:1/647: dread - d4/df/f91 zero size 2024-09-16T23:48:57.657 INFO:tasks.workunit.client.1.smithi043.stdout:4/575: truncate d8/d15/d9b/d8b/fa8 8747605 0 2024-09-16T23:48:57.657 INFO:tasks.workunit.client.1.smithi043.stdout:1/648: chown d4/d9/d12/d2b/d3d 7308053 1 2024-09-16T23:48:57.657 INFO:tasks.workunit.client.1.smithi043.stdout:3/548: dread d4/fb [4194304,4194304] 0 2024-09-16T23:48:57.658 INFO:tasks.workunit.client.1.smithi043.stdout:4/576: readlink d8/d15/d20/d2b/d54/db3/d4c/l8e 0 2024-09-16T23:48:57.659 INFO:tasks.workunit.client.1.smithi043.stdout:2/475: mknod d8/d1d/d20/d23/d38/da2/ca3 0 2024-09-16T23:48:57.660 INFO:tasks.workunit.client.1.smithi043.stdout:2/476: chown d8/d5e/d83/l8f 83 1 2024-09-16T23:48:57.660 INFO:tasks.workunit.client.1.smithi043.stdout:2/477: chown d8/d11/f32 8182 1 2024-09-16T23:48:57.661 INFO:tasks.workunit.client.1.smithi043.stdout:2/478: write d8/d1d/f2a [3462766,116716] 0 2024-09-16T23:48:57.662 INFO:tasks.workunit.client.0.smithi042.stdout:1/443: dwrite d0/d17/f36 [0,4194304] 0 2024-09-16T23:48:57.663 INFO:tasks.workunit.client.1.smithi043.stdout:2/479: read - d8/d5e/d83/fa0 zero size 2024-09-16T23:48:57.664 INFO:tasks.workunit.client.1.smithi043.stdout:2/480: chown d8/d4b/d4f 794 1 2024-09-16T23:48:57.664 INFO:tasks.workunit.client.1.smithi043.stdout:7/515: rename d4/db/d11/d21/d31/f88 to d4/fab 0 2024-09-16T23:48:57.666 INFO:tasks.workunit.client.1.smithi043.stdout:0/547: dread d0/d2/df/f18 [0,4194304] 0 2024-09-16T23:48:57.667 INFO:tasks.workunit.client.1.smithi043.stdout:0/548: chown d0/d2/df/d5a/d9c/l86 3807 1 2024-09-16T23:48:57.669 INFO:tasks.workunit.client.1.smithi043.stdout:5/733: creat d3/d9/d22/d4a/d4c/ded/ff6 x:0 0 0 2024-09-16T23:48:57.671 INFO:tasks.workunit.client.1.smithi043.stdout:2/481: dread d8/d1d/d20/d23/d38/f47 [0,4194304] 0 2024-09-16T23:48:57.672 INFO:tasks.workunit.client.1.smithi043.stdout:3/549: fsync d4/fb 0 2024-09-16T23:48:57.674 INFO:tasks.workunit.client.1.smithi043.stdout:2/482: write d8/d5e/d83/f55 [3803126,130317] 0 2024-09-16T23:48:57.676 INFO:tasks.workunit.client.1.smithi043.stdout:6/522: mknod de/d1a/cab 0 2024-09-16T23:48:57.678 INFO:tasks.workunit.client.1.smithi043.stdout:5/734: dwrite d3/d79/d91/daf/fc2 [0,4194304] 0 2024-09-16T23:48:57.679 INFO:tasks.workunit.client.0.smithi042.stdout:1/444: creat d0/d9/f9b x:0 0 0 2024-09-16T23:48:57.679 INFO:tasks.workunit.client.1.smithi043.stdout:5/735: chown d3/d9/d22/d4a/fb1 201 1 2024-09-16T23:48:57.682 INFO:tasks.workunit.client.0.smithi042.stdout:1/445: dread d0/d17/f2d [0,4194304] 0 2024-09-16T23:48:57.683 INFO:tasks.workunit.client.1.smithi043.stdout:7/516: mknod d4/db/d43/cac 0 2024-09-16T23:48:57.686 INFO:tasks.workunit.client.1.smithi043.stdout:1/649: mkdir d4/df/d13/ddf 0 2024-09-16T23:48:57.689 INFO:tasks.workunit.client.1.smithi043.stdout:6/523: rename de/d23/d34/d66/f93 to de/d23/d34/d66/fac 0 2024-09-16T23:48:57.691 INFO:tasks.workunit.client.1.smithi043.stdout:5/736: creat d3/db/d19/d3e/d28/dcb/ff7 x:0 0 0 2024-09-16T23:48:57.691 INFO:tasks.workunit.client.1.smithi043.stdout:5/737: dread - d3/d1b/fa6 zero size 2024-09-16T23:48:57.693 INFO:tasks.workunit.client.1.smithi043.stdout:8/548: rename de/d18/d1f/d2f/l55 to de/d18/d1f/d2f/d39/d7b/d83/lab 0 2024-09-16T23:48:57.694 INFO:tasks.workunit.client.1.smithi043.stdout:7/517: fsync d4/db/d11/d15/d29/d7a/f84 0 2024-09-16T23:48:57.697 INFO:tasks.workunit.client.1.smithi043.stdout:0/549: write d0/d2/df/d1a/d46/d39/f61 [173854,123879] 0 2024-09-16T23:48:57.699 INFO:tasks.workunit.client.1.smithi043.stdout:2/483: link d8/d4b/d4f/f93 d8/d1d/d25/fa4 0 2024-09-16T23:48:57.700 INFO:tasks.workunit.client.1.smithi043.stdout:2/484: chown d8/d4b/d4f/f93 4858 1 2024-09-16T23:48:57.702 INFO:tasks.workunit.client.0.smithi042.stdout:1/446: dwrite d0/d9/fe [0,4194304] 0 2024-09-16T23:48:57.706 INFO:tasks.workunit.client.1.smithi043.stdout:2/485: dwrite d8/d1d/d20/d23/d38/f66 [0,4194304] 0 2024-09-16T23:48:57.706 INFO:tasks.workunit.client.1.smithi043.stdout:2/486: readlink d8/l7f 0 2024-09-16T23:48:57.707 INFO:tasks.workunit.client.0.smithi042.stdout:1/447: unlink d0/d17/l4c 0 2024-09-16T23:48:57.708 INFO:tasks.workunit.client.1.smithi043.stdout:2/487: read d8/d1d/d20/d23/d38/f66 [2676691,62556] 0 2024-09-16T23:48:57.711 INFO:tasks.workunit.client.1.smithi043.stdout:3/550: creat d4/de/fa2 x:0 0 0 2024-09-16T23:48:57.711 INFO:tasks.workunit.client.0.smithi042.stdout:1/448: dread d0/d9/d90/d6b/d88/f2a [4194304,4194304] 0 2024-09-16T23:48:57.713 INFO:tasks.workunit.client.0.smithi042.stdout:1/449: dread d0/d9/d90/d6b/d88/f2b [0,4194304] 0 2024-09-16T23:48:57.714 INFO:tasks.workunit.client.1.smithi043.stdout:4/577: getdents d8/d15/d20/d2b/d23/d27/d4b 0 2024-09-16T23:48:57.723 INFO:tasks.workunit.client.1.smithi043.stdout:6/524: truncate de/d23/d34/d66/d70/f76 609035 0 2024-09-16T23:48:57.724 INFO:tasks.workunit.client.1.smithi043.stdout:0/550: symlink d0/d2/df/lba 0 2024-09-16T23:48:57.725 INFO:tasks.workunit.client.1.smithi043.stdout:8/549: dwrite de/d18/d27/d41/d5a/d37/f60 [4194304,4194304] 0 2024-09-16T23:48:57.726 INFO:tasks.workunit.client.1.smithi043.stdout:0/551: chown d0/d2/df/d1a/d46/d39/daa 109 1 2024-09-16T23:48:57.727 INFO:tasks.workunit.client.1.smithi043.stdout:0/552: dread - d0/d2/df/d1a/d49/d71/f8f zero size 2024-09-16T23:48:57.728 INFO:tasks.workunit.client.1.smithi043.stdout:9/484: rename d4/d1c/d6b/f57 to d4/d1c/d25/d2b/d4c/f93 0 2024-09-16T23:48:57.729 INFO:tasks.workunit.client.0.smithi042.stdout:1/450: dwrite d0/d17/d79/f7c [0,4194304] 0 2024-09-16T23:48:57.731 INFO:tasks.workunit.client.1.smithi043.stdout:3/551: write d4/de/d40/f69 [2798795,66556] 0 2024-09-16T23:48:57.732 INFO:tasks.workunit.client.1.smithi043.stdout:2/488: creat d8/d5e/d83/d9e/fa5 x:0 0 0 2024-09-16T23:48:57.732 INFO:tasks.workunit.client.1.smithi043.stdout:2/489: stat d8/d4b 0 2024-09-16T23:48:57.734 INFO:tasks.workunit.client.0.smithi042.stdout:1/451: mknod d0/d9/d26/c9c 0 2024-09-16T23:48:57.739 INFO:tasks.workunit.client.1.smithi043.stdout:8/550: fdatasync de/d18/d27/f94 0 2024-09-16T23:48:57.740 INFO:tasks.workunit.client.1.smithi043.stdout:8/551: fsync de/d18/d1f/f32 0 2024-09-16T23:48:57.743 INFO:tasks.workunit.client.1.smithi043.stdout:9/485: symlink d4/d28/d5f/l94 0 2024-09-16T23:48:57.745 INFO:tasks.workunit.client.1.smithi043.stdout:3/552: creat d4/de/d5d/fa3 x:0 0 0 2024-09-16T23:48:57.745 INFO:tasks.workunit.client.1.smithi043.stdout:7/518: getdents d4/d27/d8a 0 2024-09-16T23:48:57.746 INFO:tasks.workunit.client.1.smithi043.stdout:3/553: chown d4/de/d40/d4d/f78 2774020 1 2024-09-16T23:48:57.746 INFO:tasks.workunit.client.1.smithi043.stdout:7/519: dread - d4/db/d43/d83/d85/f92 zero size 2024-09-16T23:48:57.748 INFO:tasks.workunit.client.0.smithi042.stdout:1/452: truncate d0/d9/d90/d6b/d88/f12 610815 0 2024-09-16T23:48:57.750 INFO:tasks.workunit.client.0.smithi042.stdout:1/453: dread d0/d17/f2d [0,4194304] 0 2024-09-16T23:48:57.751 INFO:tasks.workunit.client.1.smithi043.stdout:9/486: mkdir d4/d21/d33/d95 0 2024-09-16T23:48:57.754 INFO:tasks.workunit.client.1.smithi043.stdout:3/554: dread d4/de/d1f/f79 [0,4194304] 0 2024-09-16T23:48:57.754 INFO:tasks.workunit.client.1.smithi043.stdout:6/525: creat de/d1a/fad x:0 0 0 2024-09-16T23:48:57.754 INFO:tasks.workunit.client.1.smithi043.stdout:7/520: dread d4/db/f73 [0,4194304] 0 2024-09-16T23:48:57.755 INFO:tasks.workunit.client.1.smithi043.stdout:8/552: dwrite de/d18/d1f/f59 [0,4194304] 0 2024-09-16T23:48:57.755 INFO:tasks.workunit.client.1.smithi043.stdout:6/526: dread - de/d1a/d28/fa5 zero size 2024-09-16T23:48:57.756 INFO:tasks.workunit.client.1.smithi043.stdout:3/555: chown d4/de/d1f/f29 1894 1 2024-09-16T23:48:57.756 INFO:tasks.workunit.client.1.smithi043.stdout:4/578: link d8/d15/d20/d2b/c87 d8/d15/d20/d2b/d23/d27/cb6 0 2024-09-16T23:48:57.758 INFO:tasks.workunit.client.0.smithi042.stdout:1/454: fsync d0/d9/f56 0 2024-09-16T23:48:57.761 INFO:tasks.workunit.client.1.smithi043.stdout:0/553: rename d0/dc to d0/d2/df/dbb 0 2024-09-16T23:48:57.761 INFO:tasks.workunit.client.1.smithi043.stdout:0/554: readlink d0/de/l79 0 2024-09-16T23:48:57.766 INFO:tasks.workunit.client.1.smithi043.stdout:2/490: write d8/d1f/f5c [1362432,91097] 0 2024-09-16T23:48:57.767 INFO:tasks.workunit.client.0.smithi042.stdout:1/455: mknod d0/d9/d90/d6b/d88/d32/d2f/c9d 0 2024-09-16T23:48:57.770 INFO:tasks.workunit.client.1.smithi043.stdout:0/555: dwrite d0/d2/df/dbb/f50 [0,4194304] 0 2024-09-16T23:48:57.771 INFO:tasks.workunit.client.0.smithi042.stdout:1/456: creat d0/d17/f9e x:0 0 0 2024-09-16T23:48:57.772 INFO:tasks.workunit.client.0.smithi042.stdout:1/457: stat d0/d3b/l64 0 2024-09-16T23:48:57.773 INFO:tasks.workunit.client.1.smithi043.stdout:0/556: read d0/d2/df/d1a/d3b/f6c [1501893,105030] 0 2024-09-16T23:48:57.774 INFO:tasks.workunit.client.1.smithi043.stdout:2/491: dread d8/d1d/d20/d23/d38/f66 [0,4194304] 0 2024-09-16T23:48:57.774 INFO:tasks.workunit.client.1.smithi043.stdout:6/527: chown de/d1a/d28/l91 7463825 1 2024-09-16T23:48:57.779 INFO:tasks.workunit.client.1.smithi043.stdout:8/553: creat de/d18/d27/d41/d5a/fac x:0 0 0 2024-09-16T23:48:57.779 INFO:tasks.workunit.client.1.smithi043.stdout:4/579: mknod d8/d15/d6e/cb7 0 2024-09-16T23:48:57.781 INFO:tasks.workunit.client.0.smithi042.stdout:1/458: mknod d0/c9f 0 2024-09-16T23:48:57.782 INFO:tasks.workunit.client.0.smithi042.stdout:1/459: chown d0/d3b/f82 1749010 1 2024-09-16T23:48:57.783 INFO:tasks.workunit.client.1.smithi043.stdout:9/487: unlink d4/d1c/d4f/f52 0 2024-09-16T23:48:57.785 INFO:tasks.workunit.client.1.smithi043.stdout:7/521: mknod d4/d27/d41/d1b/d75/cad 0 2024-09-16T23:48:57.786 INFO:tasks.workunit.client.1.smithi043.stdout:3/556: symlink d4/de/d1f/la4 0 2024-09-16T23:48:57.788 INFO:tasks.workunit.client.1.smithi043.stdout:6/528: dread de/d1a/f42 [0,4194304] 0 2024-09-16T23:48:57.788 INFO:tasks.workunit.client.1.smithi043.stdout:0/557: unlink d0/de/l79 0 2024-09-16T23:48:57.789 INFO:tasks.workunit.client.1.smithi043.stdout:6/529: dread - de/d23/d63/f96 zero size 2024-09-16T23:48:57.792 INFO:tasks.workunit.client.1.smithi043.stdout:2/492: unlink d8/d4b/d4f/f93 0 2024-09-16T23:48:57.794 INFO:tasks.workunit.client.1.smithi043.stdout:6/530: dwrite de/d23/d34/d9b/d5a/fa7 [0,4194304] 0 2024-09-16T23:48:57.794 INFO:tasks.workunit.client.0.smithi042.stdout:1/460: getdents d0/d9/d90/d6b/d88/d32 0 2024-09-16T23:48:57.795 INFO:tasks.workunit.client.1.smithi043.stdout:6/531: fsync de/d23/d49/faa 0 2024-09-16T23:48:57.802 INFO:tasks.workunit.client.0.smithi042.stdout:1/461: link d0/d9/d90/d6b/d88/f1e d0/d9/d90/d6b/d88/d32/d2f/fa0 0 2024-09-16T23:48:57.810 INFO:tasks.workunit.client.0.smithi042.stdout:1/462: creat d0/fa1 x:0 0 0 2024-09-16T23:48:57.811 INFO:tasks.workunit.client.0.smithi042.stdout:1/463: chown d0/d17/d79/d92 626 1 2024-09-16T23:48:57.816 INFO:tasks.workunit.client.1.smithi043.stdout:4/580: rename d8/d15/d9b/f88 to d8/d25/d39/d58/fb8 0 2024-09-16T23:48:57.817 INFO:tasks.workunit.client.1.smithi043.stdout:9/488: rmdir d4/d28/d49 39 2024-09-16T23:48:57.817 INFO:tasks.workunit.client.1.smithi043.stdout:9/489: fsync d4/d21/d35/d4b/f80 0 2024-09-16T23:48:57.823 INFO:tasks.workunit.client.1.smithi043.stdout:2/493: unlink d8/d1d/f2a 0 2024-09-16T23:48:57.830 INFO:tasks.workunit.client.0.smithi042.stdout:7/429: sync 2024-09-16T23:48:57.830 INFO:tasks.workunit.client.0.smithi042.stdout:9/264: sync 2024-09-16T23:48:57.835 INFO:tasks.workunit.client.0.smithi042.stdout:1/464: link d0/d9/d90/d6b/d88/f12 d0/d17/d79/fa2 0 2024-09-16T23:48:57.835 INFO:tasks.workunit.client.0.smithi042.stdout:7/430: link d3/d15/d41/f47 d3/d1f/d49/f97 0 2024-09-16T23:48:57.838 INFO:tasks.workunit.client.0.smithi042.stdout:9/265: rmdir d4 39 2024-09-16T23:48:57.839 INFO:tasks.workunit.client.1.smithi043.stdout:2/494: write d8/d4b/d4f/d56/f95 [725361,4320] 0 2024-09-16T23:48:57.841 INFO:tasks.workunit.client.0.smithi042.stdout:7/431: rename d3/d15/d24/d4d to d3/d98 0 2024-09-16T23:48:57.841 INFO:tasks.workunit.client.1.smithi043.stdout:4/581: dread d8/d15/d9b/d8b/fa8 [4194304,4194304] 0 2024-09-16T23:48:57.841 INFO:tasks.workunit.client.1.smithi043.stdout:6/532: mkdir de/d23/d34/d66/d9d/dae 0 2024-09-16T23:48:57.842 INFO:tasks.workunit.client.1.smithi043.stdout:4/582: fdatasync d8/d15/f21 0 2024-09-16T23:48:57.842 INFO:tasks.workunit.client.0.smithi042.stdout:1/465: creat d0/d3b/d8d/fa3 x:0 0 0 2024-09-16T23:48:57.843 INFO:tasks.workunit.client.1.smithi043.stdout:8/554: creat de/d18/d1f/d2f/d39/fad x:0 0 0 2024-09-16T23:48:57.844 INFO:tasks.workunit.client.0.smithi042.stdout:1/466: write d0/d3b/d8d/fa3 [498869,77775] 0 2024-09-16T23:48:57.844 INFO:tasks.workunit.client.1.smithi043.stdout:7/522: link d4/db/d28/f61 d4/db/d11/d21/fae 0 2024-09-16T23:48:57.845 INFO:tasks.workunit.client.1.smithi043.stdout:9/490: getdents d4/d1c/d4f/d45/d85 0 2024-09-16T23:48:57.849 INFO:tasks.workunit.client.1.smithi043.stdout:6/533: dwrite f2 [0,4194304] 0 2024-09-16T23:48:57.851 INFO:tasks.workunit.client.0.smithi042.stdout:1/467: dwrite d0/d17/f36 [0,4194304] 0 2024-09-16T23:48:57.851 INFO:tasks.workunit.client.1.smithi043.stdout:2/495: creat d8/d11/d16/fa6 x:0 0 0 2024-09-16T23:48:57.854 INFO:tasks.workunit.client.0.smithi042.stdout:7/432: link d3/d15/d24/l4c d3/d1f/d4f/l99 0 2024-09-16T23:48:57.855 INFO:tasks.workunit.client.0.smithi042.stdout:7/433: chown d3/d15/d41/f8b 3744 1 2024-09-16T23:48:57.856 INFO:tasks.workunit.client.0.smithi042.stdout:9/266: dread - d4/d10/f30 zero size 2024-09-16T23:48:57.856 INFO:tasks.workunit.client.0.smithi042.stdout:7/434: fdatasync d3/d15/d41/f45 0 2024-09-16T23:48:57.857 INFO:tasks.workunit.client.1.smithi043.stdout:4/583: truncate d8/f1d 980766 0 2024-09-16T23:48:57.857 INFO:tasks.workunit.client.0.smithi042.stdout:7/435: stat d3/d15/c42 0 2024-09-16T23:48:57.859 INFO:tasks.workunit.client.0.smithi042.stdout:7/436: write d3/d15/d41/f8b [663313,86836] 0 2024-09-16T23:48:57.860 INFO:tasks.workunit.client.0.smithi042.stdout:7/437: write d3/f30 [434596,110207] 0 2024-09-16T23:48:57.863 INFO:tasks.workunit.client.1.smithi043.stdout:3/557: dwrite d4/fb [0,4194304] 0 2024-09-16T23:48:57.863 INFO:tasks.workunit.client.1.smithi043.stdout:8/555: rmdir de/d18/d27/d41/d5a 39 2024-09-16T23:48:57.864 INFO:tasks.workunit.client.0.smithi042.stdout:7/438: write d3/d15/d32/d6b/f60 [422189,26762] 0 2024-09-16T23:48:57.864 INFO:tasks.workunit.client.1.smithi043.stdout:0/558: dwrite d0/d2/df/d1a/d3b/f7d [0,4194304] 0 2024-09-16T23:48:57.866 INFO:tasks.workunit.client.1.smithi043.stdout:0/559: dread - d0/d2/df/fad zero size 2024-09-16T23:48:57.871 INFO:tasks.workunit.client.1.smithi043.stdout:8/556: readlink de/d70/l5c 0 2024-09-16T23:48:57.871 INFO:tasks.workunit.client.1.smithi043.stdout:8/557: chown de/d18/d1f/d25/f6b 32726 1 2024-09-16T23:48:57.872 INFO:tasks.workunit.client.1.smithi043.stdout:8/558: fsync de/d18/d27/d8e/f7e 0 2024-09-16T23:48:57.872 INFO:tasks.workunit.client.0.smithi042.stdout:9/267: dread d4/d12/d2d/f32 [0,4194304] 0 2024-09-16T23:48:57.872 INFO:tasks.workunit.client.1.smithi043.stdout:8/559: write de/d18/d1f/d25/f3b [1960638,38848] 0 2024-09-16T23:48:57.876 INFO:tasks.workunit.client.1.smithi043.stdout:7/523: creat d4/db/d43/d83/d85/d8b/faf x:0 0 0 2024-09-16T23:48:57.880 INFO:tasks.workunit.client.1.smithi043.stdout:7/524: dread d4/db/d11/d21/f2f [0,4194304] 0 2024-09-16T23:48:57.881 INFO:tasks.workunit.client.1.smithi043.stdout:9/491: creat d4/d28/d5f/d86/f96 x:0 0 0 2024-09-16T23:48:57.882 INFO:tasks.workunit.client.0.smithi042.stdout:9/268: dwrite d4/d10/f15 [0,4194304] 0 2024-09-16T23:48:57.886 INFO:tasks.workunit.client.0.smithi042.stdout:1/468: dread d0/d17/d79/fa2 [0,4194304] 0 2024-09-16T23:48:57.886 INFO:tasks.workunit.client.1.smithi043.stdout:5/738: sync 2024-09-16T23:48:57.886 INFO:tasks.workunit.client.1.smithi043.stdout:1/650: sync 2024-09-16T23:48:57.887 INFO:tasks.workunit.client.0.smithi042.stdout:9/269: read d4/d10/f1b [2517942,71809] 0 2024-09-16T23:48:57.887 INFO:tasks.workunit.client.1.smithi043.stdout:4/584: dread d8/d25/f42 [4194304,4194304] 0 2024-09-16T23:48:57.888 INFO:tasks.workunit.client.1.smithi043.stdout:1/651: write d4/d9/d12/d2b/fb0 [2227272,18392] 0 2024-09-16T23:48:57.889 INFO:tasks.workunit.client.0.smithi042.stdout:7/439: dwrite d3/d15/d41/f47 [0,4194304] 0 2024-09-16T23:48:57.889 INFO:tasks.workunit.client.0.smithi042.stdout:8/316: sync 2024-09-16T23:48:57.889 INFO:tasks.workunit.client.0.smithi042.stdout:5/465: sync 2024-09-16T23:48:57.889 INFO:tasks.workunit.client.0.smithi042.stdout:0/296: sync 2024-09-16T23:48:57.889 INFO:tasks.workunit.client.0.smithi042.stdout:6/354: sync 2024-09-16T23:48:57.889 INFO:tasks.workunit.client.0.smithi042.stdout:3/474: sync 2024-09-16T23:48:57.889 INFO:tasks.workunit.client.0.smithi042.stdout:2/431: sync 2024-09-16T23:48:57.890 INFO:tasks.workunit.client.0.smithi042.stdout:4/512: sync 2024-09-16T23:48:57.890 INFO:tasks.workunit.client.0.smithi042.stdout:9/270: write d4/fb [3494383,99900] 0 2024-09-16T23:48:57.895 INFO:tasks.workunit.client.0.smithi042.stdout:3/475: chown d2/d1b/d89/l53 6 1 2024-09-16T23:48:57.896 INFO:tasks.workunit.client.0.smithi042.stdout:7/440: chown d3/d15/d24/d28/f3e 3 1 2024-09-16T23:48:57.896 INFO:tasks.workunit.client.0.smithi042.stdout:2/432: read - d0/d5d/f82 zero size 2024-09-16T23:48:57.898 INFO:tasks.workunit.client.0.smithi042.stdout:4/513: mkdir de/d12/db0 0 2024-09-16T23:48:57.898 INFO:tasks.workunit.client.0.smithi042.stdout:1/469: creat d0/d9/d4e/fa4 x:0 0 0 2024-09-16T23:48:57.900 INFO:tasks.workunit.client.1.smithi043.stdout:0/560: creat d0/d2/df/d1a/d46/d39/d6e/fbc x:0 0 0 2024-09-16T23:48:57.903 INFO:tasks.workunit.client.0.smithi042.stdout:8/317: rmdir da/d19/d1d/d30 39 2024-09-16T23:48:57.905 INFO:tasks.workunit.client.0.smithi042.stdout:4/514: dread de/d12/d7d/fa1 [0,4194304] 0 2024-09-16T23:48:57.909 INFO:tasks.workunit.client.0.smithi042.stdout:9/271: stat d4/d10/d4b/c52 0 2024-09-16T23:48:57.910 INFO:tasks.workunit.client.0.smithi042.stdout:7/441: write d3/fc [576315,68612] 0 2024-09-16T23:48:57.915 INFO:tasks.workunit.client.0.smithi042.stdout:5/466: dwrite d9/db/de/d10/f20 [0,4194304] 0 2024-09-16T23:48:57.918 INFO:tasks.workunit.client.0.smithi042.stdout:2/433: chown d0/c1e 285841075 1 2024-09-16T23:48:57.919 INFO:tasks.workunit.client.0.smithi042.stdout:3/476: dwrite d2/d1b/d1d/d30/f7c [0,4194304] 0 2024-09-16T23:48:57.922 INFO:tasks.workunit.client.1.smithi043.stdout:4/585: unlink d8/d15/f21 0 2024-09-16T23:48:57.925 INFO:tasks.workunit.client.1.smithi043.stdout:8/560: dread de/d18/d27/f75 [0,4194304] 0 2024-09-16T23:48:57.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/558: mknod d4/d30/d53/d88/ca5 0 2024-09-16T23:48:57.925 INFO:tasks.workunit.client.0.smithi042.stdout:4/515: creat de/d42/d68/fb1 x:0 0 0 2024-09-16T23:48:57.926 INFO:tasks.workunit.client.1.smithi043.stdout:0/561: creat d0/d2/df/da7/fbd x:0 0 0 2024-09-16T23:48:57.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/467: creat d9/db/d19/d21/d41/d7d/fa0 x:0 0 0 2024-09-16T23:48:57.929 INFO:tasks.workunit.client.0.smithi042.stdout:9/272: rmdir d4 39 2024-09-16T23:48:57.929 INFO:tasks.workunit.client.0.smithi042.stdout:1/470: write d0/d9/d90/d6b/d88/f12 [877316,114563] 0 2024-09-16T23:48:57.929 INFO:tasks.workunit.client.1.smithi043.stdout:9/492: mkdir d4/d28/d5f/d91/d97 0 2024-09-16T23:48:57.931 INFO:tasks.workunit.client.0.smithi042.stdout:7/442: creat d3/f9a x:0 0 0 2024-09-16T23:48:57.932 INFO:tasks.workunit.client.0.smithi042.stdout:3/477: chown d2/c3 235581607 1 2024-09-16T23:48:57.935 INFO:tasks.workunit.client.1.smithi043.stdout:4/586: dwrite d8/d25/d64/f7f [0,4194304] 0 2024-09-16T23:48:57.935 INFO:tasks.workunit.client.1.smithi043.stdout:6/534: rmdir de/d23/d34/d9b/d5a/da4 0 2024-09-16T23:48:57.935 INFO:tasks.workunit.client.0.smithi042.stdout:0/297: dread d0/df/d23/d33/d51/f59 [0,4194304] 0 2024-09-16T23:48:57.936 INFO:tasks.workunit.client.0.smithi042.stdout:3/478: chown d2/d1b/d1d/d34/d62 0 1 2024-09-16T23:48:57.936 INFO:tasks.workunit.client.0.smithi042.stdout:8/318: dwrite da/d19/d1d/f33 [0,4194304] 0 2024-09-16T23:48:57.937 INFO:tasks.workunit.client.0.smithi042.stdout:4/516: mkdir de/d12/d1b/d8a/db2 0 2024-09-16T23:48:57.939 INFO:tasks.workunit.client.0.smithi042.stdout:9/273: truncate d4/d10/f4a 553752 0 2024-09-16T23:48:57.941 INFO:tasks.workunit.client.0.smithi042.stdout:7/443: rmdir d3/d15/d24 39 2024-09-16T23:48:57.942 INFO:tasks.workunit.client.0.smithi042.stdout:2/434: dwrite d0/da/d17/f28 [0,4194304] 0 2024-09-16T23:48:57.942 INFO:tasks.workunit.client.1.smithi043.stdout:8/561: dwrite de/d18/d1f/d2f/f43 [0,4194304] 0 2024-09-16T23:48:57.943 INFO:tasks.workunit.client.0.smithi042.stdout:1/471: unlink d0/d17/c4b 0 2024-09-16T23:48:57.943 INFO:tasks.workunit.client.1.smithi043.stdout:2/496: creat d8/d5e/d83/fa7 x:0 0 0 2024-09-16T23:48:57.944 INFO:tasks.workunit.client.0.smithi042.stdout:8/319: truncate da/d19/d1d/f3c 726099 0 2024-09-16T23:48:57.945 INFO:tasks.workunit.client.0.smithi042.stdout:5/468: creat d9/db/de/d92/fa1 x:0 0 0 2024-09-16T23:48:57.945 INFO:tasks.workunit.client.0.smithi042.stdout:3/479: symlink d2/d1b/la7 0 2024-09-16T23:48:57.947 INFO:tasks.workunit.client.1.smithi043.stdout:2/497: chown d8/d1d/d20/d23/d38/l50 4 1 2024-09-16T23:48:57.947 INFO:tasks.workunit.client.1.smithi043.stdout:3/559: creat d4/d30/fa6 x:0 0 0 2024-09-16T23:48:57.948 INFO:tasks.workunit.client.0.smithi042.stdout:1/472: chown d0/d9/d4e/f7e 5344202 1 2024-09-16T23:48:57.950 INFO:tasks.workunit.client.1.smithi043.stdout:9/493: chown d4/d1c/f20 36967 1 2024-09-16T23:48:57.955 INFO:tasks.workunit.client.0.smithi042.stdout:3/480: mknod d2/d7/d61/d73/ca8 0 2024-09-16T23:48:57.955 INFO:tasks.workunit.client.0.smithi042.stdout:4/517: mkdir de/d42/d85/db3 0 2024-09-16T23:48:57.955 INFO:tasks.workunit.client.1.smithi043.stdout:6/535: creat de/d23/d34/d66/d9d/faf x:0 0 0 2024-09-16T23:48:57.956 INFO:tasks.workunit.client.0.smithi042.stdout:7/444: mkdir d3/d15/d24/d28/d9b 0 2024-09-16T23:48:57.959 INFO:tasks.workunit.client.1.smithi043.stdout:4/587: creat d8/d25/d39/d41/fb9 x:0 0 0 2024-09-16T23:48:57.959 INFO:tasks.workunit.client.1.smithi043.stdout:1/652: creat d4/df/fe0 x:0 0 0 2024-09-16T23:48:57.959 INFO:tasks.workunit.client.1.smithi043.stdout:4/588: write d8/d15/d20/d2b/d54/d55/f94 [5131606,111452] 0 2024-09-16T23:48:57.959 INFO:tasks.workunit.client.1.smithi043.stdout:0/562: symlink d0/d2/df/d1a/d46/d39/d83/lbe 0 2024-09-16T23:48:57.960 INFO:tasks.workunit.client.0.smithi042.stdout:0/298: getdents d0/d48 0 2024-09-16T23:48:57.961 INFO:tasks.workunit.client.0.smithi042.stdout:4/518: write de/d42/d68/f69 [1665145,3971] 0 2024-09-16T23:48:57.962 INFO:tasks.workunit.client.0.smithi042.stdout:5/469: rmdir d9/db/d11/d64 0 2024-09-16T23:48:57.962 INFO:tasks.workunit.client.1.smithi043.stdout:2/498: mkdir d8/d1d/d20/d23/da8 0 2024-09-16T23:48:57.962 INFO:tasks.workunit.client.0.smithi042.stdout:7/445: stat d3/d15/d32/d6b/d64/c65 0 2024-09-16T23:48:57.964 INFO:tasks.workunit.client.1.smithi043.stdout:3/560: rename d4/d30/l7a to d4/de/d5d/la7 0 2024-09-16T23:48:57.964 INFO:tasks.workunit.client.1.smithi043.stdout:3/561: chown d4/d30/c31 2365596 1 2024-09-16T23:48:57.965 INFO:tasks.workunit.client.0.smithi042.stdout:9/274: dwrite d4/f5 [4194304,4194304] 0 2024-09-16T23:48:57.966 INFO:tasks.workunit.client.0.smithi042.stdout:1/473: mknod d0/d9/d90/d6b/d88/d32/d2f/d98/ca5 0 2024-09-16T23:48:57.966 INFO:tasks.workunit.client.0.smithi042.stdout:0/299: dread d0/d15/f3c [0,4194304] 0 2024-09-16T23:48:57.968 INFO:tasks.workunit.client.0.smithi042.stdout:3/481: rename d2/d1b/d89/l74 to d2/d9b/la9 0 2024-09-16T23:48:57.968 INFO:tasks.workunit.client.0.smithi042.stdout:5/470: write d9/db/f25 [2056053,85219] 0 2024-09-16T23:48:57.970 INFO:tasks.workunit.client.0.smithi042.stdout:0/300: fdatasync d0/df/f18 0 2024-09-16T23:48:57.971 INFO:tasks.workunit.client.1.smithi043.stdout:8/562: mknod de/d18/d27/cae 0 2024-09-16T23:48:57.976 INFO:tasks.workunit.client.0.smithi042.stdout:4/519: symlink de/d2e/d37/d3c/lb4 0 2024-09-16T23:48:57.979 INFO:tasks.workunit.client.1.smithi043.stdout:7/525: dwrite d4/db/d28/f2d [0,4194304] 0 2024-09-16T23:48:57.980 INFO:tasks.workunit.client.0.smithi042.stdout:2/435: dread d0/d5d/f8c [4194304,4194304] 0 2024-09-16T23:48:57.980 INFO:tasks.workunit.client.0.smithi042.stdout:4/520: read de/d12/d7d/f88 [3093668,95679] 0 2024-09-16T23:48:57.981 INFO:tasks.workunit.client.0.smithi042.stdout:1/474: mkdir d0/da6 0 2024-09-16T23:48:57.981 INFO:tasks.workunit.client.1.smithi043.stdout:0/563: dread d0/d2/f33 [0,4194304] 0 2024-09-16T23:48:57.981 INFO:tasks.workunit.client.0.smithi042.stdout:1/475: chown d0/d3b/l64 22 1 2024-09-16T23:48:57.982 INFO:tasks.workunit.client.0.smithi042.stdout:7/446: rename d3/d1f/d49/d76/d19/d63/f79 to d3/f9c 0 2024-09-16T23:48:57.983 INFO:tasks.workunit.client.1.smithi043.stdout:8/563: dread de/d18/f4c [0,4194304] 0 2024-09-16T23:48:57.992 INFO:tasks.workunit.client.1.smithi043.stdout:8/564: dwrite de/f17 [0,4194304] 0 2024-09-16T23:48:57.993 INFO:tasks.workunit.client.0.smithi042.stdout:1/476: creat d0/d17/fa7 x:0 0 0 2024-09-16T23:48:57.994 INFO:tasks.workunit.client.0.smithi042.stdout:3/482: rename d2/d1b/d1d/f44 to d2/d9/faa 0 2024-09-16T23:48:57.995 INFO:tasks.workunit.client.0.smithi042.stdout:3/483: dread - d2/d1b/d1d/d34/f92 zero size 2024-09-16T23:48:57.997 INFO:tasks.workunit.client.1.smithi043.stdout:9/494: symlink d4/d21/d2a/d2c/l98 0 2024-09-16T23:48:57.997 INFO:tasks.workunit.client.0.smithi042.stdout:5/471: getdents d9/db/de 0 2024-09-16T23:48:57.998 INFO:tasks.workunit.client.0.smithi042.stdout:3/484: chown d2/d7/d61/d8e/d9d/ca6 5726 1 2024-09-16T23:48:57.998 INFO:tasks.workunit.client.0.smithi042.stdout:4/521: symlink de/d2e/lb5 0 2024-09-16T23:48:58.001 INFO:tasks.workunit.client.1.smithi043.stdout:5/739: rmdir d3/d79/dcc/df0 0 2024-09-16T23:48:58.002 INFO:tasks.workunit.client.0.smithi042.stdout:3/485: truncate d2/f41 690904 0 2024-09-16T23:48:58.002 INFO:tasks.workunit.client.0.smithi042.stdout:5/472: creat d9/d2c/d7b/fa2 x:0 0 0 2024-09-16T23:48:58.003 INFO:tasks.workunit.client.0.smithi042.stdout:3/486: stat d2/d7/d61/d73/ca8 0 2024-09-16T23:48:58.005 INFO:tasks.workunit.client.0.smithi042.stdout:7/447: truncate d3/d1f/d49/d76/d19/d63/f70 348036 0 2024-09-16T23:48:58.005 INFO:tasks.workunit.client.0.smithi042.stdout:9/275: truncate d4/fb 3138801 0 2024-09-16T23:48:58.007 INFO:tasks.workunit.client.0.smithi042.stdout:9/276: fsync d4/f5 0 2024-09-16T23:48:58.007 INFO:tasks.workunit.client.0.smithi042.stdout:3/487: creat d2/d7/fab x:0 0 0 2024-09-16T23:48:58.009 INFO:tasks.workunit.client.0.smithi042.stdout:5/473: symlink d9/db/d73/la3 0 2024-09-16T23:48:58.009 INFO:tasks.workunit.client.1.smithi043.stdout:6/536: creat de/d23/d34/d9b/d5a/da3/fb0 x:0 0 0 2024-09-16T23:48:58.010 INFO:tasks.workunit.client.0.smithi042.stdout:3/488: chown d2/d1b/d1d/d30 9795 1 2024-09-16T23:48:58.010 INFO:tasks.workunit.client.0.smithi042.stdout:9/277: chown d4/d12/d2d/l4f 51689 1 2024-09-16T23:48:58.011 INFO:tasks.workunit.client.0.smithi042.stdout:7/448: symlink d3/d15/d24/d28/l9d 0 2024-09-16T23:48:58.012 INFO:tasks.workunit.client.0.smithi042.stdout:4/522: dwrite de/d1a/da7/f4a [0,4194304] 0 2024-09-16T23:48:58.013 INFO:tasks.workunit.client.1.smithi043.stdout:4/589: dwrite d8/fc [0,4194304] 0 2024-09-16T23:48:58.015 INFO:tasks.workunit.client.0.smithi042.stdout:4/523: fsync de/d2e/d37/d4c/f9b 0 2024-09-16T23:48:58.015 INFO:tasks.workunit.client.0.smithi042.stdout:5/474: unlink d9/db/de/c81 0 2024-09-16T23:48:58.017 INFO:tasks.workunit.client.0.smithi042.stdout:3/489: creat d2/d1b/fac x:0 0 0 2024-09-16T23:48:58.017 INFO:tasks.workunit.client.0.smithi042.stdout:1/477: stat d0/d9/d90/d6b/d88/d32/d2f/d98 0 2024-09-16T23:48:58.023 INFO:tasks.workunit.client.0.smithi042.stdout:5/475: mkdir d9/db/d1b/da4 0 2024-09-16T23:48:58.023 INFO:tasks.workunit.client.0.smithi042.stdout:7/449: getdents d3/d15/d32/d6b/d64/d92 0 2024-09-16T23:48:58.025 INFO:tasks.workunit.client.0.smithi042.stdout:9/278: rename d4/d10/l22 to d4/d12/l56 0 2024-09-16T23:48:58.028 INFO:tasks.workunit.client.0.smithi042.stdout:7/450: rename d3/l1d to d3/d1f/d49/d78/l9e 0 2024-09-16T23:48:58.029 INFO:tasks.workunit.client.1.smithi043.stdout:3/562: link d4/de/d35/f1d d4/de/d1f/d34/fa8 0 2024-09-16T23:48:58.030 INFO:tasks.workunit.client.0.smithi042.stdout:1/478: getdents d0/d9a 0 2024-09-16T23:48:58.032 INFO:tasks.workunit.client.0.smithi042.stdout:5/476: rename d9/db/d73 to d9/db/d19/d21/d85/da5 0 2024-09-16T23:48:58.032 INFO:tasks.workunit.client.1.smithi043.stdout:9/495: rename d4/d21/d2a/f7a to d4/d21/d2a/f99 0 2024-09-16T23:48:58.034 INFO:tasks.workunit.client.0.smithi042.stdout:5/477: creat d9/db/de/d92/fa6 x:0 0 0 2024-09-16T23:48:58.036 INFO:tasks.workunit.client.1.smithi043.stdout:5/740: symlink d3/d9/d6f/d85/lf8 0 2024-09-16T23:48:58.036 INFO:tasks.workunit.client.0.smithi042.stdout:4/524: write de/f5a [500216,81028] 0 2024-09-16T23:48:58.036 INFO:tasks.workunit.client.1.smithi043.stdout:8/565: dwrite de/f23 [0,4194304] 0 2024-09-16T23:48:58.037 INFO:tasks.workunit.client.0.smithi042.stdout:5/478: chown d9/db/de/d10/l45 57 1 2024-09-16T23:48:58.038 INFO:tasks.workunit.client.0.smithi042.stdout:5/479: chown d9/db/de/d5e/l4d 1 1 2024-09-16T23:48:58.039 INFO:tasks.workunit.client.1.smithi043.stdout:8/566: write de/d18/d1f/d25/d65/d7f/f8c [186322,14995] 0 2024-09-16T23:48:58.039 INFO:tasks.workunit.client.0.smithi042.stdout:9/279: rename d4/d12/d16/f17 to d4/d10/f57 0 2024-09-16T23:48:58.039 INFO:tasks.workunit.client.1.smithi043.stdout:9/496: dread d4/f48 [0,4194304] 0 2024-09-16T23:48:58.040 INFO:tasks.workunit.client.1.smithi043.stdout:6/537: readlink de/d1a/d39/la1 0 2024-09-16T23:48:58.041 INFO:tasks.workunit.client.0.smithi042.stdout:4/525: mknod de/d12/db0/cb6 0 2024-09-16T23:48:58.041 INFO:tasks.workunit.client.0.smithi042.stdout:1/479: creat d0/d9/d90/d6b/d88/d63/fa8 x:0 0 0 2024-09-16T23:48:58.041 INFO:tasks.workunit.client.1.smithi043.stdout:7/526: mknod d4/cb0 0 2024-09-16T23:48:58.043 INFO:tasks.workunit.client.1.smithi043.stdout:2/499: dread d8/d1d/d25/f2b [0,4194304] 0 2024-09-16T23:48:58.044 INFO:tasks.workunit.client.0.smithi042.stdout:7/451: rename d3/d15/d32/d6b/d5b/c6d to d3/d98/c9f 0 2024-09-16T23:48:58.045 INFO:tasks.workunit.client.0.smithi042.stdout:5/480: fsync d9/db/d11/f1c 0 2024-09-16T23:48:58.046 INFO:tasks.workunit.client.0.smithi042.stdout:4/526: mknod de/d1a/da7/cb7 0 2024-09-16T23:48:58.050 INFO:tasks.workunit.client.1.smithi043.stdout:2/500: dwrite d8/d1f/f2e [0,4194304] 0 2024-09-16T23:48:58.050 INFO:tasks.workunit.client.0.smithi042.stdout:9/280: mknod d4/d12/d16/c58 0 2024-09-16T23:48:58.054 INFO:tasks.workunit.client.0.smithi042.stdout:9/281: dread d4/d10/f4a [0,4194304] 0 2024-09-16T23:48:58.055 INFO:tasks.workunit.client.0.smithi042.stdout:7/452: dwrite d3/d1f/f26 [4194304,4194304] 0 2024-09-16T23:48:58.055 INFO:tasks.workunit.client.1.smithi043.stdout:8/567: read de/f54 [698167,68689] 0 2024-09-16T23:48:58.057 INFO:tasks.workunit.client.1.smithi043.stdout:8/568: chown de/d18/d1f/d25/l57 149939074 1 2024-09-16T23:48:58.057 INFO:tasks.workunit.client.0.smithi042.stdout:5/481: unlink d9/d4a/d89/f72 0 2024-09-16T23:48:58.057 INFO:tasks.workunit.client.0.smithi042.stdout:9/282: dread - d4/d12/f39 zero size 2024-09-16T23:48:58.058 INFO:tasks.workunit.client.0.smithi042.stdout:1/480: mknod d0/d9/d90/d6b/d88/d32/d2f/d98/ca9 0 2024-09-16T23:48:58.059 INFO:tasks.workunit.client.1.smithi043.stdout:4/590: symlink d8/d15/d20/d2b/lba 0 2024-09-16T23:48:58.061 INFO:tasks.workunit.client.1.smithi043.stdout:4/591: readlink d8/d15/d20/d2b/d54/d55/l1e 0 2024-09-16T23:48:58.061 INFO:tasks.workunit.client.0.smithi042.stdout:5/482: chown d9/d4a/f70 103044199 1 2024-09-16T23:48:58.062 INFO:tasks.workunit.client.0.smithi042.stdout:4/527: dread de/d1a/da7/f4a [0,4194304] 0 2024-09-16T23:48:58.062 INFO:tasks.workunit.client.1.smithi043.stdout:9/497: dread d4/d1c/d6b/f3c [0,4194304] 0 2024-09-16T23:48:58.063 INFO:tasks.workunit.client.1.smithi043.stdout:1/653: rename d4/d9/c5f to d4/d9/ce1 0 2024-09-16T23:48:58.064 INFO:tasks.workunit.client.1.smithi043.stdout:4/592: read d8/d15/d20/f8d [398612,59138] 0 2024-09-16T23:48:58.065 INFO:tasks.workunit.client.1.smithi043.stdout:0/564: creat d0/d2/df/fbf x:0 0 0 2024-09-16T23:48:58.065 INFO:tasks.workunit.client.1.smithi043.stdout:1/654: truncate d4/d34/fbe 823004 0 2024-09-16T23:48:58.066 INFO:tasks.workunit.client.1.smithi043.stdout:0/565: chown d0/d2/df/f17 0 1 2024-09-16T23:48:58.066 INFO:tasks.workunit.client.0.smithi042.stdout:4/528: mkdir de/d12/d1b/db8 0 2024-09-16T23:48:58.068 INFO:tasks.workunit.client.0.smithi042.stdout:1/481: mkdir d0/d9/d26/daa 0 2024-09-16T23:48:58.068 INFO:tasks.workunit.client.1.smithi043.stdout:0/566: truncate d0/d2/df/d1a/d3b/f7d 5132246 0 2024-09-16T23:48:58.070 INFO:tasks.workunit.client.0.smithi042.stdout:7/453: rename d3/d15 to d3/d98/da0 0 2024-09-16T23:48:58.072 INFO:tasks.workunit.client.0.smithi042.stdout:5/483: dwrite d9/db/d11/f1c [4194304,4194304] 0 2024-09-16T23:48:58.074 INFO:tasks.workunit.client.0.smithi042.stdout:9/283: dwrite d4/d12/d2d/d37/d3a/f48 [0,4194304] 0 2024-09-16T23:48:58.075 INFO:tasks.workunit.client.0.smithi042.stdout:5/484: chown d9/db/d1b/f97 538975 1 2024-09-16T23:48:58.076 INFO:tasks.workunit.client.0.smithi042.stdout:5/485: chown d9/d63/f9e 116726587 1 2024-09-16T23:48:58.078 INFO:tasks.workunit.client.0.smithi042.stdout:5/486: read - d9/d4a/d89/f9f zero size 2024-09-16T23:48:58.081 INFO:tasks.workunit.client.0.smithi042.stdout:5/487: chown d9/db/de/d10/c18 5576029 1 2024-09-16T23:48:58.082 INFO:tasks.workunit.client.0.smithi042.stdout:1/482: mknod d0/d9/d90/d6b/d88/d63/cab 0 2024-09-16T23:48:58.089 INFO:tasks.workunit.client.1.smithi043.stdout:3/563: dwrite d4/de/f21 [8388608,4194304] 0 2024-09-16T23:48:58.090 INFO:tasks.workunit.client.0.smithi042.stdout:9/284: mkdir d4/d12/d2d/d37/d59 0 2024-09-16T23:48:58.092 INFO:tasks.workunit.client.1.smithi043.stdout:5/741: dwrite d3/d9/f4b [0,4194304] 0 2024-09-16T23:48:58.099 INFO:tasks.workunit.client.0.smithi042.stdout:1/483: dwrite d0/d17/f96 [0,4194304] 0 2024-09-16T23:48:58.107 INFO:tasks.workunit.client.1.smithi043.stdout:3/564: creat d4/de/d40/d4d/fa9 x:0 0 0 2024-09-16T23:48:58.108 INFO:tasks.workunit.client.0.smithi042.stdout:1/484: symlink d0/d17/d18/d60/lac 0 2024-09-16T23:48:58.109 INFO:tasks.workunit.client.1.smithi043.stdout:1/655: truncate d4/d3e/fdb 122308 0 2024-09-16T23:48:58.110 INFO:tasks.workunit.client.1.smithi043.stdout:5/742: creat d3/db/d15/d43/ff9 x:0 0 0 2024-09-16T23:48:58.113 INFO:tasks.workunit.client.1.smithi043.stdout:6/538: link de/d1a/d28/f46 de/d23/d63/fb1 0 2024-09-16T23:48:58.114 INFO:tasks.workunit.client.1.smithi043.stdout:7/527: link d4/db/d11/d15/f3d d4/db/fb1 0 2024-09-16T23:48:58.116 INFO:tasks.workunit.client.1.smithi043.stdout:7/528: write d4/db/d11/d15/f47 [1493842,9610] 0 2024-09-16T23:48:58.118 INFO:tasks.workunit.client.1.smithi043.stdout:6/539: dread de/d1a/f32 [0,4194304] 0 2024-09-16T23:48:58.118 INFO:tasks.workunit.client.0.smithi042.stdout:1/485: dwrite d0/d17/f2d [0,4194304] 0 2024-09-16T23:48:58.119 INFO:tasks.workunit.client.1.smithi043.stdout:3/565: write d4/de/f27 [4454497,89417] 0 2024-09-16T23:48:58.119 INFO:tasks.workunit.client.1.smithi043.stdout:3/566: stat d4/d30/d53/d88 0 2024-09-16T23:48:58.121 INFO:tasks.workunit.client.1.smithi043.stdout:4/593: creat d8/fbb x:0 0 0 2024-09-16T23:48:58.128 INFO:tasks.workunit.client.1.smithi043.stdout:8/569: truncate de/d18/d27/d41/d5a/d37/f69 8104351 0 2024-09-16T23:48:58.129 INFO:tasks.workunit.client.1.smithi043.stdout:2/501: getdents d8/d1d/d20 0 2024-09-16T23:48:58.129 INFO:tasks.workunit.client.1.smithi043.stdout:2/502: chown d8/d1d/d20/f80 40 1 2024-09-16T23:48:58.134 INFO:tasks.workunit.client.1.smithi043.stdout:6/540: dread - de/f71 zero size 2024-09-16T23:48:58.136 INFO:tasks.workunit.client.0.smithi042.stdout:4/529: fsync de/d12/d7d/f83 0 2024-09-16T23:48:58.136 INFO:tasks.workunit.client.1.smithi043.stdout:3/567: rename d4/fb to d4/de/d40/d86/faa 0 2024-09-16T23:48:58.137 INFO:tasks.workunit.client.1.smithi043.stdout:4/594: mkdir d8/d25/d39/d41/d57/dbc 0 2024-09-16T23:48:58.138 INFO:tasks.workunit.client.0.smithi042.stdout:5/488: mknod d9/d2c/d7b/ca7 0 2024-09-16T23:48:58.139 INFO:tasks.workunit.client.0.smithi042.stdout:9/285: rmdir d4/d12/d16 39 2024-09-16T23:48:58.140 INFO:tasks.workunit.client.1.smithi043.stdout:8/570: mkdir de/d70/d3d/daf 0 2024-09-16T23:48:58.141 INFO:tasks.workunit.client.1.smithi043.stdout:2/503: mkdir d8/d5e/d83/d3a/d77/da9 0 2024-09-16T23:48:58.141 INFO:tasks.workunit.client.1.smithi043.stdout:8/571: chown de/l7c 3 1 2024-09-16T23:48:58.142 INFO:tasks.workunit.client.1.smithi043.stdout:5/743: symlink d3/d9/d22/d60/d9d/de5/lfa 0 2024-09-16T23:48:58.144 INFO:tasks.workunit.client.1.smithi043.stdout:8/572: read de/d70/d3d/f88 [795227,122336] 0 2024-09-16T23:48:58.144 INFO:tasks.workunit.client.0.smithi042.stdout:9/286: unlink d4/c46 0 2024-09-16T23:48:58.145 INFO:tasks.workunit.client.0.smithi042.stdout:4/530: mkdir de/d1a/da7/db9 0 2024-09-16T23:48:58.145 INFO:tasks.workunit.client.1.smithi043.stdout:1/656: link d4/d9/d1c/d1e/l29 d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/le2 0 2024-09-16T23:48:58.146 INFO:tasks.workunit.client.0.smithi042.stdout:7/454: mknod d3/d98/da0/ca1 0 2024-09-16T23:48:58.146 INFO:tasks.workunit.client.0.smithi042.stdout:4/531: stat de/d1a/da7/l2a 0 2024-09-16T23:48:58.146 INFO:tasks.workunit.client.1.smithi043.stdout:1/657: fdatasync d4/df/fe0 0 2024-09-16T23:48:58.146 INFO:tasks.workunit.client.0.smithi042.stdout:7/455: stat d3/f9a 0 2024-09-16T23:48:58.147 INFO:tasks.workunit.client.1.smithi043.stdout:4/595: creat d8/d15/d20/d2b/d23/d27/d4b/fbd x:0 0 0 2024-09-16T23:48:58.148 INFO:tasks.workunit.client.0.smithi042.stdout:5/489: symlink d9/db/d1b/da4/la8 0 2024-09-16T23:48:58.149 INFO:tasks.workunit.client.0.smithi042.stdout:4/532: mknod de/d42/d85/cba 0 2024-09-16T23:48:58.158 INFO:tasks.workunit.client.0.smithi042.stdout:5/490: dwrite d9/f74 [0,4194304] 0 2024-09-16T23:48:58.160 INFO:tasks.workunit.client.0.smithi042.stdout:4/533: dwrite de/d42/d68/fb1 [0,4194304] 0 2024-09-16T23:48:58.163 INFO:tasks.workunit.client.0.smithi042.stdout:7/456: dread d3/d1f/d4f/f5d [0,4194304] 0 2024-09-16T23:48:58.164 INFO:tasks.workunit.client.0.smithi042.stdout:5/491: rmdir d9/db/de/d5e 39 2024-09-16T23:48:58.165 INFO:tasks.workunit.client.0.smithi042.stdout:4/534: symlink de/d42/d85/lbb 0 2024-09-16T23:48:58.169 INFO:tasks.workunit.client.0.smithi042.stdout:4/535: write de/f31 [3008719,126294] 0 2024-09-16T23:48:58.173 INFO:tasks.workunit.client.0.smithi042.stdout:4/536: symlink de/d1a/d70/dac/lbc 0 2024-09-16T23:48:58.174 INFO:tasks.workunit.client.0.smithi042.stdout:5/492: dwrite d9/d2c/d7b/fa2 [0,4194304] 0 2024-09-16T23:48:58.176 INFO:tasks.workunit.client.0.smithi042.stdout:5/493: mkdir d9/d2c/da9 0 2024-09-16T23:48:58.180 INFO:tasks.workunit.client.0.smithi042.stdout:5/494: truncate d9/db/de/d5e/d87/d61/f7c 4426334 0 2024-09-16T23:48:58.182 INFO:tasks.workunit.client.0.smithi042.stdout:5/495: mkdir d9/db/de/d92/daa 0 2024-09-16T23:48:58.191 INFO:tasks.workunit.client.0.smithi042.stdout:5/496: truncate d9/db/de/d5e/d87/d61/f4e 3710501 0 2024-09-16T23:48:58.200 INFO:tasks.workunit.client.0.smithi042.stdout:5/497: dwrite d9/d63/f9e [0,4194304] 0 2024-09-16T23:48:58.201 INFO:tasks.workunit.client.0.smithi042.stdout:5/498: dread - d9/db/d11/f66 zero size 2024-09-16T23:48:58.202 INFO:tasks.workunit.client.0.smithi042.stdout:5/499: rmdir d9/db/d19/d21/d85 39 2024-09-16T23:48:58.214 INFO:tasks.workunit.client.0.smithi042.stdout:5/500: dwrite d9/db/de/d10/d2f/f36 [0,4194304] 0 2024-09-16T23:48:58.214 INFO:tasks.workunit.client.1.smithi043.stdout:2/504: dread d8/d5e/d83/d3a/f62 [0,4194304] 0 2024-09-16T23:48:58.221 INFO:tasks.workunit.client.1.smithi043.stdout:7/529: creat d4/db/fb2 x:0 0 0 2024-09-16T23:48:58.222 INFO:tasks.workunit.client.1.smithi043.stdout:5/744: mkdir d3/d9/d22/d4a/d64/d8f/dfb 0 2024-09-16T23:48:58.224 INFO:tasks.workunit.client.1.smithi043.stdout:6/541: mknod de/d1a/cb2 0 2024-09-16T23:48:58.226 INFO:tasks.workunit.client.1.smithi043.stdout:8/573: truncate de/d18/d27/d41/d5a/f26 8799926 0 2024-09-16T23:48:58.228 INFO:tasks.workunit.client.1.smithi043.stdout:4/596: rmdir d8/d25/d64 39 2024-09-16T23:48:58.232 INFO:tasks.workunit.client.0.smithi042.stdout:1/486: rename d0/d9/f24 to d0/d9/d90/fad 0 2024-09-16T23:48:58.233 INFO:tasks.workunit.client.0.smithi042.stdout:1/487: fsync d0/d3b/d8d/fa3 0 2024-09-16T23:48:58.235 INFO:tasks.workunit.client.0.smithi042.stdout:4/537: rename de/d2e/d37/d3e/dae to de/d42/d82/d94/dbd 0 2024-09-16T23:48:58.238 INFO:tasks.workunit.client.1.smithi043.stdout:3/568: write d4/de/d40/f69 [3595960,108590] 0 2024-09-16T23:48:58.238 INFO:tasks.workunit.client.0.smithi042.stdout:9/287: dread d4/d10/d24/f35 [0,4194304] 0 2024-09-16T23:48:58.240 INFO:tasks.workunit.client.1.smithi043.stdout:7/530: fdatasync d4/f19 0 2024-09-16T23:48:58.241 INFO:tasks.workunit.client.0.smithi042.stdout:5/501: rename d9/db/d19/f39 to d9/db/d19/d21/d41/fab 0 2024-09-16T23:48:58.243 INFO:tasks.workunit.client.0.smithi042.stdout:1/488: dwrite d0/d9/d90/d6b/d88/f75 [0,4194304] 0 2024-09-16T23:48:58.245 INFO:tasks.workunit.client.1.smithi043.stdout:5/745: rename d3/d9/c29 to d3/d9/d22/d4a/d4c/ded/cfc 0 2024-09-16T23:48:58.246 INFO:tasks.workunit.client.0.smithi042.stdout:4/538: rename de/d42/d5d/d8f/fa0 to de/d1a/d70/fbe 0 2024-09-16T23:48:58.247 INFO:tasks.workunit.client.0.smithi042.stdout:5/502: symlink d9/db/d19/d21/d41/d7d/lac 0 2024-09-16T23:48:58.249 INFO:tasks.workunit.client.0.smithi042.stdout:4/539: symlink de/d12/lbf 0 2024-09-16T23:48:58.250 INFO:tasks.workunit.client.0.smithi042.stdout:4/540: chown de/d2e/d37/f3b 10294 1 2024-09-16T23:48:58.253 INFO:tasks.workunit.client.0.smithi042.stdout:9/288: dread d4/d12/d2d/d37/d28/f31 [0,4194304] 0 2024-09-16T23:48:58.253 INFO:tasks.workunit.client.0.smithi042.stdout:4/541: mkdir de/d42/d5d/d8f/d9f/dc0 0 2024-09-16T23:48:58.254 INFO:tasks.workunit.client.0.smithi042.stdout:5/503: link d9/db/d19/d21/d41/c5a d9/d2c/da9/cad 0 2024-09-16T23:48:58.255 INFO:tasks.workunit.client.0.smithi042.stdout:1/489: creat d0/d9/d90/d6b/fae x:0 0 0 2024-09-16T23:48:58.255 INFO:tasks.workunit.client.0.smithi042.stdout:9/289: truncate d4/d10/f3f 5033449 0 2024-09-16T23:48:58.257 INFO:tasks.workunit.client.0.smithi042.stdout:4/542: symlink de/d12/d7d/lc1 0 2024-09-16T23:48:58.258 INFO:tasks.workunit.client.1.smithi043.stdout:1/658: mknod d4/d9/d8f/db6/dcc/dda/ce3 0 2024-09-16T23:48:58.259 INFO:tasks.workunit.client.0.smithi042.stdout:9/290: chown d4/d10/f1f 2 1 2024-09-16T23:48:58.259 INFO:tasks.workunit.client.1.smithi043.stdout:2/505: symlink d8/d4b/d5f/d63/laa 0 2024-09-16T23:48:58.261 INFO:tasks.workunit.client.0.smithi042.stdout:1/490: creat d0/d9/d26/d8b/faf x:0 0 0 2024-09-16T23:48:58.262 INFO:tasks.workunit.client.0.smithi042.stdout:5/504: dread d9/db/d19/d21/d41/fab [0,4194304] 0 2024-09-16T23:48:58.265 INFO:tasks.workunit.client.0.smithi042.stdout:5/505: mknod d9/d4a/cae 0 2024-09-16T23:48:58.265 INFO:tasks.workunit.client.0.smithi042.stdout:9/291: fdatasync d4/d12/d2d/f32 0 2024-09-16T23:48:58.266 INFO:tasks.workunit.client.0.smithi042.stdout:1/491: creat d0/d9/d90/d6b/d88/d32/d2f/fb0 x:0 0 0 2024-09-16T23:48:58.267 INFO:tasks.workunit.client.0.smithi042.stdout:6/355: sync 2024-09-16T23:48:58.267 INFO:tasks.workunit.client.1.smithi043.stdout:7/531: truncate d4/db/d11/d21/d31/f4a 100188 0 2024-09-16T23:48:58.268 INFO:tasks.workunit.client.0.smithi042.stdout:5/506: mkdir d9/db/de/d92/daf 0 2024-09-16T23:48:58.269 INFO:tasks.workunit.client.0.smithi042.stdout:1/492: chown d0/d9/d90/d6b/d88/d32/d2f/fa0 877560 1 2024-09-16T23:48:58.270 INFO:tasks.workunit.client.1.smithi043.stdout:4/597: dwrite d8/d15/d20/d2b/f6b [0,4194304] 0 2024-09-16T23:48:58.270 INFO:tasks.workunit.client.0.smithi042.stdout:4/543: getdents de/d12 0 2024-09-16T23:48:58.272 INFO:tasks.workunit.client.1.smithi043.stdout:5/746: mkdir d3/db/d15/d4d/dfd 0 2024-09-16T23:48:58.272 INFO:tasks.workunit.client.0.smithi042.stdout:4/544: chown de/d2e/d37/d3e/f40 2 1 2024-09-16T23:48:58.274 INFO:tasks.workunit.client.0.smithi042.stdout:4/545: dread - de/d2e/d37/d4c/d4f/f7a zero size 2024-09-16T23:48:58.275 INFO:tasks.workunit.client.0.smithi042.stdout:6/356: creat d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f79 x:0 0 0 2024-09-16T23:48:58.275 INFO:tasks.workunit.client.1.smithi043.stdout:5/747: chown d3/d9/d22/d4a/d64/d8f/da1/ff3 549 1 2024-09-16T23:48:58.276 INFO:tasks.workunit.client.0.smithi042.stdout:5/507: creat d9/db/d19/d21/fb0 x:0 0 0 2024-09-16T23:48:58.277 INFO:tasks.workunit.client.1.smithi043.stdout:7/532: dwrite d4/db/d50/fa9 [0,4194304] 0 2024-09-16T23:48:58.278 INFO:tasks.workunit.client.0.smithi042.stdout:4/546: chown de/f2f 41624363 1 2024-09-16T23:48:58.278 INFO:tasks.workunit.client.0.smithi042.stdout:1/493: rename d0/d9/d26/c9c to d0/d9/d90/d6b/cb1 0 2024-09-16T23:48:58.280 INFO:tasks.workunit.client.1.smithi043.stdout:1/659: creat d4/d9/d1c/d1e/d42/d5d/d71/fe4 x:0 0 0 2024-09-16T23:48:58.281 INFO:tasks.workunit.client.0.smithi042.stdout:4/547: mknod de/d42/d5d/cc2 0 2024-09-16T23:48:58.282 INFO:tasks.workunit.client.0.smithi042.stdout:6/357: creat d8/db/dd/d1b/d34/d39/d5e/f7a x:0 0 0 2024-09-16T23:48:58.284 INFO:tasks.workunit.client.0.smithi042.stdout:6/358: read - d8/db/dd/d2c/d57/f60 zero size 2024-09-16T23:48:58.288 INFO:tasks.workunit.client.0.smithi042.stdout:4/548: mkdir de/d2e/d37/dc3 0 2024-09-16T23:48:58.289 INFO:tasks.workunit.client.0.smithi042.stdout:1/494: fsync d0/d9/f1c 0 2024-09-16T23:48:58.290 INFO:tasks.workunit.client.0.smithi042.stdout:1/495: readlink d0/d17/l19 0 2024-09-16T23:48:58.290 INFO:tasks.workunit.client.0.smithi042.stdout:5/508: truncate d9/db/d1b/f95 2988511 0 2024-09-16T23:48:58.291 INFO:tasks.workunit.client.0.smithi042.stdout:4/549: mknod de/d42/d82/d94/cc4 0 2024-09-16T23:48:58.292 INFO:tasks.workunit.client.1.smithi043.stdout:8/574: creat de/fb0 x:0 0 0 2024-09-16T23:48:58.292 INFO:tasks.workunit.client.0.smithi042.stdout:4/550: write de/d12/f14 [1928618,85863] 0 2024-09-16T23:48:58.292 INFO:tasks.workunit.client.1.smithi043.stdout:6/542: dwrite de/d1a/d28/f46 [0,4194304] 0 2024-09-16T23:48:58.294 INFO:tasks.workunit.client.0.smithi042.stdout:6/359: unlink d8/db/dd/d2c/d57/l46 0 2024-09-16T23:48:58.295 INFO:tasks.workunit.client.1.smithi043.stdout:6/543: write de/d23/fa2 [954435,113364] 0 2024-09-16T23:48:58.296 INFO:tasks.workunit.client.0.smithi042.stdout:5/509: mknod d9/cb1 0 2024-09-16T23:48:58.298 INFO:tasks.workunit.client.1.smithi043.stdout:5/748: creat d3/db/d84/db6/ffe x:0 0 0 2024-09-16T23:48:58.304 INFO:tasks.workunit.client.0.smithi042.stdout:5/510: mknod d9/db/d19/d21/d85/da5/cb2 0 2024-09-16T23:48:58.306 INFO:tasks.workunit.client.0.smithi042.stdout:4/551: write de/d2e/d37/d3e/f4d [899552,4464] 0 2024-09-16T23:48:58.307 INFO:tasks.workunit.client.0.smithi042.stdout:1/496: mkdir d0/db2 0 2024-09-16T23:48:58.308 INFO:tasks.workunit.client.0.smithi042.stdout:4/552: dread - de/d42/d5d/f74 zero size 2024-09-16T23:48:58.308 INFO:tasks.workunit.client.0.smithi042.stdout:1/497: write d0/f95 [801163,27138] 0 2024-09-16T23:48:58.310 INFO:tasks.workunit.client.1.smithi043.stdout:3/569: rename d4/de/d1f/f46 to d4/de/fab 0 2024-09-16T23:48:58.310 INFO:tasks.workunit.client.0.smithi042.stdout:5/511: chown d9/db/d11/c4f 154 1 2024-09-16T23:48:58.312 INFO:tasks.workunit.client.1.smithi043.stdout:3/570: fsync d4/de/d1f/d2c/f90 0 2024-09-16T23:48:58.313 INFO:tasks.workunit.client.1.smithi043.stdout:5/749: dread d3/d9/d22/d9f/fc1 [0,4194304] 0 2024-09-16T23:48:58.313 INFO:tasks.workunit.client.1.smithi043.stdout:3/571: dread - d4/de/d35/f8d zero size 2024-09-16T23:48:58.315 INFO:tasks.workunit.client.0.smithi042.stdout:4/553: mknod de/d42/d99/cc5 0 2024-09-16T23:48:58.316 INFO:tasks.workunit.client.1.smithi043.stdout:8/575: rmdir de/d18/d1f/d25 39 2024-09-16T23:48:58.319 INFO:tasks.workunit.client.0.smithi042.stdout:4/554: chown de/d2e/d37/d3c/f9a 50859 1 2024-09-16T23:48:58.320 INFO:tasks.workunit.client.0.smithi042.stdout:6/360: dread d8/db/f1d [0,4194304] 0 2024-09-16T23:48:58.320 INFO:tasks.workunit.client.0.smithi042.stdout:5/512: dwrite d9/db/d1b/d31/d46/f6d [0,4194304] 0 2024-09-16T23:48:58.322 INFO:tasks.workunit.client.0.smithi042.stdout:1/498: chown d0/d9/d90/d43/l49 6 1 2024-09-16T23:48:58.327 INFO:tasks.workunit.client.1.smithi043.stdout:5/750: symlink d3/d9/d22/d60/d9d/de5/lff 0 2024-09-16T23:48:58.329 INFO:tasks.workunit.client.1.smithi043.stdout:5/751: chown d3/db/d15/d4d/d6b/d73/ldb 1122 1 2024-09-16T23:48:58.329 INFO:tasks.workunit.client.0.smithi042.stdout:6/361: mkdir d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d7b 0 2024-09-16T23:48:58.334 INFO:tasks.workunit.client.0.smithi042.stdout:1/499: creat d0/d9/d90/d6b/d88/d32/d2f/fb3 x:0 0 0 2024-09-16T23:48:58.335 INFO:tasks.workunit.client.0.smithi042.stdout:4/555: dwrite de/d1a/d70/fbe [0,4194304] 0 2024-09-16T23:48:58.335 INFO:tasks.workunit.client.1.smithi043.stdout:8/576: write de/d70/d3d/f88 [4538970,99008] 0 2024-09-16T23:48:58.336 INFO:tasks.workunit.client.1.smithi043.stdout:5/752: dwrite d3/db/d15/d4d/d6b/ff5 [0,4194304] 0 2024-09-16T23:48:58.337 INFO:tasks.workunit.client.0.smithi042.stdout:5/513: dwrite d9/db/d19/d21/fb0 [0,4194304] 0 2024-09-16T23:48:58.339 INFO:tasks.workunit.client.1.smithi043.stdout:1/660: creat d4/d9/d12/d2b/fe5 x:0 0 0 2024-09-16T23:48:58.341 INFO:tasks.workunit.client.0.smithi042.stdout:5/514: stat d9/d2c/l84 0 2024-09-16T23:48:58.341 INFO:tasks.workunit.client.0.smithi042.stdout:6/362: unlink d8/c6f 0 2024-09-16T23:48:58.342 INFO:tasks.workunit.client.1.smithi043.stdout:8/577: dwrite de/f71 [0,4194304] 0 2024-09-16T23:48:58.343 INFO:tasks.workunit.client.1.smithi043.stdout:3/572: creat d4/d30/d53/fac x:0 0 0 2024-09-16T23:48:58.345 INFO:tasks.workunit.client.1.smithi043.stdout:6/544: creat de/d1a/fb3 x:0 0 0 2024-09-16T23:48:58.345 INFO:tasks.workunit.client.0.smithi042.stdout:5/515: dread - d9/db/de/d5e/d87/d61/f5d zero size 2024-09-16T23:48:58.348 INFO:tasks.workunit.client.0.smithi042.stdout:1/500: symlink d0/d9/lb4 0 2024-09-16T23:48:58.350 INFO:tasks.workunit.client.1.smithi043.stdout:1/661: symlink d4/df/d13/le6 0 2024-09-16T23:48:58.351 INFO:tasks.workunit.client.1.smithi043.stdout:6/545: mknod de/d23/d34/d66/d70/cb4 0 2024-09-16T23:48:58.351 INFO:tasks.workunit.client.1.smithi043.stdout:3/573: dread d4/de/fab [0,4194304] 0 2024-09-16T23:48:58.353 INFO:tasks.workunit.client.0.smithi042.stdout:4/556: dwrite de/d42/d68/f72 [0,4194304] 0 2024-09-16T23:48:58.353 INFO:tasks.workunit.client.1.smithi043.stdout:3/574: dread - d4/de/d40/f80 zero size 2024-09-16T23:48:58.354 INFO:tasks.workunit.client.0.smithi042.stdout:1/501: truncate d0/d17/f2d 4738253 0 2024-09-16T23:48:58.356 INFO:tasks.workunit.client.1.smithi043.stdout:1/662: fsync d4/d9/d72/f73 0 2024-09-16T23:48:58.358 INFO:tasks.workunit.client.0.smithi042.stdout:6/363: chown d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f51 91502 1 2024-09-16T23:48:58.358 INFO:tasks.workunit.client.1.smithi043.stdout:8/578: creat de/d18/fb1 x:0 0 0 2024-09-16T23:48:58.359 INFO:tasks.workunit.client.1.smithi043.stdout:3/575: creat d4/de/d1f/d34/fad x:0 0 0 2024-09-16T23:48:58.362 INFO:tasks.workunit.client.1.smithi043.stdout:5/753: dread d3/db/d19/d3e/f70 [0,4194304] 0 2024-09-16T23:48:58.362 INFO:tasks.workunit.client.0.smithi042.stdout:5/516: creat d9/db/de/fb3 x:0 0 0 2024-09-16T23:48:58.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/517: write d9/db/d11/f66 [111020,84225] 0 2024-09-16T23:48:58.366 INFO:tasks.workunit.client.0.smithi042.stdout:1/502: fdatasync d0/d9/d90/d6b/d88/d32/f33 0 2024-09-16T23:48:58.366 INFO:tasks.workunit.client.0.smithi042.stdout:4/557: rename lb to de/d2e/d37/lc6 0 2024-09-16T23:48:58.368 INFO:tasks.workunit.client.1.smithi043.stdout:1/663: mknod d4/d9/d8f/db6/dcc/ce7 0 2024-09-16T23:48:58.368 INFO:tasks.workunit.client.1.smithi043.stdout:8/579: dread - de/d18/d1f/d25/f6e zero size 2024-09-16T23:48:58.371 INFO:tasks.workunit.client.0.smithi042.stdout:6/364: dwrite d8/db/dd/d1b/f4b [0,4194304] 0 2024-09-16T23:48:58.373 INFO:tasks.workunit.client.1.smithi043.stdout:1/664: mkdir d4/d3e/d95/de8 0 2024-09-16T23:48:58.378 INFO:tasks.workunit.client.1.smithi043.stdout:8/580: creat de/d18/d1f/d25/d65/d7f/fb2 x:0 0 0 2024-09-16T23:48:58.381 INFO:tasks.workunit.client.0.smithi042.stdout:1/503: creat d0/d9a/fb5 x:0 0 0 2024-09-16T23:48:58.381 INFO:tasks.workunit.client.1.smithi043.stdout:1/665: symlink d4/d9/d12/d2b/d3d/le9 0 2024-09-16T23:48:58.382 INFO:tasks.workunit.client.0.smithi042.stdout:6/365: truncate d8/db/dd/d1b/d34/f76 178243 0 2024-09-16T23:48:58.384 INFO:tasks.workunit.client.0.smithi042.stdout:4/558: dwrite de/d12/d1b/f98 [0,4194304] 0 2024-09-16T23:48:58.385 INFO:tasks.workunit.client.1.smithi043.stdout:8/581: dwrite de/d18/f36 [4194304,4194304] 0 2024-09-16T23:48:58.385 INFO:tasks.workunit.client.0.smithi042.stdout:6/366: chown d8/db/dd 487 1 2024-09-16T23:48:58.386 INFO:tasks.workunit.client.1.smithi043.stdout:6/546: write de/d1a/d28/f5b [791466,11415] 0 2024-09-16T23:48:58.387 INFO:tasks.workunit.client.0.smithi042.stdout:5/518: dwrite d9/d4a/d89/f9f [0,4194304] 0 2024-09-16T23:48:58.389 INFO:tasks.workunit.client.0.smithi042.stdout:4/559: read de/d2e/d37/f3b [469520,94644] 0 2024-09-16T23:48:58.390 INFO:tasks.workunit.client.1.smithi043.stdout:3/576: truncate d4/de/d1f/f3e 788829 0 2024-09-16T23:48:58.391 INFO:tasks.workunit.client.0.smithi042.stdout:6/367: dread - d8/db/dd/d1b/d2d/f3e zero size 2024-09-16T23:48:58.392 INFO:tasks.workunit.client.1.smithi043.stdout:1/666: dwrite d4/d9/d1c/d1e/d42/d5d/d71/fe4 [0,4194304] 0 2024-09-16T23:48:58.394 INFO:tasks.workunit.client.0.smithi042.stdout:4/560: creat de/d42/d99/fc7 x:0 0 0 2024-09-16T23:48:58.394 INFO:tasks.workunit.client.1.smithi043.stdout:6/547: creat de/d23/d49/fb5 x:0 0 0 2024-09-16T23:48:58.396 INFO:tasks.workunit.client.1.smithi043.stdout:8/582: dwrite de/d70/d3d/f88 [0,4194304] 0 2024-09-16T23:48:58.396 INFO:tasks.workunit.client.0.smithi042.stdout:4/561: write de/d42/d82/d94/fa3 [84219,120775] 0 2024-09-16T23:48:58.399 INFO:tasks.workunit.client.0.smithi042.stdout:5/519: creat d9/db/de/d92/daf/fb4 x:0 0 0 2024-09-16T23:48:58.399 INFO:tasks.workunit.client.0.smithi042.stdout:6/368: creat d8/db/dd/d1b/d34/d39/d5e/f7c x:0 0 0 2024-09-16T23:48:58.404 INFO:tasks.workunit.client.0.smithi042.stdout:1/504: creat d0/d9/d90/d6b/d88/d32/fb6 x:0 0 0 2024-09-16T23:48:58.405 INFO:tasks.workunit.client.0.smithi042.stdout:4/562: mknod de/d2e/d37/dc3/cc8 0 2024-09-16T23:48:58.405 INFO:tasks.workunit.client.1.smithi043.stdout:6/548: creat de/d23/d34/d9b/d8b/fb6 x:0 0 0 2024-09-16T23:48:58.407 INFO:tasks.workunit.client.1.smithi043.stdout:3/577: dread d4/d30/d53/f5c [0,4194304] 0 2024-09-16T23:48:58.408 INFO:tasks.workunit.client.1.smithi043.stdout:3/578: write d4/de/f7f [846482,27508] 0 2024-09-16T23:48:58.409 INFO:tasks.workunit.client.1.smithi043.stdout:3/579: dread - d4/de/d40/f98 zero size 2024-09-16T23:48:58.411 INFO:tasks.workunit.client.0.smithi042.stdout:4/563: write de/d2e/d37/d3e/f5c [697002,18692] 0 2024-09-16T23:48:58.412 INFO:tasks.workunit.client.0.smithi042.stdout:5/520: read d9/db/de/d5e/d87/d61/f7c [2240068,86815] 0 2024-09-16T23:48:58.413 INFO:tasks.workunit.client.0.smithi042.stdout:8/320: sync 2024-09-16T23:48:58.413 INFO:tasks.workunit.client.0.smithi042.stdout:0/301: sync 2024-09-16T23:48:58.415 INFO:tasks.workunit.client.0.smithi042.stdout:6/369: dwrite d8/f37 [0,4194304] 0 2024-09-16T23:48:58.419 INFO:tasks.workunit.client.0.smithi042.stdout:4/564: unlink de/d1a/da7/f21 0 2024-09-16T23:48:58.419 INFO:tasks.workunit.client.0.smithi042.stdout:2/436: sync 2024-09-16T23:48:58.419 INFO:tasks.workunit.client.0.smithi042.stdout:1/505: creat d0/d9a/fb7 x:0 0 0 2024-09-16T23:48:58.420 INFO:tasks.workunit.client.0.smithi042.stdout:4/565: readlink de/d2e/d37/lc6 0 2024-09-16T23:48:58.423 INFO:tasks.workunit.client.0.smithi042.stdout:4/566: chown de/d2e/d37/d3e 3 1 2024-09-16T23:48:58.423 INFO:tasks.workunit.client.0.smithi042.stdout:4/567: chown de/d1a/f33 2480551 1 2024-09-16T23:48:58.423 INFO:tasks.workunit.client.0.smithi042.stdout:5/521: chown d9/d4a/d89/f8a 31 1 2024-09-16T23:48:58.425 INFO:tasks.workunit.client.0.smithi042.stdout:5/522: chown d9/db/d1b/d79/l8b 25 1 2024-09-16T23:48:58.429 INFO:tasks.workunit.client.0.smithi042.stdout:0/302: unlink d0/df/d13/d2d/f44 0 2024-09-16T23:48:58.430 INFO:tasks.workunit.client.1.smithi043.stdout:3/580: symlink d4/de/d5d/lae 0 2024-09-16T23:48:58.431 INFO:tasks.workunit.client.1.smithi043.stdout:1/667: dread d4/d34/f3a [0,4194304] 0 2024-09-16T23:48:58.431 INFO:tasks.workunit.client.0.smithi042.stdout:8/321: unlink da/d19/f4d 0 2024-09-16T23:48:58.432 INFO:tasks.workunit.client.1.smithi043.stdout:1/668: readlink d4/df/d26/d79/lb4 0 2024-09-16T23:48:58.432 INFO:tasks.workunit.client.0.smithi042.stdout:1/506: creat d0/d17/fb8 x:0 0 0 2024-09-16T23:48:58.434 INFO:tasks.workunit.client.1.smithi043.stdout:6/549: rename de/d23/d49/c53 to de/d23/d63/cb7 0 2024-09-16T23:48:58.436 INFO:tasks.workunit.client.1.smithi043.stdout:6/550: readlink de/d23/d63/d7f/l97 0 2024-09-16T23:48:58.438 INFO:tasks.workunit.client.1.smithi043.stdout:3/581: dwrite d4/de/d40/d86/f8c [0,4194304] 0 2024-09-16T23:48:58.438 INFO:tasks.workunit.client.1.smithi043.stdout:1/669: readlink d4/d9/l4c 0 2024-09-16T23:48:58.439 INFO:tasks.workunit.client.1.smithi043.stdout:1/670: stat d4/df 0 2024-09-16T23:48:58.439 INFO:tasks.workunit.client.0.smithi042.stdout:4/568: write de/f41 [2645344,43936] 0 2024-09-16T23:48:58.440 INFO:tasks.workunit.client.1.smithi043.stdout:3/582: truncate d4/d30/d53/fac 998322 0 2024-09-16T23:48:58.440 INFO:tasks.workunit.client.0.smithi042.stdout:5/523: dwrite d9/d4a/d89/f9f [0,4194304] 0 2024-09-16T23:48:58.441 INFO:tasks.workunit.client.1.smithi043.stdout:3/583: fsync d4/de/d5d/fa3 0 2024-09-16T23:48:58.442 INFO:tasks.workunit.client.1.smithi043.stdout:8/583: truncate de/d18/f36 127312 0 2024-09-16T23:48:58.443 INFO:tasks.workunit.client.0.smithi042.stdout:5/524: fdatasync d9/db/d1b/f8c 0 2024-09-16T23:48:58.445 INFO:tasks.workunit.client.0.smithi042.stdout:5/525: chown d9/db/de/d10/d2f 1 1 2024-09-16T23:48:58.445 INFO:tasks.workunit.client.0.smithi042.stdout:4/569: read de/d12/f54 [1049105,111503] 0 2024-09-16T23:48:58.453 INFO:tasks.workunit.client.0.smithi042.stdout:5/526: dwrite d9/f74 [0,4194304] 0 2024-09-16T23:48:58.454 INFO:tasks.workunit.client.1.smithi043.stdout:1/671: creat d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fea x:0 0 0 2024-09-16T23:48:58.456 INFO:tasks.workunit.client.0.smithi042.stdout:4/570: chown de/d12/c13 10154144 1 2024-09-16T23:48:58.456 INFO:tasks.workunit.client.0.smithi042.stdout:5/527: write d9/db/d11/f1c [1975890,122619] 0 2024-09-16T23:48:58.457 INFO:tasks.workunit.client.0.smithi042.stdout:4/571: creat de/d42/d82/d94/fc9 x:0 0 0 2024-09-16T23:48:58.458 INFO:tasks.workunit.client.0.smithi042.stdout:8/322: creat da/d2f/f5e x:0 0 0 2024-09-16T23:48:58.458 INFO:tasks.workunit.client.0.smithi042.stdout:4/572: readlink de/d1a/l20 0 2024-09-16T23:48:58.460 INFO:tasks.workunit.client.0.smithi042.stdout:4/573: write de/d2e/d37/f3b [797858,46224] 0 2024-09-16T23:48:58.466 INFO:tasks.workunit.client.1.smithi043.stdout:1/672: symlink d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/leb 0 2024-09-16T23:48:58.467 INFO:tasks.workunit.client.0.smithi042.stdout:2/437: link d0/da/d1b/l83 d0/da/d17/d3e/d86/l9d 0 2024-09-16T23:48:58.468 INFO:tasks.workunit.client.0.smithi042.stdout:1/507: rename d0/d9/f56 to d0/d9/fb9 0 2024-09-16T23:48:58.470 INFO:tasks.workunit.client.0.smithi042.stdout:2/438: fdatasync d0/f2b 0 2024-09-16T23:48:58.471 INFO:tasks.workunit.client.0.smithi042.stdout:1/508: read d0/d9/f94 [1074483,123521] 0 2024-09-16T23:48:58.472 INFO:tasks.workunit.client.0.smithi042.stdout:0/303: dread d0/f9 [0,4194304] 0 2024-09-16T23:48:58.479 INFO:tasks.workunit.client.1.smithi043.stdout:6/551: getdents de/d23/d34/d9b/d5a 0 2024-09-16T23:48:58.480 INFO:tasks.workunit.client.1.smithi043.stdout:6/552: chown de/c15 17889060 1 2024-09-16T23:48:58.480 INFO:tasks.workunit.client.0.smithi042.stdout:0/304: dwrite d0/d48/f53 [0,4194304] 0 2024-09-16T23:48:58.481 INFO:tasks.workunit.client.1.smithi043.stdout:6/553: dread - de/d1a/d28/f77 zero size 2024-09-16T23:48:58.482 INFO:tasks.workunit.client.1.smithi043.stdout:6/554: write de/d23/d34/d9b/d8b/fb6 [673272,18651] 0 2024-09-16T23:48:58.484 INFO:tasks.workunit.client.0.smithi042.stdout:0/305: symlink d0/df/d23/d3e/l5a 0 2024-09-16T23:48:58.484 INFO:tasks.workunit.client.0.smithi042.stdout:2/439: unlink d0/da/d17/f4d 0 2024-09-16T23:48:58.485 INFO:tasks.workunit.client.0.smithi042.stdout:2/440: write d0/d78/d8b/f99 [831776,84633] 0 2024-09-16T23:48:58.485 INFO:tasks.workunit.client.0.smithi042.stdout:0/306: write d0/d48/f53 [1164402,64595] 0 2024-09-16T23:48:58.486 INFO:tasks.workunit.client.0.smithi042.stdout:2/441: write d0/da/d17/f28 [4132438,107613] 0 2024-09-16T23:48:58.487 INFO:tasks.workunit.client.1.smithi043.stdout:1/673: symlink d4/lec 0 2024-09-16T23:48:58.488 INFO:tasks.workunit.client.0.smithi042.stdout:2/442: chown d0/da/d1b/d48/d81 55169678 1 2024-09-16T23:48:58.489 INFO:tasks.workunit.client.1.smithi043.stdout:8/584: truncate de/d18/d27/d41/d5a/f56 760000 0 2024-09-16T23:48:58.489 INFO:tasks.workunit.client.1.smithi043.stdout:8/585: readlink de/d18/d1f/l63 0 2024-09-16T23:48:58.490 INFO:tasks.workunit.client.0.smithi042.stdout:1/509: fsync d0/f4a 0 2024-09-16T23:48:58.490 INFO:tasks.workunit.client.1.smithi043.stdout:3/584: getdents d4/d30/d53/d88 0 2024-09-16T23:48:58.491 INFO:tasks.workunit.client.0.smithi042.stdout:0/307: mkdir d0/df/d23/d33/d51/d5b 0 2024-09-16T23:48:58.492 INFO:tasks.workunit.client.1.smithi043.stdout:3/585: chown d4/de/d40/d86/f89 8037 1 2024-09-16T23:48:58.493 INFO:tasks.workunit.client.0.smithi042.stdout:1/510: dread d0/d9/d90/d6b/d88/d32/f33 [0,4194304] 0 2024-09-16T23:48:58.495 INFO:tasks.workunit.client.0.smithi042.stdout:0/308: creat d0/df/d23/d33/f5c x:0 0 0 2024-09-16T23:48:58.497 INFO:tasks.workunit.client.0.smithi042.stdout:0/309: chown d0/df/d23/f38 3915 1 2024-09-16T23:48:58.501 INFO:tasks.workunit.client.1.smithi043.stdout:8/586: dread - de/d18/d27/d41/f8f zero size 2024-09-16T23:48:58.501 INFO:tasks.workunit.client.1.smithi043.stdout:8/587: chown de/l52 69822 1 2024-09-16T23:48:58.501 INFO:tasks.workunit.client.0.smithi042.stdout:0/310: write d0/f4b [118076,128545] 0 2024-09-16T23:48:58.502 INFO:tasks.workunit.client.0.smithi042.stdout:1/511: dwrite d0/d17/f20 [4194304,4194304] 0 2024-09-16T23:48:58.504 INFO:tasks.workunit.client.0.smithi042.stdout:0/311: truncate d0/df/d23/f55 86652 0 2024-09-16T23:48:58.505 INFO:tasks.workunit.client.0.smithi042.stdout:0/312: chown d0/l11 109209 1 2024-09-16T23:48:58.507 INFO:tasks.workunit.client.0.smithi042.stdout:1/512: truncate d0/d9/d90/d6b/d88/d32/fb6 693973 0 2024-09-16T23:48:58.510 INFO:tasks.workunit.client.1.smithi043.stdout:8/588: unlink de/d18/d27/d41/d5a/d37/f60 0 2024-09-16T23:48:58.511 INFO:tasks.workunit.client.0.smithi042.stdout:0/313: link d0/df/f18 d0/df/d23/d33/d56/f5d 0 2024-09-16T23:48:58.512 INFO:tasks.workunit.client.1.smithi043.stdout:6/555: creat de/d23/fb8 x:0 0 0 2024-09-16T23:48:58.512 INFO:tasks.workunit.client.0.smithi042.stdout:1/513: fsync d0/d9/f1c 0 2024-09-16T23:48:58.512 INFO:tasks.workunit.client.0.smithi042.stdout:0/314: stat d0/df/l41 0 2024-09-16T23:48:58.513 INFO:tasks.workunit.client.1.smithi043.stdout:1/674: dwrite d4/d9/d12/d2b/d3d/d83/fc4 [0,4194304] 0 2024-09-16T23:48:58.513 INFO:tasks.workunit.client.1.smithi043.stdout:6/556: stat de/d1a/f41 0 2024-09-16T23:48:58.513 INFO:tasks.workunit.client.0.smithi042.stdout:2/443: dread d0/d30/f38 [0,4194304] 0 2024-09-16T23:48:58.514 INFO:tasks.workunit.client.0.smithi042.stdout:2/444: dread - d0/f92 zero size 2024-09-16T23:48:58.518 INFO:tasks.workunit.client.0.smithi042.stdout:2/445: truncate d0/f1a 3207125 0 2024-09-16T23:48:58.519 INFO:tasks.workunit.client.1.smithi043.stdout:3/586: link d4/de/d40/d4d/l5e d4/de/d40/laf 0 2024-09-16T23:48:58.519 INFO:tasks.workunit.client.0.smithi042.stdout:0/315: rename d0/c3 to d0/df/d13/d49/c5e 0 2024-09-16T23:48:58.520 INFO:tasks.workunit.client.1.smithi043.stdout:6/557: dwrite de/d1a/f41 [4194304,4194304] 0 2024-09-16T23:48:58.521 INFO:tasks.workunit.client.0.smithi042.stdout:0/316: chown d0/df/d13/d49/l4f 57 1 2024-09-16T23:48:58.522 INFO:tasks.workunit.client.0.smithi042.stdout:1/514: mkdir d0/d9/dba 0 2024-09-16T23:48:58.525 INFO:tasks.workunit.client.0.smithi042.stdout:3/490: sync 2024-09-16T23:48:58.525 INFO:tasks.workunit.client.0.smithi042.stdout:7/457: sync 2024-09-16T23:48:58.526 INFO:tasks.workunit.client.0.smithi042.stdout:7/458: dread - d3/f9a zero size 2024-09-16T23:48:58.527 INFO:tasks.workunit.client.0.smithi042.stdout:0/317: read d0/df/d23/d33/d56/f5d [2787235,14314] 0 2024-09-16T23:48:58.530 INFO:tasks.workunit.client.0.smithi042.stdout:2/446: dwrite d0/f95 [0,4194304] 0 2024-09-16T23:48:58.531 INFO:tasks.workunit.client.0.smithi042.stdout:7/459: creat d3/d1f/d49/d78/fa2 x:0 0 0 2024-09-16T23:48:58.534 INFO:tasks.workunit.client.0.smithi042.stdout:1/515: creat d0/d9a/fbb x:0 0 0 2024-09-16T23:48:58.536 INFO:tasks.workunit.client.0.smithi042.stdout:0/318: mknod d0/c5f 0 2024-09-16T23:48:58.536 INFO:tasks.workunit.client.0.smithi042.stdout:7/460: stat d3/d1f/d4f/f5d 0 2024-09-16T23:48:58.539 INFO:tasks.workunit.client.0.smithi042.stdout:1/516: write d0/d9a/fbb [536527,15578] 0 2024-09-16T23:48:58.539 INFO:tasks.workunit.client.0.smithi042.stdout:0/319: chown d0/df/d23/d33/d56/f5d 99 1 2024-09-16T23:48:58.539 INFO:tasks.workunit.client.0.smithi042.stdout:2/447: unlink d0/da/d1b/l97 0 2024-09-16T23:48:58.544 INFO:tasks.workunit.client.0.smithi042.stdout:2/448: symlink d0/da/d5b/l9e 0 2024-09-16T23:48:58.546 INFO:tasks.workunit.client.0.smithi042.stdout:7/461: write d3/f84 [494952,92460] 0 2024-09-16T23:48:58.547 INFO:tasks.workunit.client.0.smithi042.stdout:3/491: dwrite d2/d1b/fac [0,4194304] 0 2024-09-16T23:48:58.547 INFO:tasks.workunit.client.0.smithi042.stdout:2/449: chown d0/da/d17/f42 297825479 1 2024-09-16T23:48:58.548 INFO:tasks.workunit.client.0.smithi042.stdout:1/517: write d0/d9/fb9 [1224103,99099] 0 2024-09-16T23:48:58.550 INFO:tasks.workunit.client.0.smithi042.stdout:3/492: chown d2/c3 0 1 2024-09-16T23:48:58.551 INFO:tasks.workunit.client.0.smithi042.stdout:2/450: creat d0/d26/f9f x:0 0 0 2024-09-16T23:48:58.554 INFO:tasks.workunit.client.0.smithi042.stdout:7/462: creat d3/d98/da0/d24/d28/fa3 x:0 0 0 2024-09-16T23:48:58.554 INFO:tasks.workunit.client.0.smithi042.stdout:0/320: dwrite d0/f7 [0,4194304] 0 2024-09-16T23:48:58.556 INFO:tasks.workunit.client.0.smithi042.stdout:2/451: mkdir d0/d8d/da0 0 2024-09-16T23:48:58.558 INFO:tasks.workunit.client.0.smithi042.stdout:7/463: stat d3/d1f/d49/d76/d19/l4b 0 2024-09-16T23:48:58.558 INFO:tasks.workunit.client.0.smithi042.stdout:0/321: write d0/f37 [529152,56873] 0 2024-09-16T23:48:58.561 INFO:tasks.workunit.client.0.smithi042.stdout:3/493: rename d2/d1b/d1d/d34/d50/d93/f99 to d2/d1b/d1d/d34/d50/d93/d96/fad 0 2024-09-16T23:48:58.561 INFO:tasks.workunit.client.0.smithi042.stdout:1/518: dwrite d0/d9/d90/f3a [0,4194304] 0 2024-09-16T23:48:58.564 INFO:tasks.workunit.client.0.smithi042.stdout:1/519: chown d0/d9/d90/d6b/d88/d63/fa8 0 1 2024-09-16T23:48:58.565 INFO:tasks.workunit.client.0.smithi042.stdout:1/520: read - d0/d17/d5e/f74 zero size 2024-09-16T23:48:58.565 INFO:tasks.workunit.client.0.smithi042.stdout:0/322: creat d0/f60 x:0 0 0 2024-09-16T23:48:58.567 INFO:tasks.workunit.client.0.smithi042.stdout:1/521: mkdir d0/d9/d90/d6b/d88/d32/dbc 0 2024-09-16T23:48:58.568 INFO:tasks.workunit.client.0.smithi042.stdout:3/494: rename d2/d7/d61/d8e/d9d/ca6 to d2/d1b/d1d/d34/cae 0 2024-09-16T23:48:58.570 INFO:tasks.workunit.client.0.smithi042.stdout:1/522: truncate d0/d9/d90/d6b/d88/d63/fa8 344206 0 2024-09-16T23:48:58.571 INFO:tasks.workunit.client.0.smithi042.stdout:0/323: creat d0/df/d23/d33/d56/f61 x:0 0 0 2024-09-16T23:48:58.573 INFO:tasks.workunit.client.0.smithi042.stdout:7/464: rename d3/d98/da0/d24/d28/c29 to d3/d98/da0/d32/d6b/d5b/ca4 0 2024-09-16T23:48:58.576 INFO:tasks.workunit.client.1.smithi043.stdout:1/675: creat d4/d9/d1c/d1e/d42/d5d/d71/fed x:0 0 0 2024-09-16T23:48:58.577 INFO:tasks.workunit.client.1.smithi043.stdout:6/558: symlink de/d23/d34/d9b/d8b/lb9 0 2024-09-16T23:48:58.577 INFO:tasks.workunit.client.1.smithi043.stdout:3/587: write d4/f1b [1729561,2428] 0 2024-09-16T23:48:58.579 INFO:tasks.workunit.client.0.smithi042.stdout:1/523: getdents d0/d17/d79/d92 0 2024-09-16T23:48:58.579 INFO:tasks.workunit.client.0.smithi042.stdout:2/452: write d0/da/f23 [773330,74710] 0 2024-09-16T23:48:58.579 INFO:tasks.workunit.client.0.smithi042.stdout:3/495: link d2/f1f d2/d1b/d1d/d34/d62/d80/faf 0 2024-09-16T23:48:58.579 INFO:tasks.workunit.client.0.smithi042.stdout:1/524: readlink d0/l86 0 2024-09-16T23:48:58.579 INFO:tasks.workunit.client.0.smithi042.stdout:1/525: fdatasync d0/d17/fb8 0 2024-09-16T23:48:58.579 INFO:tasks.workunit.client.0.smithi042.stdout:2/453: read - d0/d26/f84 zero size 2024-09-16T23:48:58.582 INFO:tasks.workunit.client.0.smithi042.stdout:3/496: mknod d2/cb0 0 2024-09-16T23:48:58.582 INFO:tasks.workunit.client.0.smithi042.stdout:1/526: creat d0/d9a/fbd x:0 0 0 2024-09-16T23:48:58.585 INFO:tasks.workunit.client.1.smithi043.stdout:6/559: truncate de/d1a/f20 1276223 0 2024-09-16T23:48:58.586 INFO:tasks.workunit.client.0.smithi042.stdout:3/497: creat d2/d1b/d1d/d34/d62/d6a/fb1 x:0 0 0 2024-09-16T23:48:58.586 INFO:tasks.workunit.client.1.smithi043.stdout:6/560: stat de/d23/d34/d66/fac 0 2024-09-16T23:48:58.587 INFO:tasks.workunit.client.1.smithi043.stdout:6/561: chown de/l59 16943 1 2024-09-16T23:48:58.590 INFO:tasks.workunit.client.0.smithi042.stdout:2/454: creat d0/da/d17/d3e/d86/fa1 x:0 0 0 2024-09-16T23:48:58.591 INFO:tasks.workunit.client.0.smithi042.stdout:3/498: getdents d2/d1b/d1d/d30 0 2024-09-16T23:48:58.591 INFO:tasks.workunit.client.0.smithi042.stdout:7/465: dread d3/d1f/f23 [0,4194304] 0 2024-09-16T23:48:58.593 INFO:tasks.workunit.client.0.smithi042.stdout:3/499: fdatasync d2/d1b/d89/f23 0 2024-09-16T23:48:58.597 INFO:tasks.workunit.client.0.smithi042.stdout:1/527: mknod d0/d17/d5e/cbe 0 2024-09-16T23:48:58.597 INFO:tasks.workunit.client.1.smithi043.stdout:6/562: symlink de/d23/d34/d9b/d50/lba 0 2024-09-16T23:48:58.599 INFO:tasks.workunit.client.0.smithi042.stdout:3/500: write d2/d7/d61/d8e/d2c/f64 [3805596,44188] 0 2024-09-16T23:48:58.600 INFO:tasks.workunit.client.1.smithi043.stdout:6/563: read de/d23/d34/d66/fac [65278,75084] 0 2024-09-16T23:48:58.601 INFO:tasks.workunit.client.0.smithi042.stdout:1/528: truncate d0/d9/d4e/f7e 720049 0 2024-09-16T23:48:58.602 INFO:tasks.workunit.client.1.smithi043.stdout:3/588: dwrite d4/de/d1f/d2c/f4a [0,4194304] 0 2024-09-16T23:48:58.602 INFO:tasks.workunit.client.1.smithi043.stdout:1/676: link d4/d9/d1c/d1e/f66 d4/d9/d12/d2b/dc9/fee 0 2024-09-16T23:48:58.603 INFO:tasks.workunit.client.0.smithi042.stdout:2/455: dwrite d0/da/d17/d3e/d86/fa1 [0,4194304] 0 2024-09-16T23:48:58.604 INFO:tasks.workunit.client.0.smithi042.stdout:7/466: dread d3/d1f/d49/d76/f1c [0,4194304] 0 2024-09-16T23:48:58.605 INFO:tasks.workunit.client.0.smithi042.stdout:1/529: mknod d0/d72/cbf 0 2024-09-16T23:48:58.606 INFO:tasks.workunit.client.1.smithi043.stdout:3/589: mknod d4/de/d35/cb0 0 2024-09-16T23:48:58.608 INFO:tasks.workunit.client.1.smithi043.stdout:6/564: truncate de/d1a/d39/f17 2794640 0 2024-09-16T23:48:58.609 INFO:tasks.workunit.client.0.smithi042.stdout:7/467: mkdir d3/d98/da0/d32/d6b/da5 0 2024-09-16T23:48:58.610 INFO:tasks.workunit.client.1.smithi043.stdout:3/590: mkdir d4/d30/d53/d88/db1 0 2024-09-16T23:48:58.611 INFO:tasks.workunit.client.1.smithi043.stdout:1/677: symlink d4/df/d13/ddf/lef 0 2024-09-16T23:48:58.613 INFO:tasks.workunit.client.0.smithi042.stdout:7/468: write d3/d1f/f3d [2693384,122506] 0 2024-09-16T23:48:58.616 INFO:tasks.workunit.client.0.smithi042.stdout:1/530: rename d0/d9/l8a to d0/d9/d4e/lc0 0 2024-09-16T23:48:58.616 INFO:tasks.workunit.client.0.smithi042.stdout:7/469: write d3/d98/da0/d41/f96 [763611,86909] 0 2024-09-16T23:48:58.618 INFO:tasks.workunit.client.0.smithi042.stdout:2/456: dwrite d0/d5d/f79 [0,4194304] 0 2024-09-16T23:48:58.620 INFO:tasks.workunit.client.0.smithi042.stdout:2/457: fsync d0/da/f23 0 2024-09-16T23:48:58.621 INFO:tasks.workunit.client.1.smithi043.stdout:1/678: read d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f30 [2000953,16798] 0 2024-09-16T23:48:58.622 INFO:tasks.workunit.client.1.smithi043.stdout:1/679: chown d4/d9/d1c 0 1 2024-09-16T23:48:58.624 INFO:tasks.workunit.client.1.smithi043.stdout:6/565: rename de/d1a/d28/f5f to de/d23/d34/d9b/d8b/fbb 0 2024-09-16T23:48:58.625 INFO:tasks.workunit.client.1.smithi043.stdout:3/591: mkdir d4/d30/d53/d88/db1/db2 0 2024-09-16T23:48:58.625 INFO:tasks.workunit.client.0.smithi042.stdout:7/470: creat d3/d1f/d49/d76/d19/d63/fa6 x:0 0 0 2024-09-16T23:48:58.629 INFO:tasks.workunit.client.0.smithi042.stdout:7/471: fdatasync d3/d98/da0/d24/f6e 0 2024-09-16T23:48:58.629 INFO:tasks.workunit.client.0.smithi042.stdout:2/458: dread d0/d30/f38 [0,4194304] 0 2024-09-16T23:48:58.629 INFO:tasks.workunit.client.0.smithi042.stdout:3/501: dread d2/d9/faa [0,4194304] 0 2024-09-16T23:48:58.632 INFO:tasks.workunit.client.1.smithi043.stdout:6/566: dwrite de/d23/fb8 [0,4194304] 0 2024-09-16T23:48:58.632 INFO:tasks.workunit.client.0.smithi042.stdout:3/502: stat d2/d7/d61/d8e/d2c/f64 0 2024-09-16T23:48:58.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/531: rename d0/d9/d90/c53 to d0/d9/d90/cc1 0 2024-09-16T23:48:58.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/532: chown d0/d17/d79 298474 1 2024-09-16T23:48:58.633 INFO:tasks.workunit.client.0.smithi042.stdout:7/472: symlink d3/d98/da0/d24/la7 0 2024-09-16T23:48:58.634 INFO:tasks.workunit.client.0.smithi042.stdout:3/503: creat d2/d1b/fb2 x:0 0 0 2024-09-16T23:48:58.637 INFO:tasks.workunit.client.0.smithi042.stdout:2/459: mknod d0/da/d1b/d48/d81/ca2 0 2024-09-16T23:48:58.637 INFO:tasks.workunit.client.0.smithi042.stdout:3/504: mkdir d2/d1b/d1d/db3 0 2024-09-16T23:48:58.640 INFO:tasks.workunit.client.0.smithi042.stdout:2/460: truncate d0/da/d5b/f6b 495190 0 2024-09-16T23:48:58.640 INFO:tasks.workunit.client.0.smithi042.stdout:7/473: getdents d3/d3f 0 2024-09-16T23:48:58.641 INFO:tasks.workunit.client.1.smithi043.stdout:6/567: dread - de/d23/d34/f5d zero size 2024-09-16T23:48:58.642 INFO:tasks.workunit.client.0.smithi042.stdout:7/474: chown d3/d98/da0/d32/c81 0 1 2024-09-16T23:48:58.643 INFO:tasks.workunit.client.0.smithi042.stdout:1/533: rename d0/d9/d90/d6b/d88/d32/dbc to d0/d9/d90/d6b/d88/dc2 0 2024-09-16T23:48:58.645 INFO:tasks.workunit.client.0.smithi042.stdout:2/461: mknod d0/d8d/ca3 0 2024-09-16T23:48:58.647 INFO:tasks.workunit.client.1.smithi043.stdout:6/568: dwrite de/f13 [4194304,4194304] 0 2024-09-16T23:48:58.647 INFO:tasks.workunit.client.0.smithi042.stdout:3/505: rename d2/d9b to d2/d1b/d89/db4 0 2024-09-16T23:48:58.648 INFO:tasks.workunit.client.0.smithi042.stdout:7/475: mknod d3/d98/da0/d32/d6b/d64/d92/ca8 0 2024-09-16T23:48:58.649 INFO:tasks.workunit.client.0.smithi042.stdout:1/534: mknod d0/d72/cc3 0 2024-09-16T23:48:58.650 INFO:tasks.workunit.client.0.smithi042.stdout:3/506: write d2/d7/d61/d73/f9a [301088,119520] 0 2024-09-16T23:48:58.652 INFO:tasks.workunit.client.0.smithi042.stdout:3/507: dread - d2/d1b/d89/f47 zero size 2024-09-16T23:48:58.652 INFO:tasks.workunit.client.0.smithi042.stdout:2/462: dread d0/f95 [0,4194304] 0 2024-09-16T23:48:58.653 INFO:tasks.workunit.client.0.smithi042.stdout:1/535: readlink d0/d17/d79/d92/l97 0 2024-09-16T23:48:58.656 INFO:tasks.workunit.client.1.smithi043.stdout:1/680: write d4/d9/d12/d2b/dc9/fee [3665983,58736] 0 2024-09-16T23:48:58.658 INFO:tasks.workunit.client.1.smithi043.stdout:1/681: chown d4/d9/d1c/d1e/d47/l49 119 1 2024-09-16T23:48:58.658 INFO:tasks.workunit.client.0.smithi042.stdout:2/463: read d0/da/d1b/d3b/d44/d54/f5e [382746,13581] 0 2024-09-16T23:48:58.660 INFO:tasks.workunit.client.1.smithi043.stdout:3/592: truncate d4/de/d40/f69 1880196 0 2024-09-16T23:48:58.660 INFO:tasks.workunit.client.1.smithi043.stdout:6/569: dread de/f67 [0,4194304] 0 2024-09-16T23:48:58.661 INFO:tasks.workunit.client.0.smithi042.stdout:3/508: rename d2/d7/d61/d73/l76 to d2/d7/d61/lb5 0 2024-09-16T23:48:58.663 INFO:tasks.workunit.client.0.smithi042.stdout:7/476: dwrite d3/d98/f7d [0,4194304] 0 2024-09-16T23:48:58.666 INFO:tasks.workunit.client.0.smithi042.stdout:7/477: write d3/f30 [636219,45020] 0 2024-09-16T23:48:58.666 INFO:tasks.workunit.client.0.smithi042.stdout:1/536: getdents d0/d9/d90/d6b/d8e 0 2024-09-16T23:48:58.668 INFO:tasks.workunit.client.1.smithi043.stdout:3/593: dread d4/de/d40/f82 [0,4194304] 0 2024-09-16T23:48:58.670 INFO:tasks.workunit.client.1.smithi043.stdout:6/570: rename de/d1a/d39/la1 to de/d23/d34/d66/d9d/lbc 0 2024-09-16T23:48:58.670 INFO:tasks.workunit.client.0.smithi042.stdout:3/509: symlink d2/d1b/d1d/d30/da2/lb6 0 2024-09-16T23:48:58.670 INFO:tasks.workunit.client.0.smithi042.stdout:7/478: chown d3/d1f/d49/d76 2028062 1 2024-09-16T23:48:58.670 INFO:tasks.workunit.client.0.smithi042.stdout:2/464: dwrite d0/d26/f9f [0,4194304] 0 2024-09-16T23:48:58.674 INFO:tasks.workunit.client.1.smithi043.stdout:3/594: creat d4/d30/d53/d88/fb3 x:0 0 0 2024-09-16T23:48:58.676 INFO:tasks.workunit.client.0.smithi042.stdout:3/510: write d2/d1b/d1d/d34/d50/d93/d96/fad [192667,75022] 0 2024-09-16T23:48:58.679 INFO:tasks.workunit.client.0.smithi042.stdout:7/479: creat d3/d98/da0/d24/d28/d9b/fa9 x:0 0 0 2024-09-16T23:48:58.680 INFO:tasks.workunit.client.0.smithi042.stdout:3/511: mknod d2/d7/d61/d8e/d2c/cb7 0 2024-09-16T23:48:58.681 INFO:tasks.workunit.client.0.smithi042.stdout:2/465: creat d0/da/d88/fa4 x:0 0 0 2024-09-16T23:48:58.683 INFO:tasks.workunit.client.0.smithi042.stdout:2/466: write d0/f2b [2667979,38574] 0 2024-09-16T23:48:58.684 INFO:tasks.workunit.client.0.smithi042.stdout:7/480: symlink d3/d1f/d49/d76/d19/d63/laa 0 2024-09-16T23:48:58.686 INFO:tasks.workunit.client.0.smithi042.stdout:3/512: rename d2/d7/d61/d8e/d7f/l6f to d2/d1b/d89/db4/lb8 0 2024-09-16T23:48:58.686 INFO:tasks.workunit.client.0.smithi042.stdout:7/481: dread - d3/d1f/d49/f8d zero size 2024-09-16T23:48:58.689 INFO:tasks.workunit.client.0.smithi042.stdout:2/467: symlink d0/da/d1b/la5 0 2024-09-16T23:48:58.689 INFO:tasks.workunit.client.0.smithi042.stdout:3/513: mkdir d2/d7/d61/db9 0 2024-09-16T23:48:58.689 INFO:tasks.workunit.client.0.smithi042.stdout:7/482: rmdir d3/d98/da0/d32/d6b/d64/d92 39 2024-09-16T23:48:58.692 INFO:tasks.workunit.client.0.smithi042.stdout:1/537: dwrite d0/d9/d90/d6b/d88/d32/d2f/f50 [0,4194304] 0 2024-09-16T23:48:58.692 INFO:tasks.workunit.client.0.smithi042.stdout:7/483: creat d3/d3f/fab x:0 0 0 2024-09-16T23:48:58.693 INFO:tasks.workunit.client.0.smithi042.stdout:1/538: chown d0/d9a 0 1 2024-09-16T23:48:58.697 INFO:tasks.workunit.client.0.smithi042.stdout:7/484: creat d3/d98/da0/d24/d28/d9b/fac x:0 0 0 2024-09-16T23:48:58.699 INFO:tasks.workunit.client.0.smithi042.stdout:7/485: write d3/fc [4427419,7232] 0 2024-09-16T23:48:58.700 INFO:tasks.workunit.client.0.smithi042.stdout:1/539: link d0/d72/c5b d0/d9/d90/cc4 0 2024-09-16T23:48:58.701 INFO:tasks.workunit.client.0.smithi042.stdout:2/468: dwrite d0/da/d17/d3e/d86/fa1 [0,4194304] 0 2024-09-16T23:48:58.705 INFO:tasks.workunit.client.0.smithi042.stdout:7/486: dread - d3/d98/da0/d32/d6b/d5b/f7a zero size 2024-09-16T23:48:58.705 INFO:tasks.workunit.client.0.smithi042.stdout:1/540: creat d0/d9/dba/fc5 x:0 0 0 2024-09-16T23:48:58.705 INFO:tasks.workunit.client.0.smithi042.stdout:6/370: sync 2024-09-16T23:48:58.705 INFO:tasks.workunit.client.0.smithi042.stdout:9/292: sync 2024-09-16T23:48:58.705 INFO:tasks.workunit.client.0.smithi042.stdout:2/469: symlink d0/da/d17/la6 0 2024-09-16T23:48:58.707 INFO:tasks.workunit.client.0.smithi042.stdout:3/514: truncate d2/d7/d61/d8e/f3f 1322815 0 2024-09-16T23:48:58.711 INFO:tasks.workunit.client.0.smithi042.stdout:2/470: dread d0/d30/d43/f89 [0,4194304] 0 2024-09-16T23:48:58.712 INFO:tasks.workunit.client.0.smithi042.stdout:3/515: mknod d2/d7/d61/d8e/d2c/cba 0 2024-09-16T23:48:58.716 INFO:tasks.workunit.client.0.smithi042.stdout:9/293: dread d4/d12/d2d/d37/d3a/f48 [0,4194304] 0 2024-09-16T23:48:58.716 INFO:tasks.workunit.client.0.smithi042.stdout:2/471: chown d0/da/d1b/d48/d81/l7f 228834 1 2024-09-16T23:48:58.716 INFO:tasks.workunit.client.0.smithi042.stdout:6/371: dwrite d8/db/dd/f1f [0,4194304] 0 2024-09-16T23:48:58.719 INFO:tasks.workunit.client.0.smithi042.stdout:3/516: write d2/d1b/d1d/d34/d62/d6a/fa5 [697243,101309] 0 2024-09-16T23:48:58.719 INFO:tasks.workunit.client.0.smithi042.stdout:1/541: dwrite d0/d9/d90/d6b/d88/f75 [0,4194304] 0 2024-09-16T23:48:58.720 INFO:tasks.workunit.client.0.smithi042.stdout:9/294: readlink d4/d12/l2f 0 2024-09-16T23:48:58.722 INFO:tasks.workunit.client.0.smithi042.stdout:1/542: write d0/d9a/fb5 [355175,62193] 0 2024-09-16T23:48:58.723 INFO:tasks.workunit.client.0.smithi042.stdout:3/517: write d2/d1b/d1d/f40 [3331875,56874] 0 2024-09-16T23:48:58.725 INFO:tasks.workunit.client.0.smithi042.stdout:9/295: dread d4/d10/f57 [4194304,4194304] 0 2024-09-16T23:48:58.732 INFO:tasks.workunit.client.0.smithi042.stdout:9/296: mknod d4/d12/d2d/d37/d59/c5a 0 2024-09-16T23:48:58.734 INFO:tasks.workunit.client.0.smithi042.stdout:3/518: dwrite d2/d1b/d1d/f38 [0,4194304] 0 2024-09-16T23:48:58.737 INFO:tasks.workunit.client.0.smithi042.stdout:2/472: dread d0/d5d/f8c [0,4194304] 0 2024-09-16T23:48:58.737 INFO:tasks.workunit.client.0.smithi042.stdout:9/297: creat d4/d12/d2d/d37/d3a/f5b x:0 0 0 2024-09-16T23:48:58.741 INFO:tasks.workunit.client.0.smithi042.stdout:3/519: chown d2/d1b/d1d/l5e 192 1 2024-09-16T23:48:58.744 INFO:tasks.workunit.client.0.smithi042.stdout:9/298: creat d4/d12/d2d/d37/d3a/f5c x:0 0 0 2024-09-16T23:48:58.744 INFO:tasks.workunit.client.0.smithi042.stdout:3/520: fsync d2/d7/fa4 0 2024-09-16T23:48:58.745 INFO:tasks.workunit.client.0.smithi042.stdout:1/543: dwrite d0/f6 [0,4194304] 0 2024-09-16T23:48:58.746 INFO:tasks.workunit.client.1.smithi043.stdout:6/571: mkdir de/d23/d34/d9b/d8b/dbd 0 2024-09-16T23:48:58.746 INFO:tasks.workunit.client.1.smithi043.stdout:1/682: getdents d4/d9/d72 0 2024-09-16T23:48:58.748 INFO:tasks.workunit.client.1.smithi043.stdout:1/683: chown d4/d9/d8f/db6/dcc/dda 321523 1 2024-09-16T23:48:58.748 INFO:tasks.workunit.client.0.smithi042.stdout:2/473: rename d0/da/d17/f4c to d0/da/d5b/fa7 0 2024-09-16T23:48:58.752 INFO:tasks.workunit.client.0.smithi042.stdout:1/544: chown d0/d9/d90/d6b/cb1 5 1 2024-09-16T23:48:58.752 INFO:tasks.workunit.client.1.smithi043.stdout:6/572: truncate de/d1a/f42 4683287 0 2024-09-16T23:48:58.754 INFO:tasks.workunit.client.1.smithi043.stdout:6/573: write de/d23/d63/fb1 [4468492,122045] 0 2024-09-16T23:48:58.758 INFO:tasks.workunit.client.0.smithi042.stdout:9/299: dwrite d4/d12/d2d/d37/d3a/f3d [0,4194304] 0 2024-09-16T23:48:58.760 INFO:tasks.workunit.client.0.smithi042.stdout:3/521: dwrite d2/d7/d61/f90 [0,4194304] 0 2024-09-16T23:48:58.764 INFO:tasks.workunit.client.0.smithi042.stdout:2/474: link d0/f3a d0/da/d17/d5c/fa8 0 2024-09-16T23:48:58.764 INFO:tasks.workunit.client.0.smithi042.stdout:3/522: chown d2/f1f 36591 1 2024-09-16T23:48:58.766 INFO:tasks.workunit.client.1.smithi043.stdout:3/595: dread d4/de/d35/f1d [0,4194304] 0 2024-09-16T23:48:58.767 INFO:tasks.workunit.client.0.smithi042.stdout:2/475: write d0/da/d1b/f29 [406629,107562] 0 2024-09-16T23:48:58.768 INFO:tasks.workunit.client.0.smithi042.stdout:3/523: truncate d2/d7/f33 3622872 0 2024-09-16T23:48:58.768 INFO:tasks.workunit.client.1.smithi043.stdout:6/574: dread de/d23/d49/f7c [0,4194304] 0 2024-09-16T23:48:58.771 INFO:tasks.workunit.client.1.smithi043.stdout:1/684: write d4/d9/f46 [347847,82716] 0 2024-09-16T23:48:58.773 INFO:tasks.workunit.client.0.smithi042.stdout:3/524: chown d2/d1b/d89/db4/la9 8 1 2024-09-16T23:48:58.774 INFO:tasks.workunit.client.1.smithi043.stdout:1/685: readlink d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/l77 0 2024-09-16T23:48:58.774 INFO:tasks.workunit.client.1.smithi043.stdout:3/596: dwrite d4/de/d1f/d2c/f32 [0,4194304] 0 2024-09-16T23:48:58.775 INFO:tasks.workunit.client.0.smithi042.stdout:2/476: symlink d0/d78/la9 0 2024-09-16T23:48:58.775 INFO:tasks.workunit.client.0.smithi042.stdout:9/300: dwrite d4/f19 [4194304,4194304] 0 2024-09-16T23:48:58.776 INFO:tasks.workunit.client.0.smithi042.stdout:1/545: write d0/d3b/f44 [221287,126593] 0 2024-09-16T23:48:58.778 INFO:tasks.workunit.client.0.smithi042.stdout:1/546: readlink d0/d9/l3d 0 2024-09-16T23:48:58.780 INFO:tasks.workunit.client.0.smithi042.stdout:9/301: stat d4/d12/d2d/d37/d3a/f5b 0 2024-09-16T23:48:58.781 INFO:tasks.workunit.client.0.smithi042.stdout:9/302: dread - d4/d10/f30 zero size 2024-09-16T23:48:58.783 INFO:tasks.workunit.client.0.smithi042.stdout:3/525: write d2/d7/f36 [784167,119195] 0 2024-09-16T23:48:58.785 INFO:tasks.workunit.client.1.smithi043.stdout:1/686: rename d4/df/d13/le6 to d4/d3e/lf0 0 2024-09-16T23:48:58.786 INFO:tasks.workunit.client.1.smithi043.stdout:6/575: mkdir de/dbe 0 2024-09-16T23:48:58.786 INFO:tasks.workunit.client.1.smithi043.stdout:6/576: dread - de/d1a/d28/fa5 zero size 2024-09-16T23:48:58.789 INFO:tasks.workunit.client.1.smithi043.stdout:3/597: mknod d4/de/cb4 0 2024-09-16T23:48:58.791 INFO:tasks.workunit.client.0.smithi042.stdout:9/303: rename d4/d12/d2d/d37/l2a to d4/d12/d2d/d37/d28/l5d 0 2024-09-16T23:48:58.792 INFO:tasks.workunit.client.1.smithi043.stdout:1/687: dread d4/d9/d12/f5c [0,4194304] 0 2024-09-16T23:48:58.792 INFO:tasks.workunit.client.0.smithi042.stdout:1/547: dwrite d0/d9a/fbd [0,4194304] 0 2024-09-16T23:48:58.793 INFO:tasks.workunit.client.0.smithi042.stdout:9/304: chown d4/d10/f1b 174420 1 2024-09-16T23:48:58.793 INFO:tasks.workunit.client.0.smithi042.stdout:2/477: mknod d0/da/d17/d3e/d86/caa 0 2024-09-16T23:48:58.797 INFO:tasks.workunit.client.0.smithi042.stdout:1/548: write d0/d9/d90/d6b/d88/d32/fb6 [351026,60415] 0 2024-09-16T23:48:58.799 INFO:tasks.workunit.client.0.smithi042.stdout:9/305: mkdir d4/d12/d2d/d37/d28/d5e 0 2024-09-16T23:48:58.800 INFO:tasks.workunit.client.0.smithi042.stdout:3/526: truncate d2/f28 500718 0 2024-09-16T23:48:58.802 INFO:tasks.workunit.client.0.smithi042.stdout:2/478: dread d0/da/d1b/d48/d81/f80 [0,4194304] 0 2024-09-16T23:48:58.803 INFO:tasks.workunit.client.0.smithi042.stdout:2/479: readlink d0/da/d17/d5c/l74 0 2024-09-16T23:48:58.803 INFO:tasks.workunit.client.0.smithi042.stdout:3/527: write d2/d1b/d1d/f84 [830657,118804] 0 2024-09-16T23:48:58.804 INFO:tasks.workunit.client.0.smithi042.stdout:1/549: fdatasync d0/d9/f94 0 2024-09-16T23:48:58.807 INFO:tasks.workunit.client.0.smithi042.stdout:9/306: mknod d4/d12/d44/c5f 0 2024-09-16T23:48:58.808 INFO:tasks.workunit.client.0.smithi042.stdout:3/528: dread d2/d1b/d89/f3c [0,4194304] 0 2024-09-16T23:48:58.810 INFO:tasks.workunit.client.0.smithi042.stdout:3/529: creat d2/d1b/d1d/db3/fbb x:0 0 0 2024-09-16T23:48:58.812 INFO:tasks.workunit.client.0.smithi042.stdout:3/530: creat d2/d1b/d89/fbc x:0 0 0 2024-09-16T23:48:58.812 INFO:tasks.workunit.client.0.smithi042.stdout:1/550: mkdir d0/d9/d26/dc6 0 2024-09-16T23:48:58.813 INFO:tasks.workunit.client.0.smithi042.stdout:3/531: stat d2/d1b/d1d/d34/d62/l7b 0 2024-09-16T23:48:58.823 INFO:tasks.workunit.client.1.smithi043.stdout:3/598: creat d4/de/d40/fb5 x:0 0 0 2024-09-16T23:48:58.823 INFO:tasks.workunit.client.1.smithi043.stdout:1/688: mknod d4/d9/d1c/d1e/d42/cf1 0 2024-09-16T23:48:58.826 INFO:tasks.workunit.client.1.smithi043.stdout:1/689: rmdir d4/d3e/d95 39 2024-09-16T23:48:58.828 INFO:tasks.workunit.client.1.smithi043.stdout:1/690: chown d4/d9/d12/d2b/d3d/dc8/cd3 645 1 2024-09-16T23:48:58.828 INFO:tasks.workunit.client.1.smithi043.stdout:6/577: dread de/d1a/f20 [0,4194304] 0 2024-09-16T23:48:58.828 INFO:tasks.workunit.client.1.smithi043.stdout:3/599: link d4/de/d1f/d2c/f32 d4/de/d40/fb6 0 2024-09-16T23:48:58.830 INFO:tasks.workunit.client.1.smithi043.stdout:1/691: write d4/d9/d1c/d1e/d42/f81 [248398,51824] 0 2024-09-16T23:48:58.832 INFO:tasks.workunit.client.1.smithi043.stdout:3/600: mkdir d4/de/d1f/d2c/db7 0 2024-09-16T23:48:58.834 INFO:tasks.workunit.client.1.smithi043.stdout:6/578: mkdir de/dbf 0 2024-09-16T23:48:58.835 INFO:tasks.workunit.client.1.smithi043.stdout:1/692: dread - d4/d9/d1c/d1e/d47/fab zero size 2024-09-16T23:48:58.838 INFO:tasks.workunit.client.1.smithi043.stdout:6/579: creat de/d23/d34/d9b/d5a/fc0 x:0 0 0 2024-09-16T23:48:58.840 INFO:tasks.workunit.client.1.smithi043.stdout:6/580: read - de/d23/d34/d9b/d5a/da3/fb0 zero size 2024-09-16T23:48:58.840 INFO:tasks.workunit.client.1.smithi043.stdout:1/693: dwrite d4/d9/d12/fce [0,4194304] 0 2024-09-16T23:48:58.843 INFO:tasks.workunit.client.1.smithi043.stdout:3/601: creat d4/fb8 x:0 0 0 2024-09-16T23:48:58.843 INFO:tasks.workunit.client.1.smithi043.stdout:3/602: dread - d4/fb8 zero size 2024-09-16T23:48:58.845 INFO:tasks.workunit.client.1.smithi043.stdout:1/694: truncate d4/d9/d1c/d1e/d42/f70 314476 0 2024-09-16T23:48:58.845 INFO:tasks.workunit.client.1.smithi043.stdout:1/695: chown d4/d9/d12/d2b/d3d 130997 1 2024-09-16T23:48:58.849 INFO:tasks.workunit.client.1.smithi043.stdout:6/581: creat de/d23/fc1 x:0 0 0 2024-09-16T23:48:58.855 INFO:tasks.workunit.client.1.smithi043.stdout:6/582: creat de/d23/d34/d66/d9d/dae/fc2 x:0 0 0 2024-09-16T23:48:58.856 INFO:tasks.workunit.client.1.smithi043.stdout:3/603: rename d4/de/fa2 to d4/fb9 0 2024-09-16T23:48:58.860 INFO:tasks.workunit.client.1.smithi043.stdout:1/696: rename d4/f94 to d4/d9/d12/d15/dac/ff2 0 2024-09-16T23:48:58.861 INFO:tasks.workunit.client.1.smithi043.stdout:3/604: creat d4/de/d40/d86/fba x:0 0 0 2024-09-16T23:48:58.863 INFO:tasks.workunit.client.1.smithi043.stdout:6/583: mkdir de/d23/dc3 0 2024-09-16T23:48:58.864 INFO:tasks.workunit.client.1.smithi043.stdout:6/584: truncate de/d1a/fb3 913034 0 2024-09-16T23:48:58.865 INFO:tasks.workunit.client.1.smithi043.stdout:6/585: readlink de/d1a/d28/l5c 0 2024-09-16T23:48:58.866 INFO:tasks.workunit.client.1.smithi043.stdout:1/697: dread d4/d34/f4d [0,4194304] 0 2024-09-16T23:48:58.867 INFO:tasks.workunit.client.1.smithi043.stdout:6/586: getdents de/dbf 0 2024-09-16T23:48:58.869 INFO:tasks.workunit.client.1.smithi043.stdout:6/587: mknod de/dbf/cc4 0 2024-09-16T23:48:58.876 INFO:tasks.workunit.client.1.smithi043.stdout:6/588: dwrite de/f40 [0,4194304] 0 2024-09-16T23:48:58.877 INFO:tasks.workunit.client.1.smithi043.stdout:6/589: readlink de/l10 0 2024-09-16T23:48:58.880 INFO:tasks.workunit.client.1.smithi043.stdout:6/590: unlink de/d23/d49/faa 0 2024-09-16T23:48:58.882 INFO:tasks.workunit.client.1.smithi043.stdout:1/698: dread d4/d34/f7b [0,4194304] 0 2024-09-16T23:48:58.883 INFO:tasks.workunit.client.1.smithi043.stdout:6/591: symlink de/dbe/lc5 0 2024-09-16T23:48:58.885 INFO:tasks.workunit.client.1.smithi043.stdout:1/699: mknod d4/cf3 0 2024-09-16T23:48:58.892 INFO:tasks.workunit.client.1.smithi043.stdout:6/592: dwrite de/f6c [0,4194304] 0 2024-09-16T23:48:58.892 INFO:tasks.workunit.client.1.smithi043.stdout:1/700: dwrite d4/d9/d12/d2b/dc9/fee [0,4194304] 0 2024-09-16T23:48:58.893 INFO:tasks.workunit.client.1.smithi043.stdout:1/701: chown d4/df/fd1 20079095 1 2024-09-16T23:48:58.901 INFO:tasks.workunit.client.1.smithi043.stdout:1/702: rename d4/d9/d12/d2b/d3d to d4/df/df4 0 2024-09-16T23:48:58.901 INFO:tasks.workunit.client.0.smithi042.stdout:5/528: sync 2024-09-16T23:48:58.901 INFO:tasks.workunit.client.0.smithi042.stdout:4/574: sync 2024-09-16T23:48:58.901 INFO:tasks.workunit.client.0.smithi042.stdout:0/324: sync 2024-09-16T23:48:58.902 INFO:tasks.workunit.client.0.smithi042.stdout:8/323: sync 2024-09-16T23:48:58.902 INFO:tasks.workunit.client.0.smithi042.stdout:5/529: stat d9/db/d11 0 2024-09-16T23:48:58.903 INFO:tasks.workunit.client.0.smithi042.stdout:1/551: mkdir d0/d17/dc7 0 2024-09-16T23:48:58.906 INFO:tasks.workunit.client.0.smithi042.stdout:3/532: write d2/f1f [3392780,118395] 0 2024-09-16T23:48:58.906 INFO:tasks.workunit.client.0.smithi042.stdout:0/325: mkdir d0/df/d62 0 2024-09-16T23:48:58.910 INFO:tasks.workunit.client.1.smithi043.stdout:1/703: rename d4/df/fd1 to d4/d9/d1c/ff5 0 2024-09-16T23:48:58.911 INFO:tasks.workunit.client.0.smithi042.stdout:3/533: write d2/d1b/f94 [888650,19122] 0 2024-09-16T23:48:58.912 INFO:tasks.workunit.client.1.smithi043.stdout:1/704: write d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f64 [1099942,63113] 0 2024-09-16T23:48:58.912 INFO:tasks.workunit.client.0.smithi042.stdout:0/326: fsync d0/df/d23/d33/d51/f59 0 2024-09-16T23:48:58.913 INFO:tasks.workunit.client.0.smithi042.stdout:9/307: unlink d4/l40 0 2024-09-16T23:48:58.914 INFO:tasks.workunit.client.1.smithi043.stdout:1/705: write d4/d9/d1c/d1e/d42/d5d/d71/fe4 [416428,121072] 0 2024-09-16T23:48:58.916 INFO:tasks.workunit.client.0.smithi042.stdout:3/534: readlink d2/d7/d61/d8e/d2c/l72 0 2024-09-16T23:48:58.916 INFO:tasks.workunit.client.0.smithi042.stdout:4/575: dwrite de/d42/d68/fb1 [0,4194304] 0 2024-09-16T23:48:58.920 INFO:tasks.workunit.client.0.smithi042.stdout:4/576: truncate de/d2e/d37/d3e/f7f 1316896 0 2024-09-16T23:48:58.920 INFO:tasks.workunit.client.0.smithi042.stdout:0/327: creat d0/df/d13/f63 x:0 0 0 2024-09-16T23:48:58.922 INFO:tasks.workunit.client.0.smithi042.stdout:1/552: write d0/d9/d90/d6b/d88/f2a [1817942,81542] 0 2024-09-16T23:48:58.923 INFO:tasks.workunit.client.0.smithi042.stdout:4/577: chown de/d42/d68/f72 350215 1 2024-09-16T23:48:58.924 INFO:tasks.workunit.client.0.smithi042.stdout:4/578: readlink de/d1a/l24 0 2024-09-16T23:48:58.925 INFO:tasks.workunit.client.0.smithi042.stdout:0/328: readlink d0/df/d13/d2d/l43 0 2024-09-16T23:48:58.929 INFO:tasks.workunit.client.0.smithi042.stdout:4/579: rename de/d42/d99/caf to de/d2e/d37/d4c/cca 0 2024-09-16T23:48:58.930 INFO:tasks.workunit.client.0.smithi042.stdout:5/530: dread d9/db/d19/d21/d41/d7d/f7e [0,4194304] 0 2024-09-16T23:48:58.932 INFO:tasks.workunit.client.0.smithi042.stdout:4/580: read de/d2e/d37/d4c/d4f/f58 [3614295,105288] 0 2024-09-16T23:48:58.933 INFO:tasks.workunit.client.0.smithi042.stdout:4/581: chown de/d12/f25 2815571 1 2024-09-16T23:48:58.939 INFO:tasks.workunit.client.0.smithi042.stdout:5/531: dread d9/db/d1b/d31/d46/f49 [0,4194304] 0 2024-09-16T23:48:58.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/329: truncate d0/df/d23/f38 471311 0 2024-09-16T23:48:58.943 INFO:tasks.workunit.client.0.smithi042.stdout:5/532: mkdir d9/db/de/d92/db5 0 2024-09-16T23:48:58.944 INFO:tasks.workunit.client.0.smithi042.stdout:5/533: stat d9/db/d1b/d79 0 2024-09-16T23:48:58.947 INFO:tasks.workunit.client.0.smithi042.stdout:0/330: rename d0/df/c2e to d0/df/d23/d33/d51/c64 0 2024-09-16T23:48:58.954 INFO:tasks.workunit.client.0.smithi042.stdout:0/331: rename d0/df/d23/d33/d51/f59 to d0/df/d62/f65 0 2024-09-16T23:48:58.954 INFO:tasks.workunit.client.0.smithi042.stdout:4/582: dread de/d12/f15 [0,4194304] 0 2024-09-16T23:48:58.955 INFO:tasks.workunit.client.0.smithi042.stdout:5/534: dwrite d9/db/d19/d21/d85/da5/f88 [0,4194304] 0 2024-09-16T23:48:58.956 INFO:tasks.workunit.client.0.smithi042.stdout:0/332: write d0/f4b [505374,76364] 0 2024-09-16T23:48:58.960 INFO:tasks.workunit.client.0.smithi042.stdout:4/583: creat de/d2e/d37/d3e/fcb x:0 0 0 2024-09-16T23:48:58.962 INFO:tasks.workunit.client.0.smithi042.stdout:0/333: symlink d0/df/d13/d49/l66 0 2024-09-16T23:48:58.964 INFO:tasks.workunit.client.0.smithi042.stdout:4/584: creat de/d42/d82/fcc x:0 0 0 2024-09-16T23:48:58.966 INFO:tasks.workunit.client.0.smithi042.stdout:4/585: symlink de/d12/db0/lcd 0 2024-09-16T23:48:58.969 INFO:tasks.workunit.client.0.smithi042.stdout:0/334: rename d0/df/d13/f22 to d0/f67 0 2024-09-16T23:48:58.976 INFO:tasks.workunit.client.0.smithi042.stdout:0/335: dread d0/f37 [0,4194304] 0 2024-09-16T23:48:58.980 INFO:tasks.workunit.client.0.smithi042.stdout:0/336: mknod d0/df/c68 0 2024-09-16T23:48:58.983 INFO:tasks.workunit.client.0.smithi042.stdout:4/586: truncate de/d1a/da7/f56 4243453 0 2024-09-16T23:48:58.984 INFO:tasks.workunit.client.0.smithi042.stdout:0/337: symlink d0/l69 0 2024-09-16T23:48:58.984 INFO:tasks.workunit.client.1.smithi043.stdout:1/706: mknod d4/df/cf6 0 2024-09-16T23:48:58.987 INFO:tasks.workunit.client.0.smithi042.stdout:1/553: unlink d0/d9/d90/d6b/d88/d32/d2f/l42 0 2024-09-16T23:48:58.988 INFO:tasks.workunit.client.0.smithi042.stdout:5/535: dwrite d9/db/d1b/f95 [0,4194304] 0 2024-09-16T23:48:58.989 INFO:tasks.workunit.client.0.smithi042.stdout:4/587: symlink de/d2e/d37/d4c/d4f/lce 0 2024-09-16T23:48:58.990 INFO:tasks.workunit.client.1.smithi043.stdout:1/707: dwrite d4/df/d26/d79/fb9 [0,4194304] 0 2024-09-16T23:48:58.995 INFO:tasks.workunit.client.0.smithi042.stdout:9/308: dwrite d4/f5 [0,4194304] 0 2024-09-16T23:48:58.996 INFO:tasks.workunit.client.0.smithi042.stdout:4/588: fdatasync de/d12/f54 0 2024-09-16T23:48:58.999 INFO:tasks.workunit.client.1.smithi043.stdout:1/708: chown d4/d9/d12/d15/dac/ff2 1 1 2024-09-16T23:48:59.000 INFO:tasks.workunit.client.1.smithi043.stdout:2/506: sync 2024-09-16T23:48:59.000 INFO:tasks.workunit.client.1.smithi043.stdout:9/498: sync 2024-09-16T23:48:59.000 INFO:tasks.workunit.client.1.smithi043.stdout:0/567: sync 2024-09-16T23:48:59.001 INFO:tasks.workunit.client.0.smithi042.stdout:0/338: dwrite d0/df/d13/f34 [0,4194304] 0 2024-09-16T23:48:59.003 INFO:tasks.workunit.client.1.smithi043.stdout:1/709: read d4/df/d26/d79/fb9 [3150960,92555] 0 2024-09-16T23:48:59.005 INFO:tasks.workunit.client.0.smithi042.stdout:4/589: unlink de/d42/d82/f8e 0 2024-09-16T23:48:59.006 INFO:tasks.workunit.client.0.smithi042.stdout:5/536: link d9/db/de/d92/fa1 d9/db/fb6 0 2024-09-16T23:48:59.007 INFO:tasks.workunit.client.1.smithi043.stdout:6/593: dwrite de/d23/d34/d9b/fa9 [0,4194304] 0 2024-09-16T23:48:59.008 INFO:tasks.workunit.client.0.smithi042.stdout:4/590: fsync de/d42/d82/d94/fa3 0 2024-09-16T23:48:59.008 INFO:tasks.workunit.client.1.smithi043.stdout:6/594: write de/d23/d49/fb5 [484740,53828] 0 2024-09-16T23:48:59.010 INFO:tasks.workunit.client.0.smithi042.stdout:1/554: dwrite d0/d9/d4e/fa4 [0,4194304] 0 2024-09-16T23:48:59.013 INFO:tasks.workunit.client.0.smithi042.stdout:0/339: read d0/df/d23/f27 [232941,3108] 0 2024-09-16T23:48:59.014 INFO:tasks.workunit.client.0.smithi042.stdout:4/591: read de/d12/d7d/f83 [482802,107957] 0 2024-09-16T23:48:59.017 INFO:tasks.workunit.client.0.smithi042.stdout:0/340: chown d0/d48/f53 211495291 1 2024-09-16T23:48:59.023 INFO:tasks.workunit.client.0.smithi042.stdout:0/341: mkdir d0/df/d13/d6a 0 2024-09-16T23:48:59.026 INFO:tasks.workunit.client.0.smithi042.stdout:4/592: dread de/d2e/d37/f3b [0,4194304] 0 2024-09-16T23:48:59.029 INFO:tasks.workunit.client.0.smithi042.stdout:4/593: readlink de/d12/lbf 0 2024-09-16T23:48:59.031 INFO:tasks.workunit.client.0.smithi042.stdout:5/537: dwrite d9/db/d19/d21/f2b [0,4194304] 0 2024-09-16T23:48:59.032 INFO:tasks.workunit.client.1.smithi043.stdout:1/710: symlink d4/d9/d8f/db6/dcc/dda/lf7 0 2024-09-16T23:48:59.033 INFO:tasks.workunit.client.1.smithi043.stdout:1/711: chown d4/df/df4/dc8/ca6 60957 1 2024-09-16T23:48:59.033 INFO:tasks.workunit.client.0.smithi042.stdout:0/342: creat d0/df/d23/f6b x:0 0 0 2024-09-16T23:48:59.038 INFO:tasks.workunit.client.1.smithi043.stdout:9/499: dread d4/d1c/d4f/d8c/f66 [0,4194304] 0 2024-09-16T23:48:59.039 INFO:tasks.workunit.client.0.smithi042.stdout:1/555: mknod d0/da6/cc8 0 2024-09-16T23:48:59.041 INFO:tasks.workunit.client.0.smithi042.stdout:4/594: dwrite de/d12/f14 [0,4194304] 0 2024-09-16T23:48:59.042 INFO:tasks.workunit.client.1.smithi043.stdout:0/568: dread d0/d2/df/d1a/d46/d39/f6a [0,4194304] 0 2024-09-16T23:48:59.045 INFO:tasks.workunit.client.0.smithi042.stdout:5/538: rename d9/d4a/f70 to d9/db/d1b/d79/fb7 0 2024-09-16T23:48:59.046 INFO:tasks.workunit.client.0.smithi042.stdout:0/343: mknod d0/df/d13/c6c 0 2024-09-16T23:48:59.047 INFO:tasks.workunit.client.0.smithi042.stdout:0/344: read - d0/df/d23/d33/d56/f61 zero size 2024-09-16T23:48:59.047 INFO:tasks.workunit.client.1.smithi043.stdout:6/595: creat de/d23/d34/d9b/d5a/fc6 x:0 0 0 2024-09-16T23:48:59.049 INFO:tasks.workunit.client.1.smithi043.stdout:6/596: readlink de/d23/d34/d9b/d50/lba 0 2024-09-16T23:48:59.049 INFO:tasks.workunit.client.1.smithi043.stdout:2/507: dread d8/d1d/d20/d23/f79 [0,4194304] 0 2024-09-16T23:48:59.050 INFO:tasks.workunit.client.0.smithi042.stdout:0/345: read - d0/df/d23/d33/f5c zero size 2024-09-16T23:48:59.052 INFO:tasks.workunit.client.0.smithi042.stdout:4/595: dread de/d12/f14 [0,4194304] 0 2024-09-16T23:48:59.052 INFO:tasks.workunit.client.1.smithi043.stdout:6/597: read de/d1a/d39/f1f [2607210,122271] 0 2024-09-16T23:48:59.056 INFO:tasks.workunit.client.1.smithi043.stdout:9/500: mknod d4/d28/d5f/d91/c9a 0 2024-09-16T23:48:59.057 INFO:tasks.workunit.client.1.smithi043.stdout:0/569: fsync d0/d2/f33 0 2024-09-16T23:48:59.059 INFO:tasks.workunit.client.0.smithi042.stdout:4/596: dread de/f2f [0,4194304] 0 2024-09-16T23:48:59.061 INFO:tasks.workunit.client.0.smithi042.stdout:4/597: readlink de/d42/d68/l81 0 2024-09-16T23:48:59.062 INFO:tasks.workunit.client.0.smithi042.stdout:5/539: dwrite d9/db/d1b/d31/d6c/f93 [0,4194304] 0 2024-09-16T23:48:59.062 INFO:tasks.workunit.client.1.smithi043.stdout:2/508: unlink d8/d4b/d5f/d8e/c96 0 2024-09-16T23:48:59.063 INFO:tasks.workunit.client.1.smithi043.stdout:2/509: fdatasync d8/d11/d16/f81 0 2024-09-16T23:48:59.064 INFO:tasks.workunit.client.0.smithi042.stdout:5/540: dread - d9/db/de/fb3 zero size 2024-09-16T23:48:59.064 INFO:tasks.workunit.client.0.smithi042.stdout:4/598: stat de/d42/d85/lbb 0 2024-09-16T23:48:59.064 INFO:tasks.workunit.client.1.smithi043.stdout:0/570: dwrite d0/d2/df/d1a/d46/d39/d6e/f65 [0,4194304] 0 2024-09-16T23:48:59.065 INFO:tasks.workunit.client.1.smithi043.stdout:1/712: truncate d4/df/d26/d79/fd7 1497418 0 2024-09-16T23:48:59.071 INFO:tasks.workunit.client.0.smithi042.stdout:5/541: mknod d9/db/d1b/d31/d6c/cb8 0 2024-09-16T23:48:59.072 INFO:tasks.workunit.client.0.smithi042.stdout:5/542: chown d9/db/de/d10/f1a 1807 1 2024-09-16T23:48:59.076 INFO:tasks.workunit.client.1.smithi043.stdout:4/598: sync 2024-09-16T23:48:59.076 INFO:tasks.workunit.client.0.smithi042.stdout:4/599: dwrite de/d42/d82/d94/fa3 [0,4194304] 0 2024-09-16T23:48:59.076 INFO:tasks.workunit.client.0.smithi042.stdout:5/543: write d9/db/de/d10/d2f/f36 [1824950,7562] 0 2024-09-16T23:48:59.078 INFO:tasks.workunit.client.0.smithi042.stdout:0/346: dread d0/df/f14 [0,4194304] 0 2024-09-16T23:48:59.079 INFO:tasks.workunit.client.0.smithi042.stdout:0/347: readlink d0/d15/l30 0 2024-09-16T23:48:59.080 INFO:tasks.workunit.client.0.smithi042.stdout:0/348: readlink d0/df/l58 0 2024-09-16T23:48:59.081 INFO:tasks.workunit.client.0.smithi042.stdout:5/544: stat d9/db/d19/d21/d85/da5/f88 0 2024-09-16T23:48:59.081 INFO:tasks.workunit.client.1.smithi043.stdout:2/510: dread d8/d4b/d5f/f61 [0,4194304] 0 2024-09-16T23:48:59.082 INFO:tasks.workunit.client.1.smithi043.stdout:9/501: creat d4/d21/d33/d95/f9b x:0 0 0 2024-09-16T23:48:59.084 INFO:tasks.workunit.client.1.smithi043.stdout:0/571: dread d0/d2/df/d5a/f63 [0,4194304] 0 2024-09-16T23:48:59.084 INFO:tasks.workunit.client.1.smithi043.stdout:4/599: dwrite d8/d25/f4f [0,4194304] 0 2024-09-16T23:48:59.085 INFO:tasks.workunit.client.1.smithi043.stdout:6/598: creat de/d23/d34/d9b/fc7 x:0 0 0 2024-09-16T23:48:59.085 INFO:tasks.workunit.client.0.smithi042.stdout:0/349: write d0/df/f14 [748220,1376] 0 2024-09-16T23:48:59.087 INFO:tasks.workunit.client.1.smithi043.stdout:0/572: write d0/d2/df/dbb/f50 [2311293,125578] 0 2024-09-16T23:48:59.088 INFO:tasks.workunit.client.1.smithi043.stdout:9/502: write d4/d21/d2a/d2c/f69 [414183,8974] 0 2024-09-16T23:48:59.090 INFO:tasks.workunit.client.1.smithi043.stdout:9/503: chown d4/d1c/d25/d2b/c83 59 1 2024-09-16T23:48:59.091 INFO:tasks.workunit.client.1.smithi043.stdout:9/504: stat d4/d28/d5f/d86/f96 0 2024-09-16T23:48:59.093 INFO:tasks.workunit.client.1.smithi043.stdout:2/511: mkdir d8/d5e/d83/d9e/dab 0 2024-09-16T23:48:59.095 INFO:tasks.workunit.client.0.smithi042.stdout:5/545: link d9/db/de/d10/f20 d9/db/de/d5e/d87/fb9 0 2024-09-16T23:48:59.095 INFO:tasks.workunit.client.1.smithi043.stdout:4/600: chown d8/f1d 94732584 1 2024-09-16T23:48:59.098 INFO:tasks.workunit.client.1.smithi043.stdout:2/512: dread d8/d1f/f6e [0,4194304] 0 2024-09-16T23:48:59.102 INFO:tasks.workunit.client.1.smithi043.stdout:0/573: symlink d0/d2/df/da7/lc0 0 2024-09-16T23:48:59.105 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:58 smithi043 ceph-mon[35191]: pgmap v8: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 151 MiB/s wr, 290 op/s 2024-09-16T23:48:59.106 INFO:tasks.workunit.client.0.smithi042.stdout:5/546: link d9/db/d1b/d31/c80 d9/db/d19/cba 0 2024-09-16T23:48:59.108 INFO:tasks.workunit.client.0.smithi042.stdout:4/600: dwrite de/d1a/f39 [0,4194304] 0 2024-09-16T23:48:59.108 INFO:tasks.workunit.client.0.smithi042.stdout:5/547: write d9/db/de/d5e/d87/d61/f5d [935161,62146] 0 2024-09-16T23:48:59.108 INFO:tasks.workunit.client.1.smithi043.stdout:6/599: dread de/d23/d34/d9b/f98 [0,4194304] 0 2024-09-16T23:48:59.110 INFO:tasks.workunit.client.0.smithi042.stdout:5/548: stat d9/db/de/d10/c18 0 2024-09-16T23:48:59.111 INFO:tasks.workunit.client.0.smithi042.stdout:4/601: write de/d2e/d37/d4c/d4f/f7a [1042408,128796] 0 2024-09-16T23:48:59.114 INFO:tasks.workunit.client.0.smithi042.stdout:5/549: creat d9/db/d1b/d31/d46/d58/fbb x:0 0 0 2024-09-16T23:48:59.116 INFO:tasks.workunit.client.0.smithi042.stdout:4/602: dread de/d42/d82/d94/fa3 [0,4194304] 0 2024-09-16T23:48:59.125 INFO:tasks.workunit.client.0.smithi042.stdout:5/550: truncate d9/db/de/d5e/f67 1344792 0 2024-09-16T23:48:59.125 INFO:tasks.workunit.client.1.smithi043.stdout:4/601: link d8/d25/d39/d58/fb8 d8/fbe 0 2024-09-16T23:48:59.126 INFO:tasks.workunit.client.0.smithi042.stdout:5/551: write d9/db/d19/d21/d85/da5/f88 [1481862,105979] 0 2024-09-16T23:48:59.131 INFO:tasks.workunit.client.0.smithi042.stdout:5/552: read d9/db/d19/d21/f2e [3360397,43290] 0 2024-09-16T23:48:59.132 INFO:tasks.workunit.client.0.smithi042.stdout:5/553: truncate d9/d4a/d89/f8a 301667 0 2024-09-16T23:48:59.132 INFO:tasks.workunit.client.0.smithi042.stdout:4/603: dread de/d2e/d37/d4c/d4f/f6b [0,4194304] 0 2024-09-16T23:48:59.133 INFO:tasks.workunit.client.1.smithi043.stdout:9/505: getdents d4 0 2024-09-16T23:48:59.133 INFO:tasks.workunit.client.1.smithi043.stdout:9/506: readlink d4/l29 0 2024-09-16T23:48:59.134 INFO:tasks.workunit.client.0.smithi042.stdout:4/604: fdatasync de/d2e/d37/d4c/d4f/f7a 0 2024-09-16T23:48:59.136 INFO:tasks.workunit.client.0.smithi042.stdout:5/554: mknod d9/db/de/d10/d2f/cbc 0 2024-09-16T23:48:59.136 INFO:tasks.workunit.client.1.smithi043.stdout:9/507: read d4/f1a [74664,110136] 0 2024-09-16T23:48:59.139 INFO:tasks.workunit.client.1.smithi043.stdout:4/602: dread d8/d15/d20/d2b/f8c [0,4194304] 0 2024-09-16T23:48:59.140 INFO:tasks.workunit.client.0.smithi042.stdout:4/605: creat de/d1a/da7/db9/fcf x:0 0 0 2024-09-16T23:48:59.140 INFO:tasks.workunit.client.1.smithi043.stdout:9/508: creat d4/d28/d5f/d91/f9c x:0 0 0 2024-09-16T23:48:59.141 INFO:tasks.workunit.client.1.smithi043.stdout:9/509: fdatasync d4/d28/d5f/d86/f96 0 2024-09-16T23:48:59.142 INFO:tasks.workunit.client.1.smithi043.stdout:4/603: mkdir d8/d15/d9b/dbf 0 2024-09-16T23:48:59.142 INFO:tasks.workunit.client.0.smithi042.stdout:4/606: creat de/d2e/d37/dc3/fd0 x:0 0 0 2024-09-16T23:48:59.142 INFO:tasks.workunit.client.0.smithi042.stdout:5/555: link d9/db/d19/d21/d41/l9c d9/db/de/d5e/d87/d61/d43/lbd 0 2024-09-16T23:48:59.144 INFO:tasks.workunit.client.0.smithi042.stdout:4/607: fsync de/d12/d7d/fa1 0 2024-09-16T23:48:59.144 INFO:tasks.workunit.client.0.smithi042.stdout:9/309: mkdir d4/d12/d60 0 2024-09-16T23:48:59.145 INFO:tasks.workunit.client.1.smithi043.stdout:9/510: fdatasync d4/d1c/d25/d2b/d4c/f93 0 2024-09-16T23:48:59.145 INFO:tasks.workunit.client.0.smithi042.stdout:4/608: write de/d42/d82/fa5 [276610,103808] 0 2024-09-16T23:48:59.147 INFO:tasks.workunit.client.1.smithi043.stdout:4/604: creat d8/d25/d39/d58/fc0 x:0 0 0 2024-09-16T23:48:59.148 INFO:tasks.workunit.client.1.smithi043.stdout:4/605: chown d8/d15/d20/d2b/lba 116961 1 2024-09-16T23:48:59.148 INFO:tasks.workunit.client.0.smithi042.stdout:4/609: creat de/d2e/d37/d3e/fd1 x:0 0 0 2024-09-16T23:48:59.149 INFO:tasks.workunit.client.1.smithi043.stdout:9/511: chown d4/d28/d49/f76 207 1 2024-09-16T23:48:59.150 INFO:tasks.workunit.client.0.smithi042.stdout:4/610: dread - de/d42/d82/d94/fc9 zero size 2024-09-16T23:48:59.151 INFO:tasks.workunit.client.0.smithi042.stdout:9/310: symlink d4/l61 0 2024-09-16T23:48:59.153 INFO:tasks.workunit.client.0.smithi042.stdout:9/311: chown d4/d12/d2d/d37/c2b 469 1 2024-09-16T23:48:59.153 INFO:tasks.workunit.client.1.smithi043.stdout:4/606: dwrite d8/d15/d20/d2b/f6b [0,4194304] 0 2024-09-16T23:48:59.155 INFO:tasks.workunit.client.1.smithi043.stdout:9/512: mknod d4/d28/d5f/d91/c9d 0 2024-09-16T23:48:59.157 INFO:tasks.workunit.client.0.smithi042.stdout:5/556: dwrite d9/db/de/d10/f20 [4194304,4194304] 0 2024-09-16T23:48:59.157 INFO:tasks.workunit.client.0.smithi042.stdout:4/611: dread de/d1a/f39 [0,4194304] 0 2024-09-16T23:48:59.157 INFO:tasks.workunit.client.1.smithi043.stdout:4/607: symlink d8/d25/d39/d41/lc1 0 2024-09-16T23:48:59.160 INFO:tasks.workunit.client.0.smithi042.stdout:5/557: truncate d9/db/de/d92/fa6 531815 0 2024-09-16T23:48:59.162 INFO:tasks.workunit.client.0.smithi042.stdout:9/312: fdatasync d4/d10/d24/f35 0 2024-09-16T23:48:59.163 INFO:tasks.workunit.client.1.smithi043.stdout:9/513: readlink d4/d1c/d4f/d8c/l40 0 2024-09-16T23:48:59.164 INFO:tasks.workunit.client.1.smithi043.stdout:9/514: fsync d4/d21/d2a/d2c/d7b/f7e 0 2024-09-16T23:48:59.164 INFO:tasks.workunit.client.0.smithi042.stdout:5/558: rmdir d9/db/de/d10 39 2024-09-16T23:48:59.166 INFO:tasks.workunit.client.1.smithi043.stdout:4/608: creat d8/d15/d20/d2b/d22/fc2 x:0 0 0 2024-09-16T23:48:59.167 INFO:tasks.workunit.client.0.smithi042.stdout:1/556: write d0/d17/f21 [4123909,87718] 0 2024-09-16T23:48:59.168 INFO:tasks.workunit.client.0.smithi042.stdout:5/559: symlink d9/db/de/d5e/lbe 0 2024-09-16T23:48:59.168 INFO:tasks.workunit.client.1.smithi043.stdout:4/609: read d8/d15/d20/d2b/d54/f91 [3424434,2320] 0 2024-09-16T23:48:59.170 INFO:tasks.workunit.client.1.smithi043.stdout:4/610: truncate d8/d25/d39/d41/fb9 1048300 0 2024-09-16T23:48:59.171 INFO:tasks.workunit.client.0.smithi042.stdout:1/557: truncate d0/d17/d5e/f74 132629 0 2024-09-16T23:48:59.172 INFO:tasks.workunit.client.0.smithi042.stdout:4/612: dwrite de/d2e/d37/d4c/d4f/f64 [0,4194304] 0 2024-09-16T23:48:59.174 INFO:tasks.workunit.client.0.smithi042.stdout:1/558: write d0/d72/f65 [1849868,101331] 0 2024-09-16T23:48:59.174 INFO:tasks.workunit.client.0.smithi042.stdout:5/560: creat d9/db/d19/d21/d41/d7d/fbf x:0 0 0 2024-09-16T23:48:59.176 INFO:tasks.workunit.client.0.smithi042.stdout:5/561: dread - d9/db/de/d92/fa1 zero size 2024-09-16T23:48:59.181 INFO:tasks.workunit.client.0.smithi042.stdout:5/562: fsync d9/db/f3e 0 2024-09-16T23:48:59.181 INFO:tasks.workunit.client.1.smithi043.stdout:4/611: unlink d8/d15/d6e/lad 0 2024-09-16T23:48:59.182 INFO:tasks.workunit.client.0.smithi042.stdout:1/559: rmdir d0/d72 39 2024-09-16T23:48:59.184 INFO:tasks.workunit.client.0.smithi042.stdout:4/613: rename de/d1a/d70/fa2 to de/d1a/da7/fd2 0 2024-09-16T23:48:59.185 INFO:tasks.workunit.client.1.smithi043.stdout:9/515: rename d4/d21/f6d to d4/d21/d2a/d2c/f9e 0 2024-09-16T23:48:59.187 INFO:tasks.workunit.client.1.smithi043.stdout:4/612: truncate d8/d25/d64/fb4 261829 0 2024-09-16T23:48:59.188 INFO:tasks.workunit.client.0.smithi042.stdout:4/614: creat de/d42/d82/d94/dbd/fd3 x:0 0 0 2024-09-16T23:48:59.189 INFO:tasks.workunit.client.1.smithi043.stdout:9/516: mknod d4/d28/d5f/d91/c9f 0 2024-09-16T23:48:59.190 INFO:tasks.workunit.client.0.smithi042.stdout:5/563: dwrite d9/db/de/f7f [0,4194304] 0 2024-09-16T23:48:59.190 INFO:tasks.workunit.client.0.smithi042.stdout:4/615: rmdir de/d42 39 2024-09-16T23:48:59.194 INFO:tasks.workunit.client.1.smithi043.stdout:1/713: dwrite d4/d9/d1c/d1e/f85 [0,4194304] 0 2024-09-16T23:48:59.195 INFO:tasks.workunit.client.0.smithi042.stdout:1/560: dwrite d0/d9/d90/d6b/d88/d32/d2f/f68 [0,4194304] 0 2024-09-16T23:48:59.195 INFO:tasks.workunit.client.1.smithi043.stdout:9/517: creat d4/d5/d16/fa0 x:0 0 0 2024-09-16T23:48:59.195 INFO:tasks.workunit.client.0.smithi042.stdout:5/564: chown d9/db/d1b/d31/c80 3744476 1 2024-09-16T23:48:59.197 INFO:tasks.workunit.client.0.smithi042.stdout:1/561: readlink d0/d17/l1b 0 2024-09-16T23:48:59.198 INFO:tasks.workunit.client.0.smithi042.stdout:5/565: dread - d9/db/d19/f98 zero size 2024-09-16T23:48:59.198 INFO:tasks.workunit.client.1.smithi043.stdout:0/574: mknod d0/d2/df/cc1 0 2024-09-16T23:48:59.200 INFO:tasks.workunit.client.1.smithi043.stdout:0/575: write d0/de/f78 [660877,120845] 0 2024-09-16T23:48:59.200 INFO:tasks.workunit.client.1.smithi043.stdout:4/613: rmdir d8/d25/d39/d41/d57/dbc 0 2024-09-16T23:48:59.202 INFO:tasks.workunit.client.1.smithi043.stdout:0/576: fsync d0/d2/df/d1a/d46/d39/f62 0 2024-09-16T23:48:59.204 INFO:tasks.workunit.client.1.smithi043.stdout:1/714: rmdir d4/df/df4/dc8 39 2024-09-16T23:48:59.205 INFO:tasks.workunit.client.0.smithi042.stdout:5/566: mknod d9/d2c/cc0 0 2024-09-16T23:48:59.205 INFO:tasks.workunit.client.0.smithi042.stdout:4/616: dwrite de/d42/d5d/f8c [0,4194304] 0 2024-09-16T23:48:59.209 INFO:tasks.workunit.client.0.smithi042.stdout:1/562: rmdir d0/d17/d18/d60 39 2024-09-16T23:48:59.211 INFO:tasks.workunit.client.0.smithi042.stdout:4/617: mkdir de/d2e/d37/d4c/dd4 0 2024-09-16T23:48:59.211 INFO:tasks.workunit.client.0.smithi042.stdout:1/563: read d0/f3 [5201354,62900] 0 2024-09-16T23:48:59.215 INFO:tasks.workunit.client.1.smithi043.stdout:4/614: dread d8/d25/d39/d41/f60 [0,4194304] 0 2024-09-16T23:48:59.215 INFO:tasks.workunit.client.0.smithi042.stdout:5/567: getdents d9/db/de/d92/daf 0 2024-09-16T23:48:59.216 INFO:tasks.workunit.client.1.smithi043.stdout:1/715: rename d4/f62 to d4/df/d13/ff8 0 2024-09-16T23:48:59.217 INFO:tasks.workunit.client.1.smithi043.stdout:1/716: chown d4/df/d26/c60 11843802 1 2024-09-16T23:48:59.217 INFO:tasks.workunit.client.0.smithi042.stdout:4/618: read de/d1a/da7/f1e [29887,66640] 0 2024-09-16T23:48:59.218 INFO:tasks.workunit.client.0.smithi042.stdout:8/324: mknod da/d19/c5f 0 2024-09-16T23:48:59.221 INFO:tasks.workunit.client.0.smithi042.stdout:5/568: dread d9/db/d1b/f95 [0,4194304] 0 2024-09-16T23:48:59.221 INFO:tasks.workunit.client.0.smithi042.stdout:4/619: unlink de/d2e/d37/d4c/f5f 0 2024-09-16T23:48:59.224 INFO:tasks.workunit.client.0.smithi042.stdout:4/620: creat de/d1a/da7/db9/fd5 x:0 0 0 2024-09-16T23:48:59.224 INFO:tasks.workunit.client.0.smithi042.stdout:1/564: mknod d0/d17/d5e/cc9 0 2024-09-16T23:48:59.225 INFO:tasks.workunit.client.0.smithi042.stdout:5/569: symlink d9/db/d1b/d31/d46/lc1 0 2024-09-16T23:48:59.228 INFO:tasks.workunit.client.0.smithi042.stdout:4/621: mkdir de/d42/d5d/d8f/dd6 0 2024-09-16T23:48:59.228 INFO:tasks.workunit.client.0.smithi042.stdout:5/570: truncate d9/db/f3e 4591444 0 2024-09-16T23:48:59.228 INFO:tasks.workunit.client.1.smithi043.stdout:9/518: getdents d4/d21/d35/d4b 0 2024-09-16T23:48:59.229 INFO:tasks.workunit.client.0.smithi042.stdout:1/565: mknod d0/d9/d90/d6b/d88/dc2/cca 0 2024-09-16T23:48:59.230 INFO:tasks.workunit.client.0.smithi042.stdout:8/325: rmdir da/d19/d1d/d30/d49 39 2024-09-16T23:48:59.231 INFO:tasks.workunit.client.0.smithi042.stdout:5/571: symlink d9/db/de/d10/d2f/lc2 0 2024-09-16T23:48:59.232 INFO:tasks.workunit.client.1.smithi043.stdout:4/615: dread d8/d25/f5b [0,4194304] 0 2024-09-16T23:48:59.232 INFO:tasks.workunit.client.1.smithi043.stdout:9/519: unlink d4/d28/d5f/d91/f9c 0 2024-09-16T23:48:59.234 INFO:tasks.workunit.client.0.smithi042.stdout:5/572: mknod d9/db/d19/d21/d41/cc3 0 2024-09-16T23:48:59.235 INFO:tasks.workunit.client.0.smithi042.stdout:4/622: dwrite de/d12/d1b/f98 [0,4194304] 0 2024-09-16T23:48:59.236 INFO:tasks.workunit.client.0.smithi042.stdout:5/573: mknod d9/d2c/cc4 0 2024-09-16T23:48:59.237 INFO:tasks.workunit.client.1.smithi043.stdout:6/600: dwrite de/d1a/d28/f3e [0,4194304] 0 2024-09-16T23:48:59.238 INFO:tasks.workunit.client.0.smithi042.stdout:1/566: symlink d0/d17/d18/lcb 0 2024-09-16T23:48:59.239 INFO:tasks.workunit.client.1.smithi043.stdout:6/601: write de/f40 [3249365,82423] 0 2024-09-16T23:48:59.240 INFO:tasks.workunit.client.1.smithi043.stdout:9/520: mkdir d4/d1c/d25/d2b/da1 0 2024-09-16T23:48:59.241 INFO:tasks.workunit.client.0.smithi042.stdout:1/567: readlink d0/l7 0 2024-09-16T23:48:59.243 INFO:tasks.workunit.client.0.smithi042.stdout:8/326: mknod da/c60 0 2024-09-16T23:48:59.243 INFO:tasks.workunit.client.0.smithi042.stdout:5/574: mknod d9/db/de/cc5 0 2024-09-16T23:48:59.244 INFO:tasks.workunit.client.1.smithi043.stdout:4/616: mknod d8/d15/d20/d2b/d22/db2/cc3 0 2024-09-16T23:48:59.248 INFO:tasks.workunit.client.0.smithi042.stdout:7/487: sync 2024-09-16T23:48:59.249 INFO:tasks.workunit.client.1.smithi043.stdout:6/602: dwrite de/d23/fb8 [4194304,4194304] 0 2024-09-16T23:48:59.251 INFO:tasks.workunit.client.0.smithi042.stdout:7/488: rmdir d3/d1f/d49/d76/d19/d63 39 2024-09-16T23:48:59.252 INFO:tasks.workunit.client.1.smithi043.stdout:9/521: rename d4/d28/d5f/d91/d97 to d4/d21/d33/da2 0 2024-09-16T23:48:59.253 INFO:tasks.workunit.client.0.smithi042.stdout:7/489: write d3/f5f [226681,33912] 0 2024-09-16T23:48:59.253 INFO:tasks.workunit.client.1.smithi043.stdout:8/589: sync 2024-09-16T23:48:59.254 INFO:tasks.workunit.client.1.smithi043.stdout:5/754: sync 2024-09-16T23:48:59.254 INFO:tasks.workunit.client.1.smithi043.stdout:7/533: sync 2024-09-16T23:48:59.254 INFO:tasks.workunit.client.1.smithi043.stdout:3/605: sync 2024-09-16T23:48:59.257 INFO:tasks.workunit.client.1.smithi043.stdout:4/617: dwrite d8/d25/d39/d58/f4a [4194304,4194304] 0 2024-09-16T23:48:59.257 INFO:tasks.workunit.client.1.smithi043.stdout:6/603: rmdir de/d23/d63 39 2024-09-16T23:48:59.258 INFO:tasks.workunit.client.0.smithi042.stdout:4/623: dwrite de/d2e/d37/d4c/d4f/f6b [8388608,4194304] 0 2024-09-16T23:48:59.259 INFO:tasks.workunit.client.0.smithi042.stdout:7/490: creat d3/d98/da0/d32/d6b/fad x:0 0 0 2024-09-16T23:48:59.261 INFO:tasks.workunit.client.1.smithi043.stdout:3/606: truncate d4/de/f55 4960972 0 2024-09-16T23:48:59.261 INFO:tasks.workunit.client.0.smithi042.stdout:7/491: dread - d3/f9a zero size 2024-09-16T23:48:59.262 INFO:tasks.workunit.client.1.smithi043.stdout:4/618: chown d8/d25/d39/d58/fc0 7168933 1 2024-09-16T23:48:59.264 INFO:tasks.workunit.client.1.smithi043.stdout:3/607: truncate d4/fa0 21227 0 2024-09-16T23:48:59.265 INFO:tasks.workunit.client.0.smithi042.stdout:4/624: creat de/d42/d99/d9e/fd7 x:0 0 0 2024-09-16T23:48:59.266 INFO:tasks.workunit.client.0.smithi042.stdout:5/575: dread d9/db/de/d5e/d87/fb9 [0,4194304] 0 2024-09-16T23:48:59.266 INFO:tasks.workunit.client.0.smithi042.stdout:9/313: truncate d4/f5 283653 0 2024-09-16T23:48:59.267 INFO:tasks.workunit.client.0.smithi042.stdout:4/625: dread - de/d42/d99/d9e/fd7 zero size 2024-09-16T23:48:59.267 INFO:tasks.workunit.client.1.smithi043.stdout:7/534: rmdir d4/db/d43/d83 39 2024-09-16T23:48:59.269 INFO:tasks.workunit.client.1.smithi043.stdout:5/755: symlink d3/d9/d22/d4a/d4c/ded/l100 0 2024-09-16T23:48:59.270 INFO:tasks.workunit.client.0.smithi042.stdout:5/576: creat d9/db/d19/d21/d85/fc6 x:0 0 0 2024-09-16T23:48:59.271 INFO:tasks.workunit.client.1.smithi043.stdout:7/535: chown d4/db/d11/d15/l38 633 1 2024-09-16T23:48:59.271 INFO:tasks.workunit.client.0.smithi042.stdout:4/626: truncate de/d42/f50 4481117 0 2024-09-16T23:48:59.272 INFO:tasks.workunit.client.1.smithi043.stdout:6/604: dwrite de/d23/d34/d9b/d5a/fc6 [0,4194304] 0 2024-09-16T23:48:59.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:58 smithi042 ceph-mon[27914]: pgmap v8: 65 pgs: 65 active+clean; 1.6 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 151 MiB/s wr, 290 op/s 2024-09-16T23:48:59.273 INFO:tasks.workunit.client.0.smithi042.stdout:4/627: write de/d12/f97 [486237,45279] 0 2024-09-16T23:48:59.276 INFO:tasks.workunit.client.0.smithi042.stdout:5/577: fdatasync d9/db/de/d5e/d87/d61/f4e 0 2024-09-16T23:48:59.277 INFO:tasks.workunit.client.1.smithi043.stdout:3/608: fsync d4/d30/d53/d88/f93 0 2024-09-16T23:48:59.279 INFO:tasks.workunit.client.0.smithi042.stdout:9/314: dwrite d4/d10/f3f [0,4194304] 0 2024-09-16T23:48:59.280 INFO:tasks.workunit.client.1.smithi043.stdout:4/619: dread d8/fc [0,4194304] 0 2024-09-16T23:48:59.281 INFO:tasks.workunit.client.1.smithi043.stdout:3/609: write d4/de/f55 [704790,38717] 0 2024-09-16T23:48:59.282 INFO:tasks.workunit.client.0.smithi042.stdout:4/628: creat de/d42/d5d/d8f/d9f/dc0/fd8 x:0 0 0 2024-09-16T23:48:59.282 INFO:tasks.workunit.client.0.smithi042.stdout:9/315: readlink d4/d12/d2d/d37/d3a/d50/l55 0 2024-09-16T23:48:59.284 INFO:tasks.workunit.client.1.smithi043.stdout:8/590: dread de/f23 [0,4194304] 0 2024-09-16T23:48:59.285 INFO:tasks.workunit.client.0.smithi042.stdout:5/578: creat d9/db/de/fc7 x:0 0 0 2024-09-16T23:48:59.285 INFO:tasks.workunit.client.1.smithi043.stdout:5/756: mkdir d3/db/d15/d4d/d6b/d73/d101 0 2024-09-16T23:48:59.286 INFO:tasks.workunit.client.0.smithi042.stdout:4/629: truncate de/d2e/d37/d3e/f4d 1516691 0 2024-09-16T23:48:59.286 INFO:tasks.workunit.client.1.smithi043.stdout:0/577: write d0/de/f91 [217058,59542] 0 2024-09-16T23:48:59.288 INFO:tasks.workunit.client.0.smithi042.stdout:4/630: write de/d2e/d37/d3c/f9a [3863980,43349] 0 2024-09-16T23:48:59.290 INFO:tasks.workunit.client.0.smithi042.stdout:5/579: mkdir d9/db/de/d10/dc8 0 2024-09-16T23:48:59.291 INFO:tasks.workunit.client.0.smithi042.stdout:9/316: symlink d4/d10/d4b/l62 0 2024-09-16T23:48:59.292 INFO:tasks.workunit.client.0.smithi042.stdout:4/631: write de/d1a/da7/f4a [3140500,128561] 0 2024-09-16T23:48:59.298 INFO:tasks.workunit.client.1.smithi043.stdout:8/591: fsync de/d18/d1f/d2f/f43 0 2024-09-16T23:48:59.298 INFO:tasks.workunit.client.0.smithi042.stdout:5/580: getdents d9/db/d19/d21 0 2024-09-16T23:48:59.301 INFO:tasks.workunit.client.1.smithi043.stdout:0/578: dwrite d0/d2/df/d5a/d9c/fb6 [0,4194304] 0 2024-09-16T23:48:59.301 INFO:tasks.workunit.client.1.smithi043.stdout:1/717: truncate d4/df/d13/ff8 1169563 0 2024-09-16T23:48:59.301 INFO:tasks.workunit.client.1.smithi043.stdout:5/757: fdatasync d3/f9b 0 2024-09-16T23:48:59.301 INFO:tasks.workunit.client.0.smithi042.stdout:4/632: dwrite de/d2e/d37/d4c/f9d [0,4194304] 0 2024-09-16T23:48:59.304 INFO:tasks.workunit.client.0.smithi042.stdout:5/581: read d9/db/de/d5e/d87/d61/f5d [206367,259] 0 2024-09-16T23:48:59.305 INFO:tasks.workunit.client.1.smithi043.stdout:4/620: dread d8/d25/d39/d41/d57/fae [0,4194304] 0 2024-09-16T23:48:59.305 INFO:tasks.workunit.client.1.smithi043.stdout:7/536: dread d4/f32 [0,4194304] 0 2024-09-16T23:48:59.305 INFO:tasks.workunit.client.1.smithi043.stdout:6/605: link de/d23/d63/f96 de/dbe/fc8 0 2024-09-16T23:48:59.306 INFO:tasks.workunit.client.0.smithi042.stdout:5/582: readlink d9/db/l3f 0 2024-09-16T23:48:59.307 INFO:tasks.workunit.client.1.smithi043.stdout:4/621: stat d8/fbe 0 2024-09-16T23:48:59.307 INFO:tasks.workunit.client.1.smithi043.stdout:6/606: stat de/d23/c43 0 2024-09-16T23:48:59.307 INFO:tasks.workunit.client.1.smithi043.stdout:0/579: stat d0/d2/df/d1a/d49/d80/d88/c95 0 2024-09-16T23:48:59.308 INFO:tasks.workunit.client.1.smithi043.stdout:4/622: stat d8/d15/l1c 0 2024-09-16T23:48:59.308 INFO:tasks.workunit.client.0.smithi042.stdout:4/633: mknod de/d42/d5d/cd9 0 2024-09-16T23:48:59.309 INFO:tasks.workunit.client.0.smithi042.stdout:9/317: creat d4/d12/f63 x:0 0 0 2024-09-16T23:48:59.311 INFO:tasks.workunit.client.0.smithi042.stdout:8/327: write da/f47 [976363,52948] 0 2024-09-16T23:48:59.311 INFO:tasks.workunit.client.1.smithi043.stdout:4/623: chown d8/d15/d20/c7b 1 1 2024-09-16T23:48:59.314 INFO:tasks.workunit.client.1.smithi043.stdout:6/607: truncate de/d23/d34/d66/f8d 486788 0 2024-09-16T23:48:59.315 INFO:tasks.workunit.client.0.smithi042.stdout:4/634: rename de/d1a/l24 to de/d1a/d70/dac/lda 0 2024-09-16T23:48:59.316 INFO:tasks.workunit.client.1.smithi043.stdout:6/608: truncate de/d1a/d28/f9a 4077982 0 2024-09-16T23:48:59.316 INFO:tasks.workunit.client.0.smithi042.stdout:8/328: write da/f47 [1818550,78730] 0 2024-09-16T23:48:59.317 INFO:tasks.workunit.client.1.smithi043.stdout:6/609: dread - de/d23/d34/d66/d9d/faf zero size 2024-09-16T23:48:59.318 INFO:tasks.workunit.client.0.smithi042.stdout:9/318: write d4/d10/f4a [1215015,58130] 0 2024-09-16T23:48:59.321 INFO:tasks.workunit.client.0.smithi042.stdout:5/583: write d9/db/de/d10/f22 [294329,123650] 0 2024-09-16T23:48:59.322 INFO:tasks.workunit.client.1.smithi043.stdout:9/522: dwrite d4/d5/f1d [0,4194304] 0 2024-09-16T23:48:59.323 INFO:tasks.workunit.client.0.smithi042.stdout:5/584: chown d9/db/d19/d21/l29 311 1 2024-09-16T23:48:59.327 INFO:tasks.workunit.client.0.smithi042.stdout:5/585: mknod d9/d2c/cc9 0 2024-09-16T23:48:59.327 INFO:tasks.workunit.client.1.smithi043.stdout:5/758: read d3/d9/d22/d60/f82 [1471716,44825] 0 2024-09-16T23:48:59.328 INFO:tasks.workunit.client.0.smithi042.stdout:9/319: rename d4/d12/l23 to d4/d12/d2d/d37/d41/l64 0 2024-09-16T23:48:59.328 INFO:tasks.workunit.client.1.smithi043.stdout:9/523: dread - d4/d1c/d25/d2b/f8a zero size 2024-09-16T23:48:59.330 INFO:tasks.workunit.client.1.smithi043.stdout:6/610: dwrite de/d23/d49/fb5 [0,4194304] 0 2024-09-16T23:48:59.330 INFO:tasks.workunit.client.1.smithi043.stdout:5/759: dread - d3/da4/fb8 zero size 2024-09-16T23:48:59.330 INFO:tasks.workunit.client.1.smithi043.stdout:8/592: dread de/d18/d27/f94 [0,4194304] 0 2024-09-16T23:48:59.331 INFO:tasks.workunit.client.1.smithi043.stdout:0/580: symlink d0/d2/df/d1a/d49/lc2 0 2024-09-16T23:48:59.331 INFO:tasks.workunit.client.0.smithi042.stdout:4/635: dwrite de/d42/f50 [0,4194304] 0 2024-09-16T23:48:59.334 INFO:tasks.workunit.client.0.smithi042.stdout:9/320: fdatasync d4/d12/d2d/d37/d3a/f3d 0 2024-09-16T23:48:59.336 INFO:tasks.workunit.client.1.smithi043.stdout:6/611: dwrite de/f6c [0,4194304] 0 2024-09-16T23:48:59.337 INFO:tasks.workunit.client.0.smithi042.stdout:9/321: chown d4/d12/d2d/d37/d3a/f5c 481206705 1 2024-09-16T23:48:59.337 INFO:tasks.workunit.client.0.smithi042.stdout:4/636: creat de/d12/db0/fdb x:0 0 0 2024-09-16T23:48:59.337 INFO:tasks.workunit.client.1.smithi043.stdout:6/612: write de/f13 [8332175,58191] 0 2024-09-16T23:48:59.341 INFO:tasks.workunit.client.1.smithi043.stdout:1/718: truncate d4/d9/f17 813336 0 2024-09-16T23:48:59.344 INFO:tasks.workunit.client.0.smithi042.stdout:8/329: write da/f14 [2856899,68204] 0 2024-09-16T23:48:59.344 INFO:tasks.workunit.client.0.smithi042.stdout:4/637: write de/d42/d68/f69 [800664,26103] 0 2024-09-16T23:48:59.346 INFO:tasks.workunit.client.0.smithi042.stdout:5/586: dwrite d9/db/d1b/f56 [0,4194304] 0 2024-09-16T23:48:59.348 INFO:tasks.workunit.client.1.smithi043.stdout:9/524: write d4/d1c/d4f/d8c/f66 [1253377,94555] 0 2024-09-16T23:48:59.349 INFO:tasks.workunit.client.1.smithi043.stdout:9/525: chown d4/d1c/d6b/d67/c8d 30061020 1 2024-09-16T23:48:59.350 INFO:tasks.workunit.client.1.smithi043.stdout:8/593: rmdir de/d18/d1f/d2f/d39/d7b 39 2024-09-16T23:48:59.353 INFO:tasks.workunit.client.1.smithi043.stdout:0/581: rename d0/d2/df/d1a/l82 to d0/de/lc3 0 2024-09-16T23:48:59.354 INFO:tasks.workunit.client.1.smithi043.stdout:0/582: read d0/d2/df/d1a/f30 [2574758,5894] 0 2024-09-16T23:48:59.354 INFO:tasks.workunit.client.1.smithi043.stdout:0/583: fsync d0/d2/f33 0 2024-09-16T23:48:59.355 INFO:tasks.workunit.client.1.smithi043.stdout:4/624: creat d8/d15/d9b/dbf/fc4 x:0 0 0 2024-09-16T23:48:59.355 INFO:tasks.workunit.client.0.smithi042.stdout:4/638: dread de/d2e/d37/d4c/d4f/f6b [4194304,4194304] 0 2024-09-16T23:48:59.357 INFO:tasks.workunit.client.0.smithi042.stdout:4/639: fsync de/d12/f48 0 2024-09-16T23:48:59.361 INFO:tasks.workunit.client.1.smithi043.stdout:6/613: mkdir de/d23/d49/dc9 0 2024-09-16T23:48:59.361 INFO:tasks.workunit.client.1.smithi043.stdout:6/614: chown de/d1a/d39/l30 11025 1 2024-09-16T23:48:59.363 INFO:tasks.workunit.client.1.smithi043.stdout:1/719: creat d4/df/d26/ff9 x:0 0 0 2024-09-16T23:48:59.366 INFO:tasks.workunit.client.0.smithi042.stdout:4/640: dwrite de/d42/d82/fcc [0,4194304] 0 2024-09-16T23:48:59.366 INFO:tasks.workunit.client.0.smithi042.stdout:8/330: link da/d2f/f5e da/d19/d1d/d30/d3f/f61 0 2024-09-16T23:48:59.367 INFO:tasks.workunit.client.1.smithi043.stdout:5/760: creat d3/d79/d91/df4/f102 x:0 0 0 2024-09-16T23:48:59.367 INFO:tasks.workunit.client.1.smithi043.stdout:6/615: dwrite de/f24 [0,4194304] 0 2024-09-16T23:48:59.369 INFO:tasks.workunit.client.1.smithi043.stdout:0/584: mkdir d0/d2/df/d1a/d49/d80/d88/dc4 0 2024-09-16T23:48:59.370 INFO:tasks.workunit.client.1.smithi043.stdout:5/761: readlink d3/db/d19/d76/l86 0 2024-09-16T23:48:59.373 INFO:tasks.workunit.client.0.smithi042.stdout:4/641: mkdir de/d42/d5d/d8f/dd6/ddc 0 2024-09-16T23:48:59.374 INFO:tasks.workunit.client.1.smithi043.stdout:4/625: rmdir d8/d15/d20/d2b/d22 39 2024-09-16T23:48:59.379 INFO:tasks.workunit.client.0.smithi042.stdout:5/587: dread d9/db/d19/d21/f2e [0,4194304] 0 2024-09-16T23:48:59.381 INFO:tasks.workunit.client.1.smithi043.stdout:9/526: mkdir d4/d1c/d25/d2b/d4c/da3 0 2024-09-16T23:48:59.381 INFO:tasks.workunit.client.1.smithi043.stdout:0/585: rmdir d0/d2/df/d5a 39 2024-09-16T23:48:59.381 INFO:tasks.workunit.client.0.smithi042.stdout:5/588: write d9/db/de/f6f [1528463,83765] 0 2024-09-16T23:48:59.382 INFO:tasks.workunit.client.1.smithi043.stdout:7/537: dwrite d4/db/d28/f97 [0,4194304] 0 2024-09-16T23:48:59.383 INFO:tasks.workunit.client.0.smithi042.stdout:4/642: dwrite de/d2e/d37/f65 [0,4194304] 0 2024-09-16T23:48:59.383 INFO:tasks.workunit.client.0.smithi042.stdout:8/331: creat da/d19/d1d/f62 x:0 0 0 2024-09-16T23:48:59.383 INFO:tasks.workunit.client.1.smithi043.stdout:0/586: stat d0/d2/df/d1a/c6b 0 2024-09-16T23:48:59.384 INFO:tasks.workunit.client.1.smithi043.stdout:0/587: read - d0/fa1 zero size 2024-09-16T23:48:59.385 INFO:tasks.workunit.client.0.smithi042.stdout:5/589: write d9/db/de/fb3 [970391,58869] 0 2024-09-16T23:48:59.386 INFO:tasks.workunit.client.0.smithi042.stdout:4/643: chown de/d1a/da7/l3d 34 1 2024-09-16T23:48:59.388 INFO:tasks.workunit.client.0.smithi042.stdout:9/322: truncate d4/d10/f4a 189420 0 2024-09-16T23:48:59.395 INFO:tasks.workunit.client.1.smithi043.stdout:9/527: symlink d4/d21/d33/da2/la4 0 2024-09-16T23:48:59.396 INFO:tasks.workunit.client.1.smithi043.stdout:9/528: dread - d4/d5/d16/fa0 zero size 2024-09-16T23:48:59.398 INFO:tasks.workunit.client.1.smithi043.stdout:9/529: chown d4/d21/c74 95855022 1 2024-09-16T23:48:59.398 INFO:tasks.workunit.client.1.smithi043.stdout:5/762: mknod d3/d79/dcc/c103 0 2024-09-16T23:48:59.400 INFO:tasks.workunit.client.1.smithi043.stdout:2/513: sync 2024-09-16T23:48:59.401 INFO:tasks.workunit.client.1.smithi043.stdout:9/530: chown d4/d1c/d6b/f3c 4216 1 2024-09-16T23:48:59.404 INFO:tasks.workunit.client.1.smithi043.stdout:0/588: rename d0/d2/df/d1a/d49/f76 to d0/d2/df/d5a/d9c/fc5 0 2024-09-16T23:48:59.405 INFO:tasks.workunit.client.1.smithi043.stdout:5/763: mknod d3/db/d84/db6/c104 0 2024-09-16T23:48:59.407 INFO:tasks.workunit.client.1.smithi043.stdout:1/720: write d4/df/d26/d79/fd7 [2458706,58520] 0 2024-09-16T23:48:59.409 INFO:tasks.workunit.client.0.smithi042.stdout:2/480: sync 2024-09-16T23:48:59.409 INFO:tasks.workunit.client.0.smithi042.stdout:6/372: sync 2024-09-16T23:48:59.409 INFO:tasks.workunit.client.0.smithi042.stdout:3/535: sync 2024-09-16T23:48:59.409 INFO:tasks.workunit.client.0.smithi042.stdout:0/350: sync 2024-09-16T23:48:59.410 INFO:tasks.workunit.client.1.smithi043.stdout:7/538: dwrite d4/d27/d41/d1b/d66/f8c [0,4194304] 0 2024-09-16T23:48:59.410 INFO:tasks.workunit.client.0.smithi042.stdout:6/373: chown d8/db/dd 416650467 1 2024-09-16T23:48:59.411 INFO:tasks.workunit.client.0.smithi042.stdout:3/536: readlink d2/d1b/d1d/d30/l8b 0 2024-09-16T23:48:59.412 INFO:tasks.workunit.client.1.smithi043.stdout:6/616: write de/f47 [462959,99182] 0 2024-09-16T23:48:59.413 INFO:tasks.workunit.client.0.smithi042.stdout:0/351: truncate d0/df/d13/f63 604926 0 2024-09-16T23:48:59.413 INFO:tasks.workunit.client.0.smithi042.stdout:6/374: dread - d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f79 zero size 2024-09-16T23:48:59.413 INFO:tasks.workunit.client.0.smithi042.stdout:0/352: fdatasync d0/f4 0 2024-09-16T23:48:59.414 INFO:tasks.workunit.client.1.smithi043.stdout:5/764: stat d3/db/d19/d3e/f70 0 2024-09-16T23:48:59.414 INFO:tasks.workunit.client.0.smithi042.stdout:6/375: chown d8/db/f1d 6 1 2024-09-16T23:48:59.414 INFO:tasks.workunit.client.1.smithi043.stdout:4/626: dwrite d8/f1d [0,4194304] 0 2024-09-16T23:48:59.419 INFO:tasks.workunit.client.1.smithi043.stdout:0/589: rename d0/d2/c68 to d0/d2/df/d1a/d9a/cc6 0 2024-09-16T23:48:59.421 INFO:tasks.workunit.client.1.smithi043.stdout:0/590: write d0/de/f91 [495890,75315] 0 2024-09-16T23:48:59.424 INFO:tasks.workunit.client.1.smithi043.stdout:6/617: mkdir de/dbe/dca 0 2024-09-16T23:48:59.427 INFO:tasks.workunit.client.0.smithi042.stdout:6/376: unlink d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/c63 0 2024-09-16T23:48:59.429 INFO:tasks.workunit.client.1.smithi043.stdout:5/765: fdatasync d3/d9/d22/f61 0 2024-09-16T23:48:59.432 INFO:tasks.workunit.client.0.smithi042.stdout:4/644: symlink de/d12/d1b/ldd 0 2024-09-16T23:48:59.432 INFO:tasks.workunit.client.0.smithi042.stdout:6/377: unlink d8/db/dd/d1b/d2d/f3e 0 2024-09-16T23:48:59.433 INFO:tasks.workunit.client.0.smithi042.stdout:4/645: stat de/d42/d5d/f8c 0 2024-09-16T23:48:59.434 INFO:tasks.workunit.client.0.smithi042.stdout:5/590: rename d9/db/d1b/d31/d6c/cb8 to d9/db/de/d92/daf/cca 0 2024-09-16T23:48:59.434 INFO:tasks.workunit.client.0.smithi042.stdout:4/646: chown de/d42/d5d/d8f/d9f 9103 1 2024-09-16T23:48:59.435 INFO:tasks.workunit.client.1.smithi043.stdout:6/618: rmdir de 39 2024-09-16T23:48:59.436 INFO:tasks.workunit.client.0.smithi042.stdout:3/537: fsync d2/d1b/f2a 0 2024-09-16T23:48:59.436 INFO:tasks.workunit.client.1.smithi043.stdout:9/531: dread d4/d1c/d25/d2b/d4c/f93 [0,4194304] 0 2024-09-16T23:48:59.436 INFO:tasks.workunit.client.1.smithi043.stdout:9/532: chown d4/l29 2972669 1 2024-09-16T23:48:59.437 INFO:tasks.workunit.client.0.smithi042.stdout:6/378: mknod d8/db/dd/d2c/c7d 0 2024-09-16T23:48:59.439 INFO:tasks.workunit.client.1.smithi043.stdout:2/514: dread d8/d1f/f2e [0,4194304] 0 2024-09-16T23:48:59.441 INFO:tasks.workunit.client.1.smithi043.stdout:7/539: rename d4/db/d43/d83/c99 to d4/db/d50/cb3 0 2024-09-16T23:48:59.441 INFO:tasks.workunit.client.0.smithi042.stdout:2/481: truncate d0/da/d17/f1c 1725672 0 2024-09-16T23:48:59.443 INFO:tasks.workunit.client.0.smithi042.stdout:5/591: creat d9/db/de/d5e/fcb x:0 0 0 2024-09-16T23:48:59.444 INFO:tasks.workunit.client.0.smithi042.stdout:9/323: rename d4/f19 to d4/d12/d2d/d37/d28/f65 0 2024-09-16T23:48:59.447 INFO:tasks.workunit.client.0.smithi042.stdout:6/379: rmdir d8/db/dd/d1b 39 2024-09-16T23:48:59.451 INFO:tasks.workunit.client.0.smithi042.stdout:0/353: dwrite d0/d15/f2f [0,4194304] 0 2024-09-16T23:48:59.452 INFO:tasks.workunit.client.0.smithi042.stdout:3/538: dwrite d2/d7/d61/d73/f9a [0,4194304] 0 2024-09-16T23:48:59.453 INFO:tasks.workunit.client.1.smithi043.stdout:5/766: stat d3/d9/d22/d4a/d4c/ded/cfc 0 2024-09-16T23:48:59.454 INFO:tasks.workunit.client.0.smithi042.stdout:5/592: mkdir d9/db/de/d5e/d87/d61/dcc 0 2024-09-16T23:48:59.457 INFO:tasks.workunit.client.0.smithi042.stdout:4/647: dread de/d2e/f77 [0,4194304] 0 2024-09-16T23:48:59.457 INFO:tasks.workunit.client.0.smithi042.stdout:6/380: truncate d8/db/dd/d2c/d57/f5d 153707 0 2024-09-16T23:48:59.459 INFO:tasks.workunit.client.0.smithi042.stdout:0/354: write d0/df/d13/f4e [1422608,35301] 0 2024-09-16T23:48:59.462 INFO:tasks.workunit.client.1.smithi043.stdout:4/627: mknod d8/d15/d20/d2b/d54/d55/d5e/cc5 0 2024-09-16T23:48:59.463 INFO:tasks.workunit.client.1.smithi043.stdout:0/591: write d0/d2/df/d1a/d46/d39/f7a [15978637,45417] 0 2024-09-16T23:48:59.463 INFO:tasks.workunit.client.1.smithi043.stdout:4/628: write d8/fbb [96409,33183] 0 2024-09-16T23:48:59.466 INFO:tasks.workunit.client.0.smithi042.stdout:2/482: truncate d0/da/f23 544429 0 2024-09-16T23:48:59.467 INFO:tasks.workunit.client.1.smithi043.stdout:4/629: dread d8/d25/d64/fb4 [0,4194304] 0 2024-09-16T23:48:59.467 INFO:tasks.workunit.client.1.smithi043.stdout:9/533: fsync d4/d1c/d6b/f3c 0 2024-09-16T23:48:59.467 INFO:tasks.workunit.client.0.smithi042.stdout:9/324: symlink d4/d12/d2d/d51/l66 0 2024-09-16T23:48:59.469 INFO:tasks.workunit.client.1.smithi043.stdout:9/534: dread d4/d1c/d4f/d8c/f60 [0,4194304] 0 2024-09-16T23:48:59.469 INFO:tasks.workunit.client.1.smithi043.stdout:2/515: creat d8/d5e/fac x:0 0 0 2024-09-16T23:48:59.470 INFO:tasks.workunit.client.0.smithi042.stdout:0/355: dread d0/df/d23/f38 [0,4194304] 0 2024-09-16T23:48:59.473 INFO:tasks.workunit.client.0.smithi042.stdout:3/539: getdents d2/d95 0 2024-09-16T23:48:59.474 INFO:tasks.workunit.client.0.smithi042.stdout:5/593: link d9/cb1 d9/d63/ccd 0 2024-09-16T23:48:59.474 INFO:tasks.workunit.client.0.smithi042.stdout:2/483: mknod d0/da/d88/cab 0 2024-09-16T23:48:59.477 INFO:tasks.workunit.client.0.smithi042.stdout:0/356: rmdir d0/df/d23/d33 39 2024-09-16T23:48:59.478 INFO:tasks.workunit.client.0.smithi042.stdout:4/648: link de/d2e/d37/lc6 de/d42/d68/lde 0 2024-09-16T23:48:59.481 INFO:tasks.workunit.client.0.smithi042.stdout:5/594: creat d9/d2c/da9/fce x:0 0 0 2024-09-16T23:48:59.481 INFO:tasks.workunit.client.1.smithi043.stdout:6/619: mknod de/d23/d34/d66/d9d/ccb 0 2024-09-16T23:48:59.484 INFO:tasks.workunit.client.0.smithi042.stdout:4/649: rename de/d2e/d37/d4c/f9d to de/d12/d1b/d8a/db2/fdf 0 2024-09-16T23:48:59.485 INFO:tasks.workunit.client.1.smithi043.stdout:2/516: dread d8/d5e/f87 [0,4194304] 0 2024-09-16T23:48:59.486 INFO:tasks.workunit.client.1.smithi043.stdout:0/592: unlink d0/d2/df/c9d 0 2024-09-16T23:48:59.487 INFO:tasks.workunit.client.0.smithi042.stdout:0/357: dwrite d0/df/d62/f65 [0,4194304] 0 2024-09-16T23:48:59.487 INFO:tasks.workunit.client.0.smithi042.stdout:4/650: write de/f31 [2118151,87846] 0 2024-09-16T23:48:59.489 INFO:tasks.workunit.client.0.smithi042.stdout:9/325: creat d4/d10/d24/f67 x:0 0 0 2024-09-16T23:48:59.489 INFO:tasks.workunit.client.0.smithi042.stdout:5/595: rmdir d9/db/d19 39 2024-09-16T23:48:59.491 INFO:tasks.workunit.client.1.smithi043.stdout:7/540: write d4/d27/d41/f40 [333052,71012] 0 2024-09-16T23:48:59.492 INFO:tasks.workunit.client.1.smithi043.stdout:4/630: read d8/d15/d6e/fa9 [2738429,38039] 0 2024-09-16T23:48:59.493 INFO:tasks.workunit.client.0.smithi042.stdout:5/596: chown d9/d4a/l71 169 1 2024-09-16T23:48:59.495 INFO:tasks.workunit.client.0.smithi042.stdout:0/358: fdatasync d0/f67 0 2024-09-16T23:48:59.500 INFO:tasks.workunit.client.1.smithi043.stdout:1/721: link d4/d9/ce1 d4/df/cfa 0 2024-09-16T23:48:59.500 INFO:tasks.workunit.client.1.smithi043.stdout:5/767: mknod d3/db/d15/c105 0 2024-09-16T23:48:59.500 INFO:tasks.workunit.client.1.smithi043.stdout:2/517: creat d8/d4b/d4f/d7c/fad x:0 0 0 2024-09-16T23:48:59.501 INFO:tasks.workunit.client.1.smithi043.stdout:9/535: dread d4/f48 [0,4194304] 0 2024-09-16T23:48:59.501 INFO:tasks.workunit.client.0.smithi042.stdout:9/326: dwrite d4/d10/f1b [4194304,4194304] 0 2024-09-16T23:48:59.502 INFO:tasks.workunit.client.1.smithi043.stdout:2/518: stat d8/d1f/c34 0 2024-09-16T23:48:59.502 INFO:tasks.workunit.client.1.smithi043.stdout:4/631: readlink d8/d15/d20/d2b/d22/l6c 0 2024-09-16T23:48:59.504 INFO:tasks.workunit.client.1.smithi043.stdout:0/593: mknod d0/cc7 0 2024-09-16T23:48:59.504 INFO:tasks.workunit.client.1.smithi043.stdout:2/519: write d8/d5e/d83/d3a/d77/f9b [951894,46967] 0 2024-09-16T23:48:59.506 INFO:tasks.workunit.client.0.smithi042.stdout:0/359: rename d0/d15/c50 to d0/df/d23/d33/c6d 0 2024-09-16T23:48:59.507 INFO:tasks.workunit.client.1.smithi043.stdout:7/541: fsync d4/f62 0 2024-09-16T23:48:59.507 INFO:tasks.workunit.client.0.smithi042.stdout:4/651: dread de/d12/f54 [0,4194304] 0 2024-09-16T23:48:59.508 INFO:tasks.workunit.client.1.smithi043.stdout:2/520: write d8/d5e/d83/fa0 [498164,75660] 0 2024-09-16T23:48:59.509 INFO:tasks.workunit.client.0.smithi042.stdout:5/597: dread d9/db/d19/d21/f2e [0,4194304] 0 2024-09-16T23:48:59.510 INFO:tasks.workunit.client.1.smithi043.stdout:5/768: creat d3/db/d19/d3e/d28/dcb/f106 x:0 0 0 2024-09-16T23:48:59.511 INFO:tasks.workunit.client.0.smithi042.stdout:0/360: write d0/df/d23/f6b [423138,113343] 0 2024-09-16T23:48:59.512 INFO:tasks.workunit.client.0.smithi042.stdout:4/652: rename de/d2e/d37/c5b to de/d12/d1b/ce0 0 2024-09-16T23:48:59.512 INFO:tasks.workunit.client.1.smithi043.stdout:5/769: chown d3/db/d26/l87 82538 1 2024-09-16T23:48:59.513 INFO:tasks.workunit.client.1.smithi043.stdout:9/536: dwrite d4/d1c/d4f/f5b [0,4194304] 0 2024-09-16T23:48:59.513 INFO:tasks.workunit.client.0.smithi042.stdout:0/361: chown d0/df/d13/d2d/d35 20842 1 2024-09-16T23:48:59.513 INFO:tasks.workunit.client.0.smithi042.stdout:5/598: mknod d9/db/d1b/d79/ccf 0 2024-09-16T23:48:59.514 INFO:tasks.workunit.client.1.smithi043.stdout:1/722: dread d4/d9/d1c/d1e/d47/f53 [0,4194304] 0 2024-09-16T23:48:59.514 INFO:tasks.workunit.client.1.smithi043.stdout:0/594: unlink d0/d2/df/d1a/f31 0 2024-09-16T23:48:59.515 INFO:tasks.workunit.client.0.smithi042.stdout:5/599: stat d9/db/de/d10/d2f/cbc 0 2024-09-16T23:48:59.515 INFO:tasks.workunit.client.0.smithi042.stdout:0/362: dread d0/df/d13/f63 [0,4194304] 0 2024-09-16T23:48:59.517 INFO:tasks.workunit.client.1.smithi043.stdout:2/521: dread - d8/d1d/d20/d26/f97 zero size 2024-09-16T23:48:59.518 INFO:tasks.workunit.client.0.smithi042.stdout:0/363: creat d0/d48/f6e x:0 0 0 2024-09-16T23:48:59.518 INFO:tasks.workunit.client.0.smithi042.stdout:5/600: rmdir d9/db/d19/d21/d85/da5 39 2024-09-16T23:48:59.519 INFO:tasks.workunit.client.1.smithi043.stdout:5/770: rmdir d3/db/d19/d76 39 2024-09-16T23:48:59.519 INFO:tasks.workunit.client.0.smithi042.stdout:5/601: stat d9/f74 0 2024-09-16T23:48:59.525 INFO:tasks.workunit.client.1.smithi043.stdout:5/771: write d3/d79/d91/daf/fc2 [2982100,46031] 0 2024-09-16T23:48:59.525 INFO:tasks.workunit.client.0.smithi042.stdout:5/602: creat d9/db/d19/d21/d41/fd0 x:0 0 0 2024-09-16T23:48:59.525 INFO:tasks.workunit.client.1.smithi043.stdout:9/537: dwrite d4/d28/d5f/d86/f96 [0,4194304] 0 2024-09-16T23:48:59.527 INFO:tasks.workunit.client.1.smithi043.stdout:2/522: symlink d8/d4b/d4f/lae 0 2024-09-16T23:48:59.527 INFO:tasks.workunit.client.0.smithi042.stdout:5/603: creat d9/db/d19/d21/d85/fd1 x:0 0 0 2024-09-16T23:48:59.528 INFO:tasks.workunit.client.1.smithi043.stdout:1/723: dwrite d4/d34/f6e [0,4194304] 0 2024-09-16T23:48:59.528 INFO:tasks.workunit.client.0.smithi042.stdout:5/604: fdatasync d9/f74 0 2024-09-16T23:48:59.528 INFO:tasks.workunit.client.1.smithi043.stdout:9/538: truncate d4/d1c/f6f 897267 0 2024-09-16T23:48:59.529 INFO:tasks.workunit.client.1.smithi043.stdout:0/595: creat d0/d2/df/fc8 x:0 0 0 2024-09-16T23:48:59.529 INFO:tasks.workunit.client.0.smithi042.stdout:5/605: chown d9/d2c/cc0 29584 1 2024-09-16T23:48:59.531 INFO:tasks.workunit.client.1.smithi043.stdout:2/523: chown d8/d1f/f6e 558826 1 2024-09-16T23:48:59.531 INFO:tasks.workunit.client.1.smithi043.stdout:5/772: truncate d3/db/d15/f75 605744 0 2024-09-16T23:48:59.531 INFO:tasks.workunit.client.1.smithi043.stdout:0/596: write d0/f4d [158409,51691] 0 2024-09-16T23:48:59.533 INFO:tasks.workunit.client.1.smithi043.stdout:1/724: mkdir d4/d9/d8f/db6/dfb 0 2024-09-16T23:48:59.534 INFO:tasks.workunit.client.1.smithi043.stdout:1/725: chown d4/d9/d12/l78 971023 1 2024-09-16T23:48:59.535 INFO:tasks.workunit.client.0.smithi042.stdout:5/606: rename d9/db/de/d5e/d87/d61/f5f to d9/d2c/d7b/fd2 0 2024-09-16T23:48:59.538 INFO:tasks.workunit.client.1.smithi043.stdout:5/773: rename d3/db/d15/d4d/dfd to d3/d79/d107 0 2024-09-16T23:48:59.539 INFO:tasks.workunit.client.1.smithi043.stdout:0/597: dwrite d0/d2/df/dbb/f50 [0,4194304] 0 2024-09-16T23:48:59.540 INFO:tasks.workunit.client.0.smithi042.stdout:5/607: readlink d9/db/d19/d21/d85/da5/la3 0 2024-09-16T23:48:59.542 INFO:tasks.workunit.client.1.smithi043.stdout:1/726: creat d4/df/df4/ffc x:0 0 0 2024-09-16T23:48:59.546 INFO:tasks.workunit.client.1.smithi043.stdout:1/727: write d4/f69 [4754298,88233] 0 2024-09-16T23:48:59.549 INFO:tasks.workunit.client.1.smithi043.stdout:9/539: dread d4/d1c/f20 [0,4194304] 0 2024-09-16T23:48:59.549 INFO:tasks.workunit.client.1.smithi043.stdout:0/598: link d0/d2/df/d1a/d3b/f6c d0/d2/df/d1a/d46/d92/fc9 0 2024-09-16T23:48:59.550 INFO:tasks.workunit.client.1.smithi043.stdout:5/774: dwrite d3/db/d19/d3e/d28/dcb/f106 [0,4194304] 0 2024-09-16T23:48:59.550 INFO:tasks.workunit.client.1.smithi043.stdout:9/540: stat d4/d1c/d6b/f3c 0 2024-09-16T23:48:59.551 INFO:tasks.workunit.client.1.smithi043.stdout:1/728: mkdir d4/df/d26/d79/dfd 0 2024-09-16T23:48:59.552 INFO:tasks.workunit.client.1.smithi043.stdout:5/775: fdatasync d3/db/d15/d4d/d6b/ff5 0 2024-09-16T23:48:59.554 INFO:tasks.workunit.client.1.smithi043.stdout:0/599: creat d0/d2/df/d1a/d49/d71/fca x:0 0 0 2024-09-16T23:48:59.557 INFO:tasks.workunit.client.1.smithi043.stdout:1/729: dread d4/d9/d12/fce [0,4194304] 0 2024-09-16T23:48:59.559 INFO:tasks.workunit.client.1.smithi043.stdout:5/776: creat d3/d9/d22/d4a/d4c/d8c/f108 x:0 0 0 2024-09-16T23:48:59.561 INFO:tasks.workunit.client.1.smithi043.stdout:1/730: write d4/d9/d1c/d1e/d47/f86 [283647,48318] 0 2024-09-16T23:48:59.562 INFO:tasks.workunit.client.1.smithi043.stdout:0/600: truncate d0/d2/df/d1a/f30 2304292 0 2024-09-16T23:48:59.565 INFO:tasks.workunit.client.1.smithi043.stdout:9/541: link d4/d21/d35/f5d d4/d1c/d25/d2b/fa5 0 2024-09-16T23:48:59.569 INFO:tasks.workunit.client.1.smithi043.stdout:9/542: readlink d4/d28/d49/l5a 0 2024-09-16T23:48:59.570 INFO:tasks.workunit.client.1.smithi043.stdout:5/777: dwrite d3/d9/f4b [0,4194304] 0 2024-09-16T23:48:59.572 INFO:tasks.workunit.client.1.smithi043.stdout:1/731: chown d4/df/df4/dc8/la5 0 1 2024-09-16T23:48:59.573 INFO:tasks.workunit.client.0.smithi042.stdout:9/327: write d4/f5 [1314101,42308] 0 2024-09-16T23:48:59.575 INFO:tasks.workunit.client.0.smithi042.stdout:4/653: read de/d1a/da7/f4a [3786303,124439] 0 2024-09-16T23:48:59.576 INFO:tasks.workunit.client.0.smithi042.stdout:4/654: read - de/d1a/da7/db9/fd5 zero size 2024-09-16T23:48:59.576 INFO:tasks.workunit.client.0.smithi042.stdout:0/364: truncate d0/d48/f53 1197810 0 2024-09-16T23:48:59.577 INFO:tasks.workunit.client.1.smithi043.stdout:9/543: chown d4/d21/d2a 4 1 2024-09-16T23:48:59.579 INFO:tasks.workunit.client.1.smithi043.stdout:5/778: mkdir d3/d9/d22/d4a/d64/d8f/d109 0 2024-09-16T23:48:59.581 INFO:tasks.workunit.client.0.smithi042.stdout:4/655: mknod de/d42/d99/d9e/ce1 0 2024-09-16T23:48:59.582 INFO:tasks.workunit.client.1.smithi043.stdout:1/732: unlink d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/leb 0 2024-09-16T23:48:59.582 INFO:tasks.workunit.client.0.smithi042.stdout:4/656: write de/d2e/d37/d3e/f5c [669967,90546] 0 2024-09-16T23:48:59.582 INFO:tasks.workunit.client.0.smithi042.stdout:9/328: mkdir d4/d12/d16/d68 0 2024-09-16T23:48:59.583 INFO:tasks.workunit.client.0.smithi042.stdout:5/608: truncate d9/db/d1b/f95 1578245 0 2024-09-16T23:48:59.585 INFO:tasks.workunit.client.1.smithi043.stdout:9/544: chown d4/d21/d33/da2/la4 34611987 1 2024-09-16T23:48:59.585 INFO:tasks.workunit.client.1.smithi043.stdout:4/632: dwrite d8/d15/d20/f8d [0,4194304] 0 2024-09-16T23:48:59.593 INFO:tasks.workunit.client.0.smithi042.stdout:4/657: rename de/l7e to de/d42/d5d/d8f/le2 0 2024-09-16T23:48:59.595 INFO:tasks.workunit.client.1.smithi043.stdout:7/542: dwrite d4/d27/d41/d1b/f6f [0,4194304] 0 2024-09-16T23:48:59.596 INFO:tasks.workunit.client.1.smithi043.stdout:2/524: write d8/d5e/d83/f59 [4939530,86091] 0 2024-09-16T23:48:59.596 INFO:tasks.workunit.client.1.smithi043.stdout:9/545: creat d4/d28/d5f/d91/fa6 x:0 0 0 2024-09-16T23:48:59.597 INFO:tasks.workunit.client.0.smithi042.stdout:0/365: write d0/df/d23/d33/d56/f5d [3858112,72308] 0 2024-09-16T23:48:59.599 INFO:tasks.workunit.client.0.smithi042.stdout:0/366: readlink d0/df/l58 0 2024-09-16T23:48:59.601 INFO:tasks.workunit.client.0.smithi042.stdout:4/658: dwrite de/d42/d82/fcc [0,4194304] 0 2024-09-16T23:48:59.603 INFO:tasks.workunit.client.1.smithi043.stdout:1/733: dwrite d4/df/fe0 [0,4194304] 0 2024-09-16T23:48:59.604 INFO:tasks.workunit.client.0.smithi042.stdout:0/367: symlink d0/df/d13/d49/l6f 0 2024-09-16T23:48:59.605 INFO:tasks.workunit.client.1.smithi043.stdout:0/601: rename d0/d2/df/d1a/d49/d71 to d0/d2/df/d1a/d46/d39/d83/dcb 0 2024-09-16T23:48:59.607 INFO:tasks.workunit.client.1.smithi043.stdout:4/633: fdatasync d8/d15/d20/d2b/d23/f38 0 2024-09-16T23:48:59.611 INFO:tasks.workunit.client.0.smithi042.stdout:0/368: truncate d0/d15/f3c 541255 0 2024-09-16T23:48:59.612 INFO:tasks.workunit.client.1.smithi043.stdout:9/546: dread d4/d1c/d25/d2b/d4c/f93 [0,4194304] 0 2024-09-16T23:48:59.612 INFO:tasks.workunit.client.0.smithi042.stdout:4/659: dwrite de/d2e/d37/d3e/f4d [0,4194304] 0 2024-09-16T23:48:59.613 INFO:tasks.workunit.client.1.smithi043.stdout:2/525: symlink d8/d11/laf 0 2024-09-16T23:48:59.613 INFO:tasks.workunit.client.1.smithi043.stdout:1/734: write d4/df/d26/d79/fd7 [239862,38551] 0 2024-09-16T23:48:59.615 INFO:tasks.workunit.client.0.smithi042.stdout:5/609: write d9/db/d1b/f95 [1771997,24322] 0 2024-09-16T23:48:59.618 INFO:tasks.workunit.client.1.smithi043.stdout:5/779: getdents d3/db 0 2024-09-16T23:48:59.623 INFO:tasks.workunit.client.1.smithi043.stdout:0/602: creat d0/d2/df/d1a/d46/d39/daa/db7/fcc x:0 0 0 2024-09-16T23:48:59.624 INFO:tasks.workunit.client.0.smithi042.stdout:5/610: dwrite d9/db/de/d92/daf/fb4 [0,4194304] 0 2024-09-16T23:48:59.630 INFO:tasks.workunit.client.1.smithi043.stdout:4/634: truncate d8/d15/d9b/d8b/fa8 2903023 0 2024-09-16T23:48:59.630 INFO:tasks.workunit.client.0.smithi042.stdout:5/611: symlink d9/ld3 0 2024-09-16T23:48:59.630 INFO:tasks.workunit.client.1.smithi043.stdout:0/603: fdatasync d0/d2/df/d1a/d46/d39/f45 0 2024-09-16T23:48:59.631 INFO:tasks.workunit.client.0.smithi042.stdout:4/660: getdents de/d42/d5d/d8f/d9f 0 2024-09-16T23:48:59.632 INFO:tasks.workunit.client.1.smithi043.stdout:9/547: dwrite d4/d28/d49/f76 [4194304,4194304] 0 2024-09-16T23:48:59.632 INFO:tasks.workunit.client.1.smithi043.stdout:7/543: dread d4/db/d28/f2d [0,4194304] 0 2024-09-16T23:48:59.633 INFO:tasks.workunit.client.0.smithi042.stdout:4/661: write de/d42/d82/d94/fa3 [3279982,58007] 0 2024-09-16T23:48:59.636 INFO:tasks.workunit.client.1.smithi043.stdout:7/544: chown d4/d27/l6d 23 1 2024-09-16T23:48:59.637 INFO:tasks.workunit.client.1.smithi043.stdout:5/780: dwrite d3/d9/f4b [0,4194304] 0 2024-09-16T23:48:59.639 INFO:tasks.workunit.client.0.smithi042.stdout:4/662: mknod de/d42/d5d/ce3 0 2024-09-16T23:48:59.639 INFO:tasks.workunit.client.0.smithi042.stdout:5/612: link d9/d2c/da9/cad d9/d63/cd4 0 2024-09-16T23:48:59.640 INFO:tasks.workunit.client.1.smithi043.stdout:2/526: truncate d8/d1d/d20/f48 560617 0 2024-09-16T23:48:59.645 INFO:tasks.workunit.client.1.smithi043.stdout:9/548: rename d4/l17 to d4/d1c/d25/d2b/la7 0 2024-09-16T23:48:59.645 INFO:tasks.workunit.client.1.smithi043.stdout:0/604: truncate d0/d2/df/d1a/d46/d39/d83/dcb/fca 1036260 0 2024-09-16T23:48:59.645 INFO:tasks.workunit.client.1.smithi043.stdout:7/545: rmdir d4/db/d43/d83/d85 39 2024-09-16T23:48:59.646 INFO:tasks.workunit.client.0.smithi042.stdout:0/369: write d0/d48/f53 [1554426,1721] 0 2024-09-16T23:48:59.646 INFO:tasks.workunit.client.1.smithi043.stdout:4/635: truncate d8/d15/d20/d2b/d23/d27/d4b/fbd 758938 0 2024-09-16T23:48:59.647 INFO:tasks.workunit.client.1.smithi043.stdout:2/527: symlink d8/d1d/d20/d26/lb0 0 2024-09-16T23:48:59.648 INFO:tasks.workunit.client.1.smithi043.stdout:1/735: write d4/df/d13/ff8 [1636492,1936] 0 2024-09-16T23:48:59.649 INFO:tasks.workunit.client.0.smithi042.stdout:5/613: mknod d9/d63/cd5 0 2024-09-16T23:48:59.651 INFO:tasks.workunit.client.1.smithi043.stdout:9/549: write d4/d21/d33/d95/f9b [825568,35346] 0 2024-09-16T23:48:59.652 INFO:tasks.workunit.client.0.smithi042.stdout:0/370: unlink d0/d48/f6e 0 2024-09-16T23:48:59.652 INFO:tasks.workunit.client.1.smithi043.stdout:0/605: creat d0/d2/df/d1a/d49/d80/fcd x:0 0 0 2024-09-16T23:48:59.653 INFO:tasks.workunit.client.1.smithi043.stdout:7/546: read d4/db/f24 [2127416,74318] 0 2024-09-16T23:48:59.655 INFO:tasks.workunit.client.0.smithi042.stdout:4/663: write de/d1a/da7/f56 [2989464,106590] 0 2024-09-16T23:48:59.657 INFO:tasks.workunit.client.0.smithi042.stdout:5/614: fsync d9/db/d1b/d31/d46/f49 0 2024-09-16T23:48:59.657 INFO:tasks.workunit.client.1.smithi043.stdout:7/547: fsync d4/db/d11/d15/f3d 0 2024-09-16T23:48:59.657 INFO:tasks.workunit.client.1.smithi043.stdout:0/606: dread - d0/d2/df/d1a/d46/d92/fac zero size 2024-09-16T23:48:59.659 INFO:tasks.workunit.client.0.smithi042.stdout:0/371: mkdir d0/df/d13/d49/d70 0 2024-09-16T23:48:59.662 INFO:tasks.workunit.client.1.smithi043.stdout:4/636: dwrite d8/d15/d20/d2b/f6b [0,4194304] 0 2024-09-16T23:48:59.663 INFO:tasks.workunit.client.1.smithi043.stdout:9/550: mknod d4/d21/d2a/d2c/d7b/ca8 0 2024-09-16T23:48:59.663 INFO:tasks.workunit.client.1.smithi043.stdout:5/781: getdents d3/d9/d22/d4a 0 2024-09-16T23:48:59.664 INFO:tasks.workunit.client.1.smithi043.stdout:9/551: stat d4/d1c/d6b/f39 0 2024-09-16T23:48:59.665 INFO:tasks.workunit.client.0.smithi042.stdout:0/372: symlink d0/df/l71 0 2024-09-16T23:48:59.665 INFO:tasks.workunit.client.1.smithi043.stdout:0/607: creat d0/d2/df/da7/fce x:0 0 0 2024-09-16T23:48:59.666 INFO:tasks.workunit.client.0.smithi042.stdout:4/664: symlink de/d42/d5d/d8f/dd6/ddc/le4 0 2024-09-16T23:48:59.666 INFO:tasks.workunit.client.1.smithi043.stdout:2/528: write d8/d1d/d20/d26/f27 [2203363,78201] 0 2024-09-16T23:48:59.667 INFO:tasks.workunit.client.1.smithi043.stdout:7/548: rename d4/db/d11/d21/fa4 to d4/d27/d41/d1b/d95/fb4 0 2024-09-16T23:48:59.667 INFO:tasks.workunit.client.0.smithi042.stdout:0/373: mknod d0/df/d13/d49/d70/c72 0 2024-09-16T23:48:59.668 INFO:tasks.workunit.client.0.smithi042.stdout:4/665: truncate de/d42/f44 749828 0 2024-09-16T23:48:59.669 INFO:tasks.workunit.client.1.smithi043.stdout:4/637: unlink d8/fbb 0 2024-09-16T23:48:59.671 INFO:tasks.workunit.client.1.smithi043.stdout:5/782: truncate d3/d9/d22/d4a/d64/d8f/fc9 4689771 0 2024-09-16T23:48:59.671 INFO:tasks.workunit.client.0.smithi042.stdout:0/374: write d0/df/d23/f31 [118885,37542] 0 2024-09-16T23:48:59.671 INFO:tasks.workunit.client.1.smithi043.stdout:7/549: chown d4/db/d11/d21/d31/d65 65 1 2024-09-16T23:48:59.672 INFO:tasks.workunit.client.0.smithi042.stdout:5/615: write d9/db/de/d92/fa1 [971955,108375] 0 2024-09-16T23:48:59.673 INFO:tasks.workunit.client.0.smithi042.stdout:4/666: rmdir de/d2e/d37/d4c/d4f 39 2024-09-16T23:48:59.673 INFO:tasks.workunit.client.0.smithi042.stdout:0/375: read d0/df/f14 [3971691,96213] 0 2024-09-16T23:48:59.673 INFO:tasks.workunit.client.0.smithi042.stdout:5/616: fsync d9/db/de/fb3 0 2024-09-16T23:48:59.675 INFO:tasks.workunit.client.0.smithi042.stdout:0/376: truncate d0/d15/f2f 4944851 0 2024-09-16T23:48:59.675 INFO:tasks.workunit.client.1.smithi043.stdout:2/529: mknod d8/d1d/d20/d23/cb1 0 2024-09-16T23:48:59.675 INFO:tasks.workunit.client.0.smithi042.stdout:5/617: fsync d9/db/d1b/f97 0 2024-09-16T23:48:59.676 INFO:tasks.workunit.client.0.smithi042.stdout:1/568: sync 2024-09-16T23:48:59.677 INFO:tasks.workunit.client.0.smithi042.stdout:7/492: sync 2024-09-16T23:48:59.677 INFO:tasks.workunit.client.0.smithi042.stdout:5/618: stat d9/db/d19/d21/d85/da5/la3 0 2024-09-16T23:48:59.678 INFO:tasks.workunit.client.0.smithi042.stdout:7/493: write d3/d98/da0/d41/f96 [1067364,119800] 0 2024-09-16T23:48:59.679 INFO:tasks.workunit.client.0.smithi042.stdout:0/377: creat d0/df/d13/d2d/d35/f73 x:0 0 0 2024-09-16T23:48:59.679 INFO:tasks.workunit.client.1.smithi043.stdout:7/550: unlink d4/d27/d41/d1b/f6f 0 2024-09-16T23:48:59.679 INFO:tasks.workunit.client.0.smithi042.stdout:1/569: write d0/d9/d90/d6b/d88/f1e [756822,59287] 0 2024-09-16T23:48:59.682 INFO:tasks.workunit.client.1.smithi043.stdout:1/736: dwrite d4/d9/d12/faa [0,4194304] 0 2024-09-16T23:48:59.683 INFO:tasks.workunit.client.1.smithi043.stdout:2/530: mkdir d8/d1d/d20/d23/d38/db2 0 2024-09-16T23:48:59.683 INFO:tasks.workunit.client.1.smithi043.stdout:7/551: stat d4/d27/d41/d1b/c20 0 2024-09-16T23:48:59.683 INFO:tasks.workunit.client.1.smithi043.stdout:0/608: dread d0/d2/f33 [0,4194304] 0 2024-09-16T23:48:59.683 INFO:tasks.workunit.client.1.smithi043.stdout:4/638: dread d8/d15/d20/d2b/d54/d55/f7a [0,4194304] 0 2024-09-16T23:48:59.684 INFO:tasks.workunit.client.0.smithi042.stdout:7/494: mkdir d3/d98/da0/d32/d6b/d5b/dae 0 2024-09-16T23:48:59.684 INFO:tasks.workunit.client.0.smithi042.stdout:5/619: dread d9/db/de/f7f [0,4194304] 0 2024-09-16T23:48:59.684 INFO:tasks.workunit.client.1.smithi043.stdout:5/783: rename d3/db/d19/d76/dba to d3/d9/d22/d4a/d4c/d7a/d10a 0 2024-09-16T23:48:59.686 INFO:tasks.workunit.client.0.smithi042.stdout:7/495: stat d3/d98/da0/d24/f6a 0 2024-09-16T23:48:59.687 INFO:tasks.workunit.client.1.smithi043.stdout:1/737: creat d4/d9/d72/ffe x:0 0 0 2024-09-16T23:48:59.689 INFO:tasks.workunit.client.1.smithi043.stdout:7/552: creat d4/db/d11/d21/d31/fb5 x:0 0 0 2024-09-16T23:48:59.689 INFO:tasks.workunit.client.0.smithi042.stdout:7/496: creat d3/d98/da0/d32/d6b/faf x:0 0 0 2024-09-16T23:48:59.691 INFO:tasks.workunit.client.1.smithi043.stdout:7/553: chown d4/d27/d41/d1b/c20 1 1 2024-09-16T23:48:59.691 INFO:tasks.workunit.client.1.smithi043.stdout:4/639: fsync d8/d15/d20/d2b/d54/d55/f7a 0 2024-09-16T23:48:59.693 INFO:tasks.workunit.client.0.smithi042.stdout:5/620: dwrite d9/db/de/d5e/fcb [0,4194304] 0 2024-09-16T23:48:59.693 INFO:tasks.workunit.client.1.smithi043.stdout:0/609: unlink d0/d2/df/d1a/d46/d39/d83/dcb/f8f 0 2024-09-16T23:48:59.694 INFO:tasks.workunit.client.0.smithi042.stdout:7/497: write d3/d98/da0/d41/f62 [77326,104165] 0 2024-09-16T23:48:59.695 INFO:tasks.workunit.client.1.smithi043.stdout:7/554: stat d4/d27/d41/d1b/d66/da3 0 2024-09-16T23:48:59.696 INFO:tasks.workunit.client.1.smithi043.stdout:2/531: dwrite d8/d1f/f2e [0,4194304] 0 2024-09-16T23:48:59.696 INFO:tasks.workunit.client.1.smithi043.stdout:0/610: stat d0/d2/df/da7 0 2024-09-16T23:48:59.697 INFO:tasks.workunit.client.0.smithi042.stdout:7/498: mkdir d3/d98/da0/d41/db0 0 2024-09-16T23:48:59.698 INFO:tasks.workunit.client.1.smithi043.stdout:1/738: symlink d4/d9/d12/d2b/dc9/lff 0 2024-09-16T23:48:59.699 INFO:tasks.workunit.client.0.smithi042.stdout:5/621: rename d9/db/d1b/da4/la8 to d9/d2c/d7b/ld6 0 2024-09-16T23:48:59.700 INFO:tasks.workunit.client.1.smithi043.stdout:5/784: dwrite d3/d9/d22/d9f/fc1 [0,4194304] 0 2024-09-16T23:48:59.700 INFO:tasks.workunit.client.1.smithi043.stdout:7/555: creat d4/d27/d41/fb6 x:0 0 0 2024-09-16T23:48:59.701 INFO:tasks.workunit.client.1.smithi043.stdout:0/611: read d0/fa3 [762778,9486] 0 2024-09-16T23:48:59.702 INFO:tasks.workunit.client.1.smithi043.stdout:2/532: symlink d8/d4b/d4f/lb3 0 2024-09-16T23:48:59.703 INFO:tasks.workunit.client.0.smithi042.stdout:7/499: rename d3/d98/da0/d41/f7f to d3/d98/da0/d32/d6b/d5b/dae/fb1 0 2024-09-16T23:48:59.703 INFO:tasks.workunit.client.1.smithi043.stdout:7/556: chown d4/f62 309818 1 2024-09-16T23:48:59.703 INFO:tasks.workunit.client.1.smithi043.stdout:1/739: symlink d4/d9/l100 0 2024-09-16T23:48:59.704 INFO:tasks.workunit.client.0.smithi042.stdout:5/622: creat d9/db/d11/fd7 x:0 0 0 2024-09-16T23:48:59.704 INFO:tasks.workunit.client.1.smithi043.stdout:5/785: creat d3/db/d84/f10b x:0 0 0 2024-09-16T23:48:59.706 INFO:tasks.workunit.client.1.smithi043.stdout:2/533: creat d8/d5e/fb4 x:0 0 0 2024-09-16T23:48:59.707 INFO:tasks.workunit.client.1.smithi043.stdout:4/640: dwrite d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:48:59.707 INFO:tasks.workunit.client.1.smithi043.stdout:5/786: chown d3/d9/d22/f61 862198124 1 2024-09-16T23:48:59.707 INFO:tasks.workunit.client.1.smithi043.stdout:7/557: dread - d4/db/d43/d83/d85/d8b/f9f zero size 2024-09-16T23:48:59.709 INFO:tasks.workunit.client.1.smithi043.stdout:1/740: mknod d4/d9/d8f/db6/c101 0 2024-09-16T23:48:59.709 INFO:tasks.workunit.client.1.smithi043.stdout:4/641: fdatasync d8/d15/d20/d2b/f26 0 2024-09-16T23:48:59.710 INFO:tasks.workunit.client.1.smithi043.stdout:2/534: creat d8/d1d/d20/d26/fb5 x:0 0 0 2024-09-16T23:48:59.711 INFO:tasks.workunit.client.0.smithi042.stdout:5/623: dwrite d9/db/d1b/d79/fb7 [4194304,4194304] 0 2024-09-16T23:48:59.711 INFO:tasks.workunit.client.1.smithi043.stdout:4/642: stat d8/d15/d20/d2b/d54/db3/d4c/c59 0 2024-09-16T23:48:59.711 INFO:tasks.workunit.client.1.smithi043.stdout:4/643: fsync d8/d25/d39/fac 0 2024-09-16T23:48:59.714 INFO:tasks.workunit.client.1.smithi043.stdout:5/787: unlink d3/d9/d22/d4a/d64/d8f/fc9 0 2024-09-16T23:48:59.714 INFO:tasks.workunit.client.0.smithi042.stdout:5/624: symlink d9/db/de/d10/ld8 0 2024-09-16T23:48:59.716 INFO:tasks.workunit.client.0.smithi042.stdout:5/625: stat d9/db/de/d5e/d87/d61/f5d 0 2024-09-16T23:48:59.716 INFO:tasks.workunit.client.1.smithi043.stdout:2/535: mkdir d8/d4b/d5f/d63/db6 0 2024-09-16T23:48:59.718 INFO:tasks.workunit.client.1.smithi043.stdout:0/612: getdents d0/d2/df/d1a/d3b 0 2024-09-16T23:48:59.720 INFO:tasks.workunit.client.0.smithi042.stdout:7/500: dread d3/d1f/d49/d76/d19/f21 [0,4194304] 0 2024-09-16T23:48:59.720 INFO:tasks.workunit.client.1.smithi043.stdout:4/644: creat d8/d15/d20/d2b/d54/db3/d4c/fc6 x:0 0 0 2024-09-16T23:48:59.721 INFO:tasks.workunit.client.1.smithi043.stdout:5/788: mkdir d3/db/d84/db6/d10c 0 2024-09-16T23:48:59.722 INFO:tasks.workunit.client.1.smithi043.stdout:2/536: creat d8/d11/d16/fb7 x:0 0 0 2024-09-16T23:48:59.723 INFO:tasks.workunit.client.1.smithi043.stdout:0/613: symlink d0/d2/df/d1a/d46/lcf 0 2024-09-16T23:48:59.723 INFO:tasks.workunit.client.0.smithi042.stdout:7/501: mknod d3/d98/da0/d41/cb2 0 2024-09-16T23:48:59.724 INFO:tasks.workunit.client.1.smithi043.stdout:2/537: truncate d8/d1d/f5a 686786 0 2024-09-16T23:48:59.724 INFO:tasks.workunit.client.1.smithi043.stdout:4/645: unlink d8/d25/d39/d41/d57/ca3 0 2024-09-16T23:48:59.726 INFO:tasks.workunit.client.0.smithi042.stdout:5/626: link d9/d63/f9e d9/db/de/d5e/d87/d61/fd9 0 2024-09-16T23:48:59.727 INFO:tasks.workunit.client.0.smithi042.stdout:5/627: stat d9/db/d1b/d31/d6c/c82 0 2024-09-16T23:48:59.728 INFO:tasks.workunit.client.1.smithi043.stdout:5/789: creat d3/d9/d6f/f10d x:0 0 0 2024-09-16T23:48:59.728 INFO:tasks.workunit.client.1.smithi043.stdout:1/741: dread d4/d9/d72/f73 [0,4194304] 0 2024-09-16T23:48:59.730 INFO:tasks.workunit.client.0.smithi042.stdout:5/628: read d9/db/de/d5e/d87/d61/f5d [891120,59729] 0 2024-09-16T23:48:59.730 INFO:tasks.workunit.client.1.smithi043.stdout:5/790: unlink d3/db/d26/ce7 0 2024-09-16T23:48:59.731 INFO:tasks.workunit.client.1.smithi043.stdout:4/646: truncate d8/d25/d39/d41/f9e 453750 0 2024-09-16T23:48:59.732 INFO:tasks.workunit.client.1.smithi043.stdout:1/742: symlink d4/df/df4/d83/d98/l102 0 2024-09-16T23:48:59.732 INFO:tasks.workunit.client.1.smithi043.stdout:5/791: creat d3/db/d19/d76/f10e x:0 0 0 2024-09-16T23:48:59.733 INFO:tasks.workunit.client.0.smithi042.stdout:5/629: mknod d9/db/de/d10/dc8/cda 0 2024-09-16T23:48:59.733 INFO:tasks.workunit.client.1.smithi043.stdout:4/647: write d8/d15/d20/d2b/d22/f24 [4431623,116250] 0 2024-09-16T23:48:59.733 INFO:tasks.workunit.client.1.smithi043.stdout:5/792: stat d3/db/c25 0 2024-09-16T23:48:59.735 INFO:tasks.workunit.client.1.smithi043.stdout:4/648: write d8/d15/d20/d2b/d22/f24 [7999427,16973] 0 2024-09-16T23:48:59.736 INFO:tasks.workunit.client.0.smithi042.stdout:5/630: unlink d9/db/d19/d21/f2e 0 2024-09-16T23:48:59.736 INFO:tasks.workunit.client.1.smithi043.stdout:1/743: creat d4/d9/d12/d15/dac/f103 x:0 0 0 2024-09-16T23:48:59.737 INFO:tasks.workunit.client.1.smithi043.stdout:5/793: readlink d3/d79/dcc/lee 0 2024-09-16T23:48:59.738 INFO:tasks.workunit.client.1.smithi043.stdout:2/538: dread d8/d1d/d20/d23/d38/f66 [0,4194304] 0 2024-09-16T23:48:59.740 INFO:tasks.workunit.client.1.smithi043.stdout:4/649: dread d8/f5d [0,4194304] 0 2024-09-16T23:48:59.741 INFO:tasks.workunit.client.1.smithi043.stdout:4/650: chown d8/d15/d20/d2b/c92 2322 1 2024-09-16T23:48:59.741 INFO:tasks.workunit.client.1.smithi043.stdout:2/539: rmdir d8/d35 39 2024-09-16T23:48:59.741 INFO:tasks.workunit.client.0.smithi042.stdout:8/332: sync 2024-09-16T23:48:59.742 INFO:tasks.workunit.client.0.smithi042.stdout:5/631: dwrite d9/db/de/fc7 [0,4194304] 0 2024-09-16T23:48:59.742 INFO:tasks.workunit.client.1.smithi043.stdout:4/651: stat d8/d15/d20/d2b/d54/d55/f95 0 2024-09-16T23:48:59.742 INFO:tasks.workunit.client.1.smithi043.stdout:4/652: chown c6 58 1 2024-09-16T23:48:59.743 INFO:tasks.workunit.client.0.smithi042.stdout:5/632: write d9/db/d1b/d31/d46/d58/fbb [247654,41650] 0 2024-09-16T23:48:59.745 INFO:tasks.workunit.client.1.smithi043.stdout:4/653: rename d8/d15/d20/d2b/d23/d27/c6f to d8/d15/d20/d2b/d54/d55/cc7 0 2024-09-16T23:48:59.746 INFO:tasks.workunit.client.1.smithi043.stdout:4/654: chown d8/d15/d9b/d8b/f93 65 1 2024-09-16T23:48:59.747 INFO:tasks.workunit.client.1.smithi043.stdout:2/540: creat d8/d1d/d20/d23/fb8 x:0 0 0 2024-09-16T23:48:59.747 INFO:tasks.workunit.client.0.smithi042.stdout:8/333: creat da/d19/d1d/f63 x:0 0 0 2024-09-16T23:48:59.748 INFO:tasks.workunit.client.1.smithi043.stdout:2/541: chown d8/d5e/fac 19294 1 2024-09-16T23:48:59.753 INFO:tasks.workunit.client.1.smithi043.stdout:2/542: dwrite d8/d1d/d20/d23/d38/f69 [0,4194304] 0 2024-09-16T23:48:59.760 INFO:tasks.workunit.client.1.smithi043.stdout:2/543: dwrite d8/d11/f6f [0,4194304] 0 2024-09-16T23:48:59.762 INFO:tasks.workunit.client.1.smithi043.stdout:2/544: fdatasync d8/d5e/f87 0 2024-09-16T23:48:59.764 INFO:tasks.workunit.client.1.smithi043.stdout:2/545: write d8/d4b/d5f/d63/f76 [521652,125061] 0 2024-09-16T23:48:59.765 INFO:tasks.workunit.client.1.smithi043.stdout:2/546: write d8/d5e/d83/fa7 [434295,124166] 0 2024-09-16T23:48:59.775 INFO:tasks.workunit.client.1.smithi043.stdout:2/547: dwrite d8/d4b/d5f/d63/f99 [0,4194304] 0 2024-09-16T23:48:59.776 INFO:tasks.workunit.client.0.smithi042.stdout:4/667: dread de/d12/f48 [0,4194304] 0 2024-09-16T23:48:59.777 INFO:tasks.workunit.client.0.smithi042.stdout:4/668: chown de/d42/l6c 4524093 1 2024-09-16T23:48:59.780 INFO:tasks.workunit.client.1.smithi043.stdout:2/548: rmdir d8 39 2024-09-16T23:48:59.782 INFO:tasks.workunit.client.1.smithi043.stdout:2/549: stat d8/d11/l12 0 2024-09-16T23:48:59.783 INFO:tasks.workunit.client.0.smithi042.stdout:4/669: mkdir de/de5 0 2024-09-16T23:48:59.783 INFO:tasks.workunit.client.1.smithi043.stdout:2/550: dread - d8/d1d/d20/d26/fb5 zero size 2024-09-16T23:48:59.788 INFO:tasks.workunit.client.0.smithi042.stdout:4/670: dread de/d42/d82/fcc [0,4194304] 0 2024-09-16T23:48:59.793 INFO:tasks.workunit.client.0.smithi042.stdout:0/378: write d0/df/d23/f38 [1334374,89637] 0 2024-09-16T23:48:59.796 INFO:tasks.workunit.client.0.smithi042.stdout:4/671: dwrite de/d42/d82/d94/fc9 [0,4194304] 0 2024-09-16T23:48:59.802 INFO:tasks.workunit.client.0.smithi042.stdout:4/672: dread de/d42/f50 [0,4194304] 0 2024-09-16T23:48:59.813 INFO:tasks.workunit.client.1.smithi043.stdout:2/551: creat d8/d35/fb9 x:0 0 0 2024-09-16T23:48:59.814 INFO:tasks.workunit.client.1.smithi043.stdout:4/655: dread d8/d15/d20/d2b/d23/f38 [0,4194304] 0 2024-09-16T23:48:59.815 INFO:tasks.workunit.client.1.smithi043.stdout:2/552: write d8/d11/d16/f81 [54998,124397] 0 2024-09-16T23:48:59.819 INFO:tasks.workunit.client.1.smithi043.stdout:7/558: creat d4/d27/d41/d1b/d66/da3/fb7 x:0 0 0 2024-09-16T23:48:59.820 INFO:tasks.workunit.client.1.smithi043.stdout:4/656: mkdir d8/d15/d20/d2b/dc8 0 2024-09-16T23:48:59.822 INFO:tasks.workunit.client.1.smithi043.stdout:7/559: creat d4/db/d11/d21/d31/fb8 x:0 0 0 2024-09-16T23:48:59.823 INFO:tasks.workunit.client.1.smithi043.stdout:7/560: chown d4/db/d43/d83/d85 6808728 1 2024-09-16T23:48:59.827 INFO:tasks.workunit.client.1.smithi043.stdout:7/561: mknod d4/db/d11/d21/d94/cb9 0 2024-09-16T23:48:59.828 INFO:tasks.workunit.client.1.smithi043.stdout:2/553: link d8/d1d/d25/c2c d8/d1d/d20/d23/cba 0 2024-09-16T23:48:59.830 INFO:tasks.workunit.client.1.smithi043.stdout:0/614: truncate d0/fa3 832389 0 2024-09-16T23:48:59.833 INFO:tasks.workunit.client.1.smithi043.stdout:7/562: creat d4/db/d43/d83/fba x:0 0 0 2024-09-16T23:48:59.834 INFO:tasks.workunit.client.1.smithi043.stdout:0/615: symlink d0/d2/df/d1a/d46/d39/daa/db7/ld0 0 2024-09-16T23:48:59.835 INFO:tasks.workunit.client.1.smithi043.stdout:2/554: link d8/d1d/d25/f2b d8/d4b/d5f/d8e/fbb 0 2024-09-16T23:48:59.837 INFO:tasks.workunit.client.1.smithi043.stdout:0/616: mkdir d0/d2/df/d1a/d49/dd1 0 2024-09-16T23:48:59.838 INFO:tasks.workunit.client.1.smithi043.stdout:1/744: write d4/d9/d1c/d1e/d42/d5d/f93 [1699009,90312] 0 2024-09-16T23:48:59.839 INFO:tasks.workunit.client.1.smithi043.stdout:2/555: fsync d8/d1d/d20/d26/f3f 0 2024-09-16T23:48:59.839 INFO:tasks.workunit.client.1.smithi043.stdout:3/610: sync 2024-09-16T23:48:59.840 INFO:tasks.workunit.client.1.smithi043.stdout:0/617: dread - d0/d2/df/d1a/d49/d80/fcd zero size 2024-09-16T23:48:59.840 INFO:tasks.workunit.client.1.smithi043.stdout:1/745: readlink d4/d34/lad 0 2024-09-16T23:48:59.841 INFO:tasks.workunit.client.1.smithi043.stdout:2/556: readlink d8/d4b/d4f/lb3 0 2024-09-16T23:48:59.844 INFO:tasks.workunit.client.1.smithi043.stdout:3/611: fsync d4/de/d40/f69 0 2024-09-16T23:48:59.847 INFO:tasks.workunit.client.1.smithi043.stdout:1/746: creat d4/d9/d12/d15/f104 x:0 0 0 2024-09-16T23:48:59.848 INFO:tasks.workunit.client.1.smithi043.stdout:7/563: dread d4/db/d11/d21/d31/f82 [0,4194304] 0 2024-09-16T23:48:59.848 INFO:tasks.workunit.client.1.smithi043.stdout:0/618: dwrite d0/d2/df/d1a/d49/d80/fcd [0,4194304] 0 2024-09-16T23:48:59.852 INFO:tasks.workunit.client.1.smithi043.stdout:3/612: mkdir d4/d30/d53/d88/db1/dbb 0 2024-09-16T23:48:59.858 INFO:tasks.workunit.client.1.smithi043.stdout:0/619: unlink d0/d2/df/dbb/c21 0 2024-09-16T23:48:59.858 INFO:tasks.workunit.client.1.smithi043.stdout:3/613: dwrite d4/de/f55 [0,4194304] 0 2024-09-16T23:48:59.860 INFO:tasks.workunit.client.1.smithi043.stdout:0/620: truncate d0/d2/df/f17 1441399 0 2024-09-16T23:48:59.863 INFO:tasks.workunit.client.1.smithi043.stdout:1/747: rmdir d4/df/d26/d79/dfd 0 2024-09-16T23:48:59.868 INFO:tasks.workunit.client.1.smithi043.stdout:1/748: unlink d4/d9/d8f/db6/dcc/ld0 0 2024-09-16T23:48:59.871 INFO:tasks.workunit.client.1.smithi043.stdout:4/657: write d8/d15/d20/d2b/d23/f38 [2044943,13014] 0 2024-09-16T23:48:59.871 INFO:tasks.workunit.client.1.smithi043.stdout:4/658: stat d8/d15/d9b/dbf 0 2024-09-16T23:48:59.873 INFO:tasks.workunit.client.1.smithi043.stdout:1/749: creat d4/df/d13/f105 x:0 0 0 2024-09-16T23:48:59.876 INFO:tasks.workunit.client.1.smithi043.stdout:1/750: fdatasync d4/df/d26/f39 0 2024-09-16T23:48:59.882 INFO:tasks.workunit.client.1.smithi043.stdout:4/659: link d8/d15/d20/d2b/c78 d8/d15/d20/d2b/d54/db3/cc9 0 2024-09-16T23:48:59.883 INFO:tasks.workunit.client.1.smithi043.stdout:4/660: creat d8/d25/d39/d58/fca x:0 0 0 2024-09-16T23:48:59.884 INFO:tasks.workunit.client.1.smithi043.stdout:4/661: write d8/d25/d39/d41/fb9 [791264,16257] 0 2024-09-16T23:48:59.886 INFO:tasks.workunit.client.1.smithi043.stdout:4/662: truncate d8/d25/d39/fac 270788 0 2024-09-16T23:48:59.887 INFO:tasks.workunit.client.1.smithi043.stdout:2/557: truncate d8/d35/f37 1889535 0 2024-09-16T23:48:59.888 INFO:tasks.workunit.client.1.smithi043.stdout:4/663: stat d8/d25/d64/fa0 0 2024-09-16T23:48:59.889 INFO:tasks.workunit.client.1.smithi043.stdout:4/664: read - d8/d25/d64/fa0 zero size 2024-09-16T23:48:59.890 INFO:tasks.workunit.client.1.smithi043.stdout:2/558: symlink d8/d1d/lbc 0 2024-09-16T23:48:59.890 INFO:tasks.workunit.client.0.smithi042.stdout:4/673: creat de/d12/d1b/d8a/fe6 x:0 0 0 2024-09-16T23:48:59.895 INFO:tasks.workunit.client.0.smithi042.stdout:0/379: creat d0/df/d13/f74 x:0 0 0 2024-09-16T23:48:59.895 INFO:tasks.workunit.client.1.smithi043.stdout:0/621: write d0/d2/df/f16 [731490,106867] 0 2024-09-16T23:48:59.896 INFO:tasks.workunit.client.1.smithi043.stdout:7/564: dwrite d4/f60 [0,4194304] 0 2024-09-16T23:48:59.898 INFO:tasks.workunit.client.0.smithi042.stdout:5/633: read d9/db/d19/d21/f2b [1779313,96129] 0 2024-09-16T23:48:59.901 INFO:tasks.workunit.client.1.smithi043.stdout:7/565: truncate d4/db/d43/f79 842904 0 2024-09-16T23:48:59.901 INFO:tasks.workunit.client.1.smithi043.stdout:2/559: creat d8/d1d/d25/fbd x:0 0 0 2024-09-16T23:48:59.901 INFO:tasks.workunit.client.0.smithi042.stdout:5/634: stat d9/db/d19/d21 0 2024-09-16T23:48:59.902 INFO:tasks.workunit.client.1.smithi043.stdout:3/614: dwrite d4/de/d1f/d2c/f47 [0,4194304] 0 2024-09-16T23:48:59.902 INFO:tasks.workunit.client.1.smithi043.stdout:7/566: dread - d4/d27/d8a/faa zero size 2024-09-16T23:48:59.904 INFO:tasks.workunit.client.0.smithi042.stdout:8/334: dwrite da/d19/d1d/d30/d3f/f53 [0,4194304] 0 2024-09-16T23:48:59.905 INFO:tasks.workunit.client.0.smithi042.stdout:7/502: dwrite d3/d1f/d49/d76/d19/f21 [4194304,4194304] 0 2024-09-16T23:48:59.906 INFO:tasks.workunit.client.0.smithi042.stdout:4/674: dread de/d12/f15 [0,4194304] 0 2024-09-16T23:48:59.907 INFO:tasks.workunit.client.0.smithi042.stdout:1/570: dread d0/d17/f20 [0,4194304] 0 2024-09-16T23:48:59.907 INFO:tasks.workunit.client.0.smithi042.stdout:5/635: creat d9/db/d19/fdb x:0 0 0 2024-09-16T23:48:59.909 INFO:tasks.workunit.client.1.smithi043.stdout:1/751: truncate d4/d9/d12/f5c 2271223 0 2024-09-16T23:48:59.913 INFO:tasks.workunit.client.1.smithi043.stdout:7/567: mknod d4/db/d11/d21/d31/d33/cbb 0 2024-09-16T23:48:59.914 INFO:tasks.workunit.client.1.smithi043.stdout:7/568: write d4/db/d43/f79 [1194663,62723] 0 2024-09-16T23:48:59.915 INFO:tasks.workunit.client.0.smithi042.stdout:4/675: mkdir de/d42/d85/db3/de7 0 2024-09-16T23:48:59.916 INFO:tasks.workunit.client.1.smithi043.stdout:7/569: fsync d4/db/d43/d83/d85/f96 0 2024-09-16T23:48:59.917 INFO:tasks.workunit.client.0.smithi042.stdout:8/335: rename da/d19/d2c/l42 to da/d19/d2c/d56/l64 0 2024-09-16T23:48:59.917 INFO:tasks.workunit.client.0.smithi042.stdout:8/336: stat da 0 2024-09-16T23:48:59.918 INFO:tasks.workunit.client.0.smithi042.stdout:4/676: mknod de/d42/d85/db3/de7/ce8 0 2024-09-16T23:48:59.919 INFO:tasks.workunit.client.1.smithi043.stdout:1/752: mkdir d4/d9/d8f/db6/dcc/dda/d106 0 2024-09-16T23:48:59.921 INFO:tasks.workunit.client.1.smithi043.stdout:3/615: dread d4/de/d1f/d34/fa8 [0,4194304] 0 2024-09-16T23:48:59.926 INFO:tasks.workunit.client.0.smithi042.stdout:8/337: dread da/d19/d2c/f4e [0,4194304] 0 2024-09-16T23:48:59.926 INFO:tasks.workunit.client.0.smithi042.stdout:7/503: write d3/f8 [1874635,2700] 0 2024-09-16T23:48:59.926 INFO:tasks.workunit.client.1.smithi043.stdout:7/570: fsync d4/d27/d41/f8d 0 2024-09-16T23:48:59.927 INFO:tasks.workunit.client.0.smithi042.stdout:7/504: fdatasync d3/d98/da0/d41/f47 0 2024-09-16T23:48:59.928 INFO:tasks.workunit.client.0.smithi042.stdout:4/677: rename de/d42/d5d/d8f/d9f/dc0/fd8 to de/d12/d7d/fe9 0 2024-09-16T23:48:59.929 INFO:tasks.workunit.client.0.smithi042.stdout:7/505: write d3/f84 [1135277,42527] 0 2024-09-16T23:48:59.930 INFO:tasks.workunit.client.1.smithi043.stdout:3/616: dwrite d4/fa0 [0,4194304] 0 2024-09-16T23:48:59.931 INFO:tasks.workunit.client.0.smithi042.stdout:1/571: getdents d0/d3b 0 2024-09-16T23:48:59.932 INFO:tasks.workunit.client.0.smithi042.stdout:4/678: creat de/d42/d85/db3/de7/fea x:0 0 0 2024-09-16T23:48:59.936 INFO:tasks.workunit.client.0.smithi042.stdout:4/679: stat de/d1a/l43 0 2024-09-16T23:48:59.937 INFO:tasks.workunit.client.0.smithi042.stdout:1/572: write d0/d9/d90/d6b/d88/d32/d2f/fb0 [542074,40121] 0 2024-09-16T23:48:59.937 INFO:tasks.workunit.client.0.smithi042.stdout:7/506: rmdir d3/d1f/d49/d78/d8a 0 2024-09-16T23:48:59.937 INFO:tasks.workunit.client.1.smithi043.stdout:1/753: rename d4/df/d13 to d4/d9/d8f/db6/dcc/dda/d107 0 2024-09-16T23:48:59.938 INFO:tasks.workunit.client.0.smithi042.stdout:4/680: readlink de/d42/d5d/d8f/le2 0 2024-09-16T23:48:59.938 INFO:tasks.workunit.client.0.smithi042.stdout:7/507: fsync d3/d98/da0/d32/d6b/faf 0 2024-09-16T23:48:59.942 INFO:tasks.workunit.client.0.smithi042.stdout:8/338: rename da/f29 to da/d4b/f65 0 2024-09-16T23:48:59.946 INFO:tasks.workunit.client.0.smithi042.stdout:4/681: dwrite de/d42/d5d/f8c [0,4194304] 0 2024-09-16T23:48:59.946 INFO:tasks.workunit.client.0.smithi042.stdout:1/573: unlink d0/d9/d4e/f7e 0 2024-09-16T23:48:59.949 INFO:tasks.workunit.client.0.smithi042.stdout:8/339: creat da/d19/d2c/f66 x:0 0 0 2024-09-16T23:48:59.949 INFO:tasks.workunit.client.0.smithi042.stdout:4/682: creat de/d12/feb x:0 0 0 2024-09-16T23:48:59.950 INFO:tasks.workunit.client.1.smithi043.stdout:2/560: truncate d8/d35/f37 1977208 0 2024-09-16T23:48:59.951 INFO:tasks.workunit.client.0.smithi042.stdout:8/340: read da/d19/f3b [6240654,4643] 0 2024-09-16T23:48:59.952 INFO:tasks.workunit.client.1.smithi043.stdout:7/571: mkdir d4/d27/dbc 0 2024-09-16T23:48:59.953 INFO:tasks.workunit.client.0.smithi042.stdout:4/683: truncate de/d12/d7d/fa1 732466 0 2024-09-16T23:48:59.954 INFO:tasks.workunit.client.1.smithi043.stdout:7/572: read d4/db/f51 [1689466,25533] 0 2024-09-16T23:48:59.955 INFO:tasks.workunit.client.0.smithi042.stdout:1/574: fsync d0/d9/f94 0 2024-09-16T23:48:59.956 INFO:tasks.workunit.client.0.smithi042.stdout:4/684: symlink de/d42/d5d/d8f/dd6/lec 0 2024-09-16T23:48:59.957 INFO:tasks.workunit.client.0.smithi042.stdout:4/685: read - de/d42/d5d/f74 zero size 2024-09-16T23:48:59.958 INFO:tasks.workunit.client.0.smithi042.stdout:4/686: chown de/d1a/d70/dac/lbc 198 1 2024-09-16T23:48:59.958 INFO:tasks.workunit.client.0.smithi042.stdout:4/687: dread - de/d12/feb zero size 2024-09-16T23:48:59.959 INFO:tasks.workunit.client.1.smithi043.stdout:1/754: fsync d4/d9/d1c/d1e/d47/fab 0 2024-09-16T23:48:59.960 INFO:tasks.workunit.client.1.smithi043.stdout:1/755: chown d4/df/df4/d83/fc4 72 1 2024-09-16T23:48:59.961 INFO:tasks.workunit.client.0.smithi042.stdout:8/341: readlink da/l15 0 2024-09-16T23:48:59.962 INFO:tasks.workunit.client.0.smithi042.stdout:4/688: creat de/d2e/fed x:0 0 0 2024-09-16T23:48:59.963 INFO:tasks.workunit.client.1.smithi043.stdout:0/622: rmdir d0/d2/df/dbb 39 2024-09-16T23:48:59.965 INFO:tasks.workunit.client.0.smithi042.stdout:1/575: chown d0/d72/l6e 381067114 1 2024-09-16T23:48:59.967 INFO:tasks.workunit.client.0.smithi042.stdout:8/342: dwrite da/d19/d1d/f2b [0,4194304] 0 2024-09-16T23:48:59.970 INFO:tasks.workunit.client.1.smithi043.stdout:7/573: creat d4/db/d11/d15/d29/fbd x:0 0 0 2024-09-16T23:48:59.970 INFO:tasks.workunit.client.1.smithi043.stdout:7/574: chown d4/l69 55 1 2024-09-16T23:48:59.973 INFO:tasks.workunit.client.0.smithi042.stdout:8/343: dwrite da/d19/d1d/f33 [0,4194304] 0 2024-09-16T23:48:59.975 INFO:tasks.workunit.client.0.smithi042.stdout:1/576: getdents d0/d9/d90/d6b 0 2024-09-16T23:48:59.976 INFO:tasks.workunit.client.0.smithi042.stdout:8/344: creat da/d19/d2c/f67 x:0 0 0 2024-09-16T23:48:59.978 INFO:tasks.workunit.client.0.smithi042.stdout:1/577: creat d0/d3b/d8d/fcc x:0 0 0 2024-09-16T23:48:59.982 INFO:tasks.workunit.client.1.smithi043.stdout:7/575: mkdir d4/db/d11/d15/dbe 0 2024-09-16T23:48:59.982 INFO:tasks.workunit.client.0.smithi042.stdout:8/345: mknod da/c68 0 2024-09-16T23:48:59.985 INFO:tasks.workunit.client.0.smithi042.stdout:1/578: creat d0/d17/d18/fcd x:0 0 0 2024-09-16T23:48:59.986 INFO:tasks.workunit.client.1.smithi043.stdout:2/561: rmdir d8/d1d/d20/d23/da8 0 2024-09-16T23:48:59.987 INFO:tasks.workunit.client.1.smithi043.stdout:3/617: getdents d4/de/d1f/d2c 0 2024-09-16T23:48:59.988 INFO:tasks.workunit.client.1.smithi043.stdout:3/618: chown d4/de/d35/f4c 636 1 2024-09-16T23:48:59.988 INFO:tasks.workunit.client.1.smithi043.stdout:3/619: dread - d4/fb8 zero size 2024-09-16T23:48:59.990 INFO:tasks.workunit.client.1.smithi043.stdout:1/756: creat d4/d9/d12/d15/f108 x:0 0 0 2024-09-16T23:48:59.990 INFO:tasks.workunit.client.0.smithi042.stdout:1/579: mknod d0/db2/cce 0 2024-09-16T23:48:59.992 INFO:tasks.workunit.client.1.smithi043.stdout:7/576: truncate d4/f2b 192016 0 2024-09-16T23:48:59.993 INFO:tasks.workunit.client.0.smithi042.stdout:8/346: creat da/d19/d1d/d30/d49/f69 x:0 0 0 2024-09-16T23:48:59.994 INFO:tasks.workunit.client.0.smithi042.stdout:8/347: readlink da/d19/l28 0 2024-09-16T23:48:59.994 INFO:tasks.workunit.client.1.smithi043.stdout:3/620: mknod d4/de/d40/cbc 0 2024-09-16T23:48:59.995 INFO:tasks.workunit.client.1.smithi043.stdout:3/621: chown d4/de/d40/fb5 842046 1 2024-09-16T23:48:59.997 INFO:tasks.workunit.client.0.smithi042.stdout:8/348: mkdir da/d19/d1d/d30/d49/d58/d6a 0 2024-09-16T23:48:59.998 INFO:tasks.workunit.client.1.smithi043.stdout:0/623: truncate d0/d2/df/d1a/d46/d39/f45 1338354 0 2024-09-16T23:48:59.998 INFO:tasks.workunit.client.1.smithi043.stdout:0/624: truncate d0/de/f91 1278990 0 2024-09-16T23:49:00.000 INFO:tasks.workunit.client.1.smithi043.stdout:1/757: mkdir d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/d109 0 2024-09-16T23:49:00.001 INFO:tasks.workunit.client.1.smithi043.stdout:1/758: write d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fd8 [750610,8376] 0 2024-09-16T23:49:00.003 INFO:tasks.workunit.client.1.smithi043.stdout:7/577: symlink d4/db/d11/d21/d31/d33/d64/lbf 0 2024-09-16T23:49:00.006 INFO:tasks.workunit.client.1.smithi043.stdout:3/622: creat d4/de/d40/d4d/fbd x:0 0 0 2024-09-16T23:49:00.008 INFO:tasks.workunit.client.1.smithi043.stdout:0/625: symlink d0/d2/df/d1a/d46/d92/ld2 0 2024-09-16T23:49:00.008 INFO:tasks.workunit.client.0.smithi042.stdout:1/580: write d0/d9/d90/fad [1435965,31414] 0 2024-09-16T23:49:00.009 INFO:tasks.workunit.client.1.smithi043.stdout:0/626: chown d0/d2/df/d1a/f9e 6538 1 2024-09-16T23:49:00.010 INFO:tasks.workunit.client.0.smithi042.stdout:1/581: chown d0/d9/d90/d6b/d88/f75 2497 1 2024-09-16T23:49:00.010 INFO:tasks.workunit.client.0.smithi042.stdout:8/349: rmdir da/d19/d1d/d30/d49/d58 39 2024-09-16T23:49:00.011 INFO:tasks.workunit.client.0.smithi042.stdout:1/582: readlink d0/d9/d90/d6b/d88/d32/d2f/l7b 0 2024-09-16T23:49:00.013 INFO:tasks.workunit.client.1.smithi043.stdout:3/623: dwrite d4/d30/d53/fac [0,4194304] 0 2024-09-16T23:49:00.013 INFO:tasks.workunit.client.1.smithi043.stdout:1/759: mknod d4/d9/d1c/d1e/d42/d5d/c10a 0 2024-09-16T23:49:00.014 INFO:tasks.workunit.client.1.smithi043.stdout:0/627: truncate d0/d2/df/d1a/d46/d39/d83/dcb/fca 1623238 0 2024-09-16T23:49:00.015 INFO:tasks.workunit.client.1.smithi043.stdout:6/620: sync 2024-09-16T23:49:00.015 INFO:tasks.workunit.client.1.smithi043.stdout:8/594: sync 2024-09-16T23:49:00.016 INFO:tasks.workunit.client.1.smithi043.stdout:2/562: link d8/d1f/f5c d8/fbe 0 2024-09-16T23:49:00.020 INFO:tasks.workunit.client.1.smithi043.stdout:3/624: dwrite d4/d30/fa6 [0,4194304] 0 2024-09-16T23:49:00.020 INFO:tasks.workunit.client.0.smithi042.stdout:1/583: dwrite d0/d9/d90/d6b/d88/d32/d2f/fb0 [0,4194304] 0 2024-09-16T23:49:00.021 INFO:tasks.workunit.client.0.smithi042.stdout:1/584: chown d0/d9/f78 33155350 1 2024-09-16T23:49:00.022 INFO:tasks.workunit.client.0.smithi042.stdout:8/350: creat da/d19/d1d/f6b x:0 0 0 2024-09-16T23:49:00.028 INFO:tasks.workunit.client.0.smithi042.stdout:8/351: mknod da/d4b/c6c 0 2024-09-16T23:49:00.032 INFO:tasks.workunit.client.0.smithi042.stdout:1/585: rename d0/d17/d18/fcd to d0/d9a/fcf 0 2024-09-16T23:49:00.035 INFO:tasks.workunit.client.0.smithi042.stdout:8/352: mkdir da/d19/d1d/d30/d3f/d6d 0 2024-09-16T23:49:00.035 INFO:tasks.workunit.client.1.smithi043.stdout:0/628: dread d0/d2/df/f16 [0,4194304] 0 2024-09-16T23:49:00.036 INFO:tasks.workunit.client.1.smithi043.stdout:6/621: truncate de/d1a/d39/f1f 1876972 0 2024-09-16T23:49:00.036 INFO:tasks.workunit.client.0.smithi042.stdout:8/353: chown da/d19/d1d/d30/d3f/f61 135 1 2024-09-16T23:49:00.037 INFO:tasks.workunit.client.0.smithi042.stdout:8/354: stat da/d19/f23 0 2024-09-16T23:49:00.037 INFO:tasks.workunit.client.1.smithi043.stdout:6/622: stat de/d1a/d28/f9a 0 2024-09-16T23:49:00.037 INFO:tasks.workunit.client.1.smithi043.stdout:0/629: fsync d0/d2/df/d1a/d46/d92/fac 0 2024-09-16T23:49:00.038 INFO:tasks.workunit.client.0.smithi042.stdout:8/355: fsync da/d19/f1f 0 2024-09-16T23:49:00.039 INFO:tasks.workunit.client.1.smithi043.stdout:1/760: fdatasync d4/d34/fc0 0 2024-09-16T23:49:00.042 INFO:tasks.workunit.client.1.smithi043.stdout:2/563: mknod d8/d35/d9f/cbf 0 2024-09-16T23:49:00.042 INFO:tasks.workunit.client.1.smithi043.stdout:2/564: readlink d8/d5e/d83/l8f 0 2024-09-16T23:49:00.043 INFO:tasks.workunit.client.0.smithi042.stdout:8/356: mknod da/d19/d1d/c6e 0 2024-09-16T23:49:00.044 INFO:tasks.workunit.client.1.smithi043.stdout:7/578: rmdir d4/d27/d41/d8f 0 2024-09-16T23:49:00.046 INFO:tasks.workunit.client.1.smithi043.stdout:3/625: symlink d4/d30/d53/d88/db1/dbb/lbe 0 2024-09-16T23:49:00.047 INFO:tasks.workunit.client.1.smithi043.stdout:3/626: write d4/de/d1f/d2c/f4a [2463546,12085] 0 2024-09-16T23:49:00.050 INFO:tasks.workunit.client.1.smithi043.stdout:0/630: creat d0/d2/df/d1a/d46/d39/daa/fd3 x:0 0 0 2024-09-16T23:49:00.054 INFO:tasks.workunit.client.1.smithi043.stdout:2/565: mkdir d8/d35/dc0 0 2024-09-16T23:49:00.054 INFO:tasks.workunit.client.0.smithi042.stdout:8/357: write da/d19/d2c/f2d [1038406,106614] 0 2024-09-16T23:49:00.055 INFO:tasks.workunit.client.1.smithi043.stdout:2/566: dread - d8/d5e/fac zero size 2024-09-16T23:49:00.057 INFO:tasks.workunit.client.1.smithi043.stdout:6/623: symlink de/dbe/dca/lcc 0 2024-09-16T23:49:00.057 INFO:tasks.workunit.client.1.smithi043.stdout:9/552: sync 2024-09-16T23:49:00.057 INFO:tasks.workunit.client.0.smithi042.stdout:8/358: chown da/c3a 3 1 2024-09-16T23:49:00.058 INFO:tasks.workunit.client.1.smithi043.stdout:8/595: truncate de/d70/d3d/fa3 1647753 0 2024-09-16T23:49:00.062 INFO:tasks.workunit.client.1.smithi043.stdout:1/761: creat d4/d3e/d5b/dca/f10b x:0 0 0 2024-09-16T23:49:00.062 INFO:tasks.workunit.client.0.smithi042.stdout:8/359: dwrite da/d19/f1f [0,4194304] 0 2024-09-16T23:49:00.063 INFO:tasks.workunit.client.1.smithi043.stdout:2/567: dread d8/d11/f32 [0,4194304] 0 2024-09-16T23:49:00.064 INFO:tasks.workunit.client.1.smithi043.stdout:1/762: dread - d4/d9/d1c/d1e/d42/d5d/f9a zero size 2024-09-16T23:49:00.066 INFO:tasks.workunit.client.1.smithi043.stdout:7/579: mkdir d4/dc0 0 2024-09-16T23:49:00.066 INFO:tasks.workunit.client.1.smithi043.stdout:0/631: dread d0/d2/df/d1a/d46/d39/f45 [0,4194304] 0 2024-09-16T23:49:00.067 INFO:tasks.workunit.client.1.smithi043.stdout:9/553: mkdir d4/d21/d2a/da9 0 2024-09-16T23:49:00.071 INFO:tasks.workunit.client.1.smithi043.stdout:2/568: chown d8/d1d/l6d 698686 1 2024-09-16T23:49:00.072 INFO:tasks.workunit.client.1.smithi043.stdout:6/624: mknod de/d1a/d28/ccd 0 2024-09-16T23:49:00.073 INFO:tasks.workunit.client.0.smithi042.stdout:8/360: rename da/f18 to da/d19/f6f 0 2024-09-16T23:49:00.074 INFO:tasks.workunit.client.1.smithi043.stdout:2/569: fsync d8/d11/f6f 0 2024-09-16T23:49:00.074 INFO:tasks.workunit.client.1.smithi043.stdout:3/627: dwrite d4/de/d40/f45 [4194304,4194304] 0 2024-09-16T23:49:00.075 INFO:tasks.workunit.client.0.smithi042.stdout:8/361: write da/d19/d2c/f67 [391584,120212] 0 2024-09-16T23:49:00.075 INFO:tasks.workunit.client.1.smithi043.stdout:2/570: read d8/d1d/f5a [225785,43504] 0 2024-09-16T23:49:00.076 INFO:tasks.workunit.client.1.smithi043.stdout:0/632: symlink d0/d2/df/d1a/d46/d39/d6e/ld4 0 2024-09-16T23:49:00.076 INFO:tasks.workunit.client.1.smithi043.stdout:3/628: write d4/de/d40/f80 [801524,81688] 0 2024-09-16T23:49:00.081 INFO:tasks.workunit.client.1.smithi043.stdout:7/580: symlink d4/d27/d8a/d93/lc1 0 2024-09-16T23:49:00.083 INFO:tasks.workunit.client.1.smithi043.stdout:6/625: creat de/d1a/d80/fce x:0 0 0 2024-09-16T23:49:00.085 INFO:tasks.workunit.client.1.smithi043.stdout:2/571: write d8/d1d/d20/d23/f6a [2443606,107993] 0 2024-09-16T23:49:00.086 INFO:tasks.workunit.client.1.smithi043.stdout:8/596: dread de/d70/d3d/f51 [0,4194304] 0 2024-09-16T23:49:00.089 INFO:tasks.workunit.client.1.smithi043.stdout:0/633: creat d0/d2/df/d1a/d46/d39/d6e/fd5 x:0 0 0 2024-09-16T23:49:00.090 INFO:tasks.workunit.client.1.smithi043.stdout:7/581: dread - d4/d27/d41/d1b/d66/da3/fb7 zero size 2024-09-16T23:49:00.091 INFO:tasks.workunit.client.1.smithi043.stdout:6/626: symlink de/dbe/dca/lcf 0 2024-09-16T23:49:00.092 INFO:tasks.workunit.client.1.smithi043.stdout:7/582: write d4/db/d28/f97 [373713,13645] 0 2024-09-16T23:49:00.092 INFO:tasks.workunit.client.1.smithi043.stdout:1/763: truncate d4/d9/d1c/d1e/f85 2670337 0 2024-09-16T23:49:00.094 INFO:tasks.workunit.client.0.smithi042.stdout:8/362: truncate da/d19/d2c/f2d 146385 0 2024-09-16T23:49:00.094 INFO:tasks.workunit.client.1.smithi043.stdout:1/764: write d4/d9/d12/d15/f104 [869559,77497] 0 2024-09-16T23:49:00.095 INFO:tasks.workunit.client.1.smithi043.stdout:0/634: dread - d0/d2/df/d1a/d3b/f99 zero size 2024-09-16T23:49:00.096 INFO:tasks.workunit.client.1.smithi043.stdout:1/765: read - d4/d9/d12/d15/f108 zero size 2024-09-16T23:49:00.097 INFO:tasks.workunit.client.1.smithi043.stdout:6/627: creat de/d1a/d80/fd0 x:0 0 0 2024-09-16T23:49:00.098 INFO:tasks.workunit.client.1.smithi043.stdout:6/628: dread - de/d2a/f9e zero size 2024-09-16T23:49:00.098 INFO:tasks.workunit.client.0.smithi042.stdout:8/363: readlink l3 0 2024-09-16T23:49:00.099 INFO:tasks.workunit.client.0.smithi042.stdout:8/364: stat da/d19/d2c/f4e 0 2024-09-16T23:49:00.099 INFO:tasks.workunit.client.1.smithi043.stdout:9/554: truncate d4/d21/d2a/d2c/f9e 3670537 0 2024-09-16T23:49:00.102 INFO:tasks.workunit.client.1.smithi043.stdout:7/583: unlink d4/l7 0 2024-09-16T23:49:00.102 INFO:tasks.workunit.client.1.smithi043.stdout:2/572: dread d8/d35/f36 [0,4194304] 0 2024-09-16T23:49:00.104 INFO:tasks.workunit.client.1.smithi043.stdout:3/629: dwrite d4/de/d40/f82 [0,4194304] 0 2024-09-16T23:49:00.105 INFO:tasks.workunit.client.1.smithi043.stdout:1/766: fdatasync d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f2f 0 2024-09-16T23:49:00.110 INFO:tasks.workunit.client.1.smithi043.stdout:4/665: sync 2024-09-16T23:49:00.110 INFO:tasks.workunit.client.1.smithi043.stdout:5/794: sync 2024-09-16T23:49:00.110 INFO:tasks.workunit.client.1.smithi043.stdout:1/767: write d4/df/d26/d79/f96 [928946,118431] 0 2024-09-16T23:49:00.111 INFO:tasks.workunit.client.1.smithi043.stdout:9/555: creat d4/d21/d2a/d2c/d7b/faa x:0 0 0 2024-09-16T23:49:00.111 INFO:tasks.workunit.client.0.smithi042.stdout:8/365: dread da/d19/f6f [0,4194304] 0 2024-09-16T23:49:00.111 INFO:tasks.workunit.client.1.smithi043.stdout:8/597: creat de/d18/d1f/d2f/d39/fb3 x:0 0 0 2024-09-16T23:49:00.114 INFO:tasks.workunit.client.1.smithi043.stdout:7/584: dwrite d4/db/d43/d83/d85/f92 [0,4194304] 0 2024-09-16T23:49:00.115 INFO:tasks.workunit.client.1.smithi043.stdout:9/556: write d4/d21/d35/d4b/f80 [721562,55694] 0 2024-09-16T23:49:00.119 INFO:tasks.workunit.client.0.smithi042.stdout:8/366: dwrite da/d19/f1e [0,4194304] 0 2024-09-16T23:49:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:59 smithi043 ceph-mon[35191]: Standby manager daemon smithi042.zrkumz started 2024-09-16T23:49:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:59 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/crt"}]: dispatch 2024-09-16T23:49:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:59 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:49:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:59 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/key"}]: dispatch 2024-09-16T23:49:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:48:59 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:49:00.130 INFO:tasks.workunit.client.1.smithi043.stdout:6/629: truncate de/d1a/d39/f17 1898477 0 2024-09-16T23:49:00.131 INFO:tasks.workunit.client.1.smithi043.stdout:6/630: chown de/d1a 233 1 2024-09-16T23:49:00.131 INFO:tasks.workunit.client.1.smithi043.stdout:4/666: fsync d8/f68 0 2024-09-16T23:49:00.132 INFO:tasks.workunit.client.1.smithi043.stdout:6/631: truncate de/d2a/f9e 685533 0 2024-09-16T23:49:00.133 INFO:tasks.workunit.client.1.smithi043.stdout:5/795: symlink d3/da4/l10f 0 2024-09-16T23:49:00.133 INFO:tasks.workunit.client.1.smithi043.stdout:4/667: chown d8/d15/d9b/fa7 28 1 2024-09-16T23:49:00.134 INFO:tasks.workunit.client.1.smithi043.stdout:1/768: symlink d4/d9/d1c/l10c 0 2024-09-16T23:49:00.134 INFO:tasks.workunit.client.1.smithi043.stdout:5/796: chown d3/d9/d22/d4a/d4c/c90 774 1 2024-09-16T23:49:00.136 INFO:tasks.workunit.client.1.smithi043.stdout:7/585: readlink d4/d27/d41/d1b/d75/l8e 0 2024-09-16T23:49:00.138 INFO:tasks.workunit.client.1.smithi043.stdout:1/769: dread d4/d34/f6e [0,4194304] 0 2024-09-16T23:49:00.139 INFO:tasks.workunit.client.1.smithi043.stdout:9/557: rmdir d4/d21/d33/da2 39 2024-09-16T23:49:00.140 INFO:tasks.workunit.client.1.smithi043.stdout:0/635: creat d0/d2/df/d1a/fd6 x:0 0 0 2024-09-16T23:49:00.144 INFO:tasks.workunit.client.1.smithi043.stdout:5/797: rmdir d3/db 39 2024-09-16T23:49:00.147 INFO:tasks.workunit.client.1.smithi043.stdout:8/598: fdatasync de/d70/d3d/fa3 0 2024-09-16T23:49:00.148 INFO:tasks.workunit.client.1.smithi043.stdout:8/599: stat de/d18/d27/d41/d5a/l19 0 2024-09-16T23:49:00.153 INFO:tasks.workunit.client.0.smithi042.stdout:8/367: dwrite da/f24 [0,4194304] 0 2024-09-16T23:49:00.159 INFO:tasks.workunit.client.1.smithi043.stdout:6/632: unlink de/d23/d34/d9b/f73 0 2024-09-16T23:49:00.161 INFO:tasks.workunit.client.1.smithi043.stdout:0/636: rmdir d0 39 2024-09-16T23:49:00.165 INFO:tasks.workunit.client.1.smithi043.stdout:4/668: link d8/d15/d20/d2b/d54/d55/f95 d8/d15/d20/d2b/d54/d55/d73/fcb 0 2024-09-16T23:49:00.166 INFO:tasks.workunit.client.1.smithi043.stdout:5/798: write d3/db/d15/d4d/f6c [1055801,33700] 0 2024-09-16T23:49:00.167 INFO:tasks.workunit.client.1.smithi043.stdout:2/573: dwrite d8/d1d/d25/f2b [4194304,4194304] 0 2024-09-16T23:49:00.167 INFO:tasks.workunit.client.1.smithi043.stdout:9/558: mknod d4/d21/d33/cab 0 2024-09-16T23:49:00.170 INFO:tasks.workunit.client.1.smithi043.stdout:1/770: mkdir d4/d9/d8f/db6/dcc/dda/d106/d10d 0 2024-09-16T23:49:00.173 INFO:tasks.workunit.client.0.smithi042.stdout:8/368: dwrite da/d19/f3b [0,4194304] 0 2024-09-16T23:49:00.173 INFO:tasks.workunit.client.1.smithi043.stdout:3/630: dwrite d4/f23 [0,4194304] 0 2024-09-16T23:49:00.175 INFO:tasks.workunit.client.1.smithi043.stdout:6/633: rename de/d23/d63/f6f to de/d23/dc3/fd1 0 2024-09-16T23:49:00.176 INFO:tasks.workunit.client.1.smithi043.stdout:8/600: symlink de/d18/d1f/d2f/d39/lb4 0 2024-09-16T23:49:00.177 INFO:tasks.workunit.client.1.smithi043.stdout:5/799: creat d3/d9/d22/d4a/d64/f110 x:0 0 0 2024-09-16T23:49:00.178 INFO:tasks.workunit.client.0.smithi042.stdout:8/369: creat da/f70 x:0 0 0 2024-09-16T23:49:00.182 INFO:tasks.workunit.client.1.smithi043.stdout:3/631: creat d4/de/d40/fbf x:0 0 0 2024-09-16T23:49:00.182 INFO:tasks.workunit.client.1.smithi043.stdout:5/800: fsync d3/d9/d22/d8e/fbb 0 2024-09-16T23:49:00.182 INFO:tasks.workunit.client.0.smithi042.stdout:8/370: readlink da/lb 0 2024-09-16T23:49:00.184 INFO:tasks.workunit.client.1.smithi043.stdout:7/586: dwrite d4/db/d11/d21/f2f [4194304,4194304] 0 2024-09-16T23:49:00.187 INFO:tasks.workunit.client.1.smithi043.stdout:9/559: rename d4/l29 to d4/d1c/d4f/d45/d85/lac 0 2024-09-16T23:49:00.187 INFO:tasks.workunit.client.1.smithi043.stdout:5/801: read d3/d9/d22/d4a/fc0 [1390523,69675] 0 2024-09-16T23:49:00.187 INFO:tasks.workunit.client.1.smithi043.stdout:9/560: readlink d4/d1c/l68 0 2024-09-16T23:49:00.189 INFO:tasks.workunit.client.1.smithi043.stdout:2/574: creat d8/d4b/d5f/d63/db6/fc1 x:0 0 0 2024-09-16T23:49:00.191 INFO:tasks.workunit.client.1.smithi043.stdout:5/802: dread - d3/d9/d22/d4a/dac/fef zero size 2024-09-16T23:49:00.191 INFO:tasks.workunit.client.1.smithi043.stdout:2/575: fsync d8/d1d/d20/f58 0 2024-09-16T23:49:00.192 INFO:tasks.workunit.client.1.smithi043.stdout:0/637: fsync d0/d2/df/dbb/f9b 0 2024-09-16T23:49:00.194 INFO:tasks.workunit.client.0.smithi042.stdout:8/371: creat da/f71 x:0 0 0 2024-09-16T23:49:00.195 INFO:tasks.workunit.client.1.smithi043.stdout:5/803: read d3/d9/d22/f4e [1787576,65602] 0 2024-09-16T23:49:00.196 INFO:tasks.workunit.client.1.smithi043.stdout:6/634: creat de/d23/d63/fd2 x:0 0 0 2024-09-16T23:49:00.198 INFO:tasks.workunit.client.1.smithi043.stdout:6/635: stat de/d23/d63/fd2 0 2024-09-16T23:49:00.199 INFO:tasks.workunit.client.1.smithi043.stdout:2/576: dread d8/d1f/f2e [0,4194304] 0 2024-09-16T23:49:00.199 INFO:tasks.workunit.client.1.smithi043.stdout:4/669: dread d8/d15/d20/d2b/d54/d55/f82 [0,4194304] 0 2024-09-16T23:49:00.200 INFO:tasks.workunit.client.1.smithi043.stdout:3/632: mkdir d4/de/d40/dc0 0 2024-09-16T23:49:00.200 INFO:tasks.workunit.client.1.smithi043.stdout:4/670: rename d8 to d8/d15/dcc 22 2024-09-16T23:49:00.202 INFO:tasks.workunit.client.0.smithi042.stdout:8/372: dread da/d19/d1d/d30/f41 [0,4194304] 0 2024-09-16T23:49:00.203 INFO:tasks.workunit.client.1.smithi043.stdout:7/587: truncate d4/d27/f6c 2561325 0 2024-09-16T23:49:00.207 INFO:tasks.workunit.client.1.smithi043.stdout:9/561: dread d4/f11 [0,4194304] 0 2024-09-16T23:49:00.208 INFO:tasks.workunit.client.1.smithi043.stdout:1/771: creat d4/d9/d12/f10e x:0 0 0 2024-09-16T23:49:00.208 INFO:tasks.workunit.client.1.smithi043.stdout:9/562: dread - d4/d21/d2a/d2c/d7b/faa zero size 2024-09-16T23:49:00.209 INFO:tasks.workunit.client.0.smithi042.stdout:8/373: truncate da/d19/d1d/f50 508970 0 2024-09-16T23:49:00.209 INFO:tasks.workunit.client.1.smithi043.stdout:1/772: stat d4/d9/d12/l78 0 2024-09-16T23:49:00.209 INFO:tasks.workunit.client.0.smithi042.stdout:8/374: stat da/d19/l21 0 2024-09-16T23:49:00.215 INFO:tasks.workunit.client.0.smithi042.stdout:8/375: rename da/d19/d1d/f33 to da/d4b/f72 0 2024-09-16T23:49:00.215 INFO:tasks.workunit.client.0.smithi042.stdout:8/376: write da/d19/d2c/f66 [126815,64621] 0 2024-09-16T23:49:00.217 INFO:tasks.workunit.client.1.smithi043.stdout:5/804: creat d3/db/d19/d3e/d28/dcb/f111 x:0 0 0 2024-09-16T23:49:00.217 INFO:tasks.workunit.client.1.smithi043.stdout:5/805: stat d3/d9/d22/d4a/d64/d8f/d109 0 2024-09-16T23:49:00.220 INFO:tasks.workunit.client.1.smithi043.stdout:3/633: rmdir d4/de/d40 39 2024-09-16T23:49:00.222 INFO:tasks.workunit.client.0.smithi042.stdout:8/377: mkdir da/d73 0 2024-09-16T23:49:00.226 INFO:tasks.workunit.client.0.smithi042.stdout:8/378: mkdir da/d4b/d74 0 2024-09-16T23:49:00.227 INFO:tasks.workunit.client.0.smithi042.stdout:8/379: write da/f70 [793573,56258] 0 2024-09-16T23:49:00.229 INFO:tasks.workunit.client.1.smithi043.stdout:2/577: truncate d8/d1d/d20/d26/f3f 1732404 0 2024-09-16T23:49:00.230 INFO:tasks.workunit.client.1.smithi043.stdout:8/601: truncate de/d18/d27/d41/d5a/f13 951793 0 2024-09-16T23:49:00.231 INFO:tasks.workunit.client.0.smithi042.stdout:8/380: creat da/d19/d1d/d30/d3f/f75 x:0 0 0 2024-09-16T23:49:00.233 INFO:tasks.workunit.client.1.smithi043.stdout:1/773: symlink d4/d34/l10f 0 2024-09-16T23:49:00.234 INFO:tasks.workunit.client.1.smithi043.stdout:1/774: fsync d4/d9/f84 0 2024-09-16T23:49:00.237 INFO:tasks.workunit.client.1.smithi043.stdout:9/563: dwrite d4/d28/d49/f76 [4194304,4194304] 0 2024-09-16T23:49:00.237 INFO:tasks.workunit.client.1.smithi043.stdout:4/671: mkdir d8/d25/dcd 0 2024-09-16T23:49:00.238 INFO:tasks.workunit.client.1.smithi043.stdout:9/564: dread - d4/d5/f92 zero size 2024-09-16T23:49:00.246 INFO:tasks.workunit.client.0.smithi042.stdout:8/381: truncate da/d19/d1d/f2b 941069 0 2024-09-16T23:49:00.246 INFO:tasks.workunit.client.1.smithi043.stdout:1/775: chown d4/df/df4/l6a 818795450 1 2024-09-16T23:49:00.247 INFO:tasks.workunit.client.1.smithi043.stdout:5/806: getdents d3/db/d84/db6/d10c 0 2024-09-16T23:49:00.247 INFO:tasks.workunit.client.1.smithi043.stdout:6/636: link de/d1a/d80/l81 de/d23/d34/d66/d9d/ld3 0 2024-09-16T23:49:00.248 INFO:tasks.workunit.client.1.smithi043.stdout:0/638: truncate d0/d2/df/d1a/d46/d39/f61 62818 0 2024-09-16T23:49:00.249 INFO:tasks.workunit.client.1.smithi043.stdout:9/565: dwrite d4/d1c/d25/f62 [0,4194304] 0 2024-09-16T23:49:00.252 INFO:tasks.workunit.client.1.smithi043.stdout:7/588: creat d4/db/d43/d83/d85/fc2 x:0 0 0 2024-09-16T23:49:00.252 INFO:tasks.workunit.client.0.smithi042.stdout:8/382: creat da/d19/d1d/d30/d3f/d6d/f76 x:0 0 0 2024-09-16T23:49:00.256 INFO:tasks.workunit.client.1.smithi043.stdout:8/602: dwrite de/f17 [4194304,4194304] 0 2024-09-16T23:49:00.256 INFO:tasks.workunit.client.1.smithi043.stdout:3/634: link d4/de/d5d/f6c d4/de/d40/d86/fc1 0 2024-09-16T23:49:00.258 INFO:tasks.workunit.client.0.smithi042.stdout:8/383: creat da/d2f/f77 x:0 0 0 2024-09-16T23:49:00.262 INFO:tasks.workunit.client.1.smithi043.stdout:8/603: write de/d18/d1f/d25/f6e [817002,101891] 0 2024-09-16T23:49:00.262 INFO:tasks.workunit.client.1.smithi043.stdout:9/566: dwrite d4/d1c/d25/d2b/d4c/f93 [0,4194304] 0 2024-09-16T23:49:00.264 INFO:tasks.workunit.client.1.smithi043.stdout:8/604: fdatasync de/d70/d3d/f88 0 2024-09-16T23:49:00.268 INFO:tasks.workunit.client.1.smithi043.stdout:5/807: creat d3/db/d26/f112 x:0 0 0 2024-09-16T23:49:00.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:59 smithi042 ceph-mon[27914]: Standby manager daemon smithi042.zrkumz started 2024-09-16T23:49:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:59 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/crt"}]: dispatch 2024-09-16T23:49:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:59 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:49:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:59 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/key"}]: dispatch 2024-09-16T23:49:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:48:59 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/2' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:49:00.273 INFO:tasks.workunit.client.1.smithi043.stdout:0/639: chown d0/d2/df/d5a/d9c/fb6 4 1 2024-09-16T23:49:00.273 INFO:tasks.workunit.client.1.smithi043.stdout:5/808: fdatasync d3/db/d84/db6/ffe 0 2024-09-16T23:49:00.275 INFO:tasks.workunit.client.0.smithi042.stdout:8/384: link da/d2f/f38 da/d19/d1d/f78 0 2024-09-16T23:49:00.276 INFO:tasks.workunit.client.1.smithi043.stdout:9/567: dwrite d4/d21/f89 [0,4194304] 0 2024-09-16T23:49:00.276 INFO:tasks.workunit.client.0.smithi042.stdout:8/385: chown da/d2f/f77 825150445 1 2024-09-16T23:49:00.277 INFO:tasks.workunit.client.1.smithi043.stdout:2/578: dwrite d8/d1d/d25/fa4 [0,4194304] 0 2024-09-16T23:49:00.277 INFO:tasks.workunit.client.0.smithi042.stdout:8/386: readlink da/d19/l21 0 2024-09-16T23:49:00.278 INFO:tasks.workunit.client.1.smithi043.stdout:1/776: dread d4/d9/d12/f82 [0,4194304] 0 2024-09-16T23:49:00.280 INFO:tasks.workunit.client.1.smithi043.stdout:3/635: unlink d4/de/d40/d4d/l99 0 2024-09-16T23:49:00.280 INFO:tasks.workunit.client.1.smithi043.stdout:6/637: dread de/d23/d49/fb5 [0,4194304] 0 2024-09-16T23:49:00.281 INFO:tasks.workunit.client.1.smithi043.stdout:4/672: rename d8/d15/d20/d2b/d22/db2 to d8/d15/d20/d2b/d54/dce 0 2024-09-16T23:49:00.283 INFO:tasks.workunit.client.1.smithi043.stdout:3/636: truncate d4/de/d40/fb5 210034 0 2024-09-16T23:49:00.289 INFO:tasks.workunit.client.1.smithi043.stdout:1/777: dwrite d4/df/d26/ff9 [0,4194304] 0 2024-09-16T23:49:00.290 INFO:tasks.workunit.client.1.smithi043.stdout:5/809: mkdir d3/d9/d22/d4a/d4c/d113 0 2024-09-16T23:49:00.292 INFO:tasks.workunit.client.1.smithi043.stdout:5/810: fdatasync d3/f96 0 2024-09-16T23:49:00.292 INFO:tasks.workunit.client.1.smithi043.stdout:2/579: creat d8/d5e/fc2 x:0 0 0 2024-09-16T23:49:00.296 INFO:tasks.workunit.client.1.smithi043.stdout:3/637: dwrite d4/de/d40/d4d/fbd [0,4194304] 0 2024-09-16T23:49:00.296 INFO:tasks.workunit.client.1.smithi043.stdout:7/589: rename d4/db/d43/d83/d85/f96 to d4/d27/d41/d1b/d95/fc3 0 2024-09-16T23:49:00.298 INFO:tasks.workunit.client.1.smithi043.stdout:4/673: creat d8/d15/d6e/fcf x:0 0 0 2024-09-16T23:49:00.301 INFO:tasks.workunit.client.1.smithi043.stdout:0/640: symlink d0/d2/df/d1a/d49/db2/ld7 0 2024-09-16T23:49:00.303 INFO:tasks.workunit.client.1.smithi043.stdout:0/641: write d0/de/f91 [2044270,16959] 0 2024-09-16T23:49:00.304 INFO:tasks.workunit.client.1.smithi043.stdout:9/568: fsync d4/d21/d35/f5d 0 2024-09-16T23:49:00.306 INFO:tasks.workunit.client.1.smithi043.stdout:9/569: write d4/d5/d16/f1f [3231567,106962] 0 2024-09-16T23:49:00.306 INFO:tasks.workunit.client.0.smithi042.stdout:8/387: write da/d19/f23 [391661,111217] 0 2024-09-16T23:49:00.312 INFO:tasks.workunit.client.1.smithi043.stdout:2/580: write d8/d1d/d20/d26/f60 [1214700,65477] 0 2024-09-16T23:49:00.314 INFO:tasks.workunit.client.1.smithi043.stdout:5/811: creat d3/d9/d22/d4a/d4c/d7a/f114 x:0 0 0 2024-09-16T23:49:00.316 INFO:tasks.workunit.client.1.smithi043.stdout:9/570: dwrite d4/d1c/d25/d2b/d4c/f72 [4194304,4194304] 0 2024-09-16T23:49:00.316 INFO:tasks.workunit.client.0.smithi042.stdout:8/388: rename da/f16 to da/d19/d1d/d5a/f79 0 2024-09-16T23:49:00.318 INFO:tasks.workunit.client.1.smithi043.stdout:6/638: write de/d1a/f32 [1067395,122216] 0 2024-09-16T23:49:00.320 INFO:tasks.workunit.client.1.smithi043.stdout:3/638: creat d4/d30/d53/d88/db1/fc2 x:0 0 0 2024-09-16T23:49:00.320 INFO:tasks.workunit.client.1.smithi043.stdout:7/590: chown d4/f2b 338 1 2024-09-16T23:49:00.322 INFO:tasks.workunit.client.1.smithi043.stdout:6/639: write de/d23/d34/d9b/fa9 [3110462,110236] 0 2024-09-16T23:49:00.327 INFO:tasks.workunit.client.0.smithi042.stdout:8/389: creat da/d19/f7a x:0 0 0 2024-09-16T23:49:00.328 INFO:tasks.workunit.client.1.smithi043.stdout:1/778: dwrite d4/d9/d12/f1f [0,4194304] 0 2024-09-16T23:49:00.330 INFO:tasks.workunit.client.1.smithi043.stdout:4/674: write d8/f3b [1835604,86977] 0 2024-09-16T23:49:00.337 INFO:tasks.workunit.client.1.smithi043.stdout:9/571: creat d4/d28/d5f/d91/fad x:0 0 0 2024-09-16T23:49:00.341 INFO:tasks.workunit.client.1.smithi043.stdout:3/639: stat d4/l50 0 2024-09-16T23:49:00.341 INFO:tasks.workunit.client.1.smithi043.stdout:2/581: write d8/d1d/d20/f80 [869137,93330] 0 2024-09-16T23:49:00.342 INFO:tasks.workunit.client.0.smithi042.stdout:8/390: rename da/d19/d1d/d30/d3f/f53 to da/d19/d1d/d30/d49/f7b 0 2024-09-16T23:49:00.342 INFO:tasks.workunit.client.1.smithi043.stdout:7/591: creat d4/db/d11/d21/d31/fc4 x:0 0 0 2024-09-16T23:49:00.343 INFO:tasks.workunit.client.0.smithi042.stdout:6/381: sync 2024-09-16T23:49:00.345 INFO:tasks.workunit.client.0.smithi042.stdout:6/382: write d8/db/dd/d2c/f10 [171983,63964] 0 2024-09-16T23:49:00.345 INFO:tasks.workunit.client.1.smithi043.stdout:0/642: creat d0/d2/df/da7/fd8 x:0 0 0 2024-09-16T23:49:00.346 INFO:tasks.workunit.client.0.smithi042.stdout:6/383: chown d8/db/dd/d2c/d57/f77 239 1 2024-09-16T23:49:00.348 INFO:tasks.workunit.client.1.smithi043.stdout:4/675: rmdir d8/d15/d20/d2b/d54/d55/d5e 39 2024-09-16T23:49:00.348 INFO:tasks.workunit.client.1.smithi043.stdout:7/592: dwrite d4/db/d50/fa9 [0,4194304] 0 2024-09-16T23:49:00.350 INFO:tasks.workunit.client.1.smithi043.stdout:4/676: stat d8/d25/d39/d41/d57/fa1 0 2024-09-16T23:49:00.350 INFO:tasks.workunit.client.1.smithi043.stdout:0/643: truncate d0/d2/df/d1a/d46/d92/fac 751528 0 2024-09-16T23:49:00.350 INFO:tasks.workunit.client.1.smithi043.stdout:5/812: creat d3/d9/d22/d4a/d4c/d113/f115 x:0 0 0 2024-09-16T23:49:00.353 INFO:tasks.workunit.client.0.smithi042.stdout:8/391: dread da/f14 [0,4194304] 0 2024-09-16T23:49:00.353 INFO:tasks.workunit.client.1.smithi043.stdout:9/572: mknod d4/d21/d2a/cae 0 2024-09-16T23:49:00.358 INFO:tasks.workunit.client.0.smithi042.stdout:8/392: unlink da/f70 0 2024-09-16T23:49:00.359 INFO:tasks.workunit.client.0.smithi042.stdout:8/393: stat da/c60 0 2024-09-16T23:49:00.361 INFO:tasks.workunit.client.0.smithi042.stdout:6/384: symlink d8/db/dd/d1b/d34/d39/d5e/d5a/l7e 0 2024-09-16T23:49:00.362 INFO:tasks.workunit.client.1.smithi043.stdout:6/640: fsync de/d1a/f42 0 2024-09-16T23:49:00.362 INFO:tasks.workunit.client.1.smithi043.stdout:9/573: dread d4/d28/d5f/d86/f96 [0,4194304] 0 2024-09-16T23:49:00.363 INFO:tasks.workunit.client.1.smithi043.stdout:6/641: stat de/d23/d49/f4c 0 2024-09-16T23:49:00.365 INFO:tasks.workunit.client.1.smithi043.stdout:7/593: rmdir d4/db/d11/d15 39 2024-09-16T23:49:00.366 INFO:tasks.workunit.client.0.smithi042.stdout:8/394: dwrite da/f71 [0,4194304] 0 2024-09-16T23:49:00.367 INFO:tasks.workunit.client.0.smithi042.stdout:6/385: rename f3 to d8/db/dd/d1b/d34/d39/d5e/d4c/d4e/f7f 0 2024-09-16T23:49:00.370 INFO:tasks.workunit.client.1.smithi043.stdout:0/644: creat d0/d2/df/da7/db8/fd9 x:0 0 0 2024-09-16T23:49:00.374 INFO:tasks.workunit.client.0.smithi042.stdout:6/386: creat d8/db/dd/d1b/d34/d39/d5e/d4c/f80 x:0 0 0 2024-09-16T23:49:00.380 INFO:tasks.workunit.client.1.smithi043.stdout:1/779: dwrite d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f30 [0,4194304] 0 2024-09-16T23:49:00.382 INFO:tasks.workunit.client.0.smithi042.stdout:8/395: rename da/d19/c25 to da/c7c 0 2024-09-16T23:49:00.383 INFO:tasks.workunit.client.0.smithi042.stdout:8/396: readlink da/d4b/l57 0 2024-09-16T23:49:00.383 INFO:tasks.workunit.client.1.smithi043.stdout:7/594: mknod d4/db/d43/d83/d85/d8b/cc5 0 2024-09-16T23:49:00.384 INFO:tasks.workunit.client.1.smithi043.stdout:4/677: mknod d8/cd0 0 2024-09-16T23:49:00.385 INFO:tasks.workunit.client.1.smithi043.stdout:7/595: readlink d4/db/d11/l54 0 2024-09-16T23:49:00.386 INFO:tasks.workunit.client.1.smithi043.stdout:5/813: dread d3/db/d19/d3e/d28/f37 [0,4194304] 0 2024-09-16T23:49:00.389 INFO:tasks.workunit.client.0.smithi042.stdout:8/397: dwrite da/d19/d1d/f3e [0,4194304] 0 2024-09-16T23:49:00.391 INFO:tasks.workunit.client.0.smithi042.stdout:6/387: rename d8/db to d8/d70/d81 0 2024-09-16T23:49:00.393 INFO:tasks.workunit.client.0.smithi042.stdout:6/388: dread d8/d70/d81/dd/d1b/d34/f76 [0,4194304] 0 2024-09-16T23:49:00.394 INFO:tasks.workunit.client.0.smithi042.stdout:6/389: dread - d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f80 zero size 2024-09-16T23:49:00.396 INFO:tasks.workunit.client.0.smithi042.stdout:6/390: read d8/d70/d81/dd/d2c/f10 [2979813,61908] 0 2024-09-16T23:49:00.397 INFO:tasks.workunit.client.1.smithi043.stdout:3/640: rmdir d4/de/d40/dc0 0 2024-09-16T23:49:00.398 INFO:tasks.workunit.client.1.smithi043.stdout:3/641: fdatasync d4/de/d40/d86/fba 0 2024-09-16T23:49:00.399 INFO:tasks.workunit.client.0.smithi042.stdout:6/391: fdatasync d8/d70/d81/dd/d1b/d2d/f41 0 2024-09-16T23:49:00.400 INFO:tasks.workunit.client.1.smithi043.stdout:0/645: dwrite d0/d2/f6d [0,4194304] 0 2024-09-16T23:49:00.502 INFO:tasks.workunit.client.1.smithi043.stdout:8/605: sync 2024-09-16T23:49:00.529 INFO:tasks.workunit.client.1.smithi043.stdout:4/678: rmdir d8/d15/d6e 39 2024-09-16T23:49:00.530 INFO:tasks.workunit.client.0.smithi042.stdout:8/398: symlink da/d19/d1d/d30/d49/d58/d6a/l7d 0 2024-09-16T23:49:00.531 INFO:tasks.workunit.client.0.smithi042.stdout:8/399: readlink da/d19/l21 0 2024-09-16T23:49:00.531 INFO:tasks.workunit.client.1.smithi043.stdout:6/642: truncate de/f40 1834402 0 2024-09-16T23:49:00.531 INFO:tasks.workunit.client.0.smithi042.stdout:8/400: chown da/d19/d2c/c54 55102834 1 2024-09-16T23:49:00.532 INFO:tasks.workunit.client.0.smithi042.stdout:8/401: fdatasync da/d19/f1f 0 2024-09-16T23:49:00.532 INFO:tasks.workunit.client.0.smithi042.stdout:6/392: write d8/d70/d81/dd/d2c/d57/f77 [533802,93685] 0 2024-09-16T23:49:00.533 INFO:tasks.workunit.client.1.smithi043.stdout:5/814: creat d3/d9/d6f/d85/f116 x:0 0 0 2024-09-16T23:49:00.534 INFO:tasks.workunit.client.0.smithi042.stdout:6/393: readlink d8/d70/d81/dd/d1b/d2d/l75 0 2024-09-16T23:49:00.535 INFO:tasks.workunit.client.1.smithi043.stdout:2/582: getdents d8/d1d/d20/d23/d38/da2 0 2024-09-16T23:49:00.536 INFO:tasks.workunit.client.0.smithi042.stdout:8/402: creat da/d19/d1d/d30/d3f/d6d/f7e x:0 0 0 2024-09-16T23:49:00.537 INFO:tasks.workunit.client.1.smithi043.stdout:3/642: unlink d4/de/d1f/d2c/l8b 0 2024-09-16T23:49:00.540 INFO:tasks.workunit.client.1.smithi043.stdout:5/815: dwrite d3/d9/d22/d4a/d4c/ded/ff6 [0,4194304] 0 2024-09-16T23:49:00.540 INFO:tasks.workunit.client.0.smithi042.stdout:8/403: rmdir da/d19/d1d/d30/d49 39 2024-09-16T23:49:00.545 INFO:tasks.workunit.client.0.smithi042.stdout:6/394: creat d8/d70/d81/f82 x:0 0 0 2024-09-16T23:49:00.548 INFO:tasks.workunit.client.1.smithi043.stdout:7/596: getdents d4/dc0 0 2024-09-16T23:49:00.548 INFO:tasks.workunit.client.0.smithi042.stdout:8/404: dwrite da/f47 [0,4194304] 0 2024-09-16T23:49:00.550 INFO:tasks.workunit.client.0.smithi042.stdout:8/405: chown da/d19/d1d/f6b 130759 1 2024-09-16T23:49:00.554 INFO:tasks.workunit.client.1.smithi043.stdout:6/643: creat de/d23/dc3/fd4 x:0 0 0 2024-09-16T23:49:00.556 INFO:tasks.workunit.client.1.smithi043.stdout:2/583: mknod d8/d1d/d20/d23/d38/da2/cc3 0 2024-09-16T23:49:00.558 INFO:tasks.workunit.client.0.smithi042.stdout:6/395: creat d8/d70/d81/dd/d1b/d34/d39/d5e/f83 x:0 0 0 2024-09-16T23:49:00.562 INFO:tasks.workunit.client.1.smithi043.stdout:0/646: link d0/d2/df/d1a/fd6 d0/d2/df/d1a/d3b/fda 0 2024-09-16T23:49:00.563 INFO:tasks.workunit.client.0.smithi042.stdout:6/396: write d8/d70/d81/dd/f11 [5113146,68954] 0 2024-09-16T23:49:00.565 INFO:tasks.workunit.client.1.smithi043.stdout:8/606: creat de/d18/d1f/d2f/d39/d7b/fb5 x:0 0 0 2024-09-16T23:49:00.568 INFO:tasks.workunit.client.0.smithi042.stdout:6/397: dread d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 [0,4194304] 0 2024-09-16T23:49:00.569 INFO:tasks.workunit.client.1.smithi043.stdout:1/780: rename d4/d9/d72 to d4/d9/d1c/d1e/d42/d5d/d110 0 2024-09-16T23:49:00.569 INFO:tasks.workunit.client.1.smithi043.stdout:1/781: stat d4/d9/d1c 0 2024-09-16T23:49:00.569 INFO:tasks.workunit.client.1.smithi043.stdout:5/816: creat d3/d9/d22/d4a/d4c/d7a/f117 x:0 0 0 2024-09-16T23:49:00.570 INFO:tasks.workunit.client.0.smithi042.stdout:3/540: sync 2024-09-16T23:49:00.571 INFO:tasks.workunit.client.1.smithi043.stdout:9/574: link d4/d21/d2a/l82 d4/d21/d33/laf 0 2024-09-16T23:49:00.572 INFO:tasks.workunit.client.1.smithi043.stdout:9/575: write d4/d1c/d4f/d8c/f66 [327963,5348] 0 2024-09-16T23:49:00.573 INFO:tasks.workunit.client.0.smithi042.stdout:6/398: mkdir d8/d70/d84 0 2024-09-16T23:49:00.574 INFO:tasks.workunit.client.0.smithi042.stdout:8/406: write da/d2f/f38 [4997684,3247] 0 2024-09-16T23:49:00.576 INFO:tasks.workunit.client.0.smithi042.stdout:6/399: read - d8/d70/d81/dd/d1b/d2d/f41 zero size 2024-09-16T23:49:00.577 INFO:tasks.workunit.client.1.smithi043.stdout:6/644: creat de/dbf/fd5 x:0 0 0 2024-09-16T23:49:00.578 INFO:tasks.workunit.client.1.smithi043.stdout:6/645: chown de/d23/d49/f7c 87 1 2024-09-16T23:49:00.578 INFO:tasks.workunit.client.0.smithi042.stdout:3/541: dwrite d2/d1b/d1d/d34/d62/d6a/fa5 [0,4194304] 0 2024-09-16T23:49:00.579 INFO:tasks.workunit.client.0.smithi042.stdout:3/542: stat d2/d1b/d1d/d34/d50/f6c 0 2024-09-16T23:49:00.580 INFO:tasks.workunit.client.0.smithi042.stdout:8/407: dread da/d19/f39 [0,4194304] 0 2024-09-16T23:49:00.580 INFO:tasks.workunit.client.0.smithi042.stdout:3/543: fsync d2/d7/f36 0 2024-09-16T23:49:00.581 INFO:tasks.workunit.client.0.smithi042.stdout:3/544: readlink d2/d1b/d1d/l5e 0 2024-09-16T23:49:00.582 INFO:tasks.workunit.client.0.smithi042.stdout:6/400: mknod d8/d70/c85 0 2024-09-16T23:49:00.584 INFO:tasks.workunit.client.1.smithi043.stdout:0/647: mknod d0/d2/df/d1a/d46/d39/d6e/cdb 0 2024-09-16T23:49:00.584 INFO:tasks.workunit.client.0.smithi042.stdout:3/545: creat d2/d1b/d1d/d34/fbd x:0 0 0 2024-09-16T23:49:00.585 INFO:tasks.workunit.client.1.smithi043.stdout:4/679: truncate d8/d15/d20/f8d 2576961 0 2024-09-16T23:49:00.585 INFO:tasks.workunit.client.1.smithi043.stdout:8/607: rmdir de 39 2024-09-16T23:49:00.586 INFO:tasks.workunit.client.0.smithi042.stdout:3/546: chown f0 269 1 2024-09-16T23:49:00.586 INFO:tasks.workunit.client.1.smithi043.stdout:0/648: read - d0/d2/df/d1a/d46/d39/daa/db7/fcc zero size 2024-09-16T23:49:00.587 INFO:tasks.workunit.client.1.smithi043.stdout:0/649: readlink d0/la2 0 2024-09-16T23:49:00.588 INFO:tasks.workunit.client.0.smithi042.stdout:8/408: dread da/d19/d1d/f50 [0,4194304] 0 2024-09-16T23:49:00.588 INFO:tasks.workunit.client.0.smithi042.stdout:3/547: mknod d2/d95/cbe 0 2024-09-16T23:49:00.588 INFO:tasks.workunit.client.1.smithi043.stdout:3/643: rename d4/de/d40/d86 to d4/de/d1f/d2c/db7/dc3 0 2024-09-16T23:49:00.588 INFO:tasks.workunit.client.1.smithi043.stdout:2/584: dread d8/d1d/d20/d26/f3f [0,4194304] 0 2024-09-16T23:49:00.589 INFO:tasks.workunit.client.0.smithi042.stdout:6/401: mknod d8/d70/d81/dd/d1b/d50/c86 0 2024-09-16T23:49:00.590 INFO:tasks.workunit.client.0.smithi042.stdout:3/548: symlink d2/d7/d61/lbf 0 2024-09-16T23:49:00.590 INFO:tasks.workunit.client.0.smithi042.stdout:6/402: dread - d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/f6b zero size 2024-09-16T23:49:00.591 INFO:tasks.workunit.client.0.smithi042.stdout:8/409: readlink da/d19/d1d/d30/d49/d58/d6a/l7d 0 2024-09-16T23:49:00.592 INFO:tasks.workunit.client.1.smithi043.stdout:6/646: mknod de/d23/d34/d66/d9d/dae/cd6 0 2024-09-16T23:49:00.593 INFO:tasks.workunit.client.0.smithi042.stdout:3/549: creat d2/d1b/d1d/d30/fc0 x:0 0 0 2024-09-16T23:49:00.594 INFO:tasks.workunit.client.1.smithi043.stdout:6/647: write de/d1a/fad [476233,114347] 0 2024-09-16T23:49:00.594 INFO:tasks.workunit.client.1.smithi043.stdout:4/680: dread - d8/d15/d6e/fcf zero size 2024-09-16T23:49:00.595 INFO:tasks.workunit.client.1.smithi043.stdout:4/681: readlink d8/d15/d20/d2b/d23/d27/l9f 0 2024-09-16T23:49:00.598 INFO:tasks.workunit.client.1.smithi043.stdout:5/817: rename d3/d9/d6f/f10d to d3/db/d19/d3e/d28/dcb/f118 0 2024-09-16T23:49:00.598 INFO:tasks.workunit.client.0.smithi042.stdout:3/550: rename d2/d1b/d1d/d34/d62/f6d to d2/d7/d61/d8e/d9d/fc1 0 2024-09-16T23:49:00.598 INFO:tasks.workunit.client.1.smithi043.stdout:3/644: creat d4/de/d1f/d2c/fc4 x:0 0 0 2024-09-16T23:49:00.600 INFO:tasks.workunit.client.0.smithi042.stdout:3/551: creat d2/d1b/d1d/db3/fc2 x:0 0 0 2024-09-16T23:49:00.602 INFO:tasks.workunit.client.1.smithi043.stdout:7/597: creat d4/db/d11/fc6 x:0 0 0 2024-09-16T23:49:00.603 INFO:tasks.workunit.client.1.smithi043.stdout:7/598: chown d4/db/d11/fc6 274530 1 2024-09-16T23:49:00.605 INFO:tasks.workunit.client.1.smithi043.stdout:3/645: dwrite d4/fa0 [0,4194304] 0 2024-09-16T23:49:00.605 INFO:tasks.workunit.client.1.smithi043.stdout:0/650: mknod d0/d2/cdc 0 2024-09-16T23:49:00.607 INFO:tasks.workunit.client.1.smithi043.stdout:4/682: creat d8/d15/d20/d2b/d23/fd1 x:0 0 0 2024-09-16T23:49:00.608 INFO:tasks.workunit.client.0.smithi042.stdout:3/552: fsync d2/d1b/d1d/d34/fbd 0 2024-09-16T23:49:00.609 INFO:tasks.workunit.client.0.smithi042.stdout:3/553: readlink d2/d7/d61/d8e/d2c/l72 0 2024-09-16T23:49:00.610 INFO:tasks.workunit.client.1.smithi043.stdout:2/585: symlink d8/d5e/d83/d3a/d77/da9/lc4 0 2024-09-16T23:49:00.611 INFO:tasks.workunit.client.0.smithi042.stdout:6/403: write d8/d70/d81/fc [4092290,3285] 0 2024-09-16T23:49:00.611 INFO:tasks.workunit.client.1.smithi043.stdout:3/646: read d4/de/d1f/d2c/f4a [2101474,85802] 0 2024-09-16T23:49:00.613 INFO:tasks.workunit.client.1.smithi043.stdout:2/586: truncate d8/d5e/fc2 454692 0 2024-09-16T23:49:00.613 INFO:tasks.workunit.client.1.smithi043.stdout:9/576: creat d4/d1c/d25/d2b/d4c/fb0 x:0 0 0 2024-09-16T23:49:00.613 INFO:tasks.workunit.client.1.smithi043.stdout:8/608: mknod de/d70/d3d/daf/cb6 0 2024-09-16T23:49:00.614 INFO:tasks.workunit.client.0.smithi042.stdout:3/554: write d2/d1b/d89/f5c [276260,30867] 0 2024-09-16T23:49:00.615 INFO:tasks.workunit.client.1.smithi043.stdout:6/648: symlink de/d23/d34/d9b/d8b/dbd/ld7 0 2024-09-16T23:49:00.617 INFO:tasks.workunit.client.1.smithi043.stdout:4/683: dwrite d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:49:00.618 INFO:tasks.workunit.client.0.smithi042.stdout:6/404: mkdir d8/d70/d81/dd/d1b/d2d/d87 0 2024-09-16T23:49:00.618 INFO:tasks.workunit.client.1.smithi043.stdout:4/684: write d8/d15/d20/d2b/d54/d55/f2a [4302971,122083] 0 2024-09-16T23:49:00.619 INFO:tasks.workunit.client.1.smithi043.stdout:7/599: read d4/f9 [3063610,71758] 0 2024-09-16T23:49:00.620 INFO:tasks.workunit.client.0.smithi042.stdout:8/410: dwrite da/d19/d1d/f2e [0,4194304] 0 2024-09-16T23:49:00.621 INFO:tasks.workunit.client.0.smithi042.stdout:3/555: symlink d2/d1b/d1d/d34/d62/lc3 0 2024-09-16T23:49:00.621 INFO:tasks.workunit.client.1.smithi043.stdout:7/600: write d4/db/d43/d83/fba [934411,128334] 0 2024-09-16T23:49:00.623 INFO:tasks.workunit.client.0.smithi042.stdout:3/556: chown d2/d1b/d1d/f25 4694550 1 2024-09-16T23:49:00.625 INFO:tasks.workunit.client.0.smithi042.stdout:3/557: dread - d2/d1b/d1d/d34/d62/f78 zero size 2024-09-16T23:49:00.628 INFO:tasks.workunit.client.0.smithi042.stdout:6/405: symlink d8/d70/d81/dd/d2c/l88 0 2024-09-16T23:49:00.629 INFO:tasks.workunit.client.1.smithi043.stdout:3/647: creat d4/de/d35/fc5 x:0 0 0 2024-09-16T23:49:00.630 INFO:tasks.workunit.client.1.smithi043.stdout:8/609: symlink de/d18/d1f/d25/lb7 0 2024-09-16T23:49:00.631 INFO:tasks.workunit.client.0.smithi042.stdout:8/411: dwrite da/d19/f1f [0,4194304] 0 2024-09-16T23:49:00.633 INFO:tasks.workunit.client.1.smithi043.stdout:2/587: rename d8/d1f/c34 to d8/d5e/d83/d3a/d77/cc5 0 2024-09-16T23:49:00.634 INFO:tasks.workunit.client.0.smithi042.stdout:6/406: read - d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f6e zero size 2024-09-16T23:49:00.636 INFO:tasks.workunit.client.0.smithi042.stdout:3/558: dwrite d2/d1b/f43 [0,4194304] 0 2024-09-16T23:49:00.637 INFO:tasks.workunit.client.0.smithi042.stdout:8/412: unlink da/l15 0 2024-09-16T23:49:00.638 INFO:tasks.workunit.client.0.smithi042.stdout:8/413: write da/f47 [2882722,65325] 0 2024-09-16T23:49:00.640 INFO:tasks.workunit.client.0.smithi042.stdout:8/414: write da/d2f/f38 [1697592,78557] 0 2024-09-16T23:49:00.641 INFO:tasks.workunit.client.0.smithi042.stdout:3/559: mkdir d2/d7/d61/d8e/d9d/dc4 0 2024-09-16T23:49:00.642 INFO:tasks.workunit.client.0.smithi042.stdout:6/407: symlink d8/d70/d81/dd/d1b/d34/d39/l89 0 2024-09-16T23:49:00.643 INFO:tasks.workunit.client.0.smithi042.stdout:3/560: fsync d2/d1b/d1d/d34/d50/f6c 0 2024-09-16T23:49:00.644 INFO:tasks.workunit.client.0.smithi042.stdout:2/484: sync 2024-09-16T23:49:00.644 INFO:tasks.workunit.client.0.smithi042.stdout:9/329: sync 2024-09-16T23:49:00.648 INFO:tasks.workunit.client.0.smithi042.stdout:3/561: dread d2/d1b/d89/f5c [0,4194304] 0 2024-09-16T23:49:00.656 INFO:tasks.workunit.client.0.smithi042.stdout:3/562: dread f0 [0,4194304] 0 2024-09-16T23:49:00.661 INFO:tasks.workunit.client.0.smithi042.stdout:9/330: symlink d4/l69 0 2024-09-16T23:49:00.662 INFO:tasks.workunit.client.0.smithi042.stdout:3/563: truncate d2/d7/d61/f90 1704876 0 2024-09-16T23:49:00.663 INFO:tasks.workunit.client.0.smithi042.stdout:3/564: read - d2/d1b/d1d/fa0 zero size 2024-09-16T23:49:00.665 INFO:tasks.workunit.client.0.smithi042.stdout:9/331: dread d4/d10/f4a [0,4194304] 0 2024-09-16T23:49:00.665 INFO:tasks.workunit.client.0.smithi042.stdout:8/415: write da/f17 [713465,43725] 0 2024-09-16T23:49:00.665 INFO:tasks.workunit.client.0.smithi042.stdout:3/565: read - d2/d1b/d1d/f85 zero size 2024-09-16T23:49:00.666 INFO:tasks.workunit.client.0.smithi042.stdout:2/485: link d0/d26/f7c d0/d30/d43/fac 0 2024-09-16T23:49:00.667 INFO:tasks.workunit.client.0.smithi042.stdout:6/408: getdents d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f 0 2024-09-16T23:49:00.667 INFO:tasks.workunit.client.0.smithi042.stdout:2/486: stat d0/da/d5b/d8e 0 2024-09-16T23:49:00.669 INFO:tasks.workunit.client.0.smithi042.stdout:6/409: fdatasync d8/d70/d81/dd/d1b/d34/f55 0 2024-09-16T23:49:00.671 INFO:tasks.workunit.client.0.smithi042.stdout:8/416: link da/d19/f23 da/d19/d1d/d30/d3f/f7f 0 2024-09-16T23:49:00.694 INFO:tasks.workunit.client.1.smithi043.stdout:4/685: creat d8/d15/d9b/fd2 x:0 0 0 2024-09-16T23:49:00.695 INFO:tasks.workunit.client.1.smithi043.stdout:3/648: dread d4/de/d40/f69 [0,4194304] 0 2024-09-16T23:49:00.695 INFO:tasks.workunit.client.1.smithi043.stdout:9/577: mkdir d4/d1c/d25/db1 0 2024-09-16T23:49:00.697 INFO:tasks.workunit.client.1.smithi043.stdout:2/588: mknod d8/d5e/d83/d3a/d77/da9/cc6 0 2024-09-16T23:49:00.698 INFO:tasks.workunit.client.1.smithi043.stdout:6/649: fsync de/d23/d34/d9b/d8b/fbb 0 2024-09-16T23:49:00.702 INFO:tasks.workunit.client.1.smithi043.stdout:0/651: dwrite d0/d2/df/d1a/d46/d39/f45 [0,4194304] 0 2024-09-16T23:49:00.704 INFO:tasks.workunit.client.1.smithi043.stdout:7/601: dwrite d4/d27/d41/d1b/d95/fc3 [0,4194304] 0 2024-09-16T23:49:00.713 INFO:tasks.workunit.client.0.smithi042.stdout:9/332: link d4/d10/f1b d4/d12/d2d/d37/f6a 0 2024-09-16T23:49:00.713 INFO:tasks.workunit.client.0.smithi042.stdout:2/487: write d0/f12 [697858,118242] 0 2024-09-16T23:49:00.716 INFO:tasks.workunit.client.1.smithi043.stdout:8/610: mknod de/d18/d1f/d2f/d39/d7b/d83/da5/cb8 0 2024-09-16T23:49:00.717 INFO:tasks.workunit.client.1.smithi043.stdout:8/611: fdatasync de/d18/d27/d8e/f7e 0 2024-09-16T23:49:00.718 INFO:tasks.workunit.client.1.smithi043.stdout:9/578: fsync d4/d21/d2a/f99 0 2024-09-16T23:49:00.719 INFO:tasks.workunit.client.0.smithi042.stdout:9/333: creat d4/d10/f6b x:0 0 0 2024-09-16T23:49:00.720 INFO:tasks.workunit.client.1.smithi043.stdout:3/649: mkdir d4/de/d1f/d2c/db7/dc6 0 2024-09-16T23:49:00.721 INFO:tasks.workunit.client.0.smithi042.stdout:6/410: write d8/d70/d81/dd/d1b/d34/d39/d5e/f47 [248559,129639] 0 2024-09-16T23:49:00.722 INFO:tasks.workunit.client.1.smithi043.stdout:2/589: symlink d8/d5e/lc7 0 2024-09-16T23:49:00.723 INFO:tasks.workunit.client.0.smithi042.stdout:2/488: truncate d0/d30/f47 778406 0 2024-09-16T23:49:00.723 INFO:tasks.workunit.client.0.smithi042.stdout:9/334: read d4/d10/d24/f35 [2208540,130455] 0 2024-09-16T23:49:00.724 INFO:tasks.workunit.client.0.smithi042.stdout:9/335: dread d4/d10/f4a [0,4194304] 0 2024-09-16T23:49:00.726 INFO:tasks.workunit.client.0.smithi042.stdout:9/336: read - d4/d10/d24/f67 zero size 2024-09-16T23:49:00.726 INFO:tasks.workunit.client.0.smithi042.stdout:6/411: unlink d8/d70/d81/dd/d1b/d34/l59 0 2024-09-16T23:49:00.727 INFO:tasks.workunit.client.0.smithi042.stdout:6/412: stat d8/d70/d81/dd/f19 0 2024-09-16T23:49:00.733 INFO:tasks.workunit.client.1.smithi043.stdout:2/590: symlink d8/d1f/lc8 0 2024-09-16T23:49:00.733 INFO:tasks.workunit.client.0.smithi042.stdout:9/337: mknod d4/d12/d2d/d37/d28/c6c 0 2024-09-16T23:49:00.735 INFO:tasks.workunit.client.1.smithi043.stdout:6/650: rename de/d1a/d39/f1f to de/d23/d63/fd8 0 2024-09-16T23:49:00.736 INFO:tasks.workunit.client.0.smithi042.stdout:8/417: getdents da/d19/d1d 0 2024-09-16T23:49:00.736 INFO:tasks.workunit.client.0.smithi042.stdout:8/418: chown da/d19/d2c/f66 6 1 2024-09-16T23:49:00.737 INFO:tasks.workunit.client.0.smithi042.stdout:8/419: stat da/d19/l21 0 2024-09-16T23:49:00.737 INFO:tasks.workunit.client.0.smithi042.stdout:6/413: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 165319 0 2024-09-16T23:49:00.738 INFO:tasks.workunit.client.1.smithi043.stdout:9/579: creat d4/d84/fb2 x:0 0 0 2024-09-16T23:49:00.739 INFO:tasks.workunit.client.0.smithi042.stdout:2/489: symlink d0/da/d17/d3e/d86/lad 0 2024-09-16T23:49:00.739 INFO:tasks.workunit.client.0.smithi042.stdout:8/420: truncate da/d19/d1d/d30/d49/f69 41095 0 2024-09-16T23:49:00.742 INFO:tasks.workunit.client.1.smithi043.stdout:0/652: link d0/d2/df/d1a/d46/d92/cb4 d0/d2/df/dbb/cdd 0 2024-09-16T23:49:00.743 INFO:tasks.workunit.client.0.smithi042.stdout:2/490: read d0/da/d17/f42 [1611680,111618] 0 2024-09-16T23:49:00.744 INFO:tasks.workunit.client.1.smithi043.stdout:9/580: dwrite d4/f31 [0,4194304] 0 2024-09-16T23:49:00.745 INFO:tasks.workunit.client.0.smithi042.stdout:6/414: rename d8/d70/d81/dd/d1b/d34/d39/c3b to d8/d70/d81/dd/d1b/d34/c8a 0 2024-09-16T23:49:00.745 INFO:tasks.workunit.client.0.smithi042.stdout:2/491: readlink d0/da/d5b/l9e 0 2024-09-16T23:49:00.748 INFO:tasks.workunit.client.0.smithi042.stdout:8/421: dwrite da/f17 [0,4194304] 0 2024-09-16T23:49:00.748 INFO:tasks.workunit.client.0.smithi042.stdout:9/338: dwrite d4/d12/d2d/d37/d28/f65 [8388608,4194304] 0 2024-09-16T23:49:00.749 INFO:tasks.workunit.client.1.smithi043.stdout:6/651: symlink de/d23/d34/d9b/ld9 0 2024-09-16T23:49:00.751 INFO:tasks.workunit.client.1.smithi043.stdout:0/653: chown d0/de/l6f 39 1 2024-09-16T23:49:00.752 INFO:tasks.workunit.client.0.smithi042.stdout:9/339: truncate d4/d12/d2d/d37/d28/f4e 529682 0 2024-09-16T23:49:00.753 INFO:tasks.workunit.client.1.smithi043.stdout:6/652: read de/d23/d34/d9b/d5a/fc6 [358443,46986] 0 2024-09-16T23:49:00.757 INFO:tasks.workunit.client.0.smithi042.stdout:2/492: rename d0/da/d5b to d0/da/d88/dae 0 2024-09-16T23:49:00.759 INFO:tasks.workunit.client.1.smithi043.stdout:8/612: getdents de/d18/d1f/d25/d65/d7f/d95 0 2024-09-16T23:49:00.762 INFO:tasks.workunit.client.1.smithi043.stdout:7/602: rename d4/db/d11/d21/d94/cb9 to d4/cc7 0 2024-09-16T23:49:00.764 INFO:tasks.workunit.client.1.smithi043.stdout:9/581: dread d4/d5/d16/f1f [0,4194304] 0 2024-09-16T23:49:00.765 INFO:tasks.workunit.client.0.smithi042.stdout:1/586: sync 2024-09-16T23:49:00.766 INFO:tasks.workunit.client.0.smithi042.stdout:0/380: sync 2024-09-16T23:49:00.766 INFO:tasks.workunit.client.0.smithi042.stdout:5/636: sync 2024-09-16T23:49:00.766 INFO:tasks.workunit.client.0.smithi042.stdout:7/508: sync 2024-09-16T23:49:00.766 INFO:tasks.workunit.client.0.smithi042.stdout:4/689: sync 2024-09-16T23:49:00.770 INFO:tasks.workunit.client.0.smithi042.stdout:4/690: chown de/d42/d82/d94/fc9 3200124 1 2024-09-16T23:49:00.773 INFO:tasks.workunit.client.1.smithi043.stdout:7/603: dwrite d4/d27/d41/d1b/d95/f9a [0,4194304] 0 2024-09-16T23:49:00.774 INFO:tasks.workunit.client.0.smithi042.stdout:8/422: mkdir da/d19/d80 0 2024-09-16T23:49:00.775 INFO:tasks.workunit.client.0.smithi042.stdout:6/415: dread d8/d70/d81/dd/f19 [0,4194304] 0 2024-09-16T23:49:00.776 INFO:tasks.workunit.client.0.smithi042.stdout:2/493: dread d0/f95 [0,4194304] 0 2024-09-16T23:49:00.776 INFO:tasks.workunit.client.0.smithi042.stdout:7/509: rename d3/cf to d3/d98/da0/d32/cb3 0 2024-09-16T23:49:00.777 INFO:tasks.workunit.client.1.smithi043.stdout:3/650: rename d4/de/d5d/c91 to d4/d30/d53/d88/db1/cc7 0 2024-09-16T23:49:00.778 INFO:tasks.workunit.client.0.smithi042.stdout:4/691: write de/f2f [964002,76014] 0 2024-09-16T23:49:00.778 INFO:tasks.workunit.client.0.smithi042.stdout:7/510: write d3/d98/da0/d41/f62 [358943,117621] 0 2024-09-16T23:49:00.779 INFO:tasks.workunit.client.1.smithi043.stdout:8/613: read de/f54 [3537865,49398] 0 2024-09-16T23:49:00.779 INFO:tasks.workunit.client.1.smithi043.stdout:7/604: chown d4/d27/d8a/faa 11148 1 2024-09-16T23:49:00.780 INFO:tasks.workunit.client.1.smithi043.stdout:9/582: dwrite d4/d21/d2a/d2c/d7b/f7e [0,4194304] 0 2024-09-16T23:49:00.786 INFO:tasks.workunit.client.0.smithi042.stdout:0/381: creat d0/f75 x:0 0 0 2024-09-16T23:49:00.786 INFO:tasks.workunit.client.0.smithi042.stdout:5/637: creat d9/db/fdc x:0 0 0 2024-09-16T23:49:00.787 INFO:tasks.workunit.client.0.smithi042.stdout:6/416: mknod d8/d70/d81/dd/d2c/c8b 0 2024-09-16T23:49:00.791 INFO:tasks.workunit.client.1.smithi043.stdout:3/651: symlink d4/de/d1f/d2c/lc8 0 2024-09-16T23:49:00.791 INFO:tasks.workunit.client.0.smithi042.stdout:4/692: creat de/d42/fee x:0 0 0 2024-09-16T23:49:00.794 INFO:tasks.workunit.client.0.smithi042.stdout:8/423: symlink da/d19/d80/l81 0 2024-09-16T23:49:00.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/417: write d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f80 [517858,48473] 0 2024-09-16T23:49:00.798 INFO:tasks.workunit.client.1.smithi043.stdout:8/614: creat de/d18/d1f/d25/fb9 x:0 0 0 2024-09-16T23:49:00.799 INFO:tasks.workunit.client.0.smithi042.stdout:1/587: link d0/d72/l6e d0/d3b/d8d/ld0 0 2024-09-16T23:49:00.800 INFO:tasks.workunit.client.0.smithi042.stdout:7/511: creat d3/d98/da0/d32/d6b/d64/d92/fb4 x:0 0 0 2024-09-16T23:49:00.801 INFO:tasks.workunit.client.1.smithi043.stdout:7/605: dread d4/db/d11/d15/d29/d7a/fa2 [0,4194304] 0 2024-09-16T23:49:00.801 INFO:tasks.workunit.client.1.smithi043.stdout:3/652: mkdir d4/de/d1f/d2c/db7/dc9 0 2024-09-16T23:49:00.802 INFO:tasks.workunit.client.1.smithi043.stdout:7/606: stat d4/db/d11/d15/d29/f76 0 2024-09-16T23:49:00.804 INFO:tasks.workunit.client.1.smithi043.stdout:6/653: write de/f40 [1255527,44383] 0 2024-09-16T23:49:00.805 INFO:tasks.workunit.client.0.smithi042.stdout:5/638: dread d9/db/de/d5e/d87/d61/f5d [0,4194304] 0 2024-09-16T23:49:00.808 INFO:tasks.workunit.client.0.smithi042.stdout:6/418: readlink d8/d70/d81/dd/d2c/l6a 0 2024-09-16T23:49:00.808 INFO:tasks.workunit.client.0.smithi042.stdout:9/340: dwrite d4/d12/d2d/d37/f6a [0,4194304] 0 2024-09-16T23:49:00.809 INFO:tasks.workunit.client.1.smithi043.stdout:7/607: dwrite d4/db/d43/d83/d85/f42 [0,4194304] 0 2024-09-16T23:49:00.810 INFO:tasks.workunit.client.0.smithi042.stdout:4/693: rename de/d1a/da7/c86 to de/d12/d1b/cef 0 2024-09-16T23:49:00.813 INFO:tasks.workunit.client.0.smithi042.stdout:0/382: dread d0/df/f18 [0,4194304] 0 2024-09-16T23:49:00.815 INFO:tasks.workunit.client.1.smithi043.stdout:1/782: sync 2024-09-16T23:49:00.816 INFO:tasks.workunit.client.0.smithi042.stdout:8/424: dwrite da/d19/d1d/f62 [0,4194304] 0 2024-09-16T23:49:00.820 INFO:tasks.workunit.client.1.smithi043.stdout:9/583: dwrite d4/d5/d16/f54 [0,4194304] 0 2024-09-16T23:49:00.822 INFO:tasks.workunit.client.0.smithi042.stdout:9/341: unlink d4/d12/f39 0 2024-09-16T23:49:00.822 INFO:tasks.workunit.client.0.smithi042.stdout:0/383: dread d0/df/d23/f31 [0,4194304] 0 2024-09-16T23:49:00.823 INFO:tasks.workunit.client.1.smithi043.stdout:7/608: dread d4/db/d43/f79 [0,4194304] 0 2024-09-16T23:49:00.828 INFO:tasks.workunit.client.0.smithi042.stdout:8/425: dread da/d4b/f72 [0,4194304] 0 2024-09-16T23:49:00.829 INFO:tasks.workunit.client.0.smithi042.stdout:0/384: write d0/f4b [1137947,31157] 0 2024-09-16T23:49:00.829 INFO:tasks.workunit.client.0.smithi042.stdout:7/512: dwrite d3/d98/da0/d24/f2d [4194304,4194304] 0 2024-09-16T23:49:00.830 INFO:tasks.workunit.client.0.smithi042.stdout:6/419: symlink d8/d70/d81/dd/d1b/d67/l8c 0 2024-09-16T23:49:00.833 INFO:tasks.workunit.client.0.smithi042.stdout:8/426: readlink da/d19/d80/l81 0 2024-09-16T23:49:00.836 INFO:tasks.workunit.client.0.smithi042.stdout:9/342: creat d4/d12/d16/f6d x:0 0 0 2024-09-16T23:49:00.838 INFO:tasks.workunit.client.0.smithi042.stdout:5/639: rename d9/db/d1b/d31/d46/d58/l83 to d9/d4a/d89/ldd 0 2024-09-16T23:49:00.838 INFO:tasks.workunit.client.0.smithi042.stdout:9/343: chown d4/d10/f1b 2019388 1 2024-09-16T23:49:00.839 INFO:tasks.workunit.client.0.smithi042.stdout:0/385: dread d0/df/d62/f65 [0,4194304] 0 2024-09-16T23:49:00.840 INFO:tasks.workunit.client.0.smithi042.stdout:7/513: chown d3/d1f/d49/d76/d19/d63/f69 2652 1 2024-09-16T23:49:00.843 INFO:tasks.workunit.client.0.smithi042.stdout:5/640: symlink d9/d2c/d7b/lde 0 2024-09-16T23:49:00.845 INFO:tasks.workunit.client.0.smithi042.stdout:0/386: rename d0/d15/l30 to d0/df/d23/d33/d56/l76 0 2024-09-16T23:49:00.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/588: truncate d0/f1d 4885791 0 2024-09-16T23:49:00.848 INFO:tasks.workunit.client.0.smithi042.stdout:6/420: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d7b/c8d 0 2024-09-16T23:49:00.850 INFO:tasks.workunit.client.0.smithi042.stdout:5/641: write d9/db/de/d5e/d87/fb9 [6685337,52276] 0 2024-09-16T23:49:00.850 INFO:tasks.workunit.client.0.smithi042.stdout:9/344: mknod d4/c6e 0 2024-09-16T23:49:00.852 INFO:tasks.workunit.client.0.smithi042.stdout:7/514: dwrite d3/d98/f8f [0,4194304] 0 2024-09-16T23:49:00.854 INFO:tasks.workunit.client.0.smithi042.stdout:6/421: mknod d8/d70/d81/dd/d1b/c8e 0 2024-09-16T23:49:00.857 INFO:tasks.workunit.client.0.smithi042.stdout:0/387: rmdir d0/df/d13/d6a 0 2024-09-16T23:49:00.858 INFO:tasks.workunit.client.0.smithi042.stdout:7/515: mkdir d3/d1f/d8e/db5 0 2024-09-16T23:49:00.858 INFO:tasks.workunit.client.0.smithi042.stdout:1/589: rename d0/d9/f78 to d0/d9/dba/fd1 0 2024-09-16T23:49:00.861 INFO:tasks.workunit.client.0.smithi042.stdout:9/345: mkdir d4/d12/d2d/d37/d3a/d50/d6f 0 2024-09-16T23:49:00.862 INFO:tasks.workunit.client.0.smithi042.stdout:5/642: dwrite d9/db/f28 [0,4194304] 0 2024-09-16T23:49:00.864 INFO:tasks.workunit.client.0.smithi042.stdout:1/590: creat d0/d9/d26/fd2 x:0 0 0 2024-09-16T23:49:00.865 INFO:tasks.workunit.client.0.smithi042.stdout:7/516: symlink d3/d98/da0/d41/db0/lb6 0 2024-09-16T23:49:00.866 INFO:tasks.workunit.client.0.smithi042.stdout:6/422: dread - d8/d38/f74 zero size 2024-09-16T23:49:00.866 INFO:tasks.workunit.client.1.smithi043.stdout:3/653: symlink d4/de/d1f/d34/lca 0 2024-09-16T23:49:00.867 INFO:tasks.workunit.client.1.smithi043.stdout:3/654: write d4/f23 [343195,119974] 0 2024-09-16T23:49:00.867 INFO:tasks.workunit.client.1.smithi043.stdout:6/654: mknod de/d23/d63/d7f/cda 0 2024-09-16T23:49:00.873 INFO:tasks.workunit.client.0.smithi042.stdout:5/643: rename d9/db/de/d10/d2f/f36 to d9/db/de/d92/fdf 0 2024-09-16T23:49:00.874 INFO:tasks.workunit.client.0.smithi042.stdout:8/427: dwrite da/d19/d1d/f50 [0,4194304] 0 2024-09-16T23:49:00.878 INFO:tasks.workunit.client.1.smithi043.stdout:9/584: mknod d4/d1c/cb3 0 2024-09-16T23:49:00.881 INFO:tasks.workunit.client.0.smithi042.stdout:1/591: truncate d0/d9/f1c 875813 0 2024-09-16T23:49:00.881 INFO:tasks.workunit.client.0.smithi042.stdout:7/517: dwrite d3/d98/f80 [0,4194304] 0 2024-09-16T23:49:00.882 INFO:tasks.workunit.client.1.smithi043.stdout:7/609: dread - d4/db/d11/d21/d94/f9c zero size 2024-09-16T23:49:00.882 INFO:tasks.workunit.client.0.smithi042.stdout:5/644: creat d9/db/de/d92/daf/fe0 x:0 0 0 2024-09-16T23:49:00.883 INFO:tasks.workunit.client.1.smithi043.stdout:7/610: write d4/db/d28/f97 [1672018,95686] 0 2024-09-16T23:49:00.884 INFO:tasks.workunit.client.0.smithi042.stdout:9/346: creat d4/d12/d2d/d37/f70 x:0 0 0 2024-09-16T23:49:00.888 INFO:tasks.workunit.client.0.smithi042.stdout:6/423: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f78 [0,4194304] 0 2024-09-16T23:49:00.888 INFO:tasks.workunit.client.0.smithi042.stdout:8/428: creat da/d19/d2c/f82 x:0 0 0 2024-09-16T23:49:00.889 INFO:tasks.workunit.client.0.smithi042.stdout:7/518: mkdir d3/db7 0 2024-09-16T23:49:00.891 INFO:tasks.workunit.client.0.smithi042.stdout:5/645: rename d9/d2c/d7b/ca7 to d9/db/d11/ce1 0 2024-09-16T23:49:00.892 INFO:tasks.workunit.client.0.smithi042.stdout:6/424: write d8/d70/d81/dd/d2c/d57/f5d [916669,42486] 0 2024-09-16T23:49:00.893 INFO:tasks.workunit.client.0.smithi042.stdout:1/592: unlink d0/d3b/l64 0 2024-09-16T23:49:00.894 INFO:tasks.workunit.client.0.smithi042.stdout:7/519: mkdir d3/d98/da0/d32/d6b/d5b/db8 0 2024-09-16T23:49:00.895 INFO:tasks.workunit.client.1.smithi043.stdout:5/818: sync 2024-09-16T23:49:00.895 INFO:tasks.workunit.client.1.smithi043.stdout:4/686: sync 2024-09-16T23:49:00.895 INFO:tasks.workunit.client.1.smithi043.stdout:6/655: mknod de/dbe/cdb 0 2024-09-16T23:49:00.896 INFO:tasks.workunit.client.0.smithi042.stdout:5/646: mkdir d9/db/d1b/da4/de2 0 2024-09-16T23:49:00.897 INFO:tasks.workunit.client.1.smithi043.stdout:5/819: write d3/db/d19/d3e/f45 [766992,2016] 0 2024-09-16T23:49:00.898 INFO:tasks.workunit.client.0.smithi042.stdout:7/520: unlink d3/d1f/c74 0 2024-09-16T23:49:00.899 INFO:tasks.workunit.client.1.smithi043.stdout:3/655: mknod d4/de/d40/d4d/ccb 0 2024-09-16T23:49:00.900 INFO:tasks.workunit.client.0.smithi042.stdout:7/521: write d3/f9a [1036391,115673] 0 2024-09-16T23:49:00.901 INFO:tasks.workunit.client.1.smithi043.stdout:8/615: write de/d18/d1f/f2d [4832893,55553] 0 2024-09-16T23:49:00.902 INFO:tasks.workunit.client.0.smithi042.stdout:7/522: stat d3/d98/da0/d32/d6b/d64/c65 0 2024-09-16T23:49:00.905 INFO:tasks.workunit.client.1.smithi043.stdout:1/783: symlink d4/d9/d12/d2b/l111 0 2024-09-16T23:49:00.907 INFO:tasks.workunit.client.0.smithi042.stdout:6/425: fsync d8/d70/d81/f12 0 2024-09-16T23:49:00.909 INFO:tasks.workunit.client.1.smithi043.stdout:9/585: truncate d4/d28/d49/f73 980780 0 2024-09-16T23:49:00.912 INFO:tasks.workunit.client.0.smithi042.stdout:9/347: creat d4/d12/d16/d68/f71 x:0 0 0 2024-09-16T23:49:00.913 INFO:tasks.workunit.client.0.smithi042.stdout:7/523: dwrite d3/d1f/d49/d76/f25 [0,4194304] 0 2024-09-16T23:49:00.917 INFO:tasks.workunit.client.1.smithi043.stdout:7/611: creat d4/d27/d41/d1b/d95/fc8 x:0 0 0 2024-09-16T23:49:00.918 INFO:tasks.workunit.client.0.smithi042.stdout:5/647: dread d9/d4a/d89/f9f [0,4194304] 0 2024-09-16T23:49:00.918 INFO:tasks.workunit.client.1.smithi043.stdout:7/612: truncate d4/fa0 280713 0 2024-09-16T23:49:00.919 INFO:tasks.workunit.client.0.smithi042.stdout:7/524: mkdir d3/d1f/d49/d78/db9 0 2024-09-16T23:49:00.922 INFO:tasks.workunit.client.1.smithi043.stdout:6/656: unlink de/d1a/f42 0 2024-09-16T23:49:00.923 INFO:tasks.workunit.client.0.smithi042.stdout:1/593: dread d0/f3 [4194304,4194304] 0 2024-09-16T23:49:00.924 INFO:tasks.workunit.client.0.smithi042.stdout:7/525: rename d3/d1f/d49/d76/f1c to d3/d98/da0/d32/d6b/d64/d92/fba 0 2024-09-16T23:49:00.926 INFO:tasks.workunit.client.1.smithi043.stdout:4/687: rename d8/d15/d6e to d8/d25/d39/dd3 0 2024-09-16T23:49:00.927 INFO:tasks.workunit.client.1.smithi043.stdout:4/688: dread - d8/d15/d20/d2b/d23/d27/faf zero size 2024-09-16T23:49:00.929 INFO:tasks.workunit.client.0.smithi042.stdout:5/648: link d9/d2c/da9/cad d9/db/d19/d21/d85/ce3 0 2024-09-16T23:49:00.930 INFO:tasks.workunit.client.1.smithi043.stdout:8/616: chown de/f71 3038902 1 2024-09-16T23:49:00.931 INFO:tasks.workunit.client.0.smithi042.stdout:5/649: write d9/db/d19/fdb [598709,57928] 0 2024-09-16T23:49:00.931 INFO:tasks.workunit.client.0.smithi042.stdout:7/526: getdents d3/d98/da0/d41 0 2024-09-16T23:49:00.931 INFO:tasks.workunit.client.0.smithi042.stdout:7/527: fdatasync d3/f30 0 2024-09-16T23:49:00.932 INFO:tasks.workunit.client.0.smithi042.stdout:5/650: stat d9/ca 0 2024-09-16T23:49:00.932 INFO:tasks.workunit.client.1.smithi043.stdout:1/784: creat d4/d9/d1c/f112 x:0 0 0 2024-09-16T23:49:00.935 INFO:tasks.workunit.client.1.smithi043.stdout:3/656: getdents d4/de/d1f/d2c/db7/dc6 0 2024-09-16T23:49:00.937 INFO:tasks.workunit.client.0.smithi042.stdout:5/651: getdents d9/db/de/d10/d2f 0 2024-09-16T23:49:00.937 INFO:tasks.workunit.client.1.smithi043.stdout:9/586: mkdir d4/d1c/d25/d2b/d4c/da3/db4 0 2024-09-16T23:49:00.937 INFO:tasks.workunit.client.1.smithi043.stdout:8/617: dwrite de/d18/d1f/f32 [0,4194304] 0 2024-09-16T23:49:00.938 INFO:tasks.workunit.client.1.smithi043.stdout:7/613: mknod d4/db/d43/d83/cc9 0 2024-09-16T23:49:00.939 INFO:tasks.workunit.client.1.smithi043.stdout:6/657: mkdir de/d23/ddc 0 2024-09-16T23:49:00.942 INFO:tasks.workunit.client.1.smithi043.stdout:1/785: unlink d4/d9/d8f/db6/dcc/dda/d107/fc1 0 2024-09-16T23:49:00.947 INFO:tasks.workunit.client.0.smithi042.stdout:5/652: truncate d9/db/d11/f1c 5875807 0 2024-09-16T23:49:00.947 INFO:tasks.workunit.client.1.smithi043.stdout:9/587: creat d4/d1c/fb5 x:0 0 0 2024-09-16T23:49:00.949 INFO:tasks.workunit.client.1.smithi043.stdout:6/658: dwrite fd [0,4194304] 0 2024-09-16T23:49:00.950 INFO:tasks.workunit.client.0.smithi042.stdout:7/528: dread d3/f5f [0,4194304] 0 2024-09-16T23:49:00.951 INFO:tasks.workunit.client.1.smithi043.stdout:7/614: rmdir d4/db/d50 39 2024-09-16T23:49:00.952 INFO:tasks.workunit.client.0.smithi042.stdout:7/529: dread - d3/d98/f71 zero size 2024-09-16T23:49:00.954 INFO:tasks.workunit.client.1.smithi043.stdout:8/618: dread de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:49:00.957 INFO:tasks.workunit.client.0.smithi042.stdout:5/653: dwrite d9/d2c/da9/fce [0,4194304] 0 2024-09-16T23:49:00.957 INFO:tasks.workunit.client.0.smithi042.stdout:7/530: unlink d3/d3f/fab 0 2024-09-16T23:49:00.957 INFO:tasks.workunit.client.1.smithi043.stdout:8/619: read de/d18/d1f/d25/f3b [1617021,11749] 0 2024-09-16T23:49:00.959 INFO:tasks.workunit.client.1.smithi043.stdout:1/786: dread d4/d9/d12/d2b/f52 [4194304,4194304] 0 2024-09-16T23:49:00.960 INFO:tasks.workunit.client.0.smithi042.stdout:5/654: chown d9/d2c/da9 1025849805 1 2024-09-16T23:49:00.961 INFO:tasks.workunit.client.1.smithi043.stdout:9/588: readlink d4/d28/d49/l5a 0 2024-09-16T23:49:00.961 INFO:tasks.workunit.client.0.smithi042.stdout:7/531: dread - d3/d98/f71 zero size 2024-09-16T23:49:00.962 INFO:tasks.workunit.client.1.smithi043.stdout:4/689: getdents d8/d15/d20/d2b/d23/d27 0 2024-09-16T23:49:00.965 INFO:tasks.workunit.client.1.smithi043.stdout:0/654: sync 2024-09-16T23:49:00.965 INFO:tasks.workunit.client.1.smithi043.stdout:2/591: sync 2024-09-16T23:49:00.967 INFO:tasks.workunit.client.0.smithi042.stdout:1/594: write d0/f3 [2382735,56559] 0 2024-09-16T23:49:00.967 INFO:tasks.workunit.client.1.smithi043.stdout:5/820: write d3/d9/d22/d4a/fc0 [2914041,46708] 0 2024-09-16T23:49:00.967 INFO:tasks.workunit.client.0.smithi042.stdout:6/426: dwrite d8/d70/d81/f20 [0,4194304] 0 2024-09-16T23:49:00.969 INFO:tasks.workunit.client.1.smithi043.stdout:0/655: rename d0/d2/df/d5a to d0/d2/df/d5a/d9c/dde 22 2024-09-16T23:49:00.970 INFO:tasks.workunit.client.1.smithi043.stdout:8/620: creat de/d18/d1f/d2f/d39/d7b/fba x:0 0 0 2024-09-16T23:49:00.971 INFO:tasks.workunit.client.0.smithi042.stdout:7/532: symlink d3/d98/da0/d32/d6b/d64/lbb 0 2024-09-16T23:49:00.975 INFO:tasks.workunit.client.1.smithi043.stdout:9/589: mkdir d4/d28/d5f/db6 0 2024-09-16T23:49:00.975 INFO:tasks.workunit.client.0.smithi042.stdout:9/348: dwrite d4/d10/d24/f3b [0,4194304] 0 2024-09-16T23:49:00.977 INFO:tasks.workunit.client.0.smithi042.stdout:9/349: stat d4/d12/d2d/d37/d3a/d50/c53 0 2024-09-16T23:49:00.977 INFO:tasks.workunit.client.1.smithi043.stdout:2/592: dwrite d8/d4b/d5f/d63/f99 [4194304,4194304] 0 2024-09-16T23:49:00.979 INFO:tasks.workunit.client.0.smithi042.stdout:7/533: write d3/d98/da0/d41/f47 [3066144,24108] 0 2024-09-16T23:49:00.980 INFO:tasks.workunit.client.0.smithi042.stdout:9/350: write d4/d10/d24/f67 [482634,59730] 0 2024-09-16T23:49:00.983 INFO:tasks.workunit.client.0.smithi042.stdout:5/655: truncate d9/db/d1b/d31/d46/f6d 941939 0 2024-09-16T23:49:00.984 INFO:tasks.workunit.client.0.smithi042.stdout:7/534: rmdir d3/d98 39 2024-09-16T23:49:00.985 INFO:tasks.workunit.client.0.smithi042.stdout:1/595: dread d0/d9/fb9 [0,4194304] 0 2024-09-16T23:49:00.986 INFO:tasks.workunit.client.0.smithi042.stdout:6/427: unlink d8/d70/d81/c48 0 2024-09-16T23:49:00.990 INFO:tasks.workunit.client.0.smithi042.stdout:5/656: truncate d9/db/d19/d21/d85/da5/f99 644413 0 2024-09-16T23:49:00.990 INFO:tasks.workunit.client.1.smithi043.stdout:3/657: dwrite d4/de/d1f/f56 [0,4194304] 0 2024-09-16T23:49:00.992 INFO:tasks.workunit.client.0.smithi042.stdout:5/657: rename d9/db/d1b/d79 to d9/db/d1b/d79/de4 22 2024-09-16T23:49:00.993 INFO:tasks.workunit.client.1.smithi043.stdout:2/593: dwrite d8/d1d/d25/f2b [0,4194304] 0 2024-09-16T23:49:00.997 INFO:tasks.workunit.client.1.smithi043.stdout:0/656: rmdir d0/d2/df/d1a 39 2024-09-16T23:49:00.997 INFO:tasks.workunit.client.0.smithi042.stdout:7/535: dread d3/d98/da0/f6c [0,4194304] 0 2024-09-16T23:49:00.997 INFO:tasks.workunit.client.0.smithi042.stdout:1/596: fsync d0/d9/d90/d6b/d88/d32/f33 0 2024-09-16T23:49:00.997 INFO:tasks.workunit.client.0.smithi042.stdout:6/428: dwrite d8/d70/d81/dd/f5f [0,4194304] 0 2024-09-16T23:49:00.997 INFO:tasks.workunit.client.0.smithi042.stdout:7/536: chown d3/d98/da0/d24/d28/fa3 37 1 2024-09-16T23:49:00.998 INFO:tasks.workunit.client.1.smithi043.stdout:1/787: dread d4/df/df4/d83/fc4 [0,4194304] 0 2024-09-16T23:49:00.999 INFO:tasks.workunit.client.1.smithi043.stdout:6/659: dwrite de/d23/d34/f5d [0,4194304] 0 2024-09-16T23:49:01.000 INFO:tasks.workunit.client.1.smithi043.stdout:9/590: rename d4/f1a to d4/d21/d35/d4b/fb7 0 2024-09-16T23:49:01.003 INFO:tasks.workunit.client.1.smithi043.stdout:3/658: dread d4/de/d40/f69 [0,4194304] 0 2024-09-16T23:49:01.005 INFO:tasks.workunit.client.0.smithi042.stdout:9/351: link d4/d12/d2d/f32 d4/d12/d16/d68/f72 0 2024-09-16T23:49:01.006 INFO:tasks.workunit.client.0.smithi042.stdout:7/537: mknod d3/d98/da0/d32/d6b/d64/cbc 0 2024-09-16T23:49:01.006 INFO:tasks.workunit.client.0.smithi042.stdout:5/658: getdents d9/db/de/d10/dc8 0 2024-09-16T23:49:01.007 INFO:tasks.workunit.client.0.smithi042.stdout:5/659: fsync d9/db/de/d10/f22 0 2024-09-16T23:49:01.008 INFO:tasks.workunit.client.0.smithi042.stdout:5/660: chown d9/db/d19/d21 137878740 1 2024-09-16T23:49:01.009 INFO:tasks.workunit.client.1.smithi043.stdout:3/659: dread d4/de/d40/fb5 [0,4194304] 0 2024-09-16T23:49:01.010 INFO:tasks.workunit.client.1.smithi043.stdout:2/594: dwrite d8/d11/d16/fb7 [0,4194304] 0 2024-09-16T23:49:01.011 INFO:tasks.workunit.client.1.smithi043.stdout:4/690: symlink d8/ld4 0 2024-09-16T23:49:01.011 INFO:tasks.workunit.client.0.smithi042.stdout:7/538: creat d3/d98/da0/d24/fbd x:0 0 0 2024-09-16T23:49:01.014 INFO:tasks.workunit.client.0.smithi042.stdout:7/539: write d3/d1f/d4f/f5d [4104763,80931] 0 2024-09-16T23:49:01.016 INFO:tasks.workunit.client.0.smithi042.stdout:6/429: rmdir d8/d70/d81/dd/d1b/d34 39 2024-09-16T23:49:01.017 INFO:tasks.workunit.client.0.smithi042.stdout:5/661: dread d9/db/de/fb3 [0,4194304] 0 2024-09-16T23:49:01.021 INFO:tasks.workunit.client.0.smithi042.stdout:7/540: dread d3/d98/da0/f6c [0,4194304] 0 2024-09-16T23:49:01.022 INFO:tasks.workunit.client.1.smithi043.stdout:6/660: rename de/d23/d34/d9b/d5a/fa7 to de/d1a/d80/fdd 0 2024-09-16T23:49:01.023 INFO:tasks.workunit.client.0.smithi042.stdout:1/597: dwrite d0/d17/f4f [0,4194304] 0 2024-09-16T23:49:01.023 INFO:tasks.workunit.client.1.smithi043.stdout:6/661: write fd [177642,44584] 0 2024-09-16T23:49:01.029 INFO:tasks.workunit.client.0.smithi042.stdout:1/598: truncate d0/d9/d26/d8b/f99 134647 0 2024-09-16T23:49:01.030 INFO:tasks.workunit.client.0.smithi042.stdout:6/430: chown d8/d70/d81/dd/d1b/d34/d39/l89 11233 1 2024-09-16T23:49:01.030 INFO:tasks.workunit.client.1.smithi043.stdout:3/660: creat d4/d30/fcc x:0 0 0 2024-09-16T23:49:01.033 INFO:tasks.workunit.client.0.smithi042.stdout:5/662: dwrite d9/f3c [0,4194304] 0 2024-09-16T23:49:01.034 INFO:tasks.workunit.client.1.smithi043.stdout:4/691: mkdir d8/d15/d20/d2b/d54/db3/dd5 0 2024-09-16T23:49:01.035 INFO:tasks.workunit.client.0.smithi042.stdout:7/541: getdents d3/d1f/d49/d76 0 2024-09-16T23:49:01.037 INFO:tasks.workunit.client.0.smithi042.stdout:1/599: unlink d0/d72/c83 0 2024-09-16T23:49:01.038 INFO:tasks.workunit.client.0.smithi042.stdout:9/352: dwrite d4/d12/f36 [0,4194304] 0 2024-09-16T23:49:01.039 INFO:tasks.workunit.client.0.smithi042.stdout:9/353: fdatasync d4/d10/f3f 0 2024-09-16T23:49:01.039 INFO:tasks.workunit.client.1.smithi043.stdout:9/591: write d4/f48 [303297,118726] 0 2024-09-16T23:49:01.040 INFO:tasks.workunit.client.0.smithi042.stdout:5/663: symlink d9/d4a/d89/le5 0 2024-09-16T23:49:01.042 INFO:tasks.workunit.client.0.smithi042.stdout:7/542: mkdir d3/d1f/d49/d78/dbe 0 2024-09-16T23:49:01.043 INFO:tasks.workunit.client.0.smithi042.stdout:6/431: mkdir d8/d70/d81/dd/d1b/d67/d8f 0 2024-09-16T23:49:01.044 INFO:tasks.workunit.client.1.smithi043.stdout:4/692: dwrite d8/d15/d20/d2b/d23/fd1 [0,4194304] 0 2024-09-16T23:49:01.046 INFO:tasks.workunit.client.0.smithi042.stdout:7/543: symlink d3/d1f/d49/d78/dbe/lbf 0 2024-09-16T23:49:01.048 INFO:tasks.workunit.client.1.smithi043.stdout:1/788: link d4/d9/d8f/db6/dcc/dda/d107/f105 d4/d9/d1c/f113 0 2024-09-16T23:49:01.050 INFO:tasks.workunit.client.0.smithi042.stdout:9/354: mkdir d4/d12/d2d/d37/d59/d73 0 2024-09-16T23:49:01.050 INFO:tasks.workunit.client.0.smithi042.stdout:7/544: mknod d3/d98/cc0 0 2024-09-16T23:49:01.050 INFO:tasks.workunit.client.1.smithi043.stdout:1/789: write d4/f69 [919690,69604] 0 2024-09-16T23:49:01.051 INFO:tasks.workunit.client.0.smithi042.stdout:9/355: write d4/f5 [126292,82666] 0 2024-09-16T23:49:01.053 INFO:tasks.workunit.client.0.smithi042.stdout:7/545: mknod d3/d3f/cc1 0 2024-09-16T23:49:01.057 INFO:tasks.workunit.client.0.smithi042.stdout:9/356: creat d4/d12/d16/d68/f74 x:0 0 0 2024-09-16T23:49:01.058 INFO:tasks.workunit.client.1.smithi043.stdout:2/595: dwrite d8/d1f/f2e [0,4194304] 0 2024-09-16T23:49:01.058 INFO:tasks.workunit.client.0.smithi042.stdout:6/432: mkdir d8/d70/d81/d90 0 2024-09-16T23:49:01.058 INFO:tasks.workunit.client.0.smithi042.stdout:7/546: rename d3/d1f/d49/d76/c2b to d3/d98/da0/d32/d6b/d5b/dae/cc2 0 2024-09-16T23:49:01.059 INFO:tasks.workunit.client.0.smithi042.stdout:9/357: read d4/d12/d2d/d37/d3a/f48 [4088438,89678] 0 2024-09-16T23:49:01.063 INFO:tasks.workunit.client.0.smithi042.stdout:7/547: mkdir d3/d98/da0/d24/dc3 0 2024-09-16T23:49:01.063 INFO:tasks.workunit.client.1.smithi043.stdout:1/790: dwrite d4/df/df4/d83/d98/fb8 [0,4194304] 0 2024-09-16T23:49:01.065 INFO:tasks.workunit.client.0.smithi042.stdout:1/600: truncate d0/d3b/f82 2974061 0 2024-09-16T23:49:01.066 INFO:tasks.workunit.client.0.smithi042.stdout:7/548: mkdir d3/d1f/d8e/db5/dc4 0 2024-09-16T23:49:01.069 INFO:tasks.workunit.client.0.smithi042.stdout:7/549: fsync d3/d98/da0/d32/d6b/d5b/dae/fb1 0 2024-09-16T23:49:01.069 INFO:tasks.workunit.client.1.smithi043.stdout:0/657: link d0/d2/df/d1a/d46/d39/d6e/lb5 d0/d2/df/d1a/d46/d39/d83/dcb/ldf 0 2024-09-16T23:49:01.070 INFO:tasks.workunit.client.0.smithi042.stdout:9/358: symlink d4/d10/d24/l75 0 2024-09-16T23:49:01.072 INFO:tasks.workunit.client.1.smithi043.stdout:9/592: symlink d4/d1c/d25/lb8 0 2024-09-16T23:49:01.072 INFO:tasks.workunit.client.1.smithi043.stdout:2/596: dwrite d8/d11/d16/f81 [0,4194304] 0 2024-09-16T23:49:01.075 INFO:tasks.workunit.client.0.smithi042.stdout:6/433: write d8/d70/d81/dd/d2c/f31 [1656219,17247] 0 2024-09-16T23:49:01.075 INFO:tasks.workunit.client.1.smithi043.stdout:3/661: dread d4/f67 [0,4194304] 0 2024-09-16T23:49:01.076 INFO:tasks.workunit.client.1.smithi043.stdout:1/791: creat d4/d9/d8f/db6/dc6/f114 x:0 0 0 2024-09-16T23:49:01.077 INFO:tasks.workunit.client.0.smithi042.stdout:6/434: chown d8/d70/d81/fc 6603799 1 2024-09-16T23:49:01.078 INFO:tasks.workunit.client.1.smithi043.stdout:0/658: symlink d0/d2/df/d1a/d9a/le0 0 2024-09-16T23:49:01.080 INFO:tasks.workunit.client.1.smithi043.stdout:9/593: write d4/d5/d64/f65 [874740,7285] 0 2024-09-16T23:49:01.082 INFO:tasks.workunit.client.0.smithi042.stdout:9/359: dwrite d4/d10/f1f [4194304,4194304] 0 2024-09-16T23:49:01.083 INFO:tasks.workunit.client.1.smithi043.stdout:6/662: getdents de/d23/d34 0 2024-09-16T23:49:01.083 INFO:tasks.workunit.client.0.smithi042.stdout:9/360: write d4/d12/d16/f6d [1000071,42585] 0 2024-09-16T23:49:01.085 INFO:tasks.workunit.client.1.smithi043.stdout:3/662: dwrite d4/de/d1f/d2c/f47 [0,4194304] 0 2024-09-16T23:49:01.086 INFO:tasks.workunit.client.1.smithi043.stdout:6/663: read de/d23/d34/d9b/d8b/fbb [12220,81506] 0 2024-09-16T23:49:01.090 INFO:tasks.workunit.client.1.smithi043.stdout:4/693: write d8/d15/d20/f8d [3486231,122162] 0 2024-09-16T23:49:01.092 INFO:tasks.workunit.client.0.smithi042.stdout:9/361: mknod d4/d12/d2d/d37/d59/c76 0 2024-09-16T23:49:01.093 INFO:tasks.workunit.client.0.smithi042.stdout:9/362: dread d4/d10/f4a [0,4194304] 0 2024-09-16T23:49:01.099 INFO:tasks.workunit.client.1.smithi043.stdout:2/597: rename d8/c33 to d8/d1d/d20/d23/d38/da2/cc9 0 2024-09-16T23:49:01.103 INFO:tasks.workunit.client.1.smithi043.stdout:2/598: fdatasync d8/d4b/d5f/d8e/fbb 0 2024-09-16T23:49:01.106 INFO:tasks.workunit.client.1.smithi043.stdout:4/694: dwrite d8/d15/d20/d2b/f6b [0,4194304] 0 2024-09-16T23:49:01.106 INFO:tasks.workunit.client.1.smithi043.stdout:9/594: dread d4/d1c/d6b/f39 [0,4194304] 0 2024-09-16T23:49:01.108 INFO:tasks.workunit.client.0.smithi042.stdout:7/550: dread d3/d1f/d49/d76/d19/d63/f75 [0,4194304] 0 2024-09-16T23:49:01.109 INFO:tasks.workunit.client.1.smithi043.stdout:1/792: dwrite d4/d3e/fdb [0,4194304] 0 2024-09-16T23:49:01.110 INFO:tasks.workunit.client.1.smithi043.stdout:0/659: dwrite d0/de/f2d [0,4194304] 0 2024-09-16T23:49:01.111 INFO:tasks.workunit.client.1.smithi043.stdout:4/695: stat d8/d25/d39/d58/fc0 0 2024-09-16T23:49:01.112 INFO:tasks.workunit.client.1.smithi043.stdout:1/793: chown d4/d9/d12 65 1 2024-09-16T23:49:01.113 INFO:tasks.workunit.client.0.smithi042.stdout:7/551: mknod d3/d1f/d49/cc5 0 2024-09-16T23:49:01.113 INFO:tasks.workunit.client.1.smithi043.stdout:3/663: rmdir d4/d30/d53/d88/db1/db2 0 2024-09-16T23:49:01.115 INFO:tasks.workunit.client.1.smithi043.stdout:0/660: chown d0/d2/df/d1a/d9a/le0 597 1 2024-09-16T23:49:01.116 INFO:tasks.workunit.client.1.smithi043.stdout:2/599: mkdir d8/d11/dca 0 2024-09-16T23:49:01.118 INFO:tasks.workunit.client.1.smithi043.stdout:6/664: truncate de/d1a/d28/f9a 3469970 0 2024-09-16T23:49:01.119 INFO:tasks.workunit.client.1.smithi043.stdout:1/794: dread d4/d34/fbe [0,4194304] 0 2024-09-16T23:49:01.120 INFO:tasks.workunit.client.1.smithi043.stdout:0/661: chown d0/d2/df/dbb/l9f 31 1 2024-09-16T23:49:01.122 INFO:tasks.workunit.client.1.smithi043.stdout:9/595: rename d4/d21/d2a/d2c/l98 to d4/d28/d5f/lb9 0 2024-09-16T23:49:01.122 INFO:tasks.workunit.client.1.smithi043.stdout:4/696: dwrite d8/f3b [4194304,4194304] 0 2024-09-16T23:49:01.123 INFO:tasks.workunit.client.0.smithi042.stdout:6/435: getdents d8/d70/d81 0 2024-09-16T23:49:01.123 INFO:tasks.workunit.client.1.smithi043.stdout:1/795: readlink d4/d9/d1c/d1e/d42/d5d/d71/d76/l9b 0 2024-09-16T23:49:01.124 INFO:tasks.workunit.client.1.smithi043.stdout:3/664: chown d4/d30/d53/l70 51573 1 2024-09-16T23:49:01.124 INFO:tasks.workunit.client.0.smithi042.stdout:6/436: chown f6 13602328 1 2024-09-16T23:49:01.125 INFO:tasks.workunit.client.1.smithi043.stdout:9/596: fdatasync d4/d1c/d4f/f8b 0 2024-09-16T23:49:01.125 INFO:tasks.workunit.client.1.smithi043.stdout:1/796: dread - d4/d9/d12/d15/dac/ff2 zero size 2024-09-16T23:49:01.126 INFO:tasks.workunit.client.0.smithi042.stdout:9/363: mkdir d4/d12/d77 0 2024-09-16T23:49:01.127 INFO:tasks.workunit.client.0.smithi042.stdout:6/437: readlink d8/d70/d81/dd/d1b/d2d/l75 0 2024-09-16T23:49:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:00 smithi043 ceph-mon[35191]: pgmap v9: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 82 MiB/s rd, 256 MiB/s wr, 493 op/s 2024-09-16T23:49:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:00 smithi043 ceph-mon[35191]: mgrmap e25: smithi043.xveyou(active, since 12s), standbys: smithi042.zrkumz 2024-09-16T23:49:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:00 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:49:01.128 INFO:tasks.workunit.client.1.smithi043.stdout:0/662: chown d0/d2/df/d1a/f30 2 1 2024-09-16T23:49:01.131 INFO:tasks.workunit.client.0.smithi042.stdout:7/552: dwrite d3/d1f/f23 [0,4194304] 0 2024-09-16T23:49:01.132 INFO:tasks.workunit.client.1.smithi043.stdout:9/597: mknod d4/d1c/d25/d2b/d4c/da3/cba 0 2024-09-16T23:49:01.133 INFO:tasks.workunit.client.1.smithi043.stdout:1/797: unlink d4/d9/d8f/db6/dc6/ccb 0 2024-09-16T23:49:01.136 INFO:tasks.workunit.client.1.smithi043.stdout:1/798: chown d4/d9/d1c/d1e/d47/f86 1 1 2024-09-16T23:49:01.136 INFO:tasks.workunit.client.1.smithi043.stdout:2/600: link d8/d1f/f6e d8/d5e/d83/d9e/dab/fcb 0 2024-09-16T23:49:01.137 INFO:tasks.workunit.client.0.smithi042.stdout:6/438: unlink d8/d70/d81/dd/d1b/c2b 0 2024-09-16T23:49:01.137 INFO:tasks.workunit.client.0.smithi042.stdout:7/553: read d3/d98/da0/d24/f59 [971169,57261] 0 2024-09-16T23:49:01.138 INFO:tasks.workunit.client.1.smithi043.stdout:1/799: chown d4/d3e/c50 38126859 1 2024-09-16T23:49:01.139 INFO:tasks.workunit.client.0.smithi042.stdout:6/439: write d8/d70/d81/dd/f5f [1845124,75146] 0 2024-09-16T23:49:01.140 INFO:tasks.workunit.client.0.smithi042.stdout:7/554: chown d3/d98/da0/d32/cb3 334320 1 2024-09-16T23:49:01.141 INFO:tasks.workunit.client.1.smithi043.stdout:4/697: creat d8/d15/d20/d2b/fd6 x:0 0 0 2024-09-16T23:49:01.142 INFO:tasks.workunit.client.1.smithi043.stdout:4/698: fsync d8/d25/d39/dd3/fcf 0 2024-09-16T23:49:01.143 INFO:tasks.workunit.client.0.smithi042.stdout:7/555: chown d3/d98/da0/d24/c33 12891902 1 2024-09-16T23:49:01.144 INFO:tasks.workunit.client.1.smithi043.stdout:9/598: creat d4/d21/d2a/da9/fbb x:0 0 0 2024-09-16T23:49:01.145 INFO:tasks.workunit.client.1.smithi043.stdout:2/601: creat d8/d4b/fcc x:0 0 0 2024-09-16T23:49:01.146 INFO:tasks.workunit.client.1.smithi043.stdout:9/599: chown d4/d21/d35/d4b 24 1 2024-09-16T23:49:01.146 INFO:tasks.workunit.client.0.smithi042.stdout:7/556: fsync d3/d1f/d49/d78/f7c 0 2024-09-16T23:49:01.147 INFO:tasks.workunit.client.0.smithi042.stdout:6/440: dwrite d8/d70/d81/dd/d2c/f10 [0,4194304] 0 2024-09-16T23:49:01.148 INFO:tasks.workunit.client.0.smithi042.stdout:6/441: chown d8/d70/d81/f23 387331957 1 2024-09-16T23:49:01.149 INFO:tasks.workunit.client.1.smithi043.stdout:0/663: dread d0/d2/df/d1a/d3b/f7d [0,4194304] 0 2024-09-16T23:49:01.151 INFO:tasks.workunit.client.1.smithi043.stdout:1/800: dwrite d4/d3e/d5b/dca/f10b [0,4194304] 0 2024-09-16T23:49:01.151 INFO:tasks.workunit.client.0.smithi042.stdout:6/442: dread - d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/f6b zero size 2024-09-16T23:49:01.152 INFO:tasks.workunit.client.1.smithi043.stdout:2/602: stat d8/d1d/d25/c74 0 2024-09-16T23:49:01.156 INFO:tasks.workunit.client.0.smithi042.stdout:7/557: rmdir d3/db7 0 2024-09-16T23:49:01.156 INFO:tasks.workunit.client.1.smithi043.stdout:2/603: write d8/d1d/d20/d23/fb8 [75184,22544] 0 2024-09-16T23:49:01.158 INFO:tasks.workunit.client.1.smithi043.stdout:1/801: creat d4/d3e/d5b/f115 x:0 0 0 2024-09-16T23:49:01.159 INFO:tasks.workunit.client.0.smithi042.stdout:6/443: creat d8/d70/d81/d90/f91 x:0 0 0 2024-09-16T23:49:01.161 INFO:tasks.workunit.client.1.smithi043.stdout:9/600: creat d4/d21/fbc x:0 0 0 2024-09-16T23:49:01.163 INFO:tasks.workunit.client.0.smithi042.stdout:7/558: dwrite d3/d98/da0/d24/d28/d9b/fac [0,4194304] 0 2024-09-16T23:49:01.164 INFO:tasks.workunit.client.1.smithi043.stdout:2/604: symlink d8/d1d/d20/d23/d38/lcd 0 2024-09-16T23:49:01.168 INFO:tasks.workunit.client.0.smithi042.stdout:7/559: unlink d3/d98/da0/d24/la7 0 2024-09-16T23:49:01.169 INFO:tasks.workunit.client.0.smithi042.stdout:6/444: rename d8/d70/d81/dd/d1b/d2d/l75 to d8/d70/d81/dd/d1b/d34/d39/l92 0 2024-09-16T23:49:01.170 INFO:tasks.workunit.client.0.smithi042.stdout:7/560: rmdir d3/d1f/d8e 39 2024-09-16T23:49:01.170 INFO:tasks.workunit.client.0.smithi042.stdout:6/445: truncate f7 4610497 0 2024-09-16T23:49:01.171 INFO:tasks.workunit.client.0.smithi042.stdout:7/561: write d3/d98/f80 [2288599,19250] 0 2024-09-16T23:49:01.172 INFO:tasks.workunit.client.1.smithi043.stdout:6/665: dwrite de/d23/dc3/fd1 [0,4194304] 0 2024-09-16T23:49:01.172 INFO:tasks.workunit.client.1.smithi043.stdout:9/601: truncate d4/d1c/f20 1582930 0 2024-09-16T23:49:01.174 INFO:tasks.workunit.client.0.smithi042.stdout:6/446: write d8/d70/d81/d90/f91 [716277,107095] 0 2024-09-16T23:49:01.174 INFO:tasks.workunit.client.1.smithi043.stdout:2/605: dwrite d8/d5e/fb4 [0,4194304] 0 2024-09-16T23:49:01.175 INFO:tasks.workunit.client.1.smithi043.stdout:4/699: getdents d8/d25/d39 0 2024-09-16T23:49:01.175 INFO:tasks.workunit.client.0.smithi042.stdout:7/562: symlink d3/d98/da0/d24/d28/lc6 0 2024-09-16T23:49:01.176 INFO:tasks.workunit.client.0.smithi042.stdout:6/447: write d8/d70/d81/dd/d2c/f10 [1873659,64519] 0 2024-09-16T23:49:01.177 INFO:tasks.workunit.client.1.smithi043.stdout:9/602: rmdir d4/d1c/d4f/d8c 39 2024-09-16T23:49:01.177 INFO:tasks.workunit.client.0.smithi042.stdout:6/448: fsync d8/d38/f74 0 2024-09-16T23:49:01.178 INFO:tasks.workunit.client.0.smithi042.stdout:7/563: truncate d3/d98/da0/d32/d6b/f83 246115 0 2024-09-16T23:49:01.180 INFO:tasks.workunit.client.1.smithi043.stdout:1/802: dread d4/f69 [0,4194304] 0 2024-09-16T23:49:01.182 INFO:tasks.workunit.client.1.smithi043.stdout:9/603: dread d4/d21/f89 [0,4194304] 0 2024-09-16T23:49:01.182 INFO:tasks.workunit.client.1.smithi043.stdout:1/803: truncate d4/d34/fd2 498719 0 2024-09-16T23:49:01.183 INFO:tasks.workunit.client.0.smithi042.stdout:7/564: unlink d3/d98/da0/d32/d6b/f60 0 2024-09-16T23:49:01.183 INFO:tasks.workunit.client.1.smithi043.stdout:2/606: getdents d8/d1d/d20/d23/d38/da2 0 2024-09-16T23:49:01.184 INFO:tasks.workunit.client.1.smithi043.stdout:9/604: chown d4/d21/d33/da2 354746490 1 2024-09-16T23:49:01.186 INFO:tasks.workunit.client.1.smithi043.stdout:9/605: creat d4/d28/d5f/fbd x:0 0 0 2024-09-16T23:49:01.187 INFO:tasks.workunit.client.1.smithi043.stdout:4/700: dread d8/d15/d20/d2b/d23/d27/d4b/fa4 [0,4194304] 0 2024-09-16T23:49:01.188 INFO:tasks.workunit.client.1.smithi043.stdout:2/607: link d8/d4b/d5f/d63/laa d8/d1d/d20/d23/lce 0 2024-09-16T23:49:01.189 INFO:tasks.workunit.client.1.smithi043.stdout:2/608: chown d8/d5e/d83/d3a/l3b 24 1 2024-09-16T23:49:01.191 INFO:tasks.workunit.client.1.smithi043.stdout:4/701: rename d8/d25/d39/dd3/l9d to d8/ld7 0 2024-09-16T23:49:01.192 INFO:tasks.workunit.client.1.smithi043.stdout:2/609: symlink d8/d5e/d83/d3a/d77/da9/lcf 0 2024-09-16T23:49:01.192 INFO:tasks.workunit.client.1.smithi043.stdout:2/610: fsync d8/d11/f46 0 2024-09-16T23:49:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:00 smithi042 ceph-mon[27914]: pgmap v9: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 82 MiB/s rd, 256 MiB/s wr, 493 op/s 2024-09-16T23:49:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:00 smithi042 ceph-mon[27914]: mgrmap e25: smithi043.xveyou(active, since 12s), standbys: smithi042.zrkumz 2024-09-16T23:49:01.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:00 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:49:01.275 INFO:tasks.workunit.client.0.smithi042.stdout:3/566: sync 2024-09-16T23:49:01.275 INFO:tasks.workunit.client.0.smithi042.stdout:3/567: chown d2/d1b/d1d/d30/da2 92207841 1 2024-09-16T23:49:01.277 INFO:tasks.workunit.client.0.smithi042.stdout:3/568: creat d2/d7/d61/fc5 x:0 0 0 2024-09-16T23:49:01.281 INFO:tasks.workunit.client.0.smithi042.stdout:3/569: link d2/d7/f81 d2/d1b/d1d/d34/d50/d93/d96/fc6 0 2024-09-16T23:49:01.281 INFO:tasks.workunit.client.0.smithi042.stdout:3/570: read d2/d1b/d1d/d34/f3e [7868748,45859] 0 2024-09-16T23:49:01.285 INFO:tasks.workunit.client.0.smithi042.stdout:3/571: creat d2/d1b/d1d/d71/fc7 x:0 0 0 2024-09-16T23:49:01.285 INFO:tasks.workunit.client.0.smithi042.stdout:3/572: mkdir d2/d1b/dc8 0 2024-09-16T23:49:01.287 INFO:tasks.workunit.client.0.smithi042.stdout:3/573: mknod d2/d7/d61/d8e/d9d/dc4/cc9 0 2024-09-16T23:49:01.288 INFO:tasks.workunit.client.0.smithi042.stdout:3/574: dread - d2/d1b/d1d/d34/f92 zero size 2024-09-16T23:49:01.292 INFO:tasks.workunit.client.0.smithi042.stdout:3/575: dwrite d2/d1b/f94 [0,4194304] 0 2024-09-16T23:49:01.299 INFO:tasks.workunit.client.0.smithi042.stdout:3/576: dwrite d2/d1b/d1d/f85 [0,4194304] 0 2024-09-16T23:49:01.302 INFO:tasks.workunit.client.0.smithi042.stdout:3/577: mknod d2/d7/d61/d8e/d2c/cca 0 2024-09-16T23:49:01.306 INFO:tasks.workunit.client.0.smithi042.stdout:3/578: symlink d2/d1b/d89/lcb 0 2024-09-16T23:49:01.309 INFO:tasks.workunit.client.0.smithi042.stdout:3/579: write d2/d7/d61/d8e/d9d/fc1 [999545,58205] 0 2024-09-16T23:49:01.310 INFO:tasks.workunit.client.0.smithi042.stdout:3/580: dread d2/d7/fa4 [0,4194304] 0 2024-09-16T23:49:01.314 INFO:tasks.workunit.client.0.smithi042.stdout:3/581: rename d2/d1b/d1d/db3 to d2/d7/d61/d8e/d9d/dc4/dcc 0 2024-09-16T23:49:01.315 INFO:tasks.workunit.client.0.smithi042.stdout:3/582: rmdir d2/d1b 39 2024-09-16T23:49:01.316 INFO:tasks.workunit.client.0.smithi042.stdout:3/583: write d2/d7/d61/fc5 [517189,74794] 0 2024-09-16T23:49:01.327 INFO:tasks.workunit.client.0.smithi042.stdout:3/584: dwrite d2/d1b/d89/f2e [0,4194304] 0 2024-09-16T23:49:01.329 INFO:tasks.workunit.client.0.smithi042.stdout:3/585: write d2/d1b/d1d/d34/d50/f6c [3177622,127160] 0 2024-09-16T23:49:01.337 INFO:tasks.workunit.client.0.smithi042.stdout:3/586: dread d2/d1b/d89/f45 [0,4194304] 0 2024-09-16T23:49:01.339 INFO:tasks.workunit.client.0.smithi042.stdout:3/587: fsync d2/d1b/d1d/d34/d50/d93/d96/fad 0 2024-09-16T23:49:01.340 INFO:tasks.workunit.client.0.smithi042.stdout:3/588: chown d2/d1b/d1d/d34/d62/d80 58360006 1 2024-09-16T23:49:01.341 INFO:tasks.workunit.client.0.smithi042.stdout:6/449: write d8/d70/d81/dd/d1b/d2d/f41 [288319,41091] 0 2024-09-16T23:49:01.343 INFO:tasks.workunit.client.0.smithi042.stdout:3/589: write d2/d1b/d1d/d34/d50/f6c [5125590,43056] 0 2024-09-16T23:49:01.344 INFO:tasks.workunit.client.1.smithi043.stdout:9/606: write d4/d21/d35/d4b/fb7 [2978638,63580] 0 2024-09-16T23:49:01.347 INFO:tasks.workunit.client.0.smithi042.stdout:2/494: sync 2024-09-16T23:49:01.347 INFO:tasks.workunit.client.1.smithi043.stdout:1/804: dwrite d4/d34/f6e [0,4194304] 0 2024-09-16T23:49:01.348 INFO:tasks.workunit.client.1.smithi043.stdout:0/664: dread d0/d2/df/d1a/d49/d80/fcd [0,4194304] 0 2024-09-16T23:49:01.348 INFO:tasks.workunit.client.0.smithi042.stdout:3/590: symlink d2/d1b/dc8/lcd 0 2024-09-16T23:49:01.349 INFO:tasks.workunit.client.1.smithi043.stdout:9/607: chown d4/d28/d49/l7f 6529149 1 2024-09-16T23:49:01.350 INFO:tasks.workunit.client.0.smithi042.stdout:6/450: creat d8/d70/d81/dd/d1b/f93 x:0 0 0 2024-09-16T23:49:01.350 INFO:tasks.workunit.client.0.smithi042.stdout:3/591: dread - d2/d7/f81 zero size 2024-09-16T23:49:01.351 INFO:tasks.workunit.client.0.smithi042.stdout:2/495: rmdir d0/da/d17/d3e/d86/d96 39 2024-09-16T23:49:01.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/451: chown d8/d70/d81/dd/f5f 118448028 1 2024-09-16T23:49:01.353 INFO:tasks.workunit.client.1.smithi043.stdout:1/805: readlink d4/d9/d8f/db6/dcc/dda/d107/ddf/lef 0 2024-09-16T23:49:01.353 INFO:tasks.workunit.client.0.smithi042.stdout:3/592: creat d2/d1b/fce x:0 0 0 2024-09-16T23:49:01.354 INFO:tasks.workunit.client.0.smithi042.stdout:2/496: symlink d0/d8d/laf 0 2024-09-16T23:49:01.356 INFO:tasks.workunit.client.0.smithi042.stdout:2/497: chown d0/da/d1b/d48/d81/l7f 274354 1 2024-09-16T23:49:01.356 INFO:tasks.workunit.client.1.smithi043.stdout:2/611: fsync d8/d5e/fb4 0 2024-09-16T23:49:01.356 INFO:tasks.workunit.client.1.smithi043.stdout:4/702: dwrite d8/d25/f5b [0,4194304] 0 2024-09-16T23:49:01.357 INFO:tasks.workunit.client.0.smithi042.stdout:2/498: readlink d0/da/d88/l8f 0 2024-09-16T23:49:01.360 INFO:tasks.workunit.client.1.smithi043.stdout:2/612: chown d8/d5e/fc2 178231656 1 2024-09-16T23:49:01.361 INFO:tasks.workunit.client.1.smithi043.stdout:4/703: dread - d8/d15/d9b/dbf/fc4 zero size 2024-09-16T23:49:01.361 INFO:tasks.workunit.client.1.smithi043.stdout:4/704: readlink d8/d15/d20/l33 0 2024-09-16T23:49:01.361 INFO:tasks.workunit.client.1.smithi043.stdout:9/608: rename d4 to d4/d21/dbe 22 2024-09-16T23:49:01.361 INFO:tasks.workunit.client.1.smithi043.stdout:1/806: write d4/d9/d12/d2b/f52 [1356868,103421] 0 2024-09-16T23:49:01.361 INFO:tasks.workunit.client.0.smithi042.stdout:3/593: creat d2/d7/d61/db9/fcf x:0 0 0 2024-09-16T23:49:01.361 INFO:tasks.workunit.client.0.smithi042.stdout:6/452: chown d8/d70/d81/dd/d2c/d57/f77 3500014 1 2024-09-16T23:49:01.362 INFO:tasks.workunit.client.0.smithi042.stdout:7/565: dread d3/fa [0,4194304] 0 2024-09-16T23:49:01.363 INFO:tasks.workunit.client.0.smithi042.stdout:3/594: creat d2/d95/fd0 x:0 0 0 2024-09-16T23:49:01.363 INFO:tasks.workunit.client.0.smithi042.stdout:7/566: fsync d3/d98/da0/d32/d6b/f83 0 2024-09-16T23:49:01.364 INFO:tasks.workunit.client.1.smithi043.stdout:9/609: mkdir d4/d1c/d6b/d67/dbf 0 2024-09-16T23:49:01.365 INFO:tasks.workunit.client.0.smithi042.stdout:6/453: fsync d8/d70/d81/dd/d1b/d34/f76 0 2024-09-16T23:49:01.366 INFO:tasks.workunit.client.1.smithi043.stdout:4/705: mkdir d8/d25/dcd/dd8 0 2024-09-16T23:49:01.367 INFO:tasks.workunit.client.0.smithi042.stdout:3/595: creat d2/d1b/d1d/d71/fd1 x:0 0 0 2024-09-16T23:49:01.367 INFO:tasks.workunit.client.0.smithi042.stdout:2/499: dread d0/da/d17/f59 [4194304,4194304] 0 2024-09-16T23:49:01.367 INFO:tasks.workunit.client.1.smithi043.stdout:0/665: getdents d0/d2/df/d5a/d9c 0 2024-09-16T23:49:01.367 INFO:tasks.workunit.client.0.smithi042.stdout:7/567: creat d3/d1f/d49/d78/dbe/fc7 x:0 0 0 2024-09-16T23:49:01.368 INFO:tasks.workunit.client.1.smithi043.stdout:4/706: read d8/d15/d20/d2b/d23/d27/d4b/fa4 [1032176,103830] 0 2024-09-16T23:49:01.368 INFO:tasks.workunit.client.0.smithi042.stdout:6/454: chown d8/d70/d81/dd/d1b/d34/d39/d5e/f7c 32755 1 2024-09-16T23:49:01.369 INFO:tasks.workunit.client.1.smithi043.stdout:9/610: creat d4/d21/d33/d95/fc0 x:0 0 0 2024-09-16T23:49:01.382 INFO:tasks.workunit.client.1.smithi043.stdout:7/615: sync 2024-09-16T23:49:01.382 INFO:tasks.workunit.client.1.smithi043.stdout:9/611: fdatasync d4/f48 0 2024-09-16T23:49:01.382 INFO:tasks.workunit.client.1.smithi043.stdout:4/707: write d8/d15/d9b/d8b/fa8 [1572570,17406] 0 2024-09-16T23:49:01.383 INFO:tasks.workunit.client.1.smithi043.stdout:9/612: fsync d4/d21/fbc 0 2024-09-16T23:49:01.384 INFO:tasks.workunit.client.1.smithi043.stdout:0/666: unlink d0/de/lc3 0 2024-09-16T23:49:01.385 INFO:tasks.workunit.client.1.smithi043.stdout:7/616: mknod d4/db/d11/d15/cca 0 2024-09-16T23:49:01.386 INFO:tasks.workunit.client.1.smithi043.stdout:4/708: creat d8/d25/dcd/fd9 x:0 0 0 2024-09-16T23:49:01.388 INFO:tasks.workunit.client.1.smithi043.stdout:0/667: mkdir d0/d2/df/d1a/d49/db2/de1 0 2024-09-16T23:49:01.390 INFO:tasks.workunit.client.1.smithi043.stdout:9/613: dwrite d4/d1c/d25/d2b/d4c/f72 [4194304,4194304] 0 2024-09-16T23:49:01.392 INFO:tasks.workunit.client.1.smithi043.stdout:0/668: rmdir d0/d2/df/dbb 39 2024-09-16T23:49:01.396 INFO:tasks.workunit.client.1.smithi043.stdout:4/709: getdents d8/d15/d20/d2b/d54/db3 0 2024-09-16T23:49:01.397 INFO:tasks.workunit.client.1.smithi043.stdout:9/614: truncate d4/d1c/d4f/d8c/f66 5049627 0 2024-09-16T23:49:01.397 INFO:tasks.workunit.client.1.smithi043.stdout:0/669: mknod d0/d2/df/d1a/ce2 0 2024-09-16T23:49:01.399 INFO:tasks.workunit.client.1.smithi043.stdout:7/617: dwrite d4/f19 [0,4194304] 0 2024-09-16T23:49:01.403 INFO:tasks.workunit.client.1.smithi043.stdout:7/618: chown d4/db/d50/f87 615101 1 2024-09-16T23:49:01.406 INFO:tasks.workunit.client.1.smithi043.stdout:4/710: dwrite d8/d15/d20/d2b/d22/fc2 [0,4194304] 0 2024-09-16T23:49:01.408 INFO:tasks.workunit.client.1.smithi043.stdout:7/619: mkdir d4/db/d11/d21/d31/dcb 0 2024-09-16T23:49:01.410 INFO:tasks.workunit.client.1.smithi043.stdout:4/711: symlink d8/d15/d20/d2b/d54/db3/d4c/lda 0 2024-09-16T23:49:01.415 INFO:tasks.workunit.client.1.smithi043.stdout:4/712: rename d8/d15/d20/d2b/c52 to d8/d15/d20/d2b/d54/db3/d4c/cdb 0 2024-09-16T23:49:01.416 INFO:tasks.workunit.client.1.smithi043.stdout:0/670: dread d0/d2/df/d1a/d46/d39/f6a [0,4194304] 0 2024-09-16T23:49:01.417 INFO:tasks.workunit.client.1.smithi043.stdout:0/671: chown d0/d2/df/f59 0 1 2024-09-16T23:49:01.418 INFO:tasks.workunit.client.1.smithi043.stdout:4/713: truncate d8/d25/d39/d41/f60 589064 0 2024-09-16T23:49:01.419 INFO:tasks.workunit.client.1.smithi043.stdout:4/714: write d8/f1d [2049967,11474] 0 2024-09-16T23:49:01.420 INFO:tasks.workunit.client.1.smithi043.stdout:0/672: write d0/d2/df/f10 [15722069,75437] 0 2024-09-16T23:49:01.422 INFO:tasks.workunit.client.1.smithi043.stdout:4/715: symlink d8/d15/d20/d2b/d23/ldc 0 2024-09-16T23:49:01.422 INFO:tasks.workunit.client.1.smithi043.stdout:0/673: dread - d0/d2/df/d1a/d46/d39/d6e/fbc zero size 2024-09-16T23:49:01.424 INFO:tasks.workunit.client.1.smithi043.stdout:0/674: fsync d0/d2/df/d1a/d46/d39/f45 0 2024-09-16T23:49:01.427 INFO:tasks.workunit.client.1.smithi043.stdout:4/716: creat d8/d15/d20/d2b/d54/d55/d5e/fdd x:0 0 0 2024-09-16T23:49:01.429 INFO:tasks.workunit.client.1.smithi043.stdout:4/717: readlink d8/d15/d20/d2b/d54/db3/d4c/l8e 0 2024-09-16T23:49:01.429 INFO:tasks.workunit.client.1.smithi043.stdout:7/620: dread d4/db/d11/d15/d29/d7a/f86 [4194304,4194304] 0 2024-09-16T23:49:01.431 INFO:tasks.workunit.client.1.smithi043.stdout:0/675: dwrite d0/d2/df/d1a/d46/d39/d6e/f65 [0,4194304] 0 2024-09-16T23:49:01.431 INFO:tasks.workunit.client.1.smithi043.stdout:7/621: dread - d4/db/d43/d83/d85/fa1 zero size 2024-09-16T23:49:01.431 INFO:tasks.workunit.client.1.smithi043.stdout:4/718: mkdir d8/d15/d9b/d8b/dde 0 2024-09-16T23:49:01.433 INFO:tasks.workunit.client.1.smithi043.stdout:4/719: mknod d8/d15/d9b/cdf 0 2024-09-16T23:49:01.434 INFO:tasks.workunit.client.1.smithi043.stdout:7/622: creat d4/db/d11/d21/d31/d33/fcc x:0 0 0 2024-09-16T23:49:01.435 INFO:tasks.workunit.client.1.smithi043.stdout:4/720: write d8/d15/d20/d2b/d54/db3/d4c/f9c [421404,129907] 0 2024-09-16T23:49:01.439 INFO:tasks.workunit.client.1.smithi043.stdout:4/721: mknod d8/d15/d20/d2b/d23/d27/d4b/ce0 0 2024-09-16T23:49:01.440 INFO:tasks.workunit.client.1.smithi043.stdout:4/722: readlink d8/d15/d20/d2b/d22/l6c 0 2024-09-16T23:49:01.441 INFO:tasks.workunit.client.1.smithi043.stdout:4/723: chown d8/d15/d9b/dbf 233124 1 2024-09-16T23:49:01.444 INFO:tasks.workunit.client.1.smithi043.stdout:0/676: dread d0/d2/df/dbb/f9b [0,4194304] 0 2024-09-16T23:49:01.445 INFO:tasks.workunit.client.1.smithi043.stdout:7/623: truncate d4/db/d50/fa9 3322047 0 2024-09-16T23:49:01.447 INFO:tasks.workunit.client.1.smithi043.stdout:0/677: truncate d0/d2/df/d1a/d46/d39/d6e/fd5 109247 0 2024-09-16T23:49:01.447 INFO:tasks.workunit.client.1.smithi043.stdout:7/624: rmdir d4/d27/d41/d1b/d66/da3 39 2024-09-16T23:49:01.448 INFO:tasks.workunit.client.0.smithi042.stdout:3/596: creat d2/d1b/d1d/d30/da2/fd2 x:0 0 0 2024-09-16T23:49:01.450 INFO:tasks.workunit.client.1.smithi043.stdout:7/625: write d4/d27/d41/d1b/d66/da3/fb7 [351402,59752] 0 2024-09-16T23:49:01.452 INFO:tasks.workunit.client.1.smithi043.stdout:0/678: dread d0/d2/df/d1a/d46/d39/f6a [0,4194304] 0 2024-09-16T23:49:01.454 INFO:tasks.workunit.client.1.smithi043.stdout:7/626: mkdir d4/db/d11/d15/d29/d7a/dcd 0 2024-09-16T23:49:01.454 INFO:tasks.workunit.client.0.smithi042.stdout:2/500: dwrite d0/d5d/f79 [0,4194304] 0 2024-09-16T23:49:01.455 INFO:tasks.workunit.client.1.smithi043.stdout:7/627: dread - d4/db/d43/d83/d85/d8b/faf zero size 2024-09-16T23:49:01.456 INFO:tasks.workunit.client.0.smithi042.stdout:6/455: rename d8/d70/d81/dd/d1b/d34/f76 to d8/d70/d81/dd/d2c/d57/f94 0 2024-09-16T23:49:01.458 INFO:tasks.workunit.client.1.smithi043.stdout:0/679: symlink d0/d2/df/d1a/d46/d39/d83/le3 0 2024-09-16T23:49:01.458 INFO:tasks.workunit.client.0.smithi042.stdout:7/568: dwrite d3/d98/da0/d41/f96 [0,4194304] 0 2024-09-16T23:49:01.459 INFO:tasks.workunit.client.1.smithi043.stdout:4/724: dread d8/d15/d9b/d8b/f8f [0,4194304] 0 2024-09-16T23:49:01.461 INFO:tasks.workunit.client.0.smithi042.stdout:3/597: rename d2/d7/d61/d8e/d9d/fc1 to d2/d1b/dc8/fd3 0 2024-09-16T23:49:01.463 INFO:tasks.workunit.client.1.smithi043.stdout:0/680: rmdir d0/d2/df/dbb 39 2024-09-16T23:49:01.463 INFO:tasks.workunit.client.0.smithi042.stdout:7/569: mkdir d3/d1f/d49/d78/dbe/dc8 0 2024-09-16T23:49:01.464 INFO:tasks.workunit.client.1.smithi043.stdout:0/681: write d0/de/f2d [4964567,88672] 0 2024-09-16T23:49:01.466 INFO:tasks.workunit.client.0.smithi042.stdout:6/456: rename f7 to d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f95 0 2024-09-16T23:49:01.466 INFO:tasks.workunit.client.0.smithi042.stdout:3/598: dread d2/d9/faa [0,4194304] 0 2024-09-16T23:49:01.467 INFO:tasks.workunit.client.1.smithi043.stdout:4/725: mknod d8/d15/d20/ce1 0 2024-09-16T23:49:01.467 INFO:tasks.workunit.client.0.smithi042.stdout:6/457: stat d8/d70/d81/dd/d1b/d34/f64 0 2024-09-16T23:49:01.472 INFO:tasks.workunit.client.0.smithi042.stdout:6/458: rmdir d8/d70/d81/dd/d1b/d50 39 2024-09-16T23:49:01.472 INFO:tasks.workunit.client.1.smithi043.stdout:7/628: write d4/db/d11/d21/d31/d33/f55 [713407,117756] 0 2024-09-16T23:49:01.473 INFO:tasks.workunit.client.0.smithi042.stdout:7/570: rename d3/d98/da0/d32/d6b/d64/d92/fba to d3/d1f/d8e/db5/dc4/fc9 0 2024-09-16T23:49:01.473 INFO:tasks.workunit.client.0.smithi042.stdout:2/501: write d0/d30/d43/f62 [2597823,20087] 0 2024-09-16T23:49:01.474 INFO:tasks.workunit.client.1.smithi043.stdout:4/726: mknod d8/d25/dcd/dd8/ce2 0 2024-09-16T23:49:01.475 INFO:tasks.workunit.client.1.smithi043.stdout:0/682: read d0/d2/df/d1a/d46/d39/d6e/f55 [44565,105170] 0 2024-09-16T23:49:01.476 INFO:tasks.workunit.client.0.smithi042.stdout:7/571: truncate d3/d98/da0/d24/fbd 46899 0 2024-09-16T23:49:01.477 INFO:tasks.workunit.client.0.smithi042.stdout:7/572: dread - d3/d1f/d49/d78/fa2 zero size 2024-09-16T23:49:01.480 INFO:tasks.workunit.client.0.smithi042.stdout:7/573: creat d3/d1f/d49/d78/dbe/fca x:0 0 0 2024-09-16T23:49:01.481 INFO:tasks.workunit.client.1.smithi043.stdout:4/727: truncate d8/d15/d20/d2b/f10 3530487 0 2024-09-16T23:49:01.482 INFO:tasks.workunit.client.0.smithi042.stdout:3/599: getdents d2/d1b/d1d/d30/da2 0 2024-09-16T23:49:01.482 INFO:tasks.workunit.client.0.smithi042.stdout:2/502: mknod d0/da/d17/d3e/d86/cb0 0 2024-09-16T23:49:01.485 INFO:tasks.workunit.client.1.smithi043.stdout:0/683: getdents d0/d2 0 2024-09-16T23:49:01.489 INFO:tasks.workunit.client.1.smithi043.stdout:7/629: dread d4/f32 [0,4194304] 0 2024-09-16T23:49:01.490 INFO:tasks.workunit.client.0.smithi042.stdout:3/600: write d2/d1b/d1d/f1a [674620,104364] 0 2024-09-16T23:49:01.490 INFO:tasks.workunit.client.0.smithi042.stdout:6/459: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 [0,4194304] 0 2024-09-16T23:49:01.492 INFO:tasks.workunit.client.1.smithi043.stdout:0/684: rename d0/d2/df/d1a/d46/f94 to d0/d2/df/d5a/fe4 0 2024-09-16T23:49:01.494 INFO:tasks.workunit.client.0.smithi042.stdout:6/460: read d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 [2493671,127343] 0 2024-09-16T23:49:01.499 INFO:tasks.workunit.client.0.smithi042.stdout:7/574: dread d3/d98/da0/d41/f45 [4194304,4194304] 0 2024-09-16T23:49:01.500 INFO:tasks.workunit.client.0.smithi042.stdout:6/461: dread d8/d70/d81/dd/d1b/d34/d39/d5e/f47 [0,4194304] 0 2024-09-16T23:49:01.502 INFO:tasks.workunit.client.0.smithi042.stdout:7/575: chown d3/d98/da0/l77 4 1 2024-09-16T23:49:01.502 INFO:tasks.workunit.client.0.smithi042.stdout:3/601: dwrite d2/d1b/d1d/d34/f5d [0,4194304] 0 2024-09-16T23:49:01.505 INFO:tasks.workunit.client.0.smithi042.stdout:7/576: truncate d3/f84 1370426 0 2024-09-16T23:49:01.505 INFO:tasks.workunit.client.1.smithi043.stdout:0/685: write d0/d2/df/d1a/f30 [2356371,87423] 0 2024-09-16T23:49:01.507 INFO:tasks.workunit.client.0.smithi042.stdout:6/462: mkdir d8/d70/d81/dd/d1b/d50/d96 0 2024-09-16T23:49:01.509 INFO:tasks.workunit.client.1.smithi043.stdout:7/630: dwrite d4/db/f5e [0,4194304] 0 2024-09-16T23:49:01.511 INFO:tasks.workunit.client.0.smithi042.stdout:3/602: dwrite d2/d1b/d1d/d34/f5d [0,4194304] 0 2024-09-16T23:49:01.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/463: mkdir d8/d70/d81/dd/d1b/d67/d97 0 2024-09-16T23:49:01.514 INFO:tasks.workunit.client.1.smithi043.stdout:7/631: rmdir d4/d27/d41 39 2024-09-16T23:49:01.514 INFO:tasks.workunit.client.0.smithi042.stdout:7/577: dwrite d3/d98/da0/d41/f96 [0,4194304] 0 2024-09-16T23:49:01.515 INFO:tasks.workunit.client.1.smithi043.stdout:0/686: dwrite d0/f4d [0,4194304] 0 2024-09-16T23:49:01.518 INFO:tasks.workunit.client.1.smithi043.stdout:7/632: fdatasync d4/db/f18 0 2024-09-16T23:49:01.519 INFO:tasks.workunit.client.0.smithi042.stdout:3/603: creat d2/d1b/d1d/d34/d62/fd4 x:0 0 0 2024-09-16T23:49:01.519 INFO:tasks.workunit.client.1.smithi043.stdout:7/633: fdatasync d4/db/fb2 0 2024-09-16T23:49:01.522 INFO:tasks.workunit.client.0.smithi042.stdout:3/604: stat d2/l56 0 2024-09-16T23:49:01.522 INFO:tasks.workunit.client.0.smithi042.stdout:6/464: rename d8/d70/d81/f20 to d8/d70/d81/dd/d1b/d50/f98 0 2024-09-16T23:49:01.524 INFO:tasks.workunit.client.0.smithi042.stdout:7/578: dread d3/d98/f80 [0,4194304] 0 2024-09-16T23:49:01.525 INFO:tasks.workunit.client.0.smithi042.stdout:3/605: write d2/d1b/d1d/d34/d50/f6c [4158926,127385] 0 2024-09-16T23:49:01.527 INFO:tasks.workunit.client.0.smithi042.stdout:7/579: dread d3/d98/da0/f6c [0,4194304] 0 2024-09-16T23:49:01.532 INFO:tasks.workunit.client.0.smithi042.stdout:7/580: rmdir d3/d98/da0/d24/d28/d9b 39 2024-09-16T23:49:01.532 INFO:tasks.workunit.client.0.smithi042.stdout:7/581: readlink d3/l13 0 2024-09-16T23:49:01.532 INFO:tasks.workunit.client.0.smithi042.stdout:7/582: readlink d3/l72 0 2024-09-16T23:49:01.534 INFO:tasks.workunit.client.1.smithi043.stdout:7/634: dread d4/db/d28/f4d [0,4194304] 0 2024-09-16T23:49:01.535 INFO:tasks.workunit.client.0.smithi042.stdout:3/606: link d2/d7/d61/lb5 d2/d1b/ld5 0 2024-09-16T23:49:01.535 INFO:tasks.workunit.client.0.smithi042.stdout:7/583: truncate d3/d1f/d49/d76/f25 4483362 0 2024-09-16T23:49:01.538 INFO:tasks.workunit.client.0.smithi042.stdout:7/584: creat d3/d1f/d8e/fcb x:0 0 0 2024-09-16T23:49:01.540 INFO:tasks.workunit.client.0.smithi042.stdout:3/607: dread d2/d7/d61/d8e/f5b [0,4194304] 0 2024-09-16T23:49:01.541 INFO:tasks.workunit.client.0.smithi042.stdout:6/465: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f6e 1 1 2024-09-16T23:49:01.541 INFO:tasks.workunit.client.0.smithi042.stdout:7/585: creat d3/d1f/d49/d78/dbe/dc8/fcc x:0 0 0 2024-09-16T23:49:01.543 INFO:tasks.workunit.client.0.smithi042.stdout:3/608: mknod d2/d1b/d1d/d34/d62/cd6 0 2024-09-16T23:49:01.543 INFO:tasks.workunit.client.1.smithi043.stdout:0/687: dwrite d0/d2/df/d1a/d49/d80/d88/fa4 [0,4194304] 0 2024-09-16T23:49:01.545 INFO:tasks.workunit.client.0.smithi042.stdout:3/609: fsync d2/d1b/d1d/f85 0 2024-09-16T23:49:01.547 INFO:tasks.workunit.client.1.smithi043.stdout:0/688: creat d0/d2/df/da7/fe5 x:0 0 0 2024-09-16T23:49:01.549 INFO:tasks.workunit.client.1.smithi043.stdout:0/689: readlink d0/d2/df/d1a/d49/db2/ld7 0 2024-09-16T23:49:01.552 INFO:tasks.workunit.client.0.smithi042.stdout:7/586: dwrite d3/d98/da0/d32/d6b/fad [0,4194304] 0 2024-09-16T23:49:01.552 INFO:tasks.workunit.client.0.smithi042.stdout:6/466: dwrite d8/d70/d81/dd/f19 [0,4194304] 0 2024-09-16T23:49:01.553 INFO:tasks.workunit.client.1.smithi043.stdout:0/690: creat d0/d2/df/d1a/d46/fe6 x:0 0 0 2024-09-16T23:49:01.553 INFO:tasks.workunit.client.0.smithi042.stdout:7/587: chown d3/d98/da0/d32/d6b/d5b/db8 1 1 2024-09-16T23:49:01.554 INFO:tasks.workunit.client.0.smithi042.stdout:3/610: rename d2/d95/cbe to d2/d1b/d1d/d34/d50/d93/cd7 0 2024-09-16T23:49:01.556 INFO:tasks.workunit.client.0.smithi042.stdout:7/588: chown d3/d1f/d49/d76/d19/l4b 4847986 1 2024-09-16T23:49:01.556 INFO:tasks.workunit.client.0.smithi042.stdout:3/611: mkdir d2/d95/dd8 0 2024-09-16T23:49:01.557 INFO:tasks.workunit.client.0.smithi042.stdout:3/612: readlink d2/d1b/dc8/lcd 0 2024-09-16T23:49:01.558 INFO:tasks.workunit.client.1.smithi043.stdout:0/691: mknod d0/d2/df/d1a/d46/d39/d83/dcb/ce7 0 2024-09-16T23:49:01.559 INFO:tasks.workunit.client.0.smithi042.stdout:6/467: mkdir d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d99 0 2024-09-16T23:49:01.559 INFO:tasks.workunit.client.1.smithi043.stdout:0/692: chown d0/d2/df/d1a/d46/ca6 0 1 2024-09-16T23:49:01.565 INFO:tasks.workunit.client.1.smithi043.stdout:0/693: mkdir d0/d2/df/d1a/de8 0 2024-09-16T23:49:01.565 INFO:tasks.workunit.client.0.smithi042.stdout:7/589: truncate d3/d98/da0/d41/f45 2837280 0 2024-09-16T23:49:01.566 INFO:tasks.workunit.client.0.smithi042.stdout:3/613: creat d2/d1b/d1d/fd9 x:0 0 0 2024-09-16T23:49:01.567 INFO:tasks.workunit.client.0.smithi042.stdout:7/590: rename d3/d1f/d8e to d3/d98/da0/d32/dcd 0 2024-09-16T23:49:01.570 INFO:tasks.workunit.client.0.smithi042.stdout:6/468: getdents d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72 0 2024-09-16T23:49:01.573 INFO:tasks.workunit.client.1.smithi043.stdout:0/694: write d0/d2/df/d1a/fd6 [325370,68326] 0 2024-09-16T23:49:01.573 INFO:tasks.workunit.client.1.smithi043.stdout:0/695: chown d0/de/l6f 119827 1 2024-09-16T23:49:01.576 INFO:tasks.workunit.client.0.smithi042.stdout:6/469: unlink d8/d70/d81/f12 0 2024-09-16T23:49:01.577 INFO:tasks.workunit.client.0.smithi042.stdout:7/591: dwrite d3/d98/da0/d24/d28/d9b/fa9 [0,4194304] 0 2024-09-16T23:49:01.577 INFO:tasks.workunit.client.1.smithi043.stdout:0/696: creat d0/d2/df/d1a/d49/db2/de1/fe9 x:0 0 0 2024-09-16T23:49:01.578 INFO:tasks.workunit.client.0.smithi042.stdout:7/592: readlink d3/l27 0 2024-09-16T23:49:01.580 INFO:tasks.workunit.client.0.smithi042.stdout:7/593: rmdir d3/d1f/d49/d76/d19 39 2024-09-16T23:49:01.581 INFO:tasks.workunit.client.0.smithi042.stdout:7/594: dread - d3/d98/f71 zero size 2024-09-16T23:49:01.584 INFO:tasks.workunit.client.1.smithi043.stdout:0/697: dwrite d0/d2/df/d1a/d46/d39/d6e/f65 [4194304,4194304] 0 2024-09-16T23:49:01.585 INFO:tasks.workunit.client.0.smithi042.stdout:7/595: dread d3/d98/f80 [0,4194304] 0 2024-09-16T23:49:01.586 INFO:tasks.workunit.client.0.smithi042.stdout:7/596: chown d3/d98/da0/d32/f4a 7875 1 2024-09-16T23:49:01.588 INFO:tasks.workunit.client.1.smithi043.stdout:0/698: write d0/d2/df/dbb/f50 [4987245,31266] 0 2024-09-16T23:49:01.592 INFO:tasks.workunit.client.0.smithi042.stdout:7/597: link d3/d98/da0/d32/d6b/d64/f94 d3/d98/fce 0 2024-09-16T23:49:01.593 INFO:tasks.workunit.client.1.smithi043.stdout:0/699: dread d0/d2/df/d1a/d46/d39/f6a [0,4194304] 0 2024-09-16T23:49:01.595 INFO:tasks.workunit.client.1.smithi043.stdout:0/700: mknod d0/d2/df/d1a/cea 0 2024-09-16T23:49:01.596 INFO:tasks.workunit.client.0.smithi042.stdout:6/470: dread d8/d70/d81/dd/d1b/f4b [0,4194304] 0 2024-09-16T23:49:01.598 INFO:tasks.workunit.client.0.smithi042.stdout:7/598: dwrite d3/d1f/d49/f8d [0,4194304] 0 2024-09-16T23:49:01.599 INFO:tasks.workunit.client.1.smithi043.stdout:0/701: creat d0/d2/df/d1a/d46/d39/d83/feb x:0 0 0 2024-09-16T23:49:01.599 INFO:tasks.workunit.client.0.smithi042.stdout:7/599: chown d3/d3f 1715 1 2024-09-16T23:49:01.601 INFO:tasks.workunit.client.0.smithi042.stdout:7/600: fsync d3/f9c 0 2024-09-16T23:49:01.601 INFO:tasks.workunit.client.0.smithi042.stdout:6/471: creat d8/d70/d81/dd/d1b/d67/d97/f9a x:0 0 0 2024-09-16T23:49:01.604 INFO:tasks.workunit.client.1.smithi043.stdout:0/702: rename d0/d2/df/d1a/d9a to d0/d2/df/d1a/de8/dec 0 2024-09-16T23:49:01.606 INFO:tasks.workunit.client.1.smithi043.stdout:5/821: sync 2024-09-16T23:49:01.607 INFO:tasks.workunit.client.1.smithi043.stdout:5/822: rmdir d3/dce 39 2024-09-16T23:49:01.609 INFO:tasks.workunit.client.1.smithi043.stdout:5/823: write d3/db/d19/d3e/ff1 [304993,105362] 0 2024-09-16T23:49:01.612 INFO:tasks.workunit.client.1.smithi043.stdout:5/824: write d3/d9/d22/d4a/d4c/fe3 [53285,46185] 0 2024-09-16T23:49:01.613 INFO:tasks.workunit.client.1.smithi043.stdout:0/703: dwrite d0/f41 [0,4194304] 0 2024-09-16T23:49:01.613 INFO:tasks.workunit.client.0.smithi042.stdout:6/472: write d8/d70/d81/dd/d2c/d57/f94 [258094,43510] 0 2024-09-16T23:49:01.619 INFO:tasks.workunit.client.0.smithi042.stdout:7/601: dread d3/d1f/d49/f97 [0,4194304] 0 2024-09-16T23:49:01.619 INFO:tasks.workunit.client.1.smithi043.stdout:0/704: getdents d0/d2/df/d1a/d49 0 2024-09-16T23:49:01.621 INFO:tasks.workunit.client.0.smithi042.stdout:6/473: dwrite d8/d38/f74 [0,4194304] 0 2024-09-16T23:49:01.623 INFO:tasks.workunit.client.0.smithi042.stdout:6/474: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/c6c 19491 1 2024-09-16T23:49:01.625 INFO:tasks.workunit.client.0.smithi042.stdout:7/602: rename d3/d98/da0/d32/d6b/d5b/dae to d3/d98/da0/d32/dcd/dcf 0 2024-09-16T23:49:01.627 INFO:tasks.workunit.client.0.smithi042.stdout:6/475: creat d8/d70/d84/f9b x:0 0 0 2024-09-16T23:49:01.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/603: read d3/d98/f8c [2340480,27121] 0 2024-09-16T23:49:01.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/604: chown d3/f5f 264079539 1 2024-09-16T23:49:01.631 INFO:tasks.workunit.client.0.smithi042.stdout:7/605: mknod d3/d1f/d4f/cd0 0 2024-09-16T23:49:01.631 INFO:tasks.workunit.client.0.smithi042.stdout:7/606: chown d3/f30 1953324924 1 2024-09-16T23:49:01.632 INFO:tasks.workunit.client.0.smithi042.stdout:6/476: dread d8/d70/d81/dd/d1b/d50/f98 [0,4194304] 0 2024-09-16T23:49:01.634 INFO:tasks.workunit.client.0.smithi042.stdout:7/607: creat d3/fd1 x:0 0 0 2024-09-16T23:49:01.635 INFO:tasks.workunit.client.0.smithi042.stdout:6/477: unlink d8/d70/d81/dd/d1b/d34/d39/d5e/f7a 0 2024-09-16T23:49:01.637 INFO:tasks.workunit.client.0.smithi042.stdout:7/608: creat d3/d98/da0/d32/d6b/fd2 x:0 0 0 2024-09-16T23:49:01.637 INFO:tasks.workunit.client.0.smithi042.stdout:6/478: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/c9c 0 2024-09-16T23:49:01.638 INFO:tasks.workunit.client.0.smithi042.stdout:6/479: chown d8/d70/d81/dd/d2c/f31 43325 1 2024-09-16T23:49:01.639 INFO:tasks.workunit.client.0.smithi042.stdout:6/480: chown d8/d70/d81/dd/d1b/c8e 195230387 1 2024-09-16T23:49:01.640 INFO:tasks.workunit.client.0.smithi042.stdout:6/481: chown f6 603925 1 2024-09-16T23:49:01.642 INFO:tasks.workunit.client.0.smithi042.stdout:6/482: dread - d8/d70/d81/dd/d2c/d57/f60 zero size 2024-09-16T23:49:01.643 INFO:tasks.workunit.client.0.smithi042.stdout:7/609: dwrite d3/d98/da0/d41/f8b [0,4194304] 0 2024-09-16T23:49:01.646 INFO:tasks.workunit.client.0.smithi042.stdout:7/610: write d3/d98/da0/d24/f59 [2065314,81617] 0 2024-09-16T23:49:01.657 INFO:tasks.workunit.client.0.smithi042.stdout:7/611: dread d3/d1f/d4f/f5d [0,4194304] 0 2024-09-16T23:49:01.662 INFO:tasks.workunit.client.0.smithi042.stdout:7/612: write d3/fe [9128058,21101] 0 2024-09-16T23:49:01.663 INFO:tasks.workunit.client.0.smithi042.stdout:7/613: write d3/f8 [595379,13341] 0 2024-09-16T23:49:01.666 INFO:tasks.workunit.client.0.smithi042.stdout:7/614: mknod d3/cd3 0 2024-09-16T23:49:01.671 INFO:tasks.workunit.client.0.smithi042.stdout:7/615: link d3/d1f/d49/d76/d19/l46 d3/d98/da0/d32/d6b/d5b/ld4 0 2024-09-16T23:49:01.671 INFO:tasks.workunit.client.0.smithi042.stdout:7/616: stat d3/d98/da0/l20 0 2024-09-16T23:49:01.674 INFO:tasks.workunit.client.0.smithi042.stdout:7/617: creat d3/d1f/d49/d76/fd5 x:0 0 0 2024-09-16T23:49:01.677 INFO:tasks.workunit.client.0.smithi042.stdout:7/618: truncate d3/d98/da0/d32/dcd/db5/dc4/fc9 929369 0 2024-09-16T23:49:01.685 INFO:tasks.workunit.client.1.smithi043.stdout:8/621: sync 2024-09-16T23:49:01.686 INFO:tasks.workunit.client.0.smithi042.stdout:4/694: sync 2024-09-16T23:49:01.686 INFO:tasks.workunit.client.0.smithi042.stdout:0/388: sync 2024-09-16T23:49:01.688 INFO:tasks.workunit.client.0.smithi042.stdout:4/695: mknod de/d42/d85/cf0 0 2024-09-16T23:49:01.691 INFO:tasks.workunit.client.0.smithi042.stdout:4/696: creat de/d2e/d37/d3e/ff1 x:0 0 0 2024-09-16T23:49:01.693 INFO:tasks.workunit.client.1.smithi043.stdout:8/622: fsync de/fb0 0 2024-09-16T23:49:01.694 INFO:tasks.workunit.client.1.smithi043.stdout:8/623: stat de/d18/d1f/d25/f6e 0 2024-09-16T23:49:01.696 INFO:tasks.workunit.client.1.smithi043.stdout:0/705: dread d0/d2/df/dbb/f50 [4194304,4194304] 0 2024-09-16T23:49:01.696 INFO:tasks.workunit.client.0.smithi042.stdout:4/697: truncate de/d2e/d37/d3e/f7f 2007534 0 2024-09-16T23:49:01.697 INFO:tasks.workunit.client.0.smithi042.stdout:4/698: chown de/d42/d85/db3/de7 15659 1 2024-09-16T23:49:01.698 INFO:tasks.workunit.client.1.smithi043.stdout:5/825: dwrite d3/db/d19/d3e/f2d [4194304,4194304] 0 2024-09-16T23:49:01.698 INFO:tasks.workunit.client.0.smithi042.stdout:4/699: truncate de/d42/fee 625876 0 2024-09-16T23:49:01.700 INFO:tasks.workunit.client.0.smithi042.stdout:0/389: dread d0/df/d13/f34 [0,4194304] 0 2024-09-16T23:49:01.700 INFO:tasks.workunit.client.0.smithi042.stdout:4/700: fsync fa 0 2024-09-16T23:49:01.700 INFO:tasks.workunit.client.1.smithi043.stdout:0/706: creat d0/d2/df/d1a/de8/fed x:0 0 0 2024-09-16T23:49:01.701 INFO:tasks.workunit.client.1.smithi043.stdout:0/707: fsync d0/f4d 0 2024-09-16T23:49:01.701 INFO:tasks.workunit.client.0.smithi042.stdout:4/701: chown de/d1a/l43 27116727 1 2024-09-16T23:49:01.705 INFO:tasks.workunit.client.0.smithi042.stdout:0/390: rename d0/l6 to d0/df/d23/d33/d51/d5b/l77 0 2024-09-16T23:49:01.705 INFO:tasks.workunit.client.1.smithi043.stdout:8/624: dwrite de/d18/f4c [0,4194304] 0 2024-09-16T23:49:01.709 INFO:tasks.workunit.client.0.smithi042.stdout:0/391: unlink d0/df/f14 0 2024-09-16T23:49:01.711 INFO:tasks.workunit.client.0.smithi042.stdout:4/702: mkdir de/d2e/d37/df2 0 2024-09-16T23:49:01.713 INFO:tasks.workunit.client.0.smithi042.stdout:4/703: write de/d42/d5d/f74 [5869,41993] 0 2024-09-16T23:49:01.714 INFO:tasks.workunit.client.1.smithi043.stdout:0/708: dwrite d0/d2/df/d1a/d3b/fda [0,4194304] 0 2024-09-16T23:49:01.716 INFO:tasks.workunit.client.0.smithi042.stdout:4/704: creat de/d42/ff3 x:0 0 0 2024-09-16T23:49:01.717 INFO:tasks.workunit.client.0.smithi042.stdout:0/392: dwrite d0/df/d13/f74 [0,4194304] 0 2024-09-16T23:49:01.718 INFO:tasks.workunit.client.1.smithi043.stdout:5/826: dread d3/d9/d22/d60/f82 [0,4194304] 0 2024-09-16T23:49:01.719 INFO:tasks.workunit.client.0.smithi042.stdout:0/393: readlink d0/df/d13/d2d/d35/l36 0 2024-09-16T23:49:01.719 INFO:tasks.workunit.client.1.smithi043.stdout:0/709: mkdir d0/d2/df/d1a/d3b/dee 0 2024-09-16T23:49:01.722 INFO:tasks.workunit.client.1.smithi043.stdout:5/827: write d3/d9/d22/d4a/d4c/d7a/f114 [22398,113762] 0 2024-09-16T23:49:01.724 INFO:tasks.workunit.client.1.smithi043.stdout:5/828: write d3/d79/d91/daf/fc2 [3831182,3634] 0 2024-09-16T23:49:01.729 INFO:tasks.workunit.client.1.smithi043.stdout:5/829: rename d3/db/d19/d3e/d28/dcb/f111 to d3/d1b/f119 0 2024-09-16T23:49:01.730 INFO:tasks.workunit.client.0.smithi042.stdout:0/394: rename d0/df/d13/f63 to d0/f78 0 2024-09-16T23:49:01.732 INFO:tasks.workunit.client.0.smithi042.stdout:0/395: mkdir d0/df/d23/d33/d51/d79 0 2024-09-16T23:49:01.733 INFO:tasks.workunit.client.1.smithi043.stdout:0/710: truncate d0/d2/df/d1a/d46/d39/d6e/f65 1118520 0 2024-09-16T23:49:01.734 INFO:tasks.workunit.client.0.smithi042.stdout:4/705: dwrite de/f10 [4194304,4194304] 0 2024-09-16T23:49:01.735 INFO:tasks.workunit.client.0.smithi042.stdout:0/396: creat d0/d48/f7a x:0 0 0 2024-09-16T23:49:01.736 INFO:tasks.workunit.client.1.smithi043.stdout:0/711: truncate d0/fa3 323020 0 2024-09-16T23:49:01.739 INFO:tasks.workunit.client.1.smithi043.stdout:0/712: mknod d0/d2/cef 0 2024-09-16T23:49:01.748 INFO:tasks.workunit.client.1.smithi043.stdout:5/830: dread d3/db/d19/d3e/f35 [4194304,4194304] 0 2024-09-16T23:49:01.755 INFO:tasks.workunit.client.1.smithi043.stdout:0/713: truncate d0/d2/df/d1a/d46/d39/f62 126501 0 2024-09-16T23:49:01.759 INFO:tasks.workunit.client.1.smithi043.stdout:5/831: dwrite d3/d9/d22/d9f/fc6 [0,4194304] 0 2024-09-16T23:49:01.765 INFO:tasks.workunit.client.1.smithi043.stdout:5/832: dread d3/d9/d22/d9f/fc6 [0,4194304] 0 2024-09-16T23:49:01.767 INFO:tasks.workunit.client.0.smithi042.stdout:0/397: symlink d0/df/d62/l7b 0 2024-09-16T23:49:01.767 INFO:tasks.workunit.client.0.smithi042.stdout:4/706: mknod de/d42/d5d/d8f/d9f/cf4 0 2024-09-16T23:49:01.767 INFO:tasks.workunit.client.1.smithi043.stdout:0/714: dwrite d0/d2/df/d1a/d46/d92/fc9 [0,4194304] 0 2024-09-16T23:49:01.769 INFO:tasks.workunit.client.0.smithi042.stdout:4/707: write de/d1a/f46 [11577135,33361] 0 2024-09-16T23:49:01.772 INFO:tasks.workunit.client.1.smithi043.stdout:0/715: mkdir d0/d2/df/d1a/d3b/dee/df0 0 2024-09-16T23:49:01.772 INFO:tasks.workunit.client.0.smithi042.stdout:0/398: creat d0/d15/f7c x:0 0 0 2024-09-16T23:49:01.773 INFO:tasks.workunit.client.0.smithi042.stdout:4/708: creat de/d2e/d37/dc3/ff5 x:0 0 0 2024-09-16T23:49:01.773 INFO:tasks.workunit.client.1.smithi043.stdout:5/833: creat d3/d9/d6f/d7c/d80/f11a x:0 0 0 2024-09-16T23:49:01.774 INFO:tasks.workunit.client.0.smithi042.stdout:0/399: write d0/df/f18 [3458218,110674] 0 2024-09-16T23:49:01.774 INFO:tasks.workunit.client.0.smithi042.stdout:4/709: fdatasync de/d2e/d37/f65 0 2024-09-16T23:49:01.775 INFO:tasks.workunit.client.0.smithi042.stdout:0/400: fdatasync d0/d48/f7a 0 2024-09-16T23:49:01.775 INFO:tasks.workunit.client.1.smithi043.stdout:0/716: unlink d0/f24 0 2024-09-16T23:49:01.776 INFO:tasks.workunit.client.0.smithi042.stdout:4/710: dread - de/d42/d68/f8d zero size 2024-09-16T23:49:01.777 INFO:tasks.workunit.client.0.smithi042.stdout:0/401: chown d0/df/d13/f4e 4168406 1 2024-09-16T23:49:01.777 INFO:tasks.workunit.client.1.smithi043.stdout:5/834: mknod d3/db/c11b 0 2024-09-16T23:49:01.778 INFO:tasks.workunit.client.0.smithi042.stdout:0/402: dread d0/df/d23/f55 [0,4194304] 0 2024-09-16T23:49:01.779 INFO:tasks.workunit.client.0.smithi042.stdout:4/711: mknod de/d12/d7d/cf6 0 2024-09-16T23:49:01.779 INFO:tasks.workunit.client.1.smithi043.stdout:0/717: creat d0/d2/df/dbb/ff1 x:0 0 0 2024-09-16T23:49:01.779 INFO:tasks.workunit.client.0.smithi042.stdout:0/403: fdatasync d0/df/f18 0 2024-09-16T23:49:01.781 INFO:tasks.workunit.client.0.smithi042.stdout:0/404: mknod d0/df/d23/d33/c7d 0 2024-09-16T23:49:01.782 INFO:tasks.workunit.client.1.smithi043.stdout:0/718: truncate d0/d2/df/d5a/f63 1424304 0 2024-09-16T23:49:01.782 INFO:tasks.workunit.client.0.smithi042.stdout:4/712: mknod de/d12/d7d/cf7 0 2024-09-16T23:49:01.782 INFO:tasks.workunit.client.1.smithi043.stdout:5/835: getdents d3/d9/d22/d4a/dac 0 2024-09-16T23:49:01.785 INFO:tasks.workunit.client.0.smithi042.stdout:0/405: rename d0/df/c1a to d0/df/d13/d49/d70/c7e 0 2024-09-16T23:49:01.787 INFO:tasks.workunit.client.0.smithi042.stdout:4/713: unlink de/d12/d7d/fa1 0 2024-09-16T23:49:01.787 INFO:tasks.workunit.client.0.smithi042.stdout:0/406: stat d0/d15/f3c 0 2024-09-16T23:49:01.789 INFO:tasks.workunit.client.1.smithi043.stdout:0/719: dwrite d0/d2/df/d1a/d49/d80/fcd [4194304,4194304] 0 2024-09-16T23:49:01.789 INFO:tasks.workunit.client.0.smithi042.stdout:4/714: creat de/d42/d99/d9e/ff8 x:0 0 0 2024-09-16T23:49:01.794 INFO:tasks.workunit.client.0.smithi042.stdout:4/715: rename de/d2e/d37/df2 to de/d1a/d70/df9 0 2024-09-16T23:49:01.795 INFO:tasks.workunit.client.1.smithi043.stdout:0/720: write d0/d2/f6d [135967,10485] 0 2024-09-16T23:49:01.797 INFO:tasks.workunit.client.0.smithi042.stdout:0/407: getdents d0/df/d13/d2d 0 2024-09-16T23:49:01.799 INFO:tasks.workunit.client.0.smithi042.stdout:4/716: mknod de/d12/d1b/db8/cfa 0 2024-09-16T23:49:01.799 INFO:tasks.workunit.client.0.smithi042.stdout:0/408: write d0/df/d23/d33/d56/f61 [874243,58886] 0 2024-09-16T23:49:01.800 INFO:tasks.workunit.client.0.smithi042.stdout:4/717: stat de/d42/d5d/d8f/dd6/lec 0 2024-09-16T23:49:01.802 INFO:tasks.workunit.client.0.smithi042.stdout:4/718: symlink de/lfb 0 2024-09-16T23:49:01.803 INFO:tasks.workunit.client.1.smithi043.stdout:0/721: readlink d0/d2/df/d1a/d46/l85 0 2024-09-16T23:49:01.804 INFO:tasks.workunit.client.0.smithi042.stdout:4/719: chown de/d42/d5d/d8f/dd6/ddc/le4 22012139 1 2024-09-16T23:49:01.804 INFO:tasks.workunit.client.1.smithi043.stdout:5/836: dread d3/d79/d91/daf/fc5 [0,4194304] 0 2024-09-16T23:49:01.805 INFO:tasks.workunit.client.0.smithi042.stdout:0/409: dread d0/f78 [0,4194304] 0 2024-09-16T23:49:01.806 INFO:tasks.workunit.client.0.smithi042.stdout:1/601: sync 2024-09-16T23:49:01.806 INFO:tasks.workunit.client.0.smithi042.stdout:5/664: sync 2024-09-16T23:49:01.806 INFO:tasks.workunit.client.0.smithi042.stdout:3/614: sync 2024-09-16T23:49:01.806 INFO:tasks.workunit.client.0.smithi042.stdout:2/503: sync 2024-09-16T23:49:01.806 INFO:tasks.workunit.client.0.smithi042.stdout:9/364: sync 2024-09-16T23:49:01.806 INFO:tasks.workunit.client.0.smithi042.stdout:8/429: sync 2024-09-16T23:49:01.807 INFO:tasks.workunit.client.0.smithi042.stdout:4/720: creat de/d42/d5d/d8f/d9f/ffc x:0 0 0 2024-09-16T23:49:01.808 INFO:tasks.workunit.client.0.smithi042.stdout:0/410: creat d0/df/d13/d49/f7f x:0 0 0 2024-09-16T23:49:01.809 INFO:tasks.workunit.client.0.smithi042.stdout:5/665: write d9/db/de/d92/fa1 [1029251,4774] 0 2024-09-16T23:49:01.810 INFO:tasks.workunit.client.0.smithi042.stdout:1/602: chown d0/d9/d26/d8b/faf 114942 1 2024-09-16T23:49:01.811 INFO:tasks.workunit.client.0.smithi042.stdout:2/504: readlink d0/l27 0 2024-09-16T23:49:01.812 INFO:tasks.workunit.client.0.smithi042.stdout:4/721: write de/d42/d99/d9e/fd7 [593043,97706] 0 2024-09-16T23:49:01.813 INFO:tasks.workunit.client.0.smithi042.stdout:0/411: write d0/d15/f2f [5800382,48928] 0 2024-09-16T23:49:01.816 INFO:tasks.workunit.client.0.smithi042.stdout:8/430: creat da/f83 x:0 0 0 2024-09-16T23:49:01.818 INFO:tasks.workunit.client.0.smithi042.stdout:3/615: rename d2/d7/d61/d8e/d2c/l49 to d2/d1b/d1d/d34/d50/d93/d96/lda 0 2024-09-16T23:49:01.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/412: unlink d0/df/d13/d2d/d35/l36 0 2024-09-16T23:49:01.821 INFO:tasks.workunit.client.0.smithi042.stdout:9/365: truncate d4/d10/f57 2837380 0 2024-09-16T23:49:01.825 INFO:tasks.workunit.client.0.smithi042.stdout:4/722: write de/d1a/da7/fd2 [847485,97225] 0 2024-09-16T23:49:01.826 INFO:tasks.workunit.client.0.smithi042.stdout:5/666: link d9/db/de/d10/d2f/lc2 d9/db/de/d92/db5/le6 0 2024-09-16T23:49:01.827 INFO:tasks.workunit.client.0.smithi042.stdout:8/431: creat da/d19/d80/f84 x:0 0 0 2024-09-16T23:49:01.828 INFO:tasks.workunit.client.0.smithi042.stdout:3/616: dwrite d2/d1b/fac [0,4194304] 0 2024-09-16T23:49:01.830 INFO:tasks.workunit.client.0.smithi042.stdout:2/505: fdatasync d0/d30/f47 0 2024-09-16T23:49:01.831 INFO:tasks.workunit.client.0.smithi042.stdout:5/667: readlink d9/db/de/d5e/l69 0 2024-09-16T23:49:01.831 INFO:tasks.workunit.client.0.smithi042.stdout:8/432: chown da/c3a 5350949 1 2024-09-16T23:49:01.832 INFO:tasks.workunit.client.1.smithi043.stdout:3/665: sync 2024-09-16T23:49:01.833 INFO:tasks.workunit.client.1.smithi043.stdout:3/666: stat d4/de/d1f/d34 0 2024-09-16T23:49:01.834 INFO:tasks.workunit.client.0.smithi042.stdout:4/723: creat de/d42/d5d/d8f/dd6/ffd x:0 0 0 2024-09-16T23:49:01.834 INFO:tasks.workunit.client.0.smithi042.stdout:3/617: chown d2/d1b/d1d/d71/fd1 4648520 1 2024-09-16T23:49:01.834 INFO:tasks.workunit.client.0.smithi042.stdout:1/603: creat d0/d17/d5e/fd3 x:0 0 0 2024-09-16T23:49:01.834 INFO:tasks.workunit.client.0.smithi042.stdout:0/413: rename d0/df/d23/d33/c6d to d0/d15/c80 0 2024-09-16T23:49:01.837 INFO:tasks.workunit.client.0.smithi042.stdout:9/366: dwrite d4/d10/f30 [0,4194304] 0 2024-09-16T23:49:01.837 INFO:tasks.workunit.client.0.smithi042.stdout:3/618: dread - d2/d95/fd0 zero size 2024-09-16T23:49:01.839 INFO:tasks.workunit.client.0.smithi042.stdout:5/668: creat d9/db/d1b/da4/fe7 x:0 0 0 2024-09-16T23:49:01.841 INFO:tasks.workunit.client.1.smithi043.stdout:3/667: dwrite d4/de/d5d/fa3 [0,4194304] 0 2024-09-16T23:49:01.842 INFO:tasks.workunit.client.0.smithi042.stdout:0/414: write d0/f37 [1310213,31076] 0 2024-09-16T23:49:01.845 INFO:tasks.workunit.client.0.smithi042.stdout:5/669: truncate d9/db/fdc 658104 0 2024-09-16T23:49:01.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/619: write d2/d1b/dc8/fd3 [1508935,48894] 0 2024-09-16T23:49:01.851 INFO:tasks.workunit.client.0.smithi042.stdout:3/620: mknod d2/d7/d61/d8e/d2c/cdb 0 2024-09-16T23:49:01.852 INFO:tasks.workunit.client.0.smithi042.stdout:9/367: creat d4/d12/d60/f78 x:0 0 0 2024-09-16T23:49:01.853 INFO:tasks.workunit.client.0.smithi042.stdout:4/724: dread de/d1a/da7/f47 [0,4194304] 0 2024-09-16T23:49:01.855 INFO:tasks.workunit.client.0.smithi042.stdout:3/621: stat d2/d1b/d1d/d34/l83 0 2024-09-16T23:49:01.856 INFO:tasks.workunit.client.1.smithi043.stdout:0/722: rename d0/d2/df/d1a/f30 to d0/d2/df/d1a/d46/d39/d83/ff2 0 2024-09-16T23:49:01.856 INFO:tasks.workunit.client.0.smithi042.stdout:8/433: dwrite da/d19/d2c/f66 [0,4194304] 0 2024-09-16T23:49:01.856 INFO:tasks.workunit.client.0.smithi042.stdout:6/483: sync 2024-09-16T23:49:01.858 INFO:tasks.workunit.client.0.smithi042.stdout:5/670: rename d9/db/de/fb3 to d9/db/de/d5e/d87/fe8 0 2024-09-16T23:49:01.859 INFO:tasks.workunit.client.0.smithi042.stdout:3/622: creat d2/d1b/fdc x:0 0 0 2024-09-16T23:49:01.863 INFO:tasks.workunit.client.1.smithi043.stdout:5/837: symlink d3/db/d19/l11c 0 2024-09-16T23:49:01.864 INFO:tasks.workunit.client.0.smithi042.stdout:2/506: getdents d0/da/d1b/d48 0 2024-09-16T23:49:01.865 INFO:tasks.workunit.client.0.smithi042.stdout:8/434: truncate da/d19/d1d/d30/d49/f69 929849 0 2024-09-16T23:49:01.865 INFO:tasks.workunit.client.0.smithi042.stdout:5/671: creat d9/db/d1b/da4/fe9 x:0 0 0 2024-09-16T23:49:01.866 INFO:tasks.workunit.client.0.smithi042.stdout:0/415: dwrite d0/df/d13/d2d/d35/f73 [0,4194304] 0 2024-09-16T23:49:01.867 INFO:tasks.workunit.client.1.smithi043.stdout:3/668: dwrite d4/f67 [0,4194304] 0 2024-09-16T23:49:01.868 INFO:tasks.workunit.client.0.smithi042.stdout:3/623: creat d2/d1b/d1d/d30/da2/fdd x:0 0 0 2024-09-16T23:49:01.869 INFO:tasks.workunit.client.0.smithi042.stdout:5/672: symlink d9/d63/lea 0 2024-09-16T23:49:01.869 INFO:tasks.workunit.client.1.smithi043.stdout:0/723: rmdir d0/de 39 2024-09-16T23:49:01.872 INFO:tasks.workunit.client.0.smithi042.stdout:2/507: creat d0/da/d1b/d3b/d44/d54/fb1 x:0 0 0 2024-09-16T23:49:01.874 INFO:tasks.workunit.client.0.smithi042.stdout:5/673: creat d9/db/de/d5e/d87/d61/d43/feb x:0 0 0 2024-09-16T23:49:01.876 INFO:tasks.workunit.client.1.smithi043.stdout:3/669: dwrite d4/de/d35/f4c [0,4194304] 0 2024-09-16T23:49:01.881 INFO:tasks.workunit.client.0.smithi042.stdout:5/674: readlink d9/db/de/d5e/d87/d61/d43/l54 0 2024-09-16T23:49:01.883 INFO:tasks.workunit.client.0.smithi042.stdout:1/604: truncate d0/d17/d79/f7c 2037745 0 2024-09-16T23:49:01.883 INFO:tasks.workunit.client.0.smithi042.stdout:3/624: dwrite d2/d1b/fdc [0,4194304] 0 2024-09-16T23:49:01.885 INFO:tasks.workunit.client.0.smithi042.stdout:2/508: fdatasync d0/da/d1b/d3b/d44/d54/f5e 0 2024-09-16T23:49:01.886 INFO:tasks.workunit.client.0.smithi042.stdout:5/675: readlink d9/db/d1b/d31/l3d 0 2024-09-16T23:49:01.889 INFO:tasks.workunit.client.1.smithi043.stdout:3/670: creat d4/fcd x:0 0 0 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/671: chown d4/de/c13 948042 1 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/672: stat d4/de/d1f/d2c/db7/dc3/fc1 0 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/673: fsync d4/de/d1f/d2c/f32 0 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/674: chown d4/de/d1f/d34/fa8 16152921 1 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/675: dread d4/d30/d53/f5c [0,4194304] 0 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/676: chown d4/de 572616 1 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/677: fsync d4/de/f55 0 2024-09-16T23:49:01.925 INFO:tasks.workunit.client.1.smithi043.stdout:0/724: dread d0/d2/df/d1a/f9e [4194304,4194304] 0 2024-09-16T23:49:01.926 INFO:tasks.workunit.client.1.smithi043.stdout:0/725: chown d0/d2/df/d1a/d3b/dee 5 1 2024-09-16T23:49:01.926 INFO:tasks.workunit.client.0.smithi042.stdout:2/509: readlink d0/l24 0 2024-09-16T23:49:01.926 INFO:tasks.workunit.client.0.smithi042.stdout:3/625: truncate d2/d7/d61/d8e/d9d/dc4/dcc/fbb 64690 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:9/368: truncate d4/d10/f1b 3424558 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:1/605: mkdir d0/d17/dd4 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:2/510: chown d0/da/d88/dae/fa7 22861430 1 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:2/511: chown d0/da/d1b/d3b/d44/d54/fb1 11288 1 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:0/416: dread d0/f9 [4194304,4194304] 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:6/484: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f51 901662 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/676: dwrite d9/db/d19/d21/d85/fd1 [0,4194304] 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:0/417: fdatasync d0/d15/f7c 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:3/626: creat d2/d7/d61/d73/fde x:0 0 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:0/418: chown d0/l69 6170755 1 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:0/419: stat d0/df/c17 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/677: getdents d9/db/d19/d21/d85 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:3/627: dread d2/d1b/d1d/d34/d50/d93/d96/fad [0,4194304] 0 2024-09-16T23:49:01.927 INFO:tasks.workunit.client.0.smithi042.stdout:9/369: link d4/d12/d2d/d37/c2b d4/d12/d2d/d37/d3a/d50/d6f/c79 0 2024-09-16T23:49:01.928 INFO:tasks.workunit.client.0.smithi042.stdout:5/678: rmdir d9/db/d1b/da4 39 2024-09-16T23:49:01.928 INFO:tasks.workunit.client.0.smithi042.stdout:5/679: read d9/db/d1b/d31/d46/d58/fbb [229968,11213] 0 2024-09-16T23:49:01.928 INFO:tasks.workunit.client.0.smithi042.stdout:0/420: link d0/df/d23/f6b d0/d15/f81 0 2024-09-16T23:49:01.928 INFO:tasks.workunit.client.1.smithi043.stdout:3/678: creat d4/de/d1f/d2c/fce x:0 0 0 2024-09-16T23:49:01.928 INFO:tasks.workunit.client.0.smithi042.stdout:5/680: mkdir d9/db/d19/d21/d85/dec 0 2024-09-16T23:49:01.928 INFO:tasks.workunit.client.0.smithi042.stdout:3/628: dwrite d2/d7/d61/d8e/d2c/f32 [0,4194304] 0 2024-09-16T23:49:01.929 INFO:tasks.workunit.client.0.smithi042.stdout:5/681: stat d9/d4a/f9b 0 2024-09-16T23:49:01.930 INFO:tasks.workunit.client.1.smithi043.stdout:0/726: creat d0/d2/df/d1a/d49/db2/ff3 x:0 0 0 2024-09-16T23:49:01.930 INFO:tasks.workunit.client.0.smithi042.stdout:3/629: write d2/d7/fab [254205,26267] 0 2024-09-16T23:49:01.932 INFO:tasks.workunit.client.0.smithi042.stdout:6/485: rmdir d8/d70/d81/dd/d1b/d67/d8f 0 2024-09-16T23:49:01.934 INFO:tasks.workunit.client.0.smithi042.stdout:0/421: dwrite d0/f60 [0,4194304] 0 2024-09-16T23:49:01.935 INFO:tasks.workunit.client.1.smithi043.stdout:3/679: chown d4/de/d1f/d34/l72 63129 1 2024-09-16T23:49:01.939 INFO:tasks.workunit.client.0.smithi042.stdout:3/630: truncate d2/d1b/d1d/f85 4867466 0 2024-09-16T23:49:01.939 INFO:tasks.workunit.client.1.smithi043.stdout:3/680: fdatasync d4/d30/d53/d88/db1/fc2 0 2024-09-16T23:49:01.940 INFO:tasks.workunit.client.1.smithi043.stdout:0/727: rename d0/d2/df/d5a to d0/d2/df/d1a/d49/db2/df4 0 2024-09-16T23:49:01.940 INFO:tasks.workunit.client.0.smithi042.stdout:2/512: getdents d0/da/d88/dae 0 2024-09-16T23:49:01.940 INFO:tasks.workunit.client.0.smithi042.stdout:3/631: fsync d2/d1b/d89/f2e 0 2024-09-16T23:49:01.940 INFO:tasks.workunit.client.0.smithi042.stdout:5/682: symlink d9/db/d1b/da4/de2/led 0 2024-09-16T23:49:01.941 INFO:tasks.workunit.client.0.smithi042.stdout:0/422: write d0/df/d23/f27 [580515,46114] 0 2024-09-16T23:49:01.942 INFO:tasks.workunit.client.0.smithi042.stdout:9/370: creat d4/d12/d2d/d37/d28/d5e/f7a x:0 0 0 2024-09-16T23:49:01.942 INFO:tasks.workunit.client.0.smithi042.stdout:3/632: rmdir d2/d1b/d1d/d30/da2 39 2024-09-16T23:49:01.945 INFO:tasks.workunit.client.0.smithi042.stdout:0/423: truncate d0/d48/f53 1869165 0 2024-09-16T23:49:01.945 INFO:tasks.workunit.client.1.smithi043.stdout:3/681: link d4/de/d1f/d2c/db7/dc3/f8e d4/d30/d53/d88/db1/fcf 0 2024-09-16T23:49:01.946 INFO:tasks.workunit.client.0.smithi042.stdout:3/633: write d2/d1b/d1d/d34/fbd [819066,8047] 0 2024-09-16T23:49:01.949 INFO:tasks.workunit.client.1.smithi043.stdout:0/728: rename d0/d2/df/d1a/d46/d39/d6e/lb5 to d0/d2/df/d1a/d46/lf5 0 2024-09-16T23:49:01.950 INFO:tasks.workunit.client.0.smithi042.stdout:0/424: write d0/d48/f7a [111090,122638] 0 2024-09-16T23:49:01.951 INFO:tasks.workunit.client.0.smithi042.stdout:5/683: dwrite d9/db/d19/f52 [0,4194304] 0 2024-09-16T23:49:01.951 INFO:tasks.workunit.client.0.smithi042.stdout:1/606: chown d0/d17/d79/f7c 7285 1 2024-09-16T23:49:01.951 INFO:tasks.workunit.client.1.smithi043.stdout:3/682: truncate d4/de/d40/fb5 500146 0 2024-09-16T23:49:01.952 INFO:tasks.workunit.client.0.smithi042.stdout:3/634: dread - d2/d1b/d1d/d34/d62/f78 zero size 2024-09-16T23:49:01.956 INFO:tasks.workunit.client.0.smithi042.stdout:0/425: readlink d0/df/d23/l46 0 2024-09-16T23:49:01.956 INFO:tasks.workunit.client.0.smithi042.stdout:3/635: write d2/d1b/f75 [654202,88684] 0 2024-09-16T23:49:01.956 INFO:tasks.workunit.client.0.smithi042.stdout:9/371: dwrite d4/d12/d16/f6d [0,4194304] 0 2024-09-16T23:49:01.959 INFO:tasks.workunit.client.0.smithi042.stdout:9/372: chown d4/d10/d24/f35 9641 1 2024-09-16T23:49:01.959 INFO:tasks.workunit.client.1.smithi043.stdout:3/683: creat d4/de/d1f/d2c/db7/dc3/fd0 x:0 0 0 2024-09-16T23:49:01.960 INFO:tasks.workunit.client.0.smithi042.stdout:1/607: mknod d0/d9/d90/d6b/cd5 0 2024-09-16T23:49:01.962 INFO:tasks.workunit.client.0.smithi042.stdout:6/486: creat d8/d70/d81/dd/d1b/f9d x:0 0 0 2024-09-16T23:49:01.963 INFO:tasks.workunit.client.1.smithi043.stdout:3/684: mkdir d4/d30/dd1 0 2024-09-16T23:49:01.963 INFO:tasks.workunit.client.1.smithi043.stdout:0/729: getdents d0/d2/df/d1a/d46/d39 0 2024-09-16T23:49:01.964 INFO:tasks.workunit.client.0.smithi042.stdout:1/608: mknod d0/d17/cd6 0 2024-09-16T23:49:01.965 INFO:tasks.workunit.client.0.smithi042.stdout:3/636: link d2/d7/d61/d8e/d9d/dc4/cc9 d2/d95/dd8/cdf 0 2024-09-16T23:49:01.966 INFO:tasks.workunit.client.0.smithi042.stdout:1/609: stat d0/d9/d90/d43/l49 0 2024-09-16T23:49:01.967 INFO:tasks.workunit.client.0.smithi042.stdout:0/426: read d0/f7 [1189995,22426] 0 2024-09-16T23:49:01.972 INFO:tasks.workunit.client.0.smithi042.stdout:3/637: truncate f0 4326221 0 2024-09-16T23:49:01.972 INFO:tasks.workunit.client.1.smithi043.stdout:3/685: getdents d4/de/d1f 0 2024-09-16T23:49:01.972 INFO:tasks.workunit.client.0.smithi042.stdout:0/427: fsync d0/f78 0 2024-09-16T23:49:01.975 INFO:tasks.workunit.client.0.smithi042.stdout:2/513: dread d0/da/d1b/d3b/d44/d54/f5f [0,4194304] 0 2024-09-16T23:49:01.976 INFO:tasks.workunit.client.0.smithi042.stdout:3/638: rename d2/d7/d61/d8e/d9d/dc4/dcc/fbb to d2/d1b/d1d/d34/d62/d6a/fe0 0 2024-09-16T23:49:01.977 INFO:tasks.workunit.client.1.smithi043.stdout:0/730: dread d0/d2/f6d [0,4194304] 0 2024-09-16T23:49:01.978 INFO:tasks.workunit.client.0.smithi042.stdout:6/487: dread d8/f37 [0,4194304] 0 2024-09-16T23:49:01.980 INFO:tasks.workunit.client.0.smithi042.stdout:2/514: mknod d0/d98/cb2 0 2024-09-16T23:49:01.980 INFO:tasks.workunit.client.0.smithi042.stdout:3/639: rmdir d2/d1b/d1d/d34/d50/d93 39 2024-09-16T23:49:01.980 INFO:tasks.workunit.client.0.smithi042.stdout:0/428: dread d0/df/d13/f4e [0,4194304] 0 2024-09-16T23:49:01.981 INFO:tasks.workunit.client.1.smithi043.stdout:0/731: mknod d0/d2/df/d1a/d49/d80/d88/cf6 0 2024-09-16T23:49:01.984 INFO:tasks.workunit.client.0.smithi042.stdout:6/488: unlink d8/d70/d81/dd/d2c/c7d 0 2024-09-16T23:49:01.985 INFO:tasks.workunit.client.0.smithi042.stdout:2/515: fdatasync d0/da/d17/d5c/fa8 0 2024-09-16T23:49:01.987 INFO:tasks.workunit.client.0.smithi042.stdout:0/429: dread d0/df/d23/f38 [0,4194304] 0 2024-09-16T23:49:01.989 INFO:tasks.workunit.client.0.smithi042.stdout:3/640: dwrite d2/d7/d61/d73/f9a [0,4194304] 0 2024-09-16T23:49:01.991 INFO:tasks.workunit.client.0.smithi042.stdout:0/430: creat d0/df/d13/d2d/d35/f82 x:0 0 0 2024-09-16T23:49:01.991 INFO:tasks.workunit.client.1.smithi043.stdout:3/686: write d4/de/d40/d4d/f78 [918337,98130] 0 2024-09-16T23:49:01.992 INFO:tasks.workunit.client.0.smithi042.stdout:6/489: symlink d8/d70/d81/dd/d1b/d34/l9e 0 2024-09-16T23:49:01.992 INFO:tasks.workunit.client.0.smithi042.stdout:3/641: creat d2/d1b/fe1 x:0 0 0 2024-09-16T23:49:01.992 INFO:tasks.workunit.client.1.smithi043.stdout:0/732: dread d0/d2/df/f17 [0,4194304] 0 2024-09-16T23:49:01.993 INFO:tasks.workunit.client.1.smithi043.stdout:3/687: write d4/de/d5d/fa3 [3457801,83620] 0 2024-09-16T23:49:01.993 INFO:tasks.workunit.client.1.smithi043.stdout:0/733: fdatasync d0/f4d 0 2024-09-16T23:49:01.996 INFO:tasks.workunit.client.0.smithi042.stdout:6/490: mkdir d8/d70/d81/d90/d9f 0 2024-09-16T23:49:01.998 INFO:tasks.workunit.client.0.smithi042.stdout:6/491: write d8/d70/d81/dd/f19 [1621369,112799] 0 2024-09-16T23:49:01.998 INFO:tasks.workunit.client.1.smithi043.stdout:6/666: sync 2024-09-16T23:49:01.999 INFO:tasks.workunit.client.1.smithi043.stdout:3/688: mknod d4/de/d40/cd2 0 2024-09-16T23:49:02.001 INFO:tasks.workunit.client.1.smithi043.stdout:0/734: symlink d0/d2/df/d1a/d46/lf7 0 2024-09-16T23:49:02.002 INFO:tasks.workunit.client.0.smithi042.stdout:0/431: truncate d0/d15/f2f 1893127 0 2024-09-16T23:49:02.002 INFO:tasks.workunit.client.0.smithi042.stdout:2/516: write d0/d26/f53 [4990226,111707] 0 2024-09-16T23:49:02.003 INFO:tasks.workunit.client.0.smithi042.stdout:0/432: stat d0/d15/f3c 0 2024-09-16T23:49:02.005 INFO:tasks.workunit.client.1.smithi043.stdout:6/667: symlink de/d23/dc3/lde 0 2024-09-16T23:49:02.007 INFO:tasks.workunit.client.0.smithi042.stdout:6/492: mkdir d8/d70/d81/dd/d1b/d50/d96/da0 0 2024-09-16T23:49:02.009 INFO:tasks.workunit.client.0.smithi042.stdout:0/433: unlink d0/df/d23/d33/l39 0 2024-09-16T23:49:02.012 INFO:tasks.workunit.client.0.smithi042.stdout:3/642: dwrite d2/d1b/d1d/d34/d50/d93/d96/fc6 [0,4194304] 0 2024-09-16T23:49:02.012 INFO:tasks.workunit.client.0.smithi042.stdout:2/517: creat d0/da/d1b/d3b/d44/fb3 x:0 0 0 2024-09-16T23:49:02.012 INFO:tasks.workunit.client.1.smithi043.stdout:3/689: unlink d4/de/d5d/la7 0 2024-09-16T23:49:02.013 INFO:tasks.workunit.client.1.smithi043.stdout:3/690: chown d4/de/c13 335460 1 2024-09-16T23:49:02.014 INFO:tasks.workunit.client.0.smithi042.stdout:2/518: chown d0/d30/d43/f89 173930859 1 2024-09-16T23:49:02.014 INFO:tasks.workunit.client.0.smithi042.stdout:6/493: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/f83 250100 0 2024-09-16T23:49:02.015 INFO:tasks.workunit.client.1.smithi043.stdout:0/735: mkdir d0/d2/df/d1a/d3b/dee/df0/df8 0 2024-09-16T23:49:02.015 INFO:tasks.workunit.client.0.smithi042.stdout:0/434: mknod d0/df/d13/d49/c83 0 2024-09-16T23:49:02.015 INFO:tasks.workunit.client.0.smithi042.stdout:3/643: truncate d2/d1b/d1d/d34/d62/f78 61208 0 2024-09-16T23:49:02.016 INFO:tasks.workunit.client.1.smithi043.stdout:3/691: mkdir d4/de/d1f/d2c/db7/dc3/dd3 0 2024-09-16T23:49:02.017 INFO:tasks.workunit.client.0.smithi042.stdout:2/519: write d0/f95 [2054823,93538] 0 2024-09-16T23:49:02.020 INFO:tasks.workunit.client.0.smithi042.stdout:6/494: mkdir d8/d70/d81/dd/d1b/d2d/da1 0 2024-09-16T23:49:02.021 INFO:tasks.workunit.client.1.smithi043.stdout:3/692: dread d4/de/d35/f4c [0,4194304] 0 2024-09-16T23:49:02.022 INFO:tasks.workunit.client.1.smithi043.stdout:0/736: write d0/d2/df/d1a/d46/d39/d83/ff2 [2547148,19653] 0 2024-09-16T23:49:02.023 INFO:tasks.workunit.client.0.smithi042.stdout:0/435: link d0/df/d13/d2d/d35/c4d d0/df/d13/d2d/c84 0 2024-09-16T23:49:02.023 INFO:tasks.workunit.client.0.smithi042.stdout:2/520: dread d0/da/d17/d3e/f91 [0,4194304] 0 2024-09-16T23:49:02.024 INFO:tasks.workunit.client.1.smithi043.stdout:6/668: write de/d1a/d28/f84 [117685,40252] 0 2024-09-16T23:49:02.025 INFO:tasks.workunit.client.1.smithi043.stdout:6/669: truncate de/d1a/f32 1487370 0 2024-09-16T23:49:02.026 INFO:tasks.workunit.client.0.smithi042.stdout:6/495: mkdir d8/d70/d81/dd/d1b/d50/da2 0 2024-09-16T23:49:02.027 INFO:tasks.workunit.client.1.smithi043.stdout:0/737: rmdir d0/d2/df/d1a/d49/db2/df4/d9c 39 2024-09-16T23:49:02.028 INFO:tasks.workunit.client.0.smithi042.stdout:0/436: creat d0/d15/f85 x:0 0 0 2024-09-16T23:49:02.029 INFO:tasks.workunit.client.1.smithi043.stdout:3/693: mknod d4/d30/dd1/cd4 0 2024-09-16T23:49:02.030 INFO:tasks.workunit.client.0.smithi042.stdout:6/496: read d8/d70/d81/dd/f19 [1307391,13857] 0 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:9/615: sync 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:4/728: sync 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:8/625: sync 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:2/613: sync 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:7/635: sync 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:1/807: sync 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:5/838: sync 2024-09-16T23:49:02.031 INFO:tasks.workunit.client.1.smithi043.stdout:6/670: rename de/d23/d49/f6a to de/d23/dc3/fdf 0 2024-09-16T23:49:02.032 INFO:tasks.workunit.client.0.smithi042.stdout:3/644: dwrite d2/d1b/d1d/f25 [4194304,4194304] 0 2024-09-16T23:49:02.034 INFO:tasks.workunit.client.0.smithi042.stdout:2/521: getdents d0/da/d1b 0 2024-09-16T23:49:02.034 INFO:tasks.workunit.client.0.smithi042.stdout:0/437: creat d0/df/d23/d3e/f86 x:0 0 0 2024-09-16T23:49:02.035 INFO:tasks.workunit.client.1.smithi043.stdout:3/694: truncate d4/de/f58 1182884 0 2024-09-16T23:49:02.035 INFO:tasks.workunit.client.1.smithi043.stdout:7/636: write d4/d27/d8a/faa [797841,124897] 0 2024-09-16T23:49:02.035 INFO:tasks.workunit.client.1.smithi043.stdout:6/671: dread - de/d1a/d80/fd0 zero size 2024-09-16T23:49:02.036 INFO:tasks.workunit.client.1.smithi043.stdout:9/616: fdatasync d4/d5/d16/f54 0 2024-09-16T23:49:02.037 INFO:tasks.workunit.client.1.smithi043.stdout:5/839: chown d3/d79/d91/daf/fbc 57382857 1 2024-09-16T23:49:02.037 INFO:tasks.workunit.client.0.smithi042.stdout:3/645: write d2/d1b/d1d/d30/da2/fd2 [304299,119902] 0 2024-09-16T23:49:02.040 INFO:tasks.workunit.client.1.smithi043.stdout:1/808: mkdir d4/d9/d1c/d1e/d47/d116 0 2024-09-16T23:49:02.040 INFO:tasks.workunit.client.1.smithi043.stdout:7/637: chown d4/db/d11/d15/d29/d7a 1 1 2024-09-16T23:49:02.040 INFO:tasks.workunit.client.1.smithi043.stdout:0/738: dwrite d0/d2/df/d1a/d46/d92/fc9 [0,4194304] 0 2024-09-16T23:49:02.041 INFO:tasks.workunit.client.0.smithi042.stdout:2/522: truncate d0/da/d1b/d48/f58 1031267 0 2024-09-16T23:49:02.041 INFO:tasks.workunit.client.0.smithi042.stdout:3/646: chown d2/d95/dd8/cdf 0 1 2024-09-16T23:49:02.041 INFO:tasks.workunit.client.0.smithi042.stdout:2/523: stat d0/d78/d8b 0 2024-09-16T23:49:02.042 INFO:tasks.workunit.client.1.smithi043.stdout:2/614: mknod d8/d35/dc0/cd0 0 2024-09-16T23:49:02.042 INFO:tasks.workunit.client.0.smithi042.stdout:3/647: dread - d2/d7/d61/d8e/d9d/dc4/dcc/fc2 zero size 2024-09-16T23:49:02.044 INFO:tasks.workunit.client.1.smithi043.stdout:3/695: write d4/de/d1f/d2c/db7/dc3/fba [183965,73634] 0 2024-09-16T23:49:02.045 INFO:tasks.workunit.client.1.smithi043.stdout:4/729: creat d8/d15/d20/d2b/dc8/fe3 x:0 0 0 2024-09-16T23:49:02.048 INFO:tasks.workunit.client.0.smithi042.stdout:0/438: truncate d0/df/d13/d2d/d35/f73 1652486 0 2024-09-16T23:49:02.050 INFO:tasks.workunit.client.1.smithi043.stdout:5/840: dread - d3/d9/d22/d4a/fb1 zero size 2024-09-16T23:49:02.051 INFO:tasks.workunit.client.1.smithi043.stdout:2/615: dread d8/d1f/f6e [0,4194304] 0 2024-09-16T23:49:02.051 INFO:tasks.workunit.client.0.smithi042.stdout:0/439: symlink d0/df/d62/l87 0 2024-09-16T23:49:02.051 INFO:tasks.workunit.client.0.smithi042.stdout:3/648: link d2/d7/d61/d8e/c29 d2/d7/d61/d73/ce2 0 2024-09-16T23:49:02.052 INFO:tasks.workunit.client.1.smithi043.stdout:6/672: symlink de/d23/ddc/le0 0 2024-09-16T23:49:02.052 INFO:tasks.workunit.client.1.smithi043.stdout:4/730: truncate d8/d25/d39/fac 1315918 0 2024-09-16T23:49:02.053 INFO:tasks.workunit.client.0.smithi042.stdout:0/440: write d0/f60 [55689,98788] 0 2024-09-16T23:49:02.054 INFO:tasks.workunit.client.0.smithi042.stdout:0/441: read d0/df/d13/f74 [3094261,98328] 0 2024-09-16T23:49:02.055 INFO:tasks.workunit.client.0.smithi042.stdout:3/649: creat d2/d7/d61/db9/fe3 x:0 0 0 2024-09-16T23:49:02.056 INFO:tasks.workunit.client.1.smithi043.stdout:1/809: dread d4/df/df4/d83/d98/fb8 [0,4194304] 0 2024-09-16T23:49:02.057 INFO:tasks.workunit.client.1.smithi043.stdout:8/626: dread de/d18/d27/d41/d5a/f24 [0,4194304] 0 2024-09-16T23:49:02.059 INFO:tasks.workunit.client.1.smithi043.stdout:0/739: truncate d0/d2/df/d1a/d3b/f99 252068 0 2024-09-16T23:49:02.061 INFO:tasks.workunit.client.0.smithi042.stdout:3/650: fsync d2/d7/d61/d8e/f3f 0 2024-09-16T23:49:02.063 INFO:tasks.workunit.client.0.smithi042.stdout:3/651: write d2/d1b/fce [639673,87582] 0 2024-09-16T23:49:02.064 INFO:tasks.workunit.client.0.smithi042.stdout:0/442: creat d0/f88 x:0 0 0 2024-09-16T23:49:02.065 INFO:tasks.workunit.client.0.smithi042.stdout:0/443: chown d0/f78 7418 1 2024-09-16T23:49:02.066 INFO:tasks.workunit.client.0.smithi042.stdout:3/652: truncate d2/d1b/d89/fbc 302932 0 2024-09-16T23:49:02.067 INFO:tasks.workunit.client.1.smithi043.stdout:6/673: dwrite de/d2a/f9e [0,4194304] 0 2024-09-16T23:49:02.067 INFO:tasks.workunit.client.0.smithi042.stdout:3/653: write d2/d1b/d1d/d34/f58 [5092623,114412] 0 2024-09-16T23:49:02.068 INFO:tasks.workunit.client.1.smithi043.stdout:9/617: creat d4/d21/d2a/d2c/fc1 x:0 0 0 2024-09-16T23:49:02.069 INFO:tasks.workunit.client.0.smithi042.stdout:7/619: sync 2024-09-16T23:49:02.071 INFO:tasks.workunit.client.1.smithi043.stdout:4/731: rename d8/d15/d20/d2b/d54/d55/d73/fcb to d8/d25/d39/dd3/fe4 0 2024-09-16T23:49:02.071 INFO:tasks.workunit.client.0.smithi042.stdout:6/497: truncate d8/d70/d81/dd/d2c/f10 1519553 0 2024-09-16T23:49:02.072 INFO:tasks.workunit.client.0.smithi042.stdout:0/444: mknod d0/df/d13/c89 0 2024-09-16T23:49:02.073 INFO:tasks.workunit.client.0.smithi042.stdout:3/654: mkdir d2/d7/d61/d8e/d9d/dc4/de4 0 2024-09-16T23:49:02.074 INFO:tasks.workunit.client.1.smithi043.stdout:1/810: symlink d4/d9/d12/d15/dac/l117 0 2024-09-16T23:49:02.075 INFO:tasks.workunit.client.0.smithi042.stdout:7/620: mkdir d3/d1f/d4f/dd6 0 2024-09-16T23:49:02.076 INFO:tasks.workunit.client.1.smithi043.stdout:1/811: fdatasync d4/d9/d12/d2b/dc9/fee 0 2024-09-16T23:49:02.076 INFO:tasks.workunit.client.1.smithi043.stdout:5/841: dread d3/f50 [4194304,4194304] 0 2024-09-16T23:49:02.077 INFO:tasks.workunit.client.0.smithi042.stdout:0/445: rmdir d0/df/d13/d49 39 2024-09-16T23:49:02.079 INFO:tasks.workunit.client.1.smithi043.stdout:5/842: fsync d3/db/d19/d3e/fea 0 2024-09-16T23:49:02.079 INFO:tasks.workunit.client.1.smithi043.stdout:0/740: dread d0/d2/df/d1a/d49/db2/df4/fe4 [0,4194304] 0 2024-09-16T23:49:02.084 INFO:tasks.workunit.client.0.smithi042.stdout:0/446: unlink d0/df/d13/d49/f7f 0 2024-09-16T23:49:02.085 INFO:tasks.workunit.client.1.smithi043.stdout:7/638: dread d4/d27/d41/f8d [0,4194304] 0 2024-09-16T23:49:02.086 INFO:tasks.workunit.client.1.smithi043.stdout:7/639: fdatasync d4/f60 0 2024-09-16T23:49:02.086 INFO:tasks.workunit.client.1.smithi043.stdout:5/843: dwrite d3/d9/d22/d4a/f7f [0,4194304] 0 2024-09-16T23:49:02.087 INFO:tasks.workunit.client.1.smithi043.stdout:5/844: chown d3/db/d15/l6d 230892574 1 2024-09-16T23:49:02.088 INFO:tasks.workunit.client.1.smithi043.stdout:5/845: fdatasync d3/db/d19/d76/f10e 0 2024-09-16T23:49:02.089 INFO:tasks.workunit.client.0.smithi042.stdout:7/621: truncate d3/d98/da0/d41/f47 2740812 0 2024-09-16T23:49:02.090 INFO:tasks.workunit.client.0.smithi042.stdout:3/655: dwrite d2/d7/d61/d8e/f3f [0,4194304] 0 2024-09-16T23:49:02.093 INFO:tasks.workunit.client.1.smithi043.stdout:5/846: dwrite d3/db/d26/f112 [0,4194304] 0 2024-09-16T23:49:02.094 INFO:tasks.workunit.client.1.smithi043.stdout:5/847: chown d3/db/d15/lb4 69918455 1 2024-09-16T23:49:02.095 INFO:tasks.workunit.client.1.smithi043.stdout:5/848: chown d3/d9/d22/d60/f82 687988 1 2024-09-16T23:49:02.097 INFO:tasks.workunit.client.0.smithi042.stdout:0/447: dwrite d0/df/d23/f31 [0,4194304] 0 2024-09-16T23:49:02.101 INFO:tasks.workunit.client.0.smithi042.stdout:3/656: dwrite d2/d1b/d89/f42 [0,4194304] 0 2024-09-16T23:49:02.102 INFO:tasks.workunit.client.0.smithi042.stdout:0/448: mknod d0/df/c8a 0 2024-09-16T23:49:02.116 INFO:tasks.workunit.client.0.smithi042.stdout:3/657: dwrite d2/d1b/d1d/f85 [0,4194304] 0 2024-09-16T23:49:02.117 INFO:tasks.workunit.client.0.smithi042.stdout:0/449: creat d0/df/d23/d33/d56/f8b x:0 0 0 2024-09-16T23:49:02.122 INFO:tasks.workunit.client.0.smithi042.stdout:3/658: rename f0 to d2/d7/d61/d8e/fe5 0 2024-09-16T23:49:02.126 INFO:tasks.workunit.client.0.smithi042.stdout:3/659: dread d2/d1b/d89/f3a [0,4194304] 0 2024-09-16T23:49:02.127 INFO:tasks.workunit.client.0.smithi042.stdout:3/660: dread - d2/d1b/d1d/fd9 zero size 2024-09-16T23:49:02.129 INFO:tasks.workunit.client.0.smithi042.stdout:3/661: mkdir d2/d1b/d1d/d34/d50/de6 0 2024-09-16T23:49:02.136 INFO:tasks.workunit.client.0.smithi042.stdout:3/662: dwrite d2/d1b/fb2 [0,4194304] 0 2024-09-16T23:49:02.139 INFO:tasks.workunit.client.0.smithi042.stdout:0/450: dwrite d0/df/d13/d2d/d35/f73 [0,4194304] 0 2024-09-16T23:49:02.140 INFO:tasks.workunit.client.0.smithi042.stdout:3/663: write d2/d1b/d89/f5c [291068,32067] 0 2024-09-16T23:49:02.143 INFO:tasks.workunit.client.0.smithi042.stdout:3/664: symlink d2/d95/dd8/le7 0 2024-09-16T23:49:02.143 INFO:tasks.workunit.client.0.smithi042.stdout:3/665: chown d2/d9/f22 47100 1 2024-09-16T23:49:02.144 INFO:tasks.workunit.client.0.smithi042.stdout:3/666: chown d2/d1b/d89/l53 357 1 2024-09-16T23:49:02.145 INFO:tasks.workunit.client.0.smithi042.stdout:0/451: link d0/d15/f2f d0/df/f8c 0 2024-09-16T23:49:02.149 INFO:tasks.workunit.client.0.smithi042.stdout:0/452: stat d0/df/d13/d2d/c84 0 2024-09-16T23:49:02.149 INFO:tasks.workunit.client.0.smithi042.stdout:3/667: dread d2/d7/f33 [0,4194304] 0 2024-09-16T23:49:02.155 INFO:tasks.workunit.client.0.smithi042.stdout:3/668: read d2/f4a [587020,92943] 0 2024-09-16T23:49:02.155 INFO:tasks.workunit.client.0.smithi042.stdout:3/669: stat d2/d1b/d1d/d30/f79 0 2024-09-16T23:49:02.157 INFO:tasks.workunit.client.0.smithi042.stdout:3/670: dread - d2/d1b/d1d/d34/d62/d6a/fb1 zero size 2024-09-16T23:49:02.158 INFO:tasks.workunit.client.0.smithi042.stdout:3/671: write d2/d1b/f43 [1146472,74605] 0 2024-09-16T23:49:02.159 INFO:tasks.workunit.client.0.smithi042.stdout:3/672: fsync d2/d9/f22 0 2024-09-16T23:49:02.161 INFO:tasks.workunit.client.0.smithi042.stdout:3/673: stat d2/d7/d61/f90 0 2024-09-16T23:49:02.162 INFO:tasks.workunit.client.0.smithi042.stdout:0/453: dwrite d0/f9 [0,4194304] 0 2024-09-16T23:49:02.164 INFO:tasks.workunit.client.0.smithi042.stdout:0/454: dread - d0/df/d23/d3e/f86 zero size 2024-09-16T23:49:02.167 INFO:tasks.workunit.client.0.smithi042.stdout:0/455: rename d0/df/l41 to d0/df/d13/d2d/d35/l8d 0 2024-09-16T23:49:02.169 INFO:tasks.workunit.client.0.smithi042.stdout:0/456: mknod d0/df/d62/c8e 0 2024-09-16T23:49:02.171 INFO:tasks.workunit.client.0.smithi042.stdout:0/457: mkdir d0/df/d23/d3e/d8f 0 2024-09-16T23:49:02.181 INFO:tasks.workunit.client.1.smithi043.stdout:6/674: creat de/dbf/fe1 x:0 0 0 2024-09-16T23:49:02.181 INFO:tasks.workunit.client.0.smithi042.stdout:6/498: write d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 [2686705,73220] 0 2024-09-16T23:49:02.185 INFO:tasks.workunit.client.1.smithi043.stdout:4/732: chown d8/d25/c56 23976275 1 2024-09-16T23:49:02.186 INFO:tasks.workunit.client.1.smithi043.stdout:3/696: dwrite d4/de/d1f/d2c/db7/dc3/fc1 [0,4194304] 0 2024-09-16T23:49:02.193 INFO:tasks.workunit.client.1.smithi043.stdout:8/627: write de/d70/d3d/f51 [4236640,124842] 0 2024-09-16T23:49:02.212 INFO:tasks.workunit.client.1.smithi043.stdout:9/618: link d4/d28/d5f/d91/fad d4/d1c/d25/d2b/d4c/da3/fc2 0 2024-09-16T23:49:02.213 INFO:tasks.workunit.client.1.smithi043.stdout:9/619: readlink d4/d28/d5f/l6c 0 2024-09-16T23:49:02.213 INFO:tasks.workunit.client.1.smithi043.stdout:6/675: dwrite de/d1a/d80/fdd [4194304,4194304] 0 2024-09-16T23:49:02.217 INFO:tasks.workunit.client.0.smithi042.stdout:6/499: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/ca3 0 2024-09-16T23:49:02.218 INFO:tasks.workunit.client.1.smithi043.stdout:4/733: mknod d8/d15/d20/d2b/d23/ce5 0 2024-09-16T23:49:02.219 INFO:tasks.workunit.client.1.smithi043.stdout:4/734: chown d8/d15/d20/d2b/d23/fd1 0 1 2024-09-16T23:49:02.219 INFO:tasks.workunit.client.0.smithi042.stdout:6/500: read d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f95 [1383137,79096] 0 2024-09-16T23:49:02.222 INFO:tasks.workunit.client.0.smithi042.stdout:6/501: write d8/d70/d81/dd/d2c/d57/f5d [1639531,56933] 0 2024-09-16T23:49:02.222 INFO:tasks.workunit.client.1.smithi043.stdout:3/697: unlink d4/de/d1f/d2c/db7/dc3/fba 0 2024-09-16T23:49:02.225 INFO:tasks.workunit.client.0.smithi042.stdout:6/502: rename d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e to d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/da4 22 2024-09-16T23:49:02.226 INFO:tasks.workunit.client.1.smithi043.stdout:3/698: dread d4/d30/d53/d88/db1/fcf [0,4194304] 0 2024-09-16T23:49:02.227 INFO:tasks.workunit.client.1.smithi043.stdout:0/741: truncate d0/f4d 2202477 0 2024-09-16T23:49:02.229 INFO:tasks.workunit.client.1.smithi043.stdout:7/640: mkdir d4/d27/dce 0 2024-09-16T23:49:02.231 INFO:tasks.workunit.client.0.smithi042.stdout:6/503: dread d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f51 [0,4194304] 0 2024-09-16T23:49:02.234 INFO:tasks.workunit.client.1.smithi043.stdout:4/735: truncate d8/d25/d39/d58/fab 341630 0 2024-09-16T23:49:02.236 INFO:tasks.workunit.client.0.smithi042.stdout:6/504: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/ca5 0 2024-09-16T23:49:02.237 INFO:tasks.workunit.client.1.smithi043.stdout:8/628: fdatasync de/d18/d27/d41/d5a/f56 0 2024-09-16T23:49:02.239 INFO:tasks.workunit.client.0.smithi042.stdout:6/505: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/ca6 0 2024-09-16T23:49:02.240 INFO:tasks.workunit.client.0.smithi042.stdout:6/506: chown d8/d70/d81/dd/d2c 61833 1 2024-09-16T23:49:02.242 INFO:tasks.workunit.client.1.smithi043.stdout:5/849: link d3/db/d15/d4d/f6c d3/d9/f11d 0 2024-09-16T23:49:02.245 INFO:tasks.workunit.client.1.smithi043.stdout:9/620: mknod d4/d21/d33/cc3 0 2024-09-16T23:49:02.247 INFO:tasks.workunit.client.1.smithi043.stdout:6/676: symlink de/d23/le2 0 2024-09-16T23:49:02.249 INFO:tasks.workunit.client.1.smithi043.stdout:1/812: dwrite d4/d9/d1c/d1e/f85 [0,4194304] 0 2024-09-16T23:49:02.249 INFO:tasks.workunit.client.1.smithi043.stdout:9/621: write d4/d21/d35/d4b/fb7 [2524478,41205] 0 2024-09-16T23:49:02.250 INFO:tasks.workunit.client.0.smithi042.stdout:6/507: dread d8/d70/d81/dd/f15 [0,4194304] 0 2024-09-16T23:49:02.250 INFO:tasks.workunit.client.1.smithi043.stdout:9/622: write d4/d5/d64/f65 [4477870,114838] 0 2024-09-16T23:49:02.250 INFO:tasks.workunit.client.1.smithi043.stdout:4/736: creat d8/d15/d20/d2b/d54/db3/fe6 x:0 0 0 2024-09-16T23:49:02.252 INFO:tasks.workunit.client.1.smithi043.stdout:8/629: fsync de/d18/d1f/d2f/d39/d4d/f5d 0 2024-09-16T23:49:02.253 INFO:tasks.workunit.client.0.smithi042.stdout:6/508: rmdir d8/d70/d81/dd/d1b/d50 39 2024-09-16T23:49:02.255 INFO:tasks.workunit.client.1.smithi043.stdout:5/850: mknod d3/d9/d6f/d7c/c11e 0 2024-09-16T23:49:02.257 INFO:tasks.workunit.client.1.smithi043.stdout:3/699: write d4/de/d40/fb6 [2600197,35465] 0 2024-09-16T23:49:02.259 INFO:tasks.workunit.client.1.smithi043.stdout:6/677: creat de/d23/d34/d9b/d50/fe3 x:0 0 0 2024-09-16T23:49:02.260 INFO:tasks.workunit.client.1.smithi043.stdout:6/678: dread - de/d23/d34/d9b/fc7 zero size 2024-09-16T23:49:02.261 INFO:tasks.workunit.client.0.smithi042.stdout:6/509: truncate d8/d70/d81/fc 2875177 0 2024-09-16T23:49:02.261 INFO:tasks.workunit.client.1.smithi043.stdout:6/679: fdatasync de/d1a/d80/fce 0 2024-09-16T23:49:02.263 INFO:tasks.workunit.client.1.smithi043.stdout:1/813: rename d4/db5 to d4/d9/d8f/db6/dc6/d118 0 2024-09-16T23:49:02.265 INFO:tasks.workunit.client.1.smithi043.stdout:9/623: mknod d4/d1c/cc4 0 2024-09-16T23:49:02.267 INFO:tasks.workunit.client.1.smithi043.stdout:8/630: write de/d18/f36 [753416,94396] 0 2024-09-16T23:49:02.267 INFO:tasks.workunit.client.1.smithi043.stdout:7/641: creat d4/db/d11/fcf x:0 0 0 2024-09-16T23:49:02.268 INFO:tasks.workunit.client.1.smithi043.stdout:9/624: truncate d4/d21/d2a/d2c/d7b/faa 460762 0 2024-09-16T23:49:02.271 INFO:tasks.workunit.client.1.smithi043.stdout:8/631: write de/d18/d1f/d2f/d39/fa4 [427721,49533] 0 2024-09-16T23:49:02.271 INFO:tasks.workunit.client.1.smithi043.stdout:4/737: mkdir d8/d15/de7 0 2024-09-16T23:49:02.272 INFO:tasks.workunit.client.1.smithi043.stdout:1/814: write d4/d9/d12/fce [3529543,70081] 0 2024-09-16T23:49:02.275 INFO:tasks.workunit.client.0.smithi042.stdout:6/510: unlink d8/d70/d81/fc 0 2024-09-16T23:49:02.276 INFO:tasks.workunit.client.1.smithi043.stdout:5/851: dwrite d3/d1b/f119 [0,4194304] 0 2024-09-16T23:49:02.276 INFO:tasks.workunit.client.0.smithi042.stdout:6/511: write d8/d70/d81/dd/d1b/f93 [236064,83430] 0 2024-09-16T23:49:02.279 INFO:tasks.workunit.client.1.smithi043.stdout:9/625: truncate d4/d1c/f27 6125551 0 2024-09-16T23:49:02.282 INFO:tasks.workunit.client.1.smithi043.stdout:3/700: write d4/d30/d53/d88/db1/fcf [4346781,56040] 0 2024-09-16T23:49:02.283 INFO:tasks.workunit.client.1.smithi043.stdout:3/701: fdatasync d4/de/d40/d4d/f78 0 2024-09-16T23:49:02.285 INFO:tasks.workunit.client.1.smithi043.stdout:7/642: creat d4/dc0/fd0 x:0 0 0 2024-09-16T23:49:02.288 INFO:tasks.workunit.client.1.smithi043.stdout:8/632: mkdir de/d18/d1f/d2f/d39/d7b/d83/dbb 0 2024-09-16T23:49:02.288 INFO:tasks.workunit.client.1.smithi043.stdout:1/815: dread - d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fa1 zero size 2024-09-16T23:49:02.293 INFO:tasks.workunit.client.1.smithi043.stdout:4/738: symlink d8/le8 0 2024-09-16T23:49:02.293 INFO:tasks.workunit.client.1.smithi043.stdout:4/739: read - d8/d15/d9b/fd2 zero size 2024-09-16T23:49:02.298 INFO:tasks.workunit.client.0.smithi042.stdout:6/512: dwrite d8/d70/d81/dd/f11 [0,4194304] 0 2024-09-16T23:49:02.299 INFO:tasks.workunit.client.0.smithi042.stdout:6/513: fsync d8/d38/f74 0 2024-09-16T23:49:02.308 INFO:tasks.workunit.client.0.smithi042.stdout:6/514: creat d8/fa7 x:0 0 0 2024-09-16T23:49:02.311 INFO:tasks.workunit.client.0.smithi042.stdout:6/515: readlink d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/l7e 0 2024-09-16T23:49:02.312 INFO:tasks.workunit.client.0.smithi042.stdout:6/516: fsync d8/d70/d81/d90/f91 0 2024-09-16T23:49:02.322 INFO:tasks.workunit.client.0.smithi042.stdout:6/517: dread d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f95 [0,4194304] 0 2024-09-16T23:49:02.327 INFO:tasks.workunit.client.1.smithi043.stdout:2/616: sync 2024-09-16T23:49:02.331 INFO:tasks.workunit.client.0.smithi042.stdout:6/518: dwrite d8/d70/d81/dd/d1b/d34/f64 [0,4194304] 0 2024-09-16T23:49:02.335 INFO:tasks.workunit.client.0.smithi042.stdout:6/519: mknod d8/d70/d81/dd/d1b/d50/da2/ca8 0 2024-09-16T23:49:02.336 INFO:tasks.workunit.client.1.smithi043.stdout:7/643: creat d4/d27/d41/d1b/d75/fd1 x:0 0 0 2024-09-16T23:49:02.337 INFO:tasks.workunit.client.1.smithi043.stdout:2/617: read - d8/d1d/d25/fbd zero size 2024-09-16T23:49:02.337 INFO:tasks.workunit.client.0.smithi042.stdout:6/520: dread d8/d70/d81/dd/d1b/d34/d39/d5e/f47 [0,4194304] 0 2024-09-16T23:49:02.338 INFO:tasks.workunit.client.1.smithi043.stdout:1/816: readlink d4/d9/la 0 2024-09-16T23:49:02.338 INFO:tasks.workunit.client.1.smithi043.stdout:6/680: getdents de/d23/d34/d9b/d50 0 2024-09-16T23:49:02.338 INFO:tasks.workunit.client.1.smithi043.stdout:9/626: mknod d4/d1c/d25/d2b/da1/cc5 0 2024-09-16T23:49:02.340 INFO:tasks.workunit.client.1.smithi043.stdout:9/627: read - d4/d1c/d4f/f8b zero size 2024-09-16T23:49:02.340 INFO:tasks.workunit.client.0.smithi042.stdout:6/521: creat d8/d70/d81/dd/d1b/d34/d39/fa9 x:0 0 0 2024-09-16T23:49:02.341 INFO:tasks.workunit.client.1.smithi043.stdout:2/618: fsync d8/d1d/d20/d26/f60 0 2024-09-16T23:49:02.342 INFO:tasks.workunit.client.1.smithi043.stdout:7/644: mkdir d4/db/d11/d15/dd2 0 2024-09-16T23:49:02.343 INFO:tasks.workunit.client.1.smithi043.stdout:3/702: write d4/f3b [412207,94329] 0 2024-09-16T23:49:02.347 INFO:tasks.workunit.client.1.smithi043.stdout:2/619: creat d8/d1d/d20/d23/d38/fd1 x:0 0 0 2024-09-16T23:49:02.350 INFO:tasks.workunit.client.1.smithi043.stdout:9/628: truncate d4/d1c/f27 4900096 0 2024-09-16T23:49:02.350 INFO:tasks.workunit.client.1.smithi043.stdout:3/703: chown d4/de/d1f/d2c/lc8 1438 1 2024-09-16T23:49:02.352 INFO:tasks.workunit.client.1.smithi043.stdout:6/681: dwrite de/f24 [4194304,4194304] 0 2024-09-16T23:49:02.353 INFO:tasks.workunit.client.1.smithi043.stdout:2/620: write d8/d5e/d83/f55 [2646286,54026] 0 2024-09-16T23:49:02.354 INFO:tasks.workunit.client.1.smithi043.stdout:1/817: dwrite d4/d34/f3a [0,4194304] 0 2024-09-16T23:49:02.357 INFO:tasks.workunit.client.1.smithi043.stdout:3/704: symlink d4/de/ld5 0 2024-09-16T23:49:02.357 INFO:tasks.workunit.client.1.smithi043.stdout:2/621: fsync d8/d4b/d5f/d8e/fbb 0 2024-09-16T23:49:02.358 INFO:tasks.workunit.client.1.smithi043.stdout:8/633: dread de/d18/d1f/d25/f3b [0,4194304] 0 2024-09-16T23:49:02.358 INFO:tasks.workunit.client.1.smithi043.stdout:3/705: readlink d4/d30/l57 0 2024-09-16T23:49:02.359 INFO:tasks.workunit.client.1.smithi043.stdout:9/629: creat d4/d1c/d25/d2b/fc6 x:0 0 0 2024-09-16T23:49:02.360 INFO:tasks.workunit.client.0.smithi042.stdout:6/522: dread d8/d70/d81/dd/f22 [0,4194304] 0 2024-09-16T23:49:02.360 INFO:tasks.workunit.client.1.smithi043.stdout:1/818: creat d4/df/df4/dc8/f119 x:0 0 0 2024-09-16T23:49:02.364 INFO:tasks.workunit.client.1.smithi043.stdout:7/645: dread d4/d27/d41/f4c [0,4194304] 0 2024-09-16T23:49:02.366 INFO:tasks.workunit.client.1.smithi043.stdout:8/634: unlink de/c4f 0 2024-09-16T23:49:02.369 INFO:tasks.workunit.client.1.smithi043.stdout:2/622: rename d8/d1d/d20/d23/d38/da2/cc3 to d8/d4b/d4f/d7c/cd2 0 2024-09-16T23:49:02.372 INFO:tasks.workunit.client.1.smithi043.stdout:8/635: dwrite de/d18/d27/f94 [0,4194304] 0 2024-09-16T23:49:02.374 INFO:tasks.workunit.client.1.smithi043.stdout:1/819: creat d4/d9/d1c/d1e/d42/d5d/d71/f11a x:0 0 0 2024-09-16T23:49:02.375 INFO:tasks.workunit.client.0.smithi042.stdout:6/523: dread d8/d70/d81/dd/f22 [0,4194304] 0 2024-09-16T23:49:02.380 INFO:tasks.workunit.client.0.smithi042.stdout:6/524: rename d8/d70/d81/dd/d1b/d50 to d8/d70/d81/dd/d1b/d50/daa 22 2024-09-16T23:49:02.381 INFO:tasks.workunit.client.0.smithi042.stdout:6/525: write d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f79 [327674,20133] 0 2024-09-16T23:49:02.381 INFO:tasks.workunit.client.1.smithi043.stdout:3/706: link d4/fcd d4/d30/d53/d88/db1/fd6 0 2024-09-16T23:49:02.381 INFO:tasks.workunit.client.1.smithi043.stdout:3/707: stat d4/de/d40/d4d/f78 0 2024-09-16T23:49:02.381 INFO:tasks.workunit.client.1.smithi043.stdout:3/708: truncate d4/de/d40/fbf 225789 0 2024-09-16T23:49:02.381 INFO:tasks.workunit.client.1.smithi043.stdout:6/682: write de/d23/d34/d9b/d50/fa8 [772362,86098] 0 2024-09-16T23:49:02.382 INFO:tasks.workunit.client.1.smithi043.stdout:3/709: chown d4/de/d40/f71 0 1 2024-09-16T23:49:02.385 INFO:tasks.workunit.client.1.smithi043.stdout:9/630: write d4/d28/d49/f73 [1501598,49357] 0 2024-09-16T23:49:02.386 INFO:tasks.workunit.client.0.smithi042.stdout:6/526: rmdir d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d7b 39 2024-09-16T23:49:02.388 INFO:tasks.workunit.client.1.smithi043.stdout:6/683: creat de/dbf/fe4 x:0 0 0 2024-09-16T23:49:02.388 INFO:tasks.workunit.client.0.smithi042.stdout:6/527: truncate d8/d70/d81/d90/f91 951139 0 2024-09-16T23:49:02.388 INFO:tasks.workunit.client.0.smithi042.stdout:6/528: write f6 [3730164,12032] 0 2024-09-16T23:49:02.389 INFO:tasks.workunit.client.1.smithi043.stdout:8/636: truncate de/f45 851436 0 2024-09-16T23:49:02.390 INFO:tasks.workunit.client.0.smithi042.stdout:6/529: chown d8/d70/d81/dd/d1b/d34/d39/d5e 3352 1 2024-09-16T23:49:02.391 INFO:tasks.workunit.client.0.smithi042.stdout:6/530: fdatasync f6 0 2024-09-16T23:49:02.392 INFO:tasks.workunit.client.1.smithi043.stdout:3/710: rename d4/de/d40/f98 to d4/de/d1f/d2c/db7/dc3/fd7 0 2024-09-16T23:49:02.394 INFO:tasks.workunit.client.1.smithi043.stdout:7/646: truncate d4/f32 1807716 0 2024-09-16T23:49:02.395 INFO:tasks.workunit.client.1.smithi043.stdout:9/631: symlink d4/d1c/d4f/d45/d85/lc7 0 2024-09-16T23:49:02.396 INFO:tasks.workunit.client.1.smithi043.stdout:1/820: symlink d4/d3e/d95/de8/l11b 0 2024-09-16T23:49:02.399 INFO:tasks.workunit.client.1.smithi043.stdout:2/623: dwrite d8/d5e/d83/d9e/dab/fcb [0,4194304] 0 2024-09-16T23:49:02.400 INFO:tasks.workunit.client.1.smithi043.stdout:9/632: write d4/d28/d49/f73 [2302509,93295] 0 2024-09-16T23:49:02.401 INFO:tasks.workunit.client.1.smithi043.stdout:3/711: unlink d4/c1a 0 2024-09-16T23:49:02.403 INFO:tasks.workunit.client.1.smithi043.stdout:6/684: dread f2 [0,4194304] 0 2024-09-16T23:49:02.404 INFO:tasks.workunit.client.1.smithi043.stdout:1/821: creat d4/df/df4/d83/d98/f11c x:0 0 0 2024-09-16T23:49:02.405 INFO:tasks.workunit.client.1.smithi043.stdout:6/685: truncate de/d23/d63/fd2 556694 0 2024-09-16T23:49:02.407 INFO:tasks.workunit.client.1.smithi043.stdout:6/686: write de/d1a/d28/fa5 [742640,107645] 0 2024-09-16T23:49:02.409 INFO:tasks.workunit.client.1.smithi043.stdout:8/637: getdents de/d18/d1f 0 2024-09-16T23:49:02.410 INFO:tasks.workunit.client.1.smithi043.stdout:9/633: symlink d4/d21/d35/d4b/lc8 0 2024-09-16T23:49:02.411 INFO:tasks.workunit.client.1.smithi043.stdout:9/634: chown d4/d21/f4d 2588358 1 2024-09-16T23:49:02.411 INFO:tasks.workunit.client.1.smithi043.stdout:7/647: dread d4/db/d11/d15/f47 [0,4194304] 0 2024-09-16T23:49:02.412 INFO:tasks.workunit.client.1.smithi043.stdout:9/635: readlink d4/d21/d33/da2/la4 0 2024-09-16T23:49:02.412 INFO:tasks.workunit.client.1.smithi043.stdout:3/712: rename d4/de/c1c to d4/de/d5d/cd8 0 2024-09-16T23:49:02.414 INFO:tasks.workunit.client.1.smithi043.stdout:2/624: mkdir d8/d5e/d83/d9e/dab/dd3 0 2024-09-16T23:49:02.417 INFO:tasks.workunit.client.1.smithi043.stdout:8/638: creat de/d18/d1f/d25/d65/d7f/fbc x:0 0 0 2024-09-16T23:49:02.418 INFO:tasks.workunit.client.1.smithi043.stdout:8/639: write de/fb0 [126900,103135] 0 2024-09-16T23:49:02.420 INFO:tasks.workunit.client.1.smithi043.stdout:3/713: symlink d4/d30/d53/d88/ld9 0 2024-09-16T23:49:02.421 INFO:tasks.workunit.client.1.smithi043.stdout:2/625: symlink d8/d1d/d20/d23/d38/ld4 0 2024-09-16T23:49:02.423 INFO:tasks.workunit.client.1.smithi043.stdout:0/742: sync 2024-09-16T23:49:02.424 INFO:tasks.workunit.client.1.smithi043.stdout:8/640: symlink de/d18/d27/d41/d5a/d37/lbd 0 2024-09-16T23:49:02.425 INFO:tasks.workunit.client.1.smithi043.stdout:8/641: fdatasync de/d18/fb1 0 2024-09-16T23:49:02.426 INFO:tasks.workunit.client.1.smithi043.stdout:3/714: creat d4/de/d40/d4d/fda x:0 0 0 2024-09-16T23:49:02.428 INFO:tasks.workunit.client.1.smithi043.stdout:1/822: write d4/d9/d12/d15/dac/ff2 [157682,109986] 0 2024-09-16T23:49:02.430 INFO:tasks.workunit.client.1.smithi043.stdout:2/626: truncate d8/fd 2521777 0 2024-09-16T23:49:02.431 INFO:tasks.workunit.client.1.smithi043.stdout:1/823: chown d4/d9/d1c 266562007 1 2024-09-16T23:49:02.432 INFO:tasks.workunit.client.1.smithi043.stdout:9/636: getdents d4/d1c 0 2024-09-16T23:49:02.433 INFO:tasks.workunit.client.1.smithi043.stdout:8/642: mknod de/d70/cbe 0 2024-09-16T23:49:02.434 INFO:tasks.workunit.client.1.smithi043.stdout:0/743: dwrite d0/d2/df/d1a/d46/d39/daa/fd3 [0,4194304] 0 2024-09-16T23:49:02.435 INFO:tasks.workunit.client.1.smithi043.stdout:7/648: dread d4/db/f51 [0,4194304] 0 2024-09-16T23:49:02.436 INFO:tasks.workunit.client.1.smithi043.stdout:8/643: chown de/d18/d1f/d25/f5f 210611692 1 2024-09-16T23:49:02.437 INFO:tasks.workunit.client.1.smithi043.stdout:2/627: write d8/d1d/f5a [969497,106462] 0 2024-09-16T23:49:02.440 INFO:tasks.workunit.client.1.smithi043.stdout:2/628: chown d8/d4b/d4f/d56 2 1 2024-09-16T23:49:02.442 INFO:tasks.workunit.client.1.smithi043.stdout:3/715: getdents d4/d30/d53/d88/db1 0 2024-09-16T23:49:02.443 INFO:tasks.workunit.client.1.smithi043.stdout:7/649: symlink d4/d27/d41/d1b/d75/ld3 0 2024-09-16T23:49:02.444 INFO:tasks.workunit.client.1.smithi043.stdout:8/644: creat de/d18/d1f/d2f/d39/d4d/fbf x:0 0 0 2024-09-16T23:49:02.444 INFO:tasks.workunit.client.1.smithi043.stdout:1/824: truncate d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fa1 506584 0 2024-09-16T23:49:02.446 INFO:tasks.workunit.client.1.smithi043.stdout:2/629: rename d8/d4b/d4f/d7c/l82 to d8/d5e/ld5 0 2024-09-16T23:49:02.447 INFO:tasks.workunit.client.1.smithi043.stdout:3/716: read d4/de/d1f/d2c/f32 [490926,9623] 0 2024-09-16T23:49:02.450 INFO:tasks.workunit.client.1.smithi043.stdout:1/825: mknod d4/d9/d8f/db6/dcc/c11d 0 2024-09-16T23:49:02.451 INFO:tasks.workunit.client.1.smithi043.stdout:7/650: symlink d4/db/d11/d15/d29/da7/ld4 0 2024-09-16T23:49:02.502 INFO:tasks.workunit.client.1.smithi043.stdout:3/717: creat d4/de/d1f/fdb x:0 0 0 2024-09-16T23:49:02.504 INFO:tasks.workunit.client.1.smithi043.stdout:8/645: dwrite de/d18/d1f/f59 [0,4194304] 0 2024-09-16T23:49:02.504 INFO:tasks.workunit.client.1.smithi043.stdout:1/826: read - d4/d9/d1c/d1e/d42/d5d/d110/ffe zero size 2024-09-16T23:49:02.504 INFO:tasks.workunit.client.1.smithi043.stdout:9/637: dwrite d4/d1c/d4f/d45/f6a [0,4194304] 0 2024-09-16T23:49:02.505 INFO:tasks.workunit.client.1.smithi043.stdout:0/744: dwrite d0/d2/df/da7/db8/f60 [0,4194304] 0 2024-09-16T23:49:02.508 INFO:tasks.workunit.client.1.smithi043.stdout:7/651: mknod d4/d27/d41/d1b/cd5 0 2024-09-16T23:49:02.511 INFO:tasks.workunit.client.1.smithi043.stdout:8/646: creat de/d18/d1f/d2f/d39/d7b/d83/fc0 x:0 0 0 2024-09-16T23:49:02.513 INFO:tasks.workunit.client.1.smithi043.stdout:1/827: rename d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/l77 to d4/df/d26/dd5/l11e 0 2024-09-16T23:49:02.514 INFO:tasks.workunit.client.1.smithi043.stdout:8/647: rename de/d18/d1f/d25 to de/d18/d1f/d25/dc1 22 2024-09-16T23:49:02.516 INFO:tasks.workunit.client.1.smithi043.stdout:0/745: link d0/l4b d0/d2/df/d1a/d46/lf9 0 2024-09-16T23:49:02.516 INFO:tasks.workunit.client.1.smithi043.stdout:7/652: getdents d4/db/d11/d15/dd2 0 2024-09-16T23:49:02.517 INFO:tasks.workunit.client.1.smithi043.stdout:1/828: mknod d4/df/df4/c11f 0 2024-09-16T23:49:02.518 INFO:tasks.workunit.client.1.smithi043.stdout:8/648: dread - de/d18/d1f/d2f/d39/f81 zero size 2024-09-16T23:49:02.519 INFO:tasks.workunit.client.1.smithi043.stdout:7/653: dread - d4/db/f81 zero size 2024-09-16T23:49:02.519 INFO:tasks.workunit.client.1.smithi043.stdout:1/829: fsync d4/f69 0 2024-09-16T23:49:02.520 INFO:tasks.workunit.client.1.smithi043.stdout:0/746: chown d0/d2/df/dbb/c89 14815318 1 2024-09-16T23:49:02.523 INFO:tasks.workunit.client.1.smithi043.stdout:9/638: dread d4/d21/f89 [0,4194304] 0 2024-09-16T23:49:02.523 INFO:tasks.workunit.client.1.smithi043.stdout:7/654: rename d4/db/d43/d83/d85/fa1 to d4/dc0/fd6 0 2024-09-16T23:49:02.525 INFO:tasks.workunit.client.1.smithi043.stdout:7/655: fdatasync d4/d27/d8a/faa 0 2024-09-16T23:49:02.527 INFO:tasks.workunit.client.1.smithi043.stdout:0/747: mkdir d0/d2/df/d1a/d49/db2/df4/d9c/dfa 0 2024-09-16T23:49:02.530 INFO:tasks.workunit.client.1.smithi043.stdout:0/748: mkdir d0/d2/df/d1a/d46/d39/d83/dfb 0 2024-09-16T23:49:02.530 INFO:tasks.workunit.client.1.smithi043.stdout:8/649: write de/d18/d1f/d25/f5f [2816542,30915] 0 2024-09-16T23:49:02.531 INFO:tasks.workunit.client.1.smithi043.stdout:0/749: fsync d0/d2/df/da7/fce 0 2024-09-16T23:49:02.533 INFO:tasks.workunit.client.1.smithi043.stdout:1/830: rename d4/d3e/d95/de8 to d4/d9/d1c/d1e/d42/d5d/d120 0 2024-09-16T23:49:02.534 INFO:tasks.workunit.client.1.smithi043.stdout:1/831: read - d4/d9/d1c/d1e/d42/d5d/d71/f11a zero size 2024-09-16T23:49:02.535 INFO:tasks.workunit.client.1.smithi043.stdout:0/750: truncate d0/f3c 695018 0 2024-09-16T23:49:02.538 INFO:tasks.workunit.client.1.smithi043.stdout:9/639: rename d4/d1c/d4f/f8b to d4/d21/d33/d95/fc9 0 2024-09-16T23:49:02.539 INFO:tasks.workunit.client.1.smithi043.stdout:7/656: truncate d4/fab 2607558 0 2024-09-16T23:49:02.539 INFO:tasks.workunit.client.1.smithi043.stdout:4/740: sync 2024-09-16T23:49:02.539 INFO:tasks.workunit.client.1.smithi043.stdout:5/852: sync 2024-09-16T23:49:02.541 INFO:tasks.workunit.client.1.smithi043.stdout:1/832: creat d4/d9/d8f/db6/dcc/dda/d107/f121 x:0 0 0 2024-09-16T23:49:02.542 INFO:tasks.workunit.client.1.smithi043.stdout:8/650: write de/d18/d27/d41/d5a/d37/f99 [806378,96742] 0 2024-09-16T23:49:02.544 INFO:tasks.workunit.client.1.smithi043.stdout:8/651: dread - de/d18/d1f/d25/fb9 zero size 2024-09-16T23:49:02.544 INFO:tasks.workunit.client.1.smithi043.stdout:0/751: creat d0/d2/df/d1a/d49/db2/ffc x:0 0 0 2024-09-16T23:49:02.548 INFO:tasks.workunit.client.1.smithi043.stdout:4/741: truncate d8/d15/f3c 2339831 0 2024-09-16T23:49:02.551 INFO:tasks.workunit.client.1.smithi043.stdout:7/657: symlink d4/d27/ld7 0 2024-09-16T23:49:02.553 INFO:tasks.workunit.client.0.smithi042.stdout:4/725: sync 2024-09-16T23:49:02.554 INFO:tasks.workunit.client.1.smithi043.stdout:4/742: mkdir d8/d25/d39/d41/de9 0 2024-09-16T23:49:02.554 INFO:tasks.workunit.client.1.smithi043.stdout:9/640: write d4/d1c/d4f/d8c/f60 [1700406,125363] 0 2024-09-16T23:49:02.556 INFO:tasks.workunit.client.0.smithi042.stdout:4/726: mkdir de/d2e/d37/d4c/dfe 0 2024-09-16T23:49:02.557 INFO:tasks.workunit.client.1.smithi043.stdout:8/652: dwrite de/d18/d1f/d2f/d39/fb3 [0,4194304] 0 2024-09-16T23:49:02.557 INFO:tasks.workunit.client.0.smithi042.stdout:4/727: write de/d42/d99/fc7 [183382,101429] 0 2024-09-16T23:49:02.559 INFO:tasks.workunit.client.1.smithi043.stdout:4/743: chown d8/d15/d20/d2b/fd6 0 1 2024-09-16T23:49:02.563 INFO:tasks.workunit.client.1.smithi043.stdout:1/833: getdents d4/d9/d8f 0 2024-09-16T23:49:02.563 INFO:tasks.workunit.client.1.smithi043.stdout:7/658: write d4/d27/d41/f8d [1672059,100688] 0 2024-09-16T23:49:02.563 INFO:tasks.workunit.client.0.smithi042.stdout:4/728: creat de/d2e/d37/d4c/fff x:0 0 0 2024-09-16T23:49:02.566 INFO:tasks.workunit.client.1.smithi043.stdout:8/653: mknod de/d18/d27/d41/d5a/cc2 0 2024-09-16T23:49:02.566 INFO:tasks.workunit.client.1.smithi043.stdout:5/853: dwrite d3/db/d19/d3e/d28/f37 [0,4194304] 0 2024-09-16T23:49:02.568 INFO:tasks.workunit.client.1.smithi043.stdout:0/752: getdents d0/d2/df/d1a/d46/d39 0 2024-09-16T23:49:02.570 INFO:tasks.workunit.client.0.smithi042.stdout:4/729: link de/d1a/f55 de/d12/d1b/db8/f100 0 2024-09-16T23:49:02.570 INFO:tasks.workunit.client.1.smithi043.stdout:9/641: dread d4/d1c/d4f/f77 [0,4194304] 0 2024-09-16T23:49:02.571 INFO:tasks.workunit.client.1.smithi043.stdout:5/854: stat d3/dce 0 2024-09-16T23:49:02.571 INFO:tasks.workunit.client.1.smithi043.stdout:1/834: readlink d4/d9/d1c/d1e/d42/d5d/d71/d76/l88 0 2024-09-16T23:49:02.575 INFO:tasks.workunit.client.1.smithi043.stdout:4/744: creat d8/d15/fea x:0 0 0 2024-09-16T23:49:02.575 INFO:tasks.workunit.client.1.smithi043.stdout:7/659: creat d4/db/d43/d83/fd8 x:0 0 0 2024-09-16T23:49:02.575 INFO:tasks.workunit.client.1.smithi043.stdout:1/835: truncate d4/d9/d12/f10e 949016 0 2024-09-16T23:49:02.577 INFO:tasks.workunit.client.1.smithi043.stdout:5/855: truncate d3/db/d19/d3e/fea 1751331 0 2024-09-16T23:49:02.577 INFO:tasks.workunit.client.1.smithi043.stdout:0/753: creat d0/d2/df/d1a/d3b/dee/df0/df8/ffd x:0 0 0 2024-09-16T23:49:02.579 INFO:tasks.workunit.client.0.smithi042.stdout:4/730: dwrite de/d2e/d37/f3b [0,4194304] 0 2024-09-16T23:49:02.579 INFO:tasks.workunit.client.1.smithi043.stdout:4/745: mknod d8/d15/d9b/dbf/ceb 0 2024-09-16T23:49:02.582 INFO:tasks.workunit.client.1.smithi043.stdout:7/660: creat d4/db/d11/d15/fd9 x:0 0 0 2024-09-16T23:49:02.583 INFO:tasks.workunit.client.1.smithi043.stdout:8/654: dwrite de/d18/d27/d41/d5a/d37/f99 [0,4194304] 0 2024-09-16T23:49:02.590 INFO:tasks.workunit.client.1.smithi043.stdout:5/856: rmdir d3/d9/d22/d4a/d64/d8f/da1 39 2024-09-16T23:49:02.590 INFO:tasks.workunit.client.1.smithi043.stdout:8/655: write de/d18/f36 [319041,82650] 0 2024-09-16T23:49:02.590 INFO:tasks.workunit.client.1.smithi043.stdout:4/746: fsync d8/d15/d20/d2b/d54/d55/f2a 0 2024-09-16T23:49:02.596 INFO:tasks.workunit.client.1.smithi043.stdout:7/661: getdents d4/db/d11/d15/d29 0 2024-09-16T23:49:02.597 INFO:tasks.workunit.client.1.smithi043.stdout:1/836: dwrite d4/d9/d12/faa [4194304,4194304] 0 2024-09-16T23:49:02.598 INFO:tasks.workunit.client.1.smithi043.stdout:4/747: dread - d8/d25/d39/dd3/fcf zero size 2024-09-16T23:49:02.600 INFO:tasks.workunit.client.1.smithi043.stdout:1/837: chown d4/df/df4/dc8 7573988 1 2024-09-16T23:49:02.601 INFO:tasks.workunit.client.1.smithi043.stdout:0/754: dwrite d0/d2/df/d1a/de8/fed [0,4194304] 0 2024-09-16T23:49:02.602 INFO:tasks.workunit.client.1.smithi043.stdout:7/662: mkdir d4/db/d11/d15/dda 0 2024-09-16T23:49:02.605 INFO:tasks.workunit.client.1.smithi043.stdout:0/755: chown d0/d2/df/dbb/c89 24 1 2024-09-16T23:49:02.610 INFO:tasks.workunit.client.1.smithi043.stdout:4/748: unlink d8/d15/d20/d2b/c92 0 2024-09-16T23:49:02.612 INFO:tasks.workunit.client.1.smithi043.stdout:7/663: creat d4/d27/dbc/fdb x:0 0 0 2024-09-16T23:49:02.613 INFO:tasks.workunit.client.1.smithi043.stdout:1/838: dread d4/d9/d1c/d1e/d42/d5d/f93 [0,4194304] 0 2024-09-16T23:49:02.613 INFO:tasks.workunit.client.1.smithi043.stdout:0/756: getdents d0/d2/df/d1a/d49/d80/d88/dc4 0 2024-09-16T23:49:02.614 INFO:tasks.workunit.client.1.smithi043.stdout:4/749: chown d8/d25/d64/fb4 4651 1 2024-09-16T23:49:02.615 INFO:tasks.workunit.client.1.smithi043.stdout:1/839: chown d4/fc 0 1 2024-09-16T23:49:02.615 INFO:tasks.workunit.client.1.smithi043.stdout:7/664: unlink d4/dc0/fd0 0 2024-09-16T23:49:02.616 INFO:tasks.workunit.client.1.smithi043.stdout:4/750: chown d8/d15/d20/d2b/f5a 49 1 2024-09-16T23:49:02.618 INFO:tasks.workunit.client.1.smithi043.stdout:0/757: dread d0/d2/df/d1a/d49/db2/df4/fe4 [0,4194304] 0 2024-09-16T23:49:02.620 INFO:tasks.workunit.client.1.smithi043.stdout:7/665: creat d4/fdc x:0 0 0 2024-09-16T23:49:02.622 INFO:tasks.workunit.client.1.smithi043.stdout:4/751: unlink d8/d15/d20/d2b/d54/d55/f7a 0 2024-09-16T23:49:02.623 INFO:tasks.workunit.client.1.smithi043.stdout:1/840: rename d4/d9/d1c/d1e/d42/d5d/d71/d76/l88 to d4/d9/d8f/db6/dcc/l122 0 2024-09-16T23:49:02.624 INFO:tasks.workunit.client.1.smithi043.stdout:8/656: dread de/d18/d27/d8e/f80 [0,4194304] 0 2024-09-16T23:49:02.625 INFO:tasks.workunit.client.1.smithi043.stdout:4/752: mkdir d8/d25/d39/dd3/dec 0 2024-09-16T23:49:02.625 INFO:tasks.workunit.client.1.smithi043.stdout:7/666: write d4/d27/d41/d1b/d95/fb4 [1000541,82724] 0 2024-09-16T23:49:02.627 INFO:tasks.workunit.client.1.smithi043.stdout:5/857: dread d3/f9b [0,4194304] 0 2024-09-16T23:49:02.628 INFO:tasks.workunit.client.1.smithi043.stdout:0/758: rename d0/d2/df/fad to d0/d2/df/d1a/d49/d80/ffe 0 2024-09-16T23:49:02.628 INFO:tasks.workunit.client.1.smithi043.stdout:5/858: chown d3/d9/d22/d60 475266526 1 2024-09-16T23:49:02.629 INFO:tasks.workunit.client.1.smithi043.stdout:8/657: symlink de/d18/d1f/d2f/d39/d4d/lc3 0 2024-09-16T23:49:02.629 INFO:tasks.workunit.client.1.smithi043.stdout:4/753: rmdir d8/d25/d39/dd3 39 2024-09-16T23:49:02.630 INFO:tasks.workunit.client.1.smithi043.stdout:5/859: write d3/db/d15/d4d/d6b/ff5 [810179,91699] 0 2024-09-16T23:49:02.631 INFO:tasks.workunit.client.1.smithi043.stdout:4/754: stat d8/d25/d39/d58/fc0 0 2024-09-16T23:49:02.631 INFO:tasks.workunit.client.1.smithi043.stdout:0/759: creat d0/d2/df/d1a/d46/d39/fff x:0 0 0 2024-09-16T23:49:02.633 INFO:tasks.workunit.client.1.smithi043.stdout:4/755: read d8/d25/d64/f7f [1141549,118573] 0 2024-09-16T23:49:02.633 INFO:tasks.workunit.client.1.smithi043.stdout:8/658: rmdir de/d18/d1f/d2f 39 2024-09-16T23:49:02.634 INFO:tasks.workunit.client.1.smithi043.stdout:7/667: link d4/db/d43/f79 d4/db/d11/d15/dda/fdd 0 2024-09-16T23:49:02.635 INFO:tasks.workunit.client.1.smithi043.stdout:7/668: dread - d4/d27/d41/d1b/d95/fc8 zero size 2024-09-16T23:49:02.635 INFO:tasks.workunit.client.1.smithi043.stdout:0/760: creat d0/d2/df/d1a/d46/d92/f100 x:0 0 0 2024-09-16T23:49:02.636 INFO:tasks.workunit.client.1.smithi043.stdout:1/841: link d4/df/d26/d79/ldd d4/d9/d12/d15/l123 0 2024-09-16T23:49:02.638 INFO:tasks.workunit.client.1.smithi043.stdout:7/669: chown d4/db/d43/d83/d85/d8b/f9f 0 1 2024-09-16T23:49:02.638 INFO:tasks.workunit.client.1.smithi043.stdout:8/659: symlink de/d18/d27/d41/d5a/d37/d6a/lc4 0 2024-09-16T23:49:02.639 INFO:tasks.workunit.client.1.smithi043.stdout:1/842: creat d4/d9/d1c/d1e/f124 x:0 0 0 2024-09-16T23:49:02.640 INFO:tasks.workunit.client.1.smithi043.stdout:7/670: write d4/d27/d41/fb6 [239974,18493] 0 2024-09-16T23:49:02.641 INFO:tasks.workunit.client.1.smithi043.stdout:8/660: chown de/f45 217763 1 2024-09-16T23:49:02.644 INFO:tasks.workunit.client.1.smithi043.stdout:0/761: dwrite d0/f41 [4194304,4194304] 0 2024-09-16T23:49:02.644 INFO:tasks.workunit.client.1.smithi043.stdout:1/843: dread d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fa1 [0,4194304] 0 2024-09-16T23:49:02.647 INFO:tasks.workunit.client.1.smithi043.stdout:1/844: dread - d4/d9/d1c/d1e/d42/d5d/d71/fed zero size 2024-09-16T23:49:02.650 INFO:tasks.workunit.client.1.smithi043.stdout:8/661: dread - de/d18/d1f/d2f/d39/d7b/fba zero size 2024-09-16T23:49:02.654 INFO:tasks.workunit.client.1.smithi043.stdout:1/845: creat d4/df/df4/d83/d98/f125 x:0 0 0 2024-09-16T23:49:02.655 INFO:tasks.workunit.client.1.smithi043.stdout:0/762: dwrite d0/de/f2d [4194304,4194304] 0 2024-09-16T23:49:02.658 INFO:tasks.workunit.client.1.smithi043.stdout:1/846: chown d4/d9/d1c/d1e/d42/d5d/d71/d76 75689 1 2024-09-16T23:49:02.660 INFO:tasks.workunit.client.1.smithi043.stdout:8/662: creat de/fc5 x:0 0 0 2024-09-16T23:49:02.663 INFO:tasks.workunit.client.1.smithi043.stdout:4/756: dwrite d8/d15/f98 [0,4194304] 0 2024-09-16T23:49:02.664 INFO:tasks.workunit.client.1.smithi043.stdout:1/847: creat d4/df/d26/dd5/f126 x:0 0 0 2024-09-16T23:49:02.669 INFO:tasks.workunit.client.1.smithi043.stdout:4/757: write d8/d25/d39/d41/fb9 [1711826,44542] 0 2024-09-16T23:49:02.671 INFO:tasks.workunit.client.1.smithi043.stdout:0/763: dwrite d0/de/f2d [0,4194304] 0 2024-09-16T23:49:02.673 INFO:tasks.workunit.client.1.smithi043.stdout:7/671: dwrite d4/db/d50/fa9 [0,4194304] 0 2024-09-16T23:49:02.674 INFO:tasks.workunit.client.1.smithi043.stdout:4/758: chown d8/d25/d64/f7d 41 1 2024-09-16T23:49:02.677 INFO:tasks.workunit.client.1.smithi043.stdout:8/663: dread de/d18/d1f/d25/f3b [0,4194304] 0 2024-09-16T23:49:02.682 INFO:tasks.workunit.client.1.smithi043.stdout:0/764: write d0/d2/df/f59 [2814981,124654] 0 2024-09-16T23:49:02.683 INFO:tasks.workunit.client.1.smithi043.stdout:8/664: dread de/d18/d27/f94 [0,4194304] 0 2024-09-16T23:49:02.684 INFO:tasks.workunit.client.1.smithi043.stdout:6/687: sync 2024-09-16T23:49:02.684 INFO:tasks.workunit.client.1.smithi043.stdout:4/759: truncate d8/d15/d20/d2b/f10 4489194 0 2024-09-16T23:49:02.690 INFO:tasks.workunit.client.1.smithi043.stdout:7/672: dwrite d4/f2b [0,4194304] 0 2024-09-16T23:49:02.691 INFO:tasks.workunit.client.1.smithi043.stdout:6/688: write de/d23/d34/d66/d9d/dae/fc2 [784740,61555] 0 2024-09-16T23:49:02.692 INFO:tasks.workunit.client.1.smithi043.stdout:0/765: symlink d0/l101 0 2024-09-16T23:49:02.693 INFO:tasks.workunit.client.1.smithi043.stdout:8/665: rename de/d18/d2b/l35 to de/d18/d1f/d25/d65/d7f/lc6 0 2024-09-16T23:49:02.695 INFO:tasks.workunit.client.1.smithi043.stdout:1/848: dwrite d4/d9/d1c/d1e/d42/f70 [0,4194304] 0 2024-09-16T23:49:02.696 INFO:tasks.workunit.client.1.smithi043.stdout:0/766: readlink d0/l11 0 2024-09-16T23:49:02.696 INFO:tasks.workunit.client.1.smithi043.stdout:7/673: chown d4/db/d11/d15/cca 3933 1 2024-09-16T23:49:02.697 INFO:tasks.workunit.client.1.smithi043.stdout:1/849: write d4/d9/f84 [844684,113207] 0 2024-09-16T23:49:02.698 INFO:tasks.workunit.client.1.smithi043.stdout:4/760: mkdir d8/d25/d39/d41/d57/ded 0 2024-09-16T23:49:02.700 INFO:tasks.workunit.client.1.smithi043.stdout:6/689: creat de/d23/d34/d9b/d8b/fe5 x:0 0 0 2024-09-16T23:49:02.701 INFO:tasks.workunit.client.1.smithi043.stdout:8/666: symlink de/d18/d1f/d2f/d39/d7b/lc7 0 2024-09-16T23:49:02.707 INFO:tasks.workunit.client.1.smithi043.stdout:0/767: mknod d0/d2/df/d1a/c102 0 2024-09-16T23:49:02.708 INFO:tasks.workunit.client.1.smithi043.stdout:8/667: mkdir de/d18/d27/d41/d5a/dc8 0 2024-09-16T23:49:02.709 INFO:tasks.workunit.client.1.smithi043.stdout:6/690: dread de/d1a/d39/f75 [0,4194304] 0 2024-09-16T23:49:02.710 INFO:tasks.workunit.client.1.smithi043.stdout:4/761: rename d8/d25/d39/dd3/dec to d8/d15/d20/d2b/d54/db3/d4c/dee 0 2024-09-16T23:49:02.713 INFO:tasks.workunit.client.1.smithi043.stdout:7/674: dwrite d4/db/d28/f4d [0,4194304] 0 2024-09-16T23:49:02.713 INFO:tasks.workunit.client.1.smithi043.stdout:0/768: write d0/d2/df/d1a/d46/d39/d83/ff2 [3122305,6263] 0 2024-09-16T23:49:02.720 INFO:tasks.workunit.client.1.smithi043.stdout:4/762: symlink d8/d15/d20/lef 0 2024-09-16T23:49:02.720 INFO:tasks.workunit.client.1.smithi043.stdout:7/675: chown d4/db/d43/d83 503865780 1 2024-09-16T23:49:02.723 INFO:tasks.workunit.client.1.smithi043.stdout:8/668: getdents de/d18/d1f/d25 0 2024-09-16T23:49:02.724 INFO:tasks.workunit.client.1.smithi043.stdout:7/676: chown d4/db/d11/d15/dd2 123103 1 2024-09-16T23:49:02.725 INFO:tasks.workunit.client.1.smithi043.stdout:1/850: dwrite d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fa1 [0,4194304] 0 2024-09-16T23:49:02.725 INFO:tasks.workunit.client.1.smithi043.stdout:4/763: chown d8/d15/d9b/fa7 7434 1 2024-09-16T23:49:02.726 INFO:tasks.workunit.client.1.smithi043.stdout:6/691: dwrite de/d23/fb8 [0,4194304] 0 2024-09-16T23:49:02.727 INFO:tasks.workunit.client.1.smithi043.stdout:0/769: getdents d0/d2/df/d1a/d3b/dee 0 2024-09-16T23:49:02.728 INFO:tasks.workunit.client.1.smithi043.stdout:8/669: dread - de/d18/d1f/d2f/d39/d7b/fb5 zero size 2024-09-16T23:49:02.731 INFO:tasks.workunit.client.1.smithi043.stdout:8/670: write de/fb0 [838244,106382] 0 2024-09-16T23:49:02.733 INFO:tasks.workunit.client.1.smithi043.stdout:6/692: unlink de/d23/d34/l55 0 2024-09-16T23:49:02.735 INFO:tasks.workunit.client.1.smithi043.stdout:7/677: dwrite d4/f2b [0,4194304] 0 2024-09-16T23:49:02.735 INFO:tasks.workunit.client.1.smithi043.stdout:8/671: read - de/d18/d1f/d2f/d39/d4d/fbf zero size 2024-09-16T23:49:02.735 INFO:tasks.workunit.client.0.smithi042.stdout:8/435: sync 2024-09-16T23:49:02.735 INFO:tasks.workunit.client.0.smithi042.stdout:5/684: sync 2024-09-16T23:49:02.738 INFO:tasks.workunit.client.1.smithi043.stdout:4/764: creat d8/d15/ff0 x:0 0 0 2024-09-16T23:49:02.738 INFO:tasks.workunit.client.1.smithi043.stdout:0/770: truncate d0/d2/df/d1a/d3b/f2a 2952552 0 2024-09-16T23:49:02.739 INFO:tasks.workunit.client.0.smithi042.stdout:5/685: dread d9/db/fdc [0,4194304] 0 2024-09-16T23:49:02.744 INFO:tasks.workunit.client.0.smithi042.stdout:8/436: creat da/d19/d1d/d30/d49/f85 x:0 0 0 2024-09-16T23:49:02.744 INFO:tasks.workunit.client.0.smithi042.stdout:5/686: mknod d9/d4a/d89/cee 0 2024-09-16T23:49:02.744 INFO:tasks.workunit.client.1.smithi043.stdout:7/678: dwrite d4/db/d43/d83/d85/f42 [0,4194304] 0 2024-09-16T23:49:02.749 INFO:tasks.workunit.client.1.smithi043.stdout:4/765: unlink d8/d15/d20/d2b/d22/ca2 0 2024-09-16T23:49:02.750 INFO:tasks.workunit.client.1.smithi043.stdout:0/771: creat d0/d2/df/d1a/d3b/f103 x:0 0 0 2024-09-16T23:49:02.752 INFO:tasks.workunit.client.0.smithi042.stdout:8/437: mknod da/d4b/d74/c86 0 2024-09-16T23:49:02.753 INFO:tasks.workunit.client.0.smithi042.stdout:8/438: fsync da/f47 0 2024-09-16T23:49:02.754 INFO:tasks.workunit.client.0.smithi042.stdout:8/439: write da/d19/d1d/f62 [3880840,31308] 0 2024-09-16T23:49:02.754 INFO:tasks.workunit.client.1.smithi043.stdout:8/672: creat de/d70/fc9 x:0 0 0 2024-09-16T23:49:02.759 INFO:tasks.workunit.client.1.smithi043.stdout:3/718: sync 2024-09-16T23:49:02.760 INFO:tasks.workunit.client.1.smithi043.stdout:2/630: sync 2024-09-16T23:49:02.762 INFO:tasks.workunit.client.1.smithi043.stdout:1/851: dread d4/fc [4194304,4194304] 0 2024-09-16T23:49:02.762 INFO:tasks.workunit.client.1.smithi043.stdout:6/693: read de/d23/d34/d66/fac [48379,8616] 0 2024-09-16T23:49:02.764 INFO:tasks.workunit.client.1.smithi043.stdout:8/673: mkdir de/d18/d1f/d25/d65/d7f/dca 0 2024-09-16T23:49:02.768 INFO:tasks.workunit.client.1.smithi043.stdout:1/852: mknod d4/d9/d1c/d1e/d42/d5d/d120/c127 0 2024-09-16T23:49:02.768 INFO:tasks.workunit.client.1.smithi043.stdout:7/679: truncate d4/db/d28/f2d 2772596 0 2024-09-16T23:49:02.769 INFO:tasks.workunit.client.1.smithi043.stdout:7/680: chown d4/db/d11/d15/dbe 1 1 2024-09-16T23:49:02.770 INFO:tasks.workunit.client.1.smithi043.stdout:1/853: truncate d4/d9/d12/d15/dac/ff2 706058 0 2024-09-16T23:49:02.770 INFO:tasks.workunit.client.1.smithi043.stdout:0/772: creat d0/d2/df/f104 x:0 0 0 2024-09-16T23:49:02.771 INFO:tasks.workunit.client.0.smithi042.stdout:8/440: write da/d19/d1d/d30/d49/d58/f5b [50630,63819] 0 2024-09-16T23:49:02.772 INFO:tasks.workunit.client.0.smithi042.stdout:8/441: write da/d19/f7a [666458,121248] 0 2024-09-16T23:49:02.776 INFO:tasks.workunit.client.1.smithi043.stdout:2/631: creat d8/d4b/fd6 x:0 0 0 2024-09-16T23:49:02.778 INFO:tasks.workunit.client.1.smithi043.stdout:7/681: creat d4/d27/d41/d1b/d75/fde x:0 0 0 2024-09-16T23:49:02.784 INFO:tasks.workunit.client.1.smithi043.stdout:2/632: symlink d8/d35/ld7 0 2024-09-16T23:49:02.784 INFO:tasks.workunit.client.1.smithi043.stdout:7/682: fsync d4/db/d11/d21/d94/f9c 0 2024-09-16T23:49:02.788 INFO:tasks.workunit.client.1.smithi043.stdout:3/719: getdents d4/d30/d53/d88 0 2024-09-16T23:49:02.788 INFO:tasks.workunit.client.1.smithi043.stdout:8/674: dwrite de/d18/d27/d41/f50 [0,4194304] 0 2024-09-16T23:49:02.788 INFO:tasks.workunit.client.1.smithi043.stdout:6/694: symlink de/d1a/d39/le6 0 2024-09-16T23:49:02.788 INFO:tasks.workunit.client.1.smithi043.stdout:2/633: chown d8/d4b/d5f/d63/db6 28 1 2024-09-16T23:49:02.789 INFO:tasks.workunit.client.1.smithi043.stdout:3/720: truncate d4/d30/fcc 22010 0 2024-09-16T23:49:02.791 INFO:tasks.workunit.client.1.smithi043.stdout:3/721: chown d4/de 211951 1 2024-09-16T23:49:02.792 INFO:tasks.workunit.client.1.smithi043.stdout:6/695: dread - de/d1a/d80/fd0 zero size 2024-09-16T23:49:02.793 INFO:tasks.workunit.client.1.smithi043.stdout:7/683: creat d4/db/d11/d21/d31/d65/fdf x:0 0 0 2024-09-16T23:49:02.794 INFO:tasks.workunit.client.1.smithi043.stdout:1/854: dread d4/df/f38 [0,4194304] 0 2024-09-16T23:49:02.795 INFO:tasks.workunit.client.1.smithi043.stdout:3/722: fdatasync d4/de/f55 0 2024-09-16T23:49:02.795 INFO:tasks.workunit.client.1.smithi043.stdout:2/634: dread d8/d5e/d83/fa7 [0,4194304] 0 2024-09-16T23:49:02.796 INFO:tasks.workunit.client.1.smithi043.stdout:8/675: creat de/d18/d27/d41/fcb x:0 0 0 2024-09-16T23:49:02.797 INFO:tasks.workunit.client.1.smithi043.stdout:7/684: mknod d4/db/d28/ce0 0 2024-09-16T23:49:02.799 INFO:tasks.workunit.client.1.smithi043.stdout:6/696: stat de/d23/d63/d7f/cda 0 2024-09-16T23:49:02.800 INFO:tasks.workunit.client.1.smithi043.stdout:2/635: symlink d8/d1d/d20/ld8 0 2024-09-16T23:49:02.800 INFO:tasks.workunit.client.1.smithi043.stdout:0/773: dwrite d0/d2/df/dbb/f9b [0,4194304] 0 2024-09-16T23:49:02.801 INFO:tasks.workunit.client.1.smithi043.stdout:8/676: stat de/d18/d1f/d2f/d39/d7b/l8b 0 2024-09-16T23:49:02.808 INFO:tasks.workunit.client.1.smithi043.stdout:6/697: mknod de/ce7 0 2024-09-16T23:49:02.809 INFO:tasks.workunit.client.1.smithi043.stdout:8/677: read - de/d18/d1f/d2f/d39/f81 zero size 2024-09-16T23:49:02.809 INFO:tasks.workunit.client.1.smithi043.stdout:7/685: rmdir d4/db/d11/d21/d94 39 2024-09-16T23:49:02.810 INFO:tasks.workunit.client.1.smithi043.stdout:2/636: rmdir d8/d11/d16 39 2024-09-16T23:49:02.810 INFO:tasks.workunit.client.1.smithi043.stdout:7/686: chown d4/f19 33600 1 2024-09-16T23:49:02.811 INFO:tasks.workunit.client.1.smithi043.stdout:2/637: stat d8/d4b/d5f/c92 0 2024-09-16T23:49:02.830 INFO:tasks.workunit.client.1.smithi043.stdout:1/855: link d4/d9/d1c/d1e/d42/l67 d4/d34/l128 0 2024-09-16T23:49:02.831 INFO:tasks.workunit.client.1.smithi043.stdout:3/723: rename d4/de/d1f/c9a to d4/d30/d53/d88/db1/dbb/cdc 0 2024-09-16T23:49:02.834 INFO:tasks.workunit.client.1.smithi043.stdout:1/856: dread d4/d9/d12/d15/f104 [0,4194304] 0 2024-09-16T23:49:02.835 INFO:tasks.workunit.client.1.smithi043.stdout:0/774: rename d0/d2/df/d1a/d46/d92/la9 to d0/d2/df/d1a/d49/d80/d88/l105 0 2024-09-16T23:49:02.839 INFO:tasks.workunit.client.1.smithi043.stdout:6/698: link de/d1a/d28/l5c de/le8 0 2024-09-16T23:49:02.840 INFO:tasks.workunit.client.1.smithi043.stdout:2/638: dread d8/d1d/d20/f48 [0,4194304] 0 2024-09-16T23:49:02.841 INFO:tasks.workunit.client.1.smithi043.stdout:0/775: creat d0/d2/df/d1a/d46/d39/d83/dfb/f106 x:0 0 0 2024-09-16T23:49:02.842 INFO:tasks.workunit.client.1.smithi043.stdout:6/699: readlink de/d23/d34/l86 0 2024-09-16T23:49:02.843 INFO:tasks.workunit.client.1.smithi043.stdout:2/639: write d8/d5e/fac [52608,55408] 0 2024-09-16T23:49:02.845 INFO:tasks.workunit.client.1.smithi043.stdout:1/857: dwrite d4/d3e/fdb [0,4194304] 0 2024-09-16T23:49:02.845 INFO:tasks.workunit.client.1.smithi043.stdout:7/687: write d4/db/d11/d15/d29/d7a/f86 [3293909,10252] 0 2024-09-16T23:49:02.848 INFO:tasks.workunit.client.1.smithi043.stdout:6/700: dread de/d23/d34/d66/f8d [0,4194304] 0 2024-09-16T23:49:02.848 INFO:tasks.workunit.client.1.smithi043.stdout:7/688: chown d4/dc0 45100569 1 2024-09-16T23:49:02.848 INFO:tasks.workunit.client.1.smithi043.stdout:8/678: dread de/d18/d1f/d2f/d39/d4d/f5d [0,4194304] 0 2024-09-16T23:49:02.849 INFO:tasks.workunit.client.1.smithi043.stdout:3/724: creat d4/de/d1f/d34/fdd x:0 0 0 2024-09-16T23:49:02.851 INFO:tasks.workunit.client.1.smithi043.stdout:0/776: creat d0/d2/df/d1a/d49/d80/d88/dc4/f107 x:0 0 0 2024-09-16T23:49:02.856 INFO:tasks.workunit.client.1.smithi043.stdout:6/701: dread de/d1a/f38 [0,4194304] 0 2024-09-16T23:49:02.857 INFO:tasks.workunit.client.1.smithi043.stdout:7/689: symlink d4/db/d43/d83/d85/le1 0 2024-09-16T23:49:02.859 INFO:tasks.workunit.client.1.smithi043.stdout:2/640: truncate d8/d4b/d5f/d8e/fbb 3042559 0 2024-09-16T23:49:02.860 INFO:tasks.workunit.client.1.smithi043.stdout:3/725: truncate d4/f19 1222902 0 2024-09-16T23:49:02.861 INFO:tasks.workunit.client.1.smithi043.stdout:6/702: dread - de/dbf/fe1 zero size 2024-09-16T23:49:02.862 INFO:tasks.workunit.client.1.smithi043.stdout:0/777: write d0/d2/df/d1a/d3b/f7d [183157,102899] 0 2024-09-16T23:49:02.863 INFO:tasks.workunit.client.1.smithi043.stdout:6/703: dread - de/dbf/fe4 zero size 2024-09-16T23:49:02.864 INFO:tasks.workunit.client.1.smithi043.stdout:7/690: mkdir d4/d27/d41/d1b/d95/de2 0 2024-09-16T23:49:02.867 INFO:tasks.workunit.client.1.smithi043.stdout:1/858: dread d4/d9/d8f/db6/dcc/dda/d107/f57 [0,4194304] 0 2024-09-16T23:49:02.869 INFO:tasks.workunit.client.1.smithi043.stdout:8/679: truncate de/d18/d1f/f32 5085639 0 2024-09-16T23:49:02.870 INFO:tasks.workunit.client.1.smithi043.stdout:8/680: fsync de/d18/d27/d8e/f7e 0 2024-09-16T23:49:02.874 INFO:tasks.workunit.client.1.smithi043.stdout:0/778: rename d0/d2/df/d1a/d46/d39/d83/dcb/f7e to d0/d2/df/d1a/d46/d39/daa/db7/f108 0 2024-09-16T23:49:02.874 INFO:tasks.workunit.client.1.smithi043.stdout:9/642: sync 2024-09-16T23:49:02.874 INFO:tasks.workunit.client.1.smithi043.stdout:5/860: sync 2024-09-16T23:49:02.876 INFO:tasks.workunit.client.1.smithi043.stdout:6/704: mkdir de/d23/d34/d9b/d8b/de9 0 2024-09-16T23:49:02.878 INFO:tasks.workunit.client.1.smithi043.stdout:2/641: creat d8/d1d/d25/fd9 x:0 0 0 2024-09-16T23:49:02.878 INFO:tasks.workunit.client.1.smithi043.stdout:7/691: dwrite d4/d27/d41/f40 [0,4194304] 0 2024-09-16T23:49:02.880 INFO:tasks.workunit.client.1.smithi043.stdout:2/642: dread - d8/d4b/d5f/d63/db6/fc1 zero size 2024-09-16T23:49:02.880 INFO:tasks.workunit.client.1.smithi043.stdout:7/692: readlink d4/d27/d41/d1b/d75/ld3 0 2024-09-16T23:49:02.885 INFO:tasks.workunit.client.1.smithi043.stdout:6/705: creat de/d23/d34/d9b/d8b/fea x:0 0 0 2024-09-16T23:49:02.886 INFO:tasks.workunit.client.1.smithi043.stdout:5/861: rename d3/db/d15/d4d/d6b/d73/l7d to d3/d9/d22/d4a/d64/d8f/d109/l11f 0 2024-09-16T23:49:02.891 INFO:tasks.workunit.client.1.smithi043.stdout:1/859: rmdir d4/d9/d8f/db6/dfb 0 2024-09-16T23:49:02.893 INFO:tasks.workunit.client.1.smithi043.stdout:7/693: dwrite d4/d27/d41/f8d [0,4194304] 0 2024-09-16T23:49:02.894 INFO:tasks.workunit.client.1.smithi043.stdout:6/706: creat de/d23/d34/d9b/d8b/feb x:0 0 0 2024-09-16T23:49:02.895 INFO:tasks.workunit.client.1.smithi043.stdout:8/681: link de/d70/d3d/daf/cb6 de/d18/d27/d41/d5a/da9/ccc 0 2024-09-16T23:49:02.896 INFO:tasks.workunit.client.1.smithi043.stdout:5/862: mkdir d3/d9/d22/d4a/d4c/d7a/d120 0 2024-09-16T23:49:02.898 INFO:tasks.workunit.client.0.smithi042.stdout:7/622: sync 2024-09-16T23:49:02.898 INFO:tasks.workunit.client.0.smithi042.stdout:9/373: sync 2024-09-16T23:49:02.898 INFO:tasks.workunit.client.0.smithi042.stdout:2/524: sync 2024-09-16T23:49:02.898 INFO:tasks.workunit.client.0.smithi042.stdout:3/674: sync 2024-09-16T23:49:02.898 INFO:tasks.workunit.client.0.smithi042.stdout:0/458: sync 2024-09-16T23:49:02.898 INFO:tasks.workunit.client.0.smithi042.stdout:1/610: sync 2024-09-16T23:49:02.898 INFO:tasks.workunit.client.0.smithi042.stdout:6/531: sync 2024-09-16T23:49:02.899 INFO:tasks.workunit.client.1.smithi043.stdout:9/643: dread d4/d1c/d25/d2b/d4c/f72 [4194304,4194304] 0 2024-09-16T23:49:02.900 INFO:tasks.workunit.client.1.smithi043.stdout:7/694: fdatasync d4/d27/d41/d1b/d95/fb4 0 2024-09-16T23:49:02.901 INFO:tasks.workunit.client.0.smithi042.stdout:3/675: creat d2/d95/fe8 x:0 0 0 2024-09-16T23:49:02.901 INFO:tasks.workunit.client.0.smithi042.stdout:0/459: truncate d0/d15/f7c 921558 0 2024-09-16T23:49:02.901 INFO:tasks.workunit.client.0.smithi042.stdout:6/532: rmdir d8/d70 39 2024-09-16T23:49:02.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/676: chown d2/d1b/d1d/d34 46889872 1 2024-09-16T23:49:02.904 INFO:tasks.workunit.client.0.smithi042.stdout:7/623: creat d3/d98/da0/d32/d6b/d5b/db8/fd7 x:0 0 0 2024-09-16T23:49:02.904 INFO:tasks.workunit.client.0.smithi042.stdout:0/460: rmdir d0/df/d23/d33/d51 39 2024-09-16T23:49:02.905 INFO:tasks.workunit.client.1.smithi043.stdout:1/860: dwrite d4/f69 [0,4194304] 0 2024-09-16T23:49:02.905 INFO:tasks.workunit.client.0.smithi042.stdout:3/677: rmdir d2/d7/d61/d8e/d9d/dc4/dcc 39 2024-09-16T23:49:02.905 INFO:tasks.workunit.client.1.smithi043.stdout:6/707: dread de/d23/d34/d9b/d8b/fb6 [0,4194304] 0 2024-09-16T23:49:02.906 INFO:tasks.workunit.client.0.smithi042.stdout:1/611: chown d0/d9/d90/d6b/d88/f2a 2 1 2024-09-16T23:49:02.906 INFO:tasks.workunit.client.1.smithi043.stdout:1/861: read - d4/d9/d1c/f112 zero size 2024-09-16T23:49:02.908 INFO:tasks.workunit.client.0.smithi042.stdout:3/678: mknod d2/d7/d61/d8e/d9d/ce9 0 2024-09-16T23:49:02.909 INFO:tasks.workunit.client.1.smithi043.stdout:0/779: getdents d0/d2/df/d1a/d46/d39/d83/dfb 0 2024-09-16T23:49:02.911 INFO:tasks.workunit.client.0.smithi042.stdout:0/461: mkdir d0/df/d23/d90 0 2024-09-16T23:49:02.911 INFO:tasks.workunit.client.0.smithi042.stdout:9/374: symlink d4/d12/d2d/d37/d3a/l7b 0 2024-09-16T23:49:02.912 INFO:tasks.workunit.client.0.smithi042.stdout:0/462: dread - d0/f75 zero size 2024-09-16T23:49:02.913 INFO:tasks.workunit.client.1.smithi043.stdout:8/682: symlink de/d18/d1f/d2f/d39/d7b/lcd 0 2024-09-16T23:49:02.914 INFO:tasks.workunit.client.1.smithi043.stdout:8/683: stat de/d18/d1f/d2f/d39/fad 0 2024-09-16T23:49:02.915 INFO:tasks.workunit.client.1.smithi043.stdout:8/684: truncate de/d18/d27/f94 5009660 0 2024-09-16T23:49:02.918 INFO:tasks.workunit.client.1.smithi043.stdout:2/643: getdents d8/d5e/d83/d3a/d77 0 2024-09-16T23:49:02.920 INFO:tasks.workunit.client.0.smithi042.stdout:0/463: symlink d0/df/d13/l91 0 2024-09-16T23:49:02.921 INFO:tasks.workunit.client.0.smithi042.stdout:3/679: dwrite d2/d7/fab [0,4194304] 0 2024-09-16T23:49:02.922 INFO:tasks.workunit.client.0.smithi042.stdout:2/525: dread d0/da/f2c [0,4194304] 0 2024-09-16T23:49:02.924 INFO:tasks.workunit.client.1.smithi043.stdout:5/863: fsync d3/d9/d22/d60/d9d/fd8 0 2024-09-16T23:49:02.925 INFO:tasks.workunit.client.1.smithi043.stdout:4/766: sync 2024-09-16T23:49:02.926 INFO:tasks.workunit.client.0.smithi042.stdout:1/612: symlink d0/d17/dd4/ld7 0 2024-09-16T23:49:02.926 INFO:tasks.workunit.client.0.smithi042.stdout:7/624: dwrite d3/d1f/d49/d76/d19/d63/f70 [0,4194304] 0 2024-09-16T23:49:02.927 INFO:tasks.workunit.client.1.smithi043.stdout:7/695: dread - d4/db/d43/f7f zero size 2024-09-16T23:49:02.930 INFO:tasks.workunit.client.1.smithi043.stdout:9/644: mkdir d4/d1c/d25/d2b/d4c/da3/dca 0 2024-09-16T23:49:02.932 INFO:tasks.workunit.client.0.smithi042.stdout:0/464: link d0/d15/f3c d0/df/d62/f92 0 2024-09-16T23:49:02.933 INFO:tasks.workunit.client.0.smithi042.stdout:0/465: read d0/df/d13/f34 [385454,94600] 0 2024-09-16T23:49:02.934 INFO:tasks.workunit.client.0.smithi042.stdout:3/680: creat d2/d1b/d1d/d34/d50/fea x:0 0 0 2024-09-16T23:49:02.934 INFO:tasks.workunit.client.1.smithi043.stdout:6/708: mkdir de/d23/d63/dec 0 2024-09-16T23:49:02.937 INFO:tasks.workunit.client.1.smithi043.stdout:1/862: creat d4/d9/d8f/db6/dcc/dda/d107/ddf/f129 x:0 0 0 2024-09-16T23:49:02.937 INFO:tasks.workunit.client.0.smithi042.stdout:7/625: rename d3/d98/da0/ca1 to d3/cd8 0 2024-09-16T23:49:02.937 INFO:tasks.workunit.client.0.smithi042.stdout:2/526: symlink d0/da/d17/d3e/lb4 0 2024-09-16T23:49:02.938 INFO:tasks.workunit.client.0.smithi042.stdout:0/466: creat d0/df/d62/f93 x:0 0 0 2024-09-16T23:49:02.938 INFO:tasks.workunit.client.1.smithi043.stdout:0/780: creat d0/d2/df/da7/f109 x:0 0 0 2024-09-16T23:49:02.939 INFO:tasks.workunit.client.0.smithi042.stdout:1/613: mknod d0/d9/d90/d6b/d88/d32/d2f/cd8 0 2024-09-16T23:49:02.939 INFO:tasks.workunit.client.1.smithi043.stdout:0/781: chown d0/d2/df/d1a/d46/f5e 58118647 1 2024-09-16T23:49:02.940 INFO:tasks.workunit.client.0.smithi042.stdout:3/681: write d2/d1b/d89/f45 [1081121,102645] 0 2024-09-16T23:49:02.941 INFO:tasks.workunit.client.0.smithi042.stdout:7/626: creat d3/d98/da0/d32/dcd/db5/fd9 x:0 0 0 2024-09-16T23:49:02.942 INFO:tasks.workunit.client.0.smithi042.stdout:1/614: write d0/d17/f9e [179760,61017] 0 2024-09-16T23:49:02.943 INFO:tasks.workunit.client.0.smithi042.stdout:9/375: rmdir d4/d12/d2d/d37/d59/d73 0 2024-09-16T23:49:02.944 INFO:tasks.workunit.client.1.smithi043.stdout:0/782: dread d0/d2/df/dbb/f9b [0,4194304] 0 2024-09-16T23:49:02.945 INFO:tasks.workunit.client.1.smithi043.stdout:3/726: truncate d4/de/d1f/d2c/f4a 984004 0 2024-09-16T23:49:02.945 INFO:tasks.workunit.client.0.smithi042.stdout:3/682: truncate d2/d7/d61/d8e/fe5 5157024 0 2024-09-16T23:49:02.949 INFO:tasks.workunit.client.0.smithi042.stdout:0/467: rename d0/df/d62 to d0/df/d13/d49/d70/d94 0 2024-09-16T23:49:02.951 INFO:tasks.workunit.client.1.smithi043.stdout:5/864: symlink d3/d9/d22/d4a/d64/l121 0 2024-09-16T23:49:02.952 INFO:tasks.workunit.client.0.smithi042.stdout:9/376: mknod d4/d12/d2d/d37/d28/c7c 0 2024-09-16T23:49:02.952 INFO:tasks.workunit.client.1.smithi043.stdout:3/727: dwrite d4/de/d40/d4d/f78 [0,4194304] 0 2024-09-16T23:49:02.953 INFO:tasks.workunit.client.1.smithi043.stdout:4/767: mknod d8/d15/d20/d2b/d54/dce/cf1 0 2024-09-16T23:49:02.953 INFO:tasks.workunit.client.0.smithi042.stdout:3/683: rename d2/d1b/fac to d2/d7/feb 0 2024-09-16T23:49:02.953 INFO:tasks.workunit.client.0.smithi042.stdout:9/377: stat d4/d12/d2d/d37/d59/c5a 0 2024-09-16T23:49:02.954 INFO:tasks.workunit.client.0.smithi042.stdout:6/533: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/fab x:0 0 0 2024-09-16T23:49:02.958 INFO:tasks.workunit.client.0.smithi042.stdout:9/378: creat d4/d12/d2d/d37/d3a/d50/f7d x:0 0 0 2024-09-16T23:49:02.958 INFO:tasks.workunit.client.0.smithi042.stdout:0/468: link d0/df/c17 d0/df/d13/d49/c95 0 2024-09-16T23:49:02.959 INFO:tasks.workunit.client.1.smithi043.stdout:3/728: write d4/d30/fa6 [1980524,22389] 0 2024-09-16T23:49:02.959 INFO:tasks.workunit.client.0.smithi042.stdout:3/684: dread d2/d1b/d1d/d34/f5d [0,4194304] 0 2024-09-16T23:49:02.959 INFO:tasks.workunit.client.1.smithi043.stdout:6/709: mknod de/d23/d34/d9b/d5a/ced 0 2024-09-16T23:49:02.960 INFO:tasks.workunit.client.0.smithi042.stdout:1/615: creat d0/d9/fd9 x:0 0 0 2024-09-16T23:49:02.961 INFO:tasks.workunit.client.1.smithi043.stdout:8/685: dwrite de/f45 [0,4194304] 0 2024-09-16T23:49:02.961 INFO:tasks.workunit.client.0.smithi042.stdout:0/469: write d0/f4b [1600435,104265] 0 2024-09-16T23:49:02.961 INFO:tasks.workunit.client.1.smithi043.stdout:1/863: mkdir d4/d3e/d5b/dca/d12a 0 2024-09-16T23:49:02.963 INFO:tasks.workunit.client.1.smithi043.stdout:8/686: chown de/d18/d27/d41/d5a/d37/d6a/lc4 87599547 1 2024-09-16T23:49:02.965 INFO:tasks.workunit.client.0.smithi042.stdout:3/685: write d2/d7/d61/d8e/f3f [2352265,92927] 0 2024-09-16T23:49:02.965 INFO:tasks.workunit.client.0.smithi042.stdout:0/470: mknod d0/df/c96 0 2024-09-16T23:49:02.968 INFO:tasks.workunit.client.0.smithi042.stdout:2/527: dread d0/d26/f2f [0,4194304] 0 2024-09-16T23:49:02.970 INFO:tasks.workunit.client.0.smithi042.stdout:2/528: dread - d0/da/d88/dae/f75 zero size 2024-09-16T23:49:02.970 INFO:tasks.workunit.client.1.smithi043.stdout:8/687: dwrite de/fb0 [0,4194304] 0 2024-09-16T23:49:02.972 INFO:tasks.workunit.client.1.smithi043.stdout:0/783: mkdir d0/d2/df/d1a/d46/d92/d10a 0 2024-09-16T23:49:02.973 INFO:tasks.workunit.client.0.smithi042.stdout:0/471: creat d0/df/d13/f97 x:0 0 0 2024-09-16T23:49:02.975 INFO:tasks.workunit.client.0.smithi042.stdout:0/472: chown d0/df/d23/d33/d56/f5d 22 1 2024-09-16T23:49:02.975 INFO:tasks.workunit.client.0.smithi042.stdout:6/534: rename d8/d70/d81/f82 to d8/d70/d81/dd/d1b/d2d/d87/fac 0 2024-09-16T23:49:02.976 INFO:tasks.workunit.client.0.smithi042.stdout:0/473: chown d0/df/f4a 945 1 2024-09-16T23:49:02.977 INFO:tasks.workunit.client.0.smithi042.stdout:3/686: dread d2/d1b/d89/f23 [0,4194304] 0 2024-09-16T23:49:02.980 INFO:tasks.workunit.client.1.smithi043.stdout:2/644: creat d8/d1d/d20/d23/d38/da2/fda x:0 0 0 2024-09-16T23:49:02.981 INFO:tasks.workunit.client.0.smithi042.stdout:0/474: creat d0/df/d13/d49/d70/f98 x:0 0 0 2024-09-16T23:49:02.983 INFO:tasks.workunit.client.1.smithi043.stdout:7/696: dwrite d4/d27/d41/d1b/f44 [0,4194304] 0 2024-09-16T23:49:02.983 INFO:tasks.workunit.client.1.smithi043.stdout:1/864: dread d4/df/f91 [0,4194304] 0 2024-09-16T23:49:02.986 INFO:tasks.workunit.client.0.smithi042.stdout:2/529: creat d0/da/d1b/d48/d81/fb5 x:0 0 0 2024-09-16T23:49:02.987 INFO:tasks.workunit.client.1.smithi043.stdout:0/784: dwrite d0/d2/df/d1a/d49/d80/d88/fa4 [0,4194304] 0 2024-09-16T23:49:02.992 INFO:tasks.workunit.client.0.smithi042.stdout:6/535: dwrite d8/d70/d81/dd/d1b/d34/f64 [0,4194304] 0 2024-09-16T23:49:02.992 INFO:tasks.workunit.client.0.smithi042.stdout:3/687: dwrite d2/d1b/d1d/d34/d50/d93/d96/fc6 [4194304,4194304] 0 2024-09-16T23:49:02.993 INFO:tasks.workunit.client.1.smithi043.stdout:9/645: symlink d4/d1c/d6b/d67/dbf/lcb 0 2024-09-16T23:49:02.993 INFO:tasks.workunit.client.1.smithi043.stdout:9/646: readlink d4/d21/d2a/l82 0 2024-09-16T23:49:02.994 INFO:tasks.workunit.client.0.smithi042.stdout:6/536: dread d8/d70/d81/dd/d1b/d34/d39/d5e/f83 [0,4194304] 0 2024-09-16T23:49:02.994 INFO:tasks.workunit.client.1.smithi043.stdout:4/768: dread d8/d15/d9b/f86 [0,4194304] 0 2024-09-16T23:49:02.994 INFO:tasks.workunit.client.1.smithi043.stdout:3/729: dread d4/de/d1f/d2c/db7/dc3/f8c [0,4194304] 0 2024-09-16T23:49:02.996 INFO:tasks.workunit.client.0.smithi042.stdout:1/616: link d0/d9/d90/d6b/d88/d32/d2f/d98/ca5 d0/d9/d26/cda 0 2024-09-16T23:49:02.996 INFO:tasks.workunit.client.1.smithi043.stdout:3/730: write d4/de/d1f/d2c/f47 [5017879,93655] 0 2024-09-16T23:49:02.997 INFO:tasks.workunit.client.1.smithi043.stdout:3/731: chown d4/f3b 128 1 2024-09-16T23:49:02.999 INFO:tasks.workunit.client.0.smithi042.stdout:6/537: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/f7c 961510 0 2024-09-16T23:49:03.002 INFO:tasks.workunit.client.0.smithi042.stdout:6/538: write d8/d70/d81/dd/d1b/d67/d97/f9a [621509,88146] 0 2024-09-16T23:49:03.004 INFO:tasks.workunit.client.0.smithi042.stdout:0/475: rename d0/df/d13/d49/c83 to d0/df/d23/d33/c99 0 2024-09-16T23:49:03.007 INFO:tasks.workunit.client.0.smithi042.stdout:9/379: getdents d4/d12/d2d 0 2024-09-16T23:49:03.007 INFO:tasks.workunit.client.0.smithi042.stdout:3/688: getdents d2/d7/d61/d8e 0 2024-09-16T23:49:03.008 INFO:tasks.workunit.client.0.smithi042.stdout:3/689: chown d2/d7/d61/d8e/f3f 22277387 1 2024-09-16T23:49:03.010 INFO:tasks.workunit.client.0.smithi042.stdout:2/530: fsync d0/f1a 0 2024-09-16T23:49:03.011 INFO:tasks.workunit.client.1.smithi043.stdout:6/710: dread de/f7a [0,4194304] 0 2024-09-16T23:49:03.015 INFO:tasks.workunit.client.0.smithi042.stdout:3/690: creat d2/d7/d61/d8e/d9d/dc4/de4/fec x:0 0 0 2024-09-16T23:49:03.015 INFO:tasks.workunit.client.0.smithi042.stdout:6/539: unlink d8/d70/d84/f9b 0 2024-09-16T23:49:03.018 INFO:tasks.workunit.client.0.smithi042.stdout:9/380: dwrite d4/d12/d2d/d37/d28/f65 [0,4194304] 0 2024-09-16T23:49:03.019 INFO:tasks.workunit.client.0.smithi042.stdout:1/617: mkdir d0/ddb 0 2024-09-16T23:49:03.020 INFO:tasks.workunit.client.0.smithi042.stdout:0/476: dwrite d0/df/d23/d33/d56/f5d [0,4194304] 0 2024-09-16T23:49:03.020 INFO:tasks.workunit.client.0.smithi042.stdout:9/381: readlink d4/d12/l2f 0 2024-09-16T23:49:03.023 INFO:tasks.workunit.client.1.smithi043.stdout:8/688: dread de/d18/d27/d41/d5a/f4a [0,4194304] 0 2024-09-16T23:49:03.025 INFO:tasks.workunit.client.1.smithi043.stdout:7/697: creat d4/db/d11/d15/d29/d7a/fe3 x:0 0 0 2024-09-16T23:49:03.027 INFO:tasks.workunit.client.1.smithi043.stdout:5/865: getdents d3/db/d84/db6/d10c 0 2024-09-16T23:49:03.028 INFO:tasks.workunit.client.0.smithi042.stdout:2/531: read - d0/da/d1b/d3b/d44/f90 zero size 2024-09-16T23:49:03.029 INFO:tasks.workunit.client.0.smithi042.stdout:3/691: dwrite d2/f4a [0,4194304] 0 2024-09-16T23:49:03.030 INFO:tasks.workunit.client.0.smithi042.stdout:6/540: fdatasync d8/d70/d81/dd/d1b/f4b 0 2024-09-16T23:49:03.032 INFO:tasks.workunit.client.0.smithi042.stdout:3/692: stat d2/d1b/d1d/d34/d62/d6a/fa5 0 2024-09-16T23:49:03.033 INFO:tasks.workunit.client.0.smithi042.stdout:1/618: creat d0/d9/d90/fdc x:0 0 0 2024-09-16T23:49:03.035 INFO:tasks.workunit.client.0.smithi042.stdout:0/477: link d0/df/d13/d2d/d35/c4d d0/d48/c9a 0 2024-09-16T23:49:03.037 INFO:tasks.workunit.client.0.smithi042.stdout:9/382: dwrite d4/d10/f6b [0,4194304] 0 2024-09-16T23:49:03.038 INFO:tasks.workunit.client.1.smithi043.stdout:1/865: write d4/d9/d1c/d1e/f55 [4398306,89034] 0 2024-09-16T23:49:03.039 INFO:tasks.workunit.client.1.smithi043.stdout:6/711: fsync de/d23/d49/fb5 0 2024-09-16T23:49:03.039 INFO:tasks.workunit.client.1.smithi043.stdout:1/866: fdatasync d4/df/df4/ffc 0 2024-09-16T23:49:03.040 INFO:tasks.workunit.client.1.smithi043.stdout:1/867: stat d4/d9/d12/d15/dac/l117 0 2024-09-16T23:49:03.041 INFO:tasks.workunit.client.0.smithi042.stdout:9/383: readlink d4/d12/d2d/d37/l26 0 2024-09-16T23:49:03.043 INFO:tasks.workunit.client.0.smithi042.stdout:3/693: unlink d2/d1b/d1d/c1c 0 2024-09-16T23:49:03.047 INFO:tasks.workunit.client.0.smithi042.stdout:1/619: dwrite d0/d9a/fb5 [0,4194304] 0 2024-09-16T23:49:03.049 INFO:tasks.workunit.client.1.smithi043.stdout:7/698: creat d4/db/d11/d15/fe4 x:0 0 0 2024-09-16T23:49:03.051 INFO:tasks.workunit.client.1.smithi043.stdout:4/769: write d8/d15/f69 [3893079,82269] 0 2024-09-16T23:49:03.051 INFO:tasks.workunit.client.0.smithi042.stdout:0/478: getdents d0/df/d23/d33/d56 0 2024-09-16T23:49:03.052 INFO:tasks.workunit.client.0.smithi042.stdout:9/384: rmdir d4/d12/d60 39 2024-09-16T23:49:03.054 INFO:tasks.workunit.client.1.smithi043.stdout:9/647: getdents d4/d28/d49 0 2024-09-16T23:49:03.055 INFO:tasks.workunit.client.0.smithi042.stdout:1/620: truncate d0/f4a 104933 0 2024-09-16T23:49:03.057 INFO:tasks.workunit.client.0.smithi042.stdout:6/541: dwrite d8/d70/d81/dd/d2c/d57/f77 [4194304,4194304] 0 2024-09-16T23:49:03.058 INFO:tasks.workunit.client.0.smithi042.stdout:0/479: stat d0/df/d13/d49/d70/c7e 0 2024-09-16T23:49:03.064 INFO:tasks.workunit.client.0.smithi042.stdout:1/621: rmdir d0/d9/d90/d6b/d88 39 2024-09-16T23:49:03.064 INFO:tasks.workunit.client.1.smithi043.stdout:2/645: dwrite d8/d11/d16/f28 [0,4194304] 0 2024-09-16T23:49:03.066 INFO:tasks.workunit.client.1.smithi043.stdout:2/646: chown d8/d5e/d83/d3a/d77/ca1 24 1 2024-09-16T23:49:03.069 INFO:tasks.workunit.client.0.smithi042.stdout:1/622: write d0/d17/f21 [2105829,59975] 0 2024-09-16T23:49:03.070 INFO:tasks.workunit.client.1.smithi043.stdout:0/785: rename d0/de/l6f to d0/d2/df/d1a/l10b 0 2024-09-16T23:49:03.071 INFO:tasks.workunit.client.0.smithi042.stdout:3/694: dread d2/d1b/d1d/d34/f58 [0,4194304] 0 2024-09-16T23:49:03.071 INFO:tasks.workunit.client.0.smithi042.stdout:9/385: dread d4/f5 [0,4194304] 0 2024-09-16T23:49:03.071 INFO:tasks.workunit.client.0.smithi042.stdout:2/532: dwrite d0/d26/f7c [0,4194304] 0 2024-09-16T23:49:03.074 INFO:tasks.workunit.client.0.smithi042.stdout:4/731: sync 2024-09-16T23:49:03.075 INFO:tasks.workunit.client.0.smithi042.stdout:6/542: truncate d8/d38/f74 2550227 0 2024-09-16T23:49:03.076 INFO:tasks.workunit.client.1.smithi043.stdout:7/699: dread d4/d27/d41/d1b/d66/f8c [0,4194304] 0 2024-09-16T23:49:03.076 INFO:tasks.workunit.client.0.smithi042.stdout:0/480: dread d0/df/d13/f74 [0,4194304] 0 2024-09-16T23:49:03.082 INFO:tasks.workunit.client.0.smithi042.stdout:3/695: mknod d2/d1b/d1d/d34/d50/ced 0 2024-09-16T23:49:03.086 INFO:tasks.workunit.client.0.smithi042.stdout:4/732: link de/d42/d68/f72 de/d1a/d70/f101 0 2024-09-16T23:49:03.087 INFO:tasks.workunit.client.1.smithi043.stdout:8/689: dwrite de/d18/d27/f85 [0,4194304] 0 2024-09-16T23:49:03.088 INFO:tasks.workunit.client.1.smithi043.stdout:4/770: creat d8/d15/d9b/d8b/dde/ff2 x:0 0 0 2024-09-16T23:49:03.089 INFO:tasks.workunit.client.1.smithi043.stdout:1/868: dwrite d4/fc [4194304,4194304] 0 2024-09-16T23:49:03.089 INFO:tasks.workunit.client.0.smithi042.stdout:3/696: symlink d2/d95/dd8/lee 0 2024-09-16T23:49:03.090 INFO:tasks.workunit.client.1.smithi043.stdout:3/732: rename d4/de/cb4 to d4/de/d5d/cde 0 2024-09-16T23:49:03.090 INFO:tasks.workunit.client.0.smithi042.stdout:1/623: dwrite d0/d17/d79/fa2 [0,4194304] 0 2024-09-16T23:49:03.090 INFO:tasks.workunit.client.0.smithi042.stdout:6/543: unlink l4 0 2024-09-16T23:49:03.094 INFO:tasks.workunit.client.1.smithi043.stdout:2/647: mkdir d8/d5e/d83/ddb 0 2024-09-16T23:49:03.094 INFO:tasks.workunit.client.1.smithi043.stdout:5/866: dread f2 [0,4194304] 0 2024-09-16T23:49:03.095 INFO:tasks.workunit.client.1.smithi043.stdout:8/690: chown de/d18/d1f/d2f/d39/d7b/fba 547 1 2024-09-16T23:49:03.097 INFO:tasks.workunit.client.1.smithi043.stdout:7/700: creat d4/d27/dbc/fe5 x:0 0 0 2024-09-16T23:49:03.098 INFO:tasks.workunit.client.0.smithi042.stdout:0/481: dwrite d0/df/f18 [0,4194304] 0 2024-09-16T23:49:03.099 INFO:tasks.workunit.client.0.smithi042.stdout:2/533: dread d0/d26/f53 [0,4194304] 0 2024-09-16T23:49:03.100 INFO:tasks.workunit.client.1.smithi043.stdout:0/786: dwrite d0/d2/df/d1a/d46/d39/d83/ff2 [0,4194304] 0 2024-09-16T23:49:03.102 INFO:tasks.workunit.client.0.smithi042.stdout:3/697: mkdir d2/d1b/d1d/d34/d62/d80/def 0 2024-09-16T23:49:03.104 INFO:tasks.workunit.client.1.smithi043.stdout:4/771: rmdir d8/d25/d39/dd3 39 2024-09-16T23:49:03.109 INFO:tasks.workunit.client.1.smithi043.stdout:2/648: creat d8/d35/d9f/fdc x:0 0 0 2024-09-16T23:49:03.110 INFO:tasks.workunit.client.0.smithi042.stdout:4/733: dwrite de/d2e/d37/d3e/f7f [0,4194304] 0 2024-09-16T23:49:03.113 INFO:tasks.workunit.client.0.smithi042.stdout:2/534: chown d0/da/d1b/d3b/d44/d54/c7a 1244131 1 2024-09-16T23:49:03.113 INFO:tasks.workunit.client.0.smithi042.stdout:1/624: dwrite d0/d17/d18/f4d [0,4194304] 0 2024-09-16T23:49:03.113 INFO:tasks.workunit.client.0.smithi042.stdout:3/698: mknod d2/d1b/d89/db4/cf0 0 2024-09-16T23:49:03.113 INFO:tasks.workunit.client.1.smithi043.stdout:4/772: chown d8/d15/d20/d2b/d23/fd1 3469917 1 2024-09-16T23:49:03.114 INFO:tasks.workunit.client.1.smithi043.stdout:1/869: dwrite d4/df/d26/d79/fd7 [0,4194304] 0 2024-09-16T23:49:03.115 INFO:tasks.workunit.client.1.smithi043.stdout:6/712: rename de/d1a/d80/l81 to de/d23/d34/d9b/d50/lee 0 2024-09-16T23:49:03.117 INFO:tasks.workunit.client.1.smithi043.stdout:9/648: rename d4 to d4/d28/d49/dcc 22 2024-09-16T23:49:03.118 INFO:tasks.workunit.client.1.smithi043.stdout:4/773: stat d8/d15/d20/d2b/d54/d55/f94 0 2024-09-16T23:49:03.119 INFO:tasks.workunit.client.1.smithi043.stdout:3/733: dread d4/de/d1f/d2c/db7/dc3/f8c [0,4194304] 0 2024-09-16T23:49:03.121 INFO:tasks.workunit.client.0.smithi042.stdout:6/544: fdatasync d8/d70/d81/f23 0 2024-09-16T23:49:03.122 INFO:tasks.workunit.client.1.smithi043.stdout:7/701: mknod d4/db/d11/d15/d29/da7/ce6 0 2024-09-16T23:49:03.122 INFO:tasks.workunit.client.0.smithi042.stdout:4/734: write de/d2e/d37/d3e/fd1 [674693,28677] 0 2024-09-16T23:49:03.123 INFO:tasks.workunit.client.1.smithi043.stdout:5/867: mkdir d3/db/d19/d122 0 2024-09-16T23:49:03.123 INFO:tasks.workunit.client.0.smithi042.stdout:3/699: readlink d2/d95/dd8/le7 0 2024-09-16T23:49:03.126 INFO:tasks.workunit.client.1.smithi043.stdout:8/691: mkdir de/d18/d1f/d2f/d39/d7b/d83/dbb/dce 0 2024-09-16T23:49:03.126 INFO:tasks.workunit.client.1.smithi043.stdout:8/692: stat de/d18/d27/d41/d5a/fac 0 2024-09-16T23:49:03.127 INFO:tasks.workunit.client.1.smithi043.stdout:8/693: chown de/d18/d27/d41 36 1 2024-09-16T23:49:03.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:02 smithi043 ceph-mon[35191]: pgmap v10: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 74 MiB/s rd, 232 MiB/s wr, 448 op/s 2024-09-16T23:49:03.128 INFO:tasks.workunit.client.0.smithi042.stdout:0/482: dwrite d0/df/f18 [0,4194304] 0 2024-09-16T23:49:03.128 INFO:tasks.workunit.client.0.smithi042.stdout:3/700: rename d2/d7 to d2/d7/d61/d8e/d9d/dc4/df1 22 2024-09-16T23:49:03.129 INFO:tasks.workunit.client.0.smithi042.stdout:3/701: readlink d2/d1b/d1d/d34/l83 0 2024-09-16T23:49:03.131 INFO:tasks.workunit.client.0.smithi042.stdout:3/702: stat d2/d95/dd8/cdf 0 2024-09-16T23:49:03.132 INFO:tasks.workunit.client.1.smithi043.stdout:6/713: creat de/d23/d34/d9b/d5a/da3/fef x:0 0 0 2024-09-16T23:49:03.132 INFO:tasks.workunit.client.0.smithi042.stdout:3/703: write d2/d1b/d89/f47 [68764,22877] 0 2024-09-16T23:49:03.133 INFO:tasks.workunit.client.0.smithi042.stdout:4/735: rename de/d42/d85/cba to de/d1a/da7/db9/c102 0 2024-09-16T23:49:03.134 INFO:tasks.workunit.client.0.smithi042.stdout:6/545: dread d8/d70/d81/dd/d1b/d34/d39/d5e/f7c [0,4194304] 0 2024-09-16T23:49:03.134 INFO:tasks.workunit.client.0.smithi042.stdout:0/483: mknod d0/df/d23/c9b 0 2024-09-16T23:49:03.134 INFO:tasks.workunit.client.1.smithi043.stdout:5/868: creat d3/d1b/dde/f123 x:0 0 0 2024-09-16T23:49:03.137 INFO:tasks.workunit.client.0.smithi042.stdout:4/736: write de/d1a/da7/f56 [5180835,88718] 0 2024-09-16T23:49:03.137 INFO:tasks.workunit.client.0.smithi042.stdout:3/704: write d2/d1b/d1d/d34/d62/d80/faf [1240919,130518] 0 2024-09-16T23:49:03.139 INFO:tasks.workunit.client.0.smithi042.stdout:0/484: creat d0/d48/f9c x:0 0 0 2024-09-16T23:49:03.141 INFO:tasks.workunit.client.0.smithi042.stdout:3/705: write d2/d95/fe8 [834356,20123] 0 2024-09-16T23:49:03.141 INFO:tasks.workunit.client.0.smithi042.stdout:6/546: write d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f79 [196417,88799] 0 2024-09-16T23:49:03.142 INFO:tasks.workunit.client.0.smithi042.stdout:4/737: truncate de/d12/d1b/f36 692827 0 2024-09-16T23:49:03.145 INFO:tasks.workunit.client.0.smithi042.stdout:6/547: write d8/d70/d81/dd/d1b/d34/f64 [1390465,34408] 0 2024-09-16T23:49:03.147 INFO:tasks.workunit.client.0.smithi042.stdout:9/386: link d4/d12/d2d/d37/d3a/d50/c53 d4/c7e 0 2024-09-16T23:49:03.147 INFO:tasks.workunit.client.0.smithi042.stdout:3/706: rename d2/d1b/d1d/d34/d62/d80/faf to d2/d7/ff2 0 2024-09-16T23:49:03.148 INFO:tasks.workunit.client.1.smithi043.stdout:4/774: dread d8/f3b [4194304,4194304] 0 2024-09-16T23:49:03.149 INFO:tasks.workunit.client.0.smithi042.stdout:4/738: mknod de/d2e/d37/d3e/c103 0 2024-09-16T23:49:03.150 INFO:tasks.workunit.client.0.smithi042.stdout:6/548: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/c6c 89284100 1 2024-09-16T23:49:03.150 INFO:tasks.workunit.client.1.smithi043.stdout:3/734: rename d4/d30/d53/l70 to d4/d30/ldf 0 2024-09-16T23:49:03.152 INFO:tasks.workunit.client.0.smithi042.stdout:6/549: stat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e 0 2024-09-16T23:49:03.152 INFO:tasks.workunit.client.0.smithi042.stdout:3/707: unlink d2/d7/d61/d8e/d2c/l72 0 2024-09-16T23:49:03.153 INFO:tasks.workunit.client.1.smithi043.stdout:2/649: link d8/d11/d16/f28 d8/d1d/d20/d23/d38/da2/fdd 0 2024-09-16T23:49:03.156 INFO:tasks.workunit.client.1.smithi043.stdout:9/649: creat d4/d1c/d25/db1/fcd x:0 0 0 2024-09-16T23:49:03.158 INFO:tasks.workunit.client.0.smithi042.stdout:4/739: dwrite de/d1a/da7/f56 [0,4194304] 0 2024-09-16T23:49:03.159 INFO:tasks.workunit.client.1.smithi043.stdout:6/714: rmdir de/d23/d63 39 2024-09-16T23:49:03.161 INFO:tasks.workunit.client.0.smithi042.stdout:4/740: write de/d1a/f39 [2508407,58613] 0 2024-09-16T23:49:03.164 INFO:tasks.workunit.client.0.smithi042.stdout:4/741: symlink de/d42/d82/l104 0 2024-09-16T23:49:03.165 INFO:tasks.workunit.client.0.smithi042.stdout:1/625: mkdir d0/ddb/ddd 0 2024-09-16T23:49:03.166 INFO:tasks.workunit.client.1.smithi043.stdout:0/787: write d0/d2/df/f18 [145962,60292] 0 2024-09-16T23:49:03.167 INFO:tasks.workunit.client.1.smithi043.stdout:5/869: creat d3/d1b/f124 x:0 0 0 2024-09-16T23:49:03.168 INFO:tasks.workunit.client.0.smithi042.stdout:9/387: mknod d4/d12/d2d/d37/d59/c7f 0 2024-09-16T23:49:03.168 INFO:tasks.workunit.client.0.smithi042.stdout:4/742: truncate de/d12/f54 148512 0 2024-09-16T23:49:03.169 INFO:tasks.workunit.client.1.smithi043.stdout:0/788: truncate d0/d2/df/d1a/d49/d80/ffe 543842 0 2024-09-16T23:49:03.172 INFO:tasks.workunit.client.1.smithi043.stdout:2/650: creat d8/d35/fde x:0 0 0 2024-09-16T23:49:03.173 INFO:tasks.workunit.client.1.smithi043.stdout:2/651: dread - d8/d4b/fcc zero size 2024-09-16T23:49:03.174 INFO:tasks.workunit.client.0.smithi042.stdout:0/485: truncate d0/f7 2889144 0 2024-09-16T23:49:03.176 INFO:tasks.workunit.client.0.smithi042.stdout:3/708: truncate d2/f1f 4054975 0 2024-09-16T23:49:03.176 INFO:tasks.workunit.client.1.smithi043.stdout:9/650: rename d4/d1c/d25/d2b/d4c/da3/db4 to d4/d5/dce 0 2024-09-16T23:49:03.178 INFO:tasks.workunit.client.0.smithi042.stdout:9/388: dwrite d4/d12/d2d/d37/f70 [0,4194304] 0 2024-09-16T23:49:03.180 INFO:tasks.workunit.client.0.smithi042.stdout:3/709: dread - d2/d7/d61/db9/fe3 zero size 2024-09-16T23:49:03.181 INFO:tasks.workunit.client.1.smithi043.stdout:1/870: truncate d4/df/d26/f39 2599120 0 2024-09-16T23:49:03.181 INFO:tasks.workunit.client.1.smithi043.stdout:5/870: dread d3/d1b/f119 [0,4194304] 0 2024-09-16T23:49:03.183 INFO:tasks.workunit.client.0.smithi042.stdout:9/389: dread - d4/d12/d16/d68/f71 zero size 2024-09-16T23:49:03.185 INFO:tasks.workunit.client.1.smithi043.stdout:8/694: write de/fb0 [5163786,81541] 0 2024-09-16T23:49:03.186 INFO:tasks.workunit.client.0.smithi042.stdout:6/550: mknod d8/d70/d81/dd/d1b/cad 0 2024-09-16T23:49:03.186 INFO:tasks.workunit.client.0.smithi042.stdout:3/710: symlink d2/lf3 0 2024-09-16T23:49:03.187 INFO:tasks.workunit.client.0.smithi042.stdout:4/743: dread de/f10 [4194304,4194304] 0 2024-09-16T23:49:03.189 INFO:tasks.workunit.client.0.smithi042.stdout:0/486: truncate d0/df/f18 3247487 0 2024-09-16T23:49:03.191 INFO:tasks.workunit.client.0.smithi042.stdout:4/744: truncate de/d42/d82/fa5 1053128 0 2024-09-16T23:49:03.192 INFO:tasks.workunit.client.0.smithi042.stdout:0/487: write d0/f75 [291294,123622] 0 2024-09-16T23:49:03.195 INFO:tasks.workunit.client.0.smithi042.stdout:6/551: symlink d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/lae 0 2024-09-16T23:49:03.196 INFO:tasks.workunit.client.0.smithi042.stdout:4/745: unlink de/d42/d68/l81 0 2024-09-16T23:49:03.199 INFO:tasks.workunit.client.1.smithi043.stdout:7/702: truncate d4/d27/d41/f40 349317 0 2024-09-16T23:49:03.200 INFO:tasks.workunit.client.0.smithi042.stdout:2/535: dwrite d0/d26/f2f [0,4194304] 0 2024-09-16T23:49:03.202 INFO:tasks.workunit.client.1.smithi043.stdout:5/871: unlink d3/d1b/f119 0 2024-09-16T23:49:03.204 INFO:tasks.workunit.client.1.smithi043.stdout:1/871: dwrite d4/df/df4/d83/d98/fb8 [0,4194304] 0 2024-09-16T23:49:03.205 INFO:tasks.workunit.client.0.smithi042.stdout:3/711: dread d2/d1b/fdc [0,4194304] 0 2024-09-16T23:49:03.205 INFO:tasks.workunit.client.0.smithi042.stdout:0/488: mknod d0/df/d23/d33/d51/d79/c9d 0 2024-09-16T23:49:03.205 INFO:tasks.workunit.client.0.smithi042.stdout:3/712: stat d2/d7/d61/d73/fde 0 2024-09-16T23:49:03.206 INFO:tasks.workunit.client.0.smithi042.stdout:3/713: stat d2/d1b/d1d/d34/d62/d6a/fb1 0 2024-09-16T23:49:03.207 INFO:tasks.workunit.client.0.smithi042.stdout:9/390: rename d4/d12/c1a to d4/d12/d2d/d37/d28/d5e/c80 0 2024-09-16T23:49:03.208 INFO:tasks.workunit.client.0.smithi042.stdout:9/391: chown d4/d12/d44 362844771 1 2024-09-16T23:49:03.208 INFO:tasks.workunit.client.1.smithi043.stdout:4/775: write d8/d15/d20/d2b/f5a [637841,73853] 0 2024-09-16T23:49:03.209 INFO:tasks.workunit.client.0.smithi042.stdout:0/489: symlink d0/df/d23/d33/d51/l9e 0 2024-09-16T23:49:03.210 INFO:tasks.workunit.client.0.smithi042.stdout:1/626: creat d0/d9/d90/d6b/d88/d32/fde x:0 0 0 2024-09-16T23:49:03.210 INFO:tasks.workunit.client.1.smithi043.stdout:4/776: readlink d8/d15/d20/lef 0 2024-09-16T23:49:03.210 INFO:tasks.workunit.client.1.smithi043.stdout:4/777: dread - d8/f68 zero size 2024-09-16T23:49:03.210 INFO:tasks.workunit.client.0.smithi042.stdout:0/490: write d0/d15/f7c [657910,119325] 0 2024-09-16T23:49:03.215 INFO:tasks.workunit.client.0.smithi042.stdout:6/552: rename d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/lae to d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/laf 0 2024-09-16T23:49:03.217 INFO:tasks.workunit.client.0.smithi042.stdout:0/491: mkdir d0/df/d13/d2d/d35/d9f 0 2024-09-16T23:49:03.217 INFO:tasks.workunit.client.1.smithi043.stdout:4/778: dwrite d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:49:03.218 INFO:tasks.workunit.client.1.smithi043.stdout:9/651: mkdir d4/d28/dcf 0 2024-09-16T23:49:03.219 INFO:tasks.workunit.client.1.smithi043.stdout:4/779: write d8/d25/d39/d58/fca [363974,67881] 0 2024-09-16T23:49:03.222 INFO:tasks.workunit.client.0.smithi042.stdout:2/536: rename d0/da/d88/dae/l9e to d0/d98/lb6 0 2024-09-16T23:49:03.223 INFO:tasks.workunit.client.0.smithi042.stdout:3/714: truncate d2/d1b/f94 420171 0 2024-09-16T23:49:03.227 INFO:tasks.workunit.client.0.smithi042.stdout:4/746: dwrite de/d2e/d37/d3e/f40 [0,4194304] 0 2024-09-16T23:49:03.229 INFO:tasks.workunit.client.0.smithi042.stdout:6/553: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f51 16773721 1 2024-09-16T23:49:03.229 INFO:tasks.workunit.client.0.smithi042.stdout:4/747: chown c9 283330 1 2024-09-16T23:49:03.230 INFO:tasks.workunit.client.0.smithi042.stdout:1/627: dwrite d0/d9/d26/d8b/f99 [0,4194304] 0 2024-09-16T23:49:03.231 INFO:tasks.workunit.client.0.smithi042.stdout:2/537: read d0/d26/f9f [2928673,73827] 0 2024-09-16T23:49:03.231 INFO:tasks.workunit.client.0.smithi042.stdout:9/392: fdatasync d4/d12/f1e 0 2024-09-16T23:49:03.232 INFO:tasks.workunit.client.1.smithi043.stdout:0/789: write d0/d2/df/d1a/d3b/f5d [796902,111723] 0 2024-09-16T23:49:03.233 INFO:tasks.workunit.client.0.smithi042.stdout:3/715: write d2/d7/d61/d8e/d2c/f64 [1176918,111154] 0 2024-09-16T23:49:03.237 INFO:tasks.workunit.client.0.smithi042.stdout:3/716: chown d2/d1b/d1d/d34/d62/d80/def 2044 1 2024-09-16T23:49:03.238 INFO:tasks.workunit.client.0.smithi042.stdout:6/554: write d8/d70/d81/dd/d2c/f31 [2540455,42365] 0 2024-09-16T23:49:03.238 INFO:tasks.workunit.client.1.smithi043.stdout:1/872: mkdir d4/df/d26/dd5/d12b 0 2024-09-16T23:49:03.239 INFO:tasks.workunit.client.0.smithi042.stdout:9/393: chown d4/d12/d2d/d37/d28/f31 169595555 1 2024-09-16T23:49:03.241 INFO:tasks.workunit.client.1.smithi043.stdout:8/695: write de/d18/d1f/d2f/d39/d7b/d83/f84 [3936645,126887] 0 2024-09-16T23:49:03.243 INFO:tasks.workunit.client.1.smithi043.stdout:0/790: dwrite d0/d2/df/d1a/d3b/dee/df0/df8/ffd [0,4194304] 0 2024-09-16T23:49:03.244 INFO:tasks.workunit.client.0.smithi042.stdout:4/748: link de/d42/d82/d94/fa3 de/d2e/d37/d4c/f105 0 2024-09-16T23:49:03.244 INFO:tasks.workunit.client.0.smithi042.stdout:0/492: dwrite d0/df/f10 [0,4194304] 0 2024-09-16T23:49:03.244 INFO:tasks.workunit.client.0.smithi042.stdout:9/394: chown d4/d12/d2d/d37/d3a/f48 90454645 1 2024-09-16T23:49:03.244 INFO:tasks.workunit.client.1.smithi043.stdout:8/696: stat de/d18/d27/d41/d5a/f1b 0 2024-09-16T23:49:03.245 INFO:tasks.workunit.client.1.smithi043.stdout:2/652: write d8/d4b/d5f/d8e/fbb [2138243,111010] 0 2024-09-16T23:49:03.247 INFO:tasks.workunit.client.0.smithi042.stdout:9/395: chown d4/d12/d16/c2e 95920002 1 2024-09-16T23:49:03.247 INFO:tasks.workunit.client.0.smithi042.stdout:1/628: mknod d0/d17/cdf 0 2024-09-16T23:49:03.248 INFO:tasks.workunit.client.0.smithi042.stdout:9/396: readlink d4/d10/l13 0 2024-09-16T23:49:03.249 INFO:tasks.workunit.client.0.smithi042.stdout:9/397: chown d4/d12 2 1 2024-09-16T23:49:03.250 INFO:tasks.workunit.client.0.smithi042.stdout:4/749: mkdir de/d2e/d37/d4c/dfe/d106 0 2024-09-16T23:49:03.251 INFO:tasks.workunit.client.0.smithi042.stdout:2/538: mkdir d0/da/d1b/d48/db7 0 2024-09-16T23:49:03.252 INFO:tasks.workunit.client.0.smithi042.stdout:2/539: chown d0/f95 21082 1 2024-09-16T23:49:03.253 INFO:tasks.workunit.client.0.smithi042.stdout:2/540: dread - d0/da/d1b/d3b/d44/d54/fb1 zero size 2024-09-16T23:49:03.253 INFO:tasks.workunit.client.0.smithi042.stdout:3/717: rmdir d2/d1b/d1d/d34/d62/d80/def 0 2024-09-16T23:49:03.254 INFO:tasks.workunit.client.0.smithi042.stdout:0/493: rmdir d0/df/d23/d90 0 2024-09-16T23:49:03.254 INFO:tasks.workunit.client.1.smithi043.stdout:4/780: mkdir d8/d15/d20/d2b/d54/db3/d4c/df3 0 2024-09-16T23:49:03.255 INFO:tasks.workunit.client.0.smithi042.stdout:3/718: dread - d2/d1b/d1d/d34/d62/fd4 zero size 2024-09-16T23:49:03.255 INFO:tasks.workunit.client.0.smithi042.stdout:0/494: fsync d0/d15/f85 0 2024-09-16T23:49:03.256 INFO:tasks.workunit.client.1.smithi043.stdout:7/703: truncate d4/db/d28/f2d 3321225 0 2024-09-16T23:49:03.258 INFO:tasks.workunit.client.0.smithi042.stdout:0/495: read d0/df/d13/d49/d70/d94/f65 [3771321,9183] 0 2024-09-16T23:49:03.260 INFO:tasks.workunit.client.1.smithi043.stdout:5/872: symlink d3/d9/d22/d4a/d64/d8f/da1/l125 0 2024-09-16T23:49:03.261 INFO:tasks.workunit.client.0.smithi042.stdout:0/496: chown d0/d48/f53 397 1 2024-09-16T23:49:03.261 INFO:tasks.workunit.client.0.smithi042.stdout:3/719: symlink d2/d9/lf4 0 2024-09-16T23:49:03.263 INFO:tasks.workunit.client.0.smithi042.stdout:4/750: dread de/d42/f44 [0,4194304] 0 2024-09-16T23:49:03.264 INFO:tasks.workunit.client.0.smithi042.stdout:3/720: chown d2/d95/fe8 0 1 2024-09-16T23:49:03.268 INFO:tasks.workunit.client.1.smithi043.stdout:0/791: dread d0/d2/df/d1a/d3b/fda [0,4194304] 0 2024-09-16T23:49:03.270 INFO:tasks.workunit.client.1.smithi043.stdout:8/697: creat de/d70/d3d/fcf x:0 0 0 2024-09-16T23:49:03.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:02 smithi042 ceph-mon[27914]: pgmap v10: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 74 MiB/s rd, 232 MiB/s wr, 448 op/s 2024-09-16T23:49:03.273 INFO:tasks.workunit.client.1.smithi043.stdout:9/652: mkdir d4/d28/dcf/dd0 0 2024-09-16T23:49:03.274 INFO:tasks.workunit.client.0.smithi042.stdout:0/497: dread d0/df/d23/f6b [0,4194304] 0 2024-09-16T23:49:03.274 INFO:tasks.workunit.client.0.smithi042.stdout:1/629: dwrite d0/d17/f20 [0,4194304] 0 2024-09-16T23:49:03.275 INFO:tasks.workunit.client.0.smithi042.stdout:1/630: fdatasync d0/d9a/fbb 0 2024-09-16T23:49:03.276 INFO:tasks.workunit.client.1.smithi043.stdout:4/781: mkdir d8/d25/dcd/df4 0 2024-09-16T23:49:03.276 INFO:tasks.workunit.client.1.smithi043.stdout:9/653: write d4/d1c/d6b/f3c [541718,102632] 0 2024-09-16T23:49:03.278 INFO:tasks.workunit.client.0.smithi042.stdout:5/687: sync 2024-09-16T23:49:03.278 INFO:tasks.workunit.client.0.smithi042.stdout:7/627: sync 2024-09-16T23:49:03.278 INFO:tasks.workunit.client.0.smithi042.stdout:8/442: sync 2024-09-16T23:49:03.278 INFO:tasks.workunit.client.0.smithi042.stdout:3/721: dwrite d2/d1b/d1d/f25 [0,4194304] 0 2024-09-16T23:49:03.278 INFO:tasks.workunit.client.0.smithi042.stdout:0/498: read d0/f4b [1473042,129852] 0 2024-09-16T23:49:03.279 INFO:tasks.workunit.client.1.smithi043.stdout:0/792: dwrite d0/d2/df/d1a/d46/d39/fff [0,4194304] 0 2024-09-16T23:49:03.279 INFO:tasks.workunit.client.1.smithi043.stdout:9/654: fsync d4/d5/f1d 0 2024-09-16T23:49:03.280 INFO:tasks.workunit.client.0.smithi042.stdout:9/398: getdents d4/d12/d2d/d37/d41 0 2024-09-16T23:49:03.281 INFO:tasks.workunit.client.1.smithi043.stdout:0/793: write d0/d2/df/d1a/d3b/f7d [4643328,78247] 0 2024-09-16T23:49:03.284 INFO:tasks.workunit.client.0.smithi042.stdout:9/399: dread - d4/d12/f63 zero size 2024-09-16T23:49:03.285 INFO:tasks.workunit.client.1.smithi043.stdout:1/873: symlink d4/df/d26/dd5/d12b/l12c 0 2024-09-16T23:49:03.286 INFO:tasks.workunit.client.0.smithi042.stdout:4/751: write fd [1031412,100549] 0 2024-09-16T23:49:03.286 INFO:tasks.workunit.client.1.smithi043.stdout:4/782: dread d8/d25/d39/d58/fca [0,4194304] 0 2024-09-16T23:49:03.289 INFO:tasks.workunit.client.0.smithi042.stdout:5/688: rename d9/db/d11/f32 to d9/fef 0 2024-09-16T23:49:03.291 INFO:tasks.workunit.client.0.smithi042.stdout:7/628: creat d3/d1f/d49/d78/dbe/fda x:0 0 0 2024-09-16T23:49:03.292 INFO:tasks.workunit.client.1.smithi043.stdout:9/655: dwrite d4/d5/d64/f65 [4194304,4194304] 0 2024-09-16T23:49:03.294 INFO:tasks.workunit.client.0.smithi042.stdout:3/722: mkdir d2/d1b/d1d/d34/d50/df5 0 2024-09-16T23:49:03.295 INFO:tasks.workunit.client.1.smithi043.stdout:9/656: chown d4/d1c/d25/d2b/d4c/da3/cba 38368447 1 2024-09-16T23:49:03.295 INFO:tasks.workunit.client.0.smithi042.stdout:6/555: write d8/d70/d81/dd/f15 [2627345,4692] 0 2024-09-16T23:49:03.296 INFO:tasks.workunit.client.1.smithi043.stdout:9/657: readlink d4/d1c/d4f/d45/d85/lc7 0 2024-09-16T23:49:03.297 INFO:tasks.workunit.client.1.smithi043.stdout:9/658: chown d4/d1c/d25/d2b/d4c/fb0 3677 1 2024-09-16T23:49:03.298 INFO:tasks.workunit.client.0.smithi042.stdout:2/541: write d0/da/d17/f1c [2579157,40443] 0 2024-09-16T23:49:03.298 INFO:tasks.workunit.client.0.smithi042.stdout:1/631: dwrite d0/d9/d90/d6b/d88/f75 [0,4194304] 0 2024-09-16T23:49:03.303 INFO:tasks.workunit.client.0.smithi042.stdout:3/723: chown d2/d1b/d1d/d34/d50/d93 19923051 1 2024-09-16T23:49:03.305 INFO:tasks.workunit.client.0.smithi042.stdout:4/752: rename de/d2e/d37/d3c/f9a to de/d2e/d37/d3e/f107 0 2024-09-16T23:49:03.306 INFO:tasks.workunit.client.0.smithi042.stdout:7/629: write d3/d1f/d49/d78/dbe/fca [652877,77910] 0 2024-09-16T23:49:03.308 INFO:tasks.workunit.client.0.smithi042.stdout:8/443: dwrite da/f47 [0,4194304] 0 2024-09-16T23:49:03.310 INFO:tasks.workunit.client.0.smithi042.stdout:6/556: write d8/d70/d81/d90/f91 [645059,43873] 0 2024-09-16T23:49:03.313 INFO:tasks.workunit.client.1.smithi043.stdout:2/653: dwrite d8/d1d/d20/d23/d38/f69 [0,4194304] 0 2024-09-16T23:49:03.313 INFO:tasks.workunit.client.0.smithi042.stdout:9/400: dread d4/d10/f6b [0,4194304] 0 2024-09-16T23:49:03.313 INFO:tasks.workunit.client.0.smithi042.stdout:3/724: chown d2/d1b/d89/f3a 8065177 1 2024-09-16T23:49:03.314 INFO:tasks.workunit.client.1.smithi043.stdout:7/704: write d4/fab [147936,258] 0 2024-09-16T23:49:03.317 INFO:tasks.workunit.client.0.smithi042.stdout:6/557: write d8/d70/d81/dd/f15 [2619633,104909] 0 2024-09-16T23:49:03.317 INFO:tasks.workunit.client.0.smithi042.stdout:3/725: stat d2/lf3 0 2024-09-16T23:49:03.318 INFO:tasks.workunit.client.0.smithi042.stdout:8/444: write da/d19/d1d/d30/d3f/d6d/f7e [955518,56570] 0 2024-09-16T23:49:03.321 INFO:tasks.workunit.client.0.smithi042.stdout:0/499: dread d0/f4 [0,4194304] 0 2024-09-16T23:49:03.321 INFO:tasks.workunit.client.1.smithi043.stdout:4/783: mknod d8/d15/d20/d2b/d54/cf5 0 2024-09-16T23:49:03.324 INFO:tasks.workunit.client.1.smithi043.stdout:4/784: write d8/d15/d9b/d8b/fa8 [3534764,60317] 0 2024-09-16T23:49:03.324 INFO:tasks.workunit.client.0.smithi042.stdout:2/542: rmdir d0 39 2024-09-16T23:49:03.324 INFO:tasks.workunit.client.0.smithi042.stdout:8/445: write da/d19/f1e [4349387,8530] 0 2024-09-16T23:49:03.325 INFO:tasks.workunit.client.1.smithi043.stdout:8/698: creat de/fd0 x:0 0 0 2024-09-16T23:49:03.326 INFO:tasks.workunit.client.1.smithi043.stdout:7/705: dwrite d4/d27/d8a/faa [0,4194304] 0 2024-09-16T23:49:03.328 INFO:tasks.workunit.client.0.smithi042.stdout:3/726: creat d2/d1b/dc8/ff6 x:0 0 0 2024-09-16T23:49:03.330 INFO:tasks.workunit.client.1.smithi043.stdout:2/654: symlink d8/d5e/d83/d3a/d77/ldf 0 2024-09-16T23:49:03.334 INFO:tasks.workunit.client.0.smithi042.stdout:4/753: dwrite de/d1a/f4b [0,4194304] 0 2024-09-16T23:49:03.336 INFO:tasks.workunit.client.1.smithi043.stdout:4/785: dwrite d8/d15/d20/d2b/d23/d27/faf [0,4194304] 0 2024-09-16T23:49:03.337 INFO:tasks.workunit.client.0.smithi042.stdout:7/630: rename d3/d1f/d49/d76/d19/c73 to d3/d98/da0/d32/d6b/d64/cdb 0 2024-09-16T23:49:03.338 INFO:tasks.workunit.client.0.smithi042.stdout:3/727: unlink d2/d7/f81 0 2024-09-16T23:49:03.340 INFO:tasks.workunit.client.1.smithi043.stdout:9/659: creat d4/d1c/d25/d2b/d4c/fd1 x:0 0 0 2024-09-16T23:49:03.341 INFO:tasks.workunit.client.0.smithi042.stdout:8/446: creat da/d4b/f87 x:0 0 0 2024-09-16T23:49:03.345 INFO:tasks.workunit.client.1.smithi043.stdout:8/699: chown de/d18/d1f/f59 55333 1 2024-09-16T23:49:03.345 INFO:tasks.workunit.client.1.smithi043.stdout:8/700: fdatasync de/f17 0 2024-09-16T23:49:03.349 INFO:tasks.workunit.client.1.smithi043.stdout:2/655: fdatasync d8/d4b/f5b 0 2024-09-16T23:49:03.350 INFO:tasks.workunit.client.1.smithi043.stdout:2/656: stat d8/d11/d16/f28 0 2024-09-16T23:49:03.350 INFO:tasks.workunit.client.0.smithi042.stdout:5/689: dread d9/db/d1b/d31/d46/d58/f9a [0,4194304] 0 2024-09-16T23:49:03.351 INFO:tasks.workunit.client.1.smithi043.stdout:2/657: chown d8/d1d/d20/d23/cb1 226301844 1 2024-09-16T23:49:03.353 INFO:tasks.workunit.client.1.smithi043.stdout:7/706: dread d4/db/d11/d21/f2f [4194304,4194304] 0 2024-09-16T23:49:03.354 INFO:tasks.workunit.client.1.smithi043.stdout:1/874: getdents d4/df/d26/dd5/d12b 0 2024-09-16T23:49:03.355 INFO:tasks.workunit.client.0.smithi042.stdout:6/558: rename d8/d70/d81/dd/d1b/d50/c86 to d8/d70/d81/dd/d1b/d34/cb0 0 2024-09-16T23:49:03.355 INFO:tasks.workunit.client.1.smithi043.stdout:4/786: stat d8/d25/d39/dd3/l48 0 2024-09-16T23:49:03.356 INFO:tasks.workunit.client.0.smithi042.stdout:1/632: dwrite d0/d17/d18/f73 [0,4194304] 0 2024-09-16T23:49:03.359 INFO:tasks.workunit.client.0.smithi042.stdout:0/500: write d0/df/f4a [792569,88862] 0 2024-09-16T23:49:03.363 INFO:tasks.workunit.client.0.smithi042.stdout:1/633: chown d0/f3 3 1 2024-09-16T23:49:03.366 INFO:tasks.workunit.client.0.smithi042.stdout:9/401: write d4/d12/d2d/d37/f6a [1536216,104948] 0 2024-09-16T23:49:03.366 INFO:tasks.workunit.client.0.smithi042.stdout:7/631: dwrite d3/d98/da0/f6c [0,4194304] 0 2024-09-16T23:49:03.368 INFO:tasks.workunit.client.1.smithi043.stdout:2/658: creat d8/d5e/fe0 x:0 0 0 2024-09-16T23:49:03.369 INFO:tasks.workunit.client.0.smithi042.stdout:4/754: write de/d1a/f33 [1327737,7221] 0 2024-09-16T23:49:03.370 INFO:tasks.workunit.client.1.smithi043.stdout:9/660: mkdir d4/d1c/d25/d2b/d4c/dd2 0 2024-09-16T23:49:03.371 INFO:tasks.workunit.client.0.smithi042.stdout:3/728: creat d2/d7/d61/d8e/d9d/dc4/ff7 x:0 0 0 2024-09-16T23:49:03.374 INFO:tasks.workunit.client.1.smithi043.stdout:1/875: mknod d4/c12d 0 2024-09-16T23:49:03.374 INFO:tasks.workunit.client.1.smithi043.stdout:9/661: write d4/d5/f1d [4258096,129765] 0 2024-09-16T23:49:03.375 INFO:tasks.workunit.client.1.smithi043.stdout:8/701: creat de/d18/d27/d41/fd1 x:0 0 0 2024-09-16T23:49:03.378 INFO:tasks.workunit.client.1.smithi043.stdout:4/787: rename d8/d15/d9b/f86 to d8/d15/ff6 0 2024-09-16T23:49:03.379 INFO:tasks.workunit.client.0.smithi042.stdout:8/447: dread da/d19/d1d/d30/d49/f7b [0,4194304] 0 2024-09-16T23:49:03.379 INFO:tasks.workunit.client.1.smithi043.stdout:1/876: stat d4/df/d26/dd5/d12b/l12c 0 2024-09-16T23:49:03.379 INFO:tasks.workunit.client.1.smithi043.stdout:7/707: dwrite d4/db/d11/d21/d31/f82 [4194304,4194304] 0 2024-09-16T23:49:03.380 INFO:tasks.workunit.client.0.smithi042.stdout:0/501: creat d0/df/d13/fa0 x:0 0 0 2024-09-16T23:49:03.380 INFO:tasks.workunit.client.0.smithi042.stdout:2/543: read d0/da/d1b/d3b/f56 [502948,113657] 0 2024-09-16T23:49:03.380 INFO:tasks.workunit.client.0.smithi042.stdout:6/559: unlink d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f68 0 2024-09-16T23:49:03.381 INFO:tasks.workunit.client.0.smithi042.stdout:4/755: chown de/d42/d5d/f8c 5799 1 2024-09-16T23:49:03.383 INFO:tasks.workunit.client.1.smithi043.stdout:8/702: mknod de/d18/d1f/d2f/d39/d7b/d83/cd2 0 2024-09-16T23:49:03.384 INFO:tasks.workunit.client.0.smithi042.stdout:2/544: readlink d0/da/d1b/l1d 0 2024-09-16T23:49:03.384 INFO:tasks.workunit.client.0.smithi042.stdout:7/632: write d3/d98/da0/f1b [7073736,57344] 0 2024-09-16T23:49:03.384 INFO:tasks.workunit.client.0.smithi042.stdout:0/502: chown d0/df/d13 104 1 2024-09-16T23:49:03.384 INFO:tasks.workunit.client.1.smithi043.stdout:4/788: fdatasync d8/d25/d64/f7f 0 2024-09-16T23:49:03.385 INFO:tasks.workunit.client.0.smithi042.stdout:9/402: chown d4/d12/d2d/d37/d3a/d50/d6f/c79 3919 1 2024-09-16T23:49:03.388 INFO:tasks.workunit.client.0.smithi042.stdout:6/560: readlink d8/d70/d81/dd/d2c/l6a 0 2024-09-16T23:49:03.389 INFO:tasks.workunit.client.1.smithi043.stdout:4/789: mknod d8/d25/dcd/cf7 0 2024-09-16T23:49:03.390 INFO:tasks.workunit.client.0.smithi042.stdout:2/545: rename d0 to d0/d30/d43/db8 22 2024-09-16T23:49:03.390 INFO:tasks.workunit.client.1.smithi043.stdout:8/703: creat de/d18/d1f/d2f/d39/d7b/d83/fd3 x:0 0 0 2024-09-16T23:49:03.391 INFO:tasks.workunit.client.0.smithi042.stdout:4/756: chown de/d2e/d37/d4c/dfe/d106 10233292 1 2024-09-16T23:49:03.393 INFO:tasks.workunit.client.0.smithi042.stdout:7/633: read - d3/d98/da0/d32/dcd/fcb zero size 2024-09-16T23:49:03.393 INFO:tasks.workunit.client.0.smithi042.stdout:1/634: dwrite d0/d3b/d8d/fcc [0,4194304] 0 2024-09-16T23:49:03.393 INFO:tasks.workunit.client.0.smithi042.stdout:5/690: dread d9/db/de/d5e/d87/fb9 [0,4194304] 0 2024-09-16T23:49:03.400 INFO:tasks.workunit.client.0.smithi042.stdout:3/729: link d2/d7/d61/d8e/d9d/dc4/cc9 d2/d7/d61/d8e/d9d/dc4/de4/cf8 0 2024-09-16T23:49:03.400 INFO:tasks.workunit.client.0.smithi042.stdout:4/757: dread - de/d2e/d37/d3e/fcb zero size 2024-09-16T23:49:03.400 INFO:tasks.workunit.client.0.smithi042.stdout:1/635: chown d0/d17/l19 38 1 2024-09-16T23:49:03.401 INFO:tasks.workunit.client.0.smithi042.stdout:0/503: write d0/df/d23/f38 [1699139,30684] 0 2024-09-16T23:49:03.401 INFO:tasks.workunit.client.0.smithi042.stdout:8/448: dwrite da/f83 [0,4194304] 0 2024-09-16T23:49:03.402 INFO:tasks.workunit.client.0.smithi042.stdout:9/403: dread d4/d12/d2d/d37/d28/f4e [0,4194304] 0 2024-09-16T23:49:03.403 INFO:tasks.workunit.client.0.smithi042.stdout:4/758: chown de/f5a 60 1 2024-09-16T23:49:03.407 INFO:tasks.workunit.client.1.smithi043.stdout:9/662: truncate d4/d21/d35/d4b/fb7 2459040 0 2024-09-16T23:49:03.411 INFO:tasks.workunit.client.1.smithi043.stdout:7/708: truncate d4/d27/d41/d1b/d95/fc3 1981906 0 2024-09-16T23:49:03.412 INFO:tasks.workunit.client.0.smithi042.stdout:0/504: write d0/df/d23/d33/f5c [322883,102228] 0 2024-09-16T23:49:03.413 INFO:tasks.workunit.client.1.smithi043.stdout:8/704: symlink de/d18/d1f/d2f/d39/d7b/d83/dbb/dce/ld4 0 2024-09-16T23:49:03.413 INFO:tasks.workunit.client.1.smithi043.stdout:1/877: truncate d4/d9/d1c/d1e/d42/f70 2028393 0 2024-09-16T23:49:03.415 INFO:tasks.workunit.client.0.smithi042.stdout:5/691: symlink d9/db/de/d5e/d87/d61/lf0 0 2024-09-16T23:49:03.416 INFO:tasks.workunit.client.0.smithi042.stdout:3/730: truncate d2/d1b/d1d/d34/d50/fea 642617 0 2024-09-16T23:49:03.416 INFO:tasks.workunit.client.1.smithi043.stdout:9/663: fdatasync d4/d28/d5f/d86/f96 0 2024-09-16T23:49:03.417 INFO:tasks.workunit.client.1.smithi043.stdout:4/790: write d8/d25/d64/f7d [1463407,116761] 0 2024-09-16T23:49:03.417 INFO:tasks.workunit.client.1.smithi043.stdout:9/664: chown d4/d28/d5f/d91 146684 1 2024-09-16T23:49:03.418 INFO:tasks.workunit.client.1.smithi043.stdout:9/665: chown d4/d1c/d25/db1/fcd 599912402 1 2024-09-16T23:49:03.418 INFO:tasks.workunit.client.1.smithi043.stdout:9/666: chown d4/d21/d33/da2/la4 372579376 1 2024-09-16T23:49:03.419 INFO:tasks.workunit.client.0.smithi042.stdout:8/449: rmdir da/d19/d1d/d30/d49 39 2024-09-16T23:49:03.420 INFO:tasks.workunit.client.0.smithi042.stdout:7/634: dwrite d3/d98/f8f [0,4194304] 0 2024-09-16T23:49:03.421 INFO:tasks.workunit.client.0.smithi042.stdout:2/546: dwrite d0/d30/d43/f63 [0,4194304] 0 2024-09-16T23:49:03.423 INFO:tasks.workunit.client.0.smithi042.stdout:3/731: mknod d2/d95/cf9 0 2024-09-16T23:49:03.426 INFO:tasks.workunit.client.0.smithi042.stdout:0/505: chown d0/df/d23/d33/d56/l76 1 1 2024-09-16T23:49:03.427 INFO:tasks.workunit.client.0.smithi042.stdout:2/547: chown d0/d78/d8b/f99 199004 1 2024-09-16T23:49:03.429 INFO:tasks.workunit.client.1.smithi043.stdout:1/878: unlink d4/d34/l58 0 2024-09-16T23:49:03.430 INFO:tasks.workunit.client.0.smithi042.stdout:5/692: creat d9/db/d19/d21/d85/ff1 x:0 0 0 2024-09-16T23:49:03.432 INFO:tasks.workunit.client.0.smithi042.stdout:4/759: dwrite de/d42/d68/f8d [0,4194304] 0 2024-09-16T23:49:03.433 INFO:tasks.workunit.client.0.smithi042.stdout:2/548: chown d0/da/d1b/d3b/d44/l4b 2 1 2024-09-16T23:49:03.434 INFO:tasks.workunit.client.1.smithi043.stdout:9/667: rename d4/d28/f61 to d4/d28/d5f/fd3 0 2024-09-16T23:49:03.434 INFO:tasks.workunit.client.0.smithi042.stdout:0/506: dread d0/f75 [0,4194304] 0 2024-09-16T23:49:03.439 INFO:tasks.workunit.client.0.smithi042.stdout:8/450: mknod da/c88 0 2024-09-16T23:49:03.440 INFO:tasks.workunit.client.1.smithi043.stdout:8/705: symlink de/d18/d1f/ld5 0 2024-09-16T23:49:03.441 INFO:tasks.workunit.client.0.smithi042.stdout:4/760: write de/d42/d99/fc7 [388840,108684] 0 2024-09-16T23:49:03.441 INFO:tasks.workunit.client.1.smithi043.stdout:8/706: fsync de/d18/d1f/d25/f6b 0 2024-09-16T23:49:03.442 INFO:tasks.workunit.client.0.smithi042.stdout:0/507: mkdir d0/df/d13/d49/d70/da1 0 2024-09-16T23:49:03.444 INFO:tasks.workunit.client.0.smithi042.stdout:5/693: chown d9/c14 0 1 2024-09-16T23:49:03.446 INFO:tasks.workunit.client.1.smithi043.stdout:9/668: mkdir d4/d21/d35/dd4 0 2024-09-16T23:49:03.446 INFO:tasks.workunit.client.0.smithi042.stdout:8/451: creat da/d19/d1d/f89 x:0 0 0 2024-09-16T23:49:03.447 INFO:tasks.workunit.client.0.smithi042.stdout:9/404: write d4/d12/d2d/d37/d28/f31 [547742,14117] 0 2024-09-16T23:49:03.447 INFO:tasks.workunit.client.1.smithi043.stdout:7/709: dwrite d4/db/f18 [0,4194304] 0 2024-09-16T23:49:03.448 INFO:tasks.workunit.client.1.smithi043.stdout:9/669: chown d4/d1c/d4f/d8c/f60 33262254 1 2024-09-16T23:49:03.449 INFO:tasks.workunit.client.0.smithi042.stdout:6/561: truncate d8/d70/d81/dd/d1b/d50/f98 1368553 0 2024-09-16T23:49:03.450 INFO:tasks.workunit.client.0.smithi042.stdout:3/732: dwrite d2/d7/d61/d8e/d9d/dc4/dcc/fc2 [0,4194304] 0 2024-09-16T23:49:03.450 INFO:tasks.workunit.client.1.smithi043.stdout:8/707: chown de/l7c 5819 1 2024-09-16T23:49:03.451 INFO:tasks.workunit.client.1.smithi043.stdout:8/708: chown de/d18/d1f/d25 28622 1 2024-09-16T23:49:03.451 INFO:tasks.workunit.client.0.smithi042.stdout:7/635: rename d3/d98/da0/d32 to d3/d1f/d49/d76/d19/ddc 0 2024-09-16T23:49:03.452 INFO:tasks.workunit.client.0.smithi042.stdout:2/549: mknod d0/da/d17/d3e/d86/cb9 0 2024-09-16T23:49:03.456 INFO:tasks.workunit.client.1.smithi043.stdout:1/879: creat d4/f12e x:0 0 0 2024-09-16T23:49:03.457 INFO:tasks.workunit.client.0.smithi042.stdout:0/508: stat d0/df/d23/d33/d51/d5b/l77 0 2024-09-16T23:49:03.458 INFO:tasks.workunit.client.0.smithi042.stdout:4/761: creat de/d42/d82/d94/f108 x:0 0 0 2024-09-16T23:49:03.458 INFO:tasks.workunit.client.1.smithi043.stdout:9/670: creat d4/d21/d35/fd5 x:0 0 0 2024-09-16T23:49:03.460 INFO:tasks.workunit.client.1.smithi043.stdout:4/791: getdents d8/d15/d20/d2b/d54 0 2024-09-16T23:49:03.462 INFO:tasks.workunit.client.1.smithi043.stdout:4/792: truncate d8/d25/f42 9344419 0 2024-09-16T23:49:03.463 INFO:tasks.workunit.client.1.smithi043.stdout:1/880: dread d4/d9/d8f/db6/dcc/dda/d107/fa4 [0,4194304] 0 2024-09-16T23:49:03.542 INFO:tasks.workunit.client.1.smithi043.stdout:7/710: rename d4/db/d11/d21/fae to d4/db/d43/d83/d85/d8b/fe7 0 2024-09-16T23:49:03.547 INFO:tasks.workunit.client.0.smithi042.stdout:3/733: mkdir d2/d7/d61/d8e/d7f/dfa 0 2024-09-16T23:49:03.548 INFO:tasks.workunit.client.0.smithi042.stdout:0/509: dread - d0/df/d23/d33/d56/f8b zero size 2024-09-16T23:49:03.548 INFO:tasks.workunit.client.0.smithi042.stdout:9/405: dwrite d4/d10/d24/f3b [4194304,4194304] 0 2024-09-16T23:49:03.549 INFO:tasks.workunit.client.0.smithi042.stdout:4/762: mkdir de/d42/d99/d9e/d109 0 2024-09-16T23:49:03.550 INFO:tasks.workunit.client.0.smithi042.stdout:7/636: dread d3/f8 [0,4194304] 0 2024-09-16T23:49:03.553 INFO:tasks.workunit.client.0.smithi042.stdout:0/510: dread - d0/df/d13/d49/d70/d94/f93 zero size 2024-09-16T23:49:03.554 INFO:tasks.workunit.client.0.smithi042.stdout:3/734: rename d2/d7/d61/db9 to d2/d7/dfb 0 2024-09-16T23:49:03.555 INFO:tasks.workunit.client.0.smithi042.stdout:4/763: getdents de/de5 0 2024-09-16T23:49:03.558 INFO:tasks.workunit.client.0.smithi042.stdout:5/694: link d9/db/d19/d21/d41/l9c d9/db/lf2 0 2024-09-16T23:49:03.558 INFO:tasks.workunit.client.0.smithi042.stdout:4/764: read de/d12/d7d/f88 [2604095,121920] 0 2024-09-16T23:49:03.560 INFO:tasks.workunit.client.0.smithi042.stdout:0/511: write d0/df/d13/d49/d70/f98 [986364,80888] 0 2024-09-16T23:49:03.561 INFO:tasks.workunit.client.0.smithi042.stdout:4/765: stat de/d2e/d37/d3e/f107 0 2024-09-16T23:49:03.563 INFO:tasks.workunit.client.0.smithi042.stdout:3/735: link d2/d1b/d1d/d34/f58 d2/d7/d61/d8e/d7f/ffc 0 2024-09-16T23:49:03.563 INFO:tasks.workunit.client.0.smithi042.stdout:9/406: dwrite d4/d10/d24/f67 [0,4194304] 0 2024-09-16T23:49:03.565 INFO:tasks.workunit.client.0.smithi042.stdout:5/695: rename d9/f3c to d9/db/d1b/d31/d46/d58/ff3 0 2024-09-16T23:49:03.566 INFO:tasks.workunit.client.0.smithi042.stdout:4/766: truncate de/d12/d7d/fe9 654511 0 2024-09-16T23:49:03.566 INFO:tasks.workunit.client.0.smithi042.stdout:0/512: unlink d0/df/d23/d33/l57 0 2024-09-16T23:49:03.569 INFO:tasks.workunit.client.0.smithi042.stdout:3/736: mkdir d2/d1b/d1d/d34/d62/dfd 0 2024-09-16T23:49:03.569 INFO:tasks.workunit.client.0.smithi042.stdout:0/513: unlink d0/l11 0 2024-09-16T23:49:03.572 INFO:tasks.workunit.client.0.smithi042.stdout:5/696: fdatasync d9/db/de/d10/f1a 0 2024-09-16T23:49:03.572 INFO:tasks.workunit.client.0.smithi042.stdout:3/737: fsync d2/d1b/f86 0 2024-09-16T23:49:03.573 INFO:tasks.workunit.client.0.smithi042.stdout:4/767: symlink de/d1a/l10a 0 2024-09-16T23:49:03.573 INFO:tasks.workunit.client.0.smithi042.stdout:7/637: dwrite d3/d1f/d49/d76/d19/ddc/d6b/d64/f94 [0,4194304] 0 2024-09-16T23:49:03.575 INFO:tasks.workunit.client.0.smithi042.stdout:0/514: creat d0/df/d23/d33/d51/d5b/fa2 x:0 0 0 2024-09-16T23:49:03.582 INFO:tasks.workunit.client.0.smithi042.stdout:3/738: unlink d2/d7/d61/d8e/d2c/cba 0 2024-09-16T23:49:03.584 INFO:tasks.workunit.client.0.smithi042.stdout:3/739: dread - d2/d1b/d1d/d34/d62/fd4 zero size 2024-09-16T23:49:03.585 INFO:tasks.workunit.client.0.smithi042.stdout:0/515: dread d0/df/d13/d49/d70/d94/f92 [0,4194304] 0 2024-09-16T23:49:03.588 INFO:tasks.workunit.client.0.smithi042.stdout:4/768: dread de/d12/d1b/f78 [4194304,4194304] 0 2024-09-16T23:49:03.588 INFO:tasks.workunit.client.0.smithi042.stdout:3/740: creat d2/d7/d61/ffe x:0 0 0 2024-09-16T23:49:03.589 INFO:tasks.workunit.client.0.smithi042.stdout:0/516: stat d0/df/d23/l25 0 2024-09-16T23:49:03.590 INFO:tasks.workunit.client.0.smithi042.stdout:4/769: chown de/d42/d99/d9e/fd7 256731 1 2024-09-16T23:49:03.590 INFO:tasks.workunit.client.0.smithi042.stdout:3/741: read d2/d7/f36 [208908,15572] 0 2024-09-16T23:49:03.591 INFO:tasks.workunit.client.0.smithi042.stdout:0/517: write d0/df/f10 [142459,26283] 0 2024-09-16T23:49:03.593 INFO:tasks.workunit.client.0.smithi042.stdout:4/770: symlink de/d1a/d70/dac/l10b 0 2024-09-16T23:49:03.594 INFO:tasks.workunit.client.0.smithi042.stdout:5/697: write d9/db/de/d5e/d87/d61/fd9 [326224,48590] 0 2024-09-16T23:49:03.598 INFO:tasks.workunit.client.0.smithi042.stdout:7/638: dwrite d3/fa [0,4194304] 0 2024-09-16T23:49:03.604 INFO:tasks.workunit.client.0.smithi042.stdout:3/742: link d2/d1b/d1d/d34/d62/f78 d2/d1b/d1d/d34/d50/d93/fff 0 2024-09-16T23:49:03.604 INFO:tasks.workunit.client.0.smithi042.stdout:5/698: creat d9/db/d19/d21/ff4 x:0 0 0 2024-09-16T23:49:03.606 INFO:tasks.workunit.client.0.smithi042.stdout:4/771: link de/d42/d82/d94/dbd/fd3 de/d42/d68/f10c 0 2024-09-16T23:49:03.609 INFO:tasks.workunit.client.0.smithi042.stdout:5/699: fdatasync d9/db/de/d5e/d87/d61/f5d 0 2024-09-16T23:49:03.615 INFO:tasks.workunit.client.0.smithi042.stdout:3/743: link d2/d1b/d1d/d34/d62/d6a/l8d d2/d1b/d1d/d34/d50/df5/l100 0 2024-09-16T23:49:03.616 INFO:tasks.workunit.client.0.smithi042.stdout:3/744: write d2/d1b/f43 [2402921,63999] 0 2024-09-16T23:49:03.617 INFO:tasks.workunit.client.0.smithi042.stdout:7/639: getdents d3/d1f/d49/d76/d19/ddc/d6b/d5b 0 2024-09-16T23:49:03.617 INFO:tasks.workunit.client.0.smithi042.stdout:3/745: stat d2/d1b/f75 0 2024-09-16T23:49:03.621 INFO:tasks.workunit.client.0.smithi042.stdout:4/772: rename de/d12/d1b to de/d10d 0 2024-09-16T23:49:03.623 INFO:tasks.workunit.client.0.smithi042.stdout:5/700: mknod d9/db/de/d92/db5/cf5 0 2024-09-16T23:49:03.625 INFO:tasks.workunit.client.0.smithi042.stdout:3/746: symlink d2/d7/dfb/l101 0 2024-09-16T23:49:03.627 INFO:tasks.workunit.client.0.smithi042.stdout:7/640: creat d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/fdd x:0 0 0 2024-09-16T23:49:03.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/641: fdatasync d3/d1f/d49/d76/d19/f21 0 2024-09-16T23:49:03.628 INFO:tasks.workunit.client.0.smithi042.stdout:4/773: creat de/d10d/f10e x:0 0 0 2024-09-16T23:49:03.630 INFO:tasks.workunit.client.0.smithi042.stdout:4/774: chown de/d1a/da7/l3d 60785 1 2024-09-16T23:49:03.630 INFO:tasks.workunit.client.0.smithi042.stdout:7/642: read d3/d1f/f23 [990410,119137] 0 2024-09-16T23:49:03.639 INFO:tasks.workunit.client.0.smithi042.stdout:3/747: dread d2/d1b/d1d/d34/f3e [0,4194304] 0 2024-09-16T23:49:03.640 INFO:tasks.workunit.client.0.smithi042.stdout:7/643: mknod d3/d1f/cde 0 2024-09-16T23:49:03.640 INFO:tasks.workunit.client.0.smithi042.stdout:5/701: dwrite d9/db/d19/d21/d85/da5/f99 [0,4194304] 0 2024-09-16T23:49:03.646 INFO:tasks.workunit.client.0.smithi042.stdout:7/644: stat d3/l10 0 2024-09-16T23:49:03.648 INFO:tasks.workunit.client.0.smithi042.stdout:4/775: link de/d2e/d37/d3e/f7f de/d2e/d37/d4c/dfe/d106/f10f 0 2024-09-16T23:49:03.648 INFO:tasks.workunit.client.0.smithi042.stdout:3/748: dwrite d2/d1b/dc8/fd3 [0,4194304] 0 2024-09-16T23:49:03.655 INFO:tasks.workunit.client.0.smithi042.stdout:5/702: truncate d9/db/de/d10/f22 1424955 0 2024-09-16T23:49:03.656 INFO:tasks.workunit.client.0.smithi042.stdout:7/645: creat d3/d98/da0/d24/dc3/fdf x:0 0 0 2024-09-16T23:49:03.658 INFO:tasks.workunit.client.0.smithi042.stdout:3/749: dwrite d2/d1b/d89/f2e [0,4194304] 0 2024-09-16T23:49:03.660 INFO:tasks.workunit.client.0.smithi042.stdout:4/776: rename de/d42/d5d/d8f/d9f/ffc to de/d12/f110 0 2024-09-16T23:49:03.661 INFO:tasks.workunit.client.0.smithi042.stdout:7/646: fsync d3/d3f/f5e 0 2024-09-16T23:49:03.668 INFO:tasks.workunit.client.0.smithi042.stdout:5/703: link d9/d63/f9e d9/db/d19/ff6 0 2024-09-16T23:49:03.669 INFO:tasks.workunit.client.0.smithi042.stdout:5/704: write d9/db/de/d5e/d87/d61/d43/feb [801904,97294] 0 2024-09-16T23:49:03.672 INFO:tasks.workunit.client.0.smithi042.stdout:4/777: rename de/d12/c13 to de/d10d/c111 0 2024-09-16T23:49:03.672 INFO:tasks.workunit.client.0.smithi042.stdout:4/778: readlink de/d1a/l10a 0 2024-09-16T23:49:03.674 INFO:tasks.workunit.client.0.smithi042.stdout:7/647: creat d3/d1f/d49/d76/d19/fe0 x:0 0 0 2024-09-16T23:49:03.674 INFO:tasks.workunit.client.0.smithi042.stdout:5/705: symlink d9/db/de/d5e/d87/d61/lf7 0 2024-09-16T23:49:03.675 INFO:tasks.workunit.client.0.smithi042.stdout:3/750: dread d2/d7/d61/d8e/d2c/f64 [0,4194304] 0 2024-09-16T23:49:03.675 INFO:tasks.workunit.client.0.smithi042.stdout:4/779: rmdir de/d42/d5d/d8f/d9f 39 2024-09-16T23:49:03.677 INFO:tasks.workunit.client.0.smithi042.stdout:7/648: write d3/d98/f80 [4592522,32993] 0 2024-09-16T23:49:03.677 INFO:tasks.workunit.client.0.smithi042.stdout:3/751: mkdir d2/d9/d102 0 2024-09-16T23:49:03.678 INFO:tasks.workunit.client.0.smithi042.stdout:4/780: rmdir de/d42/d85/db3 39 2024-09-16T23:49:03.679 INFO:tasks.workunit.client.0.smithi042.stdout:3/752: stat d2/d1b/d1d/d34/d62/l7b 0 2024-09-16T23:49:03.680 INFO:tasks.workunit.client.0.smithi042.stdout:7/649: symlink d3/d1f/d4f/le1 0 2024-09-16T23:49:03.681 INFO:tasks.workunit.client.0.smithi042.stdout:4/781: stat de/d42/d5d/d8f/d9f 0 2024-09-16T23:49:03.682 INFO:tasks.workunit.client.0.smithi042.stdout:3/753: creat d2/d1b/d1d/d71/f103 x:0 0 0 2024-09-16T23:49:03.682 INFO:tasks.workunit.client.0.smithi042.stdout:3/754: chown d2/d1b/d1d/d34/d50 14249 1 2024-09-16T23:49:03.683 INFO:tasks.workunit.client.0.smithi042.stdout:3/755: chown d2/d9/l63 7488 1 2024-09-16T23:49:03.689 INFO:tasks.workunit.client.0.smithi042.stdout:5/706: link d9/db/d19/d21/d41/l9c d9/db/de/lf8 0 2024-09-16T23:49:03.690 INFO:tasks.workunit.client.0.smithi042.stdout:3/756: dwrite d2/d1b/d89/f5c [0,4194304] 0 2024-09-16T23:49:03.690 INFO:tasks.workunit.client.0.smithi042.stdout:4/782: dread de/d2e/d37/d4c/dfe/d106/f10f [0,4194304] 0 2024-09-16T23:49:03.692 INFO:tasks.workunit.client.0.smithi042.stdout:3/757: chown d2/d1b/d1d/l5e 15788 1 2024-09-16T23:49:03.694 INFO:tasks.workunit.client.0.smithi042.stdout:3/758: stat d2/d1b/d1d/d30/c9c 0 2024-09-16T23:49:03.694 INFO:tasks.workunit.client.0.smithi042.stdout:5/707: creat d9/db/d1b/d31/d46/d58/ff9 x:0 0 0 2024-09-16T23:49:03.697 INFO:tasks.workunit.client.0.smithi042.stdout:7/650: dwrite d3/d1f/d49/d76/d19/ddc/dcd/db5/dc4/fc9 [0,4194304] 0 2024-09-16T23:49:03.699 INFO:tasks.workunit.client.0.smithi042.stdout:7/651: write d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/fb4 [996256,24150] 0 2024-09-16T23:49:03.699 INFO:tasks.workunit.client.0.smithi042.stdout:3/759: symlink d2/d7/l104 0 2024-09-16T23:49:03.700 INFO:tasks.workunit.client.0.smithi042.stdout:5/708: mkdir d9/db/d1b/d31/d46/d58/dfa 0 2024-09-16T23:49:03.704 INFO:tasks.workunit.client.0.smithi042.stdout:3/760: rmdir d2/d1b/d1d 39 2024-09-16T23:49:03.706 INFO:tasks.workunit.client.0.smithi042.stdout:7/652: mknod d3/d1f/d49/ce2 0 2024-09-16T23:49:03.708 INFO:tasks.workunit.client.0.smithi042.stdout:4/783: dwrite de/d10d/f36 [0,4194304] 0 2024-09-16T23:49:03.708 INFO:tasks.workunit.client.0.smithi042.stdout:5/709: rename d9/db/de/d92/fa6 to d9/d4a/ffb 0 2024-09-16T23:49:03.710 INFO:tasks.workunit.client.1.smithi043.stdout:3/735: sync 2024-09-16T23:49:03.712 INFO:tasks.workunit.client.0.smithi042.stdout:5/710: write d9/db/de/d92/daf/fe0 [579961,82815] 0 2024-09-16T23:49:03.713 INFO:tasks.workunit.client.0.smithi042.stdout:3/761: mknod d2/d1b/d1d/d34/c105 0 2024-09-16T23:49:03.716 INFO:tasks.workunit.client.0.smithi042.stdout:3/762: chown d2/d1b/d89/db4/la9 7540 1 2024-09-16T23:49:03.718 INFO:tasks.workunit.client.0.smithi042.stdout:7/653: dwrite d3/d1f/d49/d76/d19/d63/f70 [0,4194304] 0 2024-09-16T23:49:03.719 INFO:tasks.workunit.client.0.smithi042.stdout:4/784: getdents de/d42/d85/db3 0 2024-09-16T23:49:03.720 INFO:tasks.workunit.client.0.smithi042.stdout:5/711: dread d9/db/d1b/d31/f7a [0,4194304] 0 2024-09-16T23:49:03.721 INFO:tasks.workunit.client.0.smithi042.stdout:4/785: chown de/d2e/d37/d3e/f5c 28661416 1 2024-09-16T23:49:03.723 INFO:tasks.workunit.client.0.smithi042.stdout:5/712: read d9/db/d19/fdb [620101,109406] 0 2024-09-16T23:49:03.724 INFO:tasks.workunit.client.0.smithi042.stdout:4/786: write de/d42/d5d/f8c [3772443,27372] 0 2024-09-16T23:49:03.725 INFO:tasks.workunit.client.0.smithi042.stdout:7/654: rename d3/d1f/d4f/dd6 to d3/d1f/d49/d76/d19/ddc/d6b/da5/de3 0 2024-09-16T23:49:03.728 INFO:tasks.workunit.client.0.smithi042.stdout:5/713: creat d9/db/d1b/da4/ffc x:0 0 0 2024-09-16T23:49:03.729 INFO:tasks.workunit.client.0.smithi042.stdout:3/763: dwrite d2/d1b/d1d/d34/d62/d6a/fe0 [0,4194304] 0 2024-09-16T23:49:03.731 INFO:tasks.workunit.client.0.smithi042.stdout:4/787: read de/f2f [866996,120737] 0 2024-09-16T23:49:03.732 INFO:tasks.workunit.client.0.smithi042.stdout:5/714: rmdir d9/db/d1b/d79 39 2024-09-16T23:49:03.734 INFO:tasks.workunit.client.0.smithi042.stdout:5/715: chown d9/db/de/d10/d2f/f34 373526 1 2024-09-16T23:49:03.735 INFO:tasks.workunit.client.0.smithi042.stdout:3/764: symlink d2/d7/d61/d8e/d9d/dc4/dcc/l106 0 2024-09-16T23:49:03.736 INFO:tasks.workunit.client.0.smithi042.stdout:7/655: dwrite d3/d1f/d49/d78/dbe/fc7 [0,4194304] 0 2024-09-16T23:49:03.738 INFO:tasks.workunit.client.0.smithi042.stdout:5/716: fsync d9/db/d11/fd7 0 2024-09-16T23:49:03.741 INFO:tasks.workunit.client.0.smithi042.stdout:3/765: truncate d2/d7/d61/f90 1669682 0 2024-09-16T23:49:03.744 INFO:tasks.workunit.client.0.smithi042.stdout:7/656: rename d3/d98/da0/f6c to d3/d1f/d49/d78/dbe/fe4 0 2024-09-16T23:49:03.745 INFO:tasks.workunit.client.0.smithi042.stdout:4/788: dwrite de/d12/f25 [4194304,4194304] 0 2024-09-16T23:49:03.749 INFO:tasks.workunit.client.0.smithi042.stdout:3/766: creat d2/d7/d61/f107 x:0 0 0 2024-09-16T23:49:03.752 INFO:tasks.workunit.client.1.smithi043.stdout:5/873: sync 2024-09-16T23:49:03.752 INFO:tasks.workunit.client.1.smithi043.stdout:6/715: sync 2024-09-16T23:49:03.753 INFO:tasks.workunit.client.0.smithi042.stdout:3/767: creat d2/d7/d61/d8e/d9d/f108 x:0 0 0 2024-09-16T23:49:03.753 INFO:tasks.workunit.client.1.smithi043.stdout:1/881: symlink d4/d9/d1c/d1e/l12f 0 2024-09-16T23:49:03.755 INFO:tasks.workunit.client.1.smithi043.stdout:7/711: fsync d4/d27/d41/d1b/d66/f8c 0 2024-09-16T23:49:03.755 INFO:tasks.workunit.client.0.smithi042.stdout:7/657: getdents d3/d1f/d49 0 2024-09-16T23:49:03.755 INFO:tasks.workunit.client.1.smithi043.stdout:7/712: stat d4/db/d11/d21/d31/fb5 0 2024-09-16T23:49:03.757 INFO:tasks.workunit.client.0.smithi042.stdout:6/562: symlink d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d7b/lb1 0 2024-09-16T23:49:03.758 INFO:tasks.workunit.client.0.smithi042.stdout:7/658: creat d3/d3f/fe5 x:0 0 0 2024-09-16T23:49:03.758 INFO:tasks.workunit.client.0.smithi042.stdout:4/789: truncate de/d10d/f36 2415096 0 2024-09-16T23:49:03.760 INFO:tasks.workunit.client.0.smithi042.stdout:3/768: truncate d2/d7/d61/d8e/fe5 5003317 0 2024-09-16T23:49:03.761 INFO:tasks.workunit.client.0.smithi042.stdout:6/563: fdatasync d8/d70/d81/dd/d2c/d57/f77 0 2024-09-16T23:49:03.761 INFO:tasks.workunit.client.1.smithi043.stdout:7/713: dwrite d4/db/d43/d83/d85/fc2 [0,4194304] 0 2024-09-16T23:49:03.762 INFO:tasks.workunit.client.1.smithi043.stdout:9/671: symlink d4/d28/d5f/db6/ld6 0 2024-09-16T23:49:03.763 INFO:tasks.workunit.client.0.smithi042.stdout:2/550: getdents d0 0 2024-09-16T23:49:03.763 INFO:tasks.workunit.client.1.smithi043.stdout:9/672: fsync d4/d1c/d4f/d8c/f60 0 2024-09-16T23:49:03.764 INFO:tasks.workunit.client.0.smithi042.stdout:8/452: rmdir da/d73 0 2024-09-16T23:49:03.765 INFO:tasks.workunit.client.1.smithi043.stdout:4/793: getdents d8/d15/d20/d2b/d54/db3/d4c/df3 0 2024-09-16T23:49:03.768 INFO:tasks.workunit.client.0.smithi042.stdout:7/659: write d3/d1f/d49/d76/fd5 [574283,20709] 0 2024-09-16T23:49:03.770 INFO:tasks.workunit.client.0.smithi042.stdout:4/790: rmdir de/de5 0 2024-09-16T23:49:03.772 INFO:tasks.workunit.client.1.smithi043.stdout:7/714: dwrite d4/d27/d8a/faa [4194304,4194304] 0 2024-09-16T23:49:03.772 INFO:tasks.workunit.client.0.smithi042.stdout:9/407: fdatasync d4/f5 0 2024-09-16T23:49:03.772 INFO:tasks.workunit.client.0.smithi042.stdout:7/660: symlink d3/d1f/d49/d76/d19/ddc/d6b/da5/de3/le6 0 2024-09-16T23:49:03.774 INFO:tasks.workunit.client.1.smithi043.stdout:9/673: dwrite d4/d21/d33/d95/f9b [0,4194304] 0 2024-09-16T23:49:03.774 INFO:tasks.workunit.client.0.smithi042.stdout:3/769: mkdir d2/d1b/d1d/d34/d50/d93/d96/d109 0 2024-09-16T23:49:03.775 INFO:tasks.workunit.client.0.smithi042.stdout:4/791: mknod de/d2e/d37/d4c/c112 0 2024-09-16T23:49:03.777 INFO:tasks.workunit.client.0.smithi042.stdout:7/661: chown d3/d1f/d49/d76/l2f 5358 1 2024-09-16T23:49:03.777 INFO:tasks.workunit.client.1.smithi043.stdout:5/874: creat d3/db/d15/d4d/d6b/f126 x:0 0 0 2024-09-16T23:49:03.778 INFO:tasks.workunit.client.0.smithi042.stdout:4/792: fsync de/d42/ff3 0 2024-09-16T23:49:03.780 INFO:tasks.workunit.client.0.smithi042.stdout:3/770: symlink d2/d9/l10a 0 2024-09-16T23:49:03.780 INFO:tasks.workunit.client.0.smithi042.stdout:2/551: mknod d0/d78/d8b/cba 0 2024-09-16T23:49:03.781 INFO:tasks.workunit.client.0.smithi042.stdout:3/771: chown d2/d1b/d89/l65 499 1 2024-09-16T23:49:03.783 INFO:tasks.workunit.client.0.smithi042.stdout:2/552: write d0/da/d1b/d3b/d44/fb3 [601696,70736] 0 2024-09-16T23:49:03.784 INFO:tasks.workunit.client.1.smithi043.stdout:1/882: dread d4/d9/f17 [0,4194304] 0 2024-09-16T23:49:03.784 INFO:tasks.workunit.client.0.smithi042.stdout:3/772: chown d2/d1b/d1d 72273 1 2024-09-16T23:49:03.784 INFO:tasks.workunit.client.0.smithi042.stdout:7/662: creat d3/d1f/d49/d78/dbe/fe7 x:0 0 0 2024-09-16T23:49:03.785 INFO:tasks.workunit.client.0.smithi042.stdout:4/793: mknod de/d42/d85/db3/de7/c113 0 2024-09-16T23:49:03.788 INFO:tasks.workunit.client.0.smithi042.stdout:6/564: mknod d8/d70/d81/d90/d9f/cb2 0 2024-09-16T23:49:03.790 INFO:tasks.workunit.client.0.smithi042.stdout:3/773: unlink d2/d1b/d89/f2e 0 2024-09-16T23:49:03.794 INFO:tasks.workunit.client.0.smithi042.stdout:7/663: unlink d3/d98/da0/d24/d28/l9d 0 2024-09-16T23:49:03.795 INFO:tasks.workunit.client.0.smithi042.stdout:7/664: stat d3/l13 0 2024-09-16T23:49:03.795 INFO:tasks.workunit.client.0.smithi042.stdout:9/408: dread d4/d12/f36 [0,4194304] 0 2024-09-16T23:49:03.796 INFO:tasks.workunit.client.0.smithi042.stdout:9/409: chown d4/d12/d2d/d37/d41 37 1 2024-09-16T23:49:03.797 INFO:tasks.workunit.client.0.smithi042.stdout:9/410: readlink d4/l61 0 2024-09-16T23:49:03.797 INFO:tasks.workunit.client.1.smithi043.stdout:4/794: rename d8/d15/d20/d2b/f6b to d8/d15/d9b/d8b/ff8 0 2024-09-16T23:49:03.797 INFO:tasks.workunit.client.0.smithi042.stdout:7/665: symlink d3/d1f/d4f/le8 0 2024-09-16T23:49:03.800 INFO:tasks.workunit.client.0.smithi042.stdout:7/666: chown d3/d1f/d49/d78/dbe/fe4 0 1 2024-09-16T23:49:03.805 INFO:tasks.workunit.client.0.smithi042.stdout:7/667: mknod d3/d1f/d49/d76/d19/ddc/d6b/da5/ce9 0 2024-09-16T23:49:03.808 INFO:tasks.workunit.client.0.smithi042.stdout:3/774: rename d2/d7/d61/d8e/fe5 to d2/d1b/f10b 0 2024-09-16T23:49:03.811 INFO:tasks.workunit.client.0.smithi042.stdout:3/775: mknod d2/d7/c10c 0 2024-09-16T23:49:03.811 INFO:tasks.workunit.client.0.smithi042.stdout:9/411: mkdir d4/d12/d16/d81 0 2024-09-16T23:49:03.812 INFO:tasks.workunit.client.1.smithi043.stdout:1/883: symlink d4/d9/d1c/d1e/l130 0 2024-09-16T23:49:03.812 INFO:tasks.workunit.client.1.smithi043.stdout:3/736: write d4/f7e [33649,55031] 0 2024-09-16T23:49:03.813 INFO:tasks.workunit.client.0.smithi042.stdout:4/794: rename de/d2e/d37/d4c/d4f/c92 to de/d10d/d8a/db2/c114 0 2024-09-16T23:49:03.813 INFO:tasks.workunit.client.0.smithi042.stdout:6/565: dread d8/d70/d81/dd/f11 [0,4194304] 0 2024-09-16T23:49:03.817 INFO:tasks.workunit.client.0.smithi042.stdout:2/553: creat d0/d30/fbb x:0 0 0 2024-09-16T23:49:03.818 INFO:tasks.workunit.client.0.smithi042.stdout:3/776: mknod d2/d1b/d1d/c10d 0 2024-09-16T23:49:03.819 INFO:tasks.workunit.client.1.smithi043.stdout:4/795: write d8/d15/d9b/d8b/ff8 [2812281,99142] 0 2024-09-16T23:49:03.821 INFO:tasks.workunit.client.1.smithi043.stdout:0/794: sync 2024-09-16T23:49:03.821 INFO:tasks.workunit.client.1.smithi043.stdout:8/709: sync 2024-09-16T23:49:03.821 INFO:tasks.workunit.client.1.smithi043.stdout:2/659: sync 2024-09-16T23:49:03.821 INFO:tasks.workunit.client.1.smithi043.stdout:6/716: creat de/d23/d34/ff0 x:0 0 0 2024-09-16T23:49:03.823 INFO:tasks.workunit.client.1.smithi043.stdout:5/875: dwrite d3/d9/d6f/d85/f98 [0,4194304] 0 2024-09-16T23:49:03.823 INFO:tasks.workunit.client.0.smithi042.stdout:3/777: mknod d2/d1b/d1d/d34/d50/df5/c10e 0 2024-09-16T23:49:03.824 INFO:tasks.workunit.client.1.smithi043.stdout:7/715: fsync d4/d27/d41/f40 0 2024-09-16T23:49:03.824 INFO:tasks.workunit.client.0.smithi042.stdout:8/453: rename da/d19/d2c/c4c to da/d19/d1d/d30/d49/d58/d6a/c8a 0 2024-09-16T23:49:03.826 INFO:tasks.workunit.client.0.smithi042.stdout:3/778: chown d2/d1b/d1d/d34/d62/d6a/fa5 759407784 1 2024-09-16T23:49:03.827 INFO:tasks.workunit.client.0.smithi042.stdout:3/779: chown d2/d1b 3507 1 2024-09-16T23:49:03.828 INFO:tasks.workunit.client.1.smithi043.stdout:0/795: readlink d0/d2/df/d1a/d46/d92/ld2 0 2024-09-16T23:49:03.828 INFO:tasks.workunit.client.0.smithi042.stdout:9/412: symlink d4/d10/d24/l82 0 2024-09-16T23:49:03.829 INFO:tasks.workunit.client.0.smithi042.stdout:9/413: fsync d4/d10/f30 0 2024-09-16T23:49:03.831 INFO:tasks.workunit.client.0.smithi042.stdout:6/566: mkdir d8/d70/d81/dd/d1b/d2d/da1/db3 0 2024-09-16T23:49:03.831 INFO:tasks.workunit.client.0.smithi042.stdout:7/668: truncate d3/d1f/d49/d76/f25 2611709 0 2024-09-16T23:49:03.831 INFO:tasks.workunit.client.0.smithi042.stdout:9/414: dread - d4/d12/d2d/d37/d41/f49 zero size 2024-09-16T23:49:03.834 INFO:tasks.workunit.client.0.smithi042.stdout:8/454: creat da/d19/d1d/d30/d49/d58/d6a/f8b x:0 0 0 2024-09-16T23:49:03.835 INFO:tasks.workunit.client.0.smithi042.stdout:2/554: getdents d0/d8d/da0 0 2024-09-16T23:49:03.837 INFO:tasks.workunit.client.1.smithi043.stdout:4/796: dread d8/d15/d20/d2b/d23/fd1 [0,4194304] 0 2024-09-16T23:49:03.837 INFO:tasks.workunit.client.0.smithi042.stdout:6/567: dread d8/d70/d81/dd/d1b/f93 [0,4194304] 0 2024-09-16T23:49:03.839 INFO:tasks.workunit.client.0.smithi042.stdout:6/568: dread - d8/d70/d81/dd/d2c/d57/f60 zero size 2024-09-16T23:49:03.844 INFO:tasks.workunit.client.1.smithi043.stdout:4/797: dwrite d8/f3b [8388608,4194304] 0 2024-09-16T23:49:03.845 INFO:tasks.workunit.client.0.smithi042.stdout:7/669: truncate d3/fc 2280188 0 2024-09-16T23:49:03.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/780: read d2/d7/f33 [1728920,110840] 0 2024-09-16T23:49:03.846 INFO:tasks.workunit.client.1.smithi043.stdout:0/796: dread d0/d2/df/d1a/d3b/f6c [0,4194304] 0 2024-09-16T23:49:03.847 INFO:tasks.workunit.client.0.smithi042.stdout:3/781: chown d2/d7/d61/d8e/d9d 0 1 2024-09-16T23:49:03.848 INFO:tasks.workunit.client.0.smithi042.stdout:7/670: fsync d3/d1f/d49/d76/d19/ddc/d6b/fd2 0 2024-09-16T23:49:03.850 INFO:tasks.workunit.client.0.smithi042.stdout:8/455: chown da/d19/d1d/f78 6433 1 2024-09-16T23:49:03.850 INFO:tasks.workunit.client.0.smithi042.stdout:6/569: dread d8/d70/d81/dd/f15 [0,4194304] 0 2024-09-16T23:49:03.851 INFO:tasks.workunit.client.0.smithi042.stdout:2/555: write d0/da/d17/f37 [4729870,45649] 0 2024-09-16T23:49:03.851 INFO:tasks.workunit.client.0.smithi042.stdout:3/782: rmdir d2/d1b/d1d/d34/d50 39 2024-09-16T23:49:03.854 INFO:tasks.workunit.client.1.smithi043.stdout:0/797: dwrite d0/d2/df/d1a/d46/d39/daa/fd3 [0,4194304] 0 2024-09-16T23:49:03.855 INFO:tasks.workunit.client.0.smithi042.stdout:7/671: creat d3/d1f/d49/d76/d19/fea x:0 0 0 2024-09-16T23:49:03.859 INFO:tasks.workunit.client.0.smithi042.stdout:8/456: dwrite da/d19/f1e [0,4194304] 0 2024-09-16T23:49:03.859 INFO:tasks.workunit.client.0.smithi042.stdout:3/783: rename d2/d7/d61/ffe to d2/d1b/dc8/f10f 0 2024-09-16T23:49:03.864 INFO:tasks.workunit.client.0.smithi042.stdout:7/672: unlink d3/d98/da0/l77 0 2024-09-16T23:49:03.868 INFO:tasks.workunit.client.0.smithi042.stdout:6/570: mkdir d8/d70/d81/dd/d1b/d34/d39/db4 0 2024-09-16T23:49:03.869 INFO:tasks.workunit.client.0.smithi042.stdout:8/457: mknod da/d19/d1d/d30/d49/d58/c8c 0 2024-09-16T23:49:03.874 INFO:tasks.workunit.client.0.smithi042.stdout:2/556: creat d0/d30/d43/d77/fbc x:0 0 0 2024-09-16T23:49:03.875 INFO:tasks.workunit.client.0.smithi042.stdout:3/784: dwrite d2/d7/fa4 [0,4194304] 0 2024-09-16T23:49:03.875 INFO:tasks.workunit.client.0.smithi042.stdout:6/571: creat d8/d70/d84/fb5 x:0 0 0 2024-09-16T23:49:03.880 INFO:tasks.workunit.client.0.smithi042.stdout:9/415: dwrite d4/d12/f1e [0,4194304] 0 2024-09-16T23:49:03.880 INFO:tasks.workunit.client.0.smithi042.stdout:7/673: write d3/d1f/f23 [590670,75157] 0 2024-09-16T23:49:03.884 INFO:tasks.workunit.client.0.smithi042.stdout:9/416: write d4/d12/d16/f6d [1528310,104192] 0 2024-09-16T23:49:03.888 INFO:tasks.workunit.client.0.smithi042.stdout:7/674: rename d3/d98/cc0 to d3/d98/da0/d41/db0/ceb 0 2024-09-16T23:49:03.889 INFO:tasks.workunit.client.0.smithi042.stdout:3/785: write d2/d7/f33 [3615703,2843] 0 2024-09-16T23:49:03.891 INFO:tasks.workunit.client.1.smithi043.stdout:3/737: read - d4/de/d40/d4d/fa9 zero size 2024-09-16T23:49:03.892 INFO:tasks.workunit.client.1.smithi043.stdout:1/884: dread d4/d9/f46 [0,4194304] 0 2024-09-16T23:49:03.893 INFO:tasks.workunit.client.1.smithi043.stdout:1/885: fdatasync d4/f12e 0 2024-09-16T23:49:03.895 INFO:tasks.workunit.client.0.smithi042.stdout:8/458: link da/c88 da/d19/d1d/d30/c8d 0 2024-09-16T23:49:03.896 INFO:tasks.workunit.client.1.smithi043.stdout:6/717: dread - de/d23/d34/d9b/d50/f79 zero size 2024-09-16T23:49:03.896 INFO:tasks.workunit.client.0.smithi042.stdout:7/675: link d3/d98/da0/c91 d3/d1f/d49/d76/d19/d63/cec 0 2024-09-16T23:49:03.896 INFO:tasks.workunit.client.0.smithi042.stdout:3/786: fsync d2/d9/faa 0 2024-09-16T23:49:03.897 INFO:tasks.workunit.client.1.smithi043.stdout:2/660: symlink d8/d5e/d83/d9e/dab/le1 0 2024-09-16T23:49:03.900 INFO:tasks.workunit.client.1.smithi043.stdout:2/661: write d8/d5e/d83/d9e/dab/fcb [3178707,7122] 0 2024-09-16T23:49:03.900 INFO:tasks.workunit.client.1.smithi043.stdout:7/716: mknod d4/d27/dbc/ce8 0 2024-09-16T23:49:03.901 INFO:tasks.workunit.client.1.smithi043.stdout:3/738: dwrite d4/de/d1f/d2c/db7/dc3/f8e [0,4194304] 0 2024-09-16T23:49:03.903 INFO:tasks.workunit.client.0.smithi042.stdout:6/572: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d99/fb6 x:0 0 0 2024-09-16T23:49:03.903 INFO:tasks.workunit.client.0.smithi042.stdout:6/573: fdatasync d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 0 2024-09-16T23:49:03.906 INFO:tasks.workunit.client.1.smithi043.stdout:9/674: link d4/d1c/d4f/d45/d85/lc7 d4/d21/d35/ld7 0 2024-09-16T23:49:03.909 INFO:tasks.workunit.client.1.smithi043.stdout:3/739: dread d4/f67 [0,4194304] 0 2024-09-16T23:49:03.913 INFO:tasks.workunit.client.0.smithi042.stdout:3/787: dread d2/d1b/d89/f23 [0,4194304] 0 2024-09-16T23:49:03.915 INFO:tasks.workunit.client.0.smithi042.stdout:7/676: dread d3/d98/da0/d24/d28/d9b/fac [0,4194304] 0 2024-09-16T23:49:03.916 INFO:tasks.workunit.client.0.smithi042.stdout:3/788: symlink d2/d95/dd8/l110 0 2024-09-16T23:49:03.917 INFO:tasks.workunit.client.1.smithi043.stdout:5/876: mknod d3/db/d84/db6/c127 0 2024-09-16T23:49:03.922 INFO:tasks.workunit.client.0.smithi042.stdout:7/677: rename d3/d98/da0/d24/d28/fa3 to d3/d98/fed 0 2024-09-16T23:49:03.923 INFO:tasks.workunit.client.0.smithi042.stdout:8/459: creat da/d19/d1d/d5a/f8e x:0 0 0 2024-09-16T23:49:03.925 INFO:tasks.workunit.client.0.smithi042.stdout:6/574: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/f47 [0,4194304] 0 2024-09-16T23:49:03.929 INFO:tasks.workunit.client.0.smithi042.stdout:6/575: fsync d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f80 0 2024-09-16T23:49:03.929 INFO:tasks.workunit.client.0.smithi042.stdout:8/460: truncate da/d19/d1d/d30/d3f/d6d/f76 525686 0 2024-09-16T23:49:03.933 INFO:tasks.workunit.client.0.smithi042.stdout:3/789: dwrite d2/d1b/fce [0,4194304] 0 2024-09-16T23:49:03.941 INFO:tasks.workunit.client.0.smithi042.stdout:3/790: rename d2/l6 to d2/d1b/d1d/l111 0 2024-09-16T23:49:03.949 INFO:tasks.workunit.client.0.smithi042.stdout:3/791: dwrite d2/d7/d61/d8e/d9d/dc4/dcc/fc2 [0,4194304] 0 2024-09-16T23:49:03.954 INFO:tasks.workunit.client.0.smithi042.stdout:6/576: dread d8/d70/d81/dd/d1b/f4b [0,4194304] 0 2024-09-16T23:49:03.954 INFO:tasks.workunit.client.0.smithi042.stdout:3/792: symlink d2/d95/dd8/l112 0 2024-09-16T23:49:03.959 INFO:tasks.workunit.client.0.smithi042.stdout:3/793: rmdir d2/d1b/d1d/d34/d50/df5 39 2024-09-16T23:49:03.962 INFO:tasks.workunit.client.0.smithi042.stdout:3/794: creat d2/d7/d61/d8e/d9d/dc4/dcc/f113 x:0 0 0 2024-09-16T23:49:03.964 INFO:tasks.workunit.client.0.smithi042.stdout:3/795: dread - d2/d7/d61/d8e/d7f/f82 zero size 2024-09-16T23:49:03.968 INFO:tasks.workunit.client.0.smithi042.stdout:3/796: truncate d2/d7/ff2 1036742 0 2024-09-16T23:49:03.975 INFO:tasks.workunit.client.0.smithi042.stdout:3/797: dwrite d2/d7/d61/d73/fde [0,4194304] 0 2024-09-16T23:49:03.981 INFO:tasks.workunit.client.0.smithi042.stdout:3/798: getdents d2/d7/d61/d8e/d7f/dfa 0 2024-09-16T23:49:03.986 INFO:tasks.workunit.client.0.smithi042.stdout:3/799: mkdir d2/d7/d61/d8e/d7f/dfa/d114 0 2024-09-16T23:49:03.988 INFO:tasks.workunit.client.0.smithi042.stdout:3/800: symlink d2/d9/d102/l115 0 2024-09-16T23:49:03.990 INFO:tasks.workunit.client.0.smithi042.stdout:3/801: mknod d2/d1b/c116 0 2024-09-16T23:49:04.004 INFO:tasks.workunit.client.0.smithi042.stdout:3/802: dwrite d2/f1f [0,4194304] 0 2024-09-16T23:49:04.015 INFO:tasks.workunit.client.0.smithi042.stdout:3/803: truncate d2/d1b/d1d/f85 1169609 0 2024-09-16T23:49:04.015 INFO:tasks.workunit.client.0.smithi042.stdout:3/804: chown d2/d95/dd8/l110 144 1 2024-09-16T23:49:04.017 INFO:tasks.workunit.client.0.smithi042.stdout:3/805: fsync d2/d7/d61/d8e/d2c/f32 0 2024-09-16T23:49:04.019 INFO:tasks.workunit.client.0.smithi042.stdout:3/806: rmdir d2/d1b/d1d/d71 39 2024-09-16T23:49:04.020 INFO:tasks.workunit.client.0.smithi042.stdout:2/557: truncate d0/d5d/f79 1150604 0 2024-09-16T23:49:04.025 INFO:tasks.workunit.client.0.smithi042.stdout:8/461: creat da/d19/d80/f8f x:0 0 0 2024-09-16T23:49:04.029 INFO:tasks.workunit.client.0.smithi042.stdout:3/807: truncate d2/d1b/d1d/f85 47047 0 2024-09-16T23:49:04.031 INFO:tasks.workunit.client.0.smithi042.stdout:8/462: stat da/d19/d1d/f2e 0 2024-09-16T23:49:04.032 INFO:tasks.workunit.client.0.smithi042.stdout:9/417: write d4/fb [1976146,22282] 0 2024-09-16T23:49:04.032 INFO:tasks.workunit.client.0.smithi042.stdout:3/808: creat d2/d1b/d1d/d34/d50/d93/f117 x:0 0 0 2024-09-16T23:49:04.034 INFO:tasks.workunit.client.0.smithi042.stdout:3/809: dread - d2/d7/d61/f107 zero size 2024-09-16T23:49:04.034 INFO:tasks.workunit.client.0.smithi042.stdout:8/463: write da/d19/d2c/f66 [564320,90870] 0 2024-09-16T23:49:04.043 INFO:tasks.workunit.client.1.smithi043.stdout:1/886: symlink d4/d9/d8f/db6/dcc/dda/d106/l131 0 2024-09-16T23:49:04.046 INFO:tasks.workunit.client.0.smithi042.stdout:9/418: mkdir d4/d12/d2d/d37/d28/d5e/d83 0 2024-09-16T23:49:04.049 INFO:tasks.workunit.client.0.smithi042.stdout:3/810: dwrite d2/d1b/d1d/d34/f6b [0,4194304] 0 2024-09-16T23:49:04.051 INFO:tasks.workunit.client.0.smithi042.stdout:3/811: fsync d2/d1b/d1d/d34/d62/f88 0 2024-09-16T23:49:04.052 INFO:tasks.workunit.client.0.smithi042.stdout:8/464: fdatasync da/d19/d1d/d30/d3f/f7f 0 2024-09-16T23:49:04.055 INFO:tasks.workunit.client.0.smithi042.stdout:3/812: dread - d2/d1b/d1d/d71/fc7 zero size 2024-09-16T23:49:04.059 INFO:tasks.workunit.client.1.smithi043.stdout:7/717: creat d4/d27/dbc/fe9 x:0 0 0 2024-09-16T23:49:04.066 INFO:tasks.workunit.client.0.smithi042.stdout:3/813: dread d2/d1b/fdc [0,4194304] 0 2024-09-16T23:49:04.072 INFO:tasks.workunit.client.1.smithi043.stdout:2/662: rename d8/d5e/d83/d9e/dab/fcb to d8/d1d/d20/d23/d38/db2/fe2 0 2024-09-16T23:49:04.074 INFO:tasks.workunit.client.0.smithi042.stdout:3/814: dwrite d2/f4 [4194304,4194304] 0 2024-09-16T23:49:04.078 INFO:tasks.workunit.client.0.smithi042.stdout:3/815: creat d2/d7/f118 x:0 0 0 2024-09-16T23:49:04.078 INFO:tasks.workunit.client.0.smithi042.stdout:6/577: getdents d8 0 2024-09-16T23:49:04.081 INFO:tasks.workunit.client.0.smithi042.stdout:3/816: write d2/d7/d61/d8e/d9d/dc4/dcc/f113 [894563,29343] 0 2024-09-16T23:49:04.081 INFO:tasks.workunit.client.1.smithi043.stdout:3/740: dread - d4/de/d35/f8d zero size 2024-09-16T23:49:04.081 INFO:tasks.workunit.client.0.smithi042.stdout:9/419: mknod d4/d12/d77/c84 0 2024-09-16T23:49:04.085 INFO:tasks.workunit.client.0.smithi042.stdout:9/420: truncate d4/d12/d2d/d37/d3a/f5c 35279 0 2024-09-16T23:49:04.087 INFO:tasks.workunit.client.0.smithi042.stdout:3/817: dread d2/d1b/d89/f5c [0,4194304] 0 2024-09-16T23:49:04.088 INFO:tasks.workunit.client.0.smithi042.stdout:9/421: read d4/d12/d2d/d37/f6a [550918,115954] 0 2024-09-16T23:49:04.088 INFO:tasks.workunit.client.0.smithi042.stdout:3/818: chown d2/d1b/d89/f3a 15392204 1 2024-09-16T23:49:04.096 INFO:tasks.workunit.client.0.smithi042.stdout:3/819: dwrite d2/d1b/d1d/d34/d62/fd4 [0,4194304] 0 2024-09-16T23:49:04.099 INFO:tasks.workunit.client.0.smithi042.stdout:3/820: read d2/d1b/d89/f47 [89326,35781] 0 2024-09-16T23:49:04.100 INFO:tasks.workunit.client.1.smithi043.stdout:7/718: creat d4/db/d11/d21/d31/d33/fea x:0 0 0 2024-09-16T23:49:04.101 INFO:tasks.workunit.client.0.smithi042.stdout:3/821: write d2/d1b/d1d/fa0 [430399,20890] 0 2024-09-16T23:49:04.104 INFO:tasks.workunit.client.1.smithi043.stdout:7/719: truncate d4/db/d11/d21/d31/d33/fcc 352282 0 2024-09-16T23:49:04.104 INFO:tasks.workunit.client.0.smithi042.stdout:1/636: sync 2024-09-16T23:49:04.106 INFO:tasks.workunit.client.0.smithi042.stdout:6/578: rename d8/d70/d81/l28 to d8/d70/d81/dd/lb7 0 2024-09-16T23:49:04.109 INFO:tasks.workunit.client.0.smithi042.stdout:2/558: unlink d0/d78/l85 0 2024-09-16T23:49:04.110 INFO:tasks.workunit.client.0.smithi042.stdout:8/465: rmdir da/d19/d1d/d30/d49/d58 39 2024-09-16T23:49:04.111 INFO:tasks.workunit.client.0.smithi042.stdout:3/822: dwrite d2/d1b/dc8/ff6 [0,4194304] 0 2024-09-16T23:49:04.112 INFO:tasks.workunit.client.1.smithi043.stdout:2/663: rmdir d8/d4b/d5f/d8e 39 2024-09-16T23:49:04.114 INFO:tasks.workunit.client.0.smithi042.stdout:8/466: read da/d19/d1d/d30/d3f/d6d/f7e [915456,85728] 0 2024-09-16T23:49:04.116 INFO:tasks.workunit.client.0.smithi042.stdout:3/823: dread d2/d95/fe8 [0,4194304] 0 2024-09-16T23:49:04.116 INFO:tasks.workunit.client.0.smithi042.stdout:8/467: chown da/d19/f23 75986406 1 2024-09-16T23:49:04.117 INFO:tasks.workunit.client.1.smithi043.stdout:9/675: mkdir d4/d1c/d4f/dd8 0 2024-09-16T23:49:04.117 INFO:tasks.workunit.client.0.smithi042.stdout:3/824: write d2/d1b/d1d/f38 [1943978,30339] 0 2024-09-16T23:49:04.118 INFO:tasks.workunit.client.1.smithi043.stdout:9/676: chown d4/c8 525884851 1 2024-09-16T23:49:04.121 INFO:tasks.workunit.client.1.smithi043.stdout:9/677: fdatasync d4/d21/d35/d4b/f80 0 2024-09-16T23:49:04.121 INFO:tasks.workunit.client.1.smithi043.stdout:6/718: dread de/d23/d34/d66/f6e [0,4194304] 0 2024-09-16T23:49:04.122 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:03 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:04.123 INFO:tasks.workunit.client.1.smithi043.stdout:5/877: truncate d3/d9/fc3 3644561 0 2024-09-16T23:49:04.126 INFO:tasks.workunit.client.0.smithi042.stdout:6/579: creat d8/d70/d81/dd/d1b/d50/fb8 x:0 0 0 2024-09-16T23:49:04.126 INFO:tasks.workunit.client.1.smithi043.stdout:7/720: chown d4/l8 29955 1 2024-09-16T23:49:04.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:03 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:04.127 INFO:tasks.workunit.client.0.smithi042.stdout:3/825: write d2/d1b/d1d/d30/da2/fdd [789385,53332] 0 2024-09-16T23:49:04.136 INFO:tasks.workunit.client.0.smithi042.stdout:3/826: unlink d2/d1b/d1d/d30/c9c 0 2024-09-16T23:49:04.137 INFO:tasks.workunit.client.1.smithi043.stdout:9/678: fsync d4/d1c/d25/d2b/d4c/f72 0 2024-09-16T23:49:04.139 INFO:tasks.workunit.client.1.smithi043.stdout:9/679: dread - d4/d1c/d25/d2b/d4c/da3/fc2 zero size 2024-09-16T23:49:04.142 INFO:tasks.workunit.client.1.smithi043.stdout:5/878: mknod d3/d79/d91/daf/c128 0 2024-09-16T23:49:04.143 INFO:tasks.workunit.client.0.smithi042.stdout:3/827: dwrite d2/d1b/d1d/d34/f3e [0,4194304] 0 2024-09-16T23:49:04.149 INFO:tasks.workunit.client.0.smithi042.stdout:3/828: mknod d2/d7/d61/d8e/d7f/dfa/d114/c119 0 2024-09-16T23:49:04.153 INFO:tasks.workunit.client.0.smithi042.stdout:3/829: mkdir d2/d1b/d1d/d34/d62/d11a 0 2024-09-16T23:49:04.162 INFO:tasks.workunit.client.1.smithi043.stdout:9/680: creat d4/d28/d5f/db6/fd9 x:0 0 0 2024-09-16T23:49:04.170 INFO:tasks.workunit.client.1.smithi043.stdout:3/741: getdents d4/d30/d53 0 2024-09-16T23:49:04.174 INFO:tasks.workunit.client.1.smithi043.stdout:9/681: creat d4/d5/d16/fda x:0 0 0 2024-09-16T23:49:04.183 INFO:tasks.workunit.client.1.smithi043.stdout:6/719: link de/d23/d34/f5d de/d1a/d28/ff1 0 2024-09-16T23:49:04.185 INFO:tasks.workunit.client.1.smithi043.stdout:9/682: symlink d4/d28/d49/ldb 0 2024-09-16T23:49:04.186 INFO:tasks.workunit.client.1.smithi043.stdout:9/683: dread - d4/d1c/d25/d2b/f8a zero size 2024-09-16T23:49:04.187 INFO:tasks.workunit.client.1.smithi043.stdout:7/721: rename d4/d27/d41/d1b/f4e to d4/d27/d41/d1b/d66/feb 0 2024-09-16T23:49:04.188 INFO:tasks.workunit.client.0.smithi042.stdout:0/518: sync 2024-09-16T23:49:04.189 INFO:tasks.workunit.client.0.smithi042.stdout:0/519: fsync d0/f9 0 2024-09-16T23:49:04.191 INFO:tasks.workunit.client.0.smithi042.stdout:0/520: fsync d0/df/d23/d33/d51/f52 0 2024-09-16T23:49:04.193 INFO:tasks.workunit.client.0.smithi042.stdout:0/521: write d0/df/d23/d33/d51/d5b/fa2 [48461,8108] 0 2024-09-16T23:49:04.194 INFO:tasks.workunit.client.1.smithi043.stdout:6/720: creat de/d1a/d39/ff2 x:0 0 0 2024-09-16T23:49:04.196 INFO:tasks.workunit.client.0.smithi042.stdout:0/522: rmdir d0/df/d23/d33/d51/d79 39 2024-09-16T23:49:04.199 INFO:tasks.workunit.client.0.smithi042.stdout:0/523: symlink d0/df/d23/d3e/d8f/la3 0 2024-09-16T23:49:04.199 INFO:tasks.workunit.client.1.smithi043.stdout:7/722: creat d4/d27/d41/d1b/d95/fec x:0 0 0 2024-09-16T23:49:04.201 INFO:tasks.workunit.client.0.smithi042.stdout:0/524: mknod d0/df/d13/ca4 0 2024-09-16T23:49:04.206 INFO:tasks.workunit.client.1.smithi043.stdout:6/721: mkdir de/d23/d34/d66/d9d/df3 0 2024-09-16T23:49:04.206 INFO:tasks.workunit.client.0.smithi042.stdout:0/525: link d0/df/d13/d49/d70/d94/f92 d0/df/d13/d2d/d35/fa5 0 2024-09-16T23:49:04.210 INFO:tasks.workunit.client.0.smithi042.stdout:0/526: truncate d0/df/d23/d33/d56/f5d 2236553 0 2024-09-16T23:49:04.212 INFO:tasks.workunit.client.1.smithi043.stdout:3/742: link d4/de/d40/fb5 d4/de/d1f/d2c/db7/dc3/fe0 0 2024-09-16T23:49:04.212 INFO:tasks.workunit.client.1.smithi043.stdout:3/743: truncate d4/f3b 1228705 0 2024-09-16T23:49:04.218 INFO:tasks.workunit.client.1.smithi043.stdout:7/723: mknod d4/db/d43/ced 0 2024-09-16T23:49:04.219 INFO:tasks.workunit.client.1.smithi043.stdout:7/724: read - d4/db/d43/d83/d85/d8b/faf zero size 2024-09-16T23:49:04.220 INFO:tasks.workunit.client.0.smithi042.stdout:8/468: creat da/f90 x:0 0 0 2024-09-16T23:49:04.221 INFO:tasks.workunit.client.1.smithi043.stdout:3/744: mknod d4/de/d1f/d34/ce1 0 2024-09-16T23:49:04.223 INFO:tasks.workunit.client.0.smithi042.stdout:0/527: dwrite d0/f78 [0,4194304] 0 2024-09-16T23:49:04.223 INFO:tasks.workunit.client.1.smithi043.stdout:6/722: creat de/d23/d49/dc9/ff4 x:0 0 0 2024-09-16T23:49:04.226 INFO:tasks.workunit.client.0.smithi042.stdout:8/469: creat da/d19/d1d/d30/d3f/d6d/f91 x:0 0 0 2024-09-16T23:49:04.227 INFO:tasks.workunit.client.0.smithi042.stdout:0/528: chown d0/df/d13/d2d/d35/c4d 2976809 1 2024-09-16T23:49:04.227 INFO:tasks.workunit.client.0.smithi042.stdout:8/470: chown da/d19 0 1 2024-09-16T23:49:04.228 INFO:tasks.workunit.client.1.smithi043.stdout:6/723: rename de/d1a/c60 to de/d23/d34/d9b/d5a/cf5 0 2024-09-16T23:49:04.231 INFO:tasks.workunit.client.0.smithi042.stdout:8/471: mknod da/d19/d1d/d30/c92 0 2024-09-16T23:49:04.232 INFO:tasks.workunit.client.1.smithi043.stdout:6/724: write de/d23/d34/d9b/d8b/fea [5079,22230] 0 2024-09-16T23:49:04.233 INFO:tasks.workunit.client.0.smithi042.stdout:0/529: getdents d0/d48 0 2024-09-16T23:49:04.238 INFO:tasks.workunit.client.0.smithi042.stdout:0/530: link d0/df/d23/d3e/l5a d0/df/d23/d33/d51/d5b/la6 0 2024-09-16T23:49:04.239 INFO:tasks.workunit.client.0.smithi042.stdout:0/531: fdatasync d0/f67 0 2024-09-16T23:49:04.239 INFO:tasks.workunit.client.0.smithi042.stdout:8/472: dread da/d19/d1d/d5a/f79 [0,4194304] 0 2024-09-16T23:49:04.240 INFO:tasks.workunit.client.1.smithi043.stdout:3/745: dread d4/de/d1f/d2c/f32 [0,4194304] 0 2024-09-16T23:49:04.241 INFO:tasks.workunit.client.1.smithi043.stdout:0/798: rmdir d0/d2/df 39 2024-09-16T23:49:04.241 INFO:tasks.workunit.client.1.smithi043.stdout:3/746: dread d4/de/d40/fbf [0,4194304] 0 2024-09-16T23:49:04.254 INFO:tasks.workunit.client.0.smithi042.stdout:8/473: link da/c7c da/d4b/c93 0 2024-09-16T23:49:04.255 INFO:tasks.workunit.client.0.smithi042.stdout:8/474: read da/f17 [3300070,65038] 0 2024-09-16T23:49:04.258 INFO:tasks.workunit.client.1.smithi043.stdout:0/799: rename d0/d2/df/d1a/d3b/f6c to d0/d2/df/d1a/d46/d39/daa/f10c 0 2024-09-16T23:49:04.261 INFO:tasks.workunit.client.1.smithi043.stdout:4/798: symlink d8/d25/lf9 0 2024-09-16T23:49:04.262 INFO:tasks.workunit.client.0.smithi042.stdout:8/475: symlink da/d19/d1d/d30/l94 0 2024-09-16T23:49:04.265 INFO:tasks.workunit.client.1.smithi043.stdout:4/799: unlink d8/d15/d20/d2b/d54/dce/cc3 0 2024-09-16T23:49:04.266 INFO:tasks.workunit.client.0.smithi042.stdout:8/476: unlink da/d19/d80/f84 0 2024-09-16T23:49:04.267 INFO:tasks.workunit.client.1.smithi043.stdout:4/800: truncate d8/d15/d20/d2b/dc8/fe3 275553 0 2024-09-16T23:49:04.267 INFO:tasks.workunit.client.0.smithi042.stdout:8/477: fdatasync da/d4b/f87 0 2024-09-16T23:49:04.268 INFO:tasks.workunit.client.0.smithi042.stdout:8/478: read - da/d4b/f87 zero size 2024-09-16T23:49:04.271 INFO:tasks.workunit.client.0.smithi042.stdout:8/479: stat da/d19/d1d/f63 0 2024-09-16T23:49:04.272 INFO:tasks.workunit.client.0.smithi042.stdout:8/480: truncate da/d19/f7a 806380 0 2024-09-16T23:49:04.273 INFO:tasks.workunit.client.1.smithi043.stdout:8/710: truncate de/fb0 4111526 0 2024-09-16T23:49:04.277 INFO:tasks.workunit.client.1.smithi043.stdout:8/711: creat de/d70/d3d/daf/fd6 x:0 0 0 2024-09-16T23:49:04.277 INFO:tasks.workunit.client.0.smithi042.stdout:8/481: fsync da/f2a 0 2024-09-16T23:49:04.280 INFO:tasks.workunit.client.0.smithi042.stdout:8/482: mknod da/d19/d80/c95 0 2024-09-16T23:49:04.282 INFO:tasks.workunit.client.0.smithi042.stdout:8/483: readlink da/d19/d1d/d30/d49/d58/d6a/l7d 0 2024-09-16T23:49:04.282 INFO:tasks.workunit.client.1.smithi043.stdout:8/712: dwrite de/d18/fb1 [0,4194304] 0 2024-09-16T23:49:04.283 INFO:tasks.workunit.client.0.smithi042.stdout:9/422: truncate d4/d10/f1b 199065 0 2024-09-16T23:49:04.284 INFO:tasks.workunit.client.1.smithi043.stdout:8/713: stat de/d18/d27/d41/fd1 0 2024-09-16T23:49:04.287 INFO:tasks.workunit.client.0.smithi042.stdout:9/423: creat d4/d12/d16/f85 x:0 0 0 2024-09-16T23:49:04.289 INFO:tasks.workunit.client.0.smithi042.stdout:9/424: write d4/d12/d2d/d37/d3a/d50/f7d [437094,11777] 0 2024-09-16T23:49:04.289 INFO:tasks.workunit.client.0.smithi042.stdout:8/484: creat da/f96 x:0 0 0 2024-09-16T23:49:04.290 INFO:tasks.workunit.client.1.smithi043.stdout:8/714: symlink de/d18/d1f/d25/d82/ld7 0 2024-09-16T23:49:04.291 INFO:tasks.workunit.client.0.smithi042.stdout:8/485: write da/d19/f1f [1177754,32148] 0 2024-09-16T23:49:04.292 INFO:tasks.workunit.client.1.smithi043.stdout:8/715: dread - de/d18/d1f/d2f/d39/d7b/d83/fc0 zero size 2024-09-16T23:49:04.301 INFO:tasks.workunit.client.1.smithi043.stdout:2/664: write d8/d4b/d4f/d56/f95 [1248523,9532] 0 2024-09-16T23:49:04.302 INFO:tasks.workunit.client.1.smithi043.stdout:8/716: dread de/d18/d27/d41/d5a/f13 [0,4194304] 0 2024-09-16T23:49:04.303 INFO:tasks.workunit.client.1.smithi043.stdout:3/747: creat d4/de/fe2 x:0 0 0 2024-09-16T23:49:04.309 INFO:tasks.workunit.client.1.smithi043.stdout:1/887: mkdir d4/d9/d1c/d1e/d132 0 2024-09-16T23:49:04.312 INFO:tasks.workunit.client.1.smithi043.stdout:1/888: write d4/d9/d8f/db6/dcc/dda/d107/fbb [9887756,993] 0 2024-09-16T23:49:04.312 INFO:tasks.workunit.client.1.smithi043.stdout:3/748: dwrite d4/de/d35/f1d [4194304,4194304] 0 2024-09-16T23:49:04.312 INFO:tasks.workunit.client.1.smithi043.stdout:5/879: write d3/db/d19/d3e/d28/dcb/ff7 [301333,69718] 0 2024-09-16T23:49:04.314 INFO:tasks.workunit.client.1.smithi043.stdout:3/749: dread - d4/de/d1f/fdb zero size 2024-09-16T23:49:04.315 INFO:tasks.workunit.client.1.smithi043.stdout:5/880: creat d3/d9/d22/f129 x:0 0 0 2024-09-16T23:49:04.326 INFO:tasks.workunit.client.0.smithi042.stdout:1/637: rename d0/d9/d90/d6b/d88/d32/fde to d0/d3b/fe0 0 2024-09-16T23:49:04.326 INFO:tasks.workunit.client.0.smithi042.stdout:2/559: rename d0/d78 to d0/d78/d8b/dbd 22 2024-09-16T23:49:04.326 INFO:tasks.workunit.client.1.smithi043.stdout:9/684: write d4/d1c/d25/d2b/d4c/da3/fc2 [143474,93132] 0 2024-09-16T23:49:04.329 INFO:tasks.workunit.client.0.smithi042.stdout:3/830: rename d2/d9/l10a to d2/d1b/dc8/l11b 0 2024-09-16T23:49:04.331 INFO:tasks.workunit.client.0.smithi042.stdout:9/425: rename d4/d12/d2d/d37 to d4/d12/d2d/d37/d59/d86 22 2024-09-16T23:49:04.332 INFO:tasks.workunit.client.0.smithi042.stdout:3/831: readlink d2/d7/d61/d8e/d7f/l8c 0 2024-09-16T23:49:04.332 INFO:tasks.workunit.client.1.smithi043.stdout:9/685: dread d4/d1c/d6b/f39 [0,4194304] 0 2024-09-16T23:49:04.333 INFO:tasks.workunit.client.0.smithi042.stdout:2/560: dread d0/d30/d43/f63 [0,4194304] 0 2024-09-16T23:49:04.334 INFO:tasks.workunit.client.0.smithi042.stdout:1/638: dread d0/d72/f65 [0,4194304] 0 2024-09-16T23:49:04.334 INFO:tasks.workunit.client.1.smithi043.stdout:3/750: dread d4/de/d1f/d34/f74 [0,4194304] 0 2024-09-16T23:49:04.335 INFO:tasks.workunit.client.0.smithi042.stdout:3/832: symlink d2/l11c 0 2024-09-16T23:49:04.336 INFO:tasks.workunit.client.0.smithi042.stdout:9/426: creat d4/d12/d2d/d37/d28/f87 x:0 0 0 2024-09-16T23:49:04.337 INFO:tasks.workunit.client.1.smithi043.stdout:9/686: truncate d4/d21/d35/f5d 1960664 0 2024-09-16T23:49:04.337 INFO:tasks.workunit.client.0.smithi042.stdout:3/833: symlink d2/l11d 0 2024-09-16T23:49:04.338 INFO:tasks.workunit.client.1.smithi043.stdout:9/687: write d4/d21/d2a/d2c/d7b/faa [739585,20741] 0 2024-09-16T23:49:04.339 INFO:tasks.workunit.client.1.smithi043.stdout:3/751: dread d4/de/d40/fb5 [0,4194304] 0 2024-09-16T23:49:04.340 INFO:tasks.workunit.client.1.smithi043.stdout:9/688: write d4/d21/d2a/f99 [857500,101761] 0 2024-09-16T23:49:04.340 INFO:tasks.workunit.client.1.smithi043.stdout:2/665: rename d8/d5e/d83/l8f to d8/d1d/d20/d23/d38/le3 0 2024-09-16T23:49:04.341 INFO:tasks.workunit.client.0.smithi042.stdout:2/561: truncate d0/da/d88/dae/f6b 1063248 0 2024-09-16T23:49:04.341 INFO:tasks.workunit.client.0.smithi042.stdout:6/580: write d8/d70/d81/dd/d1b/d34/d39/d5e/f7c [1884178,104689] 0 2024-09-16T23:49:04.342 INFO:tasks.workunit.client.1.smithi043.stdout:3/752: dread - d4/d30/d53/d88/f93 zero size 2024-09-16T23:49:04.342 INFO:tasks.workunit.client.0.smithi042.stdout:3/834: mkdir d2/d1b/d1d/d34/d50/d93/d96/d11e 0 2024-09-16T23:49:04.345 INFO:tasks.workunit.client.0.smithi042.stdout:3/835: mkdir d2/d1b/d1d/d34/d62/d11a/d11f 0 2024-09-16T23:49:04.345 INFO:tasks.workunit.client.0.smithi042.stdout:9/427: creat d4/d12/d2d/d37/d59/f88 x:0 0 0 2024-09-16T23:49:04.347 INFO:tasks.workunit.client.1.smithi043.stdout:5/881: rename d3/d9/d6f/d7c/d80/cf2 to d3/d79/d91/c12a 0 2024-09-16T23:49:04.347 INFO:tasks.workunit.client.0.smithi042.stdout:6/581: dread d8/d70/d81/dd/d1b/d67/d97/f9a [0,4194304] 0 2024-09-16T23:49:04.348 INFO:tasks.workunit.client.0.smithi042.stdout:3/836: creat d2/d7/d61/d8e/d9d/dc4/de4/f120 x:0 0 0 2024-09-16T23:49:04.349 INFO:tasks.workunit.client.1.smithi043.stdout:2/666: creat d8/d1d/d20/d23/d38/da2/fe4 x:0 0 0 2024-09-16T23:49:04.350 INFO:tasks.workunit.client.1.smithi043.stdout:2/667: chown d8/d1d/d25/fd9 1 1 2024-09-16T23:49:04.351 INFO:tasks.workunit.client.0.smithi042.stdout:3/837: symlink d2/d1b/d1d/d34/d62/dfd/l121 0 2024-09-16T23:49:04.352 INFO:tasks.workunit.client.1.smithi043.stdout:3/753: read d4/de/d40/fbf [154015,73134] 0 2024-09-16T23:49:04.352 INFO:tasks.workunit.client.1.smithi043.stdout:8/717: rmdir de/d18/d2b 39 2024-09-16T23:49:04.353 INFO:tasks.workunit.client.1.smithi043.stdout:3/754: dread d4/d30/fcc [0,4194304] 0 2024-09-16T23:49:04.358 INFO:tasks.workunit.client.0.smithi042.stdout:3/838: dwrite d2/d1b/d1d/d34/d62/f9e [0,4194304] 0 2024-09-16T23:49:04.358 INFO:tasks.workunit.client.1.smithi043.stdout:9/689: rename d4/d21/d2a/da9 to d4/d1c/ddc 0 2024-09-16T23:49:04.360 INFO:tasks.workunit.client.0.smithi042.stdout:9/428: chown d4/d10/f3f 4360612 1 2024-09-16T23:49:04.361 INFO:tasks.workunit.client.0.smithi042.stdout:9/429: chown d4/d10/d24 4334237 1 2024-09-16T23:49:04.362 INFO:tasks.workunit.client.0.smithi042.stdout:3/839: stat d2/d7/d61/d8e/c29 0 2024-09-16T23:49:04.362 INFO:tasks.workunit.client.0.smithi042.stdout:6/582: dread d8/d70/d81/dd/d1b/d34/f64 [0,4194304] 0 2024-09-16T23:49:04.364 INFO:tasks.workunit.client.0.smithi042.stdout:3/840: write d2/d7/d61/d8e/d9d/dc4/dcc/f113 [1868069,61316] 0 2024-09-16T23:49:04.370 INFO:tasks.workunit.client.1.smithi043.stdout:6/725: dwrite de/d23/d34/d66/d70/f76 [0,4194304] 0 2024-09-16T23:49:04.371 INFO:tasks.workunit.client.1.smithi043.stdout:6/726: dread - de/d23/d34/d9b/d5a/da3/fef zero size 2024-09-16T23:49:04.371 INFO:tasks.workunit.client.1.smithi043.stdout:8/718: rmdir de/d18/d27/d41/d5a/d37/d6a 39 2024-09-16T23:49:04.372 INFO:tasks.workunit.client.0.smithi042.stdout:9/430: write d4/d10/f6b [852410,110630] 0 2024-09-16T23:49:04.373 INFO:tasks.workunit.client.0.smithi042.stdout:9/431: fdatasync d4/d10/f30 0 2024-09-16T23:49:04.373 INFO:tasks.workunit.client.1.smithi043.stdout:3/755: truncate d4/de/d35/f4c 1129715 0 2024-09-16T23:49:04.374 INFO:tasks.workunit.client.0.smithi042.stdout:1/639: getdents d0/d17/dd4 0 2024-09-16T23:49:04.374 INFO:tasks.workunit.client.0.smithi042.stdout:3/841: fsync d2/d9/f22 0 2024-09-16T23:49:04.381 INFO:tasks.workunit.client.1.smithi043.stdout:0/800: truncate d0/d2/df/d1a/d46/d39/d6e/f32 5507070 0 2024-09-16T23:49:04.382 INFO:tasks.workunit.client.1.smithi043.stdout:2/668: symlink d8/d5e/d83/d3a/le5 0 2024-09-16T23:49:04.386 INFO:tasks.workunit.client.0.smithi042.stdout:9/432: dread d4/d10/f6b [0,4194304] 0 2024-09-16T23:49:04.390 INFO:tasks.workunit.client.0.smithi042.stdout:2/562: rename d0/da/f23 to d0/da/d17/fbe 0 2024-09-16T23:49:04.391 INFO:tasks.workunit.client.1.smithi043.stdout:4/801: dwrite d8/d25/d39/d41/d57/fae [0,4194304] 0 2024-09-16T23:49:04.393 INFO:tasks.workunit.client.1.smithi043.stdout:4/802: chown d8/d15/d20/d2b/d54/db3/dd5 73 1 2024-09-16T23:49:04.397 INFO:tasks.workunit.client.0.smithi042.stdout:1/640: symlink d0/d17/dc7/le1 0 2024-09-16T23:49:04.397 INFO:tasks.workunit.client.0.smithi042.stdout:1/641: readlink d0/d17/dd4/ld7 0 2024-09-16T23:49:04.398 INFO:tasks.workunit.client.1.smithi043.stdout:9/690: dread d4/d21/d33/f79 [0,4194304] 0 2024-09-16T23:49:04.398 INFO:tasks.workunit.client.0.smithi042.stdout:8/486: rmdir da 39 2024-09-16T23:49:04.402 INFO:tasks.workunit.client.0.smithi042.stdout:9/433: unlink d4/d12/d2d/d37/d59/c5a 0 2024-09-16T23:49:04.404 INFO:tasks.workunit.client.1.smithi043.stdout:9/691: write d4/d28/d5f/db6/fd9 [165559,35134] 0 2024-09-16T23:49:04.405 INFO:tasks.workunit.client.1.smithi043.stdout:3/756: rename d4/de/d1f/d2c/db7/dc3/fd7 to d4/de/d1f/d2c/db7/dc9/fe3 0 2024-09-16T23:49:04.410 INFO:tasks.workunit.client.0.smithi042.stdout:6/583: rename f6 to d8/d70/d81/dd/d1b/d50/fb9 0 2024-09-16T23:49:04.414 INFO:tasks.workunit.client.0.smithi042.stdout:2/563: rmdir d0/da/d88 39 2024-09-16T23:49:04.416 INFO:tasks.workunit.client.0.smithi042.stdout:2/564: chown d0/d30/d43/d77/fbc 141944050 1 2024-09-16T23:49:04.416 INFO:tasks.workunit.client.0.smithi042.stdout:1/642: dread d0/d9/d90/d6b/d88/f12 [0,4194304] 0 2024-09-16T23:49:04.419 INFO:tasks.workunit.client.1.smithi043.stdout:4/803: rmdir d8/d15/d20/d2b/d23 39 2024-09-16T23:49:04.424 INFO:tasks.workunit.client.0.smithi042.stdout:2/565: dwrite d0/d26/f2f [0,4194304] 0 2024-09-16T23:49:04.425 INFO:tasks.workunit.client.1.smithi043.stdout:1/889: dwrite d4/d9/d12/d2b/fb0 [0,4194304] 0 2024-09-16T23:49:04.425 INFO:tasks.workunit.client.1.smithi043.stdout:6/727: chown de/d23/d63/cb7 6080 1 2024-09-16T23:49:04.427 INFO:tasks.workunit.client.1.smithi043.stdout:9/692: dread d4/d21/d2a/d2c/d7b/f7e [0,4194304] 0 2024-09-16T23:49:04.429 INFO:tasks.workunit.client.0.smithi042.stdout:6/584: creat d8/d70/fba x:0 0 0 2024-09-16T23:49:04.429 INFO:tasks.workunit.client.1.smithi043.stdout:9/693: chown d4/d1c/l68 801 1 2024-09-16T23:49:04.431 INFO:tasks.workunit.client.0.smithi042.stdout:6/585: write d8/d70/d81/dd/d1b/f9d [974289,120669] 0 2024-09-16T23:49:04.431 INFO:tasks.workunit.client.1.smithi043.stdout:9/694: read - d4/d1c/d25/d2b/d4c/fb0 zero size 2024-09-16T23:49:04.433 INFO:tasks.workunit.client.0.smithi042.stdout:6/586: write d8/d70/d81/dd/d1b/d34/d39/d5e/f7c [804070,119383] 0 2024-09-16T23:49:04.434 INFO:tasks.workunit.client.0.smithi042.stdout:3/842: creat d2/d1b/d1d/d34/d50/d93/d96/f122 x:0 0 0 2024-09-16T23:49:04.435 INFO:tasks.workunit.client.0.smithi042.stdout:6/587: chown d8/d70/d81/dd/d2c/d57 54 1 2024-09-16T23:49:04.435 INFO:tasks.workunit.client.0.smithi042.stdout:1/643: mkdir d0/d9/d90/d66/de2 0 2024-09-16T23:49:04.438 INFO:tasks.workunit.client.0.smithi042.stdout:6/588: chown d8/d70/d81/dd/d1b/d34/f64 45970050 1 2024-09-16T23:49:04.442 INFO:tasks.workunit.client.1.smithi043.stdout:3/757: unlink d4/de/d1f/d34/l5b 0 2024-09-16T23:49:04.445 INFO:tasks.workunit.client.0.smithi042.stdout:5/717: sync 2024-09-16T23:49:04.449 INFO:tasks.workunit.client.0.smithi042.stdout:6/589: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/fab [0,4194304] 0 2024-09-16T23:49:04.449 INFO:tasks.workunit.client.0.smithi042.stdout:5/718: chown d9/db/de/d5e/l69 22117 1 2024-09-16T23:49:04.449 INFO:tasks.workunit.client.1.smithi043.stdout:5/882: dwrite d3/d79/d91/daf/fc5 [0,4194304] 0 2024-09-16T23:49:04.457 INFO:tasks.workunit.client.1.smithi043.stdout:4/804: rename d8/d15/d20/d2b/d54/d55 to d8/d15/d20/dfa 0 2024-09-16T23:49:04.461 INFO:tasks.workunit.client.0.smithi042.stdout:5/719: fdatasync d9/db/d11/f1c 0 2024-09-16T23:49:04.462 INFO:tasks.workunit.client.0.smithi042.stdout:3/843: dread d2/d1b/d1d/d34/f5d [0,4194304] 0 2024-09-16T23:49:04.462 INFO:tasks.workunit.client.0.smithi042.stdout:5/720: dread - d9/db/d11/fd7 zero size 2024-09-16T23:49:04.465 INFO:tasks.workunit.client.0.smithi042.stdout:3/844: mknod d2/d1b/d89/c123 0 2024-09-16T23:49:04.466 INFO:tasks.workunit.client.0.smithi042.stdout:5/721: read d9/db/d11/f1c [4922917,36239] 0 2024-09-16T23:49:04.472 INFO:tasks.workunit.client.0.smithi042.stdout:1/644: mkdir d0/d9/d90/d6b/d88/d32/d2f/de3 0 2024-09-16T23:49:04.474 INFO:tasks.workunit.client.0.smithi042.stdout:5/722: link d9/db/d19/d21/d85/da5/c90 d9/db/d19/cfd 0 2024-09-16T23:49:04.474 INFO:tasks.workunit.client.0.smithi042.stdout:3/845: dwrite d2/d7/ff2 [0,4194304] 0 2024-09-16T23:49:04.474 INFO:tasks.workunit.client.0.smithi042.stdout:1/645: write d0/d9a/fb5 [1219542,48753] 0 2024-09-16T23:49:04.479 INFO:tasks.workunit.client.0.smithi042.stdout:3/846: rename d2/d9/c69 to d2/d1b/d89/db4/c124 0 2024-09-16T23:49:04.480 INFO:tasks.workunit.client.0.smithi042.stdout:5/723: fdatasync d9/db/de/d5e/d87/d61/f5d 0 2024-09-16T23:49:04.481 INFO:tasks.workunit.client.0.smithi042.stdout:5/724: stat d9/d63 0 2024-09-16T23:49:04.481 INFO:tasks.workunit.client.0.smithi042.stdout:3/847: chown d2/d7/d61/d8e/d7f 208294977 1 2024-09-16T23:49:04.482 INFO:tasks.workunit.client.1.smithi043.stdout:3/758: creat d4/d30/d53/d88/db1/dbb/fe4 x:0 0 0 2024-09-16T23:49:04.483 INFO:tasks.workunit.client.1.smithi043.stdout:8/719: write de/d18/d1f/d2f/d39/f81 [541165,10964] 0 2024-09-16T23:49:04.483 INFO:tasks.workunit.client.0.smithi042.stdout:5/725: read d9/fef [230325,124978] 0 2024-09-16T23:49:04.487 INFO:tasks.workunit.client.1.smithi043.stdout:8/720: truncate de/d18/d1f/d2f/d39/d7b/fba 961736 0 2024-09-16T23:49:04.490 INFO:tasks.workunit.client.1.smithi043.stdout:3/759: dwrite d4/de/d35/f1d [0,4194304] 0 2024-09-16T23:49:04.490 INFO:tasks.workunit.client.1.smithi043.stdout:0/801: write d0/d2/df/d1a/d49/db2/df4/fe4 [135761,55035] 0 2024-09-16T23:49:04.491 INFO:tasks.workunit.client.0.smithi042.stdout:3/848: write d2/d1b/d1d/d34/d50/d93/fff [1039017,78875] 0 2024-09-16T23:49:04.494 INFO:tasks.workunit.client.0.smithi042.stdout:8/487: mkdir da/d19/d2c/d56/d97 0 2024-09-16T23:49:04.494 INFO:tasks.workunit.client.0.smithi042.stdout:6/590: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f7f [0,4194304] 0 2024-09-16T23:49:04.495 INFO:tasks.workunit.client.1.smithi043.stdout:9/695: mknod d4/d21/cdd 0 2024-09-16T23:49:04.496 INFO:tasks.workunit.client.0.smithi042.stdout:3/849: creat d2/d7/d61/d8e/d2c/f125 x:0 0 0 2024-09-16T23:49:04.496 INFO:tasks.workunit.client.1.smithi043.stdout:9/696: chown d4/d21/d35 103409 1 2024-09-16T23:49:04.497 INFO:tasks.workunit.client.1.smithi043.stdout:9/697: write d4/d1c/d25/d2b/fc6 [1006561,103533] 0 2024-09-16T23:49:04.499 INFO:tasks.workunit.client.0.smithi042.stdout:9/434: getdents d4/d10/d24 0 2024-09-16T23:49:04.501 INFO:tasks.workunit.client.0.smithi042.stdout:9/435: chown d4/d12/d2d/d37/d59/c7f 1160329 1 2024-09-16T23:49:04.501 INFO:tasks.workunit.client.0.smithi042.stdout:3/850: mkdir d2/d7/d61/d8e/d7f/d126 0 2024-09-16T23:49:04.502 INFO:tasks.workunit.client.0.smithi042.stdout:9/436: fdatasync d4/d12/d16/d68/f71 0 2024-09-16T23:49:04.504 INFO:tasks.workunit.client.0.smithi042.stdout:9/437: write d4/d12/d2d/d37/f70 [1031226,92688] 0 2024-09-16T23:49:04.505 INFO:tasks.workunit.client.1.smithi043.stdout:9/698: dwrite d4/d21/d2a/d2c/f69 [4194304,4194304] 0 2024-09-16T23:49:04.506 INFO:tasks.workunit.client.0.smithi042.stdout:9/438: stat d4/d12/d44 0 2024-09-16T23:49:04.511 INFO:tasks.workunit.client.1.smithi043.stdout:2/669: dwrite d8/d1d/d20/d26/f7d [0,4194304] 0 2024-09-16T23:49:04.513 INFO:tasks.workunit.client.1.smithi043.stdout:8/721: mkdir de/d18/d1f/d2f/d39/d7b/dd8 0 2024-09-16T23:49:04.514 INFO:tasks.workunit.client.0.smithi042.stdout:8/488: creat da/d19/d2c/d56/f98 x:0 0 0 2024-09-16T23:49:04.514 INFO:tasks.workunit.client.0.smithi042.stdout:2/566: dwrite d0/d78/d8b/f99 [0,4194304] 0 2024-09-16T23:49:04.514 INFO:tasks.workunit.client.0.smithi042.stdout:7/678: sync 2024-09-16T23:49:04.514 INFO:tasks.workunit.client.0.smithi042.stdout:4/795: sync 2024-09-16T23:49:04.515 INFO:tasks.workunit.client.1.smithi043.stdout:8/722: dread - de/d18/d1f/d2f/d39/d7b/fb5 zero size 2024-09-16T23:49:04.516 INFO:tasks.workunit.client.0.smithi042.stdout:6/591: dread d8/d70/d81/dd/d2c/f13 [0,4194304] 0 2024-09-16T23:49:04.520 INFO:tasks.workunit.client.0.smithi042.stdout:6/592: fsync d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/f6b 0 2024-09-16T23:49:04.520 INFO:tasks.workunit.client.0.smithi042.stdout:4/796: unlink de/d2e/d37/d3e/ff1 0 2024-09-16T23:49:04.522 INFO:tasks.workunit.client.0.smithi042.stdout:6/593: chown d8/d70/d81/dd/d1b/d2d/c3a 17 1 2024-09-16T23:49:04.523 INFO:tasks.workunit.client.1.smithi043.stdout:3/760: write d4/f1b [4972520,19214] 0 2024-09-16T23:49:04.524 INFO:tasks.workunit.client.1.smithi043.stdout:1/890: creat d4/d9/f133 x:0 0 0 2024-09-16T23:49:04.524 INFO:tasks.workunit.client.0.smithi042.stdout:7/679: read d3/d98/f89 [570296,27359] 0 2024-09-16T23:49:04.524 INFO:tasks.workunit.client.0.smithi042.stdout:4/797: creat de/d42/d82/f115 x:0 0 0 2024-09-16T23:49:04.527 INFO:tasks.workunit.client.1.smithi043.stdout:0/802: creat d0/d2/df/d1a/d49/d80/d88/f10d x:0 0 0 2024-09-16T23:49:04.528 INFO:tasks.workunit.client.0.smithi042.stdout:4/798: mknod de/d42/d5d/d8f/d9f/dc0/c116 0 2024-09-16T23:49:04.528 INFO:tasks.workunit.client.1.smithi043.stdout:8/723: dwrite de/fd0 [0,4194304] 0 2024-09-16T23:49:04.530 INFO:tasks.workunit.client.0.smithi042.stdout:4/799: truncate de/d2e/fed 84307 0 2024-09-16T23:49:04.531 INFO:tasks.workunit.client.1.smithi043.stdout:5/883: write d3/db/d15/f75 [286925,105429] 0 2024-09-16T23:49:04.532 INFO:tasks.workunit.client.0.smithi042.stdout:4/800: mknod de/d2e/d37/d3c/c117 0 2024-09-16T23:49:04.532 INFO:tasks.workunit.client.0.smithi042.stdout:1/646: rename d0/d9/l28 to d0/d9/d90/d6b/le4 0 2024-09-16T23:49:04.533 INFO:tasks.workunit.client.0.smithi042.stdout:7/680: getdents d3/d1f/d49/d76/d19/ddc/d6b/d64 0 2024-09-16T23:49:04.535 INFO:tasks.workunit.client.0.smithi042.stdout:4/801: readlink de/d42/d68/lde 0 2024-09-16T23:49:04.535 INFO:tasks.workunit.client.0.smithi042.stdout:7/681: dread - d3/f9c zero size 2024-09-16T23:49:04.536 INFO:tasks.workunit.client.0.smithi042.stdout:5/726: rename d9/db/d1b/d31/d46/f57 to d9/db/d1b/da4/ffe 0 2024-09-16T23:49:04.539 INFO:tasks.workunit.client.1.smithi043.stdout:9/699: dwrite d4/d21/f89 [0,4194304] 0 2024-09-16T23:49:04.540 INFO:tasks.workunit.client.0.smithi042.stdout:9/439: creat d4/d12/d16/d81/f89 x:0 0 0 2024-09-16T23:49:04.542 INFO:tasks.workunit.client.0.smithi042.stdout:3/851: rename d2/d1b/d1d/d34/d50/fea to d2/d95/dd8/f127 0 2024-09-16T23:49:04.543 INFO:tasks.workunit.client.0.smithi042.stdout:7/682: creat d3/d98/fee x:0 0 0 2024-09-16T23:49:04.544 INFO:tasks.workunit.client.0.smithi042.stdout:4/802: rmdir de/d42/d5d/d8f/d9f/dc0 39 2024-09-16T23:49:04.546 INFO:tasks.workunit.client.0.smithi042.stdout:1/647: dwrite d0/d9/fd9 [0,4194304] 0 2024-09-16T23:49:04.547 INFO:tasks.workunit.client.0.smithi042.stdout:5/727: mknod d9/db/de/d92/cff 0 2024-09-16T23:49:04.547 INFO:tasks.workunit.client.1.smithi043.stdout:2/670: creat d8/d5e/d83/d3a/d77/fe6 x:0 0 0 2024-09-16T23:49:04.552 INFO:tasks.workunit.client.0.smithi042.stdout:7/683: rmdir d3/d1f/d49/d76/d19/ddc/d6b/d64/d92 39 2024-09-16T23:49:04.553 INFO:tasks.workunit.client.0.smithi042.stdout:9/440: mknod d4/d10/d24/c8a 0 2024-09-16T23:49:04.557 INFO:tasks.workunit.client.0.smithi042.stdout:3/852: write d2/d7/d61/d8e/d9d/dc4/de4/fec [268551,131046] 0 2024-09-16T23:49:04.557 INFO:tasks.workunit.client.1.smithi043.stdout:3/761: mkdir d4/d30/d53/de5 0 2024-09-16T23:49:04.559 INFO:tasks.workunit.client.1.smithi043.stdout:1/891: symlink d4/d9/d1c/d1e/d47/l134 0 2024-09-16T23:49:04.560 INFO:tasks.workunit.client.0.smithi042.stdout:5/728: mkdir d9/db/d1b/d31/d6c/d100 0 2024-09-16T23:49:04.561 INFO:tasks.workunit.client.1.smithi043.stdout:0/803: truncate d0/d2/df/f17 2009774 0 2024-09-16T23:49:04.562 INFO:tasks.workunit.client.0.smithi042.stdout:4/803: rename de/d2e/d37/d3e/f40 to de/d12/f118 0 2024-09-16T23:49:04.562 INFO:tasks.workunit.client.0.smithi042.stdout:7/684: mknod d3/d1f/d49/d76/d19/ddc/d6b/da5/cef 0 2024-09-16T23:49:04.563 INFO:tasks.workunit.client.1.smithi043.stdout:0/804: stat d0/d2/df/d1a/d46/d39/d83 0 2024-09-16T23:49:04.564 INFO:tasks.workunit.client.1.smithi043.stdout:8/724: mknod de/d18/d1f/d2f/d39/d7b/d83/dbb/cd9 0 2024-09-16T23:49:04.564 INFO:tasks.workunit.client.0.smithi042.stdout:6/594: dread d8/d70/d81/dd/f19 [0,4194304] 0 2024-09-16T23:49:04.566 INFO:tasks.workunit.client.0.smithi042.stdout:3/853: symlink d2/d1b/d1d/d34/d62/d11a/l128 0 2024-09-16T23:49:04.566 INFO:tasks.workunit.client.0.smithi042.stdout:2/567: dwrite d0/f1a [0,4194304] 0 2024-09-16T23:49:04.567 INFO:tasks.workunit.client.0.smithi042.stdout:4/804: dread - de/d2e/d37/d4c/fff zero size 2024-09-16T23:49:04.567 INFO:tasks.workunit.client.0.smithi042.stdout:9/441: creat d4/d12/d16/d81/f8b x:0 0 0 2024-09-16T23:49:04.568 INFO:tasks.workunit.client.1.smithi043.stdout:5/884: mkdir d3/d9/d22/d4a/d4c/d7a/d10a/d12b 0 2024-09-16T23:49:04.568 INFO:tasks.workunit.client.0.smithi042.stdout:5/729: unlink d9/d4a/d89/ldd 0 2024-09-16T23:49:04.572 INFO:tasks.workunit.client.0.smithi042.stdout:1/648: mkdir d0/d9/d90/d6b/d8e/de5 0 2024-09-16T23:49:04.572 INFO:tasks.workunit.client.0.smithi042.stdout:5/730: stat d9/db/d11 0 2024-09-16T23:49:04.572 INFO:tasks.workunit.client.0.smithi042.stdout:7/685: rmdir d3/d1f/d49/d76/d19/ddc/d6b 39 2024-09-16T23:49:04.573 INFO:tasks.workunit.client.1.smithi043.stdout:2/671: creat d8/d4b/d4f/d7c/fe7 x:0 0 0 2024-09-16T23:49:04.573 INFO:tasks.workunit.client.1.smithi043.stdout:3/762: symlink d4/de/d1f/d2c/le6 0 2024-09-16T23:49:04.574 INFO:tasks.workunit.client.0.smithi042.stdout:4/805: creat de/d10d/d8a/db2/f119 x:0 0 0 2024-09-16T23:49:04.574 INFO:tasks.workunit.client.1.smithi043.stdout:3/763: write d4/f23 [3076163,14454] 0 2024-09-16T23:49:04.575 INFO:tasks.workunit.client.0.smithi042.stdout:8/489: dwrite da/d19/d1d/d30/f45 [0,4194304] 0 2024-09-16T23:49:04.577 INFO:tasks.workunit.client.1.smithi043.stdout:1/892: creat d4/df/d26/dd5/f135 x:0 0 0 2024-09-16T23:49:04.577 INFO:tasks.workunit.client.0.smithi042.stdout:3/854: read - d2/d7/d61/d8e/d9d/f108 zero size 2024-09-16T23:49:04.577 INFO:tasks.workunit.client.0.smithi042.stdout:9/442: write d4/d12/d2d/d37/d3a/f48 [1266907,106623] 0 2024-09-16T23:49:04.578 INFO:tasks.workunit.client.0.smithi042.stdout:4/806: chown de/d2e/d37/l5e 9968436 1 2024-09-16T23:49:04.579 INFO:tasks.workunit.client.1.smithi043.stdout:1/893: chown d4/d9/d1c/d1e/d42/d5d/d71/d76/l9b 5012060 1 2024-09-16T23:49:04.580 INFO:tasks.workunit.client.1.smithi043.stdout:0/805: unlink d0/cc7 0 2024-09-16T23:49:04.583 INFO:tasks.workunit.client.0.smithi042.stdout:5/731: mkdir d9/db/d1b/d31/d6c/d101 0 2024-09-16T23:49:04.584 INFO:tasks.workunit.client.1.smithi043.stdout:5/885: mkdir d3/db/d15/d4d/d6b/d73/d12c 0 2024-09-16T23:49:04.585 INFO:tasks.workunit.client.0.smithi042.stdout:4/807: truncate de/d1a/da7/fd2 4633245 0 2024-09-16T23:49:04.586 INFO:tasks.workunit.client.1.smithi043.stdout:4/805: creat d8/d15/d20/d2b/d54/db3/dd5/ffb x:0 0 0 2024-09-16T23:49:04.588 INFO:tasks.workunit.client.1.smithi043.stdout:2/672: rmdir d8/d4b/d5f/d63/db6 39 2024-09-16T23:49:04.589 INFO:tasks.workunit.client.0.smithi042.stdout:7/686: dread - d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/fdd zero size 2024-09-16T23:49:04.589 INFO:tasks.workunit.client.0.smithi042.stdout:1/649: dread d0/d3b/d8d/fa3 [0,4194304] 0 2024-09-16T23:49:04.592 INFO:tasks.workunit.client.0.smithi042.stdout:5/732: creat d9/db/d1b/d31/d6c/f102 x:0 0 0 2024-09-16T23:49:04.593 INFO:tasks.workunit.client.1.smithi043.stdout:3/764: creat d4/de/d1f/d2c/db7/dc3/fe7 x:0 0 0 2024-09-16T23:49:04.593 INFO:tasks.workunit.client.0.smithi042.stdout:9/443: dwrite d4/d12/f1e [4194304,4194304] 0 2024-09-16T23:49:04.595 INFO:tasks.workunit.client.0.smithi042.stdout:8/490: dwrite da/d19/d1d/d30/d3f/d6d/f91 [0,4194304] 0 2024-09-16T23:49:04.596 INFO:tasks.workunit.client.1.smithi043.stdout:8/725: rename de/d18/d1f/l63 to de/d18/d27/d41/d5a/da9/lda 0 2024-09-16T23:49:04.598 INFO:tasks.workunit.client.0.smithi042.stdout:2/568: dread d0/da/d17/d5c/fa8 [0,4194304] 0 2024-09-16T23:49:04.602 INFO:tasks.workunit.client.0.smithi042.stdout:4/808: rename de/d10d/c67 to de/d12/db0/c11a 0 2024-09-16T23:49:04.602 INFO:tasks.workunit.client.1.smithi043.stdout:0/806: fdatasync d0/d2/df/d1a/d46/d39/f6a 0 2024-09-16T23:49:04.603 INFO:tasks.workunit.client.0.smithi042.stdout:5/733: symlink d9/db/d1b/l103 0 2024-09-16T23:49:04.605 INFO:tasks.workunit.client.1.smithi043.stdout:0/807: stat d0/d2/df/d1a/d49/db2/df4/d9c 0 2024-09-16T23:49:04.605 INFO:tasks.workunit.client.0.smithi042.stdout:5/734: chown d9/db/fb6 1 1 2024-09-16T23:49:04.606 INFO:tasks.workunit.client.0.smithi042.stdout:5/735: stat d9/db/de/d10/d2f/cbc 0 2024-09-16T23:49:04.607 INFO:tasks.workunit.client.0.smithi042.stdout:2/569: rmdir d0/da/d1b/d3b/d44 39 2024-09-16T23:49:04.607 INFO:tasks.workunit.client.0.smithi042.stdout:4/809: read de/d12/f97 [390396,29614] 0 2024-09-16T23:49:04.609 INFO:tasks.workunit.client.0.smithi042.stdout:5/736: write d9/db/d1b/da4/fe9 [329143,39373] 0 2024-09-16T23:49:04.610 INFO:tasks.workunit.client.0.smithi042.stdout:4/810: chown de/d42/d99/d9e/ce1 22337657 1 2024-09-16T23:49:04.617 INFO:tasks.workunit.client.0.smithi042.stdout:1/650: getdents d0/d17/d5e 0 2024-09-16T23:49:04.618 INFO:tasks.workunit.client.0.smithi042.stdout:2/570: mknod d0/da/d17/d3e/d86/d96/cbf 0 2024-09-16T23:49:04.619 INFO:tasks.workunit.client.1.smithi043.stdout:1/894: dwrite d4/d9/d1c/d1e/d42/d5d/d71/fb1 [0,4194304] 0 2024-09-16T23:49:04.619 INFO:tasks.workunit.client.1.smithi043.stdout:0/808: symlink d0/d2/df/d1a/d3b/dee/df0/l10e 0 2024-09-16T23:49:04.619 INFO:tasks.workunit.client.0.smithi042.stdout:1/651: chown d0/da6 486 1 2024-09-16T23:49:04.620 INFO:tasks.workunit.client.1.smithi043.stdout:4/806: dwrite d8/d15/d9b/d8b/f8f [0,4194304] 0 2024-09-16T23:49:04.621 INFO:tasks.workunit.client.0.smithi042.stdout:4/811: dwrite de/d1a/f39 [0,4194304] 0 2024-09-16T23:49:04.623 INFO:tasks.workunit.client.0.smithi042.stdout:1/652: stat d0/d17/d79/fa2 0 2024-09-16T23:49:04.623 INFO:tasks.workunit.client.0.smithi042.stdout:9/444: dwrite d4/d12/d2d/d37/d28/f4e [0,4194304] 0 2024-09-16T23:49:04.625 INFO:tasks.workunit.client.1.smithi043.stdout:2/673: rename d8/d1d/d20/d23/d38/db2 to d8/d4b/d5f/d8e/de8 0 2024-09-16T23:49:04.628 INFO:tasks.workunit.client.0.smithi042.stdout:9/445: chown d4/d12/d2d/l4f 119 1 2024-09-16T23:49:04.628 INFO:tasks.workunit.client.0.smithi042.stdout:7/687: truncate d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/fb4 199292 0 2024-09-16T23:49:04.629 INFO:tasks.workunit.client.1.smithi043.stdout:3/765: dwrite d4/de/d1f/d34/fad [0,4194304] 0 2024-09-16T23:49:04.630 INFO:tasks.workunit.client.1.smithi043.stdout:2/674: readlink d8/d1d/d20/d23/l71 0 2024-09-16T23:49:04.633 INFO:tasks.workunit.client.0.smithi042.stdout:1/653: rename d0/d3b/f44 to d0/d9/d4e/fe6 0 2024-09-16T23:49:04.635 INFO:tasks.workunit.client.0.smithi042.stdout:7/688: symlink d3/d1f/d49/d76/d19/ddc/dcd/db5/lf0 0 2024-09-16T23:49:04.636 INFO:tasks.workunit.client.0.smithi042.stdout:5/737: dwrite d9/db/f3e [0,4194304] 0 2024-09-16T23:49:04.637 INFO:tasks.workunit.client.1.smithi043.stdout:8/726: symlink de/d18/d27/d41/d5a/dc8/ldb 0 2024-09-16T23:49:04.639 INFO:tasks.workunit.client.1.smithi043.stdout:1/895: dwrite d4/d9/d12/d2b/f52 [0,4194304] 0 2024-09-16T23:49:04.643 INFO:tasks.workunit.client.1.smithi043.stdout:5/886: creat d3/d9/d6f/f12d x:0 0 0 2024-09-16T23:49:04.644 INFO:tasks.workunit.client.0.smithi042.stdout:4/812: write de/d2e/d37/d4c/f9b [1093610,33426] 0 2024-09-16T23:49:04.644 INFO:tasks.workunit.client.0.smithi042.stdout:5/738: stat d9/db/d19/d21/d85/da5/f94 0 2024-09-16T23:49:04.645 INFO:tasks.workunit.client.0.smithi042.stdout:9/446: link d4/d12/d2d/d37/d28/l42 d4/d12/d77/l8c 0 2024-09-16T23:49:04.647 INFO:tasks.workunit.client.1.smithi043.stdout:4/807: write d8/d15/d20/d2b/d23/d27/faf [537932,117186] 0 2024-09-16T23:49:04.647 INFO:tasks.workunit.client.0.smithi042.stdout:1/654: dwrite d0/d9/fe [0,4194304] 0 2024-09-16T23:49:04.649 INFO:tasks.workunit.client.0.smithi042.stdout:5/739: chown d9/d4a/cae 6528 1 2024-09-16T23:49:04.652 INFO:tasks.workunit.client.1.smithi043.stdout:2/675: rename d8/d1d/d20/d23/d38/fd1 to d8/d1d/d20/d23/d38/da2/fe9 0 2024-09-16T23:49:04.654 INFO:tasks.workunit.client.0.smithi042.stdout:4/813: truncate de/d42/d82/d94/dbd/fd3 224500 0 2024-09-16T23:49:04.655 INFO:tasks.workunit.client.1.smithi043.stdout:2/676: chown d8/d1d/d20/d26/f7d 945 1 2024-09-16T23:49:04.655 INFO:tasks.workunit.client.1.smithi043.stdout:2/677: fsync d8/d5e/fe0 0 2024-09-16T23:49:04.656 INFO:tasks.workunit.client.0.smithi042.stdout:5/740: stat d9/db/d1b/d31/d6c/f102 0 2024-09-16T23:49:04.656 INFO:tasks.workunit.client.0.smithi042.stdout:7/689: creat d3/d1f/d49/d76/d19/ff1 x:0 0 0 2024-09-16T23:49:04.657 INFO:tasks.workunit.client.0.smithi042.stdout:2/571: dwrite d0/f87 [0,4194304] 0 2024-09-16T23:49:04.657 INFO:tasks.workunit.client.0.smithi042.stdout:5/741: fsync d9/db/d19/d21/d85/ff1 0 2024-09-16T23:49:04.667 INFO:tasks.workunit.client.0.smithi042.stdout:2/572: dwrite d0/f87 [0,4194304] 0 2024-09-16T23:49:04.667 INFO:tasks.workunit.client.0.smithi042.stdout:5/742: read d9/db/d19/f52 [3705081,59101] 0 2024-09-16T23:49:04.671 INFO:tasks.workunit.client.0.smithi042.stdout:5/743: write d9/db/d19/f52 [3646235,75038] 0 2024-09-16T23:49:04.672 INFO:tasks.workunit.client.0.smithi042.stdout:7/690: write d3/d98/f71 [752604,12005] 0 2024-09-16T23:49:04.674 INFO:tasks.workunit.client.0.smithi042.stdout:5/744: stat d9/db/d19/d21/d41/d7d 0 2024-09-16T23:49:04.674 INFO:tasks.workunit.client.1.smithi043.stdout:5/887: write d3/d79/d91/df4/f102 [2505,105869] 0 2024-09-16T23:49:04.675 INFO:tasks.workunit.client.1.smithi043.stdout:5/888: chown d3/db/d15/d43 8977390 1 2024-09-16T23:49:04.678 INFO:tasks.workunit.client.0.smithi042.stdout:4/814: dwrite de/d10d/f36 [0,4194304] 0 2024-09-16T23:49:04.680 INFO:tasks.workunit.client.1.smithi043.stdout:7/725: sync 2024-09-16T23:49:04.696 INFO:tasks.workunit.client.0.smithi042.stdout:7/691: dread d3/d98/da0/d41/f8b [0,4194304] 0 2024-09-16T23:49:04.708 INFO:tasks.workunit.client.0.smithi042.stdout:5/745: symlink d9/db/de/d10/l104 0 2024-09-16T23:49:04.709 INFO:tasks.workunit.client.0.smithi042.stdout:7/692: mknod d3/cf2 0 2024-09-16T23:49:04.711 INFO:tasks.workunit.client.0.smithi042.stdout:0/532: sync 2024-09-16T23:49:04.711 INFO:tasks.workunit.client.0.smithi042.stdout:7/693: write d3/d98/da0/d24/f6a [1346211,15260] 0 2024-09-16T23:49:04.712 INFO:tasks.workunit.client.0.smithi042.stdout:7/694: stat d3/d3f 0 2024-09-16T23:49:04.713 INFO:tasks.workunit.client.0.smithi042.stdout:0/533: stat d0/c3f 0 2024-09-16T23:49:04.715 INFO:tasks.workunit.client.0.smithi042.stdout:4/815: dwrite de/d42/d68/f72 [0,4194304] 0 2024-09-16T23:49:04.716 INFO:tasks.workunit.client.0.smithi042.stdout:0/534: fsync d0/df/d23/d33/d56/f61 0 2024-09-16T23:49:04.723 INFO:tasks.workunit.client.0.smithi042.stdout:5/746: dwrite d9/db/d1b/d31/d46/f49 [0,4194304] 0 2024-09-16T23:49:04.723 INFO:tasks.workunit.client.0.smithi042.stdout:0/535: creat d0/df/d13/d49/d70/da1/fa7 x:0 0 0 2024-09-16T23:49:04.725 INFO:tasks.workunit.client.0.smithi042.stdout:7/695: dwrite d3/d1f/d49/d76/d19/ddc/dcd/dcf/fb1 [0,4194304] 0 2024-09-16T23:49:04.725 INFO:tasks.workunit.client.0.smithi042.stdout:5/747: chown d9/db/de/d10/l104 0 1 2024-09-16T23:49:04.734 INFO:tasks.workunit.client.0.smithi042.stdout:4/816: dwrite de/f41 [0,4194304] 0 2024-09-16T23:49:04.741 INFO:tasks.workunit.client.0.smithi042.stdout:4/817: write de/d1a/d70/fbe [4990074,4520] 0 2024-09-16T23:49:04.742 INFO:tasks.workunit.client.0.smithi042.stdout:4/818: readlink de/d42/d5d/d8f/dd6/lec 0 2024-09-16T23:49:04.745 INFO:tasks.workunit.client.0.smithi042.stdout:1/655: rename d0/d9/f94 to d0/d17/dc7/fe7 0 2024-09-16T23:49:04.746 INFO:tasks.workunit.client.0.smithi042.stdout:1/656: chown d0/d17/d79 12876 1 2024-09-16T23:49:04.747 INFO:tasks.workunit.client.0.smithi042.stdout:4/819: creat de/d12/d7d/f11b x:0 0 0 2024-09-16T23:49:04.748 INFO:tasks.workunit.client.0.smithi042.stdout:4/820: fsync de/d42/d99/fc7 0 2024-09-16T23:49:04.749 INFO:tasks.workunit.client.0.smithi042.stdout:5/748: rename d9/c14 to d9/db/d1b/d31/d6c/c105 0 2024-09-16T23:49:04.751 INFO:tasks.workunit.client.0.smithi042.stdout:4/821: rmdir de/d1a/d70/dac 39 2024-09-16T23:49:04.752 INFO:tasks.workunit.client.0.smithi042.stdout:4/822: readlink de/d42/d5d/l80 0 2024-09-16T23:49:04.752 INFO:tasks.workunit.client.0.smithi042.stdout:2/573: creat d0/da/d1b/d48/d81/fc0 x:0 0 0 2024-09-16T23:49:04.753 INFO:tasks.workunit.client.0.smithi042.stdout:7/696: rename d3/d98/da0/d41/db0/lb6 to d3/d1f/d49/d76/d19/ddc/dcd/db5/dc4/lf3 0 2024-09-16T23:49:04.754 INFO:tasks.workunit.client.0.smithi042.stdout:5/749: creat d9/d2c/f106 x:0 0 0 2024-09-16T23:49:04.757 INFO:tasks.workunit.client.0.smithi042.stdout:5/750: fdatasync d9/db/de/f6f 0 2024-09-16T23:49:04.757 INFO:tasks.workunit.client.0.smithi042.stdout:2/574: dread d0/da/d1b/d48/d81/f80 [0,4194304] 0 2024-09-16T23:49:04.759 INFO:tasks.workunit.client.1.smithi043.stdout:5/889: symlink d3/da4/l12e 0 2024-09-16T23:49:04.759 INFO:tasks.workunit.client.0.smithi042.stdout:4/823: creat de/d2e/d37/f11c x:0 0 0 2024-09-16T23:49:04.760 INFO:tasks.workunit.client.0.smithi042.stdout:4/824: chown de/d2e/d37/f7c 33 1 2024-09-16T23:49:04.760 INFO:tasks.workunit.client.0.smithi042.stdout:5/751: mkdir d9/db/d1b/d31/d46/d107 0 2024-09-16T23:49:04.761 INFO:tasks.workunit.client.1.smithi043.stdout:3/766: write d4/de/d1f/f29 [2566424,125446] 0 2024-09-16T23:49:04.762 INFO:tasks.workunit.client.1.smithi043.stdout:0/809: link d0/la2 d0/d2/df/d1a/d3b/dee/l10f 0 2024-09-16T23:49:04.762 INFO:tasks.workunit.client.1.smithi043.stdout:1/896: dread d4/d9/d1c/d1e/f2a [0,4194304] 0 2024-09-16T23:49:04.764 INFO:tasks.workunit.client.0.smithi042.stdout:2/575: unlink d0/da/d1b/l71 0 2024-09-16T23:49:04.765 INFO:tasks.workunit.client.1.smithi043.stdout:0/810: write d0/d2/df/d1a/d3b/f103 [599351,121868] 0 2024-09-16T23:49:04.765 INFO:tasks.workunit.client.0.smithi042.stdout:4/825: symlink de/d1a/d70/l11d 0 2024-09-16T23:49:04.766 INFO:tasks.workunit.client.1.smithi043.stdout:5/890: creat d3/d9/d6f/d7c/f12f x:0 0 0 2024-09-16T23:49:04.767 INFO:tasks.workunit.client.1.smithi043.stdout:7/726: symlink d4/db/d11/d21/lee 0 2024-09-16T23:49:04.768 INFO:tasks.workunit.client.0.smithi042.stdout:1/657: dwrite d0/d9/f16 [0,4194304] 0 2024-09-16T23:49:04.769 INFO:tasks.workunit.client.0.smithi042.stdout:2/576: rmdir d0/da/d1b/d3b 39 2024-09-16T23:49:04.769 INFO:tasks.workunit.client.1.smithi043.stdout:3/767: mknod d4/de/d40/ce8 0 2024-09-16T23:49:04.771 INFO:tasks.workunit.client.1.smithi043.stdout:1/897: mkdir d4/df/df4/d83/d136 0 2024-09-16T23:49:04.772 INFO:tasks.workunit.client.1.smithi043.stdout:1/898: stat d4/d34/f41 0 2024-09-16T23:49:04.773 INFO:tasks.workunit.client.1.smithi043.stdout:4/808: creat d8/d15/ffc x:0 0 0 2024-09-16T23:49:04.775 INFO:tasks.workunit.client.1.smithi043.stdout:8/727: getdents de/d18 0 2024-09-16T23:49:04.776 INFO:tasks.workunit.client.0.smithi042.stdout:7/697: dwrite d3/d1f/d49/d76/d19/d63/f69 [0,4194304] 0 2024-09-16T23:49:04.778 INFO:tasks.workunit.client.1.smithi043.stdout:1/899: mkdir d4/df/df4/d83/d98/d137 0 2024-09-16T23:49:04.780 INFO:tasks.workunit.client.0.smithi042.stdout:1/658: mknod d0/d9/d90/d6b/d88/d32/ce8 0 2024-09-16T23:49:04.781 INFO:tasks.workunit.client.0.smithi042.stdout:4/826: dwrite de/d42/d82/fa5 [0,4194304] 0 2024-09-16T23:49:04.783 INFO:tasks.workunit.client.0.smithi042.stdout:7/698: write d3/d1f/d49/d78/dbe/fe7 [735235,99710] 0 2024-09-16T23:49:04.787 INFO:tasks.workunit.client.0.smithi042.stdout:1/659: dread - d0/d9/d90/fdc zero size 2024-09-16T23:49:04.788 INFO:tasks.workunit.client.0.smithi042.stdout:4/827: chown de/d42/d82/d94/f108 93 1 2024-09-16T23:49:04.790 INFO:tasks.workunit.client.0.smithi042.stdout:2/577: dwrite d0/d30/fbb [0,4194304] 0 2024-09-16T23:49:04.790 INFO:tasks.workunit.client.0.smithi042.stdout:4/828: write de/f31 [465393,97334] 0 2024-09-16T23:49:04.797 INFO:tasks.workunit.client.0.smithi042.stdout:1/660: rename d0/d3b/c81 to d0/d9/ce9 0 2024-09-16T23:49:04.800 INFO:tasks.workunit.client.0.smithi042.stdout:2/578: creat d0/da/d17/fc1 x:0 0 0 2024-09-16T23:49:04.800 INFO:tasks.workunit.client.0.smithi042.stdout:4/829: truncate de/f10 7259722 0 2024-09-16T23:49:04.802 INFO:tasks.workunit.client.0.smithi042.stdout:4/830: mknod de/d42/d5d/c11e 0 2024-09-16T23:49:04.802 INFO:tasks.workunit.client.0.smithi042.stdout:1/661: rename d0/d9/d90/d6b/d88/d32/d2f/d98 to d0/d9/d26/daa/dea 0 2024-09-16T23:49:04.803 INFO:tasks.workunit.client.0.smithi042.stdout:2/579: symlink d0/da/d17/d5c/lc2 0 2024-09-16T23:49:04.807 INFO:tasks.workunit.client.0.smithi042.stdout:2/580: symlink d0/lc3 0 2024-09-16T23:49:04.809 INFO:tasks.workunit.client.0.smithi042.stdout:4/831: dwrite de/d42/d99/fc7 [0,4194304] 0 2024-09-16T23:49:04.811 INFO:tasks.workunit.client.0.smithi042.stdout:1/662: write d0/d9/d90/d6b/d88/d32/d2f/f50 [131797,101694] 0 2024-09-16T23:49:04.813 INFO:tasks.workunit.client.0.smithi042.stdout:4/832: creat de/d10d/db8/f11f x:0 0 0 2024-09-16T23:49:04.819 INFO:tasks.workunit.client.0.smithi042.stdout:1/663: link d0/d9/d90/d6b/cd5 d0/ddb/ddd/ceb 0 2024-09-16T23:49:04.819 INFO:tasks.workunit.client.1.smithi043.stdout:3/768: fsync d4/de/d1f/d2c/f4a 0 2024-09-16T23:49:04.820 INFO:tasks.workunit.client.0.smithi042.stdout:4/833: write de/d1a/da7/f47 [1266737,67830] 0 2024-09-16T23:49:04.820 INFO:tasks.workunit.client.0.smithi042.stdout:1/664: write d0/d9/d90/fdc [40422,119571] 0 2024-09-16T23:49:04.823 INFO:tasks.workunit.client.0.smithi042.stdout:4/834: creat de/d42/d82/d94/f120 x:0 0 0 2024-09-16T23:49:04.824 INFO:tasks.workunit.client.0.smithi042.stdout:1/665: truncate d0/d17/fa7 348183 0 2024-09-16T23:49:04.844 INFO:tasks.workunit.client.0.smithi042.stdout:1/666: truncate d0/d9/d90/fad 4025744 0 2024-09-16T23:49:04.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/667: mkdir d0/d9/d26/d8b/dec 0 2024-09-16T23:49:04.847 INFO:tasks.workunit.client.0.smithi042.stdout:1/668: chown d0/d17/l6a 5570742 1 2024-09-16T23:49:04.847 INFO:tasks.workunit.client.0.smithi042.stdout:1/669: chown d0/d9/d26/daa/dea 4766667 1 2024-09-16T23:49:04.849 INFO:tasks.workunit.client.0.smithi042.stdout:1/670: rmdir d0/da6 39 2024-09-16T23:49:04.851 INFO:tasks.workunit.client.0.smithi042.stdout:1/671: mknod d0/d9/d26/d8b/dec/ced 0 2024-09-16T23:49:04.854 INFO:tasks.workunit.client.0.smithi042.stdout:1/672: link d0/d9/d26/fd2 d0/d9/d26/daa/dea/fee 0 2024-09-16T23:49:04.857 INFO:tasks.workunit.client.0.smithi042.stdout:1/673: dread d0/d9/d90/d6b/d88/f75 [0,4194304] 0 2024-09-16T23:49:04.866 INFO:tasks.workunit.client.0.smithi042.stdout:1/674: dread d0/f1d [0,4194304] 0 2024-09-16T23:49:04.870 INFO:tasks.workunit.client.0.smithi042.stdout:1/675: getdents d0/d9a 0 2024-09-16T23:49:04.872 INFO:tasks.workunit.client.0.smithi042.stdout:1/676: read d0/d9a/fb5 [3615259,126013] 0 2024-09-16T23:49:04.875 INFO:tasks.workunit.client.1.smithi043.stdout:9/700: sync 2024-09-16T23:49:04.875 INFO:tasks.workunit.client.1.smithi043.stdout:6/728: sync 2024-09-16T23:49:04.877 INFO:tasks.workunit.client.1.smithi043.stdout:7/727: creat d4/db/d11/d21/fef x:0 0 0 2024-09-16T23:49:04.877 INFO:tasks.workunit.client.1.smithi043.stdout:0/811: write d0/f3c [1739478,27898] 0 2024-09-16T23:49:04.885 INFO:tasks.workunit.client.1.smithi043.stdout:9/701: rmdir d4/d21 39 2024-09-16T23:49:04.886 INFO:tasks.workunit.client.1.smithi043.stdout:8/728: dwrite de/d70/f76 [0,4194304] 0 2024-09-16T23:49:04.886 INFO:tasks.workunit.client.1.smithi043.stdout:3/769: rename d4/de/d1f/d2c/db7/dc3/fd0 to d4/d30/fe9 0 2024-09-16T23:49:04.889 INFO:tasks.workunit.client.1.smithi043.stdout:0/812: rmdir d0/d2/df/d1a/d46/d39/d83/dfb 39 2024-09-16T23:49:04.890 INFO:tasks.workunit.client.1.smithi043.stdout:4/809: dwrite d8/d25/d39/d58/fa5 [0,4194304] 0 2024-09-16T23:49:04.892 INFO:tasks.workunit.client.1.smithi043.stdout:6/729: creat de/d2a/ff6 x:0 0 0 2024-09-16T23:49:04.894 INFO:tasks.workunit.client.1.smithi043.stdout:9/702: stat d4/d21/f89 0 2024-09-16T23:49:04.894 INFO:tasks.workunit.client.1.smithi043.stdout:7/728: creat d4/db/d11/d21/d31/dcb/ff0 x:0 0 0 2024-09-16T23:49:04.895 INFO:tasks.workunit.client.1.smithi043.stdout:0/813: truncate d0/d2/df/d1a/d49/db2/ff3 292017 0 2024-09-16T23:49:04.897 INFO:tasks.workunit.client.1.smithi043.stdout:9/703: write d4/d21/f89 [3799345,82384] 0 2024-09-16T23:49:04.898 INFO:tasks.workunit.client.1.smithi043.stdout:4/810: mknod d8/d15/d20/d2b/d54/db3/cfd 0 2024-09-16T23:49:04.898 INFO:tasks.workunit.client.1.smithi043.stdout:6/730: mknod de/d23/d63/cf7 0 2024-09-16T23:49:04.899 INFO:tasks.workunit.client.1.smithi043.stdout:3/770: mkdir d4/de/dea 0 2024-09-16T23:49:04.901 INFO:tasks.workunit.client.1.smithi043.stdout:4/811: chown d8/d25/d39/dd3/fa9 57 1 2024-09-16T23:49:04.902 INFO:tasks.workunit.client.0.smithi042.stdout:1/677: dwrite d0/d9/d90/d6b/d88/d32/d2f/fb3 [0,4194304] 0 2024-09-16T23:49:04.902 INFO:tasks.workunit.client.1.smithi043.stdout:4/812: rename d8/d15 to d8/d15/d20/d2b/d54/db3/dd5/dfe 22 2024-09-16T23:49:04.905 INFO:tasks.workunit.client.1.smithi043.stdout:9/704: read d4/d1c/d25/d2b/fa5 [309916,110079] 0 2024-09-16T23:49:04.906 INFO:tasks.workunit.client.1.smithi043.stdout:0/814: fsync d0/d2/df/dbb/f9b 0 2024-09-16T23:49:04.907 INFO:tasks.workunit.client.1.smithi043.stdout:7/729: creat d4/db/d11/d15/d29/d7a/dcd/ff1 x:0 0 0 2024-09-16T23:49:04.908 INFO:tasks.workunit.client.0.smithi042.stdout:1/678: mkdir d0/d9/d90/d6b/d88/def 0 2024-09-16T23:49:04.911 INFO:tasks.workunit.client.1.smithi043.stdout:4/813: chown d8/ld7 6 1 2024-09-16T23:49:04.912 INFO:tasks.workunit.client.0.smithi042.stdout:1/679: symlink d0/d9/d26/d8b/lf0 0 2024-09-16T23:49:04.915 INFO:tasks.workunit.client.0.smithi042.stdout:1/680: mkdir d0/d17/df1 0 2024-09-16T23:49:04.916 INFO:tasks.workunit.client.1.smithi043.stdout:9/705: dread d4/d5/d64/f65 [4194304,4194304] 0 2024-09-16T23:49:04.917 INFO:tasks.workunit.client.1.smithi043.stdout:4/814: truncate d8/d15/d9b/d8b/dde/ff2 810539 0 2024-09-16T23:49:04.918 INFO:tasks.workunit.client.1.smithi043.stdout:3/771: mkdir d4/de/d1f/d2c/db7/dc6/deb 0 2024-09-16T23:49:04.918 INFO:tasks.workunit.client.1.smithi043.stdout:0/815: dwrite d0/d2/df/d1a/d46/d39/d83/dcb/fca [0,4194304] 0 2024-09-16T23:49:04.920 INFO:tasks.workunit.client.0.smithi042.stdout:1/681: symlink d0/d9/d26/d8b/lf2 0 2024-09-16T23:49:04.920 INFO:tasks.workunit.client.1.smithi043.stdout:3/772: fdatasync d4/de/d40/d4d/f78 0 2024-09-16T23:49:04.922 INFO:tasks.workunit.client.1.smithi043.stdout:6/731: creat de/ff8 x:0 0 0 2024-09-16T23:49:04.924 INFO:tasks.workunit.client.1.smithi043.stdout:2/678: sync 2024-09-16T23:49:04.929 INFO:tasks.workunit.client.1.smithi043.stdout:7/730: mknod d4/db/d43/d83/cf2 0 2024-09-16T23:49:04.930 INFO:tasks.workunit.client.1.smithi043.stdout:9/706: dread d4/d21/d2a/f99 [0,4194304] 0 2024-09-16T23:49:04.931 INFO:tasks.workunit.client.1.smithi043.stdout:9/707: stat d4/d1c/d4f/d8c/c15 0 2024-09-16T23:49:04.933 INFO:tasks.workunit.client.1.smithi043.stdout:0/816: mkdir d0/d2/df/d1a/de8/d110 0 2024-09-16T23:49:04.934 INFO:tasks.workunit.client.1.smithi043.stdout:8/729: link de/d18/d1f/c2c de/d18/d27/d41/cdc 0 2024-09-16T23:49:04.935 INFO:tasks.workunit.client.1.smithi043.stdout:6/732: mknod de/d23/d34/d9b/d5a/da3/cf9 0 2024-09-16T23:49:04.938 INFO:tasks.workunit.client.0.smithi042.stdout:1/682: dread d0/d17/f58 [0,4194304] 0 2024-09-16T23:49:04.940 INFO:tasks.workunit.client.1.smithi043.stdout:7/731: rename d4/db/c10 to d4/db/d50/cf3 0 2024-09-16T23:49:04.940 INFO:tasks.workunit.client.1.smithi043.stdout:4/815: dread d8/d15/d9b/d8b/fa8 [0,4194304] 0 2024-09-16T23:49:04.942 INFO:tasks.workunit.client.1.smithi043.stdout:8/730: dwrite de/d18/d1f/d2f/d39/d7b/d83/f84 [0,4194304] 0 2024-09-16T23:49:04.943 INFO:tasks.workunit.client.1.smithi043.stdout:4/816: chown d8/d15/d20/dfa/f94 740688 1 2024-09-16T23:49:04.944 INFO:tasks.workunit.client.0.smithi042.stdout:1/683: symlink d0/d9/d26/dc6/lf3 0 2024-09-16T23:49:04.948 INFO:tasks.workunit.client.0.smithi042.stdout:1/684: creat d0/d9/d90/d43/ff4 x:0 0 0 2024-09-16T23:49:04.951 INFO:tasks.workunit.client.1.smithi043.stdout:9/708: rename d4/d21/d33/l37 to d4/d1c/d25/d2b/d4c/da3/dca/lde 0 2024-09-16T23:49:04.951 INFO:tasks.workunit.client.1.smithi043.stdout:3/773: link d4/de/d1f/d34/ce1 d4/de/d40/d4d/cec 0 2024-09-16T23:49:04.953 INFO:tasks.workunit.client.1.smithi043.stdout:2/679: dwrite d8/d11/d16/f49 [4194304,4194304] 0 2024-09-16T23:49:04.954 INFO:tasks.workunit.client.1.smithi043.stdout:6/733: creat de/dbe/dca/ffa x:0 0 0 2024-09-16T23:49:04.956 INFO:tasks.workunit.client.0.smithi042.stdout:1/685: dwrite d0/d17/f9e [0,4194304] 0 2024-09-16T23:49:04.957 INFO:tasks.workunit.client.0.smithi042.stdout:1/686: stat d0/d17/d5e/c39 0 2024-09-16T23:49:04.958 INFO:tasks.workunit.client.1.smithi043.stdout:4/817: creat d8/d15/d20/dfa/d5e/fff x:0 0 0 2024-09-16T23:49:04.959 INFO:tasks.workunit.client.1.smithi043.stdout:0/817: creat d0/f111 x:0 0 0 2024-09-16T23:49:04.961 INFO:tasks.workunit.client.1.smithi043.stdout:4/818: read - d8/d15/d20/d2b/d54/db3/dd5/ffb zero size 2024-09-16T23:49:04.962 INFO:tasks.workunit.client.1.smithi043.stdout:0/818: truncate d0/d2/df/d1a/d46/fe6 407235 0 2024-09-16T23:49:04.962 INFO:tasks.workunit.client.0.smithi042.stdout:1/687: creat d0/d9/d90/d6b/d88/d32/d2f/ff5 x:0 0 0 2024-09-16T23:49:04.965 INFO:tasks.workunit.client.1.smithi043.stdout:3/774: symlink d4/d30/d53/led 0 2024-09-16T23:49:04.966 INFO:tasks.workunit.client.1.smithi043.stdout:6/734: creat de/dbe/dca/ffb x:0 0 0 2024-09-16T23:49:04.969 INFO:tasks.workunit.client.1.smithi043.stdout:2/680: mkdir d8/d5e/d83/d3a/d77/dea 0 2024-09-16T23:49:04.973 INFO:tasks.workunit.client.1.smithi043.stdout:4/819: mkdir d8/d25/d39/dd3/d100 0 2024-09-16T23:49:04.974 INFO:tasks.workunit.client.1.smithi043.stdout:5/891: sync 2024-09-16T23:49:04.974 INFO:tasks.workunit.client.1.smithi043.stdout:1/900: sync 2024-09-16T23:49:04.974 INFO:tasks.workunit.client.1.smithi043.stdout:4/820: chown d8/d25/d39/d58 7 1 2024-09-16T23:49:04.974 INFO:tasks.workunit.client.0.smithi042.stdout:1/688: rename d0/d3b/d8d/ld0 to d0/d17/d18/d60/lf6 0 2024-09-16T23:49:04.976 INFO:tasks.workunit.client.1.smithi043.stdout:0/819: dread - d0/d2/df/dbb/ff1 zero size 2024-09-16T23:49:04.977 INFO:tasks.workunit.client.1.smithi043.stdout:4/821: write d8/d15/f98 [1776713,8063] 0 2024-09-16T23:49:04.977 INFO:tasks.workunit.client.1.smithi043.stdout:1/901: chown d4/d9/d1c/d1e/d47/l134 169002 1 2024-09-16T23:49:04.977 INFO:tasks.workunit.client.1.smithi043.stdout:3/775: unlink d4/de/d5d/f6c 0 2024-09-16T23:49:04.977 INFO:tasks.workunit.client.1.smithi043.stdout:6/735: mkdir de/d23/d34/d9b/d5a/da3/dfc 0 2024-09-16T23:49:04.978 INFO:tasks.workunit.client.1.smithi043.stdout:8/731: dwrite de/d18/d27/d41/d5a/f4a [0,4194304] 0 2024-09-16T23:49:04.978 INFO:tasks.workunit.client.0.smithi042.stdout:1/689: creat d0/d17/ff7 x:0 0 0 2024-09-16T23:49:04.980 INFO:tasks.workunit.client.1.smithi043.stdout:4/822: chown d8/d15/d20/d2b/f26 166562 1 2024-09-16T23:49:04.982 INFO:tasks.workunit.client.1.smithi043.stdout:7/732: link d4/db/d43/d83/d85/d8b/faf d4/db/d11/d21/d94/ff4 0 2024-09-16T23:49:04.984 INFO:tasks.workunit.client.1.smithi043.stdout:1/902: chown d4/d9/d1c/d1e/d47/f53 1581667663 1 2024-09-16T23:49:04.984 INFO:tasks.workunit.client.1.smithi043.stdout:4/823: write d8/d15/d9b/fd2 [381861,28689] 0 2024-09-16T23:49:04.985 INFO:tasks.workunit.client.1.smithi043.stdout:2/681: link d8/d1d/d20/d23/d38/f69 d8/d4b/d5f/d8e/de8/feb 0 2024-09-16T23:49:04.987 INFO:tasks.workunit.client.1.smithi043.stdout:9/709: link d4/d28/d5f/d86/c88 d4/d28/d49/cdf 0 2024-09-16T23:49:04.987 INFO:tasks.workunit.client.1.smithi043.stdout:3/776: creat d4/de/d1f/d2c/db7/dc6/fee x:0 0 0 2024-09-16T23:49:04.990 INFO:tasks.workunit.client.0.smithi042.stdout:1/690: dread d0/d9/fb9 [0,4194304] 0 2024-09-16T23:49:04.990 INFO:tasks.workunit.client.1.smithi043.stdout:3/777: chown d4/d30/fa6 553982 1 2024-09-16T23:49:04.991 INFO:tasks.workunit.client.1.smithi043.stdout:0/820: symlink d0/d2/df/d1a/d46/d39/d83/dfb/l112 0 2024-09-16T23:49:04.993 INFO:tasks.workunit.client.1.smithi043.stdout:1/903: truncate d4/d9/d8f/db6/dc6/f114 481458 0 2024-09-16T23:49:04.995 INFO:tasks.workunit.client.1.smithi043.stdout:2/682: mkdir d8/dec 0 2024-09-16T23:49:04.998 INFO:tasks.workunit.client.1.smithi043.stdout:7/733: dwrite d4/d27/d41/f8d [4194304,4194304] 0 2024-09-16T23:49:04.998 INFO:tasks.workunit.client.1.smithi043.stdout:4/824: mknod d8/d15/d9b/d8b/c101 0 2024-09-16T23:49:04.999 INFO:tasks.workunit.client.1.smithi043.stdout:5/892: link d3/db/d15/d43/c67 d3/d79/dcc/c130 0 2024-09-16T23:49:05.000 INFO:tasks.workunit.client.1.smithi043.stdout:4/825: readlink d8/d15/l1c 0 2024-09-16T23:49:05.000 INFO:tasks.workunit.client.1.smithi043.stdout:7/734: stat d4/db/d11/d21/d31/d33/cbb 0 2024-09-16T23:49:05.001 INFO:tasks.workunit.client.1.smithi043.stdout:6/736: dread de/d23/d34/f5d [0,4194304] 0 2024-09-16T23:49:05.002 INFO:tasks.workunit.client.1.smithi043.stdout:7/735: write d4/db/fb2 [433285,8389] 0 2024-09-16T23:49:05.004 INFO:tasks.workunit.client.1.smithi043.stdout:5/893: chown d3/db/d15/d4d/c8a 481 1 2024-09-16T23:49:05.005 INFO:tasks.workunit.client.1.smithi043.stdout:0/821: rmdir d0/d2/df/d1a/d3b/dee/df0 39 2024-09-16T23:49:05.005 INFO:tasks.workunit.client.1.smithi043.stdout:4/826: dread - d8/d15/d20/d2b/d54/db3/fe6 zero size 2024-09-16T23:49:05.006 INFO:tasks.workunit.client.1.smithi043.stdout:5/894: stat d3/d1b/l1d 0 2024-09-16T23:49:05.017 INFO:tasks.workunit.client.1.smithi043.stdout:0/822: dread d0/d2/df/d1a/d3b/f7d [0,4194304] 0 2024-09-16T23:49:05.068 INFO:tasks.workunit.client.1.smithi043.stdout:2/683: truncate d8/d1d/d20/d26/fb5 394468 0 2024-09-16T23:49:05.069 INFO:tasks.workunit.client.1.smithi043.stdout:9/710: getdents d4/d1c/d25/d2b/d4c/dd2 0 2024-09-16T23:49:05.071 INFO:tasks.workunit.client.1.smithi043.stdout:9/711: truncate d4/d21/d35/fd5 406786 0 2024-09-16T23:49:05.074 INFO:tasks.workunit.client.1.smithi043.stdout:8/732: link de/d70/cbe de/d18/d27/cdd 0 2024-09-16T23:49:05.074 INFO:tasks.workunit.client.1.smithi043.stdout:2/684: dwrite d8/d1d/d25/fa4 [0,4194304] 0 2024-09-16T23:49:05.081 INFO:tasks.workunit.client.1.smithi043.stdout:5/895: creat d3/d9/d22/d4a/d64/d8f/d109/f131 x:0 0 0 2024-09-16T23:49:05.082 INFO:tasks.workunit.client.1.smithi043.stdout:0/823: readlink d0/d2/df/d1a/l10b 0 2024-09-16T23:49:05.083 INFO:tasks.workunit.client.0.smithi042.stdout:1/691: write d0/d17/f36 [2947679,64989] 0 2024-09-16T23:49:05.086 INFO:tasks.workunit.client.1.smithi043.stdout:9/712: creat d4/d28/d5f/d91/fe0 x:0 0 0 2024-09-16T23:49:05.086 INFO:tasks.workunit.client.1.smithi043.stdout:1/904: write d4/d34/fbe [668016,129375] 0 2024-09-16T23:49:05.088 INFO:tasks.workunit.client.1.smithi043.stdout:1/905: chown d4/d9/d1c/d1e/d42/d5d/d71/f11a 3817100 1 2024-09-16T23:49:05.090 INFO:tasks.workunit.client.1.smithi043.stdout:1/906: chown d4/d9/d8f/db6/dcc/dda/d106 488066 1 2024-09-16T23:49:05.091 INFO:tasks.workunit.client.1.smithi043.stdout:8/733: symlink de/d18/d2b/lde 0 2024-09-16T23:49:05.093 INFO:tasks.workunit.client.1.smithi043.stdout:7/736: truncate d4/db/f5e 2700034 0 2024-09-16T23:49:05.094 INFO:tasks.workunit.client.0.smithi042.stdout:1/692: getdents d0/d9/d4e 0 2024-09-16T23:49:05.096 INFO:tasks.workunit.client.1.smithi043.stdout:0/824: fdatasync d0/d2/df/d1a/d46/d39/f62 0 2024-09-16T23:49:05.097 INFO:tasks.workunit.client.1.smithi043.stdout:3/778: getdents d4/de/d1f/d2c/db7/dc9 0 2024-09-16T23:49:05.097 INFO:tasks.workunit.client.1.smithi043.stdout:3/779: dread - d4/de/fe2 zero size 2024-09-16T23:49:05.098 INFO:tasks.workunit.client.0.smithi042.stdout:1/693: mkdir d0/d9/d90/d6b/d8e/df8 0 2024-09-16T23:49:05.098 INFO:tasks.workunit.client.1.smithi043.stdout:0/825: readlink d0/d2/df/d1a/d46/d39/daa/lb1 0 2024-09-16T23:49:05.099 INFO:tasks.workunit.client.1.smithi043.stdout:9/713: creat d4/d1c/d25/db1/fe1 x:0 0 0 2024-09-16T23:49:05.099 INFO:tasks.workunit.client.0.smithi042.stdout:1/694: readlink d0/d9/d26/d8b/lf0 0 2024-09-16T23:49:05.102 INFO:tasks.workunit.client.1.smithi043.stdout:6/737: rename de/d23/d63/f96 to de/d23/d34/d9b/ffd 0 2024-09-16T23:49:05.108 INFO:tasks.workunit.client.1.smithi043.stdout:3/780: dwrite d4/fb8 [0,4194304] 0 2024-09-16T23:49:05.108 INFO:tasks.workunit.client.1.smithi043.stdout:4/827: creat d8/d25/d39/d41/d57/d85/f102 x:0 0 0 2024-09-16T23:49:05.110 INFO:tasks.workunit.client.1.smithi043.stdout:9/714: dread d4/d5/d64/f65 [4194304,4194304] 0 2024-09-16T23:49:05.111 INFO:tasks.workunit.client.1.smithi043.stdout:1/907: write d4/df/d26/d79/fd7 [3517583,103776] 0 2024-09-16T23:49:05.111 INFO:tasks.workunit.client.0.smithi042.stdout:1/695: link d0/d9/ce9 d0/d9/d26/dc6/cf9 0 2024-09-16T23:49:05.111 INFO:tasks.workunit.client.1.smithi043.stdout:5/896: dwrite d3/d9/f4b [0,4194304] 0 2024-09-16T23:49:05.113 INFO:tasks.workunit.client.1.smithi043.stdout:4/828: write d8/d15/d20/d2b/f5a [1714207,116353] 0 2024-09-16T23:49:05.113 INFO:tasks.workunit.client.1.smithi043.stdout:0/826: chown d0/lb 16429 1 2024-09-16T23:49:05.115 INFO:tasks.workunit.client.1.smithi043.stdout:7/737: write d4/db/d50/f70 [1170038,34915] 0 2024-09-16T23:49:05.116 INFO:tasks.workunit.client.0.smithi042.stdout:1/696: dread d0/d9/d90/d6b/d88/f75 [0,4194304] 0 2024-09-16T23:49:05.118 INFO:tasks.workunit.client.1.smithi043.stdout:6/738: mkdir de/d23/d34/d9b/d5a/dfe 0 2024-09-16T23:49:05.118 INFO:tasks.workunit.client.1.smithi043.stdout:8/734: getdents de/d18/d1f/d25/d65/d7f/dca 0 2024-09-16T23:49:05.119 INFO:tasks.workunit.client.1.smithi043.stdout:5/897: dread - d3/db/d19/d3e/d28/dcb/f118 zero size 2024-09-16T23:49:05.122 INFO:tasks.workunit.client.1.smithi043.stdout:4/829: write d8/d15/d20/d2b/dc8/fe3 [554128,98145] 0 2024-09-16T23:49:05.122 INFO:tasks.workunit.client.1.smithi043.stdout:0/827: write d0/d2/df/d1a/d46/d39/fff [2392603,82870] 0 2024-09-16T23:49:05.125 INFO:tasks.workunit.client.1.smithi043.stdout:5/898: chown d3/db/d19/l11c 854 1 2024-09-16T23:49:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:04 smithi043 ceph-mon[35191]: pgmap v11: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 74 MiB/s rd, 232 MiB/s wr, 448 op/s 2024-09-16T23:49:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:04 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:04 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:05.128 INFO:tasks.workunit.client.1.smithi043.stdout:2/685: rename d8/d1d/d20/f58 to d8/d5e/d83/d9e/dab/fed 0 2024-09-16T23:49:05.129 INFO:tasks.workunit.client.1.smithi043.stdout:3/781: unlink d4/l14 0 2024-09-16T23:49:05.130 INFO:tasks.workunit.client.1.smithi043.stdout:2/686: chown d8/d5e/d83/d9e/dab/dd3 346623 1 2024-09-16T23:49:05.131 INFO:tasks.workunit.client.1.smithi043.stdout:0/828: truncate d0/d2/df/d1a/d46/d39/d83/dfb/f106 744988 0 2024-09-16T23:49:05.131 INFO:tasks.workunit.client.0.smithi042.stdout:1/697: dwrite d0/d9/fe [0,4194304] 0 2024-09-16T23:49:05.132 INFO:tasks.workunit.client.1.smithi043.stdout:9/715: write d4/d21/d33/f79 [1575298,64454] 0 2024-09-16T23:49:05.134 INFO:tasks.workunit.client.1.smithi043.stdout:0/829: write d0/d2/df/da7/db8/f60 [3979448,87149] 0 2024-09-16T23:49:05.137 INFO:tasks.workunit.client.1.smithi043.stdout:9/716: write d4/d5/f92 [30403,56437] 0 2024-09-16T23:49:05.140 INFO:tasks.workunit.client.1.smithi043.stdout:4/830: dread d8/d25/d39/d41/f9e [0,4194304] 0 2024-09-16T23:49:05.140 INFO:tasks.workunit.client.1.smithi043.stdout:6/739: dread de/d1a/d80/fdd [0,4194304] 0 2024-09-16T23:49:05.142 INFO:tasks.workunit.client.1.smithi043.stdout:7/738: rename d4/db/c67 to d4/d27/d8a/cf5 0 2024-09-16T23:49:05.145 INFO:tasks.workunit.client.1.smithi043.stdout:7/739: read - d4/d27/d41/d1b/d95/fec zero size 2024-09-16T23:49:05.146 INFO:tasks.workunit.client.1.smithi043.stdout:3/782: mkdir d4/de/d5d/def 0 2024-09-16T23:49:05.146 INFO:tasks.workunit.client.0.smithi042.stdout:1/698: symlink d0/d9a/lfa 0 2024-09-16T23:49:05.148 INFO:tasks.workunit.client.1.smithi043.stdout:0/830: mknod d0/d2/df/da7/c113 0 2024-09-16T23:49:05.154 INFO:tasks.workunit.client.1.smithi043.stdout:5/899: mknod d3/db/d19/d3e/c132 0 2024-09-16T23:49:05.155 INFO:tasks.workunit.client.1.smithi043.stdout:6/740: symlink de/d23/ddc/lff 0 2024-09-16T23:49:05.156 INFO:tasks.workunit.client.1.smithi043.stdout:4/831: mkdir d8/d15/d20/d2b/d23/d27/d4b/d103 0 2024-09-16T23:49:05.157 INFO:tasks.workunit.client.1.smithi043.stdout:7/740: chown d4/db/d50/cf3 145 1 2024-09-16T23:49:05.158 INFO:tasks.workunit.client.1.smithi043.stdout:7/741: dread - d4/db/d11/d21/d94/ff4 zero size 2024-09-16T23:49:05.159 INFO:tasks.workunit.client.1.smithi043.stdout:2/687: truncate d8/d1f/f5c 1201076 0 2024-09-16T23:49:05.161 INFO:tasks.workunit.client.1.smithi043.stdout:5/900: mkdir d3/d9/d22/d4a/d64/d133 0 2024-09-16T23:49:05.162 INFO:tasks.workunit.client.1.smithi043.stdout:5/901: fsync d3/d9/d6f/d85/f116 0 2024-09-16T23:49:05.162 INFO:tasks.workunit.client.1.smithi043.stdout:1/908: getdents d4/d9/d1c/d1e/d42 0 2024-09-16T23:49:05.164 INFO:tasks.workunit.client.1.smithi043.stdout:4/832: dread - d8/d15/d9b/fa7 zero size 2024-09-16T23:49:05.165 INFO:tasks.workunit.client.0.smithi042.stdout:6/595: sync 2024-09-16T23:49:05.165 INFO:tasks.workunit.client.1.smithi043.stdout:9/717: write d4/d21/d35/f5d [2879052,91510] 0 2024-09-16T23:49:05.168 INFO:tasks.workunit.client.1.smithi043.stdout:7/742: creat d4/d27/d41/d1b/d95/ff6 x:0 0 0 2024-09-16T23:49:05.169 INFO:tasks.workunit.client.0.smithi042.stdout:1/699: dwrite d0/d9/d90/d6b/d88/d32/fb6 [0,4194304] 0 2024-09-16T23:49:05.173 INFO:tasks.workunit.client.1.smithi043.stdout:7/743: write d4/d27/d41/f8d [7378267,27335] 0 2024-09-16T23:49:05.173 INFO:tasks.workunit.client.1.smithi043.stdout:1/909: dwrite d4/d3e/fdb [0,4194304] 0 2024-09-16T23:49:05.173 INFO:tasks.workunit.client.0.smithi042.stdout:6/596: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/fbb x:0 0 0 2024-09-16T23:49:05.176 INFO:tasks.workunit.client.0.smithi042.stdout:1/700: fsync d0/d9a/fb7 0 2024-09-16T23:49:05.176 INFO:tasks.workunit.client.1.smithi043.stdout:8/735: rename de/d18/d1f/d2f/d39/d7b/la2 to de/d18/d1f/d2f/ldf 0 2024-09-16T23:49:05.177 INFO:tasks.workunit.client.1.smithi043.stdout:9/718: dwrite d4/d1c/d6b/f3c [0,4194304] 0 2024-09-16T23:49:05.179 INFO:tasks.workunit.client.0.smithi042.stdout:6/597: creat d8/d70/d81/dd/d2c/d57/fbc x:0 0 0 2024-09-16T23:49:05.180 INFO:tasks.workunit.client.0.smithi042.stdout:6/598: chown d8/d70/d81/dd/d2c/d57/c56 11 1 2024-09-16T23:49:05.181 INFO:tasks.workunit.client.0.smithi042.stdout:6/599: truncate d8/d70/d84/fb5 705660 0 2024-09-16T23:49:05.192 INFO:tasks.workunit.client.1.smithi043.stdout:9/719: dread d4/d5/d16/f54 [0,4194304] 0 2024-09-16T23:49:05.193 INFO:tasks.workunit.client.0.smithi042.stdout:1/701: creat d0/d17/d18/ffb x:0 0 0 2024-09-16T23:49:05.195 INFO:tasks.workunit.client.1.smithi043.stdout:0/831: creat d0/d2/df/f114 x:0 0 0 2024-09-16T23:49:05.197 INFO:tasks.workunit.client.0.smithi042.stdout:6/600: fdatasync d8/d70/d81/dd/d1b/d50/f98 0 2024-09-16T23:49:05.198 INFO:tasks.workunit.client.1.smithi043.stdout:2/688: creat d8/d5e/d83/d3a/d77/dea/fee x:0 0 0 2024-09-16T23:49:05.199 INFO:tasks.workunit.client.1.smithi043.stdout:2/689: chown d8/d4b/d4f/lb3 61 1 2024-09-16T23:49:05.204 INFO:tasks.workunit.client.0.smithi042.stdout:6/601: symlink d8/d70/d81/dd/d2c/d57/lbd 0 2024-09-16T23:49:05.204 INFO:tasks.workunit.client.1.smithi043.stdout:4/833: dwrite d8/d25/d39/d58/f9a [0,4194304] 0 2024-09-16T23:49:05.205 INFO:tasks.workunit.client.1.smithi043.stdout:7/744: mkdir d4/db/d43/d83/d85/d8b/df7 0 2024-09-16T23:49:05.206 INFO:tasks.workunit.client.1.smithi043.stdout:4/834: fsync d8/d15/d9b/d8b/f8f 0 2024-09-16T23:49:05.206 INFO:tasks.workunit.client.0.smithi042.stdout:1/702: symlink d0/d17/df1/lfc 0 2024-09-16T23:49:05.208 INFO:tasks.workunit.client.1.smithi043.stdout:8/736: creat de/d18/fe0 x:0 0 0 2024-09-16T23:49:05.214 INFO:tasks.workunit.client.1.smithi043.stdout:5/902: symlink d3/d79/dcc/l134 0 2024-09-16T23:49:05.215 INFO:tasks.workunit.client.1.smithi043.stdout:1/910: dwrite d4/df/f38 [0,4194304] 0 2024-09-16T23:49:05.216 INFO:tasks.workunit.client.1.smithi043.stdout:6/741: link de/d23/d49/l54 de/d23/d34/d9b/d8b/de9/l100 0 2024-09-16T23:49:05.216 INFO:tasks.workunit.client.1.smithi043.stdout:1/911: dread - d4/d3e/d5b/f115 zero size 2024-09-16T23:49:05.217 INFO:tasks.workunit.client.0.smithi042.stdout:6/602: dread d8/d70/d81/dd/d1b/f4b [0,4194304] 0 2024-09-16T23:49:05.217 INFO:tasks.workunit.client.1.smithi043.stdout:1/912: chown d4/d9/f46 0 1 2024-09-16T23:49:05.219 INFO:tasks.workunit.client.1.smithi043.stdout:9/720: unlink d4/d21/c74 0 2024-09-16T23:49:05.223 INFO:tasks.workunit.client.1.smithi043.stdout:2/690: rename d8/d11/f46 to d8/d5e/d83/d9e/fef 0 2024-09-16T23:49:05.225 INFO:tasks.workunit.client.0.smithi042.stdout:6/603: rename d8/d70/d81/dd/d1b/c2a to d8/d70/d84/cbe 0 2024-09-16T23:49:05.225 INFO:tasks.workunit.client.0.smithi042.stdout:1/703: dwrite d0/d17/d79/f7c [0,4194304] 0 2024-09-16T23:49:05.227 INFO:tasks.workunit.client.1.smithi043.stdout:7/745: fsync d4/dc0/fd6 0 2024-09-16T23:49:05.228 INFO:tasks.workunit.client.1.smithi043.stdout:4/835: rmdir d8/d25/d39/d41/d57 39 2024-09-16T23:49:05.230 INFO:tasks.workunit.client.1.smithi043.stdout:4/836: stat d8/d25/lf9 0 2024-09-16T23:49:05.232 INFO:tasks.workunit.client.0.smithi042.stdout:1/704: rename d0/f3 to d0/d9/d90/d6b/d88/d32/d2f/de3/ffd 0 2024-09-16T23:49:05.234 INFO:tasks.workunit.client.1.smithi043.stdout:3/783: dwrite d4/d30/d53/d88/db1/fd6 [0,4194304] 0 2024-09-16T23:49:05.235 INFO:tasks.workunit.client.0.smithi042.stdout:6/604: mkdir d8/d70/d81/dd/d1b/dbf 0 2024-09-16T23:49:05.237 INFO:tasks.workunit.client.1.smithi043.stdout:5/903: dread - d3/d79/d91/daf/fbc zero size 2024-09-16T23:49:05.237 INFO:tasks.workunit.client.1.smithi043.stdout:6/742: mknod de/d23/d34/d66/d9d/c101 0 2024-09-16T23:49:05.238 INFO:tasks.workunit.client.1.smithi043.stdout:1/913: unlink d4/c12d 0 2024-09-16T23:49:05.239 INFO:tasks.workunit.client.0.smithi042.stdout:6/605: symlink d8/d70/d81/d90/lc0 0 2024-09-16T23:49:05.240 INFO:tasks.workunit.client.1.smithi043.stdout:9/721: creat d4/d5/fe2 x:0 0 0 2024-09-16T23:49:05.242 INFO:tasks.workunit.client.1.smithi043.stdout:0/832: creat d0/d2/df/d1a/d3b/dee/df0/df8/f115 x:0 0 0 2024-09-16T23:49:05.244 INFO:tasks.workunit.client.0.smithi042.stdout:6/606: creat d8/d70/d81/dd/d1b/d2d/da1/fc1 x:0 0 0 2024-09-16T23:49:05.245 INFO:tasks.workunit.client.1.smithi043.stdout:2/691: dread d8/d1d/d25/f2b [0,4194304] 0 2024-09-16T23:49:05.245 INFO:tasks.workunit.client.1.smithi043.stdout:3/784: dwrite d4/de/fe2 [0,4194304] 0 2024-09-16T23:49:05.249 INFO:tasks.workunit.client.1.smithi043.stdout:7/746: rename d4/d27/d41/d1b/d66/da3 to d4/db/d11/d21/d31/dcb/df8 0 2024-09-16T23:49:05.252 INFO:tasks.workunit.client.1.smithi043.stdout:4/837: mkdir d8/d15/d20/d2b/d23/d27/d4b/d104 0 2024-09-16T23:49:05.254 INFO:tasks.workunit.client.0.smithi042.stdout:6/607: rename d8/d70/d81/dd/l17 to d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/lc2 0 2024-09-16T23:49:05.256 INFO:tasks.workunit.client.1.smithi043.stdout:1/914: creat d4/d9/d1c/d1e/d47/f138 x:0 0 0 2024-09-16T23:49:05.258 INFO:tasks.workunit.client.1.smithi043.stdout:9/722: creat d4/d1c/fe3 x:0 0 0 2024-09-16T23:49:05.260 INFO:tasks.workunit.client.0.smithi042.stdout:6/608: symlink d8/d70/d84/lc3 0 2024-09-16T23:49:05.262 INFO:tasks.workunit.client.1.smithi043.stdout:0/833: chown d0/d2/df/d1a/d46/d39/d6e/f65 615 1 2024-09-16T23:49:05.263 INFO:tasks.workunit.client.0.smithi042.stdout:6/609: mknod d8/d70/d81/dd/d1b/d50/cc4 0 2024-09-16T23:49:05.265 INFO:tasks.workunit.client.1.smithi043.stdout:9/723: dwrite d4/d28/d5f/d86/f96 [4194304,4194304] 0 2024-09-16T23:49:05.266 INFO:tasks.workunit.client.0.smithi042.stdout:6/610: mkdir d8/d70/d81/dd/d2c/d57/dc5 0 2024-09-16T23:49:05.269 INFO:tasks.workunit.client.1.smithi043.stdout:7/747: symlink d4/db/d11/d15/d29/lf9 0 2024-09-16T23:49:05.270 INFO:tasks.workunit.client.1.smithi043.stdout:8/737: dwrite de/fb0 [4194304,4194304] 0 2024-09-16T23:49:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:04 smithi042 ceph-mon[27914]: pgmap v11: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 74 MiB/s rd, 232 MiB/s wr, 448 op/s 2024-09-16T23:49:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:04 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:05.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:04 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:05.274 INFO:tasks.workunit.client.0.smithi042.stdout:6/611: dread d8/d70/d81/dd/d2c/d57/f5d [0,4194304] 0 2024-09-16T23:49:05.276 INFO:tasks.workunit.client.1.smithi043.stdout:6/743: link de/d23/d63/fd2 de/d23/d34/d9b/d50/f102 0 2024-09-16T23:49:05.276 INFO:tasks.workunit.client.1.smithi043.stdout:1/915: mknod d4/d9/d8f/db6/dcc/dda/d107/c139 0 2024-09-16T23:49:05.278 INFO:tasks.workunit.client.0.smithi042.stdout:6/612: symlink d8/d70/d81/dd/d1b/d67/lc6 0 2024-09-16T23:49:05.278 INFO:tasks.workunit.client.0.smithi042.stdout:6/613: readlink d8/l4d 0 2024-09-16T23:49:05.280 INFO:tasks.workunit.client.0.smithi042.stdout:6/614: dread - d8/d70/d81/dd/d2c/d57/f60 zero size 2024-09-16T23:49:05.281 INFO:tasks.workunit.client.0.smithi042.stdout:6/615: dread - d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d99/fb6 zero size 2024-09-16T23:49:05.282 INFO:tasks.workunit.client.1.smithi043.stdout:3/785: rename d4/d30/d53 to d4/de/d40/df0 0 2024-09-16T23:49:05.285 INFO:tasks.workunit.client.0.smithi042.stdout:6/616: mkdir d8/d70/d81/dd/d1b/d67/dc7 0 2024-09-16T23:49:05.288 INFO:tasks.workunit.client.1.smithi043.stdout:4/838: unlink d8/d15/f3c 0 2024-09-16T23:49:05.289 INFO:tasks.workunit.client.0.smithi042.stdout:6/617: fdatasync d8/d70/d81/dd/f5f 0 2024-09-16T23:49:05.290 INFO:tasks.workunit.client.1.smithi043.stdout:0/834: getdents d0/d2/df/d1a/de8/d110 0 2024-09-16T23:49:05.290 INFO:tasks.workunit.client.0.smithi042.stdout:6/618: readlink d8/d70/d81/dd/d2c/l88 0 2024-09-16T23:49:05.295 INFO:tasks.workunit.client.1.smithi043.stdout:6/744: rename de/ce7 to de/d23/d49/dc9/c103 0 2024-09-16T23:49:05.296 INFO:tasks.workunit.client.1.smithi043.stdout:6/745: dread - de/d23/d34/d9b/d50/f79 zero size 2024-09-16T23:49:05.297 INFO:tasks.workunit.client.1.smithi043.stdout:9/724: fsync d4/d21/d35/d4b/fb7 0 2024-09-16T23:49:05.297 INFO:tasks.workunit.client.1.smithi043.stdout:6/746: chown de/d1a/d28/f46 109 1 2024-09-16T23:49:05.299 INFO:tasks.workunit.client.1.smithi043.stdout:3/786: mknod d4/de/d40/d4d/cf1 0 2024-09-16T23:49:05.301 INFO:tasks.workunit.client.0.smithi042.stdout:6/619: creat d8/d70/d81/dd/d1b/fc8 x:0 0 0 2024-09-16T23:49:05.301 INFO:tasks.workunit.client.1.smithi043.stdout:8/738: chown de/d18/d1f/d25/d65/f8a 126467 1 2024-09-16T23:49:05.305 INFO:tasks.workunit.client.0.smithi042.stdout:6/620: dread d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/f79 [0,4194304] 0 2024-09-16T23:49:05.305 INFO:tasks.workunit.client.1.smithi043.stdout:2/692: getdents d8/d1d/d20/d26 0 2024-09-16T23:49:05.306 INFO:tasks.workunit.client.1.smithi043.stdout:8/739: dread - de/d18/d1f/d2f/d39/d7b/d83/fd3 zero size 2024-09-16T23:49:05.306 INFO:tasks.workunit.client.1.smithi043.stdout:7/748: dwrite d4/db/d43/d83/d85/d8b/f9f [0,4194304] 0 2024-09-16T23:49:05.307 INFO:tasks.workunit.client.1.smithi043.stdout:9/725: fsync d4/d21/f4d 0 2024-09-16T23:49:05.309 INFO:tasks.workunit.client.1.smithi043.stdout:6/747: fsync de/d23/fa2 0 2024-09-16T23:49:05.309 INFO:tasks.workunit.client.1.smithi043.stdout:4/839: mkdir d8/d25/d39/dd3/d100/d105 0 2024-09-16T23:49:05.310 INFO:tasks.workunit.client.1.smithi043.stdout:7/749: chown d4/f2b 229124 1 2024-09-16T23:49:05.311 INFO:tasks.workunit.client.1.smithi043.stdout:1/916: dwrite d4/d9/d1c/d1e/d47/f53 [0,4194304] 0 2024-09-16T23:49:05.313 INFO:tasks.workunit.client.1.smithi043.stdout:9/726: stat d4/d21/c75 0 2024-09-16T23:49:05.314 INFO:tasks.workunit.client.0.smithi042.stdout:6/621: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/f2e [0,4194304] 0 2024-09-16T23:49:05.318 INFO:tasks.workunit.client.1.smithi043.stdout:3/787: dread d4/de/f7f [0,4194304] 0 2024-09-16T23:49:05.319 INFO:tasks.workunit.client.1.smithi043.stdout:0/835: creat d0/d2/df/f116 x:0 0 0 2024-09-16T23:49:05.321 INFO:tasks.workunit.client.1.smithi043.stdout:7/750: chown d4/d27/d41/c63 36237436 1 2024-09-16T23:49:05.323 INFO:tasks.workunit.client.1.smithi043.stdout:6/748: truncate de/d1a/f20 134292 0 2024-09-16T23:49:05.323 INFO:tasks.workunit.client.1.smithi043.stdout:7/751: write d4/dc0/fd6 [590842,121327] 0 2024-09-16T23:49:05.324 INFO:tasks.workunit.client.1.smithi043.stdout:0/836: write d0/d2/df/d1a/de8/fed [4570107,121620] 0 2024-09-16T23:49:05.325 INFO:tasks.workunit.client.1.smithi043.stdout:9/727: fsync d4/d1c/f27 0 2024-09-16T23:49:05.325 INFO:tasks.workunit.client.0.smithi042.stdout:6/622: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d99/fb6 [0,4194304] 0 2024-09-16T23:49:05.329 INFO:tasks.workunit.client.0.smithi042.stdout:6/623: mkdir d8/d70/d81/dd/d1b/d67/d97/dc9 0 2024-09-16T23:49:05.333 INFO:tasks.workunit.client.1.smithi043.stdout:3/788: creat d4/de/d40/d4d/ff2 x:0 0 0 2024-09-16T23:49:05.333 INFO:tasks.workunit.client.1.smithi043.stdout:4/840: creat d8/d25/d39/dd3/d100/d105/f106 x:0 0 0 2024-09-16T23:49:05.339 INFO:tasks.workunit.client.1.smithi043.stdout:3/789: dread d4/de/d1f/d2c/db7/dc3/fe0 [0,4194304] 0 2024-09-16T23:49:05.340 INFO:tasks.workunit.client.1.smithi043.stdout:9/728: creat d4/d1c/d4f/d45/d85/fe4 x:0 0 0 2024-09-16T23:49:05.343 INFO:tasks.workunit.client.1.smithi043.stdout:9/729: write d4/d21/d35/f5d [3124115,125103] 0 2024-09-16T23:49:05.344 INFO:tasks.workunit.client.1.smithi043.stdout:1/917: rename d4/d3e/d95/fde to d4/df/df4/f13a 0 2024-09-16T23:49:05.344 INFO:tasks.workunit.client.1.smithi043.stdout:4/841: dwrite d8/d25/d39/d58/fa5 [0,4194304] 0 2024-09-16T23:49:05.346 INFO:tasks.workunit.client.1.smithi043.stdout:2/693: truncate d8/d1d/d20/d26/f7d 749181 0 2024-09-16T23:49:05.347 INFO:tasks.workunit.client.1.smithi043.stdout:6/749: creat de/f104 x:0 0 0 2024-09-16T23:49:05.349 INFO:tasks.workunit.client.1.smithi043.stdout:6/750: stat de/d1a/d28/l91 0 2024-09-16T23:49:05.350 INFO:tasks.workunit.client.1.smithi043.stdout:3/790: creat d4/de/d5d/def/ff3 x:0 0 0 2024-09-16T23:49:05.350 INFO:tasks.workunit.client.0.smithi042.stdout:9/447: sync 2024-09-16T23:49:05.350 INFO:tasks.workunit.client.0.smithi042.stdout:3/855: sync 2024-09-16T23:49:05.350 INFO:tasks.workunit.client.0.smithi042.stdout:5/752: sync 2024-09-16T23:49:05.351 INFO:tasks.workunit.client.0.smithi042.stdout:8/491: sync 2024-09-16T23:49:05.351 INFO:tasks.workunit.client.0.smithi042.stdout:0/536: sync 2024-09-16T23:49:05.352 INFO:tasks.workunit.client.1.smithi043.stdout:1/918: dread d4/d9/f17 [0,4194304] 0 2024-09-16T23:49:05.352 INFO:tasks.workunit.client.0.smithi042.stdout:0/537: fdatasync d0/f60 0 2024-09-16T23:49:05.355 INFO:tasks.workunit.client.0.smithi042.stdout:6/624: dwrite d8/d70/d81/dd/d2c/d57/f60 [0,4194304] 0 2024-09-16T23:49:05.355 INFO:tasks.workunit.client.0.smithi042.stdout:8/492: readlink da/d19/d1d/d30/d49/d58/d6a/l7d 0 2024-09-16T23:49:05.356 INFO:tasks.workunit.client.0.smithi042.stdout:5/753: symlink d9/db/d1b/d31/d6c/l108 0 2024-09-16T23:49:05.357 INFO:tasks.workunit.client.0.smithi042.stdout:0/538: fdatasync d0/f4b 0 2024-09-16T23:49:05.359 INFO:tasks.workunit.client.0.smithi042.stdout:3/856: mkdir d2/d1b/d1d/d34/d50/d129 0 2024-09-16T23:49:05.359 INFO:tasks.workunit.client.0.smithi042.stdout:0/539: dread - d0/df/d13/fa0 zero size 2024-09-16T23:49:05.361 INFO:tasks.workunit.client.0.smithi042.stdout:9/448: chown d4/d10/d24 899710082 1 2024-09-16T23:49:05.361 INFO:tasks.workunit.client.1.smithi043.stdout:2/694: dwrite d8/d5e/d83/d9e/fef [0,4194304] 0 2024-09-16T23:49:05.362 INFO:tasks.workunit.client.0.smithi042.stdout:9/449: dread - d4/d12/d16/d68/f71 zero size 2024-09-16T23:49:05.362 INFO:tasks.workunit.client.0.smithi042.stdout:8/493: chown da/d19/d2c/d56/l64 3330980 1 2024-09-16T23:49:05.364 INFO:tasks.workunit.client.0.smithi042.stdout:6/625: truncate d8/d70/d81/dd/d1b/f93 164335 0 2024-09-16T23:49:05.365 INFO:tasks.workunit.client.1.smithi043.stdout:9/730: mkdir d4/d1c/d25/d2b/d4c/dd2/de5 0 2024-09-16T23:49:05.365 INFO:tasks.workunit.client.1.smithi043.stdout:9/731: chown d4/d28/dcf 28799 1 2024-09-16T23:49:05.366 INFO:tasks.workunit.client.0.smithi042.stdout:9/450: read d4/d12/d2d/d37/d28/f31 [306541,71194] 0 2024-09-16T23:49:05.366 INFO:tasks.workunit.client.0.smithi042.stdout:3/857: creat d2/d1b/d1d/d30/da2/f12a x:0 0 0 2024-09-16T23:49:05.367 INFO:tasks.workunit.client.1.smithi043.stdout:3/791: symlink d4/de/d1f/d2c/db7/dc3/lf4 0 2024-09-16T23:49:05.368 INFO:tasks.workunit.client.0.smithi042.stdout:9/451: write d4/d12/d2d/d37/d3a/f48 [3362958,104838] 0 2024-09-16T23:49:05.368 INFO:tasks.workunit.client.0.smithi042.stdout:5/754: unlink d9/db/d19/d21/d41/l9c 0 2024-09-16T23:49:05.370 INFO:tasks.workunit.client.0.smithi042.stdout:6/626: write d8/d70/d81/dd/d1b/d2d/da1/fc1 [342410,130767] 0 2024-09-16T23:49:05.370 INFO:tasks.workunit.client.1.smithi043.stdout:1/919: mkdir d4/d9/d1c/d1e/d42/d5d/d110/d13b 0 2024-09-16T23:49:05.371 INFO:tasks.workunit.client.0.smithi042.stdout:0/540: creat d0/df/d23/d33/d51/d5b/fa8 x:0 0 0 2024-09-16T23:49:05.371 INFO:tasks.workunit.client.0.smithi042.stdout:5/755: truncate d9/d2c/f106 93259 0 2024-09-16T23:49:05.373 INFO:tasks.workunit.client.1.smithi043.stdout:4/842: dwrite d8/fbe [0,4194304] 0 2024-09-16T23:49:05.375 INFO:tasks.workunit.client.1.smithi043.stdout:4/843: write d8/d15/f98 [4593353,10767] 0 2024-09-16T23:49:05.376 INFO:tasks.workunit.client.0.smithi042.stdout:3/858: mkdir d2/d1b/d1d/d34/d50/df5/d12b 0 2024-09-16T23:49:05.377 INFO:tasks.workunit.client.1.smithi043.stdout:0/837: link d0/l4 d0/d2/df/d1a/d49/db2/l117 0 2024-09-16T23:49:05.380 INFO:tasks.workunit.client.1.smithi043.stdout:2/695: rmdir d8/d11/d16 39 2024-09-16T23:49:05.380 INFO:tasks.workunit.client.0.smithi042.stdout:5/756: write d9/db/d1b/da4/ffe [2025026,54084] 0 2024-09-16T23:49:05.383 INFO:tasks.workunit.client.0.smithi042.stdout:9/452: dwrite d4/d12/d16/f85 [0,4194304] 0 2024-09-16T23:49:05.384 INFO:tasks.workunit.client.0.smithi042.stdout:9/453: stat d4/d10/l13 0 2024-09-16T23:49:05.384 INFO:tasks.workunit.client.1.smithi043.stdout:6/751: link de/d1a/d80/fdd de/d23/d63/dec/f105 0 2024-09-16T23:49:05.385 INFO:tasks.workunit.client.0.smithi042.stdout:0/541: getdents d0/df/d13/d2d/d35 0 2024-09-16T23:49:05.386 INFO:tasks.workunit.client.0.smithi042.stdout:6/627: rmdir d8/d70/d81/dd/d1b/d50/d96 39 2024-09-16T23:49:05.387 INFO:tasks.workunit.client.0.smithi042.stdout:5/757: mknod d9/db/de/d92/db5/c109 0 2024-09-16T23:49:05.388 INFO:tasks.workunit.client.1.smithi043.stdout:3/792: mknod d4/de/d5d/cf5 0 2024-09-16T23:49:05.390 INFO:tasks.workunit.client.0.smithi042.stdout:3/859: mknod d2/d1b/d1d/d34/d50/d93/d96/d11e/c12c 0 2024-09-16T23:49:05.390 INFO:tasks.workunit.client.0.smithi042.stdout:6/628: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d7b/lb1 5428 1 2024-09-16T23:49:05.391 INFO:tasks.workunit.client.0.smithi042.stdout:9/454: symlink d4/d12/d2d/d51/l8d 0 2024-09-16T23:49:05.392 INFO:tasks.workunit.client.0.smithi042.stdout:9/455: dread - d4/d12/d2d/d37/d41/f49 zero size 2024-09-16T23:49:05.393 INFO:tasks.workunit.client.0.smithi042.stdout:6/629: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/ca6 1751 1 2024-09-16T23:49:05.393 INFO:tasks.workunit.client.0.smithi042.stdout:0/542: rename d0/df/d13/d49/d70/d94 to d0/df/d23/d33/d56/da9 0 2024-09-16T23:49:05.395 INFO:tasks.workunit.client.1.smithi043.stdout:1/920: read d4/df/d26/f39 [1640626,73021] 0 2024-09-16T23:49:05.395 INFO:tasks.workunit.client.0.smithi042.stdout:3/860: fdatasync d2/d1b/d89/f5c 0 2024-09-16T23:49:05.396 INFO:tasks.workunit.client.1.smithi043.stdout:6/752: dwrite de/d1a/d28/f84 [0,4194304] 0 2024-09-16T23:49:05.397 INFO:tasks.workunit.client.0.smithi042.stdout:3/861: truncate d2/d7/d61/d73/fde 4979194 0 2024-09-16T23:49:05.397 INFO:tasks.workunit.client.0.smithi042.stdout:9/456: mkdir d4/d12/d44/d8e 0 2024-09-16T23:49:05.398 INFO:tasks.workunit.client.0.smithi042.stdout:5/758: unlink d9/db/d19/d21/c91 0 2024-09-16T23:49:05.399 INFO:tasks.workunit.client.0.smithi042.stdout:6/630: creat d8/d70/d81/dd/d2c/d57/fca x:0 0 0 2024-09-16T23:49:05.402 INFO:tasks.workunit.client.0.smithi042.stdout:3/862: rename d2/d1b/d1d/d30/f79 to d2/d7/d61/f12d 0 2024-09-16T23:49:05.403 INFO:tasks.workunit.client.0.smithi042.stdout:6/631: fdatasync d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 0 2024-09-16T23:49:05.403 INFO:tasks.workunit.client.1.smithi043.stdout:3/793: symlink d4/de/d40/df0/d88/db1/dbb/lf6 0 2024-09-16T23:49:05.406 INFO:tasks.workunit.client.0.smithi042.stdout:5/759: read d9/db/de/d10/f20 [2928798,124422] 0 2024-09-16T23:49:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:3/863: write d2/d7/d61/d8e/d2c/f64 [5212111,37027] 0 2024-09-16T23:49:05.408 INFO:tasks.workunit.client.0.smithi042.stdout:8/494: truncate da/d19/d2c/f66 40458 0 2024-09-16T23:49:05.410 INFO:tasks.workunit.client.0.smithi042.stdout:5/760: write d9/db/d1b/da4/fe7 [837471,50042] 0 2024-09-16T23:49:05.412 INFO:tasks.workunit.client.0.smithi042.stdout:9/457: symlink d4/d12/d44/l8f 0 2024-09-16T23:49:05.414 INFO:tasks.workunit.client.1.smithi043.stdout:2/696: dread - d8/d11/d16/fa6 zero size 2024-09-16T23:49:05.414 INFO:tasks.workunit.client.0.smithi042.stdout:9/458: fsync d4/d12/f1e 0 2024-09-16T23:49:05.416 INFO:tasks.workunit.client.1.smithi043.stdout:4/844: write d8/d25/d64/f7f [3426443,37356] 0 2024-09-16T23:49:05.420 INFO:tasks.workunit.client.1.smithi043.stdout:9/732: creat d4/d1c/d4f/fe6 x:0 0 0 2024-09-16T23:49:05.421 INFO:tasks.workunit.client.0.smithi042.stdout:8/495: dwrite da/d2f/f77 [0,4194304] 0 2024-09-16T23:49:05.425 INFO:tasks.workunit.client.1.smithi043.stdout:1/921: dread d4/d9/d12/f1f [0,4194304] 0 2024-09-16T23:49:05.425 INFO:tasks.workunit.client.0.smithi042.stdout:6/632: rename d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/lc2 to d8/d70/d81/dd/d1b/d50/da2/lcb 0 2024-09-16T23:49:05.427 INFO:tasks.workunit.client.1.smithi043.stdout:3/794: symlink d4/de/d40/d4d/lf7 0 2024-09-16T23:49:05.428 INFO:tasks.workunit.client.0.smithi042.stdout:3/864: dwrite d2/f41 [0,4194304] 0 2024-09-16T23:49:05.428 INFO:tasks.workunit.client.1.smithi043.stdout:6/753: dread de/d1a/d28/f5b [0,4194304] 0 2024-09-16T23:49:05.430 INFO:tasks.workunit.client.0.smithi042.stdout:5/761: rename d9/db/de/d10/l3b to d9/db/d1b/d31/d46/d58/d9d/l10a 0 2024-09-16T23:49:05.433 INFO:tasks.workunit.client.0.smithi042.stdout:8/496: truncate da/f17 650877 0 2024-09-16T23:49:05.435 INFO:tasks.workunit.client.0.smithi042.stdout:5/762: fsync d9/db/de/d10/f1a 0 2024-09-16T23:49:05.436 INFO:tasks.workunit.client.0.smithi042.stdout:8/497: dread - da/d19/d2c/d56/f98 zero size 2024-09-16T23:49:05.437 INFO:tasks.workunit.client.0.smithi042.stdout:3/865: dwrite d2/d1b/d1d/f25 [0,4194304] 0 2024-09-16T23:49:05.438 INFO:tasks.workunit.client.1.smithi043.stdout:4/845: read d8/d15/d9b/d8b/f93 [1561236,15620] 0 2024-09-16T23:49:05.439 INFO:tasks.workunit.client.1.smithi043.stdout:0/838: dwrite d0/d2/df/d1a/d49/db2/df4/f70 [0,4194304] 0 2024-09-16T23:49:05.440 INFO:tasks.workunit.client.0.smithi042.stdout:6/633: rename d8/d70/d81/dd/d2c/d57/c56 to d8/d70/d81/dd/d1b/d2d/da1/ccc 0 2024-09-16T23:49:05.440 INFO:tasks.workunit.client.0.smithi042.stdout:5/763: symlink d9/db/d1b/da4/l10b 0 2024-09-16T23:49:05.442 INFO:tasks.workunit.client.1.smithi043.stdout:9/733: mknod d4/d21/d35/ce7 0 2024-09-16T23:49:05.442 INFO:tasks.workunit.client.1.smithi043.stdout:0/839: dread d0/d2/df/d1a/d46/d92/fac [0,4194304] 0 2024-09-16T23:49:05.444 INFO:tasks.workunit.client.1.smithi043.stdout:0/840: dread - d0/d2/df/d1a/d49/db2/de1/fe9 zero size 2024-09-16T23:49:05.445 INFO:tasks.workunit.client.1.smithi043.stdout:1/922: symlink d4/d9/d1c/d1e/d42/d5d/d120/l13c 0 2024-09-16T23:49:05.451 INFO:tasks.workunit.client.1.smithi043.stdout:3/795: read d4/de/d1f/f3e [212820,86276] 0 2024-09-16T23:49:05.452 INFO:tasks.workunit.client.0.smithi042.stdout:9/459: write d4/d10/f1b [1182773,7452] 0 2024-09-16T23:49:05.455 INFO:tasks.workunit.client.1.smithi043.stdout:6/754: unlink de/d23/d34/d9b/d8b/de9/l100 0 2024-09-16T23:49:05.456 INFO:tasks.workunit.client.0.smithi042.stdout:9/460: creat d4/d12/d2d/d37/d28/f90 x:0 0 0 2024-09-16T23:49:05.457 INFO:tasks.workunit.client.1.smithi043.stdout:9/734: dread d4/d1c/f20 [0,4194304] 0 2024-09-16T23:49:05.458 INFO:tasks.workunit.client.0.smithi042.stdout:6/634: rename d8/d70/d81/dd/d1b/d2d/d87 to d8/d70/d81/dd/dcd 0 2024-09-16T23:49:05.459 INFO:tasks.workunit.client.1.smithi043.stdout:2/697: dwrite d8/d35/f37 [0,4194304] 0 2024-09-16T23:49:05.460 INFO:tasks.workunit.client.1.smithi043.stdout:5/904: sync 2024-09-16T23:49:05.461 INFO:tasks.workunit.client.1.smithi043.stdout:4/846: unlink d8/d25/d39/d58/fb8 0 2024-09-16T23:49:05.465 INFO:tasks.workunit.client.0.smithi042.stdout:3/866: dwrite d2/d1b/d1d/d34/f58 [0,4194304] 0 2024-09-16T23:49:05.465 INFO:tasks.workunit.client.0.smithi042.stdout:5/764: dwrite d9/db/de/d10/f22 [0,4194304] 0 2024-09-16T23:49:05.467 INFO:tasks.workunit.client.0.smithi042.stdout:8/498: dread da/d19/d1d/d30/f31 [0,4194304] 0 2024-09-16T23:49:05.470 INFO:tasks.workunit.client.0.smithi042.stdout:6/635: unlink d8/d70/fba 0 2024-09-16T23:49:05.472 INFO:tasks.workunit.client.0.smithi042.stdout:3/867: symlink d2/d1b/d1d/d34/d50/df5/d12b/l12e 0 2024-09-16T23:49:05.473 INFO:tasks.workunit.client.0.smithi042.stdout:5/765: symlink d9/db/d19/d21/l10c 0 2024-09-16T23:49:05.475 INFO:tasks.workunit.client.1.smithi043.stdout:0/841: truncate d0/d2/df/d1a/d46/d39/f6a 4821618 0 2024-09-16T23:49:05.475 INFO:tasks.workunit.client.0.smithi042.stdout:3/868: creat d2/d9/d102/f12f x:0 0 0 2024-09-16T23:49:05.477 INFO:tasks.workunit.client.0.smithi042.stdout:3/869: stat d2/d1b/d1d/d30/da2/fdd 0 2024-09-16T23:49:05.478 INFO:tasks.workunit.client.0.smithi042.stdout:8/499: mkdir da/d19/d1d/d5a/d99 0 2024-09-16T23:49:05.480 INFO:tasks.workunit.client.0.smithi042.stdout:8/500: dread - da/d19/d2c/f82 zero size 2024-09-16T23:49:05.481 INFO:tasks.workunit.client.1.smithi043.stdout:3/796: mknod d4/de/d1f/d2c/db7/dc9/cf8 0 2024-09-16T23:49:05.481 INFO:tasks.workunit.client.0.smithi042.stdout:5/766: read d9/db/d1b/da4/ffe [1251070,64389] 0 2024-09-16T23:49:05.481 INFO:tasks.workunit.client.1.smithi043.stdout:3/797: stat d4/de/d1f/fdb 0 2024-09-16T23:49:05.483 INFO:tasks.workunit.client.0.smithi042.stdout:8/501: write da/d4b/f87 [210504,38284] 0 2024-09-16T23:49:05.484 INFO:tasks.workunit.client.0.smithi042.stdout:5/767: mkdir d9/db/de/d92/d10d 0 2024-09-16T23:49:05.486 INFO:tasks.workunit.client.1.smithi043.stdout:6/755: mknod de/d23/d34/d9b/d5a/c106 0 2024-09-16T23:49:05.488 INFO:tasks.workunit.client.0.smithi042.stdout:3/870: dread d2/d1b/d89/f45 [0,4194304] 0 2024-09-16T23:49:05.489 INFO:tasks.workunit.client.0.smithi042.stdout:6/636: creat d8/d70/d81/dd/d2c/d57/dc5/fce x:0 0 0 2024-09-16T23:49:05.490 INFO:tasks.workunit.client.0.smithi042.stdout:9/461: write d4/d10/f57 [2606208,46210] 0 2024-09-16T23:49:05.490 INFO:tasks.workunit.client.0.smithi042.stdout:3/871: stat d2/l7d 0 2024-09-16T23:49:05.494 INFO:tasks.workunit.client.0.smithi042.stdout:3/872: write d2/d1b/d1d/d34/f3e [5431083,10388] 0 2024-09-16T23:49:05.495 INFO:tasks.workunit.client.0.smithi042.stdout:5/768: dwrite d9/db/d1b/d31/d46/d58/ff9 [0,4194304] 0 2024-09-16T23:49:05.496 INFO:tasks.workunit.client.1.smithi043.stdout:2/698: unlink d8/d1d/d20/d26/f27 0 2024-09-16T23:49:05.496 INFO:tasks.workunit.client.0.smithi042.stdout:3/873: fsync d2/d1b/f43 0 2024-09-16T23:49:05.498 INFO:tasks.workunit.client.0.smithi042.stdout:8/502: link da/f47 da/d19/d1d/d5a/d99/f9a 0 2024-09-16T23:49:05.498 INFO:tasks.workunit.client.1.smithi043.stdout:1/923: dwrite d4/d9/d1c/d1e/d42/d5d/d71/fe4 [0,4194304] 0 2024-09-16T23:49:05.500 INFO:tasks.workunit.client.1.smithi043.stdout:4/847: write d8/d15/d20/dfa/f82 [3205149,10314] 0 2024-09-16T23:49:05.501 INFO:tasks.workunit.client.0.smithi042.stdout:3/874: unlink d2/d1b/d89/l53 0 2024-09-16T23:49:05.502 INFO:tasks.workunit.client.1.smithi043.stdout:5/905: rename d3/db/c59 to d3/db/d15/d4d/d6b/d73/d12c/c135 0 2024-09-16T23:49:05.503 INFO:tasks.workunit.client.0.smithi042.stdout:9/462: creat d4/d12/d2d/d37/d59/f91 x:0 0 0 2024-09-16T23:49:05.504 INFO:tasks.workunit.client.0.smithi042.stdout:8/503: mkdir da/d19/d1d/d30/d49/d58/d6a/d9b 0 2024-09-16T23:49:05.508 INFO:tasks.workunit.client.1.smithi043.stdout:6/756: stat de/d23/d34/d66/d9d/ld3 0 2024-09-16T23:49:05.509 INFO:tasks.workunit.client.1.smithi043.stdout:3/798: write d4/de/d1f/d2c/db7/dc3/faa [3999898,11163] 0 2024-09-16T23:49:05.509 INFO:tasks.workunit.client.0.smithi042.stdout:3/875: rmdir d2/d1b/d1d/d34/d62/d11a 39 2024-09-16T23:49:05.509 INFO:tasks.workunit.client.0.smithi042.stdout:9/463: chown d4/d10/d24/f3b 147931976 1 2024-09-16T23:49:05.511 INFO:tasks.workunit.client.1.smithi043.stdout:5/906: dwrite d3/d9/d6f/d7c/d80/f11a [0,4194304] 0 2024-09-16T23:49:05.512 INFO:tasks.workunit.client.0.smithi042.stdout:9/464: chown d4/d10/d24/c8a 2166929 1 2024-09-16T23:49:05.512 INFO:tasks.workunit.client.0.smithi042.stdout:5/769: dwrite d9/db/de/d5e/d87/d61/d43/feb [0,4194304] 0 2024-09-16T23:49:05.513 INFO:tasks.workunit.client.0.smithi042.stdout:9/465: readlink d4/d12/l2f 0 2024-09-16T23:49:05.514 INFO:tasks.workunit.client.0.smithi042.stdout:7/699: sync 2024-09-16T23:49:05.514 INFO:tasks.workunit.client.0.smithi042.stdout:4/835: sync 2024-09-16T23:49:05.514 INFO:tasks.workunit.client.0.smithi042.stdout:1/705: sync 2024-09-16T23:49:05.514 INFO:tasks.workunit.client.0.smithi042.stdout:2/581: sync 2024-09-16T23:49:05.516 INFO:tasks.workunit.client.1.smithi043.stdout:2/699: dread - d8/d5e/d83/d9e/fa5 zero size 2024-09-16T23:49:05.525 INFO:tasks.workunit.client.0.smithi042.stdout:1/706: symlink d0/d9/d26/daa/lfe 0 2024-09-16T23:49:05.531 INFO:tasks.workunit.client.0.smithi042.stdout:5/770: write d9/db/de/d5e/d87/d61/f5d [2026509,111919] 0 2024-09-16T23:49:05.534 INFO:tasks.workunit.client.0.smithi042.stdout:2/582: chown d0/d98/lb6 1 1 2024-09-16T23:49:05.538 INFO:tasks.workunit.client.0.smithi042.stdout:1/707: creat d0/d9/d90/d6b/d88/d32/d2f/de3/fff x:0 0 0 2024-09-16T23:49:05.539 INFO:tasks.workunit.client.0.smithi042.stdout:5/771: dread - d9/db/d19/d21/d85/ff1 zero size 2024-09-16T23:49:05.540 INFO:tasks.workunit.client.1.smithi043.stdout:9/735: creat d4/d21/d33/fe8 x:0 0 0 2024-09-16T23:49:05.541 INFO:tasks.workunit.client.0.smithi042.stdout:3/876: dread d2/d1b/d1d/d34/d50/d93/d96/fc6 [0,4194304] 0 2024-09-16T23:49:05.541 INFO:tasks.workunit.client.1.smithi043.stdout:6/757: dread de/d23/d34/d9b/d5a/fc6 [0,4194304] 0 2024-09-16T23:49:05.542 INFO:tasks.workunit.client.0.smithi042.stdout:6/637: write d8/d70/d81/dd/f1f [1667297,44622] 0 2024-09-16T23:49:05.542 INFO:tasks.workunit.client.1.smithi043.stdout:9/736: read d4/d1c/d25/d2b/d4c/f72 [8170956,75135] 0 2024-09-16T23:49:05.543 INFO:tasks.workunit.client.0.smithi042.stdout:8/504: write da/d19/d1d/d5a/f79 [1346227,111302] 0 2024-09-16T23:49:05.544 INFO:tasks.workunit.client.0.smithi042.stdout:7/700: dwrite d3/d1f/d49/d76/d19/d63/fa6 [0,4194304] 0 2024-09-16T23:49:05.547 INFO:tasks.workunit.client.1.smithi043.stdout:1/924: getdents d4/d3e/d5b/dca/d12a 0 2024-09-16T23:49:05.548 INFO:tasks.workunit.client.0.smithi042.stdout:4/836: dwrite de/d2e/d37/d3e/f7f [4194304,4194304] 0 2024-09-16T23:49:05.548 INFO:tasks.workunit.client.0.smithi042.stdout:3/877: write d2/d9/d102/f12f [73348,39214] 0 2024-09-16T23:49:05.551 INFO:tasks.workunit.client.0.smithi042.stdout:7/701: write d3/d3f/f90 [190079,2691] 0 2024-09-16T23:49:05.552 INFO:tasks.workunit.client.1.smithi043.stdout:2/700: truncate d8/d11/d16/f19 402460 0 2024-09-16T23:49:05.553 INFO:tasks.workunit.client.0.smithi042.stdout:1/708: creat d0/db2/f100 x:0 0 0 2024-09-16T23:49:05.554 INFO:tasks.workunit.client.0.smithi042.stdout:8/505: mknod da/d2f/c9c 0 2024-09-16T23:49:05.555 INFO:tasks.workunit.client.1.smithi043.stdout:3/799: unlink d4/de/d40/d4d/cec 0 2024-09-16T23:49:05.556 INFO:tasks.workunit.client.0.smithi042.stdout:6/638: write d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/f2e [4640448,91883] 0 2024-09-16T23:49:05.557 INFO:tasks.workunit.client.1.smithi043.stdout:6/758: creat de/dbe/f107 x:0 0 0 2024-09-16T23:49:05.559 INFO:tasks.workunit.client.1.smithi043.stdout:1/925: dwrite d4/df/d26/dd5/f126 [0,4194304] 0 2024-09-16T23:49:05.560 INFO:tasks.workunit.client.1.smithi043.stdout:9/737: symlink d4/d1c/d4f/d8c/le9 0 2024-09-16T23:49:05.562 INFO:tasks.workunit.client.0.smithi042.stdout:2/583: creat d0/da/d1b/d48/d81/fc4 x:0 0 0 2024-09-16T23:49:05.562 INFO:tasks.workunit.client.0.smithi042.stdout:4/837: rename de/f41 to de/d42/d68/f121 0 2024-09-16T23:49:05.565 INFO:tasks.workunit.client.1.smithi043.stdout:4/848: read d8/d25/f5b [3315023,47554] 0 2024-09-16T23:49:05.565 INFO:tasks.workunit.client.0.smithi042.stdout:5/772: getdents d9/db/d1b/d31/d46/d58/d9d 0 2024-09-16T23:49:05.567 INFO:tasks.workunit.client.0.smithi042.stdout:9/466: dwrite d4/d12/d2d/d37/d41/f49 [0,4194304] 0 2024-09-16T23:49:05.568 INFO:tasks.workunit.client.0.smithi042.stdout:1/709: chown d0/d9/d90/d6b/d88/d32/d2f/c9d 0 1 2024-09-16T23:49:05.568 INFO:tasks.workunit.client.1.smithi043.stdout:9/738: truncate d4/d5/d16/fda 322051 0 2024-09-16T23:49:05.568 INFO:tasks.workunit.client.1.smithi043.stdout:5/907: dwrite d3/da4/fb8 [0,4194304] 0 2024-09-16T23:49:05.571 INFO:tasks.workunit.client.0.smithi042.stdout:6/639: dread d8/d70/d81/f23 [0,4194304] 0 2024-09-16T23:49:05.574 INFO:tasks.workunit.client.1.smithi043.stdout:3/800: truncate d4/de/d40/f69 158848 0 2024-09-16T23:49:05.578 INFO:tasks.workunit.client.0.smithi042.stdout:2/584: creat d0/da/d1b/d48/fc5 x:0 0 0 2024-09-16T23:49:05.578 INFO:tasks.workunit.client.0.smithi042.stdout:3/878: dwrite d2/d1b/d1d/d34/d50/d93/f117 [0,4194304] 0 2024-09-16T23:49:05.581 INFO:tasks.workunit.client.0.smithi042.stdout:3/879: chown d2/d1b/d89/db4 0 1 2024-09-16T23:49:05.583 INFO:tasks.workunit.client.0.smithi042.stdout:9/467: mkdir d4/d12/d2d/d37/d41/d92 0 2024-09-16T23:49:05.585 INFO:tasks.workunit.client.1.smithi043.stdout:1/926: creat d4/d9/d1c/d1e/d132/f13d x:0 0 0 2024-09-16T23:49:05.585 INFO:tasks.workunit.client.1.smithi043.stdout:6/759: write de/d1a/d28/f3e [2819603,125926] 0 2024-09-16T23:49:05.586 INFO:tasks.workunit.client.0.smithi042.stdout:3/880: symlink d2/d7/d61/d8e/l130 0 2024-09-16T23:49:05.586 INFO:tasks.workunit.client.0.smithi042.stdout:7/702: dread d3/d1f/d49/d76/d19/f21 [4194304,4194304] 0 2024-09-16T23:49:05.586 INFO:tasks.workunit.client.0.smithi042.stdout:1/710: creat d0/d9/d26/daa/f101 x:0 0 0 2024-09-16T23:49:05.589 INFO:tasks.workunit.client.1.smithi043.stdout:9/739: creat d4/d1c/d6b/d67/fea x:0 0 0 2024-09-16T23:49:05.589 INFO:tasks.workunit.client.0.smithi042.stdout:9/468: truncate d4/d12/d2d/d37/d28/f87 144601 0 2024-09-16T23:49:05.591 INFO:tasks.workunit.client.0.smithi042.stdout:0/543: sync 2024-09-16T23:49:05.595 INFO:tasks.workunit.client.0.smithi042.stdout:6/640: truncate d8/d70/d81/dd/d1b/d34/f64 2252364 0 2024-09-16T23:49:05.598 INFO:tasks.workunit.client.1.smithi043.stdout:6/760: dwrite de/d23/d34/d9b/fc7 [0,4194304] 0 2024-09-16T23:49:05.598 INFO:tasks.workunit.client.0.smithi042.stdout:0/544: creat d0/df/d23/d3e/faa x:0 0 0 2024-09-16T23:49:05.600 INFO:tasks.workunit.client.1.smithi043.stdout:9/740: dwrite d4/d1c/d4f/d45/d85/fe4 [0,4194304] 0 2024-09-16T23:49:05.602 INFO:tasks.workunit.client.1.smithi043.stdout:2/701: link d8/d4b/d4f/d7c/f88 d8/d1f/ff0 0 2024-09-16T23:49:05.603 INFO:tasks.workunit.client.0.smithi042.stdout:8/506: dwrite da/d4b/f65 [0,4194304] 0 2024-09-16T23:49:05.604 INFO:tasks.workunit.client.1.smithi043.stdout:9/741: fsync d4/d1c/d25/d2b/d4c/fb0 0 2024-09-16T23:49:05.604 INFO:tasks.workunit.client.1.smithi043.stdout:2/702: write d8/d1d/d25/fbd [244252,117514] 0 2024-09-16T23:49:05.607 INFO:tasks.workunit.client.1.smithi043.stdout:4/849: dwrite d8/d15/d20/d2b/d23/d27/faf [0,4194304] 0 2024-09-16T23:49:05.609 INFO:tasks.workunit.client.1.smithi043.stdout:8/740: sync 2024-09-16T23:49:05.612 INFO:tasks.workunit.client.0.smithi042.stdout:4/838: dwrite de/d10d/f98 [0,4194304] 0 2024-09-16T23:49:05.616 INFO:tasks.workunit.client.1.smithi043.stdout:5/908: dwrite d3/db/d19/d3e/f8b [0,4194304] 0 2024-09-16T23:49:05.617 INFO:tasks.workunit.client.0.smithi042.stdout:5/773: dwrite d9/db/d1b/d31/d46/f6d [0,4194304] 0 2024-09-16T23:49:05.619 INFO:tasks.workunit.client.0.smithi042.stdout:6/641: symlink d8/d70/d81/dd/dcd/lcf 0 2024-09-16T23:49:05.625 INFO:tasks.workunit.client.0.smithi042.stdout:8/507: truncate da/f2a 1119407 0 2024-09-16T23:49:05.625 INFO:tasks.workunit.client.0.smithi042.stdout:0/545: getdents d0/df/d13/d2d/d35 0 2024-09-16T23:49:05.626 INFO:tasks.workunit.client.1.smithi043.stdout:6/761: truncate de/f40 2237012 0 2024-09-16T23:49:05.627 INFO:tasks.workunit.client.1.smithi043.stdout:0/842: sync 2024-09-16T23:49:05.627 INFO:tasks.workunit.client.1.smithi043.stdout:7/752: sync 2024-09-16T23:49:05.628 INFO:tasks.workunit.client.0.smithi042.stdout:3/881: dread d2/d1b/d1d/d34/f6b [0,4194304] 0 2024-09-16T23:49:05.630 INFO:tasks.workunit.client.0.smithi042.stdout:2/585: rename d0/d78/d8b/f99 to d0/fc6 0 2024-09-16T23:49:05.632 INFO:tasks.workunit.client.0.smithi042.stdout:4/839: dwrite de/d1a/d70/fbe [0,4194304] 0 2024-09-16T23:49:05.632 INFO:tasks.workunit.client.0.smithi042.stdout:1/711: unlink d0/d9/d90/fad 0 2024-09-16T23:49:05.633 INFO:tasks.workunit.client.0.smithi042.stdout:5/774: mkdir d9/db/de/d5e/d10e 0 2024-09-16T23:49:05.636 INFO:tasks.workunit.client.0.smithi042.stdout:3/882: read d2/d1b/fb2 [133971,94248] 0 2024-09-16T23:49:05.638 INFO:tasks.workunit.client.1.smithi043.stdout:9/742: rename d4/d21/d35/l51 to d4/d84/leb 0 2024-09-16T23:49:05.639 INFO:tasks.workunit.client.1.smithi043.stdout:9/743: write d4/d5/d16/fda [787878,5724] 0 2024-09-16T23:49:05.641 INFO:tasks.workunit.client.0.smithi042.stdout:1/712: write d0/d17/f9e [242325,64415] 0 2024-09-16T23:49:05.641 INFO:tasks.workunit.client.0.smithi042.stdout:7/703: rename d3/d1f/d49/d76/d19/c7b to d3/d1f/d49/d78/dbe/dc8/cf4 0 2024-09-16T23:49:05.644 INFO:tasks.workunit.client.0.smithi042.stdout:0/546: creat d0/df/d13/d2d/d35/d9f/fab x:0 0 0 2024-09-16T23:49:05.645 INFO:tasks.workunit.client.0.smithi042.stdout:4/840: symlink de/d1a/d70/l122 0 2024-09-16T23:49:05.647 INFO:tasks.workunit.client.1.smithi043.stdout:2/703: truncate d8/d5e/d83/d3a/f57 90829 0 2024-09-16T23:49:05.648 INFO:tasks.workunit.client.0.smithi042.stdout:3/883: rename d2/d1b/f43 to d2/d7/d61/d8e/f131 0 2024-09-16T23:49:05.648 INFO:tasks.workunit.client.0.smithi042.stdout:2/586: chown d0/da/d88/cab 1 1 2024-09-16T23:49:05.651 INFO:tasks.workunit.client.1.smithi043.stdout:3/801: truncate d4/de/d40/d4d/fbd 3484017 0 2024-09-16T23:49:05.651 INFO:tasks.workunit.client.0.smithi042.stdout:7/704: dread d3/f30 [0,4194304] 0 2024-09-16T23:49:05.652 INFO:tasks.workunit.client.1.smithi043.stdout:3/802: chown d4/de/d40/df0/f5c 31788236 1 2024-09-16T23:49:05.652 INFO:tasks.workunit.client.0.smithi042.stdout:5/775: getdents d9/db/d19 0 2024-09-16T23:49:05.653 INFO:tasks.workunit.client.0.smithi042.stdout:0/547: read d0/df/d23/d33/d56/f5d [284655,81534] 0 2024-09-16T23:49:05.655 INFO:tasks.workunit.client.0.smithi042.stdout:5/776: rmdir d9/d63 39 2024-09-16T23:49:05.656 INFO:tasks.workunit.client.0.smithi042.stdout:9/469: getdents d4/d12 0 2024-09-16T23:49:05.661 INFO:tasks.workunit.client.1.smithi043.stdout:5/909: fsync d3/db/d15/f5f 0 2024-09-16T23:49:05.662 INFO:tasks.workunit.client.0.smithi042.stdout:8/508: creat da/d19/d2c/d56/d97/f9d x:0 0 0 2024-09-16T23:49:05.663 INFO:tasks.workunit.client.1.smithi043.stdout:6/762: truncate de/f24 8414460 0 2024-09-16T23:49:05.665 INFO:tasks.workunit.client.0.smithi042.stdout:1/713: fdatasync d0/d3b/fe0 0 2024-09-16T23:49:05.666 INFO:tasks.workunit.client.0.smithi042.stdout:3/884: write d2/d1b/d89/f3c [925402,31006] 0 2024-09-16T23:49:05.668 INFO:tasks.workunit.client.0.smithi042.stdout:0/548: write d0/d15/f2f [2466000,79326] 0 2024-09-16T23:49:05.669 INFO:tasks.workunit.client.0.smithi042.stdout:2/587: creat d0/da/d88/fc7 x:0 0 0 2024-09-16T23:49:05.674 INFO:tasks.workunit.client.0.smithi042.stdout:6/642: mkdir d8/d70/d81/dd/d1b/d50/d96/da0/dd0 0 2024-09-16T23:49:05.674 INFO:tasks.workunit.client.0.smithi042.stdout:3/885: creat d2/d1b/d1d/d34/d50/d93/d96/f132 x:0 0 0 2024-09-16T23:49:05.675 INFO:tasks.workunit.client.0.smithi042.stdout:9/470: mknod d4/d12/d2d/d37/d28/c93 0 2024-09-16T23:49:05.676 INFO:tasks.workunit.client.0.smithi042.stdout:4/841: truncate de/d12/f14 3581783 0 2024-09-16T23:49:05.678 INFO:tasks.workunit.client.0.smithi042.stdout:8/509: mknod da/d4b/c9e 0 2024-09-16T23:49:05.680 INFO:tasks.workunit.client.0.smithi042.stdout:5/777: dwrite d9/db/f25 [4194304,4194304] 0 2024-09-16T23:49:05.682 INFO:tasks.workunit.client.0.smithi042.stdout:3/886: write d2/d1b/dc8/ff6 [2176975,94185] 0 2024-09-16T23:49:05.682 INFO:tasks.workunit.client.0.smithi042.stdout:4/842: chown de/c18 193 1 2024-09-16T23:49:05.683 INFO:tasks.workunit.client.0.smithi042.stdout:8/510: dread - da/d19/d2c/d56/f98 zero size 2024-09-16T23:49:05.684 INFO:tasks.workunit.client.1.smithi043.stdout:4/850: mknod d8/d25/d39/d41/d57/d85/c107 0 2024-09-16T23:49:05.687 INFO:tasks.workunit.client.0.smithi042.stdout:3/887: write d2/d7/d61/fc5 [526722,22332] 0 2024-09-16T23:49:05.687 INFO:tasks.workunit.client.0.smithi042.stdout:7/705: dwrite d3/d98/da0/d41/f47 [0,4194304] 0 2024-09-16T23:49:05.690 INFO:tasks.workunit.client.1.smithi043.stdout:8/741: fdatasync de/d18/d27/d41/d5a/f13 0 2024-09-16T23:49:05.691 INFO:tasks.workunit.client.0.smithi042.stdout:4/843: rmdir de/d1a/da7/db9 39 2024-09-16T23:49:05.692 INFO:tasks.workunit.client.0.smithi042.stdout:4/844: stat de/d2e/d37/d4c/c7b 0 2024-09-16T23:49:05.693 INFO:tasks.workunit.client.0.smithi042.stdout:5/778: rmdir d9/db/de/d10/d2f 39 2024-09-16T23:49:05.694 INFO:tasks.workunit.client.0.smithi042.stdout:2/588: creat d0/d98/fc8 x:0 0 0 2024-09-16T23:49:05.694 INFO:tasks.workunit.client.0.smithi042.stdout:4/845: dread - de/d10d/f10e zero size 2024-09-16T23:49:05.694 INFO:tasks.workunit.client.0.smithi042.stdout:7/706: symlink d3/d98/lf5 0 2024-09-16T23:49:05.695 INFO:tasks.workunit.client.1.smithi043.stdout:3/803: fdatasync d4/de/d40/fb6 0 2024-09-16T23:49:05.697 INFO:tasks.workunit.client.1.smithi043.stdout:3/804: write d4/de/d1f/d2c/f32 [5152487,79220] 0 2024-09-16T23:49:05.698 INFO:tasks.workunit.client.0.smithi042.stdout:3/888: dwrite d2/d1b/d89/f42 [0,4194304] 0 2024-09-16T23:49:05.699 INFO:tasks.workunit.client.0.smithi042.stdout:9/471: mkdir d4/d12/d2d/d94 0 2024-09-16T23:49:05.700 INFO:tasks.workunit.client.1.smithi043.stdout:5/910: truncate d3/db/d15/f69 210172 0 2024-09-16T23:49:05.700 INFO:tasks.workunit.client.0.smithi042.stdout:2/589: fsync d0/da/d1b/d48/f58 0 2024-09-16T23:49:05.700 INFO:tasks.workunit.client.0.smithi042.stdout:4/846: read - de/d42/d99/d9e/fa4 zero size 2024-09-16T23:49:05.701 INFO:tasks.workunit.client.0.smithi042.stdout:9/472: fdatasync d4/d12/d16/d68/f74 0 2024-09-16T23:49:05.701 INFO:tasks.workunit.client.0.smithi042.stdout:7/707: symlink d3/d98/da0/lf6 0 2024-09-16T23:49:05.706 INFO:tasks.workunit.client.0.smithi042.stdout:5/779: creat d9/db/d1b/d31/d46/d107/f10f x:0 0 0 2024-09-16T23:49:05.707 INFO:tasks.workunit.client.0.smithi042.stdout:1/714: symlink d0/d9/d90/d6b/l102 0 2024-09-16T23:49:05.708 INFO:tasks.workunit.client.0.smithi042.stdout:4/847: mkdir de/d42/d5d/d8f/dd6/d123 0 2024-09-16T23:49:05.710 INFO:tasks.workunit.client.1.smithi043.stdout:0/843: link d0/d2/df/d1a/d3b/fda d0/d2/df/da7/db8/f118 0 2024-09-16T23:49:05.710 INFO:tasks.workunit.client.1.smithi043.stdout:6/763: dread de/d1a/d39/f17 [0,4194304] 0 2024-09-16T23:49:05.710 INFO:tasks.workunit.client.0.smithi042.stdout:7/708: creat d3/d1f/d49/d78/ff7 x:0 0 0 2024-09-16T23:49:05.711 INFO:tasks.workunit.client.0.smithi042.stdout:8/511: unlink da/d2f/f77 0 2024-09-16T23:49:05.711 INFO:tasks.workunit.client.0.smithi042.stdout:3/889: mknod d2/d7/d61/d8e/d9d/c133 0 2024-09-16T23:49:05.711 INFO:tasks.workunit.client.0.smithi042.stdout:4/848: write de/d10d/f98 [745040,67799] 0 2024-09-16T23:49:05.713 INFO:tasks.workunit.client.1.smithi043.stdout:1/927: getdents d4/d9/d1c/d1e/d42/d5d/d71/d76 0 2024-09-16T23:49:05.714 INFO:tasks.workunit.client.0.smithi042.stdout:7/709: dread - d3/d1f/d49/d76/d19/fea zero size 2024-09-16T23:49:05.714 INFO:tasks.workunit.client.0.smithi042.stdout:4/849: chown c1 57 1 2024-09-16T23:49:05.715 INFO:tasks.workunit.client.0.smithi042.stdout:5/780: mkdir d9/db/d1b/d110 0 2024-09-16T23:49:05.716 INFO:tasks.workunit.client.0.smithi042.stdout:6/643: mknod d8/d70/d81/dd/d1b/d34/d39/db4/cd1 0 2024-09-16T23:49:05.716 INFO:tasks.workunit.client.0.smithi042.stdout:4/850: dread - de/d2e/d37/f11c zero size 2024-09-16T23:49:05.716 INFO:tasks.workunit.client.1.smithi043.stdout:4/851: write d8/d15/d20/d2b/f28 [9441111,69966] 0 2024-09-16T23:49:05.718 INFO:tasks.workunit.client.1.smithi043.stdout:6/764: dwrite fd [0,4194304] 0 2024-09-16T23:49:05.721 INFO:tasks.workunit.client.1.smithi043.stdout:1/928: dread d4/d9/f84 [0,4194304] 0 2024-09-16T23:49:05.722 INFO:tasks.workunit.client.0.smithi042.stdout:9/473: dread d4/d10/f3f [0,4194304] 0 2024-09-16T23:49:05.723 INFO:tasks.workunit.client.0.smithi042.stdout:7/710: creat d3/d1f/ff8 x:0 0 0 2024-09-16T23:49:05.724 INFO:tasks.workunit.client.0.smithi042.stdout:5/781: truncate d9/db/d19/d21/d41/d7d/f7e 3005698 0 2024-09-16T23:49:05.728 INFO:tasks.workunit.client.0.smithi042.stdout:5/782: fsync d9/db/d1b/da4/ffe 0 2024-09-16T23:49:05.729 INFO:tasks.workunit.client.0.smithi042.stdout:5/783: readlink d9/db/de/d5e/d87/d61/l1f 0 2024-09-16T23:49:05.730 INFO:tasks.workunit.client.1.smithi043.stdout:6/765: dread de/d23/dc3/fd1 [0,4194304] 0 2024-09-16T23:49:05.730 INFO:tasks.workunit.client.1.smithi043.stdout:1/929: read d4/d9/f46 [76200,68572] 0 2024-09-16T23:49:05.731 INFO:tasks.workunit.client.1.smithi043.stdout:3/805: mknod d4/de/d40/df0/cf9 0 2024-09-16T23:49:05.732 INFO:tasks.workunit.client.1.smithi043.stdout:1/930: write d4/f69 [3587594,94753] 0 2024-09-16T23:49:05.732 INFO:tasks.workunit.client.0.smithi042.stdout:7/711: write d3/d1f/d49/d76/d19/d63/f69 [4792083,66390] 0 2024-09-16T23:49:05.734 INFO:tasks.workunit.client.0.smithi042.stdout:5/784: mknod d9/db/de/d92/d10d/c111 0 2024-09-16T23:49:05.735 INFO:tasks.workunit.client.1.smithi043.stdout:3/806: dread d4/de/d1f/d2c/db7/dc3/fe0 [0,4194304] 0 2024-09-16T23:49:05.735 INFO:tasks.workunit.client.1.smithi043.stdout:5/911: mkdir d3/db/d84/d136 0 2024-09-16T23:49:05.735 INFO:tasks.workunit.client.0.smithi042.stdout:5/785: dread - d9/db/d1b/d31/d6c/f102 zero size 2024-09-16T23:49:05.736 INFO:tasks.workunit.client.1.smithi043.stdout:5/912: readlink d3/da4/l10f 0 2024-09-16T23:49:05.736 INFO:tasks.workunit.client.0.smithi042.stdout:7/712: chown d3/d1f/d49/d76/d19/ddc/d6b/d64/l95 757244334 1 2024-09-16T23:49:05.737 INFO:tasks.workunit.client.0.smithi042.stdout:4/851: dread de/d2e/d37/d4c/d4f/f6b [8388608,4194304] 0 2024-09-16T23:49:05.737 INFO:tasks.workunit.client.0.smithi042.stdout:5/786: stat d9/db/de/d5e/d87/d61 0 2024-09-16T23:49:05.738 INFO:tasks.workunit.client.1.smithi043.stdout:7/753: creat d4/db/d43/d83/ffa x:0 0 0 2024-09-16T23:49:05.739 INFO:tasks.workunit.client.0.smithi042.stdout:4/852: write de/d10d/f78 [5862690,86595] 0 2024-09-16T23:49:05.739 INFO:tasks.workunit.client.0.smithi042.stdout:7/713: symlink d3/d1f/d49/d76/d19/ddc/d6b/lf9 0 2024-09-16T23:49:05.741 INFO:tasks.workunit.client.0.smithi042.stdout:4/853: fdatasync de/d12/f118 0 2024-09-16T23:49:05.742 INFO:tasks.workunit.client.0.smithi042.stdout:6/644: fsync d8/d38/f74 0 2024-09-16T23:49:05.743 INFO:tasks.workunit.client.1.smithi043.stdout:2/704: truncate d8/d11/d16/f81 268515 0 2024-09-16T23:49:05.744 INFO:tasks.workunit.client.0.smithi042.stdout:1/715: link d0/d9/d90/d6b/d88/f30 d0/d9/d26/d8b/f103 0 2024-09-16T23:49:05.745 INFO:tasks.workunit.client.0.smithi042.stdout:2/590: write d0/d26/f84 [252774,110853] 0 2024-09-16T23:49:05.746 INFO:tasks.workunit.client.0.smithi042.stdout:4/854: unlink de/cf 0 2024-09-16T23:49:05.749 INFO:tasks.workunit.client.0.smithi042.stdout:7/714: rename d3/d98/l5a to d3/d1f/d49/d78/dbe/dc8/lfa 0 2024-09-16T23:49:05.750 INFO:tasks.workunit.client.0.smithi042.stdout:4/855: mkdir de/d42/d82/d94/d124 0 2024-09-16T23:49:05.751 INFO:tasks.workunit.client.1.smithi043.stdout:2/705: dwrite d8/d1d/d20/d26/f3c [0,4194304] 0 2024-09-16T23:49:05.752 INFO:tasks.workunit.client.1.smithi043.stdout:2/706: write d8/d5e/fac [267370,7879] 0 2024-09-16T23:49:05.755 INFO:tasks.workunit.client.0.smithi042.stdout:8/512: truncate da/d4b/f65 3776394 0 2024-09-16T23:49:05.755 INFO:tasks.workunit.client.1.smithi043.stdout:7/754: dread d4/f2b [0,4194304] 0 2024-09-16T23:49:05.756 INFO:tasks.workunit.client.0.smithi042.stdout:7/715: mkdir d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/dfb 0 2024-09-16T23:49:05.756 INFO:tasks.workunit.client.0.smithi042.stdout:2/591: dwrite d0/d26/f2f [0,4194304] 0 2024-09-16T23:49:05.758 INFO:tasks.workunit.client.1.smithi043.stdout:7/755: readlink d4/db/d11/d21/d31/d65/l98 0 2024-09-16T23:49:05.761 INFO:tasks.workunit.client.0.smithi042.stdout:5/787: write d9/db/d1b/d31/f7a [950213,128078] 0 2024-09-16T23:49:05.762 INFO:tasks.workunit.client.1.smithi043.stdout:4/852: rename d8/d15/d20/d2b/d23/d27 to d8/d25/d39/dd3/d100/d108 0 2024-09-16T23:49:05.764 INFO:tasks.workunit.client.0.smithi042.stdout:7/716: dread - d3/d1f/d49/d76/d19/ddc/d6b/d5b/f7a zero size 2024-09-16T23:49:05.766 INFO:tasks.workunit.client.0.smithi042.stdout:4/856: dwrite de/d42/d82/f115 [0,4194304] 0 2024-09-16T23:49:05.767 INFO:tasks.workunit.client.0.smithi042.stdout:9/474: truncate d4/d12/d2d/f32 201083 0 2024-09-16T23:49:05.768 INFO:tasks.workunit.client.0.smithi042.stdout:6/645: rename d8/d70/d84/lc3 to d8/d70/ld2 0 2024-09-16T23:49:05.771 INFO:tasks.workunit.client.0.smithi042.stdout:1/716: write d0/f95 [263035,5376] 0 2024-09-16T23:49:05.773 INFO:tasks.workunit.client.0.smithi042.stdout:4/857: fdatasync de/d12/f97 0 2024-09-16T23:49:05.776 INFO:tasks.workunit.client.0.smithi042.stdout:2/592: stat d0/f22 0 2024-09-16T23:49:05.780 INFO:tasks.workunit.client.0.smithi042.stdout:7/717: truncate d3/d1f/d49/d76/d19/ddc/f4a 466301 0 2024-09-16T23:49:05.783 INFO:tasks.workunit.client.0.smithi042.stdout:9/475: fdatasync d4/d12/d2d/d37/d3a/f3d 0 2024-09-16T23:49:05.784 INFO:tasks.workunit.client.1.smithi043.stdout:0/844: mknod d0/d2/c119 0 2024-09-16T23:49:05.790 INFO:tasks.workunit.client.1.smithi043.stdout:9/744: getdents d4/d5/d16 0 2024-09-16T23:49:05.791 INFO:tasks.workunit.client.0.smithi042.stdout:5/788: dwrite d9/db/de/d92/daf/fb4 [0,4194304] 0 2024-09-16T23:49:05.791 INFO:tasks.workunit.client.0.smithi042.stdout:6/646: read d8/d70/d81/dd/d1b/d34/f64 [1991113,35718] 0 2024-09-16T23:49:05.792 INFO:tasks.workunit.client.1.smithi043.stdout:8/742: dwrite de/d18/d27/d41/d5a/d37/f69 [4194304,4194304] 0 2024-09-16T23:49:05.793 INFO:tasks.workunit.client.1.smithi043.stdout:8/743: chown de/d18/d1f/d25/lb7 1722 1 2024-09-16T23:49:05.795 INFO:tasks.workunit.client.0.smithi042.stdout:7/718: truncate d3/d1f/d49/d78/dbe/fc7 1833316 0 2024-09-16T23:49:05.795 INFO:tasks.workunit.client.0.smithi042.stdout:8/513: dread da/d19/f3b [4194304,4194304] 0 2024-09-16T23:49:05.795 INFO:tasks.workunit.client.1.smithi043.stdout:5/913: write d3/d9/d22/d4a/d64/d8f/da1/ff3 [863936,63390] 0 2024-09-16T23:49:05.807 INFO:tasks.workunit.client.0.smithi042.stdout:9/476: unlink d4/d12/l2f 0 2024-09-16T23:49:05.807 INFO:tasks.workunit.client.0.smithi042.stdout:4/858: dread de/f31 [0,4194304] 0 2024-09-16T23:49:05.809 INFO:tasks.workunit.client.0.smithi042.stdout:5/789: dwrite d9/db/de/d5e/d87/d61/d43/feb [0,4194304] 0 2024-09-16T23:49:05.809 INFO:tasks.workunit.client.1.smithi043.stdout:4/853: write d8/d15/d20/d2b/d23/fd1 [3467207,60629] 0 2024-09-16T23:49:05.810 INFO:tasks.workunit.client.0.smithi042.stdout:1/717: dwrite d0/fa1 [0,4194304] 0 2024-09-16T23:49:05.812 INFO:tasks.workunit.client.0.smithi042.stdout:4/859: rmdir de/d42/d99 39 2024-09-16T23:49:05.813 INFO:tasks.workunit.client.0.smithi042.stdout:8/514: symlink da/d19/d1d/d5a/l9f 0 2024-09-16T23:49:05.815 INFO:tasks.workunit.client.0.smithi042.stdout:4/860: write de/d2e/d37/f3b [3501482,11243] 0 2024-09-16T23:49:05.815 INFO:tasks.workunit.client.0.smithi042.stdout:8/515: chown da/d19/d80 167387678 1 2024-09-16T23:49:05.815 INFO:tasks.workunit.client.0.smithi042.stdout:8/516: chown da/f90 15 1 2024-09-16T23:49:05.816 INFO:tasks.workunit.client.0.smithi042.stdout:2/593: mknod d0/d8d/da0/cc9 0 2024-09-16T23:49:05.819 INFO:tasks.workunit.client.0.smithi042.stdout:5/790: rename d9/db/de/d5e/d87 to d9/db/d19/d21/d85/dec/d112 0 2024-09-16T23:49:05.820 INFO:tasks.workunit.client.0.smithi042.stdout:6/647: symlink d8/d70/d81/dd/d1b/d50/ld3 0 2024-09-16T23:49:05.822 INFO:tasks.workunit.client.0.smithi042.stdout:4/861: symlink de/d2e/d37/d4c/dd4/l125 0 2024-09-16T23:49:05.824 INFO:tasks.workunit.client.0.smithi042.stdout:5/791: write d9/db/de/d10/f20 [6949585,115799] 0 2024-09-16T23:49:05.825 INFO:tasks.workunit.client.0.smithi042.stdout:2/594: truncate d0/da/d88/fa4 509296 0 2024-09-16T23:49:05.826 INFO:tasks.workunit.client.1.smithi043.stdout:9/745: readlink d4/d5/l38 0 2024-09-16T23:49:05.826 INFO:tasks.workunit.client.1.smithi043.stdout:8/744: stat de/d18/d1f/d2f/ldf 0 2024-09-16T23:49:05.829 INFO:tasks.workunit.client.0.smithi042.stdout:5/792: stat d9/db/de/d5e/lbe 0 2024-09-16T23:49:05.830 INFO:tasks.workunit.client.0.smithi042.stdout:8/517: rmdir da 39 2024-09-16T23:49:05.833 INFO:tasks.workunit.client.0.smithi042.stdout:8/518: write da/d19/d1d/d5a/f79 [1901678,108739] 0 2024-09-16T23:49:05.833 INFO:tasks.workunit.client.1.smithi043.stdout:6/766: truncate de/d1a/d28/f3e 129994 0 2024-09-16T23:49:05.833 INFO:tasks.workunit.client.1.smithi043.stdout:9/746: dwrite d4/d1c/d6b/d67/fea [0,4194304] 0 2024-09-16T23:49:05.834 INFO:tasks.workunit.client.1.smithi043.stdout:6/767: readlink de/d23/l2c 0 2024-09-16T23:49:05.834 INFO:tasks.workunit.client.0.smithi042.stdout:2/595: fsync d0/da/d17/f52 0 2024-09-16T23:49:05.835 INFO:tasks.workunit.client.1.smithi043.stdout:6/768: fsync de/d2a/f9e 0 2024-09-16T23:49:05.835 INFO:tasks.workunit.client.1.smithi043.stdout:9/747: stat d4/d1c/f20 0 2024-09-16T23:49:05.837 INFO:tasks.workunit.client.0.smithi042.stdout:9/477: dread d4/d12/d2d/d37/d3a/f3d [0,4194304] 0 2024-09-16T23:49:05.838 INFO:tasks.workunit.client.1.smithi043.stdout:9/748: fsync d4/d1c/d25/d2b/fa5 0 2024-09-16T23:49:05.838 INFO:tasks.workunit.client.1.smithi043.stdout:6/769: write de/d1a/d80/fce [706870,28139] 0 2024-09-16T23:49:05.838 INFO:tasks.workunit.client.0.smithi042.stdout:5/793: symlink d9/db/d19/d21/d85/dec/d112/d61/d43/l113 0 2024-09-16T23:49:05.839 INFO:tasks.workunit.client.1.smithi043.stdout:7/756: link d4/d27/dbc/fdb d4/db/d11/d21/d94/ffb 0 2024-09-16T23:49:05.839 INFO:tasks.workunit.client.1.smithi043.stdout:9/749: stat d4/d1c/f20 0 2024-09-16T23:49:05.839 INFO:tasks.workunit.client.0.smithi042.stdout:4/862: link de/d12/f15 de/d2e/d37/d3e/f126 0 2024-09-16T23:49:05.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/648: unlink d8/d70/d81/dd/d1b/d67/d97/f9a 0 2024-09-16T23:49:05.841 INFO:tasks.workunit.client.0.smithi042.stdout:9/478: dread d4/d12/d2d/d37/d3a/d50/f7d [0,4194304] 0 2024-09-16T23:49:05.843 INFO:tasks.workunit.client.1.smithi043.stdout:3/807: rename d4/de/d40/d4d/c9f to d4/de/d40/df0/d88/db1/cfa 0 2024-09-16T23:49:05.844 INFO:tasks.workunit.client.0.smithi042.stdout:4/863: truncate de/d12/d7d/f83 371287 0 2024-09-16T23:49:05.845 INFO:tasks.workunit.client.1.smithi043.stdout:4/854: rmdir d8/d25/d39/d41/d57 39 2024-09-16T23:49:05.847 INFO:tasks.workunit.client.0.smithi042.stdout:4/864: creat de/d42/d82/d94/dbd/f127 x:0 0 0 2024-09-16T23:49:05.850 INFO:tasks.workunit.client.0.smithi042.stdout:4/865: mkdir de/d10d/d128 0 2024-09-16T23:49:05.851 INFO:tasks.workunit.client.1.smithi043.stdout:5/914: dwrite d3/db/f2a [4194304,4194304] 0 2024-09-16T23:49:05.852 INFO:tasks.workunit.client.0.smithi042.stdout:5/794: write d9/db/d19/d21/f2b [3821272,101183] 0 2024-09-16T23:49:05.853 INFO:tasks.workunit.client.0.smithi042.stdout:2/596: mknod d0/da/d1b/d48/d81/cca 0 2024-09-16T23:49:05.855 INFO:tasks.workunit.client.1.smithi043.stdout:4/855: dwrite d8/d25/d39/d58/f4a [4194304,4194304] 0 2024-09-16T23:49:05.855 INFO:tasks.workunit.client.0.smithi042.stdout:4/866: truncate de/d42/d68/f96 604970 0 2024-09-16T23:49:05.856 INFO:tasks.workunit.client.1.smithi043.stdout:4/856: chown d8/d15/d9b/cdf 1 1 2024-09-16T23:49:05.859 INFO:tasks.workunit.client.0.smithi042.stdout:4/867: rmdir de/d10d/d8a/db2 39 2024-09-16T23:49:05.861 INFO:tasks.workunit.client.0.smithi042.stdout:1/718: getdents d0/d3b 0 2024-09-16T23:49:05.864 INFO:tasks.workunit.client.0.smithi042.stdout:4/868: write de/d1a/da7/f56 [4817654,12412] 0 2024-09-16T23:49:05.869 INFO:tasks.workunit.client.0.smithi042.stdout:4/869: fdatasync de/d10d/d8a/db2/fdf 0 2024-09-16T23:49:05.870 INFO:tasks.workunit.client.0.smithi042.stdout:8/519: dwrite da/d19/f1a [0,4194304] 0 2024-09-16T23:49:05.871 INFO:tasks.workunit.client.0.smithi042.stdout:5/795: truncate d9/db/d1b/d31/d46/f6d 1323818 0 2024-09-16T23:49:05.873 INFO:tasks.workunit.client.0.smithi042.stdout:5/796: fdatasync d9/db/de/f6f 0 2024-09-16T23:49:05.876 INFO:tasks.workunit.client.0.smithi042.stdout:1/719: rename d0/d9/f11 to d0/d9/d90/d6b/d88/def/f104 0 2024-09-16T23:49:05.877 INFO:tasks.workunit.client.0.smithi042.stdout:2/597: dwrite d0/da/d17/f1c [0,4194304] 0 2024-09-16T23:49:05.877 INFO:tasks.workunit.client.0.smithi042.stdout:5/797: mkdir d9/d2c/d7b/d114 0 2024-09-16T23:49:05.879 INFO:tasks.workunit.client.0.smithi042.stdout:5/798: stat d9/db/de/d10/f22 0 2024-09-16T23:49:05.881 INFO:tasks.workunit.client.1.smithi043.stdout:5/915: dread d3/d9/d6f/d85/f98 [0,4194304] 0 2024-09-16T23:49:05.881 INFO:tasks.workunit.client.0.smithi042.stdout:8/520: mknod da/d19/d2c/d56/d97/ca0 0 2024-09-16T23:49:05.882 INFO:tasks.workunit.client.1.smithi043.stdout:5/916: chown d3/db/d19/f54 127024 1 2024-09-16T23:49:05.884 INFO:tasks.workunit.client.0.smithi042.stdout:9/479: dwrite d4/d10/f15 [0,4194304] 0 2024-09-16T23:49:05.886 INFO:tasks.workunit.client.0.smithi042.stdout:4/870: getdents de/d42/d5d/d8f/d9f/dc0 0 2024-09-16T23:49:05.888 INFO:tasks.workunit.client.0.smithi042.stdout:4/871: dread de/d2e/fed [0,4194304] 0 2024-09-16T23:49:05.890 INFO:tasks.workunit.client.0.smithi042.stdout:8/521: dread - da/d19/d1d/f63 zero size 2024-09-16T23:49:05.896 INFO:tasks.workunit.client.0.smithi042.stdout:5/799: link d9/d63/cd4 d9/db/d1b/d31/d6c/c115 0 2024-09-16T23:49:05.897 INFO:tasks.workunit.client.0.smithi042.stdout:4/872: symlink de/d42/d99/d9e/d109/l129 0 2024-09-16T23:49:05.898 INFO:tasks.workunit.client.0.smithi042.stdout:6/649: link d8/d70/d81/dd/d1b/d50/da2/lcb d8/d70/d81/dd/d2c/d57/dc5/ld4 0 2024-09-16T23:49:05.900 INFO:tasks.workunit.client.0.smithi042.stdout:5/800: creat d9/db/d19/d21/d41/d7d/f116 x:0 0 0 2024-09-16T23:49:05.903 INFO:tasks.workunit.client.0.smithi042.stdout:4/873: read de/d42/d82/d94/fc9 [3795878,110869] 0 2024-09-16T23:49:05.903 INFO:tasks.workunit.client.0.smithi042.stdout:8/522: dwrite da/d19/d1d/d30/d49/d58/d6a/f8b [0,4194304] 0 2024-09-16T23:49:05.904 INFO:tasks.workunit.client.0.smithi042.stdout:2/598: mkdir d0/d30/dcb 0 2024-09-16T23:49:05.905 INFO:tasks.workunit.client.0.smithi042.stdout:5/801: symlink d9/db/d1b/d31/d46/l117 0 2024-09-16T23:49:05.906 INFO:tasks.workunit.client.0.smithi042.stdout:8/523: read - da/d19/d2c/f82 zero size 2024-09-16T23:49:05.907 INFO:tasks.workunit.client.0.smithi042.stdout:4/874: creat de/d42/d82/d94/d124/f12a x:0 0 0 2024-09-16T23:49:05.915 INFO:tasks.workunit.client.0.smithi042.stdout:1/720: write d0/d9a/fb7 [131582,81606] 0 2024-09-16T23:49:05.921 INFO:tasks.workunit.client.0.smithi042.stdout:8/524: unlink da/d19/l51 0 2024-09-16T23:49:05.923 INFO:tasks.workunit.client.0.smithi042.stdout:1/721: unlink d0/d9/d90/d6b/d88/c5c 0 2024-09-16T23:49:05.925 INFO:tasks.workunit.client.0.smithi042.stdout:1/722: read d0/d3b/d8d/fa3 [348450,33210] 0 2024-09-16T23:49:05.926 INFO:tasks.workunit.client.0.smithi042.stdout:2/599: creat d0/da/fcc x:0 0 0 2024-09-16T23:49:05.927 INFO:tasks.workunit.client.0.smithi042.stdout:5/802: truncate d9/db/d19/d21/d85/fd1 1873869 0 2024-09-16T23:49:05.930 INFO:tasks.workunit.client.0.smithi042.stdout:6/650: dread d8/d70/d81/f1d [0,4194304] 0 2024-09-16T23:49:05.932 INFO:tasks.workunit.client.0.smithi042.stdout:8/525: dwrite da/d4b/f87 [0,4194304] 0 2024-09-16T23:49:05.937 INFO:tasks.workunit.client.0.smithi042.stdout:1/723: creat d0/d9/d90/d66/f105 x:0 0 0 2024-09-16T23:49:05.942 INFO:tasks.workunit.client.0.smithi042.stdout:4/875: dwrite de/d2e/d37/f89 [0,4194304] 0 2024-09-16T23:49:05.949 INFO:tasks.workunit.client.0.smithi042.stdout:2/600: truncate d0/da/d17/d3e/f76 794903 0 2024-09-16T23:49:05.949 INFO:tasks.workunit.client.0.smithi042.stdout:6/651: symlink d8/d70/d81/dd/d1b/dbf/ld5 0 2024-09-16T23:49:05.949 INFO:tasks.workunit.client.0.smithi042.stdout:4/876: dread - de/d12/d7d/f11b zero size 2024-09-16T23:49:05.949 INFO:tasks.workunit.client.0.smithi042.stdout:9/480: dwrite d4/d12/d2d/f32 [0,4194304] 0 2024-09-16T23:49:05.949 INFO:tasks.workunit.client.0.smithi042.stdout:2/601: dread - d0/da/d1b/d48/d81/fb5 zero size 2024-09-16T23:49:05.952 INFO:tasks.workunit.client.0.smithi042.stdout:4/877: read - de/d12/d7d/f11b zero size 2024-09-16T23:49:05.953 INFO:tasks.workunit.client.0.smithi042.stdout:1/724: rename d0/d9/d26/d8b/faf to d0/d9/d90/d6b/d88/def/f106 0 2024-09-16T23:49:05.955 INFO:tasks.workunit.client.0.smithi042.stdout:6/652: rename d8/d70/d81/dd/d2c to d8/d70/d81/dd/d2c/d57/dd6 22 2024-09-16T23:49:05.955 INFO:tasks.workunit.client.0.smithi042.stdout:2/602: chown d0/c1 285 1 2024-09-16T23:49:05.955 INFO:tasks.workunit.client.0.smithi042.stdout:1/725: stat d0/db2/cce 0 2024-09-16T23:49:05.955 INFO:tasks.workunit.client.0.smithi042.stdout:8/526: read da/d19/f39 [319805,108334] 0 2024-09-16T23:49:05.956 INFO:tasks.workunit.client.0.smithi042.stdout:4/878: truncate de/f5a 1099480 0 2024-09-16T23:49:05.958 INFO:tasks.workunit.client.0.smithi042.stdout:1/726: chown d0/d9/fb9 3112 1 2024-09-16T23:49:05.961 INFO:tasks.workunit.client.0.smithi042.stdout:6/653: write d8/d70/d81/dd/d1b/d50/fb9 [4050267,20475] 0 2024-09-16T23:49:05.962 INFO:tasks.workunit.client.0.smithi042.stdout:5/803: dread d9/db/d1b/d31/d46/f49 [0,4194304] 0 2024-09-16T23:49:05.962 INFO:tasks.workunit.client.0.smithi042.stdout:4/879: mkdir de/d10d/db8/d12b 0 2024-09-16T23:49:05.963 INFO:tasks.workunit.client.0.smithi042.stdout:4/880: stat de/d1a/d70/l11d 0 2024-09-16T23:49:05.965 INFO:tasks.workunit.client.0.smithi042.stdout:9/481: rename d4/d12/d2d/d37/l26 to d4/d10/d4b/l95 0 2024-09-16T23:49:05.967 INFO:tasks.workunit.client.0.smithi042.stdout:8/527: chown da/d19/d1d/c20 86000 1 2024-09-16T23:49:05.967 INFO:tasks.workunit.client.0.smithi042.stdout:6/654: truncate d8/d70/d81/dd/d2c/d57/dc5/fce 587561 0 2024-09-16T23:49:05.968 INFO:tasks.workunit.client.1.smithi043.stdout:9/750: creat d4/d1c/d6b/d67/fec x:0 0 0 2024-09-16T23:49:05.969 INFO:tasks.workunit.client.0.smithi042.stdout:6/655: readlink d8/d70/d81/d90/lc0 0 2024-09-16T23:49:05.970 INFO:tasks.workunit.client.0.smithi042.stdout:4/881: write de/d1a/f55 [1918277,36988] 0 2024-09-16T23:49:05.972 INFO:tasks.workunit.client.1.smithi043.stdout:6/770: dread de/d23/d34/d9b/f88 [0,4194304] 0 2024-09-16T23:49:05.973 INFO:tasks.workunit.client.1.smithi043.stdout:6/771: chown de 23 1 2024-09-16T23:49:05.974 INFO:tasks.workunit.client.0.smithi042.stdout:4/882: read de/d2e/fed [13133,10026] 0 2024-09-16T23:49:05.977 INFO:tasks.workunit.client.1.smithi043.stdout:1/931: getdents d4/d3e/d5b/dca 0 2024-09-16T23:49:05.979 INFO:tasks.workunit.client.0.smithi042.stdout:5/804: dwrite d9/db/d19/f98 [0,4194304] 0 2024-09-16T23:49:05.979 INFO:tasks.workunit.client.0.smithi042.stdout:6/656: mkdir d8/d70/d81/dd/d1b/d50/d96/da0/dd7 0 2024-09-16T23:49:05.979 INFO:tasks.workunit.client.0.smithi042.stdout:8/528: dwrite da/d19/d1d/d5a/f8e [0,4194304] 0 2024-09-16T23:49:05.985 INFO:tasks.workunit.client.0.smithi042.stdout:6/657: chown d8/d70/d81/dd/d1b/d67 10351 1 2024-09-16T23:49:05.986 INFO:tasks.workunit.client.0.smithi042.stdout:2/603: rename d0/f12 to d0/da/d1b/d48/db7/fcd 0 2024-09-16T23:49:05.986 INFO:tasks.workunit.client.0.smithi042.stdout:5/805: symlink d9/db/d1b/da4/de2/l118 0 2024-09-16T23:49:05.990 INFO:tasks.workunit.client.0.smithi042.stdout:6/658: dread d8/d70/d84/fb5 [0,4194304] 0 2024-09-16T23:49:05.990 INFO:tasks.workunit.client.0.smithi042.stdout:2/604: write d0/da/d1b/d48/f58 [1733819,129320] 0 2024-09-16T23:49:05.991 INFO:tasks.workunit.client.0.smithi042.stdout:9/482: link d4/d10/f6b d4/d12/d2d/d37/d59/f96 0 2024-09-16T23:49:05.994 INFO:tasks.workunit.client.0.smithi042.stdout:5/806: truncate d9/db/d19/d21/d85/ff1 255982 0 2024-09-16T23:49:05.995 INFO:tasks.workunit.client.0.smithi042.stdout:8/529: symlink da/d4b/la1 0 2024-09-16T23:49:05.995 INFO:tasks.workunit.client.0.smithi042.stdout:4/883: dwrite de/d42/d82/d94/fa3 [0,4194304] 0 2024-09-16T23:49:05.995 INFO:tasks.workunit.client.0.smithi042.stdout:6/659: chown d8/d70/d81/dd/d1b/d34/c4a 48761 1 2024-09-16T23:49:05.996 INFO:tasks.workunit.client.1.smithi043.stdout:0/845: creat d0/d2/df/d1a/f11a x:0 0 0 2024-09-16T23:49:05.996 INFO:tasks.workunit.client.0.smithi042.stdout:4/884: write fd [1106026,83330] 0 2024-09-16T23:49:06.000 INFO:tasks.workunit.client.1.smithi043.stdout:8/745: mknod de/d18/d27/d41/d5a/d37/ce1 0 2024-09-16T23:49:06.000 INFO:tasks.workunit.client.0.smithi042.stdout:4/885: write de/d12/f25 [12295150,56202] 0 2024-09-16T23:49:06.001 INFO:tasks.workunit.client.0.smithi042.stdout:4/886: readlink de/d42/d82/l104 0 2024-09-16T23:49:06.001 INFO:tasks.workunit.client.1.smithi043.stdout:2/707: getdents d8/d4b/d4f/d7c 0 2024-09-16T23:49:06.002 INFO:tasks.workunit.client.0.smithi042.stdout:5/807: getdents d9/d63 0 2024-09-16T23:49:06.003 INFO:tasks.workunit.client.0.smithi042.stdout:4/887: chown de/d2e/d37/d3e/fd1 29 1 2024-09-16T23:49:06.003 INFO:tasks.workunit.client.0.smithi042.stdout:1/727: stat d0/d9/d90/d6b/d88/def/f106 0 2024-09-16T23:49:06.004 INFO:tasks.workunit.client.0.smithi042.stdout:5/808: chown d9/db/d1b/d31/d46/l117 17139 1 2024-09-16T23:49:06.004 INFO:tasks.workunit.client.0.smithi042.stdout:1/728: chown d0/d9/d90/d6b/d8e 19303 1 2024-09-16T23:49:06.006 INFO:tasks.workunit.client.1.smithi043.stdout:9/751: rename d4/d5/f92 to d4/d1c/d4f/d45/d85/fed 0 2024-09-16T23:49:06.007 INFO:tasks.workunit.client.0.smithi042.stdout:4/888: creat de/d42/d5d/d8f/d9f/f12c x:0 0 0 2024-09-16T23:49:06.008 INFO:tasks.workunit.client.0.smithi042.stdout:5/809: truncate d9/db/d1b/d31/d6c/f93 2177311 0 2024-09-16T23:49:06.008 INFO:tasks.workunit.client.1.smithi043.stdout:9/752: write d4/d1c/d4f/d8c/f66 [5017421,79310] 0 2024-09-16T23:49:06.009 INFO:tasks.workunit.client.0.smithi042.stdout:4/889: creat de/d42/d5d/d8f/dd6/f12d x:0 0 0 2024-09-16T23:49:06.010 INFO:tasks.workunit.client.0.smithi042.stdout:2/605: rmdir d0/d26 39 2024-09-16T23:49:06.011 INFO:tasks.workunit.client.0.smithi042.stdout:2/606: dread - d0/da/d88/fc7 zero size 2024-09-16T23:49:06.014 INFO:tasks.workunit.client.0.smithi042.stdout:2/607: chown d0/da/d17/d5c/l74 1015932 1 2024-09-16T23:49:06.014 INFO:tasks.workunit.client.0.smithi042.stdout:6/660: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d7b/fd8 x:0 0 0 2024-09-16T23:49:06.015 INFO:tasks.workunit.client.1.smithi043.stdout:7/757: dwrite d4/db/d43/d83/d85/d8b/faf [0,4194304] 0 2024-09-16T23:49:06.017 INFO:tasks.workunit.client.1.smithi043.stdout:1/932: mkdir d4/df/d26/d79/d13e 0 2024-09-16T23:49:06.017 INFO:tasks.workunit.client.1.smithi043.stdout:7/758: chown d4/db/d11/d21/d31/d33/fea 84751470 1 2024-09-16T23:49:06.018 INFO:tasks.workunit.client.0.smithi042.stdout:4/890: dwrite de/d12/d7d/f88 [0,4194304] 0 2024-09-16T23:49:06.020 INFO:tasks.workunit.client.1.smithi043.stdout:7/759: truncate d4/db/d11/fcf 393412 0 2024-09-16T23:49:06.020 INFO:tasks.workunit.client.1.smithi043.stdout:1/933: chown d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fea 341745839 1 2024-09-16T23:49:06.021 INFO:tasks.workunit.client.0.smithi042.stdout:9/483: mkdir d4/d12/d60/d97 0 2024-09-16T23:49:06.022 INFO:tasks.workunit.client.0.smithi042.stdout:4/891: creat de/d42/d82/d94/d124/f12e x:0 0 0 2024-09-16T23:49:06.024 INFO:tasks.workunit.client.0.smithi042.stdout:2/608: creat d0/da/d17/d5c/fce x:0 0 0 2024-09-16T23:49:06.026 INFO:tasks.workunit.client.0.smithi042.stdout:2/609: chown d0/da/d17/d3e/d86/d96 35 1 2024-09-16T23:49:06.027 INFO:tasks.workunit.client.0.smithi042.stdout:6/661: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/d99/fd9 x:0 0 0 2024-09-16T23:49:06.029 INFO:tasks.workunit.client.1.smithi043.stdout:8/746: write de/d18/f36 [900842,22704] 0 2024-09-16T23:49:06.031 INFO:tasks.workunit.client.0.smithi042.stdout:4/892: dwrite de/d42/d82/d94/dbd/f127 [0,4194304] 0 2024-09-16T23:49:06.031 INFO:tasks.workunit.client.0.smithi042.stdout:5/810: truncate d9/db/d1b/d31/d6c/f93 1710390 0 2024-09-16T23:49:06.032 INFO:tasks.workunit.client.1.smithi043.stdout:2/708: symlink d8/d1d/lf1 0 2024-09-16T23:49:06.032 INFO:tasks.workunit.client.1.smithi043.stdout:5/917: dwrite d3/d9/fdd [0,4194304] 0 2024-09-16T23:49:06.032 INFO:tasks.workunit.client.1.smithi043.stdout:8/747: truncate de/d70/f76 4983687 0 2024-09-16T23:49:06.034 INFO:tasks.workunit.client.0.smithi042.stdout:9/484: symlink d4/d12/l98 0 2024-09-16T23:49:06.035 INFO:tasks.workunit.client.0.smithi042.stdout:4/893: dread - de/d12/db0/fdb zero size 2024-09-16T23:49:06.035 INFO:tasks.workunit.client.1.smithi043.stdout:8/748: chown de/d18/d1f/d2f/d39/d4d/lc3 278393 1 2024-09-16T23:49:06.037 INFO:tasks.workunit.client.1.smithi043.stdout:0/846: rename d0/d2/df/d1a/d49/db2/df4/f70 to d0/d2/df/d1a/d49/db2/df4/d9c/dfa/f11b 0 2024-09-16T23:49:06.038 INFO:tasks.workunit.client.0.smithi042.stdout:6/662: mknod d8/d70/d81/dd/d1b/dbf/cda 0 2024-09-16T23:49:06.039 INFO:tasks.workunit.client.0.smithi042.stdout:4/894: mknod de/c12f 0 2024-09-16T23:49:06.041 INFO:tasks.workunit.client.0.smithi042.stdout:5/811: dread d9/db/d19/d21/d85/dec/d112/d61/fd9 [0,4194304] 0 2024-09-16T23:49:06.043 INFO:tasks.workunit.client.0.smithi042.stdout:9/485: rmdir d4/d12/d2d/d51 39 2024-09-16T23:49:06.044 INFO:tasks.workunit.client.1.smithi043.stdout:3/808: mknod d4/d30/dd1/cfb 0 2024-09-16T23:49:06.047 INFO:tasks.workunit.client.1.smithi043.stdout:7/760: chown d4/f32 4129982 1 2024-09-16T23:49:06.049 INFO:tasks.workunit.client.1.smithi043.stdout:1/934: mkdir d4/d9/d8f/db6/dc6/d13f 0 2024-09-16T23:49:06.050 INFO:tasks.workunit.client.0.smithi042.stdout:5/812: write d9/d2c/d7b/fd2 [4830062,109192] 0 2024-09-16T23:49:06.052 INFO:tasks.workunit.client.1.smithi043.stdout:4/857: creat d8/d25/d39/d41/d57/ded/f109 x:0 0 0 2024-09-16T23:49:06.054 INFO:tasks.workunit.client.0.smithi042.stdout:4/895: dread de/d1a/da7/f56 [0,4194304] 0 2024-09-16T23:49:06.055 INFO:tasks.workunit.client.1.smithi043.stdout:1/935: dwrite d4/d3e/fdb [4194304,4194304] 0 2024-09-16T23:49:06.055 INFO:tasks.workunit.client.0.smithi042.stdout:9/486: rmdir d4/d12/d16/d68 39 2024-09-16T23:49:06.058 INFO:tasks.workunit.client.0.smithi042.stdout:2/610: link d0/c72 d0/da/d1b/d3b/ccf 0 2024-09-16T23:49:06.059 INFO:tasks.workunit.client.0.smithi042.stdout:4/896: fsync de/d42/d85/db3/de7/fea 0 2024-09-16T23:49:06.062 INFO:tasks.workunit.client.0.smithi042.stdout:5/813: dwrite d9/db/d1b/d31/d46/f49 [0,4194304] 0 2024-09-16T23:49:06.063 INFO:tasks.workunit.client.0.smithi042.stdout:9/487: creat d4/d12/d2d/d37/d3a/d50/f99 x:0 0 0 2024-09-16T23:49:06.064 INFO:tasks.workunit.client.0.smithi042.stdout:4/897: unlink de/c12f 0 2024-09-16T23:49:06.064 INFO:tasks.workunit.client.0.smithi042.stdout:9/488: stat d4/d10/f30 0 2024-09-16T23:49:06.066 INFO:tasks.workunit.client.0.smithi042.stdout:5/814: readlink d9/db/de/lf8 0 2024-09-16T23:49:06.066 INFO:tasks.workunit.client.0.smithi042.stdout:4/898: chown de/d2e/d37/d4c/dfe/d106 11 1 2024-09-16T23:49:06.069 INFO:tasks.workunit.client.0.smithi042.stdout:5/815: readlink d9/db/d19/l78 0 2024-09-16T23:49:06.070 INFO:tasks.workunit.client.0.smithi042.stdout:4/899: mkdir de/d2e/d37/d4c/dd4/d130 0 2024-09-16T23:49:06.070 INFO:tasks.workunit.client.0.smithi042.stdout:4/900: stat de/d2e 0 2024-09-16T23:49:06.074 INFO:tasks.workunit.client.0.smithi042.stdout:4/901: dread de/d12/d7d/f88 [0,4194304] 0 2024-09-16T23:49:06.074 INFO:tasks.workunit.client.0.smithi042.stdout:2/611: rename d0/d26/f53 to d0/d78/d8b/fd0 0 2024-09-16T23:49:06.075 INFO:tasks.workunit.client.0.smithi042.stdout:2/612: read - d0/da/d17/d5c/fce zero size 2024-09-16T23:49:06.077 INFO:tasks.workunit.client.0.smithi042.stdout:9/489: mknod d4/d12/d77/c9a 0 2024-09-16T23:49:06.078 INFO:tasks.workunit.client.1.smithi043.stdout:2/709: dread d8/d1d/d20/d26/f7d [0,4194304] 0 2024-09-16T23:49:06.079 INFO:tasks.workunit.client.0.smithi042.stdout:2/613: creat d0/da/d88/dae/fd1 x:0 0 0 2024-09-16T23:49:06.081 INFO:tasks.workunit.client.0.smithi042.stdout:4/902: truncate de/d1a/f55 155798 0 2024-09-16T23:49:06.084 INFO:tasks.workunit.client.1.smithi043.stdout:0/847: truncate d0/d2/df/d1a/d46/d39/daa/db7/fcc 656602 0 2024-09-16T23:49:06.084 INFO:tasks.workunit.client.0.smithi042.stdout:4/903: symlink de/d42/d5d/d8f/d9f/dc0/l131 0 2024-09-16T23:49:06.085 INFO:tasks.workunit.client.0.smithi042.stdout:2/614: write d0/da/d1b/d3b/d44/fb3 [1090919,83502] 0 2024-09-16T23:49:06.090 INFO:tasks.workunit.client.0.smithi042.stdout:4/904: dwrite de/d1a/f4b [0,4194304] 0 2024-09-16T23:49:06.092 INFO:tasks.workunit.client.1.smithi043.stdout:4/858: mknod d8/d15/d9b/d8b/c10a 0 2024-09-16T23:49:06.095 INFO:tasks.workunit.client.0.smithi042.stdout:4/905: link de/d1a/d70/l75 de/d1a/d70/df9/l132 0 2024-09-16T23:49:06.097 INFO:tasks.workunit.client.1.smithi043.stdout:5/918: symlink d3/l137 0 2024-09-16T23:49:06.097 INFO:tasks.workunit.client.0.smithi042.stdout:9/490: write d4/d12/d2d/d37/d3a/f5b [140137,103779] 0 2024-09-16T23:49:06.099 INFO:tasks.workunit.client.1.smithi043.stdout:2/710: truncate d8/d35/fb9 47522 0 2024-09-16T23:49:06.100 INFO:tasks.workunit.client.1.smithi043.stdout:3/809: write d4/de/d40/df0/f6a [1079428,40430] 0 2024-09-16T23:49:06.100 INFO:tasks.workunit.client.1.smithi043.stdout:9/753: link d4/d21/cdd d4/d5/d16/cee 0 2024-09-16T23:49:06.101 INFO:tasks.workunit.client.0.smithi042.stdout:9/491: mknod d4/d10/d4b/c9b 0 2024-09-16T23:49:06.103 INFO:tasks.workunit.client.1.smithi043.stdout:1/936: dread d4/d34/f41 [4194304,4194304] 0 2024-09-16T23:49:06.103 INFO:tasks.workunit.client.1.smithi043.stdout:6/772: getdents de/d23/d63/dec 0 2024-09-16T23:49:06.104 INFO:tasks.workunit.client.0.smithi042.stdout:9/492: mkdir d4/d12/d16/d81/d9c 0 2024-09-16T23:49:06.104 INFO:tasks.workunit.client.1.smithi043.stdout:9/754: chown d4/d21/d2a/l82 13 1 2024-09-16T23:49:06.106 INFO:tasks.workunit.client.1.smithi043.stdout:4/859: mkdir d8/d15/d20/d2b/d54/db3/d4c/dee/d10b 0 2024-09-16T23:49:06.106 INFO:tasks.workunit.client.1.smithi043.stdout:3/810: write d4/de/d1f/f29 [202000,36771] 0 2024-09-16T23:49:06.106 INFO:tasks.workunit.client.1.smithi043.stdout:8/749: dwrite de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:49:06.107 INFO:tasks.workunit.client.1.smithi043.stdout:5/919: dread - d3/d1b/fa6 zero size 2024-09-16T23:49:06.111 INFO:tasks.workunit.client.1.smithi043.stdout:0/848: link d0/d2/df/d1a/d3b/dee/df0/df8/ffd d0/d2/df/d1a/d46/f11c 0 2024-09-16T23:49:06.113 INFO:tasks.workunit.client.0.smithi042.stdout:4/906: dwrite de/d12/f54 [0,4194304] 0 2024-09-16T23:49:06.114 INFO:tasks.workunit.client.0.smithi042.stdout:9/493: dwrite d4/d12/d2d/d37/d3a/f48 [0,4194304] 0 2024-09-16T23:49:06.115 INFO:tasks.workunit.client.1.smithi043.stdout:2/711: write d8/d4b/d5f/d8e/de8/fe2 [861762,45401] 0 2024-09-16T23:49:06.116 INFO:tasks.workunit.client.0.smithi042.stdout:9/494: chown d4/d10/d4b 31 1 2024-09-16T23:49:06.118 INFO:tasks.workunit.client.0.smithi042.stdout:9/495: rename d4/d12/d2d to d4/d12/d2d/d37/d3a/d9d 22 2024-09-16T23:49:06.120 INFO:tasks.workunit.client.1.smithi043.stdout:7/761: getdents d4/d27/dbc 0 2024-09-16T23:49:06.121 INFO:tasks.workunit.client.1.smithi043.stdout:2/712: chown d8/d1d/d20 13 1 2024-09-16T23:49:06.121 INFO:tasks.workunit.client.1.smithi043.stdout:9/755: dwrite d4/d1c/d25/db1/fe1 [0,4194304] 0 2024-09-16T23:49:06.122 INFO:tasks.workunit.client.0.smithi042.stdout:4/907: creat de/d1a/da7/db9/f133 x:0 0 0 2024-09-16T23:49:06.126 INFO:tasks.workunit.client.1.smithi043.stdout:8/750: dwrite de/d18/d1f/d25/fb9 [0,4194304] 0 2024-09-16T23:49:06.127 INFO:tasks.workunit.client.0.smithi042.stdout:9/496: dwrite d4/d12/d16/f6d [0,4194304] 0 2024-09-16T23:49:06.129 INFO:tasks.workunit.client.0.smithi042.stdout:9/497: dread d4/d12/d2d/d37/d3a/f5c [0,4194304] 0 2024-09-16T23:49:06.130 INFO:tasks.workunit.client.1.smithi043.stdout:3/811: mkdir d4/de/d1f/dfc 0 2024-09-16T23:49:06.131 INFO:tasks.workunit.client.1.smithi043.stdout:4/860: rmdir d8/d15/d20/d2b/d54/db3/d4c 39 2024-09-16T23:49:06.141 INFO:tasks.workunit.client.0.smithi042.stdout:4/908: dread de/d1a/f46 [0,4194304] 0 2024-09-16T23:49:06.142 INFO:tasks.workunit.client.1.smithi043.stdout:1/937: creat d4/f140 x:0 0 0 2024-09-16T23:49:06.144 INFO:tasks.workunit.client.1.smithi043.stdout:7/762: rmdir d4/db/d11/d15/d29/da7 39 2024-09-16T23:49:06.145 INFO:tasks.workunit.client.0.smithi042.stdout:4/909: symlink de/d2e/d37/d4c/d4f/l134 0 2024-09-16T23:49:06.146 INFO:tasks.workunit.client.1.smithi043.stdout:7/763: write d4/db/d11/d21/d31/f82 [2886455,126656] 0 2024-09-16T23:49:06.146 INFO:tasks.workunit.client.1.smithi043.stdout:6/773: write de/f13 [5307705,107342] 0 2024-09-16T23:49:06.147 INFO:tasks.workunit.client.1.smithi043.stdout:9/756: creat d4/d1c/ddc/fef x:0 0 0 2024-09-16T23:49:06.148 INFO:tasks.workunit.client.1.smithi043.stdout:7/764: dread - d4/db/d11/d21/d31/d65/fdf zero size 2024-09-16T23:49:06.151 INFO:tasks.workunit.client.1.smithi043.stdout:2/713: mkdir d8/d4b/d4f/d56/df2 0 2024-09-16T23:49:06.151 INFO:tasks.workunit.client.0.smithi042.stdout:4/910: link de/d1a/c23 de/d1a/c135 0 2024-09-16T23:49:06.153 INFO:tasks.workunit.client.1.smithi043.stdout:8/751: stat de/d18/d1f/d25/d65/d7f/lc6 0 2024-09-16T23:49:06.154 INFO:tasks.workunit.client.0.smithi042.stdout:4/911: creat de/d42/f136 x:0 0 0 2024-09-16T23:49:06.155 INFO:tasks.workunit.client.1.smithi043.stdout:3/812: truncate d4/de/fab 389752 0 2024-09-16T23:49:06.156 INFO:tasks.workunit.client.0.smithi042.stdout:4/912: dread - de/d12/f28 zero size 2024-09-16T23:49:06.159 INFO:tasks.workunit.client.0.smithi042.stdout:9/498: dwrite d4/d10/f4a [0,4194304] 0 2024-09-16T23:49:06.160 INFO:tasks.workunit.client.0.smithi042.stdout:4/913: creat de/d2e/f137 x:0 0 0 2024-09-16T23:49:06.162 INFO:tasks.workunit.client.0.smithi042.stdout:9/499: write d4/d12/d2d/d37/d3a/f5b [126016,51280] 0 2024-09-16T23:49:06.163 INFO:tasks.workunit.client.0.smithi042.stdout:4/914: truncate de/d2e/d37/d3e/f107 5228186 0 2024-09-16T23:49:06.164 INFO:tasks.workunit.client.1.smithi043.stdout:6/774: creat de/d23/d34/d9b/d50/f108 x:0 0 0 2024-09-16T23:49:06.165 INFO:tasks.workunit.client.1.smithi043.stdout:6/775: chown de/d2a/f33 20 1 2024-09-16T23:49:06.166 INFO:tasks.workunit.client.1.smithi043.stdout:6/776: dread - de/d23/d49/dc9/ff4 zero size 2024-09-16T23:49:06.169 INFO:tasks.workunit.client.1.smithi043.stdout:7/765: rename d4/db/d11/d21/d31/dcb/df8/fb7 to d4/d27/d41/d1b/d75/ffc 0 2024-09-16T23:49:06.174 INFO:tasks.workunit.client.1.smithi043.stdout:9/757: dwrite d4/d21/d33/d95/fc9 [0,4194304] 0 2024-09-16T23:49:06.177 INFO:tasks.workunit.client.0.smithi042.stdout:9/500: symlink d4/d12/d2d/d37/d28/l9e 0 2024-09-16T23:49:06.177 INFO:tasks.workunit.client.1.smithi043.stdout:1/938: write d4/df/d26/ff9 [2953714,12794] 0 2024-09-16T23:49:06.179 INFO:tasks.workunit.client.1.smithi043.stdout:4/861: unlink d8/d15/d20/d2b/c78 0 2024-09-16T23:49:06.179 INFO:tasks.workunit.client.1.smithi043.stdout:1/939: dread - d4/d9/d1c/d1e/d47/f138 zero size 2024-09-16T23:49:06.179 INFO:tasks.workunit.client.0.smithi042.stdout:9/501: truncate d4/d12/d16/f85 4444465 0 2024-09-16T23:49:06.180 INFO:tasks.workunit.client.0.smithi042.stdout:4/915: dwrite de/d2e/f77 [0,4194304] 0 2024-09-16T23:49:06.180 INFO:tasks.workunit.client.1.smithi043.stdout:0/849: getdents d0/d2/df/d1a/de8 0 2024-09-16T23:49:06.184 INFO:tasks.workunit.client.0.smithi042.stdout:4/916: getdents de/d10d/d128 0 2024-09-16T23:49:06.186 INFO:tasks.workunit.client.0.smithi042.stdout:9/502: fsync d4/d10/f57 0 2024-09-16T23:49:06.187 INFO:tasks.workunit.client.0.smithi042.stdout:4/917: symlink de/d42/d68/l138 0 2024-09-16T23:49:06.187 INFO:tasks.workunit.client.0.smithi042.stdout:9/503: chown d4/d12/d77/c9a 7 1 2024-09-16T23:49:06.188 INFO:tasks.workunit.client.0.smithi042.stdout:4/918: chown de/d2e/d37/d3e/c103 1 1 2024-09-16T23:49:06.188 INFO:tasks.workunit.client.1.smithi043.stdout:2/714: symlink d8/dec/lf3 0 2024-09-16T23:49:06.191 INFO:tasks.workunit.client.0.smithi042.stdout:9/504: truncate d4/d12/d16/d68/f74 521835 0 2024-09-16T23:49:06.191 INFO:tasks.workunit.client.0.smithi042.stdout:4/919: creat de/d42/d5d/d8f/dd6/d123/f139 x:0 0 0 2024-09-16T23:49:06.192 INFO:tasks.workunit.client.0.smithi042.stdout:4/920: stat de/lfb 0 2024-09-16T23:49:06.194 INFO:tasks.workunit.client.1.smithi043.stdout:9/758: mkdir d4/d21/d2a/d2c/d7b/df0 0 2024-09-16T23:49:06.195 INFO:tasks.workunit.client.1.smithi043.stdout:9/759: stat d4/d5/l38 0 2024-09-16T23:49:06.195 INFO:tasks.workunit.client.1.smithi043.stdout:9/760: fsync d4/d21/d35/fd5 0 2024-09-16T23:49:06.196 INFO:tasks.workunit.client.0.smithi042.stdout:9/505: creat d4/d12/d16/d68/f9f x:0 0 0 2024-09-16T23:49:06.196 INFO:tasks.workunit.client.0.smithi042.stdout:4/921: creat de/d42/d5d/d8f/f13a x:0 0 0 2024-09-16T23:49:06.197 INFO:tasks.workunit.client.1.smithi043.stdout:6/777: write de/d1a/d28/ff1 [3647362,111786] 0 2024-09-16T23:49:06.197 INFO:tasks.workunit.client.1.smithi043.stdout:4/862: write d8/d15/d20/d2b/d54/f91 [817204,73723] 0 2024-09-16T23:49:06.198 INFO:tasks.workunit.client.0.smithi042.stdout:9/506: write d4/fb [1098217,45620] 0 2024-09-16T23:49:06.199 INFO:tasks.workunit.client.1.smithi043.stdout:0/850: mkdir d0/d2/df/d1a/d46/d11d 0 2024-09-16T23:49:06.201 INFO:tasks.workunit.client.1.smithi043.stdout:8/752: write de/d18/d27/d41/d5a/f26 [2122601,114406] 0 2024-09-16T23:49:06.203 INFO:tasks.workunit.client.1.smithi043.stdout:7/766: symlink d4/db/d11/d21/lfd 0 2024-09-16T23:49:06.204 INFO:tasks.workunit.client.0.smithi042.stdout:4/922: getdents de/d42/d5d/d8f/dd6/d123 0 2024-09-16T23:49:06.204 INFO:tasks.workunit.client.0.smithi042.stdout:9/507: getdents d4/d12/d2d/d37/d28/d5e/d83 0 2024-09-16T23:49:06.205 INFO:tasks.workunit.client.1.smithi043.stdout:3/813: mknod d4/de/d1f/d2c/db7/dc6/deb/cfd 0 2024-09-16T23:49:06.206 INFO:tasks.workunit.client.0.smithi042.stdout:9/508: write d4/d12/d2d/d37/d3a/d50/f99 [500949,8458] 0 2024-09-16T23:49:06.207 INFO:tasks.workunit.client.1.smithi043.stdout:9/761: symlink d4/d1c/d6b/d67/dbf/lf1 0 2024-09-16T23:49:06.208 INFO:tasks.workunit.client.0.smithi042.stdout:9/509: write d4/d12/d16/d68/f72 [528928,21931] 0 2024-09-16T23:49:06.209 INFO:tasks.workunit.client.1.smithi043.stdout:6/778: symlink de/d23/d34/d66/l109 0 2024-09-16T23:49:06.213 INFO:tasks.workunit.client.0.smithi042.stdout:4/923: dwrite de/d12/d7d/f88 [0,4194304] 0 2024-09-16T23:49:06.215 INFO:tasks.workunit.client.1.smithi043.stdout:8/753: dread de/f45 [0,4194304] 0 2024-09-16T23:49:06.215 INFO:tasks.workunit.client.1.smithi043.stdout:8/754: readlink de/d70/l5c 0 2024-09-16T23:49:06.219 INFO:tasks.workunit.client.1.smithi043.stdout:7/767: rename d4/f19 to d4/db/d11/d15/dd2/ffe 0 2024-09-16T23:49:06.220 INFO:tasks.workunit.client.0.smithi042.stdout:4/924: truncate de/d42/d68/f72 1124923 0 2024-09-16T23:49:06.221 INFO:tasks.workunit.client.1.smithi043.stdout:7/768: readlink d4/db/d11/d21/lfd 0 2024-09-16T23:49:06.221 INFO:tasks.workunit.client.0.smithi042.stdout:9/510: creat d4/d12/d60/d97/fa0 x:0 0 0 2024-09-16T23:49:06.222 INFO:tasks.workunit.client.0.smithi042.stdout:4/925: write de/d42/d5d/d8f/f13a [94455,8386] 0 2024-09-16T23:49:06.224 INFO:tasks.workunit.client.0.smithi042.stdout:4/926: read - de/d2e/d37/dc3/fd0 zero size 2024-09-16T23:49:06.226 INFO:tasks.workunit.client.1.smithi043.stdout:6/779: creat de/d23/d34/d66/d9d/dae/f10a x:0 0 0 2024-09-16T23:49:06.227 INFO:tasks.workunit.client.0.smithi042.stdout:4/927: symlink de/d2e/d37/d4c/l13b 0 2024-09-16T23:49:06.228 INFO:tasks.workunit.client.1.smithi043.stdout:1/940: link d4/d9/d8f/db6/dcc/dda/d107/f57 d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/f141 0 2024-09-16T23:49:06.228 INFO:tasks.workunit.client.1.smithi043.stdout:6/780: read de/d23/d63/fd2 [345009,11458] 0 2024-09-16T23:49:06.229 INFO:tasks.workunit.client.1.smithi043.stdout:2/715: write d8/d11/d16/f19 [1185649,16906] 0 2024-09-16T23:49:06.230 INFO:tasks.workunit.client.1.smithi043.stdout:6/781: fdatasync de/d1a/d28/ff1 0 2024-09-16T23:49:06.231 INFO:tasks.workunit.client.0.smithi042.stdout:4/928: symlink de/d10d/d8a/l13c 0 2024-09-16T23:49:06.233 INFO:tasks.workunit.client.1.smithi043.stdout:4/863: creat d8/d15/f10c x:0 0 0 2024-09-16T23:49:06.234 INFO:tasks.workunit.client.1.smithi043.stdout:2/716: chown d8/d4b/d5f/d8e/de8/feb 1392379 1 2024-09-16T23:49:06.234 INFO:tasks.workunit.client.1.smithi043.stdout:3/814: dread d4/de/d1f/d34/f74 [0,4194304] 0 2024-09-16T23:49:06.237 INFO:tasks.workunit.client.1.smithi043.stdout:3/815: read d4/de/d40/fb5 [213113,59091] 0 2024-09-16T23:49:06.237 INFO:tasks.workunit.client.0.smithi042.stdout:4/929: getdents de/d2e/d37/d3e 0 2024-09-16T23:49:06.239 INFO:tasks.workunit.client.1.smithi043.stdout:1/941: unlink d4/df/f91 0 2024-09-16T23:49:06.239 INFO:tasks.workunit.client.1.smithi043.stdout:3/816: read - d4/de/d1f/d2c/db7/dc3/fe7 zero size 2024-09-16T23:49:06.242 INFO:tasks.workunit.client.1.smithi043.stdout:0/851: getdents d0/d2/df/d1a/d3b 0 2024-09-16T23:49:06.243 INFO:tasks.workunit.client.0.smithi042.stdout:9/511: dread d4/d10/f1f [0,4194304] 0 2024-09-16T23:49:06.243 INFO:tasks.workunit.client.1.smithi043.stdout:2/717: symlink d8/d35/d9f/lf4 0 2024-09-16T23:49:06.244 INFO:tasks.workunit.client.0.smithi042.stdout:4/930: link de/d42/d5d/d8f/dd6/f12d de/d2e/d37/d3e/f13d 0 2024-09-16T23:49:06.247 INFO:tasks.workunit.client.0.smithi042.stdout:4/931: mknod de/d42/d5d/d8f/d9f/c13e 0 2024-09-16T23:49:06.249 INFO:tasks.workunit.client.1.smithi043.stdout:1/942: truncate d4/d9/d12/d15/dac/ff2 1010908 0 2024-09-16T23:49:06.249 INFO:tasks.workunit.client.1.smithi043.stdout:0/852: mkdir d0/d2/df/d1a/d49/d80/d11e 0 2024-09-16T23:49:06.250 INFO:tasks.workunit.client.0.smithi042.stdout:9/512: mkdir d4/d12/d44/d8e/da1 0 2024-09-16T23:49:06.251 INFO:tasks.workunit.client.1.smithi043.stdout:4/864: link d8/d25/f4f d8/d25/d39/dd3/d100/d108/d4b/d104/f10d 0 2024-09-16T23:49:06.251 INFO:tasks.workunit.client.1.smithi043.stdout:8/755: dread fc [0,4194304] 0 2024-09-16T23:49:06.252 INFO:tasks.workunit.client.1.smithi043.stdout:1/943: fsync d4/fc 0 2024-09-16T23:49:06.254 INFO:tasks.workunit.client.0.smithi042.stdout:4/932: dwrite de/d42/d82/fa5 [0,4194304] 0 2024-09-16T23:49:06.254 INFO:tasks.workunit.client.1.smithi043.stdout:7/769: dwrite d4/d27/f3f [0,4194304] 0 2024-09-16T23:49:06.256 INFO:tasks.workunit.client.0.smithi042.stdout:4/933: write de/d10d/db8/f11f [246887,832] 0 2024-09-16T23:49:06.257 INFO:tasks.workunit.client.1.smithi043.stdout:3/817: dread d4/de/d1f/d2c/f4a [0,4194304] 0 2024-09-16T23:49:06.259 INFO:tasks.workunit.client.1.smithi043.stdout:4/865: unlink d8/d15/fea 0 2024-09-16T23:49:06.260 INFO:tasks.workunit.client.1.smithi043.stdout:7/770: fdatasync d4/f60 0 2024-09-16T23:49:06.262 INFO:tasks.workunit.client.1.smithi043.stdout:9/762: dwrite d4/d28/d5f/fd3 [0,4194304] 0 2024-09-16T23:49:06.264 INFO:tasks.workunit.client.1.smithi043.stdout:6/782: dwrite de/d23/d34/d9b/d50/f102 [0,4194304] 0 2024-09-16T23:49:06.264 INFO:tasks.workunit.client.0.smithi042.stdout:9/513: rename d4/d10/d24/l38 to d4/d12/d2d/d37/d28/la2 0 2024-09-16T23:49:06.268 INFO:tasks.workunit.client.1.smithi043.stdout:4/866: dread d8/d25/d39/fac [0,4194304] 0 2024-09-16T23:49:06.268 INFO:tasks.workunit.client.1.smithi043.stdout:0/853: mknod d0/d2/df/da7/db8/c11f 0 2024-09-16T23:49:06.269 INFO:tasks.workunit.client.0.smithi042.stdout:4/934: dwrite de/d2e/d37/d4c/d4f/f6b [8388608,4194304] 0 2024-09-16T23:49:06.272 INFO:tasks.workunit.client.1.smithi043.stdout:9/763: read - d4/d1c/fb5 zero size 2024-09-16T23:49:06.273 INFO:tasks.workunit.client.1.smithi043.stdout:1/944: rmdir d4/d9/d8f/db6 39 2024-09-16T23:49:06.275 INFO:tasks.workunit.client.1.smithi043.stdout:0/854: dread d0/d2/df/d1a/d46/fe6 [0,4194304] 0 2024-09-16T23:49:06.275 INFO:tasks.workunit.client.1.smithi043.stdout:2/718: dwrite d8/d4b/d5f/d8e/fbb [0,4194304] 0 2024-09-16T23:49:06.278 INFO:tasks.workunit.client.0.smithi042.stdout:9/514: mkdir d4/d12/d2d/d94/da3 0 2024-09-16T23:49:06.279 INFO:tasks.workunit.client.1.smithi043.stdout:8/756: creat de/d18/d2b/da1/fe2 x:0 0 0 2024-09-16T23:49:06.280 INFO:tasks.workunit.client.0.smithi042.stdout:4/935: dwrite de/d2e/d37/d4c/f9b [0,4194304] 0 2024-09-16T23:49:06.281 INFO:tasks.workunit.client.1.smithi043.stdout:7/771: fdatasync d4/d27/f6c 0 2024-09-16T23:49:06.281 INFO:tasks.workunit.client.1.smithi043.stdout:8/757: fsync de/d18/d1f/d2f/d39/d4d/fbf 0 2024-09-16T23:49:06.282 INFO:tasks.workunit.client.1.smithi043.stdout:4/867: rmdir d8/d15/d20/d2b/d23 39 2024-09-16T23:49:06.287 INFO:tasks.workunit.client.0.smithi042.stdout:9/515: dwrite d4/d10/f3f [0,4194304] 0 2024-09-16T23:49:06.310 INFO:tasks.workunit.client.1.smithi043.stdout:1/945: fsync d4/d9/d1c/f113 0 2024-09-16T23:49:06.312 INFO:tasks.workunit.client.0.smithi042.stdout:9/516: rmdir d4/d12/d44 39 2024-09-16T23:49:06.314 INFO:tasks.workunit.client.1.smithi043.stdout:9/764: creat d4/d1c/d25/d2b/d4c/da3/ff2 x:0 0 0 2024-09-16T23:49:06.315 INFO:tasks.workunit.client.0.smithi042.stdout:9/517: rmdir d4/d12/d2d/d37/d41 39 2024-09-16T23:49:06.316 INFO:tasks.workunit.client.1.smithi043.stdout:0/855: chown d0/d2/c4a 5 1 2024-09-16T23:49:06.320 INFO:tasks.workunit.client.1.smithi043.stdout:6/783: fdatasync de/d23/dc3/fdf 0 2024-09-16T23:49:06.322 INFO:tasks.workunit.client.0.smithi042.stdout:4/936: dwrite de/d42/d68/f69 [0,4194304] 0 2024-09-16T23:49:06.327 INFO:tasks.workunit.client.0.smithi042.stdout:4/937: write de/d12/f28 [945913,75451] 0 2024-09-16T23:49:06.328 INFO:tasks.workunit.client.0.smithi042.stdout:9/518: getdents d4/d12/d2d/d37 0 2024-09-16T23:49:06.330 INFO:tasks.workunit.client.0.smithi042.stdout:4/938: dread de/d42/fee [0,4194304] 0 2024-09-16T23:49:06.330 INFO:tasks.workunit.client.1.smithi043.stdout:2/719: write d8/d4b/f5b [2432886,27609] 0 2024-09-16T23:49:06.332 INFO:tasks.workunit.client.1.smithi043.stdout:2/720: chown d8/d4b/d4f/c90 2566 1 2024-09-16T23:49:06.332 INFO:tasks.workunit.client.0.smithi042.stdout:9/519: creat d4/d12/d2d/d37/fa4 x:0 0 0 2024-09-16T23:49:06.333 INFO:tasks.workunit.client.1.smithi043.stdout:1/946: mknod d4/d3e/d5b/dca/c142 0 2024-09-16T23:49:06.333 INFO:tasks.workunit.client.1.smithi043.stdout:4/868: write d8/d15/d9b/d8b/f93 [5238509,8126] 0 2024-09-16T23:49:06.334 INFO:tasks.workunit.client.1.smithi043.stdout:8/758: write de/d18/d27/d41/d5a/f1b [4273641,92586] 0 2024-09-16T23:49:06.335 INFO:tasks.workunit.client.1.smithi043.stdout:0/856: symlink d0/d2/df/d1a/de8/dec/l120 0 2024-09-16T23:49:06.335 INFO:tasks.workunit.client.1.smithi043.stdout:1/947: fsync d4/d9/d12/d2b/fb0 0 2024-09-16T23:49:06.336 INFO:tasks.workunit.client.0.smithi042.stdout:4/939: rename de/d42/d5d/d8f/d9f/dc0/l131 to de/d42/d99/d9e/l13f 0 2024-09-16T23:49:06.336 INFO:tasks.workunit.client.1.smithi043.stdout:8/759: truncate de/f45 5228646 0 2024-09-16T23:49:06.337 INFO:tasks.workunit.client.0.smithi042.stdout:4/940: chown de/d12/db0/fdb 3301425 1 2024-09-16T23:49:06.337 INFO:tasks.workunit.client.1.smithi043.stdout:6/784: creat de/d23/dc3/f10b x:0 0 0 2024-09-16T23:49:06.338 INFO:tasks.workunit.client.1.smithi043.stdout:7/772: mknod d4/d27/d41/d1b/cff 0 2024-09-16T23:49:06.339 INFO:tasks.workunit.client.1.smithi043.stdout:7/773: fsync d4/db/d50/fa9 0 2024-09-16T23:49:06.341 INFO:tasks.workunit.client.1.smithi043.stdout:0/857: dread d0/d2/df/d1a/d49/db2/df4/fe4 [0,4194304] 0 2024-09-16T23:49:06.341 INFO:tasks.workunit.client.1.smithi043.stdout:7/774: truncate d4/d27/dbc/fe9 539648 0 2024-09-16T23:49:06.342 INFO:tasks.workunit.client.0.smithi042.stdout:4/941: link de/d42/d82/d94/dbd/fd3 de/d42/d99/d9e/f140 0 2024-09-16T23:49:06.342 INFO:tasks.workunit.client.1.smithi043.stdout:2/721: mknod d8/dec/cf5 0 2024-09-16T23:49:06.343 INFO:tasks.workunit.client.1.smithi043.stdout:9/765: symlink d4/d1c/d25/d2b/d4c/lf3 0 2024-09-16T23:49:06.343 INFO:tasks.workunit.client.1.smithi043.stdout:4/869: dread - d8/f68 zero size 2024-09-16T23:49:06.345 INFO:tasks.workunit.client.0.smithi042.stdout:9/520: rename d4/d12/d2d/d37/d3a/l7b to d4/d12/d2d/d37/d3a/d50/d6f/la5 0 2024-09-16T23:49:06.348 INFO:tasks.workunit.client.1.smithi043.stdout:0/858: dread d0/d2/df/d1a/d46/d39/d6e/f65 [0,4194304] 0 2024-09-16T23:49:06.349 INFO:tasks.workunit.client.1.smithi043.stdout:0/859: dread d0/d2/df/d1a/d49/db2/ff3 [0,4194304] 0 2024-09-16T23:49:06.349 INFO:tasks.workunit.client.1.smithi043.stdout:7/775: mknod d4/d27/d41/d1b/d75/c100 0 2024-09-16T23:49:06.351 INFO:tasks.workunit.client.1.smithi043.stdout:9/766: rename d4/d21/d2a/l82 to d4/d1c/d25/d2b/lf4 0 2024-09-16T23:49:06.356 INFO:tasks.workunit.client.1.smithi043.stdout:2/722: symlink d8/d5e/d83/ddb/lf6 0 2024-09-16T23:49:06.358 INFO:tasks.workunit.client.1.smithi043.stdout:0/860: mkdir d0/d2/df/d1a/d46/d92/d121 0 2024-09-16T23:49:06.359 INFO:tasks.workunit.client.1.smithi043.stdout:9/767: mknod d4/d1c/d25/db1/cf5 0 2024-09-16T23:49:06.359 INFO:tasks.workunit.client.1.smithi043.stdout:0/861: write d0/d2/df/d1a/d49/d80/d88/f10d [1043173,115640] 0 2024-09-16T23:49:06.360 INFO:tasks.workunit.client.1.smithi043.stdout:4/870: symlink d8/d15/d20/d2b/d54/db3/d4c/dee/l10e 0 2024-09-16T23:49:06.361 INFO:tasks.workunit.client.1.smithi043.stdout:1/948: creat d4/f143 x:0 0 0 2024-09-16T23:49:06.364 INFO:tasks.workunit.client.1.smithi043.stdout:7/776: creat d4/db/d11/d15/dda/f101 x:0 0 0 2024-09-16T23:49:06.364 INFO:tasks.workunit.client.1.smithi043.stdout:6/785: write de/f67 [1538249,48699] 0 2024-09-16T23:49:06.365 INFO:tasks.workunit.client.1.smithi043.stdout:2/723: rename d8/d11/d16/f49 to d8/d4b/d4f/d7c/ff7 0 2024-09-16T23:49:06.366 INFO:tasks.workunit.client.1.smithi043.stdout:9/768: symlink d4/d5/lf6 0 2024-09-16T23:49:06.367 INFO:tasks.workunit.client.1.smithi043.stdout:0/862: mknod d0/d2/df/d1a/d46/d92/c122 0 2024-09-16T23:49:06.369 INFO:tasks.workunit.client.1.smithi043.stdout:9/769: dread d4/d1c/f20 [0,4194304] 0 2024-09-16T23:49:06.369 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:06 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:06.369 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:06 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:06.369 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:06 smithi043 ceph-mon[35191]: pgmap v12: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 122 MiB/s rd, 349 MiB/s wr, 687 op/s 2024-09-16T23:49:06.373 INFO:tasks.workunit.client.1.smithi043.stdout:1/949: creat d4/d9/d8f/db6/dcc/dda/f144 x:0 0 0 2024-09-16T23:49:06.374 INFO:tasks.workunit.client.1.smithi043.stdout:7/777: symlink d4/db/d43/d83/l102 0 2024-09-16T23:49:06.376 INFO:tasks.workunit.client.1.smithi043.stdout:9/770: mkdir d4/d1c/d25/d2b/df7 0 2024-09-16T23:49:06.377 INFO:tasks.workunit.client.1.smithi043.stdout:2/724: symlink d8/d5e/d83/d3a/lf8 0 2024-09-16T23:49:06.379 INFO:tasks.workunit.client.1.smithi043.stdout:0/863: unlink d0/d2/f6d 0 2024-09-16T23:49:06.380 INFO:tasks.workunit.client.1.smithi043.stdout:1/950: unlink d4/l8 0 2024-09-16T23:49:06.380 INFO:tasks.workunit.client.1.smithi043.stdout:1/951: fsync d4/f12e 0 2024-09-16T23:49:06.381 INFO:tasks.workunit.client.1.smithi043.stdout:7/778: rmdir d4/db/d11/d15/d29 39 2024-09-16T23:49:06.382 INFO:tasks.workunit.client.1.smithi043.stdout:9/771: mkdir d4/d5/d64/df8 0 2024-09-16T23:49:06.384 INFO:tasks.workunit.client.1.smithi043.stdout:0/864: mkdir d0/d2/df/d1a/d46/d92/d123 0 2024-09-16T23:49:06.385 INFO:tasks.workunit.client.1.smithi043.stdout:2/725: dread d8/d5e/f87 [0,4194304] 0 2024-09-16T23:49:06.386 INFO:tasks.workunit.client.1.smithi043.stdout:4/871: rename d8/d25/d39/dd3/d100/d108/d4b/d104/f10d to d8/d15/d20/dfa/f10f 0 2024-09-16T23:49:06.387 INFO:tasks.workunit.client.1.smithi043.stdout:9/772: truncate d4/d1c/d25/d2b/d4c/fd1 991371 0 2024-09-16T23:49:06.388 INFO:tasks.workunit.client.0.smithi042.stdout:4/942: write de/d2e/d37/d4c/d4f/f7a [40309,84671] 0 2024-09-16T23:49:06.389 INFO:tasks.workunit.client.1.smithi043.stdout:6/786: dwrite de/f71 [0,4194304] 0 2024-09-16T23:49:06.389 INFO:tasks.workunit.client.1.smithi043.stdout:2/726: read d8/d1d/d25/fa4 [2974037,98990] 0 2024-09-16T23:49:06.391 INFO:tasks.workunit.client.1.smithi043.stdout:1/952: creat d4/d3e/d5b/dca/d12a/f145 x:0 0 0 2024-09-16T23:49:06.393 INFO:tasks.workunit.client.0.smithi042.stdout:4/943: dread de/d2e/d37/d4c/f105 [0,4194304] 0 2024-09-16T23:49:06.396 INFO:tasks.workunit.client.0.smithi042.stdout:4/944: creat de/d42/d85/db3/f141 x:0 0 0 2024-09-16T23:49:06.396 INFO:tasks.workunit.client.1.smithi043.stdout:9/773: unlink d4/d21/d33/cc3 0 2024-09-16T23:49:06.397 INFO:tasks.workunit.client.1.smithi043.stdout:9/774: readlink d4/d28/d5f/l6c 0 2024-09-16T23:49:06.397 INFO:tasks.workunit.client.1.smithi043.stdout:9/775: chown d4/d28/d5f/l6c 16 1 2024-09-16T23:49:06.402 INFO:tasks.workunit.client.1.smithi043.stdout:6/787: creat de/d23/d34/d9b/d8b/dbd/f10c x:0 0 0 2024-09-16T23:49:06.409 INFO:tasks.workunit.client.1.smithi043.stdout:9/776: rename d4/d1c/d4f to d4/d1c/d25/d2b/da1/df9 0 2024-09-16T23:49:06.412 INFO:tasks.workunit.client.1.smithi043.stdout:6/788: mkdir de/d23/d34/d9b/d5a/dfe/d10d 0 2024-09-16T23:49:06.413 INFO:tasks.workunit.client.1.smithi043.stdout:7/779: dwrite d4/db/d11/d21/d31/fc4 [0,4194304] 0 2024-09-16T23:49:06.416 INFO:tasks.workunit.client.1.smithi043.stdout:1/953: dwrite d4/d34/f3a [0,4194304] 0 2024-09-16T23:49:06.418 INFO:tasks.workunit.client.1.smithi043.stdout:0/865: dwrite d0/d2/df/d1a/f9e [4194304,4194304] 0 2024-09-16T23:49:06.424 INFO:tasks.workunit.client.1.smithi043.stdout:6/789: symlink de/d23/d34/d9b/d5a/l10e 0 2024-09-16T23:49:06.424 INFO:tasks.workunit.client.1.smithi043.stdout:9/777: dread d4/d1c/d25/d2b/da1/df9/f77 [0,4194304] 0 2024-09-16T23:49:06.424 INFO:tasks.workunit.client.1.smithi043.stdout:1/954: mknod d4/d9/d8f/db6/dc6/c146 0 2024-09-16T23:49:06.424 INFO:tasks.workunit.client.1.smithi043.stdout:0/866: dread - d0/d2/df/dbb/ff1 zero size 2024-09-16T23:49:06.426 INFO:tasks.workunit.client.1.smithi043.stdout:2/727: dwrite d8/fbe [0,4194304] 0 2024-09-16T23:49:06.435 INFO:tasks.workunit.client.1.smithi043.stdout:7/780: dread d4/db/d28/f4d [0,4194304] 0 2024-09-16T23:49:06.438 INFO:tasks.workunit.client.1.smithi043.stdout:2/728: fsync d8/d5e/d83/d9e/fa5 0 2024-09-16T23:49:06.446 INFO:tasks.workunit.client.1.smithi043.stdout:7/781: dwrite d4/db/d11/d15/fd9 [0,4194304] 0 2024-09-16T23:49:06.450 INFO:tasks.workunit.client.1.smithi043.stdout:2/729: dread d8/d11/d16/f19 [0,4194304] 0 2024-09-16T23:49:06.450 INFO:tasks.workunit.client.1.smithi043.stdout:9/778: dwrite d4/d1c/d25/d2b/da1/df9/f5b [0,4194304] 0 2024-09-16T23:49:06.453 INFO:tasks.workunit.client.1.smithi043.stdout:9/779: read - d4/d1c/fb5 zero size 2024-09-16T23:49:06.456 INFO:tasks.workunit.client.1.smithi043.stdout:0/867: dwrite d0/d2/df/d1a/d46/d39/f62 [0,4194304] 0 2024-09-16T23:49:06.461 INFO:tasks.workunit.client.1.smithi043.stdout:2/730: truncate d8/d5e/d83/d3a/d77/fe6 442464 0 2024-09-16T23:49:06.467 INFO:tasks.workunit.client.1.smithi043.stdout:1/955: dwrite d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fea [0,4194304] 0 2024-09-16T23:49:06.469 INFO:tasks.workunit.client.1.smithi043.stdout:7/782: dwrite d4/db/d11/d15/f47 [0,4194304] 0 2024-09-16T23:49:06.470 INFO:tasks.workunit.client.1.smithi043.stdout:2/731: dread - d8/d1d/d20/d23/d38/da2/fe9 zero size 2024-09-16T23:49:06.472 INFO:tasks.workunit.client.1.smithi043.stdout:9/780: dwrite d4/d1c/d6b/d67/fec [0,4194304] 0 2024-09-16T23:49:06.474 INFO:tasks.workunit.client.1.smithi043.stdout:0/868: truncate d0/d2/df/d1a/d46/d39/d6e/f65 1148114 0 2024-09-16T23:49:06.476 INFO:tasks.workunit.client.1.smithi043.stdout:0/869: write d0/f111 [1024660,107755] 0 2024-09-16T23:49:06.477 INFO:tasks.workunit.client.1.smithi043.stdout:2/732: symlink d8/d1d/d20/d26/lf9 0 2024-09-16T23:49:06.477 INFO:tasks.workunit.client.1.smithi043.stdout:1/956: fdatasync d4/d9/d12/f5c 0 2024-09-16T23:49:06.478 INFO:tasks.workunit.client.0.smithi042.stdout:0/549: sync 2024-09-16T23:49:06.479 INFO:tasks.workunit.client.1.smithi043.stdout:1/957: chown d4/d9/d1c/d1e/d42/d5d/d71/fed 272887 1 2024-09-16T23:49:06.480 INFO:tasks.workunit.client.1.smithi043.stdout:7/783: rename d4/cb0 to d4/db/d11/c103 0 2024-09-16T23:49:06.480 INFO:tasks.workunit.client.0.smithi042.stdout:9/521: getdents d4/d12/d16/d68 0 2024-09-16T23:49:06.482 INFO:tasks.workunit.client.1.smithi043.stdout:2/733: chown d8/d1d/d20/d23/cba 53931 1 2024-09-16T23:49:06.482 INFO:tasks.workunit.client.0.smithi042.stdout:0/550: stat d0/df/d23/f38 0 2024-09-16T23:49:06.483 INFO:tasks.workunit.client.0.smithi042.stdout:4/945: dwrite de/d42/d5d/d8f/dd6/d123/f139 [0,4194304] 0 2024-09-16T23:49:06.484 INFO:tasks.workunit.client.1.smithi043.stdout:0/870: creat d0/d2/df/d1a/d46/d39/d6e/f124 x:0 0 0 2024-09-16T23:49:06.486 INFO:tasks.workunit.client.0.smithi042.stdout:0/551: chown d0/f88 1095 1 2024-09-16T23:49:06.486 INFO:tasks.workunit.client.0.smithi042.stdout:4/946: write de/d10d/f36 [1941642,105662] 0 2024-09-16T23:49:06.487 INFO:tasks.workunit.client.1.smithi043.stdout:1/958: dwrite d4/f140 [0,4194304] 0 2024-09-16T23:49:06.490 INFO:tasks.workunit.client.0.smithi042.stdout:9/522: unlink d4/d10/d4b/c52 0 2024-09-16T23:49:06.492 INFO:tasks.workunit.client.1.smithi043.stdout:0/871: readlink d0/d2/df/l19 0 2024-09-16T23:49:06.493 INFO:tasks.workunit.client.1.smithi043.stdout:7/784: creat d4/db/f104 x:0 0 0 2024-09-16T23:49:06.494 INFO:tasks.workunit.client.1.smithi043.stdout:1/959: fdatasync d4/d9/d1c/d1e/d42/d5d/d71/d76/d8a/fa1 0 2024-09-16T23:49:06.495 INFO:tasks.workunit.client.0.smithi042.stdout:9/523: fsync d4/d10/f1f 0 2024-09-16T23:49:06.498 INFO:tasks.workunit.client.1.smithi043.stdout:7/785: chown d4/d27/d41/f8d 5011 1 2024-09-16T23:49:06.499 INFO:tasks.workunit.client.0.smithi042.stdout:4/947: dwrite de/d2e/d37/d4c/d4f/f6b [8388608,4194304] 0 2024-09-16T23:49:06.499 INFO:tasks.workunit.client.0.smithi042.stdout:0/552: dwrite d0/df/d23/d3e/faa [0,4194304] 0 2024-09-16T23:49:06.500 INFO:tasks.workunit.client.1.smithi043.stdout:1/960: fdatasync d4/d9/d12/fc2 0 2024-09-16T23:49:06.502 INFO:tasks.workunit.client.0.smithi042.stdout:4/948: chown de/d42/d5d/d8f/dd6/ffd 2959101 1 2024-09-16T23:49:06.504 INFO:tasks.workunit.client.1.smithi043.stdout:7/786: chown d4/d27/d41/l1d 2 1 2024-09-16T23:49:06.505 INFO:tasks.workunit.client.0.smithi042.stdout:9/524: mknod d4/d10/d24/ca6 0 2024-09-16T23:49:06.507 INFO:tasks.workunit.client.0.smithi042.stdout:0/553: mkdir d0/df/d13/d49/d70/da1/dac 0 2024-09-16T23:49:06.508 INFO:tasks.workunit.client.1.smithi043.stdout:1/961: unlink d4/lec 0 2024-09-16T23:49:06.508 INFO:tasks.workunit.client.0.smithi042.stdout:4/949: mkdir de/d1a/d142 0 2024-09-16T23:49:06.509 INFO:tasks.workunit.client.1.smithi043.stdout:1/962: write d4/d3e/fdb [3048365,63211] 0 2024-09-16T23:49:06.509 INFO:tasks.workunit.client.1.smithi043.stdout:0/872: unlink d0/d2/df/d1a/d49/d80/d88/l105 0 2024-09-16T23:49:06.510 INFO:tasks.workunit.client.0.smithi042.stdout:0/554: rmdir d0/df/d23/d3e/d8f 39 2024-09-16T23:49:06.516 INFO:tasks.workunit.client.0.smithi042.stdout:4/950: link de/d42/d5d/d8f/le2 de/d2e/d37/d4c/dfe/l143 0 2024-09-16T23:49:06.516 INFO:tasks.workunit.client.1.smithi043.stdout:7/787: creat d4/db/d11/d15/dbe/f105 x:0 0 0 2024-09-16T23:49:06.517 INFO:tasks.workunit.client.1.smithi043.stdout:2/734: dread d8/d5e/d83/f59 [0,4194304] 0 2024-09-16T23:49:06.518 INFO:tasks.workunit.client.0.smithi042.stdout:4/951: fsync de/d2e/d37/d4c/d4f/f58 0 2024-09-16T23:49:06.519 INFO:tasks.workunit.client.0.smithi042.stdout:9/525: rename d4/d12/d16/d81 to d4/d12/d2d/d37/d41/d92/da7 0 2024-09-16T23:49:06.520 INFO:tasks.workunit.client.1.smithi043.stdout:1/963: dwrite d4/d3e/d5b/f115 [0,4194304] 0 2024-09-16T23:49:06.521 INFO:tasks.workunit.client.1.smithi043.stdout:0/873: mknod d0/d2/df/d1a/d49/db2/df4/c125 0 2024-09-16T23:49:06.523 INFO:tasks.workunit.client.0.smithi042.stdout:4/952: symlink de/l144 0 2024-09-16T23:49:06.527 INFO:tasks.workunit.client.1.smithi043.stdout:0/874: chown d0/d2/df/d1a/d46/d39/d6e/fbc 1418 1 2024-09-16T23:49:06.527 INFO:tasks.workunit.client.1.smithi043.stdout:7/788: mknod d4/db/d11/d21/d31/c106 0 2024-09-16T23:49:06.527 INFO:tasks.workunit.client.1.smithi043.stdout:2/735: dread d8/d1f/f5c [0,4194304] 0 2024-09-16T23:49:06.528 INFO:tasks.workunit.client.0.smithi042.stdout:9/526: rename d4/d12/d77/c9a to d4/d12/d44/d8e/da1/ca8 0 2024-09-16T23:49:06.532 INFO:tasks.workunit.client.0.smithi042.stdout:9/527: read d4/d12/d2d/d37/d28/f87 [132887,61321] 0 2024-09-16T23:49:06.533 INFO:tasks.workunit.client.0.smithi042.stdout:4/953: truncate de/d42/d5d/f8c 3933853 0 2024-09-16T23:49:06.534 INFO:tasks.workunit.client.1.smithi043.stdout:0/875: creat d0/d2/df/d1a/d49/d80/f126 x:0 0 0 2024-09-16T23:49:06.536 INFO:tasks.workunit.client.0.smithi042.stdout:4/954: creat de/d42/d85/db3/f145 x:0 0 0 2024-09-16T23:49:06.537 INFO:tasks.workunit.client.1.smithi043.stdout:0/876: chown d0/d2/f33 888597996 1 2024-09-16T23:49:06.538 INFO:tasks.workunit.client.1.smithi043.stdout:1/964: link d4/d3e/fc7 d4/d3e/d95/f147 0 2024-09-16T23:49:06.539 INFO:tasks.workunit.client.0.smithi042.stdout:4/955: mknod de/d1a/d70/dac/c146 0 2024-09-16T23:49:06.540 INFO:tasks.workunit.client.1.smithi043.stdout:1/965: chown d4/df/c3c 1314715 1 2024-09-16T23:49:06.541 INFO:tasks.workunit.client.1.smithi043.stdout:7/789: link d4/db/d11/d15/fd9 d4/f107 0 2024-09-16T23:49:06.541 INFO:tasks.workunit.client.1.smithi043.stdout:1/966: chown d4/df/df4/dc8 0 1 2024-09-16T23:49:06.541 INFO:tasks.workunit.client.0.smithi042.stdout:9/528: getdents d4/d12/d2d/d37/d41/d92 0 2024-09-16T23:49:06.541 INFO:tasks.workunit.client.1.smithi043.stdout:0/877: mkdir d0/d2/df/d1a/d46/d39/d127 0 2024-09-16T23:49:06.543 INFO:tasks.workunit.client.1.smithi043.stdout:1/967: truncate d4/df/df4/dc8/f119 800656 0 2024-09-16T23:49:06.544 INFO:tasks.workunit.client.1.smithi043.stdout:0/878: chown d0/d2/df/d1a/d46/d39/f6a 3819 1 2024-09-16T23:49:06.546 INFO:tasks.workunit.client.1.smithi043.stdout:2/736: dread d8/d11/f6f [0,4194304] 0 2024-09-16T23:49:06.546 INFO:tasks.workunit.client.0.smithi042.stdout:4/956: dwrite de/d2e/d37/d4c/d4f/f7a [0,4194304] 0 2024-09-16T23:49:06.547 INFO:tasks.workunit.client.0.smithi042.stdout:9/529: rename d4/d12/d2d/d37/d3a/d50/d6f to d4/d12/d16/da9 0 2024-09-16T23:49:06.549 INFO:tasks.workunit.client.1.smithi043.stdout:1/968: symlink d4/d9/d12/d15/dac/l148 0 2024-09-16T23:49:06.549 INFO:tasks.workunit.client.0.smithi042.stdout:4/957: mkdir de/d147 0 2024-09-16T23:49:06.555 INFO:tasks.workunit.client.0.smithi042.stdout:4/958: rename de/d12/f25 to de/d1a/d70/f148 0 2024-09-16T23:49:06.560 INFO:tasks.workunit.client.0.smithi042.stdout:4/959: dread de/d42/d99/d9e/fd7 [0,4194304] 0 2024-09-16T23:49:06.561 INFO:tasks.workunit.client.1.smithi043.stdout:0/879: dread d0/d2/df/da7/db8/f60 [0,4194304] 0 2024-09-16T23:49:06.561 INFO:tasks.workunit.client.1.smithi043.stdout:1/969: rename d4/fc to d4/df/df4/d83/d98/d137/f149 0 2024-09-16T23:49:06.563 INFO:tasks.workunit.client.1.smithi043.stdout:7/790: dwrite d4/db/d43/d83/fba [0,4194304] 0 2024-09-16T23:49:06.563 INFO:tasks.workunit.client.1.smithi043.stdout:2/737: truncate d8/d35/f37 698211 0 2024-09-16T23:49:06.564 INFO:tasks.workunit.client.1.smithi043.stdout:7/791: chown d4/db/d11/d15/dbe 58 1 2024-09-16T23:49:06.565 INFO:tasks.workunit.client.0.smithi042.stdout:4/960: rename de/d42/d5d/d8f/dd6/f12d to de/d1a/d70/dac/f149 0 2024-09-16T23:49:06.566 INFO:tasks.workunit.client.1.smithi043.stdout:2/738: chown d8/d11/d16/f81 35743683 1 2024-09-16T23:49:06.571 INFO:tasks.workunit.client.1.smithi043.stdout:1/970: fdatasync d4/d3e/fc7 0 2024-09-16T23:49:06.572 INFO:tasks.workunit.client.1.smithi043.stdout:1/971: dread - d4/df/df4/d83/d98/f125 zero size 2024-09-16T23:49:06.572 INFO:tasks.workunit.client.1.smithi043.stdout:7/792: symlink d4/db/d11/d15/d29/d7a/l108 0 2024-09-16T23:49:06.573 INFO:tasks.workunit.client.1.smithi043.stdout:1/972: stat d4/d9/d1c/d1e/d42/d5d 0 2024-09-16T23:49:06.574 INFO:tasks.workunit.client.1.smithi043.stdout:7/793: write d4/d27/f3f [4811595,23244] 0 2024-09-16T23:49:06.574 INFO:tasks.workunit.client.1.smithi043.stdout:2/739: mknod d8/cfa 0 2024-09-16T23:49:06.576 INFO:tasks.workunit.client.1.smithi043.stdout:7/794: dread - d4/db/d11/d21/d31/d65/fdf zero size 2024-09-16T23:49:06.577 INFO:tasks.workunit.client.1.smithi043.stdout:2/740: fsync d8/d1d/d20/d26/f7d 0 2024-09-16T23:49:06.585 INFO:tasks.workunit.client.1.smithi043.stdout:0/880: write d0/d2/df/fbf [165638,130525] 0 2024-09-16T23:49:06.586 INFO:tasks.workunit.client.1.smithi043.stdout:7/795: rmdir d4/db/d43/d83 39 2024-09-16T23:49:06.589 INFO:tasks.workunit.client.1.smithi043.stdout:7/796: creat d4/db/d50/f109 x:0 0 0 2024-09-16T23:49:06.590 INFO:tasks.workunit.client.1.smithi043.stdout:1/973: link d4/d9/d8f/db6/dcc/dda/d107/ddf/lef d4/df/df4/dc8/l14a 0 2024-09-16T23:49:06.594 INFO:tasks.workunit.client.1.smithi043.stdout:0/881: dwrite d0/d2/df/d1a/de8/fed [4194304,4194304] 0 2024-09-16T23:49:06.596 INFO:tasks.workunit.client.1.smithi043.stdout:2/741: link d8/d1d/d20/d23/d38/le3 d8/d1d/d20/d23/lfb 0 2024-09-16T23:49:06.599 INFO:tasks.workunit.client.1.smithi043.stdout:1/974: rename d4/d9/d8f/db6/dcc/dda/d107/c139 to d4/d3e/c14b 0 2024-09-16T23:49:06.602 INFO:tasks.workunit.client.1.smithi043.stdout:2/742: mkdir d8/d1d/d20/d23/dfc 0 2024-09-16T23:49:06.602 INFO:tasks.workunit.client.1.smithi043.stdout:1/975: dread - d4/d3e/d5b/dca/d12a/f145 zero size 2024-09-16T23:49:06.602 INFO:tasks.workunit.client.1.smithi043.stdout:7/797: dwrite d4/db/d43/d83/d85/fc2 [4194304,4194304] 0 2024-09-16T23:49:06.605 INFO:tasks.workunit.client.1.smithi043.stdout:1/976: write d4/d9/d1c/d1e/d42/d5d/f93 [4344054,25350] 0 2024-09-16T23:49:06.606 INFO:tasks.workunit.client.1.smithi043.stdout:2/743: dread d8/d1d/d25/fbd [0,4194304] 0 2024-09-16T23:49:06.609 INFO:tasks.workunit.client.1.smithi043.stdout:0/882: getdents d0/d2/df/d1a/d46/d92 0 2024-09-16T23:49:06.610 INFO:tasks.workunit.client.1.smithi043.stdout:2/744: creat d8/d5e/ffd x:0 0 0 2024-09-16T23:49:06.611 INFO:tasks.workunit.client.1.smithi043.stdout:1/977: symlink d4/l14c 0 2024-09-16T23:49:06.612 INFO:tasks.workunit.client.1.smithi043.stdout:2/745: symlink d8/d35/dc0/lfe 0 2024-09-16T23:49:06.613 INFO:tasks.workunit.client.1.smithi043.stdout:0/883: creat d0/d2/df/d1a/d46/d92/d10a/f128 x:0 0 0 2024-09-16T23:49:06.614 INFO:tasks.workunit.client.1.smithi043.stdout:1/978: creat d4/d9/d12/d2b/dc9/f14d x:0 0 0 2024-09-16T23:49:06.616 INFO:tasks.workunit.client.1.smithi043.stdout:0/884: mknod d0/d2/df/d1a/d46/d39/daa/db7/c129 0 2024-09-16T23:49:06.619 INFO:tasks.workunit.client.1.smithi043.stdout:0/885: rmdir d0/d2/df/dbb 39 2024-09-16T23:49:06.621 INFO:tasks.workunit.client.1.smithi043.stdout:1/979: dwrite d4/f140 [0,4194304] 0 2024-09-16T23:49:06.621 INFO:tasks.workunit.client.1.smithi043.stdout:2/746: getdents d8/d11 0 2024-09-16T23:49:06.621 INFO:tasks.workunit.client.1.smithi043.stdout:0/886: dread - d0/d2/df/da7/db8/fd9 zero size 2024-09-16T23:49:06.657 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:06 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:06.657 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:06 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:06.657 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:06 smithi042 ceph-mon[27914]: pgmap v12: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 122 MiB/s rd, 349 MiB/s wr, 687 op/s 2024-09-16T23:49:06.739 INFO:tasks.workunit.client.1.smithi043.stdout:7/798: dread d4/db/d28/f97 [0,4194304] 0 2024-09-16T23:49:06.740 INFO:tasks.workunit.client.1.smithi043.stdout:1/980: write d4/df/d26/dd5/f135 [768342,27355] 0 2024-09-16T23:49:06.740 INFO:tasks.workunit.client.0.smithi042.stdout:7/719: sync 2024-09-16T23:49:06.741 INFO:tasks.workunit.client.0.smithi042.stdout:3/890: sync 2024-09-16T23:49:06.741 INFO:tasks.workunit.client.1.smithi043.stdout:1/981: chown d4/d9/d1c/d1e/l130 57 1 2024-09-16T23:49:06.742 INFO:tasks.workunit.client.0.smithi042.stdout:3/891: fdatasync d2/d1b/d1d/f38 0 2024-09-16T23:49:06.743 INFO:tasks.workunit.client.1.smithi043.stdout:0/887: creat d0/d2/df/d1a/d46/d39/daa/db7/f12a x:0 0 0 2024-09-16T23:49:06.744 INFO:tasks.workunit.client.1.smithi043.stdout:7/799: symlink d4/db/d11/d21/d94/l10a 0 2024-09-16T23:49:06.748 INFO:tasks.workunit.client.1.smithi043.stdout:7/800: mknod d4/db/d11/d15/d29/d7a/c10b 0 2024-09-16T23:49:06.748 INFO:tasks.workunit.client.1.smithi043.stdout:0/888: symlink d0/d2/df/d1a/d49/db2/df4/l12b 0 2024-09-16T23:49:06.749 INFO:tasks.workunit.client.0.smithi042.stdout:3/892: creat d2/d1b/d1d/f134 x:0 0 0 2024-09-16T23:49:06.751 INFO:tasks.workunit.client.1.smithi043.stdout:1/982: dwrite d4/d3e/d5b/dca/d12a/f145 [0,4194304] 0 2024-09-16T23:49:06.754 INFO:tasks.workunit.client.1.smithi043.stdout:2/747: dwrite d8/d1d/d20/d23/d38/f66 [0,4194304] 0 2024-09-16T23:49:06.756 INFO:tasks.workunit.client.0.smithi042.stdout:7/720: dread d3/d1f/d49/d76/d19/d63/f70 [0,4194304] 0 2024-09-16T23:49:06.757 INFO:tasks.workunit.client.0.smithi042.stdout:3/893: link d2/d1b/d1d/d34/d50/df5/d12b/l12e d2/d7/d61/d8e/d9d/dc4/de4/l135 0 2024-09-16T23:49:06.758 INFO:tasks.workunit.client.1.smithi043.stdout:7/801: truncate d4/d27/d41/d1b/f90 866417 0 2024-09-16T23:49:06.760 INFO:tasks.workunit.client.1.smithi043.stdout:1/983: creat d4/d9/d8f/db6/f14e x:0 0 0 2024-09-16T23:49:06.760 INFO:tasks.workunit.client.1.smithi043.stdout:2/748: symlink d8/d5e/d83/d3a/d77/da9/lff 0 2024-09-16T23:49:06.761 INFO:tasks.workunit.client.0.smithi042.stdout:3/894: mknod d2/d1b/d1d/d34/d50/d93/c136 0 2024-09-16T23:49:06.761 INFO:tasks.workunit.client.0.smithi042.stdout:7/721: dread d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/fb4 [0,4194304] 0 2024-09-16T23:49:06.764 INFO:tasks.workunit.client.1.smithi043.stdout:1/984: symlink d4/d9/d1c/d1e/d132/l14f 0 2024-09-16T23:49:06.765 INFO:tasks.workunit.client.1.smithi043.stdout:1/985: fdatasync d4/f140 0 2024-09-16T23:49:06.765 INFO:tasks.workunit.client.1.smithi043.stdout:7/802: fsync d4/d27/d41/f26 0 2024-09-16T23:49:06.766 INFO:tasks.workunit.client.0.smithi042.stdout:7/722: creat d3/d1f/d49/d76/d19/ddc/d6b/d5b/ffc x:0 0 0 2024-09-16T23:49:06.766 INFO:tasks.workunit.client.0.smithi042.stdout:3/895: creat d2/d1b/d1d/d30/da2/f137 x:0 0 0 2024-09-16T23:49:06.767 INFO:tasks.workunit.client.1.smithi043.stdout:7/803: fdatasync d4/db/d43/d83/d85/fc2 0 2024-09-16T23:49:06.768 INFO:tasks.workunit.client.0.smithi042.stdout:3/896: dread - d2/d1b/d1d/d30/fc0 zero size 2024-09-16T23:49:06.770 INFO:tasks.workunit.client.0.smithi042.stdout:3/897: fdatasync d2/d7/d61/d8e/d9d/dc4/de4/fec 0 2024-09-16T23:49:06.770 INFO:tasks.workunit.client.1.smithi043.stdout:2/749: getdents d8/d1d/d25 0 2024-09-16T23:49:06.773 INFO:tasks.workunit.client.1.smithi043.stdout:1/986: mknod d4/d9/d8f/db6/dcc/dda/d106/d10d/c150 0 2024-09-16T23:49:06.774 INFO:tasks.workunit.client.1.smithi043.stdout:0/889: truncate d0/d2/df/d1a/de8/fed 2874405 0 2024-09-16T23:49:06.776 INFO:tasks.workunit.client.1.smithi043.stdout:0/890: stat d0/d2/df/d1a/d46/l85 0 2024-09-16T23:49:06.776 INFO:tasks.workunit.client.1.smithi043.stdout:0/891: stat d0/d2/df/d1a/d49 0 2024-09-16T23:49:06.778 INFO:tasks.workunit.client.1.smithi043.stdout:2/750: creat d8/d5e/d83/d3a/d77/da9/f100 x:0 0 0 2024-09-16T23:49:06.779 INFO:tasks.workunit.client.1.smithi043.stdout:1/987: creat d4/d9/d1c/d1e/d42/d5d/d71/f151 x:0 0 0 2024-09-16T23:49:06.780 INFO:tasks.workunit.client.0.smithi042.stdout:7/723: dread d3/d98/f8f [0,4194304] 0 2024-09-16T23:49:06.781 INFO:tasks.workunit.client.1.smithi043.stdout:5/920: sync 2024-09-16T23:49:06.785 INFO:tasks.workunit.client.0.smithi042.stdout:7/724: creat d3/d1f/d49/d76/d19/ddc/dcd/ffd x:0 0 0 2024-09-16T23:49:06.786 INFO:tasks.workunit.client.1.smithi043.stdout:5/921: write d3/d9/d22/d9f/fc6 [973262,28284] 0 2024-09-16T23:49:06.787 INFO:tasks.workunit.client.1.smithi043.stdout:0/892: mkdir d0/d2/df/dbb/d12c 0 2024-09-16T23:49:06.789 INFO:tasks.workunit.client.1.smithi043.stdout:1/988: creat d4/df/df4/f152 x:0 0 0 2024-09-16T23:49:06.790 INFO:tasks.workunit.client.1.smithi043.stdout:2/751: write d8/d4b/d4f/d7c/fad [559196,94611] 0 2024-09-16T23:49:06.793 INFO:tasks.workunit.client.0.smithi042.stdout:3/898: dwrite d2/d1b/f86 [0,4194304] 0 2024-09-16T23:49:06.793 INFO:tasks.workunit.client.1.smithi043.stdout:5/922: creat d3/f138 x:0 0 0 2024-09-16T23:49:06.795 INFO:tasks.workunit.client.1.smithi043.stdout:7/804: dwrite d4/d27/d41/d1b/d66/feb [0,4194304] 0 2024-09-16T23:49:06.795 INFO:tasks.workunit.client.1.smithi043.stdout:2/752: mknod d8/d4b/d4f/d7c/c101 0 2024-09-16T23:49:06.798 INFO:tasks.workunit.client.0.smithi042.stdout:3/899: creat d2/d1b/d1d/f138 x:0 0 0 2024-09-16T23:49:06.800 INFO:tasks.workunit.client.0.smithi042.stdout:3/900: write d2/d7/d61/fc5 [714074,77859] 0 2024-09-16T23:49:06.800 INFO:tasks.workunit.client.0.smithi042.stdout:7/725: dread d3/d1f/d49/f56 [0,4194304] 0 2024-09-16T23:49:06.800 INFO:tasks.workunit.client.1.smithi043.stdout:0/893: creat d0/d2/df/d1a/d46/d39/d83/f12d x:0 0 0 2024-09-16T23:49:06.801 INFO:tasks.workunit.client.1.smithi043.stdout:5/923: dread - d3/db/d15/d43/fec zero size 2024-09-16T23:49:06.802 INFO:tasks.workunit.client.1.smithi043.stdout:1/989: dwrite d4/d34/fd2 [0,4194304] 0 2024-09-16T23:49:06.802 INFO:tasks.workunit.client.0.smithi042.stdout:3/901: chown d2/d7/d61/d8e/d9d/dc4/dcc/l106 60 1 2024-09-16T23:49:06.803 INFO:tasks.workunit.client.0.smithi042.stdout:3/902: write d2/d1b/f75 [636150,99426] 0 2024-09-16T23:49:06.805 INFO:tasks.workunit.client.0.smithi042.stdout:3/903: fsync d2/d7/d61/d8e/d9d/dc4/ff7 0 2024-09-16T23:49:06.807 INFO:tasks.workunit.client.0.smithi042.stdout:3/904: chown d2/d1b/d1d/f134 784103845 1 2024-09-16T23:49:06.807 INFO:tasks.workunit.client.1.smithi043.stdout:7/805: dwrite d4/db/d43/d83/d85/f42 [4194304,4194304] 0 2024-09-16T23:49:06.808 INFO:tasks.workunit.client.0.smithi042.stdout:7/726: getdents d3/d1f/d49/d76/d19/d63 0 2024-09-16T23:49:06.808 INFO:tasks.workunit.client.1.smithi043.stdout:1/990: mkdir d4/d3e/d5b/d153 0 2024-09-16T23:49:06.808 INFO:tasks.workunit.client.0.smithi042.stdout:3/905: readlink d2/d1b/la7 0 2024-09-16T23:49:06.810 INFO:tasks.workunit.client.1.smithi043.stdout:5/924: mknod d3/db/d15/c139 0 2024-09-16T23:49:06.815 INFO:tasks.workunit.client.1.smithi043.stdout:7/806: mkdir d4/db/d28/d10c 0 2024-09-16T23:49:06.817 INFO:tasks.workunit.client.1.smithi043.stdout:1/991: dwrite d4/d3e/d5b/dca/d12a/f145 [0,4194304] 0 2024-09-16T23:49:06.818 INFO:tasks.workunit.client.0.smithi042.stdout:7/727: dwrite d3/d1f/d49/d76/d19/ddc/dcd/fcb [0,4194304] 0 2024-09-16T23:49:06.821 INFO:tasks.workunit.client.1.smithi043.stdout:1/992: truncate d4/d9/d1c/d1e/d42/d5d/d110/ffe 632537 0 2024-09-16T23:49:06.821 INFO:tasks.workunit.client.1.smithi043.stdout:5/925: truncate d3/db/d19/d3e/f70 3241582 0 2024-09-16T23:49:06.821 INFO:tasks.workunit.client.1.smithi043.stdout:0/894: link d0/d2/cdc d0/d2/df/d1a/d46/d11d/c12e 0 2024-09-16T23:49:06.822 INFO:tasks.workunit.client.0.smithi042.stdout:7/728: symlink d3/d3f/lfe 0 2024-09-16T23:49:06.824 INFO:tasks.workunit.client.1.smithi043.stdout:7/807: creat d4/db/d11/d15/d29/d7a/f10d x:0 0 0 2024-09-16T23:49:06.825 INFO:tasks.workunit.client.1.smithi043.stdout:0/895: symlink d0/de/l12f 0 2024-09-16T23:49:06.826 INFO:tasks.workunit.client.0.smithi042.stdout:7/729: fsync d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/fb4 0 2024-09-16T23:49:06.828 INFO:tasks.workunit.client.0.smithi042.stdout:7/730: dread - d3/f9c zero size 2024-09-16T23:49:06.829 INFO:tasks.workunit.client.0.smithi042.stdout:3/906: truncate d2/d1b/d1d/d34/d50/d93/f117 565164 0 2024-09-16T23:49:06.832 INFO:tasks.workunit.client.0.smithi042.stdout:7/731: symlink d3/d1f/d49/d76/d19/ddc/d6b/d64/lff 0 2024-09-16T23:49:06.833 INFO:tasks.workunit.client.0.smithi042.stdout:7/732: write d3/fd1 [1027189,29789] 0 2024-09-16T23:49:06.834 INFO:tasks.workunit.client.0.smithi042.stdout:3/907: dread d2/d1b/d1d/d34/fbd [0,4194304] 0 2024-09-16T23:49:06.837 INFO:tasks.workunit.client.0.smithi042.stdout:7/733: truncate d3/d1f/d49/d76/d19/fe0 977296 0 2024-09-16T23:49:06.839 INFO:tasks.workunit.client.1.smithi043.stdout:1/993: write d4/d9/d12/d15/dac/f103 [131535,105464] 0 2024-09-16T23:49:06.839 INFO:tasks.workunit.client.0.smithi042.stdout:7/734: fdatasync d3/f9a 0 2024-09-16T23:49:06.840 INFO:tasks.workunit.client.1.smithi043.stdout:5/926: write d3/db/d19/d3e/d28/dcb/f106 [2452056,72871] 0 2024-09-16T23:49:06.841 INFO:tasks.workunit.client.1.smithi043.stdout:1/994: stat d4/d9/d1c/d1e/d42/d5d/d120/l11b 0 2024-09-16T23:49:06.841 INFO:tasks.workunit.client.0.smithi042.stdout:3/908: rename d2/d1b/d1d/d34/d50/l7a to d2/d1b/d1d/d34/d50/de6/l139 0 2024-09-16T23:49:06.842 INFO:tasks.workunit.client.1.smithi043.stdout:2/753: dread d8/d35/f36 [0,4194304] 0 2024-09-16T23:49:06.846 INFO:tasks.workunit.client.0.smithi042.stdout:3/909: creat d2/d9/f13a x:0 0 0 2024-09-16T23:49:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:3/910: truncate d2/d7/d61/d8e/d9d/dc4/dcc/f113 2784832 0 2024-09-16T23:49:06.850 INFO:tasks.workunit.client.0.smithi042.stdout:7/735: dwrite d3/d98/f8f [0,4194304] 0 2024-09-16T23:49:06.852 INFO:tasks.workunit.client.0.smithi042.stdout:3/911: mkdir d2/d1b/d1d/d71/d13b 0 2024-09-16T23:49:06.854 INFO:tasks.workunit.client.0.smithi042.stdout:3/912: write d2/d1b/d1d/d71/fd1 [470727,43194] 0 2024-09-16T23:49:06.855 INFO:tasks.workunit.client.0.smithi042.stdout:7/736: getdents d3/d1f/d49/d76/d19/ddc/d6b/d64/d92/dfb 0 2024-09-16T23:49:06.858 INFO:tasks.workunit.client.0.smithi042.stdout:7/737: creat d3/d98/da0/d41/f100 x:0 0 0 2024-09-16T23:49:06.859 INFO:tasks.workunit.client.0.smithi042.stdout:3/913: mknod d2/d1b/d1d/d34/d62/c13c 0 2024-09-16T23:49:06.865 INFO:tasks.workunit.client.0.smithi042.stdout:7/738: rename d3/d1f/d49/d76/d19/ddc/d6b to d3/d1f/d49/d76/d101 0 2024-09-16T23:49:06.866 INFO:tasks.workunit.client.0.smithi042.stdout:7/739: read - d3/f9c zero size 2024-09-16T23:49:06.866 INFO:tasks.workunit.client.1.smithi043.stdout:1/995: unlink d4/df/d26/d79/fb9 0 2024-09-16T23:49:06.868 INFO:tasks.workunit.client.0.smithi042.stdout:3/914: dwrite d2/d7/d61/d8e/d9d/dc4/ff7 [0,4194304] 0 2024-09-16T23:49:06.870 INFO:tasks.workunit.client.1.smithi043.stdout:0/896: mknod d0/d2/df/d1a/d46/d11d/c130 0 2024-09-16T23:49:06.870 INFO:tasks.workunit.client.0.smithi042.stdout:8/530: sync 2024-09-16T23:49:06.871 INFO:tasks.workunit.client.1.smithi043.stdout:7/808: creat d4/db/d11/f10e x:0 0 0 2024-09-16T23:49:06.873 INFO:tasks.workunit.client.1.smithi043.stdout:1/996: symlink d4/d34/l154 0 2024-09-16T23:49:06.873 INFO:tasks.workunit.client.0.smithi042.stdout:3/915: creat d2/f13d x:0 0 0 2024-09-16T23:49:06.873 INFO:tasks.workunit.client.1.smithi043.stdout:7/809: readlink d4/db/d11/d15/d29/lf9 0 2024-09-16T23:49:06.874 INFO:tasks.workunit.client.0.smithi042.stdout:7/740: getdents d3/d98/da0/d24/d28/d9b 0 2024-09-16T23:49:06.874 INFO:tasks.workunit.client.1.smithi043.stdout:7/810: readlink d4/db/d11/d15/l1c 0 2024-09-16T23:49:06.875 INFO:tasks.workunit.client.1.smithi043.stdout:7/811: chown c3 12 1 2024-09-16T23:49:06.876 INFO:tasks.workunit.client.0.smithi042.stdout:8/531: mknod da/d19/d1d/d30/ca2 0 2024-09-16T23:49:06.876 INFO:tasks.workunit.client.0.smithi042.stdout:7/741: creat d3/d1f/d49/d78/f102 x:0 0 0 2024-09-16T23:49:06.877 INFO:tasks.workunit.client.1.smithi043.stdout:1/997: creat d4/d3e/d5b/dca/d12a/f155 x:0 0 0 2024-09-16T23:49:06.877 INFO:tasks.workunit.client.0.smithi042.stdout:7/742: dread - d3/d1f/d49/d78/f102 zero size 2024-09-16T23:49:06.879 INFO:tasks.workunit.client.0.smithi042.stdout:7/743: creat d3/d98/da0/d41/f103 x:0 0 0 2024-09-16T23:49:06.880 INFO:tasks.workunit.client.0.smithi042.stdout:7/744: chown d3/d1f/l3b 4 1 2024-09-16T23:49:06.882 INFO:tasks.workunit.client.1.smithi043.stdout:0/897: link d0/d2/df/d1a/d49/d80/f126 d0/d2/df/d1a/d49/db2/df4/f131 0 2024-09-16T23:49:06.883 INFO:tasks.workunit.client.0.smithi042.stdout:8/532: symlink da/d19/la3 0 2024-09-16T23:49:06.883 INFO:tasks.workunit.client.0.smithi042.stdout:7/745: creat d3/d1f/d49/d76/d101/da5/f104 x:0 0 0 2024-09-16T23:49:06.884 INFO:tasks.workunit.client.0.smithi042.stdout:3/916: truncate d2/d1b/d1d/d34/d62/f9e 3457562 0 2024-09-16T23:49:06.885 INFO:tasks.workunit.client.0.smithi042.stdout:7/746: creat d3/d1f/d4f/f105 x:0 0 0 2024-09-16T23:49:06.888 INFO:tasks.workunit.client.0.smithi042.stdout:7/747: creat d3/d1f/d49/d76/d101/da5/f106 x:0 0 0 2024-09-16T23:49:06.893 INFO:tasks.workunit.client.1.smithi043.stdout:3/818: sync 2024-09-16T23:49:06.893 INFO:tasks.workunit.client.1.smithi043.stdout:1/998: rename d4/d9/d12/d15/f104 to d4/d9/d1c/d1e/d42/d5d/d71/d76/f156 0 2024-09-16T23:49:06.893 INFO:tasks.workunit.client.1.smithi043.stdout:1/999: stat d4/d9/d12/d2b 0 2024-09-16T23:49:06.894 INFO:tasks.workunit.client.1.smithi043.stdout:0/898: symlink d0/d2/df/d1a/d49/db2/l132 0 2024-09-16T23:49:06.895 INFO:tasks.workunit.client.0.smithi042.stdout:7/748: link d3/d98/da0/c4e d3/d1f/d49/d76/d19/ddc/c107 0 2024-09-16T23:49:06.897 INFO:tasks.workunit.client.0.smithi042.stdout:3/917: dwrite d2/d7/d61/d73/f8f [0,4194304] 0 2024-09-16T23:49:06.899 INFO:tasks.workunit.client.0.smithi042.stdout:3/918: dread - d2/d7/d61/d8e/d7f/f82 zero size 2024-09-16T23:49:06.899 INFO:tasks.workunit.client.0.smithi042.stdout:7/749: creat d3/d98/da0/d24/d28/d9b/f108 x:0 0 0 2024-09-16T23:49:06.900 INFO:tasks.workunit.client.1.smithi043.stdout:3/819: write d4/d30/fe9 [975209,95750] 0 2024-09-16T23:49:06.900 INFO:tasks.workunit.client.0.smithi042.stdout:3/919: chown d2/d1b/d1d/d34/d50/f6c 2112 1 2024-09-16T23:49:06.901 INFO:tasks.workunit.client.0.smithi042.stdout:7/750: fdatasync d3/d1f/d49/d76/d101/d5b/ffc 0 2024-09-16T23:49:06.902 INFO:tasks.workunit.client.0.smithi042.stdout:3/920: fsync d2/d1b/d1d/f38 0 2024-09-16T23:49:06.903 INFO:tasks.workunit.client.0.smithi042.stdout:3/921: chown d2/d1b/d1d/d34/f5d 715639488 1 2024-09-16T23:49:06.905 INFO:tasks.workunit.client.0.smithi042.stdout:3/922: stat d2/d1b/d1d/d34/d62/d6a/fe0 0 2024-09-16T23:49:06.905 INFO:tasks.workunit.client.0.smithi042.stdout:7/751: rmdir d3/d98/da0/d41 39 2024-09-16T23:49:06.906 INFO:tasks.workunit.client.1.smithi043.stdout:0/899: unlink d0/d2/df/d1a/d46/l8e 0 2024-09-16T23:49:06.910 INFO:tasks.workunit.client.1.smithi043.stdout:5/927: truncate d3/db/d19/d3e/ff1 592129 0 2024-09-16T23:49:06.911 INFO:tasks.workunit.client.0.smithi042.stdout:3/923: unlink d2/d1b/d1d/d34/d50/d93/d96/fad 0 2024-09-16T23:49:06.911 INFO:tasks.workunit.client.1.smithi043.stdout:0/900: rename d0/d2/df/d1a/d46/ca6 to d0/d2/df/d1a/d49/db2/df4/d9c/dfa/c133 0 2024-09-16T23:49:06.914 INFO:tasks.workunit.client.1.smithi043.stdout:3/820: dwrite d4/de/d1f/d2c/db7/dc9/fe3 [0,4194304] 0 2024-09-16T23:49:06.916 INFO:tasks.workunit.client.1.smithi043.stdout:3/821: write d4/fcd [2221542,20582] 0 2024-09-16T23:49:06.918 INFO:tasks.workunit.client.0.smithi042.stdout:3/924: write d2/d1b/d89/f42 [4238647,74457] 0 2024-09-16T23:49:06.919 INFO:tasks.workunit.client.1.smithi043.stdout:3/822: read - d4/de/d1f/fdb zero size 2024-09-16T23:49:06.919 INFO:tasks.workunit.client.0.smithi042.stdout:3/925: dread - d2/d1b/d1d/d30/da2/f137 zero size 2024-09-16T23:49:06.924 INFO:tasks.workunit.client.1.smithi043.stdout:0/901: link d0/d2/df/d1a/d49/d80/d88/f10d d0/d2/df/d1a/d49/d80/d88/dc4/f134 0 2024-09-16T23:49:06.924 INFO:tasks.workunit.client.1.smithi043.stdout:2/754: dwrite d8/d35/f36 [0,4194304] 0 2024-09-16T23:49:06.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/823: rmdir d4/de/d40/df0/d88/db1/dbb 39 2024-09-16T23:49:06.927 INFO:tasks.workunit.client.1.smithi043.stdout:7/812: truncate d4/db/d11/d15/f47 656365 0 2024-09-16T23:49:06.929 INFO:tasks.workunit.client.1.smithi043.stdout:7/813: dread - d4/db/d11/d21/d31/f6b zero size 2024-09-16T23:49:06.929 INFO:tasks.workunit.client.1.smithi043.stdout:5/928: dread d3/db/d15/f5f [0,4194304] 0 2024-09-16T23:49:06.931 INFO:tasks.workunit.client.1.smithi043.stdout:7/814: write d4/db/d43/d83/ffa [156488,18938] 0 2024-09-16T23:49:06.935 INFO:tasks.workunit.client.0.smithi042.stdout:3/926: dread d2/d1b/d1d/d34/d50/f6c [0,4194304] 0 2024-09-16T23:49:06.936 INFO:tasks.workunit.client.1.smithi043.stdout:5/929: creat d3/d9/d22/d4a/f13a x:0 0 0 2024-09-16T23:49:06.936 INFO:tasks.workunit.client.0.smithi042.stdout:8/533: dwrite da/d19/d1d/f50 [0,4194304] 0 2024-09-16T23:49:06.936 INFO:tasks.workunit.client.0.smithi042.stdout:3/927: stat d2/d1b/d89/lcb 0 2024-09-16T23:49:06.937 INFO:tasks.workunit.client.0.smithi042.stdout:8/534: chown da 578754 1 2024-09-16T23:49:06.938 INFO:tasks.workunit.client.1.smithi043.stdout:7/815: unlink d4/f32 0 2024-09-16T23:49:06.939 INFO:tasks.workunit.client.1.smithi043.stdout:2/755: dread d8/d1f/ff0 [0,4194304] 0 2024-09-16T23:49:06.939 INFO:tasks.workunit.client.1.smithi043.stdout:7/816: readlink d4/d27/d8a/d93/lc1 0 2024-09-16T23:49:06.940 INFO:tasks.workunit.client.1.smithi043.stdout:2/756: dread - d8/d1d/d20/d23/d38/da2/fe9 zero size 2024-09-16T23:49:06.945 INFO:tasks.workunit.client.0.smithi042.stdout:3/928: dwrite d2/d9/f13a [0,4194304] 0 2024-09-16T23:49:06.946 INFO:tasks.workunit.client.1.smithi043.stdout:7/817: creat d4/d27/dbc/f10f x:0 0 0 2024-09-16T23:49:06.950 INFO:tasks.workunit.client.0.smithi042.stdout:8/535: unlink da/c7c 0 2024-09-16T23:49:06.951 INFO:tasks.workunit.client.0.smithi042.stdout:3/929: creat d2/d1b/d1d/d34/f13e x:0 0 0 2024-09-16T23:49:06.952 INFO:tasks.workunit.client.0.smithi042.stdout:3/930: chown d2/d1b/d1d/d34/d50/ced 2431450 1 2024-09-16T23:49:06.952 INFO:tasks.workunit.client.1.smithi043.stdout:0/902: dwrite d0/d2/df/d1a/d49/db2/df4/f63 [0,4194304] 0 2024-09-16T23:49:06.954 INFO:tasks.workunit.client.1.smithi043.stdout:7/818: write d4/db/fb1 [4117914,63359] 0 2024-09-16T23:49:06.958 INFO:tasks.workunit.client.1.smithi043.stdout:0/903: chown d0/d2/df/cc1 1516905328 1 2024-09-16T23:49:06.962 INFO:tasks.workunit.client.1.smithi043.stdout:5/930: write d3/d9/d22/d60/d9d/fd7 [2143229,57970] 0 2024-09-16T23:49:06.964 INFO:tasks.workunit.client.1.smithi043.stdout:2/757: dwrite d8/d4b/d5f/f61 [4194304,4194304] 0 2024-09-16T23:49:06.965 INFO:tasks.workunit.client.1.smithi043.stdout:5/931: write d3/d1b/f124 [64824,29271] 0 2024-09-16T23:49:06.965 INFO:tasks.workunit.client.1.smithi043.stdout:0/904: unlink d0/l48 0 2024-09-16T23:49:06.967 INFO:tasks.workunit.client.1.smithi043.stdout:0/905: read - d0/d2/df/da7/f109 zero size 2024-09-16T23:49:06.970 INFO:tasks.workunit.client.0.smithi042.stdout:8/536: dwrite da/f24 [0,4194304] 0 2024-09-16T23:49:06.972 INFO:tasks.workunit.client.1.smithi043.stdout:2/758: mknod d8/d1d/d20/d26/c102 0 2024-09-16T23:49:06.974 INFO:tasks.workunit.client.1.smithi043.stdout:5/932: creat d3/db/d84/f13b x:0 0 0 2024-09-16T23:49:06.974 INFO:tasks.workunit.client.0.smithi042.stdout:3/931: dread d2/d7/d61/d8e/f12 [0,4194304] 0 2024-09-16T23:49:06.974 INFO:tasks.workunit.client.1.smithi043.stdout:5/933: readlink d3/db/d15/d4d/ldc 0 2024-09-16T23:49:06.976 INFO:tasks.workunit.client.1.smithi043.stdout:0/906: rename d0/lb to d0/d2/df/dbb/d12c/l135 0 2024-09-16T23:49:06.978 INFO:tasks.workunit.client.1.smithi043.stdout:0/907: chown d0/d2/df/f104 3 1 2024-09-16T23:49:06.981 INFO:tasks.workunit.client.0.smithi042.stdout:3/932: rename d2/d7/d61/d8e/d2c/cca to d2/d1b/d1d/d34/c13f 0 2024-09-16T23:49:06.982 INFO:tasks.workunit.client.0.smithi042.stdout:8/537: mknod da/d19/d1d/d30/d3f/d6d/ca4 0 2024-09-16T23:49:06.983 INFO:tasks.workunit.client.0.smithi042.stdout:8/538: dread - da/d19/d80/f8f zero size 2024-09-16T23:49:06.983 INFO:tasks.workunit.client.1.smithi043.stdout:2/759: read d8/fd [1747186,92137] 0 2024-09-16T23:49:06.988 INFO:tasks.workunit.client.1.smithi043.stdout:2/760: chown d8/d35/f37 106 1 2024-09-16T23:49:06.989 INFO:tasks.workunit.client.1.smithi043.stdout:0/908: link d0/d2/df/d1a/d46/d92/c122 d0/d2/df/d1a/d3b/dee/df0/df8/c136 0 2024-09-16T23:49:06.990 INFO:tasks.workunit.client.0.smithi042.stdout:8/539: getdents da/d19/d1d 0 2024-09-16T23:49:06.991 INFO:tasks.workunit.client.1.smithi043.stdout:0/909: chown d0/d2/df/l84 536270 1 2024-09-16T23:49:06.994 INFO:tasks.workunit.client.1.smithi043.stdout:0/910: symlink d0/d2/df/d1a/d49/db2/df4/d9c/dfa/l137 0 2024-09-16T23:49:06.997 INFO:tasks.workunit.client.1.smithi043.stdout:5/934: dread d3/f50 [4194304,4194304] 0 2024-09-16T23:49:06.998 INFO:tasks.workunit.client.1.smithi043.stdout:0/911: mkdir d0/d2/df/d1a/d46/d39/d83/d138 0 2024-09-16T23:49:07.000 INFO:tasks.workunit.client.1.smithi043.stdout:5/935: symlink d3/d9/d6f/d7c/d80/l13c 0 2024-09-16T23:49:07.000 INFO:tasks.workunit.client.1.smithi043.stdout:5/936: stat d3/d9/d22/f4e 0 2024-09-16T23:49:07.003 INFO:tasks.workunit.client.1.smithi043.stdout:5/937: mkdir d3/db/d15/d13d 0 2024-09-16T23:49:07.003 INFO:tasks.workunit.client.1.smithi043.stdout:5/938: readlink d3/db/d84/le0 0 2024-09-16T23:49:07.003 INFO:tasks.workunit.client.0.smithi042.stdout:3/933: dread d2/d1b/d89/f5c [0,4194304] 0 2024-09-16T23:49:07.004 INFO:tasks.workunit.client.1.smithi043.stdout:5/939: read - d3/db/d19/d3e/d28/dcb/f118 zero size 2024-09-16T23:49:07.004 INFO:tasks.workunit.client.1.smithi043.stdout:0/912: rename d0/f111 to d0/d2/df/d1a/f139 0 2024-09-16T23:49:07.005 INFO:tasks.workunit.client.0.smithi042.stdout:8/540: dwrite da/d4b/f65 [0,4194304] 0 2024-09-16T23:49:07.007 INFO:tasks.workunit.client.0.smithi042.stdout:3/934: mknod d2/d7/d61/c140 0 2024-09-16T23:49:07.009 INFO:tasks.workunit.client.0.smithi042.stdout:3/935: dread d2/d1b/d89/f47 [0,4194304] 0 2024-09-16T23:49:07.009 INFO:tasks.workunit.client.1.smithi043.stdout:5/940: rename d3/db/d15/d4d/d6b/d73/d101 to d3/d9/d22/d60/d9d/de4/d13e 0 2024-09-16T23:49:07.011 INFO:tasks.workunit.client.0.smithi042.stdout:8/541: dread da/d19/d2c/f2d [0,4194304] 0 2024-09-16T23:49:07.012 INFO:tasks.workunit.client.0.smithi042.stdout:8/542: fdatasync da/d19/d1d/d30/f45 0 2024-09-16T23:49:07.014 INFO:tasks.workunit.client.1.smithi043.stdout:5/941: creat d3/d9/d22/d9f/f13f x:0 0 0 2024-09-16T23:49:07.018 INFO:tasks.workunit.client.0.smithi042.stdout:3/936: rename d2/d1b/d1d to d2/d7/d61/d8e/d9d/d141 0 2024-09-16T23:49:07.019 INFO:tasks.workunit.client.1.smithi043.stdout:5/942: mknod d3/d9/d6f/d7c/d80/c140 0 2024-09-16T23:49:07.019 INFO:tasks.workunit.client.0.smithi042.stdout:3/937: chown d2/d7/d61/d8e/d9d/d141/d34/d62/f78 1 1 2024-09-16T23:49:07.021 INFO:tasks.workunit.client.0.smithi042.stdout:8/543: mknod da/d19/ca5 0 2024-09-16T23:49:07.021 INFO:tasks.workunit.client.1.smithi043.stdout:0/913: write d0/d2/df/d1a/d46/d39/daa/db7/f108 [4466232,1037] 0 2024-09-16T23:49:07.022 INFO:tasks.workunit.client.1.smithi043.stdout:5/943: mknod d3/d9/d22/d60/d9d/de4/d13e/c141 0 2024-09-16T23:49:07.023 INFO:tasks.workunit.client.1.smithi043.stdout:5/944: dread - d3/d9/d22/d4a/f13a zero size 2024-09-16T23:49:07.024 INFO:tasks.workunit.client.1.smithi043.stdout:0/914: unlink d0/d2/df/d1a/d3b/f103 0 2024-09-16T23:49:07.025 INFO:tasks.workunit.client.0.smithi042.stdout:3/938: dwrite d2/d7/d61/d8e/f3f [0,4194304] 0 2024-09-16T23:49:07.027 INFO:tasks.workunit.client.0.smithi042.stdout:8/544: rmdir da/d19/d2c/d56 39 2024-09-16T23:49:07.028 INFO:tasks.workunit.client.0.smithi042.stdout:8/545: chown da/d4b/f65 10 1 2024-09-16T23:49:07.028 INFO:tasks.workunit.client.1.smithi043.stdout:5/945: mknod d3/d9/d6f/d7c/d80/c142 0 2024-09-16T23:49:07.030 INFO:tasks.workunit.client.0.smithi042.stdout:3/939: fdatasync d2/d7/d61/d8e/d9d/d141/d30/da2/fd2 0 2024-09-16T23:49:07.030 INFO:tasks.workunit.client.1.smithi043.stdout:0/915: dwrite d0/d2/df/fbf [0,4194304] 0 2024-09-16T23:49:07.032 INFO:tasks.workunit.client.0.smithi042.stdout:8/546: creat da/d19/d80/fa6 x:0 0 0 2024-09-16T23:49:07.035 INFO:tasks.workunit.client.1.smithi043.stdout:5/946: creat d3/d9/d22/d8e/f143 x:0 0 0 2024-09-16T23:49:07.036 INFO:tasks.workunit.client.1.smithi043.stdout:0/916: fsync d0/d2/df/da7/fbd 0 2024-09-16T23:49:07.040 INFO:tasks.workunit.client.0.smithi042.stdout:8/547: unlink da/d19/d1d/c52 0 2024-09-16T23:49:07.040 INFO:tasks.workunit.client.1.smithi043.stdout:0/917: fsync d0/d2/df/d1a/d49/db2/df4/fe4 0 2024-09-16T23:49:07.043 INFO:tasks.workunit.client.1.smithi043.stdout:5/947: getdents d3/db/d19/d3e 0 2024-09-16T23:49:07.043 INFO:tasks.workunit.client.1.smithi043.stdout:0/918: getdents d0/d2/df/d1a/d46/d92/d123 0 2024-09-16T23:49:07.044 INFO:tasks.workunit.client.0.smithi042.stdout:3/940: dwrite d2/d1b/d89/f23 [0,4194304] 0 2024-09-16T23:49:07.045 INFO:tasks.workunit.client.1.smithi043.stdout:5/948: write d3/d9/d22/d9f/f13f [117266,56497] 0 2024-09-16T23:49:07.047 INFO:tasks.workunit.client.0.smithi042.stdout:1/729: sync 2024-09-16T23:49:07.047 INFO:tasks.workunit.client.1.smithi043.stdout:0/919: mknod d0/d2/df/d1a/d49/db2/df4/d9c/dfa/c13a 0 2024-09-16T23:49:07.048 INFO:tasks.workunit.client.0.smithi042.stdout:1/730: fdatasync d0/d9/d90/fdc 0 2024-09-16T23:49:07.049 INFO:tasks.workunit.client.0.smithi042.stdout:3/941: mknod d2/d7/d61/d8e/d9d/d141/d34/d50/de6/c142 0 2024-09-16T23:49:07.051 INFO:tasks.workunit.client.0.smithi042.stdout:3/942: rmdir d2/d7/d61/d8e/d9d/d141/d34/d50 39 2024-09-16T23:49:07.051 INFO:tasks.workunit.client.1.smithi043.stdout:5/949: fdatasync d3/d9/d22/d4a/d4c/d8c/f108 0 2024-09-16T23:49:07.052 INFO:tasks.workunit.client.1.smithi043.stdout:5/950: chown d3/da4/l12e 19 1 2024-09-16T23:49:07.053 INFO:tasks.workunit.client.0.smithi042.stdout:1/731: mknod d0/d17/c107 0 2024-09-16T23:49:07.054 INFO:tasks.workunit.client.1.smithi043.stdout:0/920: rmdir d0/d2/df/d1a/d49/db2/de1 39 2024-09-16T23:49:07.056 INFO:tasks.workunit.client.0.smithi042.stdout:3/943: rename d2/d7/d61/d8e/d9d/dc4/de4/f120 to d2/d7/d61/d8e/d9d/f143 0 2024-09-16T23:49:07.057 INFO:tasks.workunit.client.1.smithi043.stdout:0/921: read d0/d2/df/d1a/d49/d80/d88/fa4 [3515915,27116] 0 2024-09-16T23:49:07.057 INFO:tasks.workunit.client.0.smithi042.stdout:3/944: write d2/d7/ff2 [3437221,14742] 0 2024-09-16T23:49:07.058 INFO:tasks.workunit.client.1.smithi043.stdout:5/951: readlink d3/db/d15/d4d/d6b/ld5 0 2024-09-16T23:49:07.059 INFO:tasks.workunit.client.0.smithi042.stdout:3/945: creat d2/d1b/dc8/f144 x:0 0 0 2024-09-16T23:49:07.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/946: read - d2/d7/d61/d8e/d9d/d141/fd9 zero size 2024-09-16T23:49:07.062 INFO:tasks.workunit.client.0.smithi042.stdout:3/947: readlink d2/d7/d61/d8e/d9d/dc4/dcc/l106 0 2024-09-16T23:49:07.062 INFO:tasks.workunit.client.1.smithi043.stdout:0/922: mknod d0/d2/df/d1a/d49/db2/df4/d9c/dfa/c13b 0 2024-09-16T23:49:07.063 INFO:tasks.workunit.client.1.smithi043.stdout:0/923: chown d0/d2/df/d1a/d46/d92/fc9 5851 1 2024-09-16T23:49:07.063 INFO:tasks.workunit.client.1.smithi043.stdout:5/952: truncate d3/db/d19/d3e/f2d 4678390 0 2024-09-16T23:49:07.064 INFO:tasks.workunit.client.0.smithi042.stdout:3/948: write d2/d7/d61/d8e/d9d/dc4/dcc/fc2 [4421127,50038] 0 2024-09-16T23:49:07.064 INFO:tasks.workunit.client.0.smithi042.stdout:1/732: write d0/d9/d90/d6b/fae [708291,46466] 0 2024-09-16T23:49:07.065 INFO:tasks.workunit.client.0.smithi042.stdout:8/548: mknod da/d19/d1d/d30/ca7 0 2024-09-16T23:49:07.067 INFO:tasks.workunit.client.1.smithi043.stdout:0/924: creat d0/d2/df/d1a/d46/d39/f13c x:0 0 0 2024-09-16T23:49:07.068 INFO:tasks.workunit.client.1.smithi043.stdout:5/953: creat d3/db/d19/d76/f144 x:0 0 0 2024-09-16T23:49:07.069 INFO:tasks.workunit.client.0.smithi042.stdout:8/549: rmdir da/d19/d1d/d30/d3f 39 2024-09-16T23:49:07.070 INFO:tasks.workunit.client.0.smithi042.stdout:3/949: write d2/d7/d61/d8e/d9d/d141/d30/f7c [1622499,85796] 0 2024-09-16T23:49:07.072 INFO:tasks.workunit.client.1.smithi043.stdout:0/925: rename d0/d2/df/d1a/d3b/c81 to d0/d2/df/d1a/d49/d80/c13d 0 2024-09-16T23:49:07.073 INFO:tasks.workunit.client.0.smithi042.stdout:8/550: mknod da/d19/d80/ca8 0 2024-09-16T23:49:07.073 INFO:tasks.workunit.client.0.smithi042.stdout:1/733: link d0/d9/d26/d8b/lf0 d0/d17/dd4/l108 0 2024-09-16T23:49:07.073 INFO:tasks.workunit.client.1.smithi043.stdout:0/926: stat d0/d2/df/d1a/d49/db2/df4/c125 0 2024-09-16T23:49:07.074 INFO:tasks.workunit.client.0.smithi042.stdout:3/950: rename d2/d7/d61/d8e/d9d/d141/d30/da2 to d2/d9/d145 0 2024-09-16T23:49:07.076 INFO:tasks.workunit.client.0.smithi042.stdout:3/951: write d2/d95/dd8/f127 [589827,90269] 0 2024-09-16T23:49:07.077 INFO:tasks.workunit.client.0.smithi042.stdout:8/551: dread da/d19/f23 [0,4194304] 0 2024-09-16T23:49:07.077 INFO:tasks.workunit.client.0.smithi042.stdout:1/734: mknod d0/d9/d90/d6b/d88/d32/d2f/de3/c109 0 2024-09-16T23:49:07.078 INFO:tasks.workunit.client.0.smithi042.stdout:3/952: write d2/d1b/d89/f42 [4561675,121127] 0 2024-09-16T23:49:07.079 INFO:tasks.workunit.client.1.smithi043.stdout:0/927: dwrite d0/de/f2d [0,4194304] 0 2024-09-16T23:49:07.079 INFO:tasks.workunit.client.1.smithi043.stdout:5/954: rename d3/db/d19/c81 to d3/d9/d6f/c145 0 2024-09-16T23:49:07.082 INFO:tasks.workunit.client.1.smithi043.stdout:6/790: sync 2024-09-16T23:49:07.082 INFO:tasks.workunit.client.1.smithi043.stdout:8/760: sync 2024-09-16T23:49:07.082 INFO:tasks.workunit.client.1.smithi043.stdout:9/781: sync 2024-09-16T23:49:07.082 INFO:tasks.workunit.client.1.smithi043.stdout:4/872: sync 2024-09-16T23:49:07.089 INFO:tasks.workunit.client.1.smithi043.stdout:0/928: dwrite d0/d2/df/dbb/ff1 [0,4194304] 0 2024-09-16T23:49:07.089 INFO:tasks.workunit.client.0.smithi042.stdout:8/552: dread da/d19/d2c/f67 [0,4194304] 0 2024-09-16T23:49:07.091 INFO:tasks.workunit.client.1.smithi043.stdout:9/782: dread - d4/d84/fb2 zero size 2024-09-16T23:49:07.092 INFO:tasks.workunit.client.1.smithi043.stdout:5/955: creat d3/d9/d6f/f146 x:0 0 0 2024-09-16T23:49:07.092 INFO:tasks.workunit.client.0.smithi042.stdout:3/953: dwrite d2/d1b/f94 [0,4194304] 0 2024-09-16T23:49:07.093 INFO:tasks.workunit.client.1.smithi043.stdout:4/873: dwrite d8/d15/f10c [0,4194304] 0 2024-09-16T23:49:07.094 INFO:tasks.workunit.client.0.smithi042.stdout:3/954: fdatasync d2/f4a 0 2024-09-16T23:49:07.095 INFO:tasks.workunit.client.0.smithi042.stdout:3/955: fdatasync d2/d1b/dc8/ff6 0 2024-09-16T23:49:07.102 INFO:tasks.workunit.client.0.smithi042.stdout:1/735: write d0/d17/f2d [807363,39338] 0 2024-09-16T23:49:07.102 INFO:tasks.workunit.client.0.smithi042.stdout:3/956: rmdir d2/d7/d61/d8e/d9d/d141/d34/d62/d80 0 2024-09-16T23:49:07.103 INFO:tasks.workunit.client.0.smithi042.stdout:1/736: chown d0/d72/l5d 466 1 2024-09-16T23:49:07.105 INFO:tasks.workunit.client.0.smithi042.stdout:0/555: sync 2024-09-16T23:49:07.105 INFO:tasks.workunit.client.0.smithi042.stdout:9/530: sync 2024-09-16T23:49:07.105 INFO:tasks.workunit.client.0.smithi042.stdout:5/816: sync 2024-09-16T23:49:07.105 INFO:tasks.workunit.client.0.smithi042.stdout:6/663: sync 2024-09-16T23:49:07.105 INFO:tasks.workunit.client.0.smithi042.stdout:7/752: sync 2024-09-16T23:49:07.105 INFO:tasks.workunit.client.0.smithi042.stdout:2/615: sync 2024-09-16T23:49:07.105 INFO:tasks.workunit.client.0.smithi042.stdout:4/961: sync 2024-09-16T23:49:07.106 INFO:tasks.workunit.client.0.smithi042.stdout:3/957: write d2/d7/d61/d8e/d9d/d141/fa0 [1248915,120513] 0 2024-09-16T23:49:07.107 INFO:tasks.workunit.client.0.smithi042.stdout:8/553: creat da/d4b/fa9 x:0 0 0 2024-09-16T23:49:07.108 INFO:tasks.workunit.client.1.smithi043.stdout:5/956: rmdir d3/db/d15/d4d 39 2024-09-16T23:49:07.110 INFO:tasks.workunit.client.1.smithi043.stdout:5/957: fdatasync d3/d79/d91/daf/fc5 0 2024-09-16T23:49:07.113 INFO:tasks.workunit.client.1.smithi043.stdout:0/929: rename d0/d2/df/d1a/d46/d39/d83/dfb to d0/d2/df/d1a/d3b/d13e 0 2024-09-16T23:49:07.113 INFO:tasks.workunit.client.1.smithi043.stdout:8/761: write de/f71 [483753,45437] 0 2024-09-16T23:49:07.114 INFO:tasks.workunit.client.0.smithi042.stdout:4/962: fsync de/d2e/d37/d4c/f105 0 2024-09-16T23:49:07.114 INFO:tasks.workunit.client.1.smithi043.stdout:6/791: dwrite de/d23/dc3/fd4 [0,4194304] 0 2024-09-16T23:49:07.115 INFO:tasks.workunit.client.0.smithi042.stdout:5/817: symlink d9/db/d1b/d79/l119 0 2024-09-16T23:49:07.116 INFO:tasks.workunit.client.1.smithi043.stdout:6/792: chown de/d23/c43 2 1 2024-09-16T23:49:07.116 INFO:tasks.workunit.client.1.smithi043.stdout:0/930: dread - d0/d2/df/f104 zero size 2024-09-16T23:49:07.117 INFO:tasks.workunit.client.0.smithi042.stdout:1/737: dread d0/f4a [0,4194304] 0 2024-09-16T23:49:07.119 INFO:tasks.workunit.client.0.smithi042.stdout:2/616: creat d0/da/d1b/d48/db7/fd2 x:0 0 0 2024-09-16T23:49:07.119 INFO:tasks.workunit.client.0.smithi042.stdout:9/531: creat d4/d10/faa x:0 0 0 2024-09-16T23:49:07.122 INFO:tasks.workunit.client.0.smithi042.stdout:9/532: readlink d4/d12/d16/l3e 0 2024-09-16T23:49:07.124 INFO:tasks.workunit.client.1.smithi043.stdout:0/931: dwrite d0/d2/df/f116 [0,4194304] 0 2024-09-16T23:49:07.124 INFO:tasks.workunit.client.1.smithi043.stdout:9/783: creat d4/d1c/d25/d2b/da1/df9/ffa x:0 0 0 2024-09-16T23:49:07.125 INFO:tasks.workunit.client.0.smithi042.stdout:2/617: chown d0/d30/d43/c7b 456 1 2024-09-16T23:49:07.127 INFO:tasks.workunit.client.1.smithi043.stdout:0/932: write d0/d2/df/d1a/d3b/f5d [608597,100644] 0 2024-09-16T23:49:07.129 INFO:tasks.workunit.client.0.smithi042.stdout:0/556: dwrite d0/df/d23/d33/d56/f61 [0,4194304] 0 2024-09-16T23:49:07.132 INFO:tasks.workunit.client.0.smithi042.stdout:5/818: rmdir d9/db/d1b/d31/d6c/d101 0 2024-09-16T23:49:07.133 INFO:tasks.workunit.client.1.smithi043.stdout:5/958: rename d3/d9/cd2 to d3/dce/c147 0 2024-09-16T23:49:07.136 INFO:tasks.workunit.client.1.smithi043.stdout:8/762: creat de/d18/d1f/d25/d65/d7f/dca/fe3 x:0 0 0 2024-09-16T23:49:07.139 INFO:tasks.workunit.client.0.smithi042.stdout:8/554: dwrite da/d19/d80/fa6 [0,4194304] 0 2024-09-16T23:49:07.140 INFO:tasks.workunit.client.1.smithi043.stdout:9/784: unlink d4/d21/d2a/d2c/d7b/f7e 0 2024-09-16T23:49:07.144 INFO:tasks.workunit.client.0.smithi042.stdout:5/819: readlink d9/d2c/d7b/lde 0 2024-09-16T23:49:07.145 INFO:tasks.workunit.client.0.smithi042.stdout:2/618: rmdir d0/da/d17/d3e 39 2024-09-16T23:49:07.146 INFO:tasks.workunit.client.0.smithi042.stdout:5/820: chown d9/ca 201997713 1 2024-09-16T23:49:07.148 INFO:tasks.workunit.client.0.smithi042.stdout:7/753: dwrite d3/d1f/d49/d76/d19/ddc/f4a [0,4194304] 0 2024-09-16T23:49:07.149 INFO:tasks.workunit.client.1.smithi043.stdout:8/763: dread de/f71 [4194304,4194304] 0 2024-09-16T23:49:07.150 INFO:tasks.workunit.client.0.smithi042.stdout:5/821: stat d9/db/f25 0 2024-09-16T23:49:07.153 INFO:tasks.workunit.client.0.smithi042.stdout:5/822: stat d9/db/d19/d21/d41/d7d/f116 0 2024-09-16T23:49:07.153 INFO:tasks.workunit.client.0.smithi042.stdout:1/738: dread d0/d9/f16 [0,4194304] 0 2024-09-16T23:49:07.156 INFO:tasks.workunit.client.0.smithi042.stdout:0/557: write d0/df/d23/f6b [1543032,20697] 0 2024-09-16T23:49:07.159 INFO:tasks.workunit.client.0.smithi042.stdout:4/963: dread de/f10 [4194304,4194304] 0 2024-09-16T23:49:07.159 INFO:tasks.workunit.client.1.smithi043.stdout:8/764: mkdir de/d18/d1f/d2f/d39/d7b/d83/dbb/dce/de4 0 2024-09-16T23:49:07.162 INFO:tasks.workunit.client.1.smithi043.stdout:6/793: dwrite de/d23/dc3/fd1 [4194304,4194304] 0 2024-09-16T23:49:07.164 INFO:tasks.workunit.client.0.smithi042.stdout:9/533: write d4/d10/d24/f67 [5136050,107861] 0 2024-09-16T23:49:07.168 INFO:tasks.workunit.client.1.smithi043.stdout:8/765: creat de/d18/d1f/d2f/d39/d7b/d83/fe5 x:0 0 0 2024-09-16T23:49:07.169 INFO:tasks.workunit.client.1.smithi043.stdout:9/785: write d4/d21/d33/d95/fc0 [218942,76363] 0 2024-09-16T23:49:07.171 INFO:tasks.workunit.client.0.smithi042.stdout:7/754: mknod d3/d1f/d49/c109 0 2024-09-16T23:49:07.172 INFO:tasks.workunit.client.0.smithi042.stdout:5/823: truncate d9/db/de/d10/f1a 1006907 0 2024-09-16T23:49:07.172 INFO:tasks.workunit.client.0.smithi042.stdout:7/755: chown d3/l13 299642443 1 2024-09-16T23:49:07.173 INFO:tasks.workunit.client.1.smithi043.stdout:5/959: dwrite d3/d9/d22/d4a/d4c/d7a/f114 [0,4194304] 0 2024-09-16T23:49:07.176 INFO:tasks.workunit.client.0.smithi042.stdout:8/555: write da/d19/d1d/d30/d49/f7b [1169162,97513] 0 2024-09-16T23:49:07.177 INFO:tasks.workunit.client.0.smithi042.stdout:5/824: dread d9/d4a/d89/f8a [0,4194304] 0 2024-09-16T23:49:07.182 INFO:tasks.workunit.client.0.smithi042.stdout:5/825: rmdir d9 39 2024-09-16T23:49:07.186 INFO:tasks.workunit.client.0.smithi042.stdout:9/534: readlink d4/d12/d2d/d51/l8d 0 2024-09-16T23:49:07.188 INFO:tasks.workunit.client.1.smithi043.stdout:8/766: truncate de/d18/d27/d41/d5a/f24 2134478 0 2024-09-16T23:49:07.188 INFO:tasks.workunit.client.0.smithi042.stdout:0/558: dwrite d0/df/d13/d2d/d35/f82 [0,4194304] 0 2024-09-16T23:49:07.190 INFO:tasks.workunit.client.0.smithi042.stdout:4/964: link de/d2e/d37/d3c/l76 de/d1a/d70/dac/l14a 0 2024-09-16T23:49:07.191 INFO:tasks.workunit.client.0.smithi042.stdout:5/826: mkdir d9/d11a 0 2024-09-16T23:49:07.192 INFO:tasks.workunit.client.0.smithi042.stdout:4/965: write de/d2e/d37/d4c/f9b [4090752,78823] 0 2024-09-16T23:49:07.193 INFO:tasks.workunit.client.0.smithi042.stdout:5/827: chown d9/db/d19/d21/d41/d7d/fa0 0 1 2024-09-16T23:49:07.194 INFO:tasks.workunit.client.0.smithi042.stdout:5/828: stat d9/db/d1b/d31/l3d 0 2024-09-16T23:49:07.195 INFO:tasks.workunit.client.0.smithi042.stdout:7/756: dread d3/d1f/d49/d78/dbe/fe4 [0,4194304] 0 2024-09-16T23:49:07.196 INFO:tasks.workunit.client.1.smithi043.stdout:5/960: dread d3/db/f2a [0,4194304] 0 2024-09-16T23:49:07.198 INFO:tasks.workunit.client.0.smithi042.stdout:2/619: write d0/f2b [12843,98743] 0 2024-09-16T23:49:07.203 INFO:tasks.workunit.client.1.smithi043.stdout:8/767: symlink de/d18/d1f/d2f/d39/d7b/d83/dbb/le6 0 2024-09-16T23:49:07.204 INFO:tasks.workunit.client.0.smithi042.stdout:1/739: truncate d0/d17/f9e 310980 0 2024-09-16T23:49:07.207 INFO:tasks.workunit.client.1.smithi043.stdout:6/794: dwrite de/d1a/d28/f3e [0,4194304] 0 2024-09-16T23:49:07.207 INFO:tasks.workunit.client.1.smithi043.stdout:5/961: dwrite d3/d9/d6f/f146 [0,4194304] 0 2024-09-16T23:49:07.209 INFO:tasks.workunit.client.0.smithi042.stdout:2/620: symlink d0/da/d1b/d48/d81/ld3 0 2024-09-16T23:49:07.210 INFO:tasks.workunit.client.0.smithi042.stdout:8/556: dread da/d19/d1d/d5a/f79 [0,4194304] 0 2024-09-16T23:49:07.215 INFO:tasks.workunit.client.0.smithi042.stdout:9/535: rename d4/d12/d2d/d37/d28/l5d to d4/d12/d44/d8e/lab 0 2024-09-16T23:49:07.216 INFO:tasks.workunit.client.1.smithi043.stdout:8/768: creat de/d18/d1f/d25/fe7 x:0 0 0 2024-09-16T23:49:07.221 INFO:tasks.workunit.client.1.smithi043.stdout:9/786: getdents d4/d1c 0 2024-09-16T23:49:07.223 INFO:tasks.workunit.client.1.smithi043.stdout:6/795: fdatasync de/d23/d34/d9b/d5a/fc0 0 2024-09-16T23:49:07.227 INFO:tasks.workunit.client.0.smithi042.stdout:1/740: dwrite d0/d17/f58 [4194304,4194304] 0 2024-09-16T23:49:07.228 INFO:tasks.workunit.client.1.smithi043.stdout:8/769: symlink de/d18/d1f/d2f/d39/d7b/d83/da5/le8 0 2024-09-16T23:49:07.231 INFO:tasks.workunit.client.0.smithi042.stdout:2/621: write d0/d30/f55 [852804,24109] 0 2024-09-16T23:49:07.234 INFO:tasks.workunit.client.0.smithi042.stdout:1/741: symlink d0/d9a/l10a 0 2024-09-16T23:49:07.235 INFO:tasks.workunit.client.1.smithi043.stdout:8/770: dwrite de/d18/d27/d41/d5a/d37/f69 [0,4194304] 0 2024-09-16T23:49:07.236 INFO:tasks.workunit.client.0.smithi042.stdout:9/536: rename d4/d12/d2d/d37/d59/c76 to d4/d12/d2d/d37/d41/d92/cac 0 2024-09-16T23:49:07.237 INFO:tasks.workunit.client.1.smithi043.stdout:9/787: dwrite d4/d5/d64/f65 [4194304,4194304] 0 2024-09-16T23:49:07.237 INFO:tasks.workunit.client.1.smithi043.stdout:8/771: fdatasync de/d18/d1f/f2d 0 2024-09-16T23:49:07.239 INFO:tasks.workunit.client.1.smithi043.stdout:5/962: truncate d3/d79/d91/daf/fc5 1778476 0 2024-09-16T23:49:07.242 INFO:tasks.workunit.client.0.smithi042.stdout:8/557: creat da/faa x:0 0 0 2024-09-16T23:49:07.245 INFO:tasks.workunit.client.0.smithi042.stdout:2/622: fsync d0/da/d88/dae/f75 0 2024-09-16T23:49:07.250 INFO:tasks.workunit.client.0.smithi042.stdout:1/742: symlink d0/d9/d90/l10b 0 2024-09-16T23:49:07.253 INFO:tasks.workunit.client.1.smithi043.stdout:8/772: creat de/d18/d27/d41/d5a/dc8/fe9 x:0 0 0 2024-09-16T23:49:07.254 INFO:tasks.workunit.client.0.smithi042.stdout:2/623: fdatasync d0/d78/d8b/fd0 0 2024-09-16T23:49:07.255 INFO:tasks.workunit.client.1.smithi043.stdout:5/963: read - d3/db/d84/f10b zero size 2024-09-16T23:49:07.259 INFO:tasks.workunit.client.1.smithi043.stdout:8/773: symlink de/d70/d3d/daf/lea 0 2024-09-16T23:49:07.261 INFO:tasks.workunit.client.1.smithi043.stdout:9/788: mknod d4/d21/d2a/d2c/d7b/df0/cfb 0 2024-09-16T23:49:07.261 INFO:tasks.workunit.client.1.smithi043.stdout:8/774: chown de/d18/d1f/d2f/d39/d4d/f5d 134 1 2024-09-16T23:49:07.263 INFO:tasks.workunit.client.1.smithi043.stdout:5/964: unlink d3/d79/lb3 0 2024-09-16T23:49:07.263 INFO:tasks.workunit.client.1.smithi043.stdout:6/796: getdents de/d23/ddc 0 2024-09-16T23:49:07.263 INFO:tasks.workunit.client.0.smithi042.stdout:1/743: dread d0/d17/f2d [0,4194304] 0 2024-09-16T23:49:07.265 INFO:tasks.workunit.client.0.smithi042.stdout:2/624: creat d0/d30/d43/fd4 x:0 0 0 2024-09-16T23:49:07.268 INFO:tasks.workunit.client.0.smithi042.stdout:1/744: creat d0/d9/d90/d6b/d88/d32/d2f/de3/f10c x:0 0 0 2024-09-16T23:49:07.270 INFO:tasks.workunit.client.0.smithi042.stdout:2/625: creat d0/da/d1b/d48/db7/fd5 x:0 0 0 2024-09-16T23:49:07.270 INFO:tasks.workunit.client.0.smithi042.stdout:7/757: truncate d3/d1f/d49/d76/d101/d5b/f7a 715370 0 2024-09-16T23:49:07.271 INFO:tasks.workunit.client.0.smithi042.stdout:0/559: mknod d0/d15/cad 0 2024-09-16T23:49:07.273 INFO:tasks.workunit.client.0.smithi042.stdout:8/558: dwrite da/d19/d1d/f63 [0,4194304] 0 2024-09-16T23:49:07.274 INFO:tasks.workunit.client.0.smithi042.stdout:0/560: truncate d0/f4 4869446 0 2024-09-16T23:49:07.274 INFO:tasks.workunit.client.1.smithi043.stdout:9/789: chown d4/d1c/d25/d2b/da1/df9/d8c/l43 974 1 2024-09-16T23:49:07.275 INFO:tasks.workunit.client.0.smithi042.stdout:0/561: chown d0/d48/c9a 84283 1 2024-09-16T23:49:07.276 INFO:tasks.workunit.client.1.smithi043.stdout:8/775: write de/d70/fa0 [90346,63968] 0 2024-09-16T23:49:07.278 INFO:tasks.workunit.client.0.smithi042.stdout:2/626: creat d0/d8d/da0/fd6 x:0 0 0 2024-09-16T23:49:07.280 INFO:tasks.workunit.client.0.smithi042.stdout:8/559: rmdir da/d19/d1d/d30 39 2024-09-16T23:49:07.281 INFO:tasks.workunit.client.1.smithi043.stdout:6/797: truncate de/d23/d49/fb5 608414 0 2024-09-16T23:49:07.283 INFO:tasks.workunit.client.0.smithi042.stdout:8/560: dread da/d19/d2c/f4e [0,4194304] 0 2024-09-16T23:49:07.285 INFO:tasks.workunit.client.0.smithi042.stdout:2/627: truncate d0/f92 145241 0 2024-09-16T23:49:07.292 INFO:tasks.workunit.client.0.smithi042.stdout:8/561: mkdir da/d19/d2c/d56/d97/dab 0 2024-09-16T23:49:07.293 INFO:tasks.workunit.client.0.smithi042.stdout:5/829: dread d9/f74 [0,4194304] 0 2024-09-16T23:49:07.293 INFO:tasks.workunit.client.1.smithi043.stdout:8/776: truncate de/d18/d1f/f32 2201763 0 2024-09-16T23:49:07.294 INFO:tasks.workunit.client.0.smithi042.stdout:1/745: truncate d0/d9a/fb5 3228233 0 2024-09-16T23:49:07.295 INFO:tasks.workunit.client.0.smithi042.stdout:2/628: unlink d0/da/d1b/d48/c5a 0 2024-09-16T23:49:07.296 INFO:tasks.workunit.client.0.smithi042.stdout:2/629: write d0/da/fcc [759083,16707] 0 2024-09-16T23:49:07.297 INFO:tasks.workunit.client.1.smithi043.stdout:9/790: link d4/d1c/d25/d2b/f8a d4/d28/ffc 0 2024-09-16T23:49:07.300 INFO:tasks.workunit.client.0.smithi042.stdout:2/630: chown d0/da/d1b/d48/f58 1 1 2024-09-16T23:49:07.301 INFO:tasks.workunit.client.0.smithi042.stdout:2/631: write d0/da/fcc [782457,64857] 0 2024-09-16T23:49:07.301 INFO:tasks.workunit.client.1.smithi043.stdout:9/791: truncate d4/d1c/d25/d2b/f8a 215393 0 2024-09-16T23:49:07.301 INFO:tasks.workunit.client.1.smithi043.stdout:5/965: getdents d3/db/d15/d4d/d6b 0 2024-09-16T23:49:07.303 INFO:tasks.workunit.client.1.smithi043.stdout:8/777: mkdir de/d18/d27/d41/d5a/dc8/deb 0 2024-09-16T23:49:07.305 INFO:tasks.workunit.client.1.smithi043.stdout:9/792: unlink d4/d1c/d6b/d67/fec 0 2024-09-16T23:49:07.305 INFO:tasks.workunit.client.1.smithi043.stdout:6/798: link de/d23/d34/d9b/ffd de/d1a/d80/f10f 0 2024-09-16T23:49:07.307 INFO:tasks.workunit.client.1.smithi043.stdout:5/966: symlink d3/d9/d22/d4a/d4c/l148 0 2024-09-16T23:49:07.309 INFO:tasks.workunit.client.1.smithi043.stdout:8/778: fsync de/d18/d1f/d2f/d39/d4d/f5d 0 2024-09-16T23:49:07.314 INFO:tasks.workunit.client.0.smithi042.stdout:2/632: creat d0/da/d17/d3e/d86/fd7 x:0 0 0 2024-09-16T23:49:07.320 INFO:tasks.workunit.client.1.smithi043.stdout:5/967: creat d3/d9/d6f/d7c/d80/f149 x:0 0 0 2024-09-16T23:49:07.322 INFO:tasks.workunit.client.1.smithi043.stdout:8/779: mknod de/d18/d1f/d25/d65/d7f/dca/cec 0 2024-09-16T23:49:07.322 INFO:tasks.workunit.client.0.smithi042.stdout:8/562: write da/d19/d1d/d30/d3f/f61 [342571,127366] 0 2024-09-16T23:49:07.326 INFO:tasks.workunit.client.1.smithi043.stdout:8/780: dwrite de/d18/d1f/d25/fb9 [0,4194304] 0 2024-09-16T23:49:07.328 INFO:tasks.workunit.client.0.smithi042.stdout:8/563: dwrite da/d19/d2c/d56/d97/f9d [0,4194304] 0 2024-09-16T23:49:07.340 INFO:tasks.workunit.client.0.smithi042.stdout:4/966: creat de/d2e/d37/d3c/f14b x:0 0 0 2024-09-16T23:49:07.343 INFO:tasks.workunit.client.0.smithi042.stdout:5/830: mknod d9/db/de/d92/db5/c11b 0 2024-09-16T23:49:07.344 INFO:tasks.workunit.client.0.smithi042.stdout:0/562: symlink d0/df/d13/d49/d70/da1/dac/lae 0 2024-09-16T23:49:07.346 INFO:tasks.workunit.client.0.smithi042.stdout:0/563: write d0/df/d23/d33/d56/f8b [265371,122989] 0 2024-09-16T23:49:07.346 INFO:tasks.workunit.client.0.smithi042.stdout:7/758: creat d3/d98/da0/d24/f10a x:0 0 0 2024-09-16T23:49:07.348 INFO:tasks.workunit.client.0.smithi042.stdout:5/831: creat d9/db/d1b/d79/f11c x:0 0 0 2024-09-16T23:49:07.351 INFO:tasks.workunit.client.0.smithi042.stdout:0/564: mkdir d0/d48/daf 0 2024-09-16T23:49:07.352 INFO:tasks.workunit.client.0.smithi042.stdout:0/565: chown d0/df/c96 1016757679 1 2024-09-16T23:49:07.353 INFO:tasks.workunit.client.0.smithi042.stdout:5/832: symlink d9/db/de/d10/dc8/l11d 0 2024-09-16T23:49:07.358 INFO:tasks.workunit.client.0.smithi042.stdout:0/566: creat d0/df/d23/d33/d51/d79/fb0 x:0 0 0 2024-09-16T23:49:07.359 INFO:tasks.workunit.client.0.smithi042.stdout:4/967: write de/d42/d68/f96 [526990,121868] 0 2024-09-16T23:49:07.360 INFO:tasks.workunit.client.0.smithi042.stdout:5/833: dwrite d9/d2c/f106 [0,4194304] 0 2024-09-16T23:49:07.365 INFO:tasks.workunit.client.0.smithi042.stdout:5/834: mkdir d9/d11e 0 2024-09-16T23:49:07.368 INFO:tasks.workunit.client.0.smithi042.stdout:4/968: creat de/d10d/d8a/f14c x:0 0 0 2024-09-16T23:49:07.369 INFO:tasks.workunit.client.0.smithi042.stdout:5/835: fdatasync d9/db/d1b/f97 0 2024-09-16T23:49:07.371 INFO:tasks.workunit.client.0.smithi042.stdout:7/759: dread d3/d98/da0/d41/f8b [0,4194304] 0 2024-09-16T23:49:07.372 INFO:tasks.workunit.client.0.smithi042.stdout:7/760: chown d3/d1f/l2a 1557 1 2024-09-16T23:49:07.374 INFO:tasks.workunit.client.0.smithi042.stdout:4/969: link de/d12/f97 de/d10d/d8a/f14d 0 2024-09-16T23:49:07.378 INFO:tasks.workunit.client.0.smithi042.stdout:7/761: link d3/d1f/d4f/le1 d3/d98/da0/d41/db0/l10b 0 2024-09-16T23:49:07.380 INFO:tasks.workunit.client.0.smithi042.stdout:0/567: dread d0/df/d13/d2d/d35/f73 [0,4194304] 0 2024-09-16T23:49:07.380 INFO:tasks.workunit.client.0.smithi042.stdout:5/836: dwrite d9/d4a/ffb [0,4194304] 0 2024-09-16T23:49:07.384 INFO:tasks.workunit.client.0.smithi042.stdout:0/568: creat d0/d48/fb1 x:0 0 0 2024-09-16T23:49:07.386 INFO:tasks.workunit.client.0.smithi042.stdout:5/837: mkdir d9/db/de/d5e/d11f 0 2024-09-16T23:49:07.387 INFO:tasks.workunit.client.0.smithi042.stdout:7/762: dwrite d3/f30 [0,4194304] 0 2024-09-16T23:49:07.393 INFO:tasks.workunit.client.0.smithi042.stdout:4/970: dwrite de/d12/f14 [0,4194304] 0 2024-09-16T23:49:07.395 INFO:tasks.workunit.client.0.smithi042.stdout:0/569: dwrite d0/df/d23/d3e/faa [0,4194304] 0 2024-09-16T23:49:07.396 INFO:tasks.workunit.client.0.smithi042.stdout:7/763: link d3/d98/da0/d41/f103 d3/d3f/f10c 0 2024-09-16T23:49:07.397 INFO:tasks.workunit.client.0.smithi042.stdout:0/570: write d0/df/d23/d33/d56/f8b [387396,106339] 0 2024-09-16T23:49:07.399 INFO:tasks.workunit.client.1.smithi043.stdout:7/819: sync 2024-09-16T23:49:07.399 INFO:tasks.workunit.client.1.smithi043.stdout:3/824: sync 2024-09-16T23:49:07.401 INFO:tasks.workunit.client.1.smithi043.stdout:3/825: write d4/de/fe2 [2308868,93246] 0 2024-09-16T23:49:07.401 INFO:tasks.workunit.client.0.smithi042.stdout:0/571: creat d0/df/fb2 x:0 0 0 2024-09-16T23:49:07.403 INFO:tasks.workunit.client.0.smithi042.stdout:7/764: rename d3/l12 to d3/d98/da0/d24/dc3/l10d 0 2024-09-16T23:49:07.404 INFO:tasks.workunit.client.1.smithi043.stdout:3/826: truncate d4/de/d1f/d2c/fce 704180 0 2024-09-16T23:49:07.404 INFO:tasks.workunit.client.0.smithi042.stdout:7/765: stat d3/d1f/d49/d76/d19 0 2024-09-16T23:49:07.405 INFO:tasks.workunit.client.0.smithi042.stdout:0/572: read d0/df/f8c [2532654,67211] 0 2024-09-16T23:49:07.409 INFO:tasks.workunit.client.0.smithi042.stdout:7/766: rename d3/d98/da0/d41/f47 to d3/d3f/f10e 0 2024-09-16T23:49:07.413 INFO:tasks.workunit.client.0.smithi042.stdout:0/573: link d0/df/d13/d49/d70/f98 d0/df/d13/d49/d70/fb3 0 2024-09-16T23:49:07.414 INFO:tasks.workunit.client.1.smithi043.stdout:7/820: dread d4/d27/d8a/faa [4194304,4194304] 0 2024-09-16T23:49:07.415 INFO:tasks.workunit.client.0.smithi042.stdout:4/971: write de/d1a/da7/db9/fd5 [703147,11966] 0 2024-09-16T23:49:07.415 INFO:tasks.workunit.client.0.smithi042.stdout:7/767: dwrite d3/d1f/f23 [0,4194304] 0 2024-09-16T23:49:07.417 INFO:tasks.workunit.client.0.smithi042.stdout:4/972: write de/d42/d82/d94/d124/f12a [40082,63671] 0 2024-09-16T23:49:07.424 INFO:tasks.workunit.client.0.smithi042.stdout:7/768: dwrite d3/d1f/d49/d76/d19/fe0 [0,4194304] 0 2024-09-16T23:49:07.431 INFO:tasks.workunit.client.1.smithi043.stdout:8/781: creat de/d18/d1f/d2f/d39/d7b/fed x:0 0 0 2024-09-16T23:49:07.432 INFO:tasks.workunit.client.1.smithi043.stdout:8/782: chown de/d70/f76 7943 1 2024-09-16T23:49:07.432 INFO:tasks.workunit.client.0.smithi042.stdout:8/564: mknod da/d19/d2c/d56/d97/cac 0 2024-09-16T23:49:07.432 INFO:tasks.workunit.client.0.smithi042.stdout:7/769: getdents d3/d98/da0/d24 0 2024-09-16T23:49:07.434 INFO:tasks.workunit.client.0.smithi042.stdout:7/770: write d3/d98/da0/d24/f2d [4145660,9898] 0 2024-09-16T23:49:07.434 INFO:tasks.workunit.client.1.smithi043.stdout:8/783: read de/d18/d27/d41/d5a/d37/f99 [401801,26850] 0 2024-09-16T23:49:07.437 INFO:tasks.workunit.client.0.smithi042.stdout:0/574: dread d0/f67 [0,4194304] 0 2024-09-16T23:49:07.439 INFO:tasks.workunit.client.0.smithi042.stdout:8/565: mkdir da/d19/d2c/d56/d97/dad 0 2024-09-16T23:49:07.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/633: write d0/da/d1b/d3b/d44/f90 [924598,90959] 0 2024-09-16T23:49:07.442 INFO:tasks.workunit.client.0.smithi042.stdout:7/771: link d3/d1f/d49/d76/d101/d64/d92/fdd d3/d1f/d49/d78/dbe/dc8/f10f 0 2024-09-16T23:49:07.443 INFO:tasks.workunit.client.1.smithi043.stdout:9/793: dwrite d4/d5/d16/f1f [0,4194304] 0 2024-09-16T23:49:07.443 INFO:tasks.workunit.client.1.smithi043.stdout:6/799: dwrite de/d23/d63/fb1 [0,4194304] 0 2024-09-16T23:49:07.444 INFO:tasks.workunit.client.0.smithi042.stdout:4/973: dread de/d1a/da7/f4a [0,4194304] 0 2024-09-16T23:49:07.444 INFO:tasks.workunit.client.0.smithi042.stdout:0/575: creat d0/df/d23/d3e/fb4 x:0 0 0 2024-09-16T23:49:07.447 INFO:tasks.workunit.client.0.smithi042.stdout:4/974: write de/d42/d68/f96 [801445,109935] 0 2024-09-16T23:49:07.448 INFO:tasks.workunit.client.1.smithi043.stdout:8/784: dread - de/d18/d27/d41/f8f zero size 2024-09-16T23:49:07.448 INFO:tasks.workunit.client.1.smithi043.stdout:8/785: chown l2 2375170 1 2024-09-16T23:49:07.449 INFO:tasks.workunit.client.1.smithi043.stdout:8/786: dread - de/d18/d27/d41/fcb zero size 2024-09-16T23:49:07.451 INFO:tasks.workunit.client.1.smithi043.stdout:8/787: chown de/d18/d1f/d2f/d39/d7b/d83/f84 261 1 2024-09-16T23:49:07.452 INFO:tasks.workunit.client.1.smithi043.stdout:7/821: mkdir d4/d27/d41/d110 0 2024-09-16T23:49:07.453 INFO:tasks.workunit.client.0.smithi042.stdout:4/975: link de/d2e/f137 de/d2e/d37/dc3/f14e 0 2024-09-16T23:49:07.455 INFO:tasks.workunit.client.1.smithi043.stdout:9/794: mknod d4/d1c/d6b/d67/cfd 0 2024-09-16T23:49:07.455 INFO:tasks.workunit.client.0.smithi042.stdout:7/772: rename d3/d1f/d49/d76/d19/ddc/c107 to d3/d1f/d49/d76/d19/ddc/dcd/db5/dc4/c110 0 2024-09-16T23:49:07.455 INFO:tasks.workunit.client.1.smithi043.stdout:7/822: truncate d4/db/d11/d15/d29/d7a/fe3 299515 0 2024-09-16T23:49:07.456 INFO:tasks.workunit.client.1.smithi043.stdout:9/795: chown d4/d21/d33/d95/fc0 14769 1 2024-09-16T23:49:07.456 INFO:tasks.workunit.client.0.smithi042.stdout:4/976: write de/f31 [3864707,15787] 0 2024-09-16T23:49:07.457 INFO:tasks.workunit.client.1.smithi043.stdout:7/823: write d4/db/d11/fcf [856375,5545] 0 2024-09-16T23:49:07.457 INFO:tasks.workunit.client.1.smithi043.stdout:9/796: stat d4/d1c/d25/d2b/da1/df9/d45/d85/lc7 0 2024-09-16T23:49:07.458 INFO:tasks.workunit.client.1.smithi043.stdout:7/824: dread - d4/db/f104 zero size 2024-09-16T23:49:07.459 INFO:tasks.workunit.client.1.smithi043.stdout:3/827: rename d4/de/d1f/d34/c3c to d4/de/cfe 0 2024-09-16T23:49:07.459 INFO:tasks.workunit.client.0.smithi042.stdout:2/634: dread d0/da/d17/f2d [0,4194304] 0 2024-09-16T23:49:07.461 INFO:tasks.workunit.client.1.smithi043.stdout:7/825: write d4/db/d43/d83/fba [109856,119685] 0 2024-09-16T23:49:07.462 INFO:tasks.workunit.client.0.smithi042.stdout:2/635: write d0/da/d1b/d48/f58 [2635114,91395] 0 2024-09-16T23:49:07.463 INFO:tasks.workunit.client.1.smithi043.stdout:7/826: write d4/db/fb1 [3989237,105466] 0 2024-09-16T23:49:07.466 INFO:tasks.workunit.client.0.smithi042.stdout:8/566: dread da/d19/f1a [0,4194304] 0 2024-09-16T23:49:07.467 INFO:tasks.workunit.client.1.smithi043.stdout:9/797: mkdir d4/d1c/d25/d2b/d4c/da3/dca/dfe 0 2024-09-16T23:49:07.469 INFO:tasks.workunit.client.0.smithi042.stdout:0/576: dwrite d0/df/d23/f31 [0,4194304] 0 2024-09-16T23:49:07.471 INFO:tasks.workunit.client.1.smithi043.stdout:3/828: mknod d4/de/d5d/cff 0 2024-09-16T23:49:07.472 INFO:tasks.workunit.client.0.smithi042.stdout:8/567: dread da/d19/d1d/d30/f41 [0,4194304] 0 2024-09-16T23:49:07.472 INFO:tasks.workunit.client.0.smithi042.stdout:0/577: rmdir d0/df/d13/d49 39 2024-09-16T23:49:07.474 INFO:tasks.workunit.client.1.smithi043.stdout:4/874: sync 2024-09-16T23:49:07.474 INFO:tasks.workunit.client.1.smithi043.stdout:2/761: sync 2024-09-16T23:49:07.475 INFO:tasks.workunit.client.1.smithi043.stdout:8/788: write de/d70/d3d/f88 [5309518,24205] 0 2024-09-16T23:49:07.475 INFO:tasks.workunit.client.0.smithi042.stdout:0/578: chown d0/d15/f7c 4955568 1 2024-09-16T23:49:07.476 INFO:tasks.workunit.client.1.smithi043.stdout:8/789: chown de/d18/f36 81773331 1 2024-09-16T23:49:07.477 INFO:tasks.workunit.client.1.smithi043.stdout:8/790: read - de/d70/d3d/daf/fd6 zero size 2024-09-16T23:49:07.479 INFO:tasks.workunit.client.0.smithi042.stdout:7/773: dwrite d3/d1f/f3d [4194304,4194304] 0 2024-09-16T23:49:07.479 INFO:tasks.workunit.client.1.smithi043.stdout:7/827: unlink d4/d27/d41/d1b/d75/ffc 0 2024-09-16T23:49:07.479 INFO:tasks.workunit.client.1.smithi043.stdout:9/798: mknod d4/d28/d5f/d86/cff 0 2024-09-16T23:49:07.479 INFO:tasks.workunit.client.1.smithi043.stdout:6/800: write de/d23/d34/d9b/ffd [787236,37109] 0 2024-09-16T23:49:07.480 INFO:tasks.workunit.client.0.smithi042.stdout:4/977: dwrite de/d42/d68/fb1 [0,4194304] 0 2024-09-16T23:49:07.481 INFO:tasks.workunit.client.1.smithi043.stdout:3/829: write d4/d30/fa6 [3659560,12391] 0 2024-09-16T23:49:07.482 INFO:tasks.workunit.client.1.smithi043.stdout:2/762: mknod d8/d5e/c103 0 2024-09-16T23:49:07.482 INFO:tasks.workunit.client.0.smithi042.stdout:7/774: dread - d3/d98/da0/d41/f100 zero size 2024-09-16T23:49:07.483 INFO:tasks.workunit.client.1.smithi043.stdout:3/830: readlink d4/de/d40/df0/l76 0 2024-09-16T23:49:07.486 INFO:tasks.workunit.client.0.smithi042.stdout:0/579: creat d0/df/d13/d2d/d35/d9f/fb5 x:0 0 0 2024-09-16T23:49:07.488 INFO:tasks.workunit.client.0.smithi042.stdout:7/775: creat d3/d3f/f111 x:0 0 0 2024-09-16T23:49:07.489 INFO:tasks.workunit.client.0.smithi042.stdout:8/568: dread da/d19/d1d/d30/d3f/d6d/f7e [0,4194304] 0 2024-09-16T23:49:07.491 INFO:tasks.workunit.client.0.smithi042.stdout:0/580: readlink d0/df/d23/d3e/d8f/la3 0 2024-09-16T23:49:07.492 INFO:tasks.workunit.client.1.smithi043.stdout:8/791: dread de/d18/d1f/d2f/d39/d4d/f5d [0,4194304] 0 2024-09-16T23:49:07.492 INFO:tasks.workunit.client.1.smithi043.stdout:9/799: unlink d4/d1c/cb3 0 2024-09-16T23:49:07.493 INFO:tasks.workunit.client.0.smithi042.stdout:8/569: chown da/d19/d2c/f36 0 1 2024-09-16T23:49:07.494 INFO:tasks.workunit.client.1.smithi043.stdout:6/801: mknod de/d23/d34/d9b/d50/c110 0 2024-09-16T23:49:07.494 INFO:tasks.workunit.client.0.smithi042.stdout:4/978: getdents de/d10d/db8 0 2024-09-16T23:49:07.494 INFO:tasks.workunit.client.1.smithi043.stdout:9/800: write d4/d1c/d25/d2b/d4c/f72 [4262993,7132] 0 2024-09-16T23:49:07.495 INFO:tasks.workunit.client.1.smithi043.stdout:8/792: chown de/d18/d1f/d25/fb9 1 1 2024-09-16T23:49:07.495 INFO:tasks.workunit.client.1.smithi043.stdout:2/763: creat d8/d5e/d83/d9c/f104 x:0 0 0 2024-09-16T23:49:07.497 INFO:tasks.workunit.client.0.smithi042.stdout:2/636: dwrite d0/da/d17/d5c/fa8 [4194304,4194304] 0 2024-09-16T23:49:07.499 INFO:tasks.workunit.client.0.smithi042.stdout:0/581: creat d0/df/d23/d3e/d8f/fb6 x:0 0 0 2024-09-16T23:49:07.502 INFO:tasks.workunit.client.1.smithi043.stdout:2/764: unlink d8/d1d/d25/fa4 0 2024-09-16T23:49:07.505 INFO:tasks.workunit.client.0.smithi042.stdout:8/570: rename da/d19/d1d/d30/d3f/d6d/f91 to da/d19/d2c/d56/fae 0 2024-09-16T23:49:07.505 INFO:tasks.workunit.client.1.smithi043.stdout:2/765: dread d8/d11/d16/f19 [0,4194304] 0 2024-09-16T23:49:07.506 INFO:tasks.workunit.client.1.smithi043.stdout:2/766: dread - d8/d5e/d83/d9c/f104 zero size 2024-09-16T23:49:07.507 INFO:tasks.workunit.client.0.smithi042.stdout:0/582: symlink d0/df/d13/d49/d70/da1/lb7 0 2024-09-16T23:49:07.507 INFO:tasks.workunit.client.0.smithi042.stdout:7/776: dread d3/d1f/f3d [0,4194304] 0 2024-09-16T23:49:07.508 INFO:tasks.workunit.client.0.smithi042.stdout:4/979: truncate de/d2e/d37/d3e/f107 2939825 0 2024-09-16T23:49:07.509 INFO:tasks.workunit.client.1.smithi043.stdout:9/801: mkdir d4/d5/d64/df8/d100 0 2024-09-16T23:49:07.509 INFO:tasks.workunit.client.0.smithi042.stdout:0/583: mknod d0/d48/daf/cb8 0 2024-09-16T23:49:07.510 INFO:tasks.workunit.client.1.smithi043.stdout:2/767: dread d8/d11/d16/f81 [0,4194304] 0 2024-09-16T23:49:07.510 INFO:tasks.workunit.client.0.smithi042.stdout:4/980: readlink de/d2e/d37/l6f 0 2024-09-16T23:49:07.511 INFO:tasks.workunit.client.1.smithi043.stdout:2/768: readlink d8/d5e/d83/d3a/d77/ldf 0 2024-09-16T23:49:07.512 INFO:tasks.workunit.client.1.smithi043.stdout:7/828: write d4/fa0 [1171442,34916] 0 2024-09-16T23:49:07.512 INFO:tasks.workunit.client.1.smithi043.stdout:2/769: dread - d8/d35/d9f/fdc zero size 2024-09-16T23:49:07.513 INFO:tasks.workunit.client.0.smithi042.stdout:0/584: write d0/df/d13/d2d/d35/f73 [227671,64989] 0 2024-09-16T23:49:07.515 INFO:tasks.workunit.client.0.smithi042.stdout:7/777: dread d3/d1f/d49/d76/d19/d63/f70 [0,4194304] 0 2024-09-16T23:49:07.515 INFO:tasks.workunit.client.0.smithi042.stdout:4/981: symlink de/d42/l14f 0 2024-09-16T23:49:07.516 INFO:tasks.workunit.client.1.smithi043.stdout:7/829: dread d4/db/d11/d15/f3d [0,4194304] 0 2024-09-16T23:49:07.517 INFO:tasks.workunit.client.0.smithi042.stdout:0/585: mknod d0/df/d23/d33/d56/cb9 0 2024-09-16T23:49:07.518 INFO:tasks.workunit.client.0.smithi042.stdout:7/778: creat d3/d1f/d49/d78/dbe/dc8/f112 x:0 0 0 2024-09-16T23:49:07.519 INFO:tasks.workunit.client.1.smithi043.stdout:9/802: fsync d4/d28/d5f/d91/fa6 0 2024-09-16T23:49:07.519 INFO:tasks.workunit.client.1.smithi043.stdout:3/831: write d4/de/d1f/d2c/f4a [693139,39911] 0 2024-09-16T23:49:07.523 INFO:tasks.workunit.client.0.smithi042.stdout:2/637: rmdir d0/da/d88/dae/d8e 0 2024-09-16T23:49:07.523 INFO:tasks.workunit.client.1.smithi043.stdout:4/875: dwrite d8/d25/d39/d41/f60 [0,4194304] 0 2024-09-16T23:49:07.524 INFO:tasks.workunit.client.0.smithi042.stdout:0/586: rename d0/df/d23/d33/d51/d79/c9d to d0/df/d23/d33/cba 0 2024-09-16T23:49:07.524 INFO:tasks.workunit.client.1.smithi043.stdout:9/803: truncate d4/d1c/d25/d2b/d4c/da3/ff2 731360 0 2024-09-16T23:49:07.527 INFO:tasks.workunit.client.0.smithi042.stdout:8/571: write da/f2a [1607306,120432] 0 2024-09-16T23:49:07.530 INFO:tasks.workunit.client.1.smithi043.stdout:8/793: link de/d18/d1f/d2f/d39/d7b/d83/fe5 de/d18/d27/d41/fee 0 2024-09-16T23:49:07.531 INFO:tasks.workunit.client.0.smithi042.stdout:4/982: truncate de/d12/d7d/fe9 652046 0 2024-09-16T23:49:07.532 INFO:tasks.workunit.client.0.smithi042.stdout:0/587: creat d0/df/d13/d2d/d35/fbb x:0 0 0 2024-09-16T23:49:07.532 INFO:tasks.workunit.client.1.smithi043.stdout:3/832: mknod d4/de/d1f/d2c/db7/dc3/c100 0 2024-09-16T23:49:07.533 INFO:tasks.workunit.client.0.smithi042.stdout:2/638: dwrite d0/da/d1b/d3b/d44/f90 [0,4194304] 0 2024-09-16T23:49:07.534 INFO:tasks.workunit.client.1.smithi043.stdout:4/876: dwrite d8/d25/d39/dd3/d100/d105/f106 [0,4194304] 0 2024-09-16T23:49:07.536 INFO:tasks.workunit.client.1.smithi043.stdout:9/804: creat d4/d84/f101 x:0 0 0 2024-09-16T23:49:07.537 INFO:tasks.workunit.client.1.smithi043.stdout:2/770: symlink d8/d1d/d20/d23/dfc/l105 0 2024-09-16T23:49:07.538 INFO:tasks.workunit.client.0.smithi042.stdout:0/588: creat d0/df/d23/d33/d56/da9/fbc x:0 0 0 2024-09-16T23:49:07.539 INFO:tasks.workunit.client.1.smithi043.stdout:9/805: write d4/d28/d5f/d91/fad [31738,127010] 0 2024-09-16T23:49:07.542 INFO:tasks.workunit.client.0.smithi042.stdout:4/983: creat de/d12/d7d/f150 x:0 0 0 2024-09-16T23:49:07.542 INFO:tasks.workunit.client.0.smithi042.stdout:2/639: read d0/f1a [3214953,53248] 0 2024-09-16T23:49:07.543 INFO:tasks.workunit.client.0.smithi042.stdout:2/640: chown d0/d78/d8b 950 1 2024-09-16T23:49:07.544 INFO:tasks.workunit.client.0.smithi042.stdout:0/589: dread d0/d15/f7c [0,4194304] 0 2024-09-16T23:49:07.545 INFO:tasks.workunit.client.0.smithi042.stdout:0/590: chown d0/l69 4989 1 2024-09-16T23:49:07.545 INFO:tasks.workunit.client.0.smithi042.stdout:8/572: link da/d19/d1d/d30/d49/f7b da/d19/d1d/d5a/faf 0 2024-09-16T23:49:07.550 INFO:tasks.workunit.client.1.smithi043.stdout:4/877: unlink d8/d15/d20/d2b/d54/db3/l4e 0 2024-09-16T23:49:07.551 INFO:tasks.workunit.client.0.smithi042.stdout:2/641: mknod d0/da/d1b/d48/cd8 0 2024-09-16T23:49:07.552 INFO:tasks.workunit.client.1.smithi043.stdout:8/794: dread de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:49:07.552 INFO:tasks.workunit.client.1.smithi043.stdout:4/878: write d8/d25/d39/d41/d57/d85/f102 [5287,114792] 0 2024-09-16T23:49:07.555 INFO:tasks.workunit.client.1.smithi043.stdout:4/879: write d8/d25/d39/d41/d57/d85/f102 [766035,124022] 0 2024-09-16T23:49:07.558 INFO:tasks.workunit.client.0.smithi042.stdout:4/984: write de/d12/f15 [5906634,89926] 0 2024-09-16T23:49:07.558 INFO:tasks.workunit.client.0.smithi042.stdout:0/591: write d0/f37 [1155620,93518] 0 2024-09-16T23:49:07.559 INFO:tasks.workunit.client.1.smithi043.stdout:4/880: write d8/d15/d20/dfa/d5e/fff [86632,98139] 0 2024-09-16T23:49:07.562 INFO:tasks.workunit.client.0.smithi042.stdout:8/573: mkdir da/d19/d2c/db0 0 2024-09-16T23:49:07.563 INFO:tasks.workunit.client.0.smithi042.stdout:4/985: read de/d42/d82/f115 [3484485,116721] 0 2024-09-16T23:49:07.565 INFO:tasks.workunit.client.1.smithi043.stdout:7/830: creat d4/db/d11/f111 x:0 0 0 2024-09-16T23:49:07.566 INFO:tasks.workunit.client.0.smithi042.stdout:4/986: creat de/d42/d99/d9e/f151 x:0 0 0 2024-09-16T23:49:07.567 INFO:tasks.workunit.client.1.smithi043.stdout:8/795: stat de/d18/d1f/d25/d65/d7f/d95/l9b 0 2024-09-16T23:49:07.567 INFO:tasks.workunit.client.0.smithi042.stdout:0/592: write d0/f4b [687180,97855] 0 2024-09-16T23:49:07.567 INFO:tasks.workunit.client.0.smithi042.stdout:2/642: dread d0/da/d17/f37 [0,4194304] 0 2024-09-16T23:49:07.569 INFO:tasks.workunit.client.1.smithi043.stdout:3/833: dread d4/de/d1f/d2c/db7/dc3/fc1 [0,4194304] 0 2024-09-16T23:49:07.569 INFO:tasks.workunit.client.0.smithi042.stdout:0/593: dread - d0/df/d23/d33/d51/f52 zero size 2024-09-16T23:49:07.571 INFO:tasks.workunit.client.1.smithi043.stdout:4/881: creat d8/d25/d39/d41/d57/ded/f110 x:0 0 0 2024-09-16T23:49:07.572 INFO:tasks.workunit.client.0.smithi042.stdout:4/987: rename de/d2e/d37/dc3/fd0 to de/d1a/d70/dac/f152 0 2024-09-16T23:49:07.573 INFO:tasks.workunit.client.1.smithi043.stdout:7/831: fsync d4/db/d11/d15/f5f 0 2024-09-16T23:49:07.574 INFO:tasks.workunit.client.0.smithi042.stdout:4/988: stat de/d2e/c32 0 2024-09-16T23:49:07.574 INFO:tasks.workunit.client.0.smithi042.stdout:0/594: symlink d0/df/d23/d33/d51/d79/lbd 0 2024-09-16T23:49:07.575 INFO:tasks.workunit.client.0.smithi042.stdout:0/595: fdatasync d0/df/d23/d3e/fb4 0 2024-09-16T23:49:07.576 INFO:tasks.workunit.client.1.smithi043.stdout:8/796: dwrite de/d18/d1f/d2f/d39/fb3 [0,4194304] 0 2024-09-16T23:49:07.576 INFO:tasks.workunit.client.1.smithi043.stdout:4/882: fdatasync d8/d25/d39/dd3/d100/d108/faf 0 2024-09-16T23:49:07.577 INFO:tasks.workunit.client.0.smithi042.stdout:4/989: unlink de/d42/d68/f96 0 2024-09-16T23:49:07.578 INFO:tasks.workunit.client.1.smithi043.stdout:9/806: dwrite d4/d84/fb2 [0,4194304] 0 2024-09-16T23:49:07.578 INFO:tasks.workunit.client.0.smithi042.stdout:0/596: rmdir d0/df/d23 39 2024-09-16T23:49:07.578 INFO:tasks.workunit.client.0.smithi042.stdout:4/990: dread de/d42/d99/d9e/f140 [0,4194304] 0 2024-09-16T23:49:07.579 INFO:tasks.workunit.client.0.smithi042.stdout:8/574: link da/l5c da/d4b/lb1 0 2024-09-16T23:49:07.580 INFO:tasks.workunit.client.0.smithi042.stdout:0/597: read d0/d15/f2f [920019,35402] 0 2024-09-16T23:49:07.581 INFO:tasks.workunit.client.1.smithi043.stdout:9/807: write d4/d28/d5f/d91/fe0 [544896,40825] 0 2024-09-16T23:49:07.581 INFO:tasks.workunit.client.0.smithi042.stdout:0/598: read d0/f67 [1971978,123349] 0 2024-09-16T23:49:07.583 INFO:tasks.workunit.client.0.smithi042.stdout:2/643: getdents d0/d30/d43/d77 0 2024-09-16T23:49:07.588 INFO:tasks.workunit.client.0.smithi042.stdout:4/991: creat de/d2e/f153 x:0 0 0 2024-09-16T23:49:07.589 INFO:tasks.workunit.client.0.smithi042.stdout:8/575: mkdir da/d19/d1d/d30/d49/db2 0 2024-09-16T23:49:07.590 INFO:tasks.workunit.client.0.smithi042.stdout:0/599: dread d0/df/d23/f55 [0,4194304] 0 2024-09-16T23:49:07.591 INFO:tasks.workunit.client.0.smithi042.stdout:2/644: symlink d0/d30/ld9 0 2024-09-16T23:49:07.592 INFO:tasks.workunit.client.0.smithi042.stdout:2/645: fdatasync d0/da/d17/fc1 0 2024-09-16T23:49:07.592 INFO:tasks.workunit.client.1.smithi043.stdout:8/797: chown de/d18/fb1 19 1 2024-09-16T23:49:07.594 INFO:tasks.workunit.client.1.smithi043.stdout:8/798: chown de/d18/d2b/l9f 3160268 1 2024-09-16T23:49:07.594 INFO:tasks.workunit.client.1.smithi043.stdout:2/771: link d8/d5e/d83/d3a/l3b d8/d5e/d83/d3a/l106 0 2024-09-16T23:49:07.595 INFO:tasks.workunit.client.1.smithi043.stdout:2/772: stat d8/d5e/d83/d3a/d77/da9/f100 0 2024-09-16T23:49:07.597 INFO:tasks.workunit.client.0.smithi042.stdout:2/646: creat d0/da/d1b/d48/d81/fda x:0 0 0 2024-09-16T23:49:07.598 INFO:tasks.workunit.client.0.smithi042.stdout:0/600: truncate d0/df/d23/f38 1541993 0 2024-09-16T23:49:07.599 INFO:tasks.workunit.client.0.smithi042.stdout:8/576: truncate da/d19/d1d/f3c 325488 0 2024-09-16T23:49:07.601 INFO:tasks.workunit.client.1.smithi043.stdout:3/834: creat d4/d30/f101 x:0 0 0 2024-09-16T23:49:07.604 INFO:tasks.workunit.client.1.smithi043.stdout:2/773: dread d8/d4b/f5b [0,4194304] 0 2024-09-16T23:49:07.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/601: rename d0/df/d13/d49/d70/da1/dac/lae to d0/df/d13/d49/d70/da1/lbe 0 2024-09-16T23:49:07.605 INFO:tasks.workunit.client.0.smithi042.stdout:2/647: dread d0/da/d1b/d3b/d44/fb3 [0,4194304] 0 2024-09-16T23:49:07.606 INFO:tasks.workunit.client.0.smithi042.stdout:0/602: write d0/f9 [5146611,100885] 0 2024-09-16T23:49:07.607 INFO:tasks.workunit.client.0.smithi042.stdout:4/992: dread de/d1a/da7/fd2 [4194304,4194304] 0 2024-09-16T23:49:07.607 INFO:tasks.workunit.client.1.smithi043.stdout:4/883: write d8/d25/f6a [13100,19858] 0 2024-09-16T23:49:07.610 INFO:tasks.workunit.client.0.smithi042.stdout:2/648: symlink d0/d26/ldb 0 2024-09-16T23:49:07.611 INFO:tasks.workunit.client.0.smithi042.stdout:4/993: creat de/d2e/d37/d4c/dd4/f154 x:0 0 0 2024-09-16T23:49:07.611 INFO:tasks.workunit.client.0.smithi042.stdout:2/649: dread - d0/da/d1b/d48/fc5 zero size 2024-09-16T23:49:07.612 INFO:tasks.workunit.client.0.smithi042.stdout:0/603: mknod d0/df/cbf 0 2024-09-16T23:49:07.613 INFO:tasks.workunit.client.1.smithi043.stdout:8/799: dread de/d70/d3d/f88 [0,4194304] 0 2024-09-16T23:49:07.613 INFO:tasks.workunit.client.1.smithi043.stdout:3/835: mknod d4/de/d1f/d2c/db7/dc3/c102 0 2024-09-16T23:49:07.613 INFO:tasks.workunit.client.0.smithi042.stdout:4/994: write de/f31 [1552098,22977] 0 2024-09-16T23:49:07.613 INFO:tasks.workunit.client.1.smithi043.stdout:7/832: truncate d4/f2b 3113238 0 2024-09-16T23:49:07.616 INFO:tasks.workunit.client.1.smithi043.stdout:9/808: dwrite d4/d1c/f6f [0,4194304] 0 2024-09-16T23:49:07.617 INFO:tasks.workunit.client.0.smithi042.stdout:4/995: read de/d42/d5d/d8f/dd6/d123/f139 [3649405,36846] 0 2024-09-16T23:49:07.617 INFO:tasks.workunit.client.1.smithi043.stdout:8/800: chown de/d18/d27/d41/d5a/dc8/ldb 95510297 1 2024-09-16T23:49:07.617 INFO:tasks.workunit.client.1.smithi043.stdout:2/774: rmdir d8/d11/d16 39 2024-09-16T23:49:07.619 INFO:tasks.workunit.client.1.smithi043.stdout:8/801: readlink de/d18/d1f/d2f/d39/d7b/d83/da5/le8 0 2024-09-16T23:49:07.619 INFO:tasks.workunit.client.1.smithi043.stdout:4/884: fdatasync d8/d15/d20/d2b/d54/db3/d4c/f9c 0 2024-09-16T23:49:07.620 INFO:tasks.workunit.client.0.smithi042.stdout:2/650: dwrite d0/d30/d43/fac [0,4194304] 0 2024-09-16T23:49:07.625 INFO:tasks.workunit.client.1.smithi043.stdout:2/775: dwrite d8/d35/f36 [4194304,4194304] 0 2024-09-16T23:49:07.628 INFO:tasks.workunit.client.1.smithi043.stdout:7/833: unlink d4/d27/dbc/ce8 0 2024-09-16T23:49:07.630 INFO:tasks.workunit.client.0.smithi042.stdout:2/651: truncate d0/da/d17/d3e/d86/fa1 4806329 0 2024-09-16T23:49:07.632 INFO:tasks.workunit.client.1.smithi043.stdout:9/809: readlink d4/d1c/d25/d2b/lf4 0 2024-09-16T23:49:07.634 INFO:tasks.workunit.client.1.smithi043.stdout:9/810: write d4/d21/d33/f79 [1729050,2208] 0 2024-09-16T23:49:07.634 INFO:tasks.workunit.client.0.smithi042.stdout:2/652: mknod d0/d26/cdc 0 2024-09-16T23:49:07.635 INFO:tasks.workunit.client.0.smithi042.stdout:4/996: truncate de/d10d/d8a/db2/fdf 3841457 0 2024-09-16T23:49:07.635 INFO:tasks.workunit.client.1.smithi043.stdout:9/811: write d4/d5/f1d [4421186,3349] 0 2024-09-16T23:49:07.636 INFO:tasks.workunit.client.0.smithi042.stdout:2/653: dread - d0/d8d/da0/fd6 zero size 2024-09-16T23:49:07.637 INFO:tasks.workunit.client.0.smithi042.stdout:0/604: write d0/df/d13/d2d/d35/fa5 [1395117,89960] 0 2024-09-16T23:49:07.639 INFO:tasks.workunit.client.1.smithi043.stdout:3/836: write d4/de/f21 [3760645,15414] 0 2024-09-16T23:49:07.639 INFO:tasks.workunit.client.1.smithi043.stdout:0/933: sync 2024-09-16T23:49:07.639 INFO:tasks.workunit.client.1.smithi043.stdout:5/968: sync 2024-09-16T23:49:07.640 INFO:tasks.workunit.client.0.smithi042.stdout:4/997: symlink de/d42/d5d/d8f/dd6/d123/l155 0 2024-09-16T23:49:07.641 INFO:tasks.workunit.client.1.smithi043.stdout:8/802: dread de/d18/d27/d41/f50 [0,4194304] 0 2024-09-16T23:49:07.643 INFO:tasks.workunit.client.1.smithi043.stdout:0/934: chown d0/d2/df/d1a/d46/d39/f7a 0 1 2024-09-16T23:49:07.644 INFO:tasks.workunit.client.0.smithi042.stdout:4/998: symlink de/d10d/d128/l156 0 2024-09-16T23:49:07.646 INFO:tasks.workunit.client.0.smithi042.stdout:4/999: dread - de/d42/d85/db3/f141 zero size 2024-09-16T23:49:07.647 INFO:tasks.workunit.client.0.smithi042.stdout:0/605: rename d0/df/d23/d3e/l4c to d0/df/d13/lc0 0 2024-09-16T23:49:07.648 INFO:tasks.workunit.client.1.smithi043.stdout:8/803: dwrite de/d18/d1f/d25/d65/d7f/dca/fe3 [0,4194304] 0 2024-09-16T23:49:07.650 INFO:tasks.workunit.client.0.smithi042.stdout:2/654: link d0/c1e d0/da/d1b/d48/cdd 0 2024-09-16T23:49:07.653 INFO:tasks.workunit.client.1.smithi043.stdout:3/837: dread d4/de/f51 [0,4194304] 0 2024-09-16T23:49:07.653 INFO:tasks.workunit.client.1.smithi043.stdout:5/969: mknod d3/d9/d6f/d7c/d80/c14a 0 2024-09-16T23:49:07.653 INFO:tasks.workunit.client.0.smithi042.stdout:2/655: truncate d0/d26/f9f 2962384 0 2024-09-16T23:49:07.655 INFO:tasks.workunit.client.0.smithi042.stdout:0/606: dwrite d0/df/d23/d33/d51/d79/fb0 [0,4194304] 0 2024-09-16T23:49:07.656 INFO:tasks.workunit.client.1.smithi043.stdout:9/812: symlink d4/d28/dcf/dd0/l102 0 2024-09-16T23:49:07.658 INFO:tasks.workunit.client.0.smithi042.stdout:6/664: sync 2024-09-16T23:49:07.658 INFO:tasks.workunit.client.1.smithi043.stdout:2/776: link d8/d5e/d83/d3a/c9a d8/d5e/d83/d3a/d77/dea/c107 0 2024-09-16T23:49:07.661 INFO:tasks.workunit.client.0.smithi042.stdout:2/656: mknod d0/d30/cde 0 2024-09-16T23:49:07.662 INFO:tasks.workunit.client.0.smithi042.stdout:0/607: symlink d0/df/d23/d33/d56/da9/lc1 0 2024-09-16T23:49:07.663 INFO:tasks.workunit.client.0.smithi042.stdout:6/665: readlink d8/d70/d81/dd/d1b/d50/da2/lcb 0 2024-09-16T23:49:07.664 INFO:tasks.workunit.client.1.smithi043.stdout:4/885: dwrite d8/d15/d20/d2b/fd6 [0,4194304] 0 2024-09-16T23:49:07.664 INFO:tasks.workunit.client.1.smithi043.stdout:3/838: rename c0 to d4/de/d1f/d2c/c103 0 2024-09-16T23:49:07.665 INFO:tasks.workunit.client.0.smithi042.stdout:0/608: unlink d0/df/d23/f27 0 2024-09-16T23:49:07.666 INFO:tasks.workunit.client.1.smithi043.stdout:7/834: dwrite d4/d27/d41/f40 [0,4194304] 0 2024-09-16T23:49:07.667 INFO:tasks.workunit.client.0.smithi042.stdout:6/666: truncate d8/d70/d84/fb5 203530 0 2024-09-16T23:49:07.668 INFO:tasks.workunit.client.0.smithi042.stdout:2/657: truncate d0/da/d17/fbe 11281 0 2024-09-16T23:49:07.669 INFO:tasks.workunit.client.1.smithi043.stdout:8/804: symlink de/d18/d27/d41/lef 0 2024-09-16T23:49:07.670 INFO:tasks.workunit.client.1.smithi043.stdout:2/777: creat d8/d5e/d83/d3a/d77/f108 x:0 0 0 2024-09-16T23:49:07.671 INFO:tasks.workunit.client.1.smithi043.stdout:3/839: chown d4/de/d1f/d2c/db7/dc3/c9c 23085 1 2024-09-16T23:49:07.671 INFO:tasks.workunit.client.1.smithi043.stdout:8/805: read - de/d70/fc9 zero size 2024-09-16T23:49:07.671 INFO:tasks.workunit.client.1.smithi043.stdout:7/835: fdatasync d4/d27/d41/d1b/d75/fde 0 2024-09-16T23:49:07.672 INFO:tasks.workunit.client.1.smithi043.stdout:2/778: readlink d8/d5e/d83/d3a/d77/ldf 0 2024-09-16T23:49:07.674 INFO:tasks.workunit.client.1.smithi043.stdout:8/806: dread - de/d18/d1f/d2f/d39/d7b/fb5 zero size 2024-09-16T23:49:07.675 INFO:tasks.workunit.client.0.smithi042.stdout:2/658: creat d0/da/d88/fdf x:0 0 0 2024-09-16T23:49:07.676 INFO:tasks.workunit.client.1.smithi043.stdout:9/813: rename d4/d1c/d25/d2b/da1/df9/d8c to d4/d84/d103 0 2024-09-16T23:49:07.676 INFO:tasks.workunit.client.1.smithi043.stdout:5/970: dread d3/db/d15/f69 [0,4194304] 0 2024-09-16T23:49:07.682 INFO:tasks.workunit.client.1.smithi043.stdout:7/836: chown d4/db/d43/d83/d85/c7d 115477 1 2024-09-16T23:49:07.685 INFO:tasks.workunit.client.1.smithi043.stdout:8/807: creat de/d18/d1f/d2f/d39/d7b/d83/da5/ff0 x:0 0 0 2024-09-16T23:49:07.686 INFO:tasks.workunit.client.1.smithi043.stdout:2/779: dread d8/f43 [0,4194304] 0 2024-09-16T23:49:07.686 INFO:tasks.workunit.client.1.smithi043.stdout:9/814: creat d4/d1c/ddc/f104 x:0 0 0 2024-09-16T23:49:07.689 INFO:tasks.workunit.client.1.smithi043.stdout:3/840: rename d4/de/d1f/d2c/fce to d4/de/d40/df0/d88/db1/dbb/f104 0 2024-09-16T23:49:07.691 INFO:tasks.workunit.client.0.smithi042.stdout:0/609: dread d0/df/f10 [0,4194304] 0 2024-09-16T23:49:07.693 INFO:tasks.workunit.client.0.smithi042.stdout:0/610: write d0/d15/f3c [837801,113529] 0 2024-09-16T23:49:07.694 INFO:tasks.workunit.client.1.smithi043.stdout:2/780: dwrite d8/d35/fde [0,4194304] 0 2024-09-16T23:49:07.695 INFO:tasks.workunit.client.0.smithi042.stdout:0/611: write d0/d15/f81 [2456407,109719] 0 2024-09-16T23:49:07.696 INFO:tasks.workunit.client.1.smithi043.stdout:8/808: mknod de/d18/d1f/d25/cf1 0 2024-09-16T23:49:07.696 INFO:tasks.workunit.client.0.smithi042.stdout:0/612: truncate d0/d15/f7c 1044401 0 2024-09-16T23:49:07.698 INFO:tasks.workunit.client.0.smithi042.stdout:0/613: write d0/df/d23/d33/d51/d79/fb0 [2024243,4166] 0 2024-09-16T23:49:07.703 INFO:tasks.workunit.client.0.smithi042.stdout:0/614: fdatasync d0/df/d23/d33/d56/da9/f65 0 2024-09-16T23:49:07.703 INFO:tasks.workunit.client.0.smithi042.stdout:6/667: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/f83 [0,4194304] 0 2024-09-16T23:49:07.704 INFO:tasks.workunit.client.0.smithi042.stdout:2/659: dread d0/da/d1b/d3b/d44/d54/f5f [0,4194304] 0 2024-09-16T23:49:07.705 INFO:tasks.workunit.client.0.smithi042.stdout:2/660: chown d0/d8d 6547 1 2024-09-16T23:49:07.706 INFO:tasks.workunit.client.1.smithi043.stdout:9/815: dwrite d4/d1c/d25/db1/fcd [0,4194304] 0 2024-09-16T23:49:07.710 INFO:tasks.workunit.client.1.smithi043.stdout:8/809: mknod de/d18/d1f/d25/d65/d7f/d95/cf2 0 2024-09-16T23:49:07.712 INFO:tasks.workunit.client.0.smithi042.stdout:2/661: truncate d0/f1a 2927612 0 2024-09-16T23:49:07.714 INFO:tasks.workunit.client.0.smithi042.stdout:6/668: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/fdb x:0 0 0 2024-09-16T23:49:07.719 INFO:tasks.workunit.client.0.smithi042.stdout:0/615: truncate d0/df/f8c 4802 0 2024-09-16T23:49:07.723 INFO:tasks.workunit.client.0.smithi042.stdout:6/669: creat d8/d70/d81/dd/d1b/d50/d96/da0/dd7/fdc x:0 0 0 2024-09-16T23:49:07.726 INFO:tasks.workunit.client.1.smithi043.stdout:9/816: dwrite d4/d21/d33/fe8 [0,4194304] 0 2024-09-16T23:49:07.727 INFO:tasks.workunit.client.0.smithi042.stdout:2/662: link d0/da/d1b/d3b/d44/d54/c6d d0/d30/d43/ce0 0 2024-09-16T23:49:07.733 INFO:tasks.workunit.client.0.smithi042.stdout:2/663: rename d0/da/d17/d3e/d86/caa to d0/d30/d43/d77/ce1 0 2024-09-16T23:49:07.735 INFO:tasks.workunit.client.1.smithi043.stdout:5/971: dwrite d3/d9/d22/d4a/d4c/fe3 [0,4194304] 0 2024-09-16T23:49:07.737 INFO:tasks.workunit.client.1.smithi043.stdout:3/841: dread d4/de/d1f/d2c/db7/dc3/f8e [0,4194304] 0 2024-09-16T23:49:07.739 INFO:tasks.workunit.client.1.smithi043.stdout:7/837: dwrite d4/db/d11/fc6 [0,4194304] 0 2024-09-16T23:49:07.739 INFO:tasks.workunit.client.1.smithi043.stdout:4/886: dwrite d8/d15/d9b/fa7 [0,4194304] 0 2024-09-16T23:49:07.740 INFO:tasks.workunit.client.0.smithi042.stdout:6/670: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/fdd x:0 0 0 2024-09-16T23:49:07.746 INFO:tasks.workunit.client.0.smithi042.stdout:0/616: dread d0/df/d13/f74 [0,4194304] 0 2024-09-16T23:49:07.746 INFO:tasks.workunit.client.1.smithi043.stdout:2/781: dwrite d8/d5e/d83/fa0 [0,4194304] 0 2024-09-16T23:49:07.753 INFO:tasks.workunit.client.0.smithi042.stdout:0/617: creat d0/df/d23/d33/d51/d79/fc2 x:0 0 0 2024-09-16T23:49:07.753 INFO:tasks.workunit.client.0.smithi042.stdout:6/671: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e/d4f/d72/f2e [0,4194304] 0 2024-09-16T23:49:07.754 INFO:tasks.workunit.client.0.smithi042.stdout:0/618: fsync d0/d15/f81 0 2024-09-16T23:49:07.755 INFO:tasks.workunit.client.1.smithi043.stdout:9/817: link d4/d84/f101 d4/d1c/d6b/d67/dbf/f105 0 2024-09-16T23:49:07.757 INFO:tasks.workunit.client.1.smithi043.stdout:5/972: dwrite d3/d9/d22/d4a/d4c/d7a/f114 [4194304,4194304] 0 2024-09-16T23:49:07.758 INFO:tasks.workunit.client.1.smithi043.stdout:7/838: rename d4/db/c12 to d4/db/d11/d15/dd2/c112 0 2024-09-16T23:49:07.759 INFO:tasks.workunit.client.1.smithi043.stdout:3/842: dwrite d4/de/f21 [4194304,4194304] 0 2024-09-16T23:49:07.759 INFO:tasks.workunit.client.0.smithi042.stdout:0/619: mkdir d0/df/d23/d33/d56/da9/dc3 0 2024-09-16T23:49:07.760 INFO:tasks.workunit.client.1.smithi043.stdout:8/810: dread de/d18/d1f/d2f/f43 [0,4194304] 0 2024-09-16T23:49:07.762 INFO:tasks.workunit.client.0.smithi042.stdout:6/672: unlink d8/d70/d81/dd/d1b/d34/l9e 0 2024-09-16T23:49:07.763 INFO:tasks.workunit.client.0.smithi042.stdout:0/620: fsync d0/df/d13/f4e 0 2024-09-16T23:49:07.764 INFO:tasks.workunit.client.1.smithi043.stdout:5/973: dread - d3/d9/d22/d4a/f13a zero size 2024-09-16T23:49:07.764 INFO:tasks.workunit.client.0.smithi042.stdout:0/621: stat d0/df/cbf 0 2024-09-16T23:49:07.765 INFO:tasks.workunit.client.0.smithi042.stdout:0/622: chown d0/f4b 531303399 1 2024-09-16T23:49:07.766 INFO:tasks.workunit.client.1.smithi043.stdout:9/818: creat d4/d1c/f106 x:0 0 0 2024-09-16T23:49:07.766 INFO:tasks.workunit.client.1.smithi043.stdout:5/974: readlink d3/db/d15/d4d/d6b/d73/ldb 0 2024-09-16T23:49:07.767 INFO:tasks.workunit.client.1.smithi043.stdout:8/811: write de/d18/d1f/d2f/d39/d7b/fb5 [137214,119673] 0 2024-09-16T23:49:07.767 INFO:tasks.workunit.client.0.smithi042.stdout:2/664: dread d0/d30/f38 [0,4194304] 0 2024-09-16T23:49:07.770 INFO:tasks.workunit.client.0.smithi042.stdout:0/623: mknod d0/df/d13/d49/d70/da1/dac/cc4 0 2024-09-16T23:49:07.773 INFO:tasks.workunit.client.1.smithi043.stdout:6/802: sync 2024-09-16T23:49:07.777 INFO:tasks.workunit.client.0.smithi042.stdout:2/665: mknod d0/da/d88/dae/ce2 0 2024-09-16T23:49:07.780 INFO:tasks.workunit.client.1.smithi043.stdout:3/843: write d4/de/d1f/d2c/db7/dc3/f8c [4022790,54149] 0 2024-09-16T23:49:07.781 INFO:tasks.workunit.client.0.smithi042.stdout:0/624: dwrite d0/df/d13/d2d/d35/fa5 [0,4194304] 0 2024-09-16T23:49:07.783 INFO:tasks.workunit.client.1.smithi043.stdout:4/887: link d8/d25/d39/d41/d57/d85/f102 d8/d15/d20/dfa/f111 0 2024-09-16T23:49:07.784 INFO:tasks.workunit.client.0.smithi042.stdout:0/625: mkdir d0/df/d23/d33/d56/dc5 0 2024-09-16T23:49:07.786 INFO:tasks.workunit.client.1.smithi043.stdout:8/812: rmdir de/d18/d1f/d2f/d39/d7b/d83/dbb/dce 39 2024-09-16T23:49:07.791 INFO:tasks.workunit.client.1.smithi043.stdout:7/839: link d4/db/d11/d15/fe4 d4/db/d11/d21/d31/d33/f113 0 2024-09-16T23:49:07.791 INFO:tasks.workunit.client.0.smithi042.stdout:6/673: dwrite d8/d70/d81/f23 [0,4194304] 0 2024-09-16T23:49:07.797 INFO:tasks.workunit.client.1.smithi043.stdout:9/819: mknod d4/d1c/d25/d2b/da1/df9/dd8/c107 0 2024-09-16T23:49:07.801 INFO:tasks.workunit.client.1.smithi043.stdout:2/782: rename d8/d4b/d4f/d7c/c101 to d8/d4b/c109 0 2024-09-16T23:49:07.803 INFO:tasks.workunit.client.0.smithi042.stdout:2/666: dwrite d0/da/d17/d3e/d86/d96/f9c [0,4194304] 0 2024-09-16T23:49:07.804 INFO:tasks.workunit.client.1.smithi043.stdout:2/783: write d8/d1d/f5a [1865584,99402] 0 2024-09-16T23:49:07.805 INFO:tasks.workunit.client.0.smithi042.stdout:2/667: write d0/d30/d43/d77/fbc [941182,90397] 0 2024-09-16T23:49:07.808 INFO:tasks.workunit.client.1.smithi043.stdout:6/803: dwrite de/d1a/d39/f78 [0,4194304] 0 2024-09-16T23:49:07.810 INFO:tasks.workunit.client.0.smithi042.stdout:2/668: rmdir d0/da/d17/d5c 39 2024-09-16T23:49:07.811 INFO:tasks.workunit.client.1.smithi043.stdout:3/844: unlink d4/de/d40/laf 0 2024-09-16T23:49:07.811 INFO:tasks.workunit.client.1.smithi043.stdout:8/813: dread de/d18/d1f/d2f/d39/d7b/d83/f84 [0,4194304] 0 2024-09-16T23:49:07.813 INFO:tasks.workunit.client.1.smithi043.stdout:7/840: truncate d4/db/d28/f97 2041318 0 2024-09-16T23:49:07.814 INFO:tasks.workunit.client.1.smithi043.stdout:3/845: fsync d4/de/d40/df0/d88/db1/dbb/fe4 0 2024-09-16T23:49:07.814 INFO:tasks.workunit.client.1.smithi043.stdout:7/841: read - d4/d27/dbc/fdb zero size 2024-09-16T23:49:07.815 INFO:tasks.workunit.client.1.smithi043.stdout:0/935: sync 2024-09-16T23:49:07.815 INFO:tasks.workunit.client.0.smithi042.stdout:6/674: write d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f78 [1689190,2341] 0 2024-09-16T23:49:07.817 INFO:tasks.workunit.client.1.smithi043.stdout:9/820: mkdir d4/d1c/d25/db1/d108 0 2024-09-16T23:49:07.819 INFO:tasks.workunit.client.1.smithi043.stdout:5/975: rename d3/da4/l12e to d3/db/d15/d13d/l14b 0 2024-09-16T23:49:07.821 INFO:tasks.workunit.client.0.smithi042.stdout:2/669: rename d0/da/d1b/c45 to d0/da/d1b/d3b/d44/ce3 0 2024-09-16T23:49:07.824 INFO:tasks.workunit.client.1.smithi043.stdout:4/888: truncate d8/d15/f69 3718239 0 2024-09-16T23:49:07.827 INFO:tasks.workunit.client.0.smithi042.stdout:6/675: rename d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d4e to d8/d70/d81/dd/d1b/dbf/dde 0 2024-09-16T23:49:07.829 INFO:tasks.workunit.client.0.smithi042.stdout:2/670: symlink d0/da/d88/dae/le4 0 2024-09-16T23:49:07.838 INFO:tasks.workunit.client.1.smithi043.stdout:8/814: mkdir de/d18/d27/d8e/df3 0 2024-09-16T23:49:07.839 INFO:tasks.workunit.client.0.smithi042.stdout:6/676: write d8/d70/d81/dd/d1b/d2d/f41 [711780,4810] 0 2024-09-16T23:49:07.839 INFO:tasks.workunit.client.1.smithi043.stdout:6/804: write de/d1a/d39/f17 [1873704,61818] 0 2024-09-16T23:49:07.839 INFO:tasks.workunit.client.1.smithi043.stdout:0/936: readlink d0/d2/df/l26 0 2024-09-16T23:49:07.841 INFO:tasks.workunit.client.1.smithi043.stdout:6/805: write de/d1a/d28/f3e [2198426,41873] 0 2024-09-16T23:49:07.842 INFO:tasks.workunit.client.1.smithi043.stdout:7/842: creat d4/db/d11/d21/d31/f114 x:0 0 0 2024-09-16T23:49:07.844 INFO:tasks.workunit.client.0.smithi042.stdout:2/671: dread d0/da/d17/f1c [0,4194304] 0 2024-09-16T23:49:07.845 INFO:tasks.workunit.client.0.smithi042.stdout:2/672: stat d0/d30/d43 0 2024-09-16T23:49:07.846 INFO:tasks.workunit.client.1.smithi043.stdout:5/976: fsync d3/d1b/fa6 0 2024-09-16T23:49:07.850 INFO:tasks.workunit.client.1.smithi043.stdout:9/821: write d4/d21/f4d [1564796,4804] 0 2024-09-16T23:49:07.853 INFO:tasks.workunit.client.0.smithi042.stdout:2/673: dwrite d0/da/d1b/d48/db7/fd2 [0,4194304] 0 2024-09-16T23:49:07.856 INFO:tasks.workunit.client.0.smithi042.stdout:6/677: dread d8/f37 [0,4194304] 0 2024-09-16T23:49:07.856 INFO:tasks.workunit.client.1.smithi043.stdout:9/822: dwrite d4/d5/d16/f1f [0,4194304] 0 2024-09-16T23:49:07.857 INFO:tasks.workunit.client.0.smithi042.stdout:6/678: stat d8/d70/d81/dd/d1b/d67/lc6 0 2024-09-16T23:49:07.859 INFO:tasks.workunit.client.1.smithi043.stdout:4/889: dwrite d8/d15/d20/d2b/d22/fc2 [0,4194304] 0 2024-09-16T23:49:07.862 INFO:tasks.workunit.client.0.smithi042.stdout:6/679: read d8/d70/d81/dd/d1b/dbf/dde/d4f/f5b [169388,5148] 0 2024-09-16T23:49:07.864 INFO:tasks.workunit.client.0.smithi042.stdout:6/680: fdatasync d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/fd8 0 2024-09-16T23:49:07.867 INFO:tasks.workunit.client.1.smithi043.stdout:9/823: dwrite d4/d1c/d25/d2b/d4c/da3/ff2 [0,4194304] 0 2024-09-16T23:49:07.868 INFO:tasks.workunit.client.1.smithi043.stdout:9/824: readlink d4/d1c/d25/d2b/lf4 0 2024-09-16T23:49:07.869 INFO:tasks.workunit.client.0.smithi042.stdout:2/674: dread d0/d26/f9f [0,4194304] 0 2024-09-16T23:49:07.875 INFO:tasks.workunit.client.0.smithi042.stdout:2/675: chown d0/da/d88/fa4 55 1 2024-09-16T23:49:07.876 INFO:tasks.workunit.client.0.smithi042.stdout:2/676: write d0/d98/fc8 [704985,1731] 0 2024-09-16T23:49:07.878 INFO:tasks.workunit.client.0.smithi042.stdout:6/681: getdents d8/d70/d81/d90 0 2024-09-16T23:49:07.881 INFO:tasks.workunit.client.0.smithi042.stdout:6/682: symlink d8/d70/d81/d90/d9f/ldf 0 2024-09-16T23:49:07.884 INFO:tasks.workunit.client.0.smithi042.stdout:2/677: fsync d0/f92 0 2024-09-16T23:49:07.895 INFO:tasks.workunit.client.0.smithi042.stdout:6/683: dread d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/fab [0,4194304] 0 2024-09-16T23:49:07.897 INFO:tasks.workunit.client.0.smithi042.stdout:2/678: dwrite d0/da/d88/dae/fa7 [0,4194304] 0 2024-09-16T23:49:07.902 INFO:tasks.workunit.client.0.smithi042.stdout:6/684: dwrite d8/d70/d81/f23 [0,4194304] 0 2024-09-16T23:49:07.903 INFO:tasks.workunit.client.0.smithi042.stdout:6/685: stat d8/d70/d81/dd/d2c/f10 0 2024-09-16T23:49:07.906 INFO:tasks.workunit.client.0.smithi042.stdout:6/686: stat d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/fd8 0 2024-09-16T23:49:07.916 INFO:tasks.workunit.client.0.smithi042.stdout:6/687: unlink d8/d70/d81/dd/d1b/d67/l8c 0 2024-09-16T23:49:07.917 INFO:tasks.workunit.client.0.smithi042.stdout:6/688: readlink d8/d70/d81/dd/d1b/d34/d39/l49 0 2024-09-16T23:49:07.919 INFO:tasks.workunit.client.0.smithi042.stdout:6/689: creat d8/d70/d81/dd/d1b/d2d/da1/fe0 x:0 0 0 2024-09-16T23:49:07.920 INFO:tasks.workunit.client.0.smithi042.stdout:6/690: chown d8/d70/d81/dd/d1b/d2d/c65 14727829 1 2024-09-16T23:49:07.924 INFO:tasks.workunit.client.0.smithi042.stdout:6/691: write d8/d70/d81/dd/d1b/dbf/dde/f79 [160222,50411] 0 2024-09-16T23:49:07.928 INFO:tasks.workunit.client.0.smithi042.stdout:6/692: readlink d8/d70/d81/dd/d1b/d34/d39/l92 0 2024-09-16T23:49:07.929 INFO:tasks.workunit.client.0.smithi042.stdout:6/693: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d4c 47 1 2024-09-16T23:49:07.932 INFO:tasks.workunit.client.0.smithi042.stdout:6/694: stat d8/d70/d81/dd/d1b/f93 0 2024-09-16T23:49:07.933 INFO:tasks.workunit.client.0.smithi042.stdout:6/695: readlink d8/d70/d81/dd/dcd/lcf 0 2024-09-16T23:49:07.934 INFO:tasks.workunit.client.1.smithi043.stdout:3/846: mknod d4/de/c105 0 2024-09-16T23:49:07.937 INFO:tasks.workunit.client.0.smithi042.stdout:6/696: rmdir d8/d70/d81/dd/d2c/d57/dc5 39 2024-09-16T23:49:07.938 INFO:tasks.workunit.client.0.smithi042.stdout:6/697: dread - d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/fdb zero size 2024-09-16T23:49:07.939 INFO:tasks.workunit.client.1.smithi043.stdout:8/815: mkdir de/d18/d2b/df4 0 2024-09-16T23:49:07.939 INFO:tasks.workunit.client.1.smithi043.stdout:8/816: fsync de/d18/d27/d41/d5a/f1b 0 2024-09-16T23:49:07.941 INFO:tasks.workunit.client.1.smithi043.stdout:0/937: read - d0/d2/df/da7/fd8 zero size 2024-09-16T23:49:07.942 INFO:tasks.workunit.client.0.smithi042.stdout:6/698: mkdir d8/d70/d81/dd/dcd/de1 0 2024-09-16T23:49:07.944 INFO:tasks.workunit.client.0.smithi042.stdout:2/679: dwrite d0/da/d17/d3e/f91 [0,4194304] 0 2024-09-16T23:49:07.949 INFO:tasks.workunit.client.0.smithi042.stdout:2/680: truncate d0/da/d88/dae/f75 87215 0 2024-09-16T23:49:07.952 INFO:tasks.workunit.client.1.smithi043.stdout:6/806: dread de/f47 [0,4194304] 0 2024-09-16T23:49:07.957 INFO:tasks.workunit.client.0.smithi042.stdout:6/699: getdents d8/d70/d81/dd/d1b/d34/d39/db4 0 2024-09-16T23:49:07.958 INFO:tasks.workunit.client.0.smithi042.stdout:6/700: write d8/d70/d81/dd/d1b/dbf/dde/f6e [2961031,18512] 0 2024-09-16T23:49:07.965 INFO:tasks.workunit.client.1.smithi043.stdout:8/817: fsync fc 0 2024-09-16T23:49:07.965 INFO:tasks.workunit.client.0.smithi042.stdout:6/701: dwrite d8/d70/d81/dd/d1b/d2d/da1/fc1 [0,4194304] 0 2024-09-16T23:49:07.967 INFO:tasks.workunit.client.1.smithi043.stdout:5/977: write d3/db/d19/d3e/fea [1642794,61160] 0 2024-09-16T23:49:07.967 INFO:tasks.workunit.client.1.smithi043.stdout:7/843: symlink d4/d27/d41/d1b/l115 0 2024-09-16T23:49:07.968 INFO:tasks.workunit.client.1.smithi043.stdout:5/978: write d3/db/d19/d3e/d28/dcb/ff7 [1023932,22193] 0 2024-09-16T23:49:07.969 INFO:tasks.workunit.client.1.smithi043.stdout:7/844: write d4/d27/f3f [3122520,32398] 0 2024-09-16T23:49:07.971 INFO:tasks.workunit.client.1.smithi043.stdout:2/784: getdents d8/d1d/d20 0 2024-09-16T23:49:07.973 INFO:tasks.workunit.client.1.smithi043.stdout:5/979: dread - d3/db/d19/d3e/d28/dcb/f118 zero size 2024-09-16T23:49:07.973 INFO:tasks.workunit.client.1.smithi043.stdout:6/807: rename de/d2a/c95 to de/d23/d34/d9b/d5a/dfe/c111 0 2024-09-16T23:49:07.973 INFO:tasks.workunit.client.1.smithi043.stdout:9/825: dread d4/d21/f63 [0,4194304] 0 2024-09-16T23:49:07.974 INFO:tasks.workunit.client.1.smithi043.stdout:4/890: mkdir d8/d15/d20/d2b/d54/dce/d112 0 2024-09-16T23:49:07.975 INFO:tasks.workunit.client.1.smithi043.stdout:3/847: mkdir d4/d106 0 2024-09-16T23:49:07.976 INFO:tasks.workunit.client.1.smithi043.stdout:8/818: mkdir de/d70/d3d/daf/df5 0 2024-09-16T23:49:07.978 INFO:tasks.workunit.client.1.smithi043.stdout:7/845: fsync d4/d27/d41/fb6 0 2024-09-16T23:49:07.981 INFO:tasks.workunit.client.1.smithi043.stdout:0/938: rmdir d0/d2/df/d1a/d49/dd1 0 2024-09-16T23:49:07.981 INFO:tasks.workunit.client.1.smithi043.stdout:3/848: rmdir d4/de/d40 39 2024-09-16T23:49:07.983 INFO:tasks.workunit.client.1.smithi043.stdout:7/846: mkdir d4/db/d11/d21/d94/d116 0 2024-09-16T23:49:07.985 INFO:tasks.workunit.client.1.smithi043.stdout:9/826: rmdir d4/d5/dce 0 2024-09-16T23:49:07.986 INFO:tasks.workunit.client.1.smithi043.stdout:6/808: dwrite de/d1a/d28/f3e [0,4194304] 0 2024-09-16T23:49:07.989 INFO:tasks.workunit.client.1.smithi043.stdout:7/847: stat d4/db/d11/d15/d29/fbd 0 2024-09-16T23:49:07.990 INFO:tasks.workunit.client.1.smithi043.stdout:2/785: dread d8/d4b/d4f/d7c/f88 [0,4194304] 0 2024-09-16T23:49:07.991 INFO:tasks.workunit.client.1.smithi043.stdout:4/891: link d8/f1d d8/d25/d39/dd3/d100/d108/d4b/d103/f113 0 2024-09-16T23:49:07.991 INFO:tasks.workunit.client.1.smithi043.stdout:9/827: chown d4/d21/d2a/d2c/c7c 15151 1 2024-09-16T23:49:07.995 INFO:tasks.workunit.client.1.smithi043.stdout:8/819: dwrite de/d18/d27/d41/d5a/f56 [0,4194304] 0 2024-09-16T23:49:07.997 INFO:tasks.workunit.client.1.smithi043.stdout:4/892: readlink d8/d15/d20/d2b/d54/db3/d4c/lda 0 2024-09-16T23:49:07.997 INFO:tasks.workunit.client.1.smithi043.stdout:9/828: fdatasync d4/f31 0 2024-09-16T23:49:07.999 INFO:tasks.workunit.client.1.smithi043.stdout:2/786: mkdir d8/d5e/d83/d3a/d77/da9/d10a 0 2024-09-16T23:49:07.999 INFO:tasks.workunit.client.0.smithi042.stdout:3/958: sync 2024-09-16T23:49:08.003 INFO:tasks.workunit.client.1.smithi043.stdout:5/980: dwrite d3/f96 [0,4194304] 0 2024-09-16T23:49:08.003 INFO:tasks.workunit.client.1.smithi043.stdout:7/848: rename d4/db/d43/d83 to d4/db/d11/d117 0 2024-09-16T23:49:08.004 INFO:tasks.workunit.client.1.smithi043.stdout:0/939: dread d0/d2/df/d1a/d49/d80/d88/f10d [0,4194304] 0 2024-09-16T23:49:08.005 INFO:tasks.workunit.client.1.smithi043.stdout:8/820: mkdir de/d18/d1f/d2f/d39/d7b/df6 0 2024-09-16T23:49:08.007 INFO:tasks.workunit.client.1.smithi043.stdout:9/829: mknod d4/d21/d2a/c109 0 2024-09-16T23:49:08.007 INFO:tasks.workunit.client.1.smithi043.stdout:6/809: dwrite de/d1a/d80/fdd [0,4194304] 0 2024-09-16T23:49:08.009 INFO:tasks.workunit.client.1.smithi043.stdout:4/893: unlink d8/d15/d20/d2b/d54/db3/d4c/dee/l10e 0 2024-09-16T23:49:08.012 INFO:tasks.workunit.client.1.smithi043.stdout:2/787: truncate d8/d1d/d25/fd9 45732 0 2024-09-16T23:49:08.013 INFO:tasks.workunit.client.1.smithi043.stdout:4/894: stat d8/d25/d39/dd3 0 2024-09-16T23:49:08.014 INFO:tasks.workunit.client.1.smithi043.stdout:9/830: mkdir d4/d21/d33/da2/d10a 0 2024-09-16T23:49:08.015 INFO:tasks.workunit.client.1.smithi043.stdout:5/981: truncate d3/d9/d22/f4e 3502810 0 2024-09-16T23:49:08.015 INFO:tasks.workunit.client.1.smithi043.stdout:6/810: symlink de/d23/d63/d7f/l112 0 2024-09-16T23:49:08.017 INFO:tasks.workunit.client.1.smithi043.stdout:8/821: mkdir de/d18/d27/df7 0 2024-09-16T23:49:08.018 INFO:tasks.workunit.client.1.smithi043.stdout:2/788: creat d8/d4b/d4f/d56/f10b x:0 0 0 2024-09-16T23:49:08.019 INFO:tasks.workunit.client.1.smithi043.stdout:6/811: fdatasync de/d23/d34/d9b/d8b/feb 0 2024-09-16T23:49:08.020 INFO:tasks.workunit.client.1.smithi043.stdout:8/822: chown de/d18/d27/f85 19627380 1 2024-09-16T23:49:08.020 INFO:tasks.workunit.client.1.smithi043.stdout:5/982: unlink d3/c88 0 2024-09-16T23:49:08.023 INFO:tasks.workunit.client.1.smithi043.stdout:9/831: mknod d4/d21/d2a/d2c/d7b/c10b 0 2024-09-16T23:49:08.023 INFO:tasks.workunit.client.1.smithi043.stdout:0/940: dwrite d0/d2/df/d1a/d46/d39/d83/dcb/fca [0,4194304] 0 2024-09-16T23:49:08.025 INFO:tasks.workunit.client.1.smithi043.stdout:6/812: mknod de/d23/d34/d9b/d8b/de9/c113 0 2024-09-16T23:49:08.030 INFO:tasks.workunit.client.1.smithi043.stdout:8/823: creat de/d18/d1f/d2f/d39/d7b/df6/ff8 x:0 0 0 2024-09-16T23:49:08.031 INFO:tasks.workunit.client.1.smithi043.stdout:0/941: rmdir d0/d2/df/da7/db8 39 2024-09-16T23:49:08.033 INFO:tasks.workunit.client.1.smithi043.stdout:9/832: creat d4/d21/d35/dd4/f10c x:0 0 0 2024-09-16T23:49:08.034 INFO:tasks.workunit.client.1.smithi043.stdout:6/813: mknod de/d23/d34/d9b/d8b/dbd/c114 0 2024-09-16T23:49:08.036 INFO:tasks.workunit.client.1.smithi043.stdout:8/824: rmdir de/d18/d2b/da1 39 2024-09-16T23:49:08.041 INFO:tasks.workunit.client.1.smithi043.stdout:0/942: readlink d0/d2/df/d1a/d3b/d13e/l112 0 2024-09-16T23:49:08.041 INFO:tasks.workunit.client.1.smithi043.stdout:4/895: dread d8/d25/d39/d58/fa5 [0,4194304] 0 2024-09-16T23:49:08.042 INFO:tasks.workunit.client.1.smithi043.stdout:7/849: dwrite d4/db/d28/f97 [0,4194304] 0 2024-09-16T23:49:08.044 INFO:tasks.workunit.client.1.smithi043.stdout:5/983: dwrite d3/db/d15/f75 [0,4194304] 0 2024-09-16T23:49:08.045 INFO:tasks.workunit.client.1.smithi043.stdout:9/833: dread - d4/d28/d5f/fbd zero size 2024-09-16T23:49:08.046 INFO:tasks.workunit.client.0.smithi042.stdout:3/959: chown d2/d7/ff2 9364 1 2024-09-16T23:49:08.046 INFO:tasks.workunit.client.1.smithi043.stdout:8/825: symlink de/d18/d27/d8e/lf9 0 2024-09-16T23:49:08.046 INFO:tasks.workunit.client.1.smithi043.stdout:6/814: mkdir de/d23/d34/d9b/d5a/dfe/d115 0 2024-09-16T23:49:08.047 INFO:tasks.workunit.client.0.smithi042.stdout:6/702: creat d8/d38/fe2 x:0 0 0 2024-09-16T23:49:08.048 INFO:tasks.workunit.client.1.smithi043.stdout:2/789: dwrite d8/d4b/d5f/d8e/f8c [0,4194304] 0 2024-09-16T23:49:08.051 INFO:tasks.workunit.client.0.smithi042.stdout:2/681: dwrite d0/f1a [0,4194304] 0 2024-09-16T23:49:08.051 INFO:tasks.workunit.client.0.smithi042.stdout:6/703: stat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/fdb 0 2024-09-16T23:49:08.051 INFO:tasks.workunit.client.1.smithi043.stdout:4/896: rmdir d8/d25/d39/dd3/d100/d108/d4b 39 2024-09-16T23:49:08.055 INFO:tasks.workunit.client.0.smithi042.stdout:3/960: dwrite d2/d1b/fce [0,4194304] 0 2024-09-16T23:49:08.056 INFO:tasks.workunit.client.1.smithi043.stdout:6/815: dread de/d23/d34/d9b/d8b/fbb [0,4194304] 0 2024-09-16T23:49:08.057 INFO:tasks.workunit.client.1.smithi043.stdout:0/943: mknod d0/d2/df/d1a/d49/d80/d88/c13f 0 2024-09-16T23:49:08.060 INFO:tasks.workunit.client.0.smithi042.stdout:3/961: unlink d2/d7/d61/d8e/f131 0 2024-09-16T23:49:08.061 INFO:tasks.workunit.client.1.smithi043.stdout:6/816: write de/d23/d34/d9b/d8b/dbd/f10c [379445,79817] 0 2024-09-16T23:49:08.062 INFO:tasks.workunit.client.1.smithi043.stdout:7/850: readlink d4/db/d11/d117/d85/d8b/l91 0 2024-09-16T23:49:08.063 INFO:tasks.workunit.client.1.smithi043.stdout:5/984: mknod d3/d9/d22/d4a/d64/d133/c14c 0 2024-09-16T23:49:08.064 INFO:tasks.workunit.client.0.smithi042.stdout:2/682: dwrite d0/da/d1b/d48/f58 [0,4194304] 0 2024-09-16T23:49:08.066 INFO:tasks.workunit.client.1.smithi043.stdout:9/834: creat d4/d1c/d6b/d67/dbf/f10d x:0 0 0 2024-09-16T23:49:08.067 INFO:tasks.workunit.client.1.smithi043.stdout:5/985: readlink d3/db/d15/d4d/d6b/ld5 0 2024-09-16T23:49:08.067 INFO:tasks.workunit.client.1.smithi043.stdout:7/851: truncate d4/db/d11/fcf 1198416 0 2024-09-16T23:49:08.068 INFO:tasks.workunit.client.1.smithi043.stdout:8/826: write de/d18/d27/d41/f50 [4924777,868] 0 2024-09-16T23:49:08.069 INFO:tasks.workunit.client.1.smithi043.stdout:4/897: dread d8/f5d [0,4194304] 0 2024-09-16T23:49:08.069 INFO:tasks.workunit.client.1.smithi043.stdout:2/790: symlink d8/d5e/l10c 0 2024-09-16T23:49:08.071 INFO:tasks.workunit.client.1.smithi043.stdout:0/944: creat d0/d2/df/d1a/d49/db2/df4/d9c/dfa/f140 x:0 0 0 2024-09-16T23:49:08.071 INFO:tasks.workunit.client.0.smithi042.stdout:3/962: dwrite d2/d7/d61/d8e/d9d/d141/d34/d62/d6a/fb1 [0,4194304] 0 2024-09-16T23:49:08.073 INFO:tasks.workunit.client.0.smithi042.stdout:2/683: rename d0/da/d1b/d48/d81 to d0/d30/dcb/de5 0 2024-09-16T23:49:08.074 INFO:tasks.workunit.client.0.smithi042.stdout:3/963: stat d2/d7/d61/d8e/d9d/d141/d34/d62/d11a 0 2024-09-16T23:49:08.078 INFO:tasks.workunit.client.0.smithi042.stdout:2/684: fdatasync d0/d30/dcb/de5/f80 0 2024-09-16T23:49:08.078 INFO:tasks.workunit.client.0.smithi042.stdout:3/964: symlink d2/d7/d61/d8e/d9d/dc4/dcc/l146 0 2024-09-16T23:49:08.079 INFO:tasks.workunit.client.0.smithi042.stdout:6/704: write d8/d38/f74 [507946,45397] 0 2024-09-16T23:49:08.080 INFO:tasks.workunit.client.0.smithi042.stdout:6/705: dread - d8/d70/d81/dd/d1b/d50/fb8 zero size 2024-09-16T23:49:08.083 INFO:tasks.workunit.client.1.smithi043.stdout:0/945: dwrite d0/d2/df/f114 [0,4194304] 0 2024-09-16T23:49:08.083 INFO:tasks.workunit.client.1.smithi043.stdout:9/835: dread - d4/d21/fbc zero size 2024-09-16T23:49:08.084 INFO:tasks.workunit.client.1.smithi043.stdout:5/986: dwrite d3/db/d19/d76/f144 [0,4194304] 0 2024-09-16T23:49:08.091 INFO:tasks.workunit.client.1.smithi043.stdout:9/836: dread d4/d1c/f6f [0,4194304] 0 2024-09-16T23:49:08.092 INFO:tasks.workunit.client.0.smithi042.stdout:6/706: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 7654132 0 2024-09-16T23:49:08.092 INFO:tasks.workunit.client.1.smithi043.stdout:6/817: write de/d23/d34/d66/d9d/faf [567582,40666] 0 2024-09-16T23:49:08.093 INFO:tasks.workunit.client.1.smithi043.stdout:2/791: creat d8/d35/dc0/f10d x:0 0 0 2024-09-16T23:49:08.094 INFO:tasks.workunit.client.0.smithi042.stdout:2/685: link d0/da/c2e d0/d8d/da0/ce6 0 2024-09-16T23:49:08.095 INFO:tasks.workunit.client.1.smithi043.stdout:4/898: unlink d8/d15/d20/dfa/f95 0 2024-09-16T23:49:08.096 INFO:tasks.workunit.client.0.smithi042.stdout:6/707: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/ce3 0 2024-09-16T23:49:08.100 INFO:tasks.workunit.client.0.smithi042.stdout:2/686: unlink d0/d5d/l73 0 2024-09-16T23:49:08.102 INFO:tasks.workunit.client.1.smithi043.stdout:8/827: rmdir de/d18/d27/d41/d5a/d37/d6a 39 2024-09-16T23:49:08.102 INFO:tasks.workunit.client.1.smithi043.stdout:4/899: dread d8/d25/f6a [0,4194304] 0 2024-09-16T23:49:08.103 INFO:tasks.workunit.client.1.smithi043.stdout:2/792: mknod d8/d1d/d20/d26/c10e 0 2024-09-16T23:49:08.104 INFO:tasks.workunit.client.1.smithi043.stdout:9/837: dwrite d4/d28/d5f/d86/f96 [4194304,4194304] 0 2024-09-16T23:49:08.114 INFO:tasks.workunit.client.0.smithi042.stdout:6/708: link d8/d70/d81/d90/d9f/cb2 d8/d70/d81/dd/d1b/d34/d39/db4/ce4 0 2024-09-16T23:49:08.117 INFO:tasks.workunit.client.0.smithi042.stdout:2/687: dread d0/f2b [0,4194304] 0 2024-09-16T23:49:08.120 INFO:tasks.workunit.client.0.smithi042.stdout:6/709: write d8/d70/d81/dd/d1b/d34/d39/d5e/f83 [1944045,91002] 0 2024-09-16T23:49:08.120 INFO:tasks.workunit.client.0.smithi042.stdout:6/710: fdatasync d8/d70/d81/dd/d1b/d2d/da1/fc1 0 2024-09-16T23:49:08.120 INFO:tasks.workunit.client.0.smithi042.stdout:2/688: dread d0/da/d1b/d3b/d44/fb3 [0,4194304] 0 2024-09-16T23:49:08.123 INFO:tasks.workunit.client.1.smithi043.stdout:7/852: rmdir d4/db/d11/d21/d94/d116 0 2024-09-16T23:49:08.124 INFO:tasks.workunit.client.0.smithi042.stdout:2/689: dread d0/da/d88/dae/fa7 [0,4194304] 0 2024-09-16T23:49:08.124 INFO:tasks.workunit.client.1.smithi043.stdout:6/818: write de/d23/d34/d9b/d5a/fc6 [748197,39933] 0 2024-09-16T23:49:08.124 INFO:tasks.workunit.client.1.smithi043.stdout:5/987: dwrite d3/d9/d22/d4a/fc0 [0,4194304] 0 2024-09-16T23:49:08.127 INFO:tasks.workunit.client.0.smithi042.stdout:6/711: chown d8/d70/d81/dd/d2c/l6a 13 1 2024-09-16T23:49:08.127 INFO:tasks.workunit.client.1.smithi043.stdout:7/853: readlink d4/d27/d41/d1b/l89 0 2024-09-16T23:49:08.128 INFO:tasks.workunit.client.1.smithi043.stdout:7/854: write d4/d27/dbc/f10f [92242,23946] 0 2024-09-16T23:49:08.129 INFO:tasks.workunit.client.1.smithi043.stdout:0/946: write d0/d2/df/d1a/d46/d39/d6e/f65 [604087,91313] 0 2024-09-16T23:49:08.131 INFO:tasks.workunit.client.0.smithi042.stdout:2/690: fdatasync d0/da/d1b/d3b/f56 0 2024-09-16T23:49:08.135 INFO:tasks.workunit.client.0.smithi042.stdout:6/712: dwrite d8/d70/d81/dd/d2c/f10 [0,4194304] 0 2024-09-16T23:49:08.136 INFO:tasks.workunit.client.1.smithi043.stdout:8/828: truncate de/d18/d1f/d25/d65/d7f/f8c 270922 0 2024-09-16T23:49:08.142 INFO:tasks.workunit.client.0.smithi042.stdout:2/691: mknod d0/da/d88/dae/ce7 0 2024-09-16T23:49:08.143 INFO:tasks.workunit.client.0.smithi042.stdout:6/713: dwrite d8/d70/d81/dd/d1b/fc8 [0,4194304] 0 2024-09-16T23:49:08.149 INFO:tasks.workunit.client.0.smithi042.stdout:2/692: rename d0/d30/d43/fd4 to d0/da/d88/fe8 0 2024-09-16T23:49:08.153 INFO:tasks.workunit.client.1.smithi043.stdout:9/838: mkdir d4/d21/d33/d10e 0 2024-09-16T23:49:08.153 INFO:tasks.workunit.client.1.smithi043.stdout:7/855: mkdir d4/db/d11/d117/d85/d8b/d118 0 2024-09-16T23:49:08.155 INFO:tasks.workunit.client.0.smithi042.stdout:6/714: dwrite d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d99/fd9 [0,4194304] 0 2024-09-16T23:49:08.159 INFO:tasks.workunit.client.1.smithi043.stdout:4/900: truncate d8/d15/d20/dfa/f111 783150 0 2024-09-16T23:49:08.161 INFO:tasks.workunit.client.1.smithi043.stdout:5/988: write d3/db/d19/f54 [2307593,41446] 0 2024-09-16T23:49:08.167 INFO:tasks.workunit.client.1.smithi043.stdout:9/839: rmdir d4/d21/d35 39 2024-09-16T23:49:08.168 INFO:tasks.workunit.client.1.smithi043.stdout:0/947: mknod d0/d2/df/d1a/d46/d39/d127/c141 0 2024-09-16T23:49:08.168 INFO:tasks.workunit.client.1.smithi043.stdout:6/819: dwrite de/d23/d34/d66/fac [0,4194304] 0 2024-09-16T23:49:08.169 INFO:tasks.workunit.client.0.smithi042.stdout:2/693: creat d0/d98/fe9 x:0 0 0 2024-09-16T23:49:08.172 INFO:tasks.workunit.client.0.smithi042.stdout:6/715: write d8/d70/d81/dd/d2c/d57/dc5/fce [1629014,108620] 0 2024-09-16T23:49:08.172 INFO:tasks.workunit.client.1.smithi043.stdout:8/829: symlink de/d18/d27/d41/lfa 0 2024-09-16T23:49:08.174 INFO:tasks.workunit.client.1.smithi043.stdout:8/830: chown de/d18/d27/d8e/f80 4341 1 2024-09-16T23:49:08.175 INFO:tasks.workunit.client.1.smithi043.stdout:9/840: dwrite d4/d21/d33/d95/f9b [0,4194304] 0 2024-09-16T23:49:08.176 INFO:tasks.workunit.client.1.smithi043.stdout:5/989: dread - d3/d9/d22/d4a/d64/f110 zero size 2024-09-16T23:49:08.177 INFO:tasks.workunit.client.0.smithi042.stdout:2/694: dread d0/d30/d43/d77/fbc [0,4194304] 0 2024-09-16T23:49:08.177 INFO:tasks.workunit.client.1.smithi043.stdout:8/831: write de/d70/fc9 [820111,122081] 0 2024-09-16T23:49:08.178 INFO:tasks.workunit.client.0.smithi042.stdout:9/537: sync 2024-09-16T23:49:08.178 INFO:tasks.workunit.client.1.smithi043.stdout:2/793: getdents d8/d4b/d4f/d7c 0 2024-09-16T23:49:08.179 INFO:tasks.workunit.client.1.smithi043.stdout:9/841: rename d4/d1c/d25/d2b/d4c/dd2/de5 to d4/d1c/d25/d2b/d4c/dd2/de5/d10f 22 2024-09-16T23:49:08.180 INFO:tasks.workunit.client.1.smithi043.stdout:9/842: fdatasync d4/d5/fe2 0 2024-09-16T23:49:08.180 INFO:tasks.workunit.client.1.smithi043.stdout:2/794: write d8/d5e/fe0 [133060,69080] 0 2024-09-16T23:49:08.184 INFO:tasks.workunit.client.1.smithi043.stdout:2/795: truncate d8/d1d/d20/d23/d38/da2/fe4 628396 0 2024-09-16T23:49:08.185 INFO:tasks.workunit.client.1.smithi043.stdout:7/856: creat d4/db/d28/d10c/f119 x:0 0 0 2024-09-16T23:49:08.189 INFO:tasks.workunit.client.1.smithi043.stdout:7/857: dread d4/db/d11/fc6 [0,4194304] 0 2024-09-16T23:49:08.190 INFO:tasks.workunit.client.1.smithi043.stdout:7/858: stat d4/db/d11/d21/d31/fc4 0 2024-09-16T23:49:08.191 INFO:tasks.workunit.client.1.smithi043.stdout:7/859: chown d4/db/d11/d15/dda/f101 50 1 2024-09-16T23:49:08.193 INFO:tasks.workunit.client.0.smithi042.stdout:6/716: link d8/d70/d81/dd/d1b/dbf/ld5 d8/le5 0 2024-09-16T23:49:08.195 INFO:tasks.workunit.client.1.smithi043.stdout:9/843: stat d4/d1c/d25/c44 0 2024-09-16T23:49:08.197 INFO:tasks.workunit.client.0.smithi042.stdout:2/695: dwrite d0/da/d17/f42 [0,4194304] 0 2024-09-16T23:49:08.197 INFO:tasks.workunit.client.1.smithi043.stdout:5/990: dwrite d3/f50 [0,4194304] 0 2024-09-16T23:49:08.198 INFO:tasks.workunit.client.1.smithi043.stdout:2/796: mkdir d8/d1d/d10f 0 2024-09-16T23:49:08.199 INFO:tasks.workunit.client.1.smithi043.stdout:5/991: fsync d3/db/d19/d3e/d28/dcb/f106 0 2024-09-16T23:49:08.199 INFO:tasks.workunit.client.0.smithi042.stdout:2/696: write d0/d98/fc8 [1571032,120952] 0 2024-09-16T23:49:08.201 INFO:tasks.workunit.client.1.smithi043.stdout:0/948: mkdir d0/d2/df/da7/db8/d142 0 2024-09-16T23:49:08.201 INFO:tasks.workunit.client.1.smithi043.stdout:5/992: readlink d3/d79/dcc/l134 0 2024-09-16T23:49:08.201 INFO:tasks.workunit.client.0.smithi042.stdout:9/538: dwrite d4/d12/d2d/d37/d28/f87 [0,4194304] 0 2024-09-16T23:49:08.206 INFO:tasks.workunit.client.1.smithi043.stdout:6/820: fsync de/d1a/d28/f9a 0 2024-09-16T23:49:08.207 INFO:tasks.workunit.client.1.smithi043.stdout:6/821: chown de/d23/d34/d9b/d8b/de9 13170 1 2024-09-16T23:49:08.207 INFO:tasks.workunit.client.0.smithi042.stdout:6/717: unlink d8/d70/d81/dd/d1b/fc8 0 2024-09-16T23:49:08.213 INFO:tasks.workunit.client.1.smithi043.stdout:8/832: truncate de/d18/d1f/f32 1032153 0 2024-09-16T23:49:08.217 INFO:tasks.workunit.client.1.smithi043.stdout:9/844: rename d4/d1c/d6b/d67 to d4/d1c/d25/d2b/d110 0 2024-09-16T23:49:08.218 INFO:tasks.workunit.client.0.smithi042.stdout:6/718: fdatasync d8/d70/d81/dd/f19 0 2024-09-16T23:49:08.221 INFO:tasks.workunit.client.1.smithi043.stdout:0/949: rmdir d0/d2/df/da7 39 2024-09-16T23:49:08.222 INFO:tasks.workunit.client.1.smithi043.stdout:0/950: fsync d0/f3c 0 2024-09-16T23:49:08.224 INFO:tasks.workunit.client.1.smithi043.stdout:7/860: write d4/db/d43/f7f [746214,103514] 0 2024-09-16T23:49:08.226 INFO:tasks.workunit.client.0.smithi042.stdout:6/719: dread d8/f18 [0,4194304] 0 2024-09-16T23:49:08.230 INFO:tasks.workunit.client.0.smithi042.stdout:9/539: truncate d4/d12/d16/f85 1786586 0 2024-09-16T23:49:08.231 INFO:tasks.workunit.client.0.smithi042.stdout:6/720: symlink d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/le6 0 2024-09-16T23:49:08.232 INFO:tasks.workunit.client.0.smithi042.stdout:6/721: chown d8/d70/d81/dd/d2c/d57 1716 1 2024-09-16T23:49:08.240 INFO:tasks.workunit.client.0.smithi042.stdout:9/540: unlink d4/d12/d16/f85 0 2024-09-16T23:49:08.240 INFO:tasks.workunit.client.1.smithi043.stdout:4/901: dwrite d8/d25/d39/dd3/d100/d108/d4b/fbd [0,4194304] 0 2024-09-16T23:49:08.240 INFO:tasks.workunit.client.0.smithi042.stdout:9/541: fdatasync d4/d12/d60/d97/fa0 0 2024-09-16T23:49:08.244 INFO:tasks.workunit.client.1.smithi043.stdout:0/951: rename d0/d2/df/d1a/d49/db2/l117 to d0/d2/df/d1a/d46/d92/d123/l143 0 2024-09-16T23:49:08.246 INFO:tasks.workunit.client.0.smithi042.stdout:6/722: write d8/d70/d81/dd/d1b/d34/f64 [952851,92257] 0 2024-09-16T23:49:08.251 INFO:tasks.workunit.client.1.smithi043.stdout:4/902: dread d8/d15/d20/d2b/f28 [0,4194304] 0 2024-09-16T23:49:08.252 INFO:tasks.workunit.client.1.smithi043.stdout:6/822: mknod de/d23/d34/d9b/c116 0 2024-09-16T23:49:08.253 INFO:tasks.workunit.client.0.smithi042.stdout:6/723: symlink d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d99/le7 0 2024-09-16T23:49:08.254 INFO:tasks.workunit.client.1.smithi043.stdout:8/833: creat de/d18/d1f/d2f/d39/d7b/dd8/ffb x:0 0 0 2024-09-16T23:49:08.255 INFO:tasks.workunit.client.0.smithi042.stdout:9/542: rmdir d4/d12/d2d/d37/d41/d92/da7/d9c 0 2024-09-16T23:49:08.256 INFO:tasks.workunit.client.0.smithi042.stdout:9/543: chown d4/d12/d2d/d37/d59 56700 1 2024-09-16T23:49:08.257 INFO:tasks.workunit.client.1.smithi043.stdout:9/845: truncate d4/d21/d35/d4b/fb7 648370 0 2024-09-16T23:49:08.258 INFO:tasks.workunit.client.1.smithi043.stdout:2/797: unlink d8/d11/c1b 0 2024-09-16T23:49:08.260 INFO:tasks.workunit.client.0.smithi042.stdout:6/724: rename d8/d70/d81/dd/d1b/d34/d39/db4/ce4 to d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/ce8 0 2024-09-16T23:49:08.261 INFO:tasks.workunit.client.1.smithi043.stdout:8/834: dwrite de/d18/d27/d41/d5a/f26 [4194304,4194304] 0 2024-09-16T23:49:08.263 INFO:tasks.workunit.client.1.smithi043.stdout:9/846: dread d4/d28/d5f/d91/fe0 [0,4194304] 0 2024-09-16T23:49:08.263 INFO:tasks.workunit.client.1.smithi043.stdout:5/993: link d3/d9/d22/d4a/dac/fef d3/d9/d6f/d7c/f14d 0 2024-09-16T23:49:08.263 INFO:tasks.workunit.client.0.smithi042.stdout:9/544: dread d4/d12/d2d/d37/d28/f31 [0,4194304] 0 2024-09-16T23:49:08.265 INFO:tasks.workunit.client.1.smithi043.stdout:0/952: creat d0/d2/df/d1a/d46/d39/d127/f144 x:0 0 0 2024-09-16T23:49:08.265 INFO:tasks.workunit.client.1.smithi043.stdout:3/849: sync 2024-09-16T23:49:08.268 INFO:tasks.workunit.client.1.smithi043.stdout:4/903: dread - d8/d25/dcd/fd9 zero size 2024-09-16T23:49:08.268 INFO:tasks.workunit.client.0.smithi042.stdout:9/545: unlink d4/d12/d2d/f32 0 2024-09-16T23:49:08.270 INFO:tasks.workunit.client.1.smithi043.stdout:6/823: creat de/d23/ddc/f117 x:0 0 0 2024-09-16T23:49:08.270 INFO:tasks.workunit.client.0.smithi042.stdout:6/725: creat d8/d70/d81/dd/d1b/d67/d97/dc9/fe9 x:0 0 0 2024-09-16T23:49:08.272 INFO:tasks.workunit.client.1.smithi043.stdout:3/850: dwrite d4/de/d1f/d34/fad [0,4194304] 0 2024-09-16T23:49:08.276 INFO:tasks.workunit.client.0.smithi042.stdout:9/546: fsync d4/d12/d2d/d37/d28/d5e/f7a 0 2024-09-16T23:49:08.279 INFO:tasks.workunit.client.0.smithi042.stdout:6/726: rename d8/d70/d81/dd/d1b/d34/d39/d5e/f83 to d8/d70/d81/dd/d2c/fea 0 2024-09-16T23:49:08.282 INFO:tasks.workunit.client.1.smithi043.stdout:9/847: creat d4/d21/d2a/d2c/d7b/f111 x:0 0 0 2024-09-16T23:49:08.285 INFO:tasks.workunit.client.1.smithi043.stdout:7/861: link d4/db/d11/fcf d4/f11a 0 2024-09-16T23:49:08.286 INFO:tasks.workunit.client.0.smithi042.stdout:6/727: truncate d8/d70/d81/dd/d2c/f13 757795 0 2024-09-16T23:49:08.287 INFO:tasks.workunit.client.1.smithi043.stdout:0/953: dread - d0/d2/df/d1a/d46/d39/d83/feb zero size 2024-09-16T23:49:08.288 INFO:tasks.workunit.client.0.smithi042.stdout:9/547: dread d4/d10/f1f [0,4194304] 0 2024-09-16T23:49:08.291 INFO:tasks.workunit.client.0.smithi042.stdout:6/728: truncate d8/d38/fe2 398236 0 2024-09-16T23:49:08.293 INFO:tasks.workunit.client.0.smithi042.stdout:9/548: rmdir d4/d12/d60/d97 39 2024-09-16T23:49:08.295 INFO:tasks.workunit.client.1.smithi043.stdout:6/824: creat de/d23/d34/d9b/d5a/dfe/f118 x:0 0 0 2024-09-16T23:49:08.296 INFO:tasks.workunit.client.1.smithi043.stdout:2/798: write d8/d4b/d4f/d7c/f88 [3919636,10407] 0 2024-09-16T23:49:08.298 INFO:tasks.workunit.client.0.smithi042.stdout:6/729: creat d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/feb x:0 0 0 2024-09-16T23:49:08.299 INFO:tasks.workunit.client.1.smithi043.stdout:2/799: chown d8/d5e/d83/ddb 2965 1 2024-09-16T23:49:08.300 INFO:tasks.workunit.client.1.smithi043.stdout:7/862: dread d4/db/d11/d15/fd9 [0,4194304] 0 2024-09-16T23:49:08.301 INFO:tasks.workunit.client.1.smithi043.stdout:4/904: dread d8/d25/f5b [0,4194304] 0 2024-09-16T23:49:08.301 INFO:tasks.workunit.client.0.smithi042.stdout:9/549: unlink d4/d10/d24/f67 0 2024-09-16T23:49:08.302 INFO:tasks.workunit.client.0.smithi042.stdout:9/550: read - d4/d12/d2d/d37/d28/d5e/f7a zero size 2024-09-16T23:49:08.306 INFO:tasks.workunit.client.1.smithi043.stdout:8/835: dwrite de/d18/d2b/f77 [0,4194304] 0 2024-09-16T23:49:08.308 INFO:tasks.workunit.client.0.smithi042.stdout:9/551: rmdir d4/d12/d2d/d37/d3a/d50 39 2024-09-16T23:49:08.311 INFO:tasks.workunit.client.0.smithi042.stdout:6/730: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 3035603 0 2024-09-16T23:49:08.315 INFO:tasks.workunit.client.0.smithi042.stdout:9/552: dread d4/d12/d2d/d37/d3a/f5c [0,4194304] 0 2024-09-16T23:49:08.316 INFO:tasks.workunit.client.1.smithi043.stdout:6/825: mknod de/d23/d49/c119 0 2024-09-16T23:49:08.317 INFO:tasks.workunit.client.0.smithi042.stdout:9/553: write d4/d10/f57 [3144080,85835] 0 2024-09-16T23:49:08.318 INFO:tasks.workunit.client.0.smithi042.stdout:6/731: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/f7c [0,4194304] 0 2024-09-16T23:49:08.331 INFO:tasks.workunit.client.0.smithi042.stdout:6/732: dwrite d8/d70/d81/dd/d1b/dbf/dde/d4f/f5b [0,4194304] 0 2024-09-16T23:49:08.333 INFO:tasks.workunit.client.0.smithi042.stdout:9/554: rename d4/d12/d44 to d4/d12/d2d/d37/d3a/dad 0 2024-09-16T23:49:08.335 INFO:tasks.workunit.client.0.smithi042.stdout:9/555: fsync d4/d12/d2d/d37/d28/f4e 0 2024-09-16T23:49:08.335 INFO:tasks.workunit.client.1.smithi043.stdout:0/954: creat d0/d2/df/d1a/de8/d110/f145 x:0 0 0 2024-09-16T23:49:08.338 INFO:tasks.workunit.client.1.smithi043.stdout:3/851: creat d4/d30/f107 x:0 0 0 2024-09-16T23:49:08.339 INFO:tasks.workunit.client.0.smithi042.stdout:6/733: rename d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/fab to d8/d70/d81/dd/dcd/fec 0 2024-09-16T23:49:08.340 INFO:tasks.workunit.client.0.smithi042.stdout:6/734: dread - d8/d70/d81/dd/d1b/d50/fb8 zero size 2024-09-16T23:49:08.342 INFO:tasks.workunit.client.1.smithi043.stdout:4/905: mknod d8/d15/c114 0 2024-09-16T23:49:08.345 INFO:tasks.workunit.client.1.smithi043.stdout:9/848: link d4/d28/d5f/fbd d4/d5/d16/f112 0 2024-09-16T23:49:08.346 INFO:tasks.workunit.client.1.smithi043.stdout:9/849: write d4/d1c/d25/d2b/fc6 [110685,57364] 0 2024-09-16T23:49:08.347 INFO:tasks.workunit.client.0.smithi042.stdout:6/735: write d8/d70/d81/dd/dcd/fac [879928,108730] 0 2024-09-16T23:49:08.350 INFO:tasks.workunit.client.0.smithi042.stdout:9/556: mknod d4/d12/d2d/d37/d3a/d50/cae 0 2024-09-16T23:49:08.351 INFO:tasks.workunit.client.0.smithi042.stdout:6/736: read d8/f37 [971697,28661] 0 2024-09-16T23:49:08.353 INFO:tasks.workunit.client.1.smithi043.stdout:2/800: write d8/fc [4242605,42709] 0 2024-09-16T23:49:08.355 INFO:tasks.workunit.client.1.smithi043.stdout:6/826: dwrite de/d2a/f33 [4194304,4194304] 0 2024-09-16T23:49:08.355 INFO:tasks.workunit.client.1.smithi043.stdout:3/852: creat d4/de/d1f/d2c/db7/f108 x:0 0 0 2024-09-16T23:49:08.355 INFO:tasks.workunit.client.0.smithi042.stdout:5/838: sync 2024-09-16T23:49:08.355 INFO:tasks.workunit.client.0.smithi042.stdout:1/746: sync 2024-09-16T23:49:08.357 INFO:tasks.workunit.client.0.smithi042.stdout:6/737: unlink d8/d38/f74 0 2024-09-16T23:49:08.358 INFO:tasks.workunit.client.1.smithi043.stdout:8/836: write de/d18/d27/d41/d5a/f24 [355086,13038] 0 2024-09-16T23:49:08.359 INFO:tasks.workunit.client.0.smithi042.stdout:6/738: write d8/d70/d81/dd/d2c/d57/fca [870750,125422] 0 2024-09-16T23:49:08.359 INFO:tasks.workunit.client.0.smithi042.stdout:5/839: mkdir d9/db/d1b/d31/d46/d58/d120 0 2024-09-16T23:49:08.361 INFO:tasks.workunit.client.1.smithi043.stdout:8/837: read de/f17 [3278677,62514] 0 2024-09-16T23:49:08.361 INFO:tasks.workunit.client.0.smithi042.stdout:5/840: mkdir d9/d2c/d121 0 2024-09-16T23:49:08.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/841: fdatasync d9/db/de/d92/daf/fe0 0 2024-09-16T23:49:08.365 INFO:tasks.workunit.client.0.smithi042.stdout:1/747: symlink d0/d9/d90/d6b/d88/d32/d2f/de3/l10d 0 2024-09-16T23:49:08.366 INFO:tasks.workunit.client.1.smithi043.stdout:9/850: creat d4/d21/d35/f113 x:0 0 0 2024-09-16T23:49:08.367 INFO:tasks.workunit.client.0.smithi042.stdout:9/557: dwrite d4/d12/d16/d68/f9f [0,4194304] 0 2024-09-16T23:49:08.370 INFO:tasks.workunit.client.1.smithi043.stdout:4/906: write d8/d25/d39/fac [771807,84305] 0 2024-09-16T23:49:08.372 INFO:tasks.workunit.client.0.smithi042.stdout:6/739: dread d8/d70/d81/dd/d1b/d50/fb9 [0,4194304] 0 2024-09-16T23:49:08.373 INFO:tasks.workunit.client.1.smithi043.stdout:0/955: rename d0/l57 to d0/d2/df/d1a/d3b/dee/df0/l146 0 2024-09-16T23:49:08.373 INFO:tasks.workunit.client.1.smithi043.stdout:9/851: dwrite d4/d28/d5f/fd3 [0,4194304] 0 2024-09-16T23:49:08.376 INFO:tasks.workunit.client.0.smithi042.stdout:1/748: dwrite d0/d9/d90/d66/f105 [0,4194304] 0 2024-09-16T23:49:08.376 INFO:tasks.workunit.client.1.smithi043.stdout:0/956: chown d0/de/f78 80 1 2024-09-16T23:49:08.376 INFO:tasks.workunit.client.0.smithi042.stdout:5/842: unlink d9/db/d19/d21/d85/dec/d112/d61/d43/c53 0 2024-09-16T23:49:08.378 INFO:tasks.workunit.client.1.smithi043.stdout:9/852: write d4/d1c/ddc/fef [383651,28749] 0 2024-09-16T23:49:08.378 INFO:tasks.workunit.client.1.smithi043.stdout:8/838: truncate de/d18/d27/d8e/f80 4593161 0 2024-09-16T23:49:08.379 INFO:tasks.workunit.client.0.smithi042.stdout:5/843: readlink d9/db/de/d10/ld8 0 2024-09-16T23:49:08.379 INFO:tasks.workunit.client.0.smithi042.stdout:9/558: dread d4/d12/d2d/d37/d3a/d50/f99 [0,4194304] 0 2024-09-16T23:49:08.380 INFO:tasks.workunit.client.0.smithi042.stdout:6/740: dread d8/d70/d81/dd/d2c/f31 [0,4194304] 0 2024-09-16T23:49:08.381 INFO:tasks.workunit.client.0.smithi042.stdout:7/779: sync 2024-09-16T23:49:08.381 INFO:tasks.workunit.client.0.smithi042.stdout:3/965: sync 2024-09-16T23:49:08.381 INFO:tasks.workunit.client.0.smithi042.stdout:8/577: sync 2024-09-16T23:49:08.381 INFO:tasks.workunit.client.0.smithi042.stdout:2/697: sync 2024-09-16T23:49:08.381 INFO:tasks.workunit.client.0.smithi042.stdout:0/626: sync 2024-09-16T23:49:08.382 INFO:tasks.workunit.client.0.smithi042.stdout:1/749: symlink d0/d17/l10e 0 2024-09-16T23:49:08.384 INFO:tasks.workunit.client.1.smithi043.stdout:6/827: mkdir de/d1a/d11a 0 2024-09-16T23:49:08.386 INFO:tasks.workunit.client.0.smithi042.stdout:6/741: stat d8/d70/d81/dd/d2c/d57 0 2024-09-16T23:49:08.388 INFO:tasks.workunit.client.1.smithi043.stdout:9/853: dwrite d4/d21/d33/d95/f9b [0,4194304] 0 2024-09-16T23:49:08.388 INFO:tasks.workunit.client.0.smithi042.stdout:1/750: creat d0/d9/d26/daa/dea/f10f x:0 0 0 2024-09-16T23:49:08.388 INFO:tasks.workunit.client.0.smithi042.stdout:0/627: read d0/f4 [1710340,97535] 0 2024-09-16T23:49:08.391 INFO:tasks.workunit.client.0.smithi042.stdout:3/966: fdatasync d2/d95/fd0 0 2024-09-16T23:49:08.391 INFO:tasks.workunit.client.1.smithi043.stdout:4/907: fsync d8/fbe 0 2024-09-16T23:49:08.394 INFO:tasks.workunit.client.1.smithi043.stdout:6/828: creat de/d1a/d39/f11b x:0 0 0 2024-09-16T23:49:08.395 INFO:tasks.workunit.client.0.smithi042.stdout:5/844: rename d9/db/de/d92/d10d to d9/db/d19/d21/d122 0 2024-09-16T23:49:08.395 INFO:tasks.workunit.client.0.smithi042.stdout:8/578: fsync da/d19/d1d/d5a/f79 0 2024-09-16T23:49:08.398 INFO:tasks.workunit.client.1.smithi043.stdout:9/854: creat d4/d28/d49/f114 x:0 0 0 2024-09-16T23:49:08.398 INFO:tasks.workunit.client.0.smithi042.stdout:3/967: mknod d2/d95/c147 0 2024-09-16T23:49:08.402 INFO:tasks.workunit.client.0.smithi042.stdout:1/751: creat d0/d72/f110 x:0 0 0 2024-09-16T23:49:08.402 INFO:tasks.workunit.client.0.smithi042.stdout:0/628: rename d0/df/d23/d33/d51/d5b/fa8 to d0/df/d23/d33/d56/dc5/fc6 0 2024-09-16T23:49:08.402 INFO:tasks.workunit.client.1.smithi043.stdout:2/801: dread d8/d1d/d25/f2b [0,4194304] 0 2024-09-16T23:49:08.403 INFO:tasks.workunit.client.0.smithi042.stdout:1/752: chown d0/d9/d26/f70 3373 1 2024-09-16T23:49:08.403 INFO:tasks.workunit.client.0.smithi042.stdout:0/629: chown d0/df/d13/d2d/c84 271 1 2024-09-16T23:49:08.404 INFO:tasks.workunit.client.1.smithi043.stdout:6/829: dwrite de/dbe/f107 [0,4194304] 0 2024-09-16T23:49:08.404 INFO:tasks.workunit.client.0.smithi042.stdout:8/579: fdatasync da/d19/d1d/d30/d49/f7b 0 2024-09-16T23:49:08.404 INFO:tasks.workunit.client.0.smithi042.stdout:5/845: rename d9/d4a/f9b to d9/d4a/f123 0 2024-09-16T23:49:08.405 INFO:tasks.workunit.client.1.smithi043.stdout:2/802: fsync d8/d4b/d5f/f61 0 2024-09-16T23:49:08.406 INFO:tasks.workunit.client.0.smithi042.stdout:9/559: truncate d4/fb 3002786 0 2024-09-16T23:49:08.406 INFO:tasks.workunit.client.0.smithi042.stdout:9/560: fdatasync d4/d12/d16/f6d 0 2024-09-16T23:49:08.407 INFO:tasks.workunit.client.1.smithi043.stdout:6/830: fsync de/d23/d34/d9b/ffd 0 2024-09-16T23:49:08.407 INFO:tasks.workunit.client.0.smithi042.stdout:9/561: chown d4/d12/d2d/d94/da3 42944119 1 2024-09-16T23:49:08.408 INFO:tasks.workunit.client.1.smithi043.stdout:4/908: mknod d8/d25/d39/dd3/c115 0 2024-09-16T23:49:08.409 INFO:tasks.workunit.client.0.smithi042.stdout:6/742: symlink d8/d70/d81/dd/d1b/d2d/da1/db3/led 0 2024-09-16T23:49:08.410 INFO:tasks.workunit.client.0.smithi042.stdout:0/630: symlink d0/df/lc7 0 2024-09-16T23:49:08.412 INFO:tasks.workunit.client.0.smithi042.stdout:3/968: creat d2/d7/d61/d8e/d9d/d141/f148 x:0 0 0 2024-09-16T23:49:08.413 INFO:tasks.workunit.client.1.smithi043.stdout:9/855: rename d4/d1c/d25/d2b/d4c/da3/fc2 to d4/d1c/d25/d2b/da1/df9/d45/d85/f115 0 2024-09-16T23:49:08.413 INFO:tasks.workunit.client.0.smithi042.stdout:3/969: write d2/d1b/dc8/ff6 [2342020,56277] 0 2024-09-16T23:49:08.414 INFO:tasks.workunit.client.0.smithi042.stdout:1/753: rmdir d0/d17/d5e 39 2024-09-16T23:49:08.415 INFO:tasks.workunit.client.0.smithi042.stdout:0/631: symlink d0/df/lc8 0 2024-09-16T23:49:08.415 INFO:tasks.workunit.client.1.smithi043.stdout:4/909: dwrite d8/d15/f10c [0,4194304] 0 2024-09-16T23:49:08.417 INFO:tasks.workunit.client.0.smithi042.stdout:0/632: truncate d0/df/d13/d49/d70/fb3 1639902 0 2024-09-16T23:49:08.421 INFO:tasks.workunit.client.0.smithi042.stdout:7/780: write d3/d1f/d49/d76/f25 [1024497,38365] 0 2024-09-16T23:49:08.427 INFO:tasks.workunit.client.0.smithi042.stdout:9/562: mknod d4/d12/d2d/d37/d3a/d50/caf 0 2024-09-16T23:49:08.427 INFO:tasks.workunit.client.0.smithi042.stdout:2/698: creat d0/d30/d43/fea x:0 0 0 2024-09-16T23:49:08.428 INFO:tasks.workunit.client.1.smithi043.stdout:6/831: rename de/f24 to de/d23/d34/d9b/d5a/dfe/f11c 0 2024-09-16T23:49:08.432 INFO:tasks.workunit.client.0.smithi042.stdout:1/754: rename d0/d9/f16 to d0/d9/d90/d66/de2/f111 0 2024-09-16T23:49:08.433 INFO:tasks.workunit.client.0.smithi042.stdout:7/781: creat d3/d1f/d49/d76/d19/d63/f113 x:0 0 0 2024-09-16T23:49:08.434 INFO:tasks.workunit.client.0.smithi042.stdout:0/633: dwrite d0/df/d13/d2d/d35/fbb [0,4194304] 0 2024-09-16T23:49:08.436 INFO:tasks.workunit.client.1.smithi043.stdout:4/910: rmdir d8/d25 39 2024-09-16T23:49:08.438 INFO:tasks.workunit.client.0.smithi042.stdout:8/580: dwrite da/d2f/f5e [0,4194304] 0 2024-09-16T23:49:08.439 INFO:tasks.workunit.client.1.smithi043.stdout:2/803: fdatasync d8/d35/fb9 0 2024-09-16T23:49:08.441 INFO:tasks.workunit.client.1.smithi043.stdout:2/804: truncate d8/d1f/ff0 4416737 0 2024-09-16T23:49:08.445 INFO:tasks.workunit.client.0.smithi042.stdout:3/970: dwrite d2/d7/fab [0,4194304] 0 2024-09-16T23:49:08.445 INFO:tasks.workunit.client.0.smithi042.stdout:0/634: write d0/f4b [2489404,21650] 0 2024-09-16T23:49:08.446 INFO:tasks.workunit.client.1.smithi043.stdout:6/832: rename de/d23/d34/d9b/d5a/c106 to de/dbe/dca/c11d 0 2024-09-16T23:49:08.448 INFO:tasks.workunit.client.0.smithi042.stdout:2/699: dwrite d0/da/d17/f42 [0,4194304] 0 2024-09-16T23:49:08.449 INFO:tasks.workunit.client.1.smithi043.stdout:9/856: unlink d4/d84/leb 0 2024-09-16T23:49:08.450 INFO:tasks.workunit.client.0.smithi042.stdout:0/635: chown d0/df/d13/c1d 0 1 2024-09-16T23:49:08.453 INFO:tasks.workunit.client.0.smithi042.stdout:1/755: dread - d0/d9/f9b zero size 2024-09-16T23:49:08.454 INFO:tasks.workunit.client.0.smithi042.stdout:9/563: dwrite d4/d10/d24/f3b [0,4194304] 0 2024-09-16T23:49:08.456 INFO:tasks.workunit.client.0.smithi042.stdout:9/564: chown d4/f5 95332870 1 2024-09-16T23:49:08.457 INFO:tasks.workunit.client.0.smithi042.stdout:7/782: write d3/d1f/d49/d76/d101/d5b/f7a [743123,34700] 0 2024-09-16T23:49:08.458 INFO:tasks.workunit.client.1.smithi043.stdout:2/805: unlink d8/d5e/fc2 0 2024-09-16T23:49:08.460 INFO:tasks.workunit.client.0.smithi042.stdout:0/636: dread d0/d15/f3c [0,4194304] 0 2024-09-16T23:49:08.461 INFO:tasks.workunit.client.1.smithi043.stdout:6/833: dread - de/d1a/d28/f77 zero size 2024-09-16T23:49:08.463 INFO:tasks.workunit.client.0.smithi042.stdout:0/637: write d0/df/d23/d33/d56/da9/fbc [248535,54055] 0 2024-09-16T23:49:08.464 INFO:tasks.workunit.client.0.smithi042.stdout:7/783: mkdir d3/d3f/d114 0 2024-09-16T23:49:08.465 INFO:tasks.workunit.client.0.smithi042.stdout:1/756: dread - d0/d17/fb8 zero size 2024-09-16T23:49:08.466 INFO:tasks.workunit.client.0.smithi042.stdout:0/638: creat d0/df/d23/d33/d51/d79/fc9 x:0 0 0 2024-09-16T23:49:08.466 INFO:tasks.workunit.client.0.smithi042.stdout:7/784: chown d3/d98/da0/c42 9871 1 2024-09-16T23:49:08.467 INFO:tasks.workunit.client.0.smithi042.stdout:1/757: readlink d0/d9/d90/d6b/l102 0 2024-09-16T23:49:08.469 INFO:tasks.workunit.client.0.smithi042.stdout:0/639: fsync d0/df/d23/d33/d56/da9/f65 0 2024-09-16T23:49:08.470 INFO:tasks.workunit.client.0.smithi042.stdout:8/581: write da/d19/f23 [1361623,85110] 0 2024-09-16T23:49:08.471 INFO:tasks.workunit.client.1.smithi043.stdout:6/834: mknod de/d23/dc3/c11e 0 2024-09-16T23:49:08.471 INFO:tasks.workunit.client.1.smithi043.stdout:9/857: mknod d4/d21/d2a/d2c/c116 0 2024-09-16T23:49:08.473 INFO:tasks.workunit.client.1.smithi043.stdout:4/911: link d8/d15/d9b/d8b/fa8 d8/d25/dcd/dd8/f116 0 2024-09-16T23:49:08.474 INFO:tasks.workunit.client.0.smithi042.stdout:0/640: mknod d0/df/d13/d2d/cca 0 2024-09-16T23:49:08.477 INFO:tasks.workunit.client.0.smithi042.stdout:1/758: symlink d0/d9/d90/d43/l112 0 2024-09-16T23:49:08.478 INFO:tasks.workunit.client.1.smithi043.stdout:6/835: symlink de/d23/d49/l11f 0 2024-09-16T23:49:08.478 INFO:tasks.workunit.client.1.smithi043.stdout:6/836: chown de/d23/dc3/fd4 3 1 2024-09-16T23:49:08.479 INFO:tasks.workunit.client.0.smithi042.stdout:3/971: dread d2/d1b/f94 [0,4194304] 0 2024-09-16T23:49:08.482 INFO:tasks.workunit.client.0.smithi042.stdout:0/641: dread d0/df/d13/d49/d70/fb3 [0,4194304] 0 2024-09-16T23:49:08.482 INFO:tasks.workunit.client.0.smithi042.stdout:2/700: rename d0/da/c2e to d0/da/d1b/d3b/ceb 0 2024-09-16T23:49:08.483 INFO:tasks.workunit.client.0.smithi042.stdout:0/642: stat d0/df/cbf 0 2024-09-16T23:49:08.483 INFO:tasks.workunit.client.0.smithi042.stdout:3/972: mkdir d2/d7/d61/d8e/d149 0 2024-09-16T23:49:08.484 INFO:tasks.workunit.client.1.smithi043.stdout:2/806: link d8/d35/ld7 d8/d5e/d83/d9e/dab/l110 0 2024-09-16T23:49:08.485 INFO:tasks.workunit.client.1.smithi043.stdout:9/858: mkdir d4/d1c/d25/d2b/d4c/da3/dca/dfe/d117 0 2024-09-16T23:49:08.486 INFO:tasks.workunit.client.0.smithi042.stdout:7/785: dread d3/d98/da0/f1b [4194304,4194304] 0 2024-09-16T23:49:08.486 INFO:tasks.workunit.client.0.smithi042.stdout:3/973: chown d2/d7/d61/d8e/d9d/d141/d30/c9f 83471 1 2024-09-16T23:49:08.487 INFO:tasks.workunit.client.1.smithi043.stdout:6/837: fdatasync de/d23/d34/d66/f6e 0 2024-09-16T23:49:08.488 INFO:tasks.workunit.client.0.smithi042.stdout:0/643: truncate d0/df/d23/d33/d56/dc5/fc6 120858 0 2024-09-16T23:49:08.488 INFO:tasks.workunit.client.0.smithi042.stdout:3/974: fsync d2/d1b/dc8/f144 0 2024-09-16T23:49:08.490 INFO:tasks.workunit.client.1.smithi043.stdout:4/912: mkdir d8/d15/d20/d2b/d23/d117 0 2024-09-16T23:49:08.490 INFO:tasks.workunit.client.0.smithi042.stdout:0/644: mkdir d0/df/d13/dcb 0 2024-09-16T23:49:08.492 INFO:tasks.workunit.client.1.smithi043.stdout:2/807: chown d8/d11/l12 30128 1 2024-09-16T23:49:08.493 INFO:tasks.workunit.client.0.smithi042.stdout:0/645: mkdir d0/df/d13/dcb/dcc 0 2024-09-16T23:49:08.494 INFO:tasks.workunit.client.0.smithi042.stdout:8/582: rename da/d19/d1d/f78 to da/d19/d1d/fb3 0 2024-09-16T23:49:08.495 INFO:tasks.workunit.client.0.smithi042.stdout:7/786: dwrite d3/f30 [0,4194304] 0 2024-09-16T23:49:08.496 INFO:tasks.workunit.client.1.smithi043.stdout:9/859: creat d4/d1c/d25/d2b/d4c/dd2/f118 x:0 0 0 2024-09-16T23:49:08.498 INFO:tasks.workunit.client.0.smithi042.stdout:0/646: chown d0/df/d13/d49/d70/da1/lb7 2787589 1 2024-09-16T23:49:08.500 INFO:tasks.workunit.client.0.smithi042.stdout:3/975: dwrite d2/d7/d61/d8e/d9d/dc4/dcc/f113 [0,4194304] 0 2024-09-16T23:49:08.501 INFO:tasks.workunit.client.0.smithi042.stdout:0/647: mkdir d0/df/d23/d33/d56/dc5/dcd 0 2024-09-16T23:49:08.504 INFO:tasks.workunit.client.1.smithi043.stdout:9/860: dwrite d4/d1c/d25/d2b/d4c/da3/ff2 [4194304,4194304] 0 2024-09-16T23:49:08.506 INFO:tasks.workunit.client.0.smithi042.stdout:0/648: mknod d0/df/d23/d3e/d8f/cce 0 2024-09-16T23:49:08.506 INFO:tasks.workunit.client.0.smithi042.stdout:8/583: mknod da/d19/d1d/d30/d3f/d6d/cb4 0 2024-09-16T23:49:08.506 INFO:tasks.workunit.client.1.smithi043.stdout:6/838: creat de/d1a/d80/f120 x:0 0 0 2024-09-16T23:49:08.507 INFO:tasks.workunit.client.0.smithi042.stdout:3/976: unlink d2/d7/d61/d8e/d9d/d141/l111 0 2024-09-16T23:49:08.508 INFO:tasks.workunit.client.1.smithi043.stdout:6/839: write de/d23/d34/d9b/d8b/fea [1040499,24331] 0 2024-09-16T23:49:08.509 INFO:tasks.workunit.client.0.smithi042.stdout:1/759: write d0/d9/f9b [921031,90614] 0 2024-09-16T23:49:08.511 INFO:tasks.workunit.client.0.smithi042.stdout:1/760: chown d0/d17/d79 665816758 1 2024-09-16T23:49:08.514 INFO:tasks.workunit.client.1.smithi043.stdout:9/861: rmdir d4/d1c/d25/d2b/da1/df9/dd8 39 2024-09-16T23:49:08.514 INFO:tasks.workunit.client.0.smithi042.stdout:1/761: write d0/d9a/fbb [286016,4146] 0 2024-09-16T23:49:08.517 INFO:tasks.workunit.client.0.smithi042.stdout:1/762: chown d0/d9/d90/l10b 12727134 1 2024-09-16T23:49:08.517 INFO:tasks.workunit.client.0.smithi042.stdout:0/649: rename d0/df/d23/d33/d51/f52 to d0/d15/fcf 0 2024-09-16T23:49:08.519 INFO:tasks.workunit.client.0.smithi042.stdout:2/701: truncate d0/da/d17/f37 3641090 0 2024-09-16T23:49:08.521 INFO:tasks.workunit.client.0.smithi042.stdout:8/584: creat da/d19/d2c/d56/d97/dad/fb5 x:0 0 0 2024-09-16T23:49:08.522 INFO:tasks.workunit.client.0.smithi042.stdout:0/650: creat d0/df/d13/d49/fd0 x:0 0 0 2024-09-16T23:49:08.522 INFO:tasks.workunit.client.1.smithi043.stdout:2/808: mkdir d8/d11/d111 0 2024-09-16T23:49:08.522 INFO:tasks.workunit.client.0.smithi042.stdout:7/787: dread d3/d1f/d49/d76/d19/d63/f75 [0,4194304] 0 2024-09-16T23:49:08.524 INFO:tasks.workunit.client.0.smithi042.stdout:1/763: rename d0/d9/l3d to d0/d9/d90/d6b/d88/d32/d2f/de3/l113 0 2024-09-16T23:49:08.525 INFO:tasks.workunit.client.0.smithi042.stdout:7/788: fsync d3/d1f/f61 0 2024-09-16T23:49:08.526 INFO:tasks.workunit.client.0.smithi042.stdout:3/977: truncate d2/d7/d61/d8e/d9d/d141/f25 2166266 0 2024-09-16T23:49:08.527 INFO:tasks.workunit.client.1.smithi043.stdout:6/840: fdatasync de/d23/d34/d9b/f88 0 2024-09-16T23:49:08.527 INFO:tasks.workunit.client.0.smithi042.stdout:0/651: mkdir d0/df/d23/d33/d56/dc5/dcd/dd1 0 2024-09-16T23:49:08.528 INFO:tasks.workunit.client.0.smithi042.stdout:3/978: chown d2/d7/d61/d8e/d9d/d141/d34/d62/d6a/fe0 6699 1 2024-09-16T23:49:08.529 INFO:tasks.workunit.client.0.smithi042.stdout:7/789: creat d3/d1f/d49/d76/d101/d5b/db8/f115 x:0 0 0 2024-09-16T23:49:08.533 INFO:tasks.workunit.client.0.smithi042.stdout:1/764: dread d0/d9/fb9 [0,4194304] 0 2024-09-16T23:49:08.535 INFO:tasks.workunit.client.0.smithi042.stdout:3/979: creat d2/d7/d61/d8e/d7f/dfa/d114/f14a x:0 0 0 2024-09-16T23:49:08.537 INFO:tasks.workunit.client.0.smithi042.stdout:7/790: creat d3/d98/da0/d24/d28/d9b/f116 x:0 0 0 2024-09-16T23:49:08.537 INFO:tasks.workunit.client.1.smithi043.stdout:4/913: dwrite d8/d25/d39/d41/d57/fa1 [0,4194304] 0 2024-09-16T23:49:08.539 INFO:tasks.workunit.client.0.smithi042.stdout:1/765: read d0/d9/d90/d66/f105 [2262516,51793] 0 2024-09-16T23:49:08.543 INFO:tasks.workunit.client.0.smithi042.stdout:0/652: dwrite d0/d15/f7c [0,4194304] 0 2024-09-16T23:49:08.543 INFO:tasks.workunit.client.0.smithi042.stdout:3/980: fsync d2/d1b/d89/f45 0 2024-09-16T23:49:08.546 INFO:tasks.workunit.client.0.smithi042.stdout:0/653: chown d0/df/f4a 36 1 2024-09-16T23:49:08.547 INFO:tasks.workunit.client.0.smithi042.stdout:2/702: dread d0/da/d1b/f29 [4194304,4194304] 0 2024-09-16T23:49:08.548 INFO:tasks.workunit.client.1.smithi043.stdout:4/914: mknod d8/d15/d20/d2b/d54/dce/d112/c118 0 2024-09-16T23:49:08.549 INFO:tasks.workunit.client.1.smithi043.stdout:9/862: link d4/d21/d2a/d2c/d7b/c7d d4/d1c/c119 0 2024-09-16T23:49:08.550 INFO:tasks.workunit.client.1.smithi043.stdout:9/863: chown d4/d1c/d25/d2b/d4c/f93 3444 1 2024-09-16T23:49:08.551 INFO:tasks.workunit.client.0.smithi042.stdout:0/654: rmdir d0/df/d13/d49/d70 39 2024-09-16T23:49:08.551 INFO:tasks.workunit.client.0.smithi042.stdout:3/981: write d2/d7/feb [2474960,2633] 0 2024-09-16T23:49:08.552 INFO:tasks.workunit.client.1.smithi043.stdout:9/864: truncate d4/d84/f101 1007038 0 2024-09-16T23:49:08.552 INFO:tasks.workunit.client.0.smithi042.stdout:1/766: mknod d0/d9/d26/daa/c114 0 2024-09-16T23:49:08.553 INFO:tasks.workunit.client.1.smithi043.stdout:4/915: mknod d8/d15/d20/d2b/d54/db3/d4c/df3/c119 0 2024-09-16T23:49:08.554 INFO:tasks.workunit.client.0.smithi042.stdout:1/767: write d0/d9a/fb7 [359292,11259] 0 2024-09-16T23:49:08.555 INFO:tasks.workunit.client.0.smithi042.stdout:2/703: mkdir d0/da/d1b/d3b/d44/dec 0 2024-09-16T23:49:08.556 INFO:tasks.workunit.client.0.smithi042.stdout:8/585: dwrite da/d19/d1d/d30/d3f/d6d/f7e [0,4194304] 0 2024-09-16T23:49:08.557 INFO:tasks.workunit.client.1.smithi043.stdout:9/865: mknod d4/d21/c11a 0 2024-09-16T23:49:08.557 INFO:tasks.workunit.client.0.smithi042.stdout:3/982: mknod d2/d7/dfb/c14b 0 2024-09-16T23:49:08.559 INFO:tasks.workunit.client.0.smithi042.stdout:7/791: link d3/d1f/d49/d78/dbe/dc8/lfa d3/d98/da0/d41/db0/l117 0 2024-09-16T23:49:08.559 INFO:tasks.workunit.client.0.smithi042.stdout:3/983: chown d2/d1b/d89/lcb 227405 1 2024-09-16T23:49:08.560 INFO:tasks.workunit.client.0.smithi042.stdout:1/768: rmdir d0/d9/dba 39 2024-09-16T23:49:08.563 INFO:tasks.workunit.client.0.smithi042.stdout:2/704: creat d0/da/d17/d3e/fed x:0 0 0 2024-09-16T23:49:08.564 INFO:tasks.workunit.client.1.smithi043.stdout:9/866: truncate d4/d1c/d25/d2b/da1/df9/d45/f6a 2824615 0 2024-09-16T23:49:08.565 INFO:tasks.workunit.client.0.smithi042.stdout:7/792: fdatasync d3/d1f/d49/d76/fd5 0 2024-09-16T23:49:08.565 INFO:tasks.workunit.client.1.smithi043.stdout:9/867: chown d4/d21/c11a 1876453 1 2024-09-16T23:49:08.567 INFO:tasks.workunit.client.1.smithi043.stdout:4/916: creat d8/d15/d20/d2b/d54/db3/f11a x:0 0 0 2024-09-16T23:49:08.567 INFO:tasks.workunit.client.1.smithi043.stdout:2/809: write d8/d1d/d20/d23/d38/f69 [1333843,71164] 0 2024-09-16T23:49:08.570 INFO:tasks.workunit.client.0.smithi042.stdout:8/586: mkdir da/d19/db6 0 2024-09-16T23:49:08.572 INFO:tasks.workunit.client.0.smithi042.stdout:0/655: dwrite d0/df/f18 [0,4194304] 0 2024-09-16T23:49:08.572 INFO:tasks.workunit.client.0.smithi042.stdout:1/769: read d0/d9/d26/f70 [2999340,38722] 0 2024-09-16T23:49:08.573 INFO:tasks.workunit.client.0.smithi042.stdout:3/984: dwrite d2/d7/d61/d8e/d9d/dc4/dcc/fc2 [0,4194304] 0 2024-09-16T23:49:08.575 INFO:tasks.workunit.client.1.smithi043.stdout:6/841: dwrite de/d23/fb8 [0,4194304] 0 2024-09-16T23:49:08.579 INFO:tasks.workunit.client.1.smithi043.stdout:9/868: dread d4/d5/d16/f54 [0,4194304] 0 2024-09-16T23:49:08.581 INFO:tasks.workunit.client.0.smithi042.stdout:2/705: dread d0/da/d1b/d3b/d44/d54/f5e [0,4194304] 0 2024-09-16T23:49:08.582 INFO:tasks.workunit.client.0.smithi042.stdout:3/985: write d2/d1b/fb2 [1712269,60510] 0 2024-09-16T23:49:08.583 INFO:tasks.workunit.client.1.smithi043.stdout:6/842: mkdir de/d23/d34/d66/d121 0 2024-09-16T23:49:08.585 INFO:tasks.workunit.client.1.smithi043.stdout:2/810: creat d8/d5e/d83/d3a/f112 x:0 0 0 2024-09-16T23:49:08.587 INFO:tasks.workunit.client.0.smithi042.stdout:3/986: chown d2/d7/f118 455 1 2024-09-16T23:49:08.588 INFO:tasks.workunit.client.0.smithi042.stdout:1/770: symlink d0/d9/dba/l115 0 2024-09-16T23:49:08.592 INFO:tasks.workunit.client.0.smithi042.stdout:0/656: dwrite d0/df/d23/d33/d51/d79/fc2 [0,4194304] 0 2024-09-16T23:49:08.593 INFO:tasks.workunit.client.0.smithi042.stdout:8/587: dwrite da/d19/d1d/d30/d3f/d6d/f7e [0,4194304] 0 2024-09-16T23:49:08.593 INFO:tasks.workunit.client.0.smithi042.stdout:0/657: fsync d0/d15/f85 0 2024-09-16T23:49:08.595 INFO:tasks.workunit.client.0.smithi042.stdout:2/706: write d0/da/d1b/d48/db7/fcd [3603756,117790] 0 2024-09-16T23:49:08.595 INFO:tasks.workunit.client.0.smithi042.stdout:3/987: truncate d2/d7/d61/d8e/d2c/f125 532867 0 2024-09-16T23:49:08.596 INFO:tasks.workunit.client.0.smithi042.stdout:1/771: creat d0/d9a/f116 x:0 0 0 2024-09-16T23:49:08.596 INFO:tasks.workunit.client.0.smithi042.stdout:7/793: dread d3/d98/f8f [0,4194304] 0 2024-09-16T23:49:08.598 INFO:tasks.workunit.client.1.smithi043.stdout:6/843: mknod de/d23/d34/c122 0 2024-09-16T23:49:08.599 INFO:tasks.workunit.client.1.smithi043.stdout:4/917: write d8/d25/d39/d58/fab [1255537,70645] 0 2024-09-16T23:49:08.599 INFO:tasks.workunit.client.0.smithi042.stdout:2/707: truncate d0/da/fcc 1869468 0 2024-09-16T23:49:08.600 INFO:tasks.workunit.client.1.smithi043.stdout:6/844: readlink de/d1a/d28/l91 0 2024-09-16T23:49:08.602 INFO:tasks.workunit.client.0.smithi042.stdout:0/658: mknod d0/df/d13/d2d/cd2 0 2024-09-16T23:49:08.602 INFO:tasks.workunit.client.0.smithi042.stdout:1/772: mknod d0/d9/d90/d43/c117 0 2024-09-16T23:49:08.603 INFO:tasks.workunit.client.0.smithi042.stdout:3/988: symlink d2/d95/l14c 0 2024-09-16T23:49:08.603 INFO:tasks.workunit.client.0.smithi042.stdout:2/708: dread - d0/d30/dcb/de5/fc0 zero size 2024-09-16T23:49:08.604 INFO:tasks.workunit.client.1.smithi043.stdout:6/845: creat de/d23/d34/d66/d9d/f123 x:0 0 0 2024-09-16T23:49:08.604 INFO:tasks.workunit.client.0.smithi042.stdout:7/794: getdents d3/d1f/d49/d76/d101/d64/d92/dfb 0 2024-09-16T23:49:08.605 INFO:tasks.workunit.client.0.smithi042.stdout:0/659: chown d0/df/d13/d2d/cca 1209456 1 2024-09-16T23:49:08.607 INFO:tasks.workunit.client.0.smithi042.stdout:8/588: truncate da/d19/d1d/f3e 912162 0 2024-09-16T23:49:08.609 INFO:tasks.workunit.client.0.smithi042.stdout:2/709: creat d0/da/d88/dae/fee x:0 0 0 2024-09-16T23:49:08.610 INFO:tasks.workunit.client.1.smithi043.stdout:6/846: mkdir de/d23/d49/dc9/d124 0 2024-09-16T23:49:08.611 INFO:tasks.workunit.client.0.smithi042.stdout:0/660: mknod d0/df/d23/d33/d56/da9/cd3 0 2024-09-16T23:49:08.612 INFO:tasks.workunit.client.1.smithi043.stdout:9/869: dwrite d4/d28/d5f/d91/fa6 [0,4194304] 0 2024-09-16T23:49:08.612 INFO:tasks.workunit.client.0.smithi042.stdout:1/773: truncate d0/d9/d90/d6b/d88/f2b 452945 0 2024-09-16T23:49:08.613 INFO:tasks.workunit.client.0.smithi042.stdout:8/589: rmdir da/d19/d1d/d30/d49/d58 39 2024-09-16T23:49:08.615 INFO:tasks.workunit.client.0.smithi042.stdout:2/710: creat d0/da/d17/d3e/fef x:0 0 0 2024-09-16T23:49:08.616 INFO:tasks.workunit.client.0.smithi042.stdout:3/989: dwrite d2/d7/dfb/fe3 [0,4194304] 0 2024-09-16T23:49:08.616 INFO:tasks.workunit.client.1.smithi043.stdout:9/870: unlink d4/d21/d35/d4b/lc8 0 2024-09-16T23:49:08.617 INFO:tasks.workunit.client.0.smithi042.stdout:0/661: truncate d0/d48/f9c 384996 0 2024-09-16T23:49:08.621 INFO:tasks.workunit.client.0.smithi042.stdout:1/774: dread - d0/d9/dba/fc5 zero size 2024-09-16T23:49:08.622 INFO:tasks.workunit.client.1.smithi043.stdout:9/871: creat d4/d84/f11b x:0 0 0 2024-09-16T23:49:08.623 INFO:tasks.workunit.client.1.smithi043.stdout:9/872: read d4/d28/d5f/d91/fad [118678,65314] 0 2024-09-16T23:49:08.625 INFO:tasks.workunit.client.0.smithi042.stdout:7/795: dwrite d3/d1f/d49/d78/dbe/fc7 [0,4194304] 0 2024-09-16T23:49:08.627 INFO:tasks.workunit.client.1.smithi043.stdout:4/918: truncate d8/d25/d39/dd3/d100/d108/d4b/fbd 162848 0 2024-09-16T23:49:08.627 INFO:tasks.workunit.client.0.smithi042.stdout:1/775: read d0/d9/d90/d66/de2/f111 [7784048,82242] 0 2024-09-16T23:49:08.628 INFO:tasks.workunit.client.1.smithi043.stdout:4/919: stat d8/d15/d20/cb5 0 2024-09-16T23:49:08.628 INFO:tasks.workunit.client.0.smithi042.stdout:3/990: mkdir d2/d7/d61/d8e/d9d/d141/d34/d50/df5/d14d 0 2024-09-16T23:49:08.629 INFO:tasks.workunit.client.0.smithi042.stdout:1/776: write d0/d17/dc7/fe7 [2336819,125100] 0 2024-09-16T23:49:08.630 INFO:tasks.workunit.client.1.smithi043.stdout:6/847: getdents de/d2a 0 2024-09-16T23:49:08.631 INFO:tasks.workunit.client.1.smithi043.stdout:6/848: readlink de/d1a/d39/le6 0 2024-09-16T23:49:08.632 INFO:tasks.workunit.client.0.smithi042.stdout:0/662: creat d0/df/d23/d33/d51/fd4 x:0 0 0 2024-09-16T23:49:08.633 INFO:tasks.workunit.client.1.smithi043.stdout:9/873: mknod d4/d1c/d25/d2b/d4c/da3/dca/dfe/c11c 0 2024-09-16T23:49:08.633 INFO:tasks.workunit.client.0.smithi042.stdout:3/991: creat d2/d9/d102/f14e x:0 0 0 2024-09-16T23:49:08.636 INFO:tasks.workunit.client.0.smithi042.stdout:3/992: mknod d2/d7/d61/d8e/d2c/c14f 0 2024-09-16T23:49:08.638 INFO:tasks.workunit.client.0.smithi042.stdout:1/777: mkdir d0/d17/d79/d118 0 2024-09-16T23:49:08.639 INFO:tasks.workunit.client.1.smithi043.stdout:6/849: dread de/dbe/f107 [0,4194304] 0 2024-09-16T23:49:08.639 INFO:tasks.workunit.client.0.smithi042.stdout:1/778: chown d0/d9/d26/l67 134631261 1 2024-09-16T23:49:08.640 INFO:tasks.workunit.client.0.smithi042.stdout:0/663: creat d0/fd5 x:0 0 0 2024-09-16T23:49:08.641 INFO:tasks.workunit.client.0.smithi042.stdout:1/779: creat d0/d9/d90/d66/de2/f119 x:0 0 0 2024-09-16T23:49:08.645 INFO:tasks.workunit.client.1.smithi043.stdout:6/850: rename c8 to de/dbf/c125 0 2024-09-16T23:49:08.646 INFO:tasks.workunit.client.0.smithi042.stdout:0/664: mkdir d0/df/d13/dd6 0 2024-09-16T23:49:08.646 INFO:tasks.workunit.client.0.smithi042.stdout:8/590: write da/d19/d1d/f3e [676441,113240] 0 2024-09-16T23:49:08.647 INFO:tasks.workunit.client.0.smithi042.stdout:1/780: write d0/d72/f65 [1126773,126947] 0 2024-09-16T23:49:08.650 INFO:tasks.workunit.client.1.smithi043.stdout:6/851: rename c5 to de/d23/d34/d66/d121/c126 0 2024-09-16T23:49:08.651 INFO:tasks.workunit.client.1.smithi043.stdout:6/852: chown de/d23/fb8 1799523068 1 2024-09-16T23:49:08.651 INFO:tasks.workunit.client.0.smithi042.stdout:0/665: creat d0/df/d23/d33/fd7 x:0 0 0 2024-09-16T23:49:08.653 INFO:tasks.workunit.client.0.smithi042.stdout:7/796: dwrite d3/d98/da0/d41/f45 [0,4194304] 0 2024-09-16T23:49:08.654 INFO:tasks.workunit.client.0.smithi042.stdout:1/781: unlink d0/d9/d90/d6b/d88/d32/ce8 0 2024-09-16T23:49:08.656 INFO:tasks.workunit.client.1.smithi043.stdout:6/853: mknod de/d23/d49/c127 0 2024-09-16T23:49:08.657 INFO:tasks.workunit.client.1.smithi043.stdout:4/920: dwrite d8/d15/ff0 [0,4194304] 0 2024-09-16T23:49:08.657 INFO:tasks.workunit.client.1.smithi043.stdout:6/854: read - de/d1a/d28/f77 zero size 2024-09-16T23:49:08.658 INFO:tasks.workunit.client.0.smithi042.stdout:0/666: getdents d0/df/d23/d3e 0 2024-09-16T23:49:08.658 INFO:tasks.workunit.client.0.smithi042.stdout:8/591: getdents da/d19/d1d/d30/d49/db2 0 2024-09-16T23:49:08.659 INFO:tasks.workunit.client.0.smithi042.stdout:0/667: fsync d0/df/d23/d33/d51/fd4 0 2024-09-16T23:49:08.659 INFO:tasks.workunit.client.0.smithi042.stdout:7/797: symlink d3/d1f/d49/d76/d101/d64/d92/dfb/l118 0 2024-09-16T23:49:08.661 INFO:tasks.workunit.client.0.smithi042.stdout:1/782: creat d0/d9/d26/dc6/f11a x:0 0 0 2024-09-16T23:49:08.663 INFO:tasks.workunit.client.0.smithi042.stdout:7/798: write d3/d1f/d49/d78/dbe/fe4 [3961509,56730] 0 2024-09-16T23:49:08.664 INFO:tasks.workunit.client.1.smithi043.stdout:9/874: truncate d4/d1c/d25/d2b/d110/fea 2711715 0 2024-09-16T23:49:08.664 INFO:tasks.workunit.client.0.smithi042.stdout:0/668: creat d0/df/d13/dd6/fd8 x:0 0 0 2024-09-16T23:49:08.666 INFO:tasks.workunit.client.0.smithi042.stdout:7/799: dread - d3/d98/da0/d41/f103 zero size 2024-09-16T23:49:08.666 INFO:tasks.workunit.client.1.smithi043.stdout:9/875: truncate d4/d1c/d25/d2b/fa5 4150737 0 2024-09-16T23:49:08.669 INFO:tasks.workunit.client.0.smithi042.stdout:1/783: creat d0/d9/d90/d6b/d88/d32/d2f/f11b x:0 0 0 2024-09-16T23:49:08.669 INFO:tasks.workunit.client.0.smithi042.stdout:0/669: link d0/df/d13/c1d d0/df/d23/d33/d56/dc5/dcd/dd1/cd9 0 2024-09-16T23:49:08.670 INFO:tasks.workunit.client.1.smithi043.stdout:6/855: mkdir de/d23/d49/dc9/d124/d128 0 2024-09-16T23:49:08.671 INFO:tasks.workunit.client.0.smithi042.stdout:0/670: read d0/df/d13/d2d/d35/fbb [3538697,74926] 0 2024-09-16T23:49:08.674 INFO:tasks.workunit.client.0.smithi042.stdout:0/671: truncate d0/df/d23/d3e/fb4 916950 0 2024-09-16T23:49:08.675 INFO:tasks.workunit.client.1.smithi043.stdout:4/921: creat d8/d25/d39/f11b x:0 0 0 2024-09-16T23:49:08.675 INFO:tasks.workunit.client.1.smithi043.stdout:4/922: fdatasync d8/d25/d39/dd3/d100/d105/f106 0 2024-09-16T23:49:08.676 INFO:tasks.workunit.client.0.smithi042.stdout:1/784: creat d0/d9a/f11c x:0 0 0 2024-09-16T23:49:08.677 INFO:tasks.workunit.client.1.smithi043.stdout:9/876: write d4/d1c/d25/d2b/da1/df9/d45/d85/fed [1102415,28852] 0 2024-09-16T23:49:08.677 INFO:tasks.workunit.client.0.smithi042.stdout:0/672: fdatasync d0/df/d13/f4e 0 2024-09-16T23:49:08.681 INFO:tasks.workunit.client.1.smithi043.stdout:4/923: unlink d8/d15/d20/d2b/c87 0 2024-09-16T23:49:08.682 INFO:tasks.workunit.client.0.smithi042.stdout:8/592: dread da/d19/d1d/fb3 [0,4194304] 0 2024-09-16T23:49:08.686 INFO:tasks.workunit.client.1.smithi043.stdout:4/924: creat d8/d15/d20/d2b/d22/f11c x:0 0 0 2024-09-16T23:49:08.689 INFO:tasks.workunit.client.1.smithi043.stdout:9/877: truncate d4/d21/d35/d4b/fb7 1150238 0 2024-09-16T23:49:08.689 INFO:tasks.workunit.client.0.smithi042.stdout:0/673: dwrite d0/df/d13/f97 [0,4194304] 0 2024-09-16T23:49:08.692 INFO:tasks.workunit.client.1.smithi043.stdout:4/925: unlink d8/d25/d39/dd3/d100/d105/f106 0 2024-09-16T23:49:08.693 INFO:tasks.workunit.client.1.smithi043.stdout:4/926: dread - d8/d25/d39/f11b zero size 2024-09-16T23:49:08.696 INFO:tasks.workunit.client.1.smithi043.stdout:6/856: dwrite de/d23/d34/d9b/d50/fa8 [0,4194304] 0 2024-09-16T23:49:08.696 INFO:tasks.workunit.client.0.smithi042.stdout:1/785: link d0/d17/d5e/cc9 d0/d9/d90/d6b/d88/d32/c11d 0 2024-09-16T23:49:08.698 INFO:tasks.workunit.client.0.smithi042.stdout:8/593: dwrite da/f14 [0,4194304] 0 2024-09-16T23:49:08.705 INFO:tasks.workunit.client.1.smithi043.stdout:6/857: dwrite de/d23/d34/d9b/d5a/dfe/f11c [0,4194304] 0 2024-09-16T23:49:08.706 INFO:tasks.workunit.client.0.smithi042.stdout:1/786: rename d0/d17/c107 to d0/d9/d90/c11e 0 2024-09-16T23:49:08.706 INFO:tasks.workunit.client.0.smithi042.stdout:0/674: dread d0/df/d23/f38 [0,4194304] 0 2024-09-16T23:49:08.706 INFO:tasks.workunit.client.0.smithi042.stdout:0/675: chown d0/df/d13/d49 2292402 1 2024-09-16T23:49:08.707 INFO:tasks.workunit.client.0.smithi042.stdout:8/594: mknod da/d19/d80/cb7 0 2024-09-16T23:49:08.711 INFO:tasks.workunit.client.0.smithi042.stdout:0/676: mkdir d0/df/d23/d3e/d8f/dda 0 2024-09-16T23:49:08.713 INFO:tasks.workunit.client.0.smithi042.stdout:8/595: creat da/d19/d2c/d56/d97/dad/fb8 x:0 0 0 2024-09-16T23:49:08.715 INFO:tasks.workunit.client.1.smithi043.stdout:6/858: write de/d1a/d28/f9a [1855531,9380] 0 2024-09-16T23:49:08.717 INFO:tasks.workunit.client.0.smithi042.stdout:8/596: truncate da/d19/d1d/fb3 5988310 0 2024-09-16T23:49:08.718 INFO:tasks.workunit.client.0.smithi042.stdout:8/597: fsync da/d19/d1d/d30/f45 0 2024-09-16T23:49:08.718 INFO:tasks.workunit.client.1.smithi043.stdout:4/927: dread d8/d15/d20/d2b/d54/f91 [0,4194304] 0 2024-09-16T23:49:08.720 INFO:tasks.workunit.client.0.smithi042.stdout:0/677: rename d0/df/d23/d33/d51/d5b to d0/df/d13/d49/d70/da1/dac/ddb 0 2024-09-16T23:49:08.721 INFO:tasks.workunit.client.1.smithi043.stdout:6/859: link de/d23/d34/d9b/fa9 de/d23/d34/d66/d9d/df3/f129 0 2024-09-16T23:49:08.723 INFO:tasks.workunit.client.0.smithi042.stdout:0/678: unlink d0/df/d23/d33/d56/da9/cd3 0 2024-09-16T23:49:08.723 INFO:tasks.workunit.client.1.smithi043.stdout:6/860: truncate de/d23/d34/d9b/d8b/dbd/f10c 1070074 0 2024-09-16T23:49:08.725 INFO:tasks.workunit.client.1.smithi043.stdout:4/928: creat d8/d15/d20/dfa/f11d x:0 0 0 2024-09-16T23:49:08.725 INFO:tasks.workunit.client.0.smithi042.stdout:0/679: creat d0/df/d13/d2d/d35/d9f/fdc x:0 0 0 2024-09-16T23:49:08.726 INFO:tasks.workunit.client.1.smithi043.stdout:4/929: truncate d8/d25/d64/f7f 4339982 0 2024-09-16T23:49:08.726 INFO:tasks.workunit.client.0.smithi042.stdout:0/680: creat d0/df/d23/d33/d56/da9/fdd x:0 0 0 2024-09-16T23:49:08.727 INFO:tasks.workunit.client.0.smithi042.stdout:8/598: rename da/d19/d2c/d56/l64 to da/d4b/lb9 0 2024-09-16T23:49:08.728 INFO:tasks.workunit.client.1.smithi043.stdout:6/861: truncate de/d1a/d39/f75 1455039 0 2024-09-16T23:49:08.728 INFO:tasks.workunit.client.1.smithi043.stdout:6/862: dread - de/dbe/dca/ffb zero size 2024-09-16T23:49:08.728 INFO:tasks.workunit.client.1.smithi043.stdout:4/930: mknod d8/d25/d64/c11e 0 2024-09-16T23:49:08.730 INFO:tasks.workunit.client.1.smithi043.stdout:6/863: creat de/dbe/f12a x:0 0 0 2024-09-16T23:49:08.733 INFO:tasks.workunit.client.0.smithi042.stdout:8/599: truncate da/d4b/f72 3626215 0 2024-09-16T23:49:08.734 INFO:tasks.workunit.client.0.smithi042.stdout:0/681: dread d0/d48/f53 [0,4194304] 0 2024-09-16T23:49:08.737 INFO:tasks.workunit.client.0.smithi042.stdout:8/600: symlink da/d19/d2c/db0/lba 0 2024-09-16T23:49:08.739 INFO:tasks.workunit.client.0.smithi042.stdout:0/682: rename d0/df/d23/d33/d56/dc5/dcd/dd1/cd9 to d0/df/d13/dd6/cde 0 2024-09-16T23:49:08.740 INFO:tasks.workunit.client.1.smithi043.stdout:4/931: write d8/d15/d20/d2b/d54/db3/d4c/f9c [565157,14636] 0 2024-09-16T23:49:08.742 INFO:tasks.workunit.client.0.smithi042.stdout:0/683: mkdir d0/df/ddf 0 2024-09-16T23:49:08.745 INFO:tasks.workunit.client.1.smithi043.stdout:6/864: dread de/d23/d63/fd2 [0,4194304] 0 2024-09-16T23:49:08.747 INFO:tasks.workunit.client.1.smithi043.stdout:6/865: dread - de/d23/d34/d9b/d5a/dfe/f118 zero size 2024-09-16T23:49:08.747 INFO:tasks.workunit.client.1.smithi043.stdout:5/994: sync 2024-09-16T23:49:08.747 INFO:tasks.workunit.client.1.smithi043.stdout:7/863: sync 2024-09-16T23:49:08.748 INFO:tasks.workunit.client.0.smithi042.stdout:8/601: truncate da/d19/f23 256144 0 2024-09-16T23:49:08.749 INFO:tasks.workunit.client.0.smithi042.stdout:8/602: write da/d19/d2c/d56/f98 [919525,49093] 0 2024-09-16T23:49:08.750 INFO:tasks.workunit.client.1.smithi043.stdout:6/866: read de/d23/d34/d66/d9d/df3/f129 [3609736,109678] 0 2024-09-16T23:49:08.751 INFO:tasks.workunit.client.0.smithi042.stdout:8/603: write da/d19/d2c/d56/fae [3212762,79816] 0 2024-09-16T23:49:08.752 INFO:tasks.workunit.client.0.smithi042.stdout:0/684: write d0/df/f8c [787578,16164] 0 2024-09-16T23:49:08.755 INFO:tasks.workunit.client.1.smithi043.stdout:6/867: mknod de/d1a/d80/c12b 0 2024-09-16T23:49:08.756 INFO:tasks.workunit.client.0.smithi042.stdout:8/604: mknod da/d19/d2c/db0/cbb 0 2024-09-16T23:49:08.756 INFO:tasks.workunit.client.1.smithi043.stdout:4/932: dwrite d8/d15/d20/d2b/f28 [4194304,4194304] 0 2024-09-16T23:49:08.764 INFO:tasks.workunit.client.0.smithi042.stdout:0/685: link d0/df/d13/lc0 d0/df/d13/d49/d70/da1/dac/ddb/le0 0 2024-09-16T23:49:08.765 INFO:tasks.workunit.client.1.smithi043.stdout:6/868: dread de/f67 [0,4194304] 0 2024-09-16T23:49:08.766 INFO:tasks.workunit.client.1.smithi043.stdout:7/864: dread d4/db/d11/d21/f2f [0,4194304] 0 2024-09-16T23:49:08.766 INFO:tasks.workunit.client.0.smithi042.stdout:0/686: fdatasync d0/df/d13/f74 0 2024-09-16T23:49:08.767 INFO:tasks.workunit.client.0.smithi042.stdout:8/605: rmdir da/d19/d1d/d30/d49/d58/d6a/d9b 0 2024-09-16T23:49:08.767 INFO:tasks.workunit.client.1.smithi043.stdout:6/869: chown de/d1a/d28/f9a 47943638 1 2024-09-16T23:49:08.768 INFO:tasks.workunit.client.1.smithi043.stdout:5/995: dread d3/d9/d22/d4a/d4c/fe3 [0,4194304] 0 2024-09-16T23:49:08.771 INFO:tasks.workunit.client.1.smithi043.stdout:6/870: truncate de/d23/d34/d66/fac 4932791 0 2024-09-16T23:49:08.771 INFO:tasks.workunit.client.1.smithi043.stdout:7/865: creat d4/d27/d41/d1b/d95/f11b x:0 0 0 2024-09-16T23:49:08.774 INFO:tasks.workunit.client.1.smithi043.stdout:6/871: creat de/d2a/f12c x:0 0 0 2024-09-16T23:49:08.775 INFO:tasks.workunit.client.0.smithi042.stdout:0/687: dwrite d0/df/d23/d33/d51/d79/fc2 [0,4194304] 0 2024-09-16T23:49:08.776 INFO:tasks.workunit.client.1.smithi043.stdout:5/996: link d3/d9/d6f/f12d d3/d9/d6f/d7c/f14e 0 2024-09-16T23:49:08.777 INFO:tasks.workunit.client.1.smithi043.stdout:4/933: dread d8/d25/d39/d58/f4a [0,4194304] 0 2024-09-16T23:49:08.779 INFO:tasks.workunit.client.0.smithi042.stdout:0/688: rename d0/df/d23/d33/d51/d79/fc9 to d0/df/d23/d3e/d8f/fe1 0 2024-09-16T23:49:08.780 INFO:tasks.workunit.client.1.smithi043.stdout:6/872: symlink de/l12d 0 2024-09-16T23:49:08.780 INFO:tasks.workunit.client.0.smithi042.stdout:0/689: chown d0/df/d23/d33 63411 1 2024-09-16T23:49:08.780 INFO:tasks.workunit.client.0.smithi042.stdout:0/690: truncate d0/df/fb2 245704 0 2024-09-16T23:49:08.784 INFO:tasks.workunit.client.0.smithi042.stdout:0/691: rename d0/df/d13/d2d/cd2 to d0/df/d23/d33/d56/dc5/dcd/ce2 0 2024-09-16T23:49:08.785 INFO:tasks.workunit.client.1.smithi043.stdout:7/866: rename d4/db/d11/d15/d29/d7a/f86 to d4/db/d11/d117/f11c 0 2024-09-16T23:49:08.785 INFO:tasks.workunit.client.0.smithi042.stdout:0/692: dread d0/df/d23/f55 [0,4194304] 0 2024-09-16T23:49:08.786 INFO:tasks.workunit.client.1.smithi043.stdout:6/873: truncate de/d23/dc3/fdf 2091752 0 2024-09-16T23:49:08.787 INFO:tasks.workunit.client.1.smithi043.stdout:7/867: fdatasync d4/d27/d41/d1b/d95/f9a 0 2024-09-16T23:49:08.787 INFO:tasks.workunit.client.1.smithi043.stdout:5/997: getdents d3/db/d19/d122 0 2024-09-16T23:49:08.787 INFO:tasks.workunit.client.1.smithi043.stdout:6/874: dread - de/d23/d34/d9b/d50/f108 zero size 2024-09-16T23:49:08.788 INFO:tasks.workunit.client.1.smithi043.stdout:4/934: fsync d8/d15/d20/dfa/f111 0 2024-09-16T23:49:08.789 INFO:tasks.workunit.client.0.smithi042.stdout:0/693: symlink d0/df/d13/d49/d70/da1/dac/le3 0 2024-09-16T23:49:08.790 INFO:tasks.workunit.client.1.smithi043.stdout:6/875: write de/d23/ddc/f117 [918132,88888] 0 2024-09-16T23:49:08.791 INFO:tasks.workunit.client.1.smithi043.stdout:7/868: read d4/db/d11/d117/ffa [99076,107310] 0 2024-09-16T23:49:08.792 INFO:tasks.workunit.client.1.smithi043.stdout:4/935: symlink d8/d15/d20/d2b/d54/dce/d112/l11f 0 2024-09-16T23:49:08.793 INFO:tasks.workunit.client.0.smithi042.stdout:8/606: dread da/d19/f1f [0,4194304] 0 2024-09-16T23:49:08.797 INFO:tasks.workunit.client.0.smithi042.stdout:0/694: dwrite d0/d48/fb1 [0,4194304] 0 2024-09-16T23:49:08.797 INFO:tasks.workunit.client.1.smithi043.stdout:4/936: mkdir d8/d25/d39/dd3/d100/d108/d4b/d104/d120 0 2024-09-16T23:49:08.799 INFO:tasks.workunit.client.0.smithi042.stdout:8/607: rmdir da/d4b 39 2024-09-16T23:49:08.801 INFO:tasks.workunit.client.0.smithi042.stdout:0/695: dread d0/df/f8c [0,4194304] 0 2024-09-16T23:49:08.803 INFO:tasks.workunit.client.0.smithi042.stdout:8/608: dread da/d19/f39 [0,4194304] 0 2024-09-16T23:49:08.806 INFO:tasks.workunit.client.0.smithi042.stdout:8/609: getdents da/d19/d2c/d56/d97/dab 0 2024-09-16T23:49:08.807 INFO:tasks.workunit.client.0.smithi042.stdout:0/696: truncate d0/df/d23/d33/d56/da9/f65 2067222 0 2024-09-16T23:49:08.809 INFO:tasks.workunit.client.0.smithi042.stdout:8/610: creat da/d2f/fbc x:0 0 0 2024-09-16T23:49:08.812 INFO:tasks.workunit.client.0.smithi042.stdout:0/697: rename d0/df/d13/d49/d70/da1/dac to d0/d15/de4 0 2024-09-16T23:49:08.813 INFO:tasks.workunit.client.0.smithi042.stdout:0/698: write d0/df/d13/d2d/d35/f82 [4285827,72673] 0 2024-09-16T23:49:08.815 INFO:tasks.workunit.client.0.smithi042.stdout:0/699: creat d0/d48/fe5 x:0 0 0 2024-09-16T23:49:08.819 INFO:tasks.workunit.client.0.smithi042.stdout:0/700: write d0/f4 [669422,6446] 0 2024-09-16T23:49:08.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/701: write d0/df/d23/d33/d56/da9/fbc [179465,100273] 0 2024-09-16T23:49:08.823 INFO:tasks.workunit.client.0.smithi042.stdout:0/702: dread d0/df/d23/d33/d56/dc5/fc6 [0,4194304] 0 2024-09-16T23:49:08.824 INFO:tasks.workunit.client.0.smithi042.stdout:0/703: write d0/df/d23/d33/d56/f61 [2076393,34506] 0 2024-09-16T23:49:08.826 INFO:tasks.workunit.client.0.smithi042.stdout:0/704: symlink d0/df/d13/d2d/le6 0 2024-09-16T23:49:08.832 INFO:tasks.workunit.client.0.smithi042.stdout:0/705: dwrite d0/df/d13/d49/d70/f98 [0,4194304] 0 2024-09-16T23:49:08.842 INFO:tasks.workunit.client.0.smithi042.stdout:0/706: dwrite d0/df/d13/f34 [0,4194304] 0 2024-09-16T23:49:08.844 INFO:tasks.workunit.client.1.smithi043.stdout:7/869: mkdir d4/db/d11/d117/d11d 0 2024-09-16T23:49:08.845 INFO:tasks.workunit.client.1.smithi043.stdout:6/876: write de/d23/d34/d9b/d50/f3a [1428830,12799] 0 2024-09-16T23:49:08.846 INFO:tasks.workunit.client.0.smithi042.stdout:0/707: creat d0/df/d13/fe7 x:0 0 0 2024-09-16T23:49:08.847 INFO:tasks.workunit.client.1.smithi043.stdout:7/870: write d4/db/d11/d117/d85/fc2 [4702794,71170] 0 2024-09-16T23:49:08.847 INFO:tasks.workunit.client.0.smithi042.stdout:0/708: dread - d0/df/d13/fa0 zero size 2024-09-16T23:49:08.849 INFO:tasks.workunit.client.1.smithi043.stdout:7/871: chown d4/d27/l6d 1941749208 1 2024-09-16T23:49:08.850 INFO:tasks.workunit.client.1.smithi043.stdout:7/872: chown d4/db/d11/d21/d31 1148748 1 2024-09-16T23:49:08.851 INFO:tasks.workunit.client.1.smithi043.stdout:7/873: chown d4/d27/f3f 386702 1 2024-09-16T23:49:08.852 INFO:tasks.workunit.client.1.smithi043.stdout:6/877: dread de/d23/fb8 [0,4194304] 0 2024-09-16T23:49:08.855 INFO:tasks.workunit.client.0.smithi042.stdout:0/709: dwrite d0/df/d23/d33/d56/da9/fbc [0,4194304] 0 2024-09-16T23:49:08.857 INFO:tasks.workunit.client.1.smithi043.stdout:6/878: dwrite de/d23/d34/d9b/d8b/feb [0,4194304] 0 2024-09-16T23:49:08.865 INFO:tasks.workunit.client.1.smithi043.stdout:6/879: dwrite de/d1a/f41 [0,4194304] 0 2024-09-16T23:49:08.865 INFO:tasks.workunit.client.0.smithi042.stdout:0/710: link d0/df/d23/d3e/f86 d0/d15/fe8 0 2024-09-16T23:49:08.871 INFO:tasks.workunit.client.0.smithi042.stdout:0/711: rename d0/df/d23/d33/d56/da9/f65 to d0/df/d23/d33/d56/dc5/fe9 0 2024-09-16T23:49:08.872 INFO:tasks.workunit.client.0.smithi042.stdout:0/712: fsync d0/f67 0 2024-09-16T23:49:08.875 INFO:tasks.workunit.client.0.smithi042.stdout:0/713: creat d0/df/d13/d49/d70/da1/fea x:0 0 0 2024-09-16T23:49:08.876 INFO:tasks.workunit.client.0.smithi042.stdout:0/714: write d0/df/d23/d3e/faa [2659827,127050] 0 2024-09-16T23:49:08.877 INFO:tasks.workunit.client.1.smithi043.stdout:4/937: link d8/d25/dcd/dd8/f116 d8/d25/d39/d58/f121 0 2024-09-16T23:49:08.877 INFO:tasks.workunit.client.0.smithi042.stdout:0/715: read d0/df/d23/f55 [42106,89034] 0 2024-09-16T23:49:08.878 INFO:tasks.workunit.client.1.smithi043.stdout:4/938: chown d8/d25/d39/dd3/d100/d108/l30 1 1 2024-09-16T23:49:08.879 INFO:tasks.workunit.client.1.smithi043.stdout:6/880: getdents de/d23/d49/dc9 0 2024-09-16T23:49:08.882 INFO:tasks.workunit.client.1.smithi043.stdout:6/881: symlink de/d23/d34/d9b/d5a/dfe/l12e 0 2024-09-16T23:49:08.882 INFO:tasks.workunit.client.0.smithi042.stdout:0/716: link d0/df/d13/d49/d70/f98 d0/df/d13/d2d/d35/feb 0 2024-09-16T23:49:08.885 INFO:tasks.workunit.client.0.smithi042.stdout:0/717: rename d0/d15/f81 to d0/d48/daf/fec 0 2024-09-16T23:49:08.885 INFO:tasks.workunit.client.0.smithi042.stdout:0/718: chown d0/df/d23/l3a 7513374 1 2024-09-16T23:49:08.888 INFO:tasks.workunit.client.0.smithi042.stdout:0/719: mknod d0/df/d23/d33/d51/ced 0 2024-09-16T23:49:08.891 INFO:tasks.workunit.client.0.smithi042.stdout:0/720: symlink d0/df/d13/dcb/dcc/lee 0 2024-09-16T23:49:08.892 INFO:tasks.workunit.client.0.smithi042.stdout:0/721: stat d0/df/d23/d33/d56/da9/l87 0 2024-09-16T23:49:08.893 INFO:tasks.workunit.client.0.smithi042.stdout:0/722: write d0/d15/f85 [234887,64491] 0 2024-09-16T23:49:08.894 INFO:tasks.workunit.client.0.smithi042.stdout:0/723: chown d0/df/d13/l91 1 1 2024-09-16T23:49:08.895 INFO:tasks.workunit.client.1.smithi043.stdout:6/882: truncate de/d1a/d80/fce 177825 0 2024-09-16T23:49:08.897 INFO:tasks.workunit.client.0.smithi042.stdout:0/724: mknod d0/df/ddf/cef 0 2024-09-16T23:49:08.898 INFO:tasks.workunit.client.1.smithi043.stdout:7/874: dwrite d4/db/d11/fcf [0,4194304] 0 2024-09-16T23:49:08.899 INFO:tasks.workunit.client.1.smithi043.stdout:6/883: rmdir de/d23/d34/d9b/d5a 39 2024-09-16T23:49:08.900 INFO:tasks.workunit.client.0.smithi042.stdout:0/725: symlink d0/d48/daf/lf0 0 2024-09-16T23:49:08.901 INFO:tasks.workunit.client.1.smithi043.stdout:7/875: creat d4/db/d11/d15/d29/d7a/f11e x:0 0 0 2024-09-16T23:49:08.902 INFO:tasks.workunit.client.0.smithi042.stdout:0/726: dread - d0/df/d23/d3e/d8f/fe1 zero size 2024-09-16T23:49:08.905 INFO:tasks.workunit.client.1.smithi043.stdout:7/876: mknod d4/dc0/c11f 0 2024-09-16T23:49:08.906 INFO:tasks.workunit.client.1.smithi043.stdout:6/884: creat de/d23/d34/d9b/d5a/dfe/d115/f12f x:0 0 0 2024-09-16T23:49:08.906 INFO:tasks.workunit.client.0.smithi042.stdout:0/727: unlink d0/df/d23/d33/d51/d79/lbd 0 2024-09-16T23:49:08.908 INFO:tasks.workunit.client.0.smithi042.stdout:0/728: write d0/d15/f3c [4319185,46640] 0 2024-09-16T23:49:08.912 INFO:tasks.workunit.client.1.smithi043.stdout:6/885: rename de/d1a/d80/c92 to de/d23/d34/d9b/d8b/dbd/c130 0 2024-09-16T23:49:08.913 INFO:tasks.workunit.client.1.smithi043.stdout:7/877: getdents d4/db/d11/d21/d31/dcb 0 2024-09-16T23:49:08.914 INFO:tasks.workunit.client.1.smithi043.stdout:6/886: write de/d23/d34/d9b/d8b/fea [1784112,14217] 0 2024-09-16T23:49:08.915 INFO:tasks.workunit.client.0.smithi042.stdout:0/729: dwrite d0/df/d23/d33/d51/fd4 [0,4194304] 0 2024-09-16T23:49:08.916 INFO:tasks.workunit.client.1.smithi043.stdout:6/887: write de/d23/dc3/fd4 [2569650,19163] 0 2024-09-16T23:49:08.916 INFO:tasks.workunit.client.0.smithi042.stdout:0/730: chown d0/df/d13/d2d 822895 1 2024-09-16T23:49:08.917 INFO:tasks.workunit.client.1.smithi043.stdout:6/888: chown de/d1a/d28/f9a 959566622 1 2024-09-16T23:49:08.917 INFO:tasks.workunit.client.1.smithi043.stdout:6/889: dread - de/d23/d34/d9b/d50/f79 zero size 2024-09-16T23:49:08.918 INFO:tasks.workunit.client.1.smithi043.stdout:0/957: sync 2024-09-16T23:49:08.918 INFO:tasks.workunit.client.1.smithi043.stdout:8/839: sync 2024-09-16T23:49:08.918 INFO:tasks.workunit.client.1.smithi043.stdout:3/853: sync 2024-09-16T23:49:08.920 INFO:tasks.workunit.client.1.smithi043.stdout:6/890: fdatasync de/d1a/d28/f7d 0 2024-09-16T23:49:08.920 INFO:tasks.workunit.client.1.smithi043.stdout:7/878: symlink d4/db/d11/d117/d85/d8b/d118/l120 0 2024-09-16T23:49:08.922 INFO:tasks.workunit.client.1.smithi043.stdout:3/854: fsync d4/de/d35/f8d 0 2024-09-16T23:49:08.923 INFO:tasks.workunit.client.0.smithi042.stdout:0/731: mkdir d0/df/d13/df1 0 2024-09-16T23:49:08.923 INFO:tasks.workunit.client.1.smithi043.stdout:6/891: chown de/d23/d34/d9b/d5a/dfe/l12e 1880 1 2024-09-16T23:49:08.924 INFO:tasks.workunit.client.1.smithi043.stdout:0/958: mkdir d0/d2/df/d1a/d46/d39/d6e/d147 0 2024-09-16T23:49:08.924 INFO:tasks.workunit.client.1.smithi043.stdout:7/879: fdatasync d4/d27/d41/d1b/d95/ff6 0 2024-09-16T23:49:08.932 INFO:tasks.workunit.client.0.smithi042.stdout:0/732: symlink d0/df/d13/d49/lf2 0 2024-09-16T23:49:08.933 INFO:tasks.workunit.client.0.smithi042.stdout:0/733: stat d0/df/d13/dcb/dcc 0 2024-09-16T23:49:08.936 INFO:tasks.workunit.client.0.smithi042.stdout:0/734: fsync d0/d48/f9c 0 2024-09-16T23:49:08.938 INFO:tasks.workunit.client.0.smithi042.stdout:0/735: rmdir d0/d15/de4/ddb 39 2024-09-16T23:49:08.943 INFO:tasks.workunit.client.0.smithi042.stdout:0/736: creat d0/df/d13/d49/ff3 x:0 0 0 2024-09-16T23:49:08.945 INFO:tasks.workunit.client.0.smithi042.stdout:0/737: creat d0/df/d23/d33/d56/dc5/ff4 x:0 0 0 2024-09-16T23:49:08.948 INFO:tasks.workunit.client.0.smithi042.stdout:0/738: fdatasync d0/df/d13/d2d/d35/feb 0 2024-09-16T23:49:08.958 INFO:tasks.workunit.client.0.smithi042.stdout:0/739: dread d0/f9 [0,4194304] 0 2024-09-16T23:49:08.959 INFO:tasks.workunit.client.0.smithi042.stdout:0/740: chown d0/df/d13/f74 2035202667 1 2024-09-16T23:49:08.960 INFO:tasks.workunit.client.0.smithi042.stdout:0/741: fdatasync d0/df/d23/d33/d56/dc5/fc6 0 2024-09-16T23:49:08.965 INFO:tasks.workunit.client.0.smithi042.stdout:0/742: creat d0/df/d23/d33/d51/ff5 x:0 0 0 2024-09-16T23:49:08.968 INFO:tasks.workunit.client.0.smithi042.stdout:0/743: creat d0/df/d13/d49/d70/ff6 x:0 0 0 2024-09-16T23:49:08.970 INFO:tasks.workunit.client.0.smithi042.stdout:0/744: mkdir d0/df/df7 0 2024-09-16T23:49:08.971 INFO:tasks.workunit.client.0.smithi042.stdout:0/745: chown d0/d15 1 1 2024-09-16T23:49:08.982 INFO:tasks.workunit.client.1.smithi043.stdout:3/855: symlink d4/de/d1f/d34/l109 0 2024-09-16T23:49:08.983 INFO:tasks.workunit.client.1.smithi043.stdout:0/959: dread - d0/fa1 zero size 2024-09-16T23:49:08.984 INFO:tasks.workunit.client.1.smithi043.stdout:6/892: creat de/dbf/f131 x:0 0 0 2024-09-16T23:49:08.986 INFO:tasks.workunit.client.1.smithi043.stdout:8/840: write de/d18/d1f/d25/f6e [1923961,106084] 0 2024-09-16T23:49:08.986 INFO:tasks.workunit.client.1.smithi043.stdout:7/880: mkdir d4/db/d11/d15/dda/d121 0 2024-09-16T23:49:08.988 INFO:tasks.workunit.client.1.smithi043.stdout:7/881: chown d4/d27/d41/d1b/d95/f11b 455 1 2024-09-16T23:49:08.990 INFO:tasks.workunit.client.1.smithi043.stdout:0/960: mkdir d0/d2/df/d1a/d49/d80/d88/dc4/d148 0 2024-09-16T23:49:08.993 INFO:tasks.workunit.client.1.smithi043.stdout:6/893: creat de/d23/d49/dc9/f132 x:0 0 0 2024-09-16T23:49:08.995 INFO:tasks.workunit.client.1.smithi043.stdout:8/841: rmdir de/d18 39 2024-09-16T23:49:08.997 INFO:tasks.workunit.client.1.smithi043.stdout:8/842: dread de/d70/fa0 [0,4194304] 0 2024-09-16T23:49:09.009 INFO:tasks.workunit.client.1.smithi043.stdout:8/843: fdatasync de/d18/d1f/d25/d65/d7f/dca/fe3 0 2024-09-16T23:49:09.012 INFO:tasks.workunit.client.1.smithi043.stdout:3/856: rename d4/de/d1f/l52 to d4/de/d1f/l10a 0 2024-09-16T23:49:09.016 INFO:tasks.workunit.client.1.smithi043.stdout:8/844: truncate de/d18/d27/d41/d5a/d37/f99 4521560 0 2024-09-16T23:49:09.020 INFO:tasks.workunit.client.1.smithi043.stdout:8/845: mknod de/d18/d1f/d2f/d39/cfc 0 2024-09-16T23:49:09.022 INFO:tasks.workunit.client.1.smithi043.stdout:3/857: dread d4/de/d40/f69 [0,4194304] 0 2024-09-16T23:49:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:08 smithi042 ceph-mon[27914]: pgmap v13: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 93 MiB/s rd, 237 MiB/s wr, 471 op/s 2024-09-16T23:49:09.022 INFO:tasks.workunit.client.1.smithi043.stdout:8/846: read de/d70/d3d/fa3 [1458189,37547] 0 2024-09-16T23:49:09.024 INFO:tasks.workunit.client.1.smithi043.stdout:3/858: fdatasync d4/de/d1f/d2c/f90 0 2024-09-16T23:49:09.025 INFO:tasks.workunit.client.1.smithi043.stdout:3/859: readlink d4/de/d40/d4d/l6f 0 2024-09-16T23:49:09.028 INFO:tasks.workunit.client.1.smithi043.stdout:3/860: mkdir d4/d30/dd1/d10b 0 2024-09-16T23:49:09.030 INFO:tasks.workunit.client.1.smithi043.stdout:3/861: chown d4/de/d40/df0/d88/db1/dbb/lf6 1353 1 2024-09-16T23:49:09.034 INFO:tasks.workunit.client.1.smithi043.stdout:2/811: sync 2024-09-16T23:49:09.035 INFO:tasks.workunit.client.1.smithi043.stdout:9/878: sync 2024-09-16T23:49:09.035 INFO:tasks.workunit.client.1.smithi043.stdout:0/961: rmdir d0/d2/df/d1a/d46/d39/d83/dcb 39 2024-09-16T23:49:09.036 INFO:tasks.workunit.client.1.smithi043.stdout:9/879: write d4/d1c/d6b/f3c [2530555,59403] 0 2024-09-16T23:49:09.037 INFO:tasks.workunit.client.1.smithi043.stdout:0/962: chown d0/d2/df/d1a/d49/db2 407 1 2024-09-16T23:49:09.042 INFO:tasks.workunit.client.1.smithi043.stdout:2/812: creat d8/d5e/d83/d3a/d77/da9/d10a/f113 x:0 0 0 2024-09-16T23:49:09.042 INFO:tasks.workunit.client.1.smithi043.stdout:0/963: dread d0/d2/df/d1a/d49/d80/ffe [0,4194304] 0 2024-09-16T23:49:09.044 INFO:tasks.workunit.client.1.smithi043.stdout:9/880: dwrite d4/d1c/d25/db1/fe1 [0,4194304] 0 2024-09-16T23:49:09.047 INFO:tasks.workunit.client.1.smithi043.stdout:0/964: creat d0/d2/df/d1a/d46/d39/d6e/d147/f149 x:0 0 0 2024-09-16T23:49:09.048 INFO:tasks.workunit.client.1.smithi043.stdout:9/881: rmdir d4/d21/d2a/d2c/d7b/df0 39 2024-09-16T23:49:09.050 INFO:tasks.workunit.client.1.smithi043.stdout:0/965: write d0/d2/df/d1a/d46/d39/daa/db7/f12a [607575,81917] 0 2024-09-16T23:49:09.053 INFO:tasks.workunit.client.1.smithi043.stdout:0/966: chown d0/d2/df/d1a/d46/d39/daa/db7/ld0 14 1 2024-09-16T23:49:09.054 INFO:tasks.workunit.client.1.smithi043.stdout:9/882: dread d4/d5/d16/f54 [0,4194304] 0 2024-09-16T23:49:09.058 INFO:tasks.workunit.client.1.smithi043.stdout:9/883: rename d4/f10 to d4/d28/d5f/db6/f11d 0 2024-09-16T23:49:09.062 INFO:tasks.workunit.client.1.smithi043.stdout:9/884: mknod d4/d1c/d25/d2b/d4c/c11e 0 2024-09-16T23:49:09.062 INFO:tasks.workunit.client.1.smithi043.stdout:6/894: write de/d23/d34/d9b/d8b/fe5 [540499,36197] 0 2024-09-16T23:49:09.063 INFO:tasks.workunit.client.0.smithi042.stdout:5/846: sync 2024-09-16T23:49:09.068 INFO:tasks.workunit.client.0.smithi042.stdout:5/847: creat d9/db/f124 x:0 0 0 2024-09-16T23:49:09.069 INFO:tasks.workunit.client.1.smithi043.stdout:7/882: rmdir d4/db 39 2024-09-16T23:49:09.070 INFO:tasks.workunit.client.1.smithi043.stdout:8/847: write de/d18/d1f/d25/d65/d7f/f8c [719847,124529] 0 2024-09-16T23:49:09.071 INFO:tasks.workunit.client.1.smithi043.stdout:0/967: dread d0/d2/df/d1a/d49/d80/d88/fa4 [0,4194304] 0 2024-09-16T23:49:09.073 INFO:tasks.workunit.client.1.smithi043.stdout:3/862: write d4/de/d40/f80 [367756,115717] 0 2024-09-16T23:49:09.074 INFO:tasks.workunit.client.1.smithi043.stdout:3/863: chown d4/de/c36 14782943 1 2024-09-16T23:49:09.076 INFO:tasks.workunit.client.1.smithi043.stdout:8/848: symlink de/d18/d2b/df4/lfd 0 2024-09-16T23:49:09.077 INFO:tasks.workunit.client.1.smithi043.stdout:7/883: stat d4/db/d11/d21/d31/d33/c1a 0 2024-09-16T23:49:09.078 INFO:tasks.workunit.client.1.smithi043.stdout:8/849: truncate de/fb0 9179581 0 2024-09-16T23:49:09.078 INFO:tasks.workunit.client.1.smithi043.stdout:3/864: mkdir d4/de/d35/d10c 0 2024-09-16T23:49:09.079 INFO:tasks.workunit.client.0.smithi042.stdout:5/848: dread d9/db/d19/d21/fb0 [0,4194304] 0 2024-09-16T23:49:09.080 INFO:tasks.workunit.client.1.smithi043.stdout:7/884: mkdir d4/dc0/d122 0 2024-09-16T23:49:09.082 INFO:tasks.workunit.client.0.smithi042.stdout:5/849: creat d9/db/d1b/d79/f125 x:0 0 0 2024-09-16T23:49:09.082 INFO:tasks.workunit.client.1.smithi043.stdout:8/850: mkdir de/d18/d1f/d2f/d39/d4d/dfe 0 2024-09-16T23:49:09.083 INFO:tasks.workunit.client.1.smithi043.stdout:3/865: mknod d4/de/d1f/d2c/c10d 0 2024-09-16T23:49:09.084 INFO:tasks.workunit.client.0.smithi042.stdout:5/850: creat d9/d11a/f126 x:0 0 0 2024-09-16T23:49:09.085 INFO:tasks.workunit.client.1.smithi043.stdout:8/851: mknod de/d70/cff 0 2024-09-16T23:49:09.088 INFO:tasks.workunit.client.1.smithi043.stdout:3/866: creat d4/de/d35/d10c/f10e x:0 0 0 2024-09-16T23:49:09.089 INFO:tasks.workunit.client.0.smithi042.stdout:5/851: rename d9/db/d19/d21/l29 to d9/db/d1b/da4/l127 0 2024-09-16T23:49:09.090 INFO:tasks.workunit.client.1.smithi043.stdout:2/813: dwrite d8/f43 [0,4194304] 0 2024-09-16T23:49:09.094 INFO:tasks.workunit.client.0.smithi042.stdout:5/852: rename d9/db/de/d5e/d11f to d9/db/de/d92/daf/d128 0 2024-09-16T23:49:09.097 INFO:tasks.workunit.client.1.smithi043.stdout:8/852: link de/d18/c1c de/d18/d1f/d25/d82/c100 0 2024-09-16T23:49:09.098 INFO:tasks.workunit.client.0.smithi042.stdout:5/853: symlink d9/db/d1b/da4/l129 0 2024-09-16T23:49:09.099 INFO:tasks.workunit.client.1.smithi043.stdout:3/867: creat d4/de/d1f/f10f x:0 0 0 2024-09-16T23:49:09.100 INFO:tasks.workunit.client.1.smithi043.stdout:0/968: link d0/d2/df/da7/db8/fd9 d0/d2/df/d1a/d46/d39/f14a 0 2024-09-16T23:49:09.104 INFO:tasks.workunit.client.0.smithi042.stdout:5/854: rename d9/db/de/d10/d2f/cbc to d9/c12a 0 2024-09-16T23:49:09.105 INFO:tasks.workunit.client.0.smithi042.stdout:5/855: chown d9/d2c/cc4 6872336 1 2024-09-16T23:49:09.105 INFO:tasks.workunit.client.1.smithi043.stdout:0/969: chown d0/de/l25 22 1 2024-09-16T23:49:09.106 INFO:tasks.workunit.client.0.smithi042.stdout:5/856: write d9/db/d1b/da4/fe7 [894864,5722] 0 2024-09-16T23:49:09.106 INFO:tasks.workunit.client.1.smithi043.stdout:8/853: dwrite de/d18/d1f/d25/fb9 [4194304,4194304] 0 2024-09-16T23:49:09.108 INFO:tasks.workunit.client.0.smithi042.stdout:5/857: fdatasync d9/db/d19/d21/d41/fab 0 2024-09-16T23:49:09.108 INFO:tasks.workunit.client.0.smithi042.stdout:5/858: chown d9/db/f124 14 1 2024-09-16T23:49:09.108 INFO:tasks.workunit.client.0.smithi042.stdout:5/859: chown d9/f74 5755135 1 2024-09-16T23:49:09.109 INFO:tasks.workunit.client.1.smithi043.stdout:6/895: dwrite de/d1a/d28/f7d [0,4194304] 0 2024-09-16T23:49:09.110 INFO:tasks.workunit.client.1.smithi043.stdout:8/854: truncate de/d18/d27/d41/d5a/f24 2411080 0 2024-09-16T23:49:09.116 INFO:tasks.workunit.client.1.smithi043.stdout:8/855: unlink de/d18/d1f/d25/cf1 0 2024-09-16T23:49:09.117 INFO:tasks.workunit.client.1.smithi043.stdout:0/970: dwrite d0/d2/df/dbb/ff1 [0,4194304] 0 2024-09-16T23:49:09.120 INFO:tasks.workunit.client.0.smithi042.stdout:5/860: dread d9/db/d19/d21/d85/dec/d112/fb9 [4194304,4194304] 0 2024-09-16T23:49:09.121 INFO:tasks.workunit.client.1.smithi043.stdout:2/814: write d8/d5e/d83/d3a/f62 [506922,99178] 0 2024-09-16T23:49:09.127 INFO:tasks.workunit.client.1.smithi043.stdout:0/971: unlink d0/d2/df/l5b 0 2024-09-16T23:49:09.127 INFO:tasks.workunit.client.1.smithi043.stdout:2/815: write d8/d4b/d4f/d56/f10b [878732,15851] 0 2024-09-16T23:49:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:08 smithi043 ceph-mon[35191]: pgmap v13: 65 pgs: 65 active+clean; 2.9 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 93 MiB/s rd, 237 MiB/s wr, 471 op/s 2024-09-16T23:49:09.131 INFO:tasks.workunit.client.0.smithi042.stdout:5/861: unlink d9/d4a/d89/cee 0 2024-09-16T23:49:09.131 INFO:tasks.workunit.client.1.smithi043.stdout:2/816: chown d8/d1d/d20/d23/d38/f47 0 1 2024-09-16T23:49:09.134 INFO:tasks.workunit.client.1.smithi043.stdout:0/972: getdents d0/d2/df/d1a/d49/d80/d88/dc4/d148 0 2024-09-16T23:49:09.141 INFO:tasks.workunit.client.1.smithi043.stdout:0/973: mkdir d0/d2/df/d1a/d3b/d14b 0 2024-09-16T23:49:09.145 INFO:tasks.workunit.client.0.smithi042.stdout:5/862: link d9/db/d19/d21/d85/fd1 d9/db/d19/d21/d41/d7d/f12b 0 2024-09-16T23:49:09.147 INFO:tasks.workunit.client.1.smithi043.stdout:3/868: dwrite d4/de/d1f/d2c/db7/dc3/f8e [0,4194304] 0 2024-09-16T23:49:09.149 INFO:tasks.workunit.client.1.smithi043.stdout:6/896: dwrite de/dbf/fe4 [0,4194304] 0 2024-09-16T23:49:09.151 INFO:tasks.workunit.client.1.smithi043.stdout:8/856: dwrite de/d18/d1f/d25/d65/d7f/fbc [0,4194304] 0 2024-09-16T23:49:09.153 INFO:tasks.workunit.client.1.smithi043.stdout:2/817: dwrite d8/d5e/d83/d3a/d77/da9/f100 [0,4194304] 0 2024-09-16T23:49:09.156 INFO:tasks.workunit.client.0.smithi042.stdout:5/863: unlink d9/db/d1b/da4/l129 0 2024-09-16T23:49:09.156 INFO:tasks.workunit.client.1.smithi043.stdout:6/897: truncate de/d23/d34/d66/d9d/faf 1396932 0 2024-09-16T23:49:09.180 INFO:tasks.workunit.client.1.smithi043.stdout:0/974: rmdir d0/d2/df/da7 39 2024-09-16T23:49:09.184 INFO:tasks.workunit.client.1.smithi043.stdout:5/998: sync 2024-09-16T23:49:09.185 INFO:tasks.workunit.client.1.smithi043.stdout:4/939: sync 2024-09-16T23:49:09.192 INFO:tasks.workunit.client.0.smithi042.stdout:5/864: mkdir d9/d11a/d12c 0 2024-09-16T23:49:09.195 INFO:tasks.workunit.client.0.smithi042.stdout:5/865: creat d9/db/de/d92/db5/f12d x:0 0 0 2024-09-16T23:49:09.198 INFO:tasks.workunit.client.0.smithi042.stdout:5/866: creat d9/d2c/d121/f12e x:0 0 0 2024-09-16T23:49:09.210 INFO:tasks.workunit.client.1.smithi043.stdout:2/818: readlink d8/d4b/d5f/d8e/l85 0 2024-09-16T23:49:09.212 INFO:tasks.workunit.client.1.smithi043.stdout:6/898: rename de/d1a/d28/fa5 to de/d23/d63/d7f/f133 0 2024-09-16T23:49:09.214 INFO:tasks.workunit.client.1.smithi043.stdout:4/940: mknod d8/d15/d20/d2b/d54/dce/d112/c122 0 2024-09-16T23:49:09.215 INFO:tasks.workunit.client.1.smithi043.stdout:3/869: dread d4/de/f58 [0,4194304] 0 2024-09-16T23:49:09.215 INFO:tasks.workunit.client.0.smithi042.stdout:6/743: sync 2024-09-16T23:49:09.216 INFO:tasks.workunit.client.0.smithi042.stdout:5/867: write d9/db/d1b/f8c [978844,91863] 0 2024-09-16T23:49:09.218 INFO:tasks.workunit.client.1.smithi043.stdout:0/975: symlink d0/d2/df/d1a/d49/db2/df4/l14c 0 2024-09-16T23:49:09.219 INFO:tasks.workunit.client.0.smithi042.stdout:6/744: mkdir d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/dee 0 2024-09-16T23:49:09.225 INFO:tasks.workunit.client.1.smithi043.stdout:8/857: creat de/d18/d1f/d2f/f101 x:0 0 0 2024-09-16T23:49:09.226 INFO:tasks.workunit.client.0.smithi042.stdout:6/745: write d8/d70/d81/dd/f22 [5010175,42106] 0 2024-09-16T23:49:09.227 INFO:tasks.workunit.client.1.smithi043.stdout:8/858: chown de/d70/d3d/daf 79003524 1 2024-09-16T23:49:09.228 INFO:tasks.workunit.client.1.smithi043.stdout:5/999: dwrite d3/d9/d22/d9f/fc1 [0,4194304] 0 2024-09-16T23:49:09.228 INFO:tasks.workunit.client.1.smithi043.stdout:4/941: dread d8/d15/f98 [0,4194304] 0 2024-09-16T23:49:09.229 INFO:tasks.workunit.client.1.smithi043.stdout:2/819: truncate d8/d11/d16/fa6 401997 0 2024-09-16T23:49:09.232 INFO:tasks.workunit.client.1.smithi043.stdout:3/870: fdatasync d4/d30/f41 0 2024-09-16T23:49:09.233 INFO:tasks.workunit.client.0.smithi042.stdout:6/746: write d8/fa7 [880643,112904] 0 2024-09-16T23:49:09.233 INFO:tasks.workunit.client.0.smithi042.stdout:5/868: dread d9/db/d11/f40 [0,4194304] 0 2024-09-16T23:49:09.235 INFO:tasks.workunit.client.1.smithi043.stdout:0/976: creat d0/d2/df/d1a/d46/d39/d83/f14d x:0 0 0 2024-09-16T23:49:09.236 INFO:tasks.workunit.client.0.smithi042.stdout:6/747: mknod d8/cef 0 2024-09-16T23:49:09.237 INFO:tasks.workunit.client.1.smithi043.stdout:2/820: mkdir d8/d5e/d83/d9e/dab/d114 0 2024-09-16T23:49:09.238 INFO:tasks.workunit.client.1.smithi043.stdout:6/899: dwrite de/d23/d34/d9b/d5a/fc0 [0,4194304] 0 2024-09-16T23:49:09.238 INFO:tasks.workunit.client.0.smithi042.stdout:5/869: truncate d9/d4a/d89/f8a 1127226 0 2024-09-16T23:49:09.239 INFO:tasks.workunit.client.1.smithi043.stdout:4/942: getdents d8/d25/d39/dd3/d100 0 2024-09-16T23:49:09.239 INFO:tasks.workunit.client.1.smithi043.stdout:3/871: truncate d4/de/d40/d4d/fa9 873915 0 2024-09-16T23:49:09.241 INFO:tasks.workunit.client.0.smithi042.stdout:6/748: creat d8/d70/d81/dd/d1b/d50/d96/da0/dd0/ff0 x:0 0 0 2024-09-16T23:49:09.242 INFO:tasks.workunit.client.0.smithi042.stdout:5/870: rmdir d9/db/d11 39 2024-09-16T23:49:09.243 INFO:tasks.workunit.client.1.smithi043.stdout:6/900: mknod de/d23/d49/c134 0 2024-09-16T23:49:09.245 INFO:tasks.workunit.client.1.smithi043.stdout:6/901: truncate de/d23/d34/d66/d9d/f123 114530 0 2024-09-16T23:49:09.245 INFO:tasks.workunit.client.0.smithi042.stdout:6/749: mknod d8/d70/d81/dd/d1b/d50/da2/cf1 0 2024-09-16T23:49:09.247 INFO:tasks.workunit.client.1.smithi043.stdout:4/943: creat d8/d15/d20/dfa/d5e/f123 x:0 0 0 2024-09-16T23:49:09.249 INFO:tasks.workunit.client.1.smithi043.stdout:3/872: mknod d4/de/d1f/d2c/c110 0 2024-09-16T23:49:09.249 INFO:tasks.workunit.client.1.smithi043.stdout:6/902: dread de/d23/d34/d66/fac [0,4194304] 0 2024-09-16T23:49:09.250 INFO:tasks.workunit.client.0.smithi042.stdout:6/750: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/f47 993496 0 2024-09-16T23:49:09.251 INFO:tasks.workunit.client.1.smithi043.stdout:2/821: creat d8/d11/dca/f115 x:0 0 0 2024-09-16T23:49:09.257 INFO:tasks.workunit.client.0.smithi042.stdout:5/871: dwrite d9/db/de/d10/f20 [0,4194304] 0 2024-09-16T23:49:09.258 INFO:tasks.workunit.client.0.smithi042.stdout:7/800: sync 2024-09-16T23:49:09.258 INFO:tasks.workunit.client.0.smithi042.stdout:3/993: sync 2024-09-16T23:49:09.258 INFO:tasks.workunit.client.0.smithi042.stdout:9/565: sync 2024-09-16T23:49:09.258 INFO:tasks.workunit.client.0.smithi042.stdout:1/787: sync 2024-09-16T23:49:09.258 INFO:tasks.workunit.client.0.smithi042.stdout:2/711: sync 2024-09-16T23:49:09.258 INFO:tasks.workunit.client.0.smithi042.stdout:8/611: sync 2024-09-16T23:49:09.258 INFO:tasks.workunit.client.0.smithi042.stdout:0/746: sync 2024-09-16T23:49:09.260 INFO:tasks.workunit.client.0.smithi042.stdout:9/566: readlink d4/d12/d16/l3e 0 2024-09-16T23:49:09.260 INFO:tasks.workunit.client.0.smithi042.stdout:8/612: fsync da/faa 0 2024-09-16T23:49:09.262 INFO:tasks.workunit.client.0.smithi042.stdout:5/872: chown d9/db/d1b/d31/d46/d58/ff9 249499 1 2024-09-16T23:49:09.264 INFO:tasks.workunit.client.0.smithi042.stdout:5/873: dread - d9/d2c/d121/f12e zero size 2024-09-16T23:49:09.267 INFO:tasks.workunit.client.0.smithi042.stdout:7/801: mkdir d3/d1f/d4f/d119 0 2024-09-16T23:49:09.268 INFO:tasks.workunit.client.0.smithi042.stdout:6/751: truncate d8/d70/d81/dd/f11 660613 0 2024-09-16T23:49:09.271 INFO:tasks.workunit.client.0.smithi042.stdout:8/613: mknod da/d19/d2c/d56/d97/cbd 0 2024-09-16T23:49:09.272 INFO:tasks.workunit.client.0.smithi042.stdout:8/614: chown da/d19/d1d/d30/d49/d58/d6a/l7d 39786086 1 2024-09-16T23:49:09.273 INFO:tasks.workunit.client.1.smithi043.stdout:6/903: readlink de/d23/d34/d66/d70/l7e 0 2024-09-16T23:49:09.273 INFO:tasks.workunit.client.1.smithi043.stdout:0/977: dwrite d0/d2/df/f18 [0,4194304] 0 2024-09-16T23:49:09.276 INFO:tasks.workunit.client.0.smithi042.stdout:8/615: write da/d19/d1d/f63 [3047076,49333] 0 2024-09-16T23:49:09.276 INFO:tasks.workunit.client.1.smithi043.stdout:4/944: truncate d8/d15/ff6 1143947 0 2024-09-16T23:49:09.277 INFO:tasks.workunit.client.1.smithi043.stdout:3/873: symlink d4/de/dea/l111 0 2024-09-16T23:49:09.280 INFO:tasks.workunit.client.1.smithi043.stdout:6/904: mknod de/d23/d34/d66/d9d/df3/c135 0 2024-09-16T23:49:09.283 INFO:tasks.workunit.client.1.smithi043.stdout:3/874: creat d4/de/d35/f112 x:0 0 0 2024-09-16T23:49:09.283 INFO:tasks.workunit.client.0.smithi042.stdout:8/616: mkdir da/d19/d1d/d30/d49/db2/dbe 0 2024-09-16T23:49:09.283 INFO:tasks.workunit.client.0.smithi042.stdout:9/567: dread d4/d10/f3f [0,4194304] 0 2024-09-16T23:49:09.287 INFO:tasks.workunit.client.1.smithi043.stdout:0/978: mkdir d0/d2/df/d1a/d49/db2/df4/d14e 0 2024-09-16T23:49:09.287 INFO:tasks.workunit.client.0.smithi042.stdout:0/747: dread d0/f67 [0,4194304] 0 2024-09-16T23:49:09.291 INFO:tasks.workunit.client.1.smithi043.stdout:2/822: dwrite d8/d11/d16/f28 [4194304,4194304] 0 2024-09-16T23:49:09.296 INFO:tasks.workunit.client.0.smithi042.stdout:9/568: dwrite d4/d12/d16/d68/f9f [0,4194304] 0 2024-09-16T23:49:09.303 INFO:tasks.workunit.client.0.smithi042.stdout:0/748: dread d0/d48/daf/fec [0,4194304] 0 2024-09-16T23:49:09.308 INFO:tasks.workunit.client.0.smithi042.stdout:7/802: mkdir d3/d98/da0/d24/d11a 0 2024-09-16T23:49:09.309 INFO:tasks.workunit.client.0.smithi042.stdout:6/752: mknod d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d99/cf2 0 2024-09-16T23:49:09.316 INFO:tasks.workunit.client.1.smithi043.stdout:3/875: dread d4/de/d1f/d2c/db7/dc3/f8c [0,4194304] 0 2024-09-16T23:49:09.317 INFO:tasks.workunit.client.0.smithi042.stdout:9/569: creat d4/d12/d2d/d37/d28/d5e/d83/fb0 x:0 0 0 2024-09-16T23:49:09.317 INFO:tasks.workunit.client.1.smithi043.stdout:3/876: write d4/de/d40/f71 [270635,117695] 0 2024-09-16T23:49:09.318 INFO:tasks.workunit.client.0.smithi042.stdout:7/803: dwrite d3/d1f/d49/d76/f25 [0,4194304] 0 2024-09-16T23:49:09.320 INFO:tasks.workunit.client.0.smithi042.stdout:3/994: write d2/d7/d61/d8e/d9d/d141/d34/d50/d93/f117 [1504730,120059] 0 2024-09-16T23:49:09.322 INFO:tasks.workunit.client.0.smithi042.stdout:5/874: dwrite d9/db/fdc [0,4194304] 0 2024-09-16T23:49:09.322 INFO:tasks.workunit.client.0.smithi042.stdout:3/995: chown d2/d7/dfb/fcf 7575778 1 2024-09-16T23:49:09.324 INFO:tasks.workunit.client.0.smithi042.stdout:9/570: unlink d4/d12/d2d/d37/d3a/d50/f7d 0 2024-09-16T23:49:09.331 INFO:tasks.workunit.client.0.smithi042.stdout:8/617: link da/faa da/d19/d2c/d56/d97/dad/fbf 0 2024-09-16T23:49:09.334 INFO:tasks.workunit.client.0.smithi042.stdout:6/753: creat d8/d70/d81/dd/d2c/ff3 x:0 0 0 2024-09-16T23:49:09.336 INFO:tasks.workunit.client.1.smithi043.stdout:0/979: mkdir d0/d2/df/d1a/d46/d39/d6e/d14f 0 2024-09-16T23:49:09.337 INFO:tasks.workunit.client.0.smithi042.stdout:6/754: chown d8/d70/d81/dd/d1b/d50/d96/da0/dd0 8650 1 2024-09-16T23:49:09.338 INFO:tasks.workunit.client.1.smithi043.stdout:0/980: stat d0/d2/df/d1a/d49/d80/d88/dc4 0 2024-09-16T23:49:09.340 INFO:tasks.workunit.client.0.smithi042.stdout:8/618: write da/d19/f3b [8280214,119253] 0 2024-09-16T23:49:09.340 INFO:tasks.workunit.client.0.smithi042.stdout:2/712: mkdir d0/d30/d43/df0 0 2024-09-16T23:49:09.341 INFO:tasks.workunit.client.0.smithi042.stdout:6/755: read d8/d70/d81/dd/d1b/d2d/da1/fc1 [3616933,19579] 0 2024-09-16T23:49:09.343 INFO:tasks.workunit.client.0.smithi042.stdout:6/756: readlink d8/d70/d81/dd/d1b/d2d/da1/db3/led 0 2024-09-16T23:49:09.343 INFO:tasks.workunit.client.1.smithi043.stdout:3/877: creat d4/de/d5d/def/f113 x:0 0 0 2024-09-16T23:49:09.344 INFO:tasks.workunit.client.1.smithi043.stdout:4/945: dwrite d8/d15/f69 [0,4194304] 0 2024-09-16T23:49:09.346 INFO:tasks.workunit.client.0.smithi042.stdout:3/996: rename d2/d7/dfb/l101 to d2/d7/d61/d8e/d9d/d141/d34/l150 0 2024-09-16T23:49:09.348 INFO:tasks.workunit.client.1.smithi043.stdout:6/905: dwrite fd [0,4194304] 0 2024-09-16T23:49:09.349 INFO:tasks.workunit.client.0.smithi042.stdout:3/997: write d2/d1b/dc8/f144 [48238,84216] 0 2024-09-16T23:49:09.350 INFO:tasks.workunit.client.1.smithi043.stdout:3/878: rmdir d4/de/d40/df0/d88/db1 39 2024-09-16T23:49:09.351 INFO:tasks.workunit.client.0.smithi042.stdout:5/875: write d9/db/d19/d21/d85/fd1 [1657096,22318] 0 2024-09-16T23:49:09.351 INFO:tasks.workunit.client.1.smithi043.stdout:2/823: getdents d8/d1f 0 2024-09-16T23:49:09.355 INFO:tasks.workunit.client.1.smithi043.stdout:6/906: mknod de/d23/d34/d9b/d8b/dbd/c136 0 2024-09-16T23:49:09.357 INFO:tasks.workunit.client.0.smithi042.stdout:8/619: dwrite da/d4b/fa9 [0,4194304] 0 2024-09-16T23:49:09.357 INFO:tasks.workunit.client.0.smithi042.stdout:9/571: rename d4/d12/d16/c2e to d4/d12/d2d/d37/d28/d5e/d83/cb1 0 2024-09-16T23:49:09.358 INFO:tasks.workunit.client.1.smithi043.stdout:4/946: getdents d8/d15/d20/d2b/d22 0 2024-09-16T23:49:09.361 INFO:tasks.workunit.client.0.smithi042.stdout:6/757: dwrite d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d99/fd9 [0,4194304] 0 2024-09-16T23:49:09.361 INFO:tasks.workunit.client.0.smithi042.stdout:3/998: rmdir d2/d1b/dc8 39 2024-09-16T23:49:09.362 INFO:tasks.workunit.client.1.smithi043.stdout:3/879: dwrite d4/de/d1f/d2c/f4a [0,4194304] 0 2024-09-16T23:49:09.364 INFO:tasks.workunit.client.0.smithi042.stdout:5/876: fsync d9/db/d19/d21/d85/dec/d112/d61/f7c 0 2024-09-16T23:49:09.367 INFO:tasks.workunit.client.0.smithi042.stdout:6/758: stat d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/c5c 0 2024-09-16T23:49:09.367 INFO:tasks.workunit.client.0.smithi042.stdout:5/877: chown d9/ld3 14 1 2024-09-16T23:49:09.370 INFO:tasks.workunit.client.1.smithi043.stdout:0/981: write d0/d2/df/f59 [115596,23728] 0 2024-09-16T23:49:09.371 INFO:tasks.workunit.client.1.smithi043.stdout:3/880: dwrite d4/de/d40/f80 [0,4194304] 0 2024-09-16T23:49:09.372 INFO:tasks.workunit.client.1.smithi043.stdout:3/881: write d4/de/d1f/f10f [944902,28848] 0 2024-09-16T23:49:09.373 INFO:tasks.workunit.client.1.smithi043.stdout:3/882: stat d4/f3b 0 2024-09-16T23:49:09.376 INFO:tasks.workunit.client.1.smithi043.stdout:7/885: sync 2024-09-16T23:49:09.377 INFO:tasks.workunit.client.1.smithi043.stdout:9/885: sync 2024-09-16T23:49:09.377 INFO:tasks.workunit.client.1.smithi043.stdout:8/859: sync 2024-09-16T23:49:09.380 INFO:tasks.workunit.client.1.smithi043.stdout:9/886: write d4/d1c/d25/d2b/d4c/fb0 [804098,60372] 0 2024-09-16T23:49:09.404 INFO:tasks.workunit.client.1.smithi043.stdout:6/907: creat de/d23/d34/d9b/d5a/dfe/f137 x:0 0 0 2024-09-16T23:49:09.405 INFO:tasks.workunit.client.1.smithi043.stdout:6/908: dread - de/d1a/d28/f77 zero size 2024-09-16T23:49:09.406 INFO:tasks.workunit.client.1.smithi043.stdout:2/824: creat d8/d4b/d5f/f116 x:0 0 0 2024-09-16T23:49:09.407 INFO:tasks.workunit.client.1.smithi043.stdout:2/825: chown d8/d1f/f6e 103423128 1 2024-09-16T23:49:09.408 INFO:tasks.workunit.client.0.smithi042.stdout:8/620: rename da/d4b/la1 to da/d19/d1d/d5a/d99/lc0 0 2024-09-16T23:49:09.412 INFO:tasks.workunit.client.1.smithi043.stdout:3/883: fdatasync d4/de/d40/df0/d88/db1/fcf 0 2024-09-16T23:49:09.412 INFO:tasks.workunit.client.1.smithi043.stdout:2/826: dread d8/d1d/d25/f2b [0,4194304] 0 2024-09-16T23:49:09.413 INFO:tasks.workunit.client.1.smithi043.stdout:8/860: rmdir de/d18/d27/d41/d5a/dc8 39 2024-09-16T23:49:09.417 INFO:tasks.workunit.client.1.smithi043.stdout:2/827: creat d8/d5e/f117 x:0 0 0 2024-09-16T23:49:09.418 INFO:tasks.workunit.client.1.smithi043.stdout:2/828: dread - d8/d1d/d20/d26/f97 zero size 2024-09-16T23:49:09.419 INFO:tasks.workunit.client.1.smithi043.stdout:9/887: creat d4/d1c/d25/d2b/df7/f11f x:0 0 0 2024-09-16T23:49:09.421 INFO:tasks.workunit.client.1.smithi043.stdout:3/884: symlink d4/d106/l114 0 2024-09-16T23:49:09.422 INFO:tasks.workunit.client.1.smithi043.stdout:9/888: write d4/d28/d5f/db6/fd9 [969150,76347] 0 2024-09-16T23:49:09.423 INFO:tasks.workunit.client.1.smithi043.stdout:3/885: fdatasync d4/d30/fe9 0 2024-09-16T23:49:09.423 INFO:tasks.workunit.client.1.smithi043.stdout:8/861: unlink de/d18/d1f/f32 0 2024-09-16T23:49:09.428 INFO:tasks.workunit.client.1.smithi043.stdout:4/947: write d8/d25/d39/dd3/fe4 [711069,104615] 0 2024-09-16T23:49:09.430 INFO:tasks.workunit.client.1.smithi043.stdout:7/886: truncate d4/db/d11/d15/d29/d7a/fe3 232184 0 2024-09-16T23:49:09.436 INFO:tasks.workunit.client.1.smithi043.stdout:4/948: write d8/d15/d20/dfa/f11d [510104,48850] 0 2024-09-16T23:49:09.437 INFO:tasks.workunit.client.0.smithi042.stdout:5/878: fsync d9/db/d1b/da4/ffc 0 2024-09-16T23:49:09.437 INFO:tasks.workunit.client.1.smithi043.stdout:9/889: dwrite d4/d28/d5f/fd3 [0,4194304] 0 2024-09-16T23:49:09.445 INFO:tasks.workunit.client.1.smithi043.stdout:4/949: write d8/d25/d39/fac [478173,112328] 0 2024-09-16T23:49:09.446 INFO:tasks.workunit.client.1.smithi043.stdout:8/862: creat de/d18/d27/d41/d5a/dc8/deb/f102 x:0 0 0 2024-09-16T23:49:09.446 INFO:tasks.workunit.client.1.smithi043.stdout:6/909: dwrite de/d23/d49/f4c [0,4194304] 0 2024-09-16T23:49:09.447 INFO:tasks.workunit.client.1.smithi043.stdout:3/886: write d4/de/d40/f82 [232007,33258] 0 2024-09-16T23:49:09.448 INFO:tasks.workunit.client.1.smithi043.stdout:7/887: mkdir d4/db/d11/d117/d123 0 2024-09-16T23:49:09.451 INFO:tasks.workunit.client.1.smithi043.stdout:0/982: dwrite d0/d2/df/da7/db8/fd9 [0,4194304] 0 2024-09-16T23:49:09.453 INFO:tasks.workunit.client.0.smithi042.stdout:3/999: dwrite d2/d7/f118 [0,4194304] 0 2024-09-16T23:49:09.459 INFO:tasks.workunit.client.1.smithi043.stdout:9/890: creat d4/d21/d33/d95/f120 x:0 0 0 2024-09-16T23:49:09.462 INFO:tasks.workunit.client.1.smithi043.stdout:0/983: chown d0/d2/df/d1a/d46/d92/ld2 7 1 2024-09-16T23:49:09.463 INFO:tasks.workunit.client.1.smithi043.stdout:2/829: dwrite d8/d1d/d25/fbd [0,4194304] 0 2024-09-16T23:49:09.464 INFO:tasks.workunit.client.1.smithi043.stdout:7/888: fsync d4/db/f18 0 2024-09-16T23:49:09.465 INFO:tasks.workunit.client.0.smithi042.stdout:1/788: mkdir d0/d17/d5e/d11f 0 2024-09-16T23:49:09.465 INFO:tasks.workunit.client.1.smithi043.stdout:3/887: dread - d4/de/d40/df0/d88/db1/dbb/fe4 zero size 2024-09-16T23:49:09.471 INFO:tasks.workunit.client.1.smithi043.stdout:9/891: creat d4/d21/d2a/f121 x:0 0 0 2024-09-16T23:49:09.471 INFO:tasks.workunit.client.1.smithi043.stdout:0/984: symlink d0/d2/l150 0 2024-09-16T23:49:09.471 INFO:tasks.workunit.client.1.smithi043.stdout:3/888: stat d4/de/d1f/d2c/db7/dc3/c102 0 2024-09-16T23:49:09.474 INFO:tasks.workunit.client.1.smithi043.stdout:3/889: fsync d4/d30/f101 0 2024-09-16T23:49:09.474 INFO:tasks.workunit.client.1.smithi043.stdout:4/950: mknod d8/d25/d39/dd3/d100/d108/c124 0 2024-09-16T23:49:09.474 INFO:tasks.workunit.client.1.smithi043.stdout:6/910: getdents de/d23/d34/d9b/d8b/de9 0 2024-09-16T23:49:09.476 INFO:tasks.workunit.client.1.smithi043.stdout:7/889: mknod d4/d27/d41/d1b/d95/de2/c124 0 2024-09-16T23:49:09.477 INFO:tasks.workunit.client.0.smithi042.stdout:6/759: rename d8/d70/d84/cbe to d8/d70/d81/dd/d1b/dbf/cf4 0 2024-09-16T23:49:09.477 INFO:tasks.workunit.client.1.smithi043.stdout:4/951: write d8/d15/d20/dfa/d5e/fff [80663,64467] 0 2024-09-16T23:49:09.478 INFO:tasks.workunit.client.0.smithi042.stdout:9/572: unlink d4/d12/d2d/d37/d28/f4e 0 2024-09-16T23:49:09.479 INFO:tasks.workunit.client.0.smithi042.stdout:5/879: mkdir d9/d12f 0 2024-09-16T23:49:09.483 INFO:tasks.workunit.client.0.smithi042.stdout:6/760: write d8/d70/d81/dd/d1b/d50/fb9 [1632254,10203] 0 2024-09-16T23:49:09.484 INFO:tasks.workunit.client.1.smithi043.stdout:3/890: symlink d4/de/d1f/d2c/db7/dc9/l115 0 2024-09-16T23:49:09.485 INFO:tasks.workunit.client.0.smithi042.stdout:9/573: creat d4/d12/d2d/d37/d3a/dad/d8e/da1/fb2 x:0 0 0 2024-09-16T23:49:09.485 INFO:tasks.workunit.client.0.smithi042.stdout:6/761: dread - d8/d70/d81/dd/d2c/ff3 zero size 2024-09-16T23:49:09.486 INFO:tasks.workunit.client.1.smithi043.stdout:7/890: chown d4/db/d11/d117/d85/d8b/fe7 6701 1 2024-09-16T23:49:09.486 INFO:tasks.workunit.client.0.smithi042.stdout:6/762: chown d8/d70/d81/dd/d1b/dbf/cda 17956 1 2024-09-16T23:49:09.488 INFO:tasks.workunit.client.1.smithi043.stdout:4/952: creat d8/d15/d9b/dbf/f125 x:0 0 0 2024-09-16T23:49:09.489 INFO:tasks.workunit.client.0.smithi042.stdout:5/880: mknod d9/db/d1b/d31/d46/d58/d120/c130 0 2024-09-16T23:49:09.489 INFO:tasks.workunit.client.1.smithi043.stdout:2/830: creat d8/d5e/d83/f118 x:0 0 0 2024-09-16T23:49:09.490 INFO:tasks.workunit.client.0.smithi042.stdout:5/881: readlink d9/db/d19/d21/d85/dec/d112/d61/d43/l113 0 2024-09-16T23:49:09.491 INFO:tasks.workunit.client.1.smithi043.stdout:4/953: stat d8/d15/d9b/fa7 0 2024-09-16T23:49:09.491 INFO:tasks.workunit.client.0.smithi042.stdout:9/574: mkdir d4/d12/d2d/d37/d28/d5e/d83/db3 0 2024-09-16T23:49:09.492 INFO:tasks.workunit.client.1.smithi043.stdout:3/891: creat d4/de/d1f/d2c/db7/dc6/deb/f116 x:0 0 0 2024-09-16T23:49:09.493 INFO:tasks.workunit.client.0.smithi042.stdout:6/763: rmdir d8/d70/d81/dd/d1b/d2d/da1/db3 39 2024-09-16T23:49:09.494 INFO:tasks.workunit.client.1.smithi043.stdout:8/863: dwrite de/d18/d27/d41/d5a/fac [0,4194304] 0 2024-09-16T23:49:09.494 INFO:tasks.workunit.client.1.smithi043.stdout:9/892: rename d4/d28/d5f/lb9 to d4/d1c/d25/l122 0 2024-09-16T23:49:09.495 INFO:tasks.workunit.client.1.smithi043.stdout:0/985: write d0/d2/df/da7/fbd [646534,84447] 0 2024-09-16T23:49:09.496 INFO:tasks.workunit.client.0.smithi042.stdout:9/575: mknod d4/d12/d2d/d37/d41/cb4 0 2024-09-16T23:49:09.497 INFO:tasks.workunit.client.1.smithi043.stdout:6/911: rename de/dbe to de/d138 0 2024-09-16T23:49:09.501 INFO:tasks.workunit.client.1.smithi043.stdout:2/831: truncate d8/d4b/d5f/d8e/de8/feb 2007084 0 2024-09-16T23:49:09.501 INFO:tasks.workunit.client.0.smithi042.stdout:6/764: mknod d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/cf5 0 2024-09-16T23:49:09.502 INFO:tasks.workunit.client.1.smithi043.stdout:8/864: dread - de/d18/d1f/d2f/d39/d7b/d83/fc0 zero size 2024-09-16T23:49:09.502 INFO:tasks.workunit.client.1.smithi043.stdout:3/892: unlink d4/de/d1f/fdb 0 2024-09-16T23:49:09.503 INFO:tasks.workunit.client.1.smithi043.stdout:4/954: mknod d8/d25/d39/c126 0 2024-09-16T23:49:09.503 INFO:tasks.workunit.client.1.smithi043.stdout:7/891: dwrite d4/db/d50/fa9 [0,4194304] 0 2024-09-16T23:49:09.506 INFO:tasks.workunit.client.1.smithi043.stdout:2/832: write d8/d35/f36 [1703518,123170] 0 2024-09-16T23:49:09.509 INFO:tasks.workunit.client.1.smithi043.stdout:3/893: chown d4/de/d1f/d2c/db7/f108 0 1 2024-09-16T23:49:09.510 INFO:tasks.workunit.client.0.smithi042.stdout:6/765: dwrite d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d99/fb6 [0,4194304] 0 2024-09-16T23:49:09.512 INFO:tasks.workunit.client.0.smithi042.stdout:5/882: link d9/db/d19/d21/d85/da5/f88 d9/f131 0 2024-09-16T23:49:09.513 INFO:tasks.workunit.client.1.smithi043.stdout:9/893: dwrite d4/d1c/d25/d2b/d4c/da3/ff2 [0,4194304] 0 2024-09-16T23:49:09.513 INFO:tasks.workunit.client.0.smithi042.stdout:6/766: truncate d8/d70/d81/dd/d1b/d2d/f41 971370 0 2024-09-16T23:49:09.514 INFO:tasks.workunit.client.0.smithi042.stdout:9/576: link d4/l69 d4/d12/d2d/d94/da3/lb5 0 2024-09-16T23:49:09.518 INFO:tasks.workunit.client.0.smithi042.stdout:6/767: dread d8/fa7 [0,4194304] 0 2024-09-16T23:49:09.519 INFO:tasks.workunit.client.1.smithi043.stdout:6/912: creat de/d23/d34/f139 x:0 0 0 2024-09-16T23:49:09.521 INFO:tasks.workunit.client.0.smithi042.stdout:6/768: dread - d8/d70/d81/dd/d1b/d50/fb8 zero size 2024-09-16T23:49:09.521 INFO:tasks.workunit.client.1.smithi043.stdout:0/986: symlink d0/d2/df/d1a/d46/d39/d83/l151 0 2024-09-16T23:49:09.522 INFO:tasks.workunit.client.0.smithi042.stdout:9/577: mknod d4/d10/d4b/cb6 0 2024-09-16T23:49:09.524 INFO:tasks.workunit.client.1.smithi043.stdout:3/894: mkdir d4/de/d1f/d2c/db7/dc6/deb/d117 0 2024-09-16T23:49:09.525 INFO:tasks.workunit.client.0.smithi042.stdout:5/883: rename d9/db/de/d10/l45 to d9/d11a/d12c/l132 0 2024-09-16T23:49:09.526 INFO:tasks.workunit.client.0.smithi042.stdout:9/578: fdatasync d4/d12/d2d/d37/d28/f90 0 2024-09-16T23:49:09.530 INFO:tasks.workunit.client.1.smithi043.stdout:6/913: rename de/d23/d34/d9b/d8b/fea to de/d23/d34/d9b/d8b/de9/f13a 0 2024-09-16T23:49:09.532 INFO:tasks.workunit.client.1.smithi043.stdout:4/955: creat d8/d15/d20/d2b/d54/f127 x:0 0 0 2024-09-16T23:49:09.534 INFO:tasks.workunit.client.1.smithi043.stdout:3/895: mknod d4/de/d1f/dfc/c118 0 2024-09-16T23:49:09.536 INFO:tasks.workunit.client.1.smithi043.stdout:6/914: mkdir de/d1a/d39/d13b 0 2024-09-16T23:49:09.536 INFO:tasks.workunit.client.1.smithi043.stdout:6/915: readlink de/d23/l9f 0 2024-09-16T23:49:09.538 INFO:tasks.workunit.client.1.smithi043.stdout:9/894: write d4/d28/d49/f76 [2764902,44494] 0 2024-09-16T23:49:09.540 INFO:tasks.workunit.client.1.smithi043.stdout:6/916: read de/d23/dc3/fd4 [724896,63038] 0 2024-09-16T23:49:09.540 INFO:tasks.workunit.client.1.smithi043.stdout:9/895: write d4/d21/d35/f113 [202090,76237] 0 2024-09-16T23:49:09.540 INFO:tasks.workunit.client.1.smithi043.stdout:7/892: truncate d4/db/d50/fa9 2575189 0 2024-09-16T23:49:09.540 INFO:tasks.workunit.client.0.smithi042.stdout:5/884: symlink d9/db/d19/l133 0 2024-09-16T23:49:09.542 INFO:tasks.workunit.client.0.smithi042.stdout:5/885: chown d9/db/de/d10/l104 23778 1 2024-09-16T23:49:09.542 INFO:tasks.workunit.client.1.smithi043.stdout:7/893: readlink d4/db/d11/d21/d31/d33/d64/lbf 0 2024-09-16T23:49:09.543 INFO:tasks.workunit.client.0.smithi042.stdout:6/769: dread d8/d70/d81/dd/d2c/d57/f60 [0,4194304] 0 2024-09-16T23:49:09.544 INFO:tasks.workunit.client.0.smithi042.stdout:5/886: chown d9/db/d19/d21/d85/dec/d112/fb9 0 1 2024-09-16T23:49:09.545 INFO:tasks.workunit.client.1.smithi043.stdout:3/896: creat d4/de/d1f/d2c/db7/dc3/dd3/f119 x:0 0 0 2024-09-16T23:49:09.547 INFO:tasks.workunit.client.0.smithi042.stdout:6/770: mknod d8/d70/d81/dd/d1b/dbf/dde/cf6 0 2024-09-16T23:49:09.547 INFO:tasks.workunit.client.1.smithi043.stdout:0/987: dwrite d0/d2/df/d1a/d3b/f2a [0,4194304] 0 2024-09-16T23:49:09.550 INFO:tasks.workunit.client.1.smithi043.stdout:6/917: creat de/d23/d34/d9b/d8b/f13c x:0 0 0 2024-09-16T23:49:09.551 INFO:tasks.workunit.client.1.smithi043.stdout:6/918: readlink de/d23/d49/l52 0 2024-09-16T23:49:09.551 INFO:tasks.workunit.client.1.smithi043.stdout:4/956: truncate d8/d15/d20/dfa/f11d 480133 0 2024-09-16T23:49:09.552 INFO:tasks.workunit.client.1.smithi043.stdout:6/919: write de/d1a/d39/f11b [750147,3965] 0 2024-09-16T23:49:09.553 INFO:tasks.workunit.client.1.smithi043.stdout:9/896: creat d4/d1c/d25/db1/f123 x:0 0 0 2024-09-16T23:49:09.554 INFO:tasks.workunit.client.1.smithi043.stdout:4/957: truncate d8/d15/d20/d2b/d22/fc2 4474140 0 2024-09-16T23:49:09.557 INFO:tasks.workunit.client.1.smithi043.stdout:7/894: mknod d4/db/d11/d21/d31/dcb/df8/c125 0 2024-09-16T23:49:09.557 INFO:tasks.workunit.client.0.smithi042.stdout:6/771: getdents d8/d70/d81/dd/d1b/d67 0 2024-09-16T23:49:09.558 INFO:tasks.workunit.client.0.smithi042.stdout:1/789: dwrite d0/d3b/fe0 [0,4194304] 0 2024-09-16T23:49:09.559 INFO:tasks.workunit.client.1.smithi043.stdout:4/958: symlink d8/d25/d64/l128 0 2024-09-16T23:49:09.562 INFO:tasks.workunit.client.1.smithi043.stdout:3/897: mknod d4/de/c11a 0 2024-09-16T23:49:09.563 INFO:tasks.workunit.client.1.smithi043.stdout:3/898: write d4/de/d35/d10c/f10e [803606,11035] 0 2024-09-16T23:49:09.569 INFO:tasks.workunit.client.0.smithi042.stdout:1/790: write d0/d9/d90/d6b/d88/f61 [4644621,84289] 0 2024-09-16T23:49:09.569 INFO:tasks.workunit.client.0.smithi042.stdout:6/772: write d8/f37 [3613080,22393] 0 2024-09-16T23:49:09.572 INFO:tasks.workunit.client.1.smithi043.stdout:4/959: rename d8/d25/d64/fa0 to d8/d25/d39/d41/d57/f129 0 2024-09-16T23:49:09.574 INFO:tasks.workunit.client.0.smithi042.stdout:9/579: dwrite d4/d12/d2d/d37/d59/f96 [0,4194304] 0 2024-09-16T23:49:09.576 INFO:tasks.workunit.client.0.smithi042.stdout:6/773: readlink d8/d70/d81/dd/d1b/d2d/da1/db3/led 0 2024-09-16T23:49:09.576 INFO:tasks.workunit.client.1.smithi043.stdout:7/895: rename d4/db/d11/d15/f5f to d4/db/d11/d15/d29/da7/f126 0 2024-09-16T23:49:09.578 INFO:tasks.workunit.client.0.smithi042.stdout:1/791: creat d0/d9/d90/d6b/d88/dc2/f120 x:0 0 0 2024-09-16T23:49:09.582 INFO:tasks.workunit.client.0.smithi042.stdout:9/580: unlink d4/d10/d24/l2c 0 2024-09-16T23:49:09.584 INFO:tasks.workunit.client.0.smithi042.stdout:9/581: write d4/d10/f3f [4106645,64162] 0 2024-09-16T23:49:09.585 INFO:tasks.workunit.client.1.smithi043.stdout:4/960: creat d8/d25/d39/f12a x:0 0 0 2024-09-16T23:49:09.586 INFO:tasks.workunit.client.1.smithi043.stdout:7/896: creat d4/db/d11/f127 x:0 0 0 2024-09-16T23:49:09.587 INFO:tasks.workunit.client.0.smithi042.stdout:1/792: link d0/l3c d0/ddb/ddd/l121 0 2024-09-16T23:49:09.587 INFO:tasks.workunit.client.0.smithi042.stdout:9/582: mkdir d4/d12/d60/db7 0 2024-09-16T23:49:09.588 INFO:tasks.workunit.client.1.smithi043.stdout:0/988: write d0/d2/df/f17 [1322236,49907] 0 2024-09-16T23:49:09.588 INFO:tasks.workunit.client.1.smithi043.stdout:6/920: write de/d1a/d80/fce [1180029,126842] 0 2024-09-16T23:49:09.588 INFO:tasks.workunit.client.1.smithi043.stdout:9/897: truncate d4/d28/d5f/d91/fa6 2569362 0 2024-09-16T23:49:09.590 INFO:tasks.workunit.client.0.smithi042.stdout:1/793: read d0/f6 [1522485,91995] 0 2024-09-16T23:49:09.593 INFO:tasks.workunit.client.0.smithi042.stdout:9/583: unlink d4/d12/d2d/d37/d3a/f3d 0 2024-09-16T23:49:09.593 INFO:tasks.workunit.client.0.smithi042.stdout:6/774: truncate d8/d70/d81/dd/d1b/d34/f64 422742 0 2024-09-16T23:49:09.594 INFO:tasks.workunit.client.1.smithi043.stdout:0/989: chown d0/d2/df/d1a/d46/d39/d83/lbe 1197 1 2024-09-16T23:49:09.595 INFO:tasks.workunit.client.0.smithi042.stdout:6/775: fsync d8/d70/d81/dd/d2c/d57/fbc 0 2024-09-16T23:49:09.595 INFO:tasks.workunit.client.1.smithi043.stdout:7/897: mknod d4/d27/d41/d110/c128 0 2024-09-16T23:49:09.598 INFO:tasks.workunit.client.0.smithi042.stdout:9/584: fdatasync d4/d12/d2d/d37/d28/f31 0 2024-09-16T23:49:09.599 INFO:tasks.workunit.client.1.smithi043.stdout:9/898: mkdir d4/d21/d124 0 2024-09-16T23:49:09.600 INFO:tasks.workunit.client.1.smithi043.stdout:3/899: dwrite d4/de/d1f/d2c/f90 [0,4194304] 0 2024-09-16T23:49:09.600 INFO:tasks.workunit.client.1.smithi043.stdout:7/898: write d4/db/d11/d15/dbe/f105 [909352,102436] 0 2024-09-16T23:49:09.603 INFO:tasks.workunit.client.1.smithi043.stdout:9/899: read - d4/d28/d5f/fbd zero size 2024-09-16T23:49:09.603 INFO:tasks.workunit.client.0.smithi042.stdout:6/776: dwrite d8/d70/d81/dd/d2c/f31 [4194304,4194304] 0 2024-09-16T23:49:09.606 INFO:tasks.workunit.client.1.smithi043.stdout:9/900: fdatasync d4/d21/d35/f113 0 2024-09-16T23:49:09.606 INFO:tasks.workunit.client.1.smithi043.stdout:0/990: rename d0/d2/df/f114 to d0/d2/df/d1a/d46/d92/d121/f152 0 2024-09-16T23:49:09.607 INFO:tasks.workunit.client.1.smithi043.stdout:9/901: fsync d4/d5/d64/f65 0 2024-09-16T23:49:09.607 INFO:tasks.workunit.client.1.smithi043.stdout:4/961: dread d8/d15/d9b/fa7 [0,4194304] 0 2024-09-16T23:49:09.607 INFO:tasks.workunit.client.0.smithi042.stdout:9/585: rename d4/d12/d2d/d37/d41/c54 to d4/d12/d60/d97/cb8 0 2024-09-16T23:49:09.609 INFO:tasks.workunit.client.1.smithi043.stdout:9/902: stat d4/d1c/d25/db1/d108 0 2024-09-16T23:49:09.609 INFO:tasks.workunit.client.1.smithi043.stdout:6/921: write de/d23/d34/d9b/d50/f79 [606764,68282] 0 2024-09-16T23:49:09.612 INFO:tasks.workunit.client.0.smithi042.stdout:6/777: symlink d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/lf7 0 2024-09-16T23:49:09.612 INFO:tasks.workunit.client.0.smithi042.stdout:9/586: mkdir d4/d12/d2d/d51/db9 0 2024-09-16T23:49:09.612 INFO:tasks.workunit.client.0.smithi042.stdout:1/794: write d0/d3b/f82 [3941687,35192] 0 2024-09-16T23:49:09.613 INFO:tasks.workunit.client.1.smithi043.stdout:9/903: chown d4/d1c/d25/d2b/fa5 213 1 2024-09-16T23:49:09.613 INFO:tasks.workunit.client.1.smithi043.stdout:6/922: chown de/d1a/d39/ff2 90 1 2024-09-16T23:49:09.615 INFO:tasks.workunit.client.0.smithi042.stdout:1/795: write d0/d9/d90/d6b/d88/d32/d2f/f11b [626619,26709] 0 2024-09-16T23:49:09.616 INFO:tasks.workunit.client.0.smithi042.stdout:6/778: mknod d8/d70/d81/d90/d9f/cf8 0 2024-09-16T23:49:09.617 INFO:tasks.workunit.client.0.smithi042.stdout:9/587: fsync d4/d12/d2d/d37/f6a 0 2024-09-16T23:49:09.618 INFO:tasks.workunit.client.1.smithi043.stdout:4/962: mknod d8/d15/d20/d2b/d54/c12b 0 2024-09-16T23:49:09.620 INFO:tasks.workunit.client.1.smithi043.stdout:9/904: symlink d4/d5/d64/df8/d100/l125 0 2024-09-16T23:49:09.620 INFO:tasks.workunit.client.1.smithi043.stdout:0/991: dwrite d0/d2/df/f18 [4194304,4194304] 0 2024-09-16T23:49:09.622 INFO:tasks.workunit.client.0.smithi042.stdout:6/779: creat d8/d70/d81/dd/d1b/d34/d39/d5e/ff9 x:0 0 0 2024-09-16T23:49:09.623 INFO:tasks.workunit.client.1.smithi043.stdout:9/905: fdatasync d4/d21/d2a/d2c/f69 0 2024-09-16T23:49:09.623 INFO:tasks.workunit.client.1.smithi043.stdout:4/963: symlink d8/d15/d9b/l12c 0 2024-09-16T23:49:09.631 INFO:tasks.workunit.client.0.smithi042.stdout:9/588: mknod d4/d12/d2d/d37/cba 0 2024-09-16T23:49:09.632 INFO:tasks.workunit.client.1.smithi043.stdout:6/923: dwrite de/d138/f107 [0,4194304] 0 2024-09-16T23:49:09.632 INFO:tasks.workunit.client.0.smithi042.stdout:9/589: write d4/d10/f1b [1850660,52078] 0 2024-09-16T23:49:09.632 INFO:tasks.workunit.client.0.smithi042.stdout:6/780: link d8/d70/d81/dd/d1b/d34/d39/d5e/ca5 d8/d70/d81/dd/d1b/d34/d39/cfa 0 2024-09-16T23:49:09.634 INFO:tasks.workunit.client.0.smithi042.stdout:1/796: getdents d0/d9/d90/d6b 0 2024-09-16T23:49:09.639 INFO:tasks.workunit.client.1.smithi043.stdout:3/900: dwrite d4/de/d40/df0/f5c [0,4194304] 0 2024-09-16T23:49:09.640 INFO:tasks.workunit.client.0.smithi042.stdout:6/781: dread d8/d70/d81/dd/d2c/d57/fca [0,4194304] 0 2024-09-16T23:49:09.642 INFO:tasks.workunit.client.0.smithi042.stdout:1/797: fdatasync d0/d17/d5e/fd3 0 2024-09-16T23:49:09.642 INFO:tasks.workunit.client.1.smithi043.stdout:0/992: dwrite d0/d2/df/d1a/d46/d39/d127/f144 [0,4194304] 0 2024-09-16T23:49:09.643 INFO:tasks.workunit.client.1.smithi043.stdout:4/964: link d8/d25/d39/dd3/d100/d108/cb6 d8/d15/d20/d2b/d23/c12d 0 2024-09-16T23:49:09.645 INFO:tasks.workunit.client.1.smithi043.stdout:3/901: write d4/de/d1f/d34/fa8 [4560400,97326] 0 2024-09-16T23:49:09.646 INFO:tasks.workunit.client.1.smithi043.stdout:9/906: dwrite d4/d28/d49/f76 [0,4194304] 0 2024-09-16T23:49:09.648 INFO:tasks.workunit.client.0.smithi042.stdout:1/798: creat d0/d9/d90/d66/de2/f122 x:0 0 0 2024-09-16T23:49:09.648 INFO:tasks.workunit.client.1.smithi043.stdout:6/924: mkdir de/d1a/d28/d13d 0 2024-09-16T23:49:09.649 INFO:tasks.workunit.client.0.smithi042.stdout:9/590: link d4/d12/d2d/d94/da3/lb5 d4/d12/d2d/d37/d28/d5e/d83/lbb 0 2024-09-16T23:49:09.654 INFO:tasks.workunit.client.0.smithi042.stdout:1/799: mknod d0/d9/d90/d43/c123 0 2024-09-16T23:49:09.659 INFO:tasks.workunit.client.0.smithi042.stdout:1/800: symlink d0/d9/d90/d6b/d8e/de5/l124 0 2024-09-16T23:49:09.660 INFO:tasks.workunit.client.1.smithi043.stdout:6/925: creat de/d23/d34/d9b/d5a/dfe/d10d/f13e x:0 0 0 2024-09-16T23:49:09.660 INFO:tasks.workunit.client.0.smithi042.stdout:1/801: readlink d0/d9/d4e/lc0 0 2024-09-16T23:49:09.660 INFO:tasks.workunit.client.0.smithi042.stdout:9/591: dread d4/d12/d2d/d37/d28/f31 [0,4194304] 0 2024-09-16T23:49:09.662 INFO:tasks.workunit.client.1.smithi043.stdout:6/926: truncate de/d23/d34/d9b/d8b/fbb 500424 0 2024-09-16T23:49:09.664 INFO:tasks.workunit.client.1.smithi043.stdout:9/907: getdents d4/d1c/d25/d2b/da1/df9 0 2024-09-16T23:49:09.666 INFO:tasks.workunit.client.0.smithi042.stdout:1/802: rmdir d0/d17/d18/d60 39 2024-09-16T23:49:09.667 INFO:tasks.workunit.client.1.smithi043.stdout:6/927: creat de/d1a/d28/d13d/f13f x:0 0 0 2024-09-16T23:49:09.669 INFO:tasks.workunit.client.0.smithi042.stdout:1/803: chown d0/d9/ce9 1 1 2024-09-16T23:49:09.669 INFO:tasks.workunit.client.1.smithi043.stdout:9/908: symlink d4/d1c/d25/d2b/d4c/da3/dca/dfe/d117/l126 0 2024-09-16T23:49:09.670 INFO:tasks.workunit.client.1.smithi043.stdout:6/928: creat de/d23/d34/d66/d70/f140 x:0 0 0 2024-09-16T23:49:09.673 INFO:tasks.workunit.client.0.smithi042.stdout:9/592: write d4/d12/d2d/d37/d28/d5e/f7a [573987,85688] 0 2024-09-16T23:49:09.674 INFO:tasks.workunit.client.1.smithi043.stdout:9/909: symlink d4/d21/d35/dd4/l127 0 2024-09-16T23:49:09.675 INFO:tasks.workunit.client.0.smithi042.stdout:1/804: link d0/d9/d90/d6b/d88/d32/d2f/de3/fff d0/d17/f125 0 2024-09-16T23:49:09.677 INFO:tasks.workunit.client.0.smithi042.stdout:6/782: dwrite d8/d70/d81/dd/d2c/d57/f77 [0,4194304] 0 2024-09-16T23:49:09.678 INFO:tasks.workunit.client.1.smithi043.stdout:9/910: mkdir d4/d21/d124/d128 0 2024-09-16T23:49:09.680 INFO:tasks.workunit.client.0.smithi042.stdout:6/783: chown d8/d70/d81/dd/f19 46 1 2024-09-16T23:49:09.681 INFO:tasks.workunit.client.0.smithi042.stdout:1/805: creat d0/d9/d90/d6b/d88/dc2/f126 x:0 0 0 2024-09-16T23:49:09.683 INFO:tasks.workunit.client.1.smithi043.stdout:4/965: dwrite d8/d25/d64/fb4 [0,4194304] 0 2024-09-16T23:49:09.685 INFO:tasks.workunit.client.1.smithi043.stdout:3/902: dwrite d4/de/f55 [4194304,4194304] 0 2024-09-16T23:49:09.686 INFO:tasks.workunit.client.0.smithi042.stdout:9/593: dwrite d4/d12/d16/d68/f71 [0,4194304] 0 2024-09-16T23:49:09.687 INFO:tasks.workunit.client.1.smithi043.stdout:4/966: dread - d8/d15/d20/d2b/d22/f11c zero size 2024-09-16T23:49:09.687 INFO:tasks.workunit.client.1.smithi043.stdout:3/903: chown d4/de/d40/cbc 2260 1 2024-09-16T23:49:09.687 INFO:tasks.workunit.client.0.smithi042.stdout:6/784: write d8/d70/d81/dd/d1b/d34/d39/d5e/ff9 [264966,96223] 0 2024-09-16T23:49:09.689 INFO:tasks.workunit.client.1.smithi043.stdout:3/904: fsync d4/de/d1f/d2c/db7/dc9/fe3 0 2024-09-16T23:49:09.690 INFO:tasks.workunit.client.1.smithi043.stdout:9/911: dread d4/d28/d5f/fd3 [0,4194304] 0 2024-09-16T23:49:09.692 INFO:tasks.workunit.client.0.smithi042.stdout:6/785: readlink d8/d70/d81/dd/d1b/d34/d39/l49 0 2024-09-16T23:49:09.693 INFO:tasks.workunit.client.1.smithi043.stdout:3/905: write d4/de/d1f/f10f [1340038,58595] 0 2024-09-16T23:49:09.694 INFO:tasks.workunit.client.0.smithi042.stdout:9/594: dread - d4/d12/d60/f78 zero size 2024-09-16T23:49:09.696 INFO:tasks.workunit.client.0.smithi042.stdout:1/806: dwrite d0/d9/d90/d6b/fae [0,4194304] 0 2024-09-16T23:49:09.697 INFO:tasks.workunit.client.1.smithi043.stdout:4/967: mkdir d8/d25/dcd/df4/d12e 0 2024-09-16T23:49:09.698 INFO:tasks.workunit.client.1.smithi043.stdout:9/912: symlink d4/d21/d35/d4b/l129 0 2024-09-16T23:49:09.702 INFO:tasks.workunit.client.1.smithi043.stdout:9/913: write d4/d28/d49/f114 [822533,95643] 0 2024-09-16T23:49:09.703 INFO:tasks.workunit.client.1.smithi043.stdout:6/929: dwrite de/d23/d63/d7f/f133 [0,4194304] 0 2024-09-16T23:49:09.705 INFO:tasks.workunit.client.0.smithi042.stdout:6/786: dread d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 [0,4194304] 0 2024-09-16T23:49:09.708 INFO:tasks.workunit.client.0.smithi042.stdout:9/595: mknod d4/d12/d2d/d37/d28/d5e/d83/db3/cbc 0 2024-09-16T23:49:09.714 INFO:tasks.workunit.client.1.smithi043.stdout:3/906: unlink d4/de/d40/df0/d88/db1/dbb/cdc 0 2024-09-16T23:49:09.714 INFO:tasks.workunit.client.1.smithi043.stdout:3/907: chown d4/de/d40/d4d/lf7 173966328 1 2024-09-16T23:49:09.715 INFO:tasks.workunit.client.0.smithi042.stdout:9/596: truncate d4/d12/d16/d68/f72 2097795 0 2024-09-16T23:49:09.715 INFO:tasks.workunit.client.1.smithi043.stdout:3/908: read - d4/de/d40/df0/d88/f93 zero size 2024-09-16T23:49:09.717 INFO:tasks.workunit.client.1.smithi043.stdout:6/930: truncate de/d23/fa2 1162467 0 2024-09-16T23:49:09.718 INFO:tasks.workunit.client.0.smithi042.stdout:1/807: dwrite d0/d9a/fb5 [0,4194304] 0 2024-09-16T23:49:09.722 INFO:tasks.workunit.client.1.smithi043.stdout:3/909: dwrite d4/de/d1f/d34/fa8 [4194304,4194304] 0 2024-09-16T23:49:09.723 INFO:tasks.workunit.client.1.smithi043.stdout:4/968: getdents d8/d15/d20/d2b/d22 0 2024-09-16T23:49:09.724 INFO:tasks.workunit.client.1.smithi043.stdout:4/969: dread - d8/f68 zero size 2024-09-16T23:49:09.726 INFO:tasks.workunit.client.0.smithi042.stdout:1/808: link d0/d3b/d8d/fa3 d0/d17/dd4/f127 0 2024-09-16T23:49:09.727 INFO:tasks.workunit.client.1.smithi043.stdout:9/914: rename d4/d28/d5f/db6/f11d to d4/d1c/d25/d2b/d4c/da3/dca/f12a 0 2024-09-16T23:49:09.727 INFO:tasks.workunit.client.0.smithi042.stdout:1/809: chown d0/d9/d26/dc6 1815427 1 2024-09-16T23:49:09.729 INFO:tasks.workunit.client.0.smithi042.stdout:9/597: write d4/d12/d2d/d37/d3a/f5c [720471,76080] 0 2024-09-16T23:49:09.729 INFO:tasks.workunit.client.1.smithi043.stdout:3/910: mknod d4/de/d40/df0/c11b 0 2024-09-16T23:49:09.731 INFO:tasks.workunit.client.1.smithi043.stdout:3/911: write d4/de/d1f/d2c/db7/dc6/fee [788804,63427] 0 2024-09-16T23:49:09.732 INFO:tasks.workunit.client.1.smithi043.stdout:3/912: chown d4/de/c105 41110 1 2024-09-16T23:49:09.733 INFO:tasks.workunit.client.0.smithi042.stdout:1/810: creat d0/f128 x:0 0 0 2024-09-16T23:49:09.734 INFO:tasks.workunit.client.1.smithi043.stdout:6/931: rename de/d23/ddc to de/d23/d49/dc9/d141 0 2024-09-16T23:49:09.734 INFO:tasks.workunit.client.1.smithi043.stdout:9/915: symlink d4/d21/d2a/l12b 0 2024-09-16T23:49:09.734 INFO:tasks.workunit.client.1.smithi043.stdout:3/913: truncate d4/de/d40/d4d/fa9 1143470 0 2024-09-16T23:49:09.736 INFO:tasks.workunit.client.1.smithi043.stdout:9/916: rmdir d4/d21/d35/d4b 39 2024-09-16T23:49:09.739 INFO:tasks.workunit.client.1.smithi043.stdout:4/970: rename d8/d25/d39/d41/d57/ded/f110 to d8/d15/d20/d2b/d23/d117/f12f 0 2024-09-16T23:49:09.740 INFO:tasks.workunit.client.1.smithi043.stdout:6/932: mknod de/d23/dc3/c142 0 2024-09-16T23:49:09.740 INFO:tasks.workunit.client.0.smithi042.stdout:9/598: getdents d4/d12/d2d/d37/d41/d92 0 2024-09-16T23:49:09.741 INFO:tasks.workunit.client.0.smithi042.stdout:9/599: readlink d4/d12/d2d/d51/l66 0 2024-09-16T23:49:09.742 INFO:tasks.workunit.client.1.smithi043.stdout:3/914: fdatasync d4/fb9 0 2024-09-16T23:49:09.744 INFO:tasks.workunit.client.0.smithi042.stdout:9/600: symlink d4/d12/d2d/d37/d41/d92/da7/lbd 0 2024-09-16T23:49:09.746 INFO:tasks.workunit.client.0.smithi042.stdout:9/601: read d4/d12/d2d/d37/d3a/d50/f99 [16172,58302] 0 2024-09-16T23:49:09.748 INFO:tasks.workunit.client.1.smithi043.stdout:6/933: creat de/d23/d34/d66/d9d/df3/f143 x:0 0 0 2024-09-16T23:49:09.748 INFO:tasks.workunit.client.0.smithi042.stdout:9/602: read d4/d10/f30 [2372061,97247] 0 2024-09-16T23:49:09.749 INFO:tasks.workunit.client.0.smithi042.stdout:9/603: fdatasync d4/d12/d60/f78 0 2024-09-16T23:49:09.751 INFO:tasks.workunit.client.1.smithi043.stdout:9/917: link d4/d1c/d25/d2b/da1/df9/d45/d85/f115 d4/d21/d2a/f12c 0 2024-09-16T23:49:09.751 INFO:tasks.workunit.client.0.smithi042.stdout:1/811: dread d0/d9/d90/d6b/d88/def/f104 [0,4194304] 0 2024-09-16T23:49:09.751 INFO:tasks.workunit.client.1.smithi043.stdout:4/971: truncate d8/d15/d20/dfa/f11d 1308753 0 2024-09-16T23:49:09.752 INFO:tasks.workunit.client.0.smithi042.stdout:1/812: stat d0/d9/d90/d6b/d88/dc2/cca 0 2024-09-16T23:49:09.753 INFO:tasks.workunit.client.1.smithi043.stdout:3/915: mknod d4/de/d1f/c11c 0 2024-09-16T23:49:09.753 INFO:tasks.workunit.client.0.smithi042.stdout:1/813: chown d0/d9/d26/daa/c114 2005439 1 2024-09-16T23:49:09.754 INFO:tasks.workunit.client.1.smithi043.stdout:3/916: chown d4/de/d1f/d34/fa8 7554987 1 2024-09-16T23:49:09.755 INFO:tasks.workunit.client.1.smithi043.stdout:6/934: read de/d23/d34/d66/d9d/dae/fc2 [776467,74227] 0 2024-09-16T23:49:09.756 INFO:tasks.workunit.client.1.smithi043.stdout:9/918: link d4/d1c/d25/d2b/la7 d4/d1c/d25/d2b/d110/dbf/l12d 0 2024-09-16T23:49:09.757 INFO:tasks.workunit.client.1.smithi043.stdout:3/917: symlink d4/l11d 0 2024-09-16T23:49:09.760 INFO:tasks.workunit.client.0.smithi042.stdout:1/814: dwrite d0/d9/d90/d66/de2/f119 [0,4194304] 0 2024-09-16T23:49:09.761 INFO:tasks.workunit.client.0.smithi042.stdout:1/815: read d0/f6 [1417759,74099] 0 2024-09-16T23:49:09.761 INFO:tasks.workunit.client.1.smithi043.stdout:3/918: symlink d4/de/d1f/d2c/db7/dc6/deb/l11e 0 2024-09-16T23:49:09.762 INFO:tasks.workunit.client.1.smithi043.stdout:9/919: dread d4/d21/d33/d95/fc0 [0,4194304] 0 2024-09-16T23:49:09.765 INFO:tasks.workunit.client.1.smithi043.stdout:4/972: dread d8/d15/d20/d2b/d23/fd1 [0,4194304] 0 2024-09-16T23:49:09.766 INFO:tasks.workunit.client.1.smithi043.stdout:4/973: dread - d8/d15/ffc zero size 2024-09-16T23:49:09.767 INFO:tasks.workunit.client.1.smithi043.stdout:6/935: write de/dbf/fd5 [906452,69716] 0 2024-09-16T23:49:09.769 INFO:tasks.workunit.client.1.smithi043.stdout:6/936: chown de/d23/d34/d9b/d8b 26 1 2024-09-16T23:49:09.770 INFO:tasks.workunit.client.1.smithi043.stdout:9/920: creat d4/d21/f12e x:0 0 0 2024-09-16T23:49:09.771 INFO:tasks.workunit.client.1.smithi043.stdout:4/974: creat d8/d15/d20/d2b/d22/f130 x:0 0 0 2024-09-16T23:49:09.773 INFO:tasks.workunit.client.1.smithi043.stdout:6/937: chown de/d23/d49/l54 26383 1 2024-09-16T23:49:09.774 INFO:tasks.workunit.client.1.smithi043.stdout:3/919: dwrite d4/de/d35/fc5 [0,4194304] 0 2024-09-16T23:49:09.777 INFO:tasks.workunit.client.1.smithi043.stdout:6/938: write de/d23/d34/d9b/d5a/dfe/d10d/f13e [265720,62884] 0 2024-09-16T23:49:09.778 INFO:tasks.workunit.client.1.smithi043.stdout:9/921: getdents d4/d1c/d25/d2b/da1/df9 0 2024-09-16T23:49:09.781 INFO:tasks.workunit.client.1.smithi043.stdout:3/920: write d4/de/d40/df0/d88/db1/fc2 [187083,53406] 0 2024-09-16T23:49:09.782 INFO:tasks.workunit.client.1.smithi043.stdout:6/939: symlink de/d23/l144 0 2024-09-16T23:49:09.782 INFO:tasks.workunit.client.1.smithi043.stdout:4/975: link d8/d15/d20/d2b/d54/db3/d4c/fc6 d8/d25/d39/f131 0 2024-09-16T23:49:09.784 INFO:tasks.workunit.client.1.smithi043.stdout:6/940: mkdir de/d23/d49/d145 0 2024-09-16T23:49:09.787 INFO:tasks.workunit.client.1.smithi043.stdout:3/921: truncate d4/de/d1f/d2c/db7/dc3/fe0 978532 0 2024-09-16T23:49:09.788 INFO:tasks.workunit.client.1.smithi043.stdout:9/922: getdents d4/d21/d2a/d2c/d7b 0 2024-09-16T23:49:09.789 INFO:tasks.workunit.client.1.smithi043.stdout:4/976: write d8/d25/f6a [1508476,14796] 0 2024-09-16T23:49:09.790 INFO:tasks.workunit.client.1.smithi043.stdout:3/922: write d4/de/d40/df0/d88/f93 [95943,15578] 0 2024-09-16T23:49:09.792 INFO:tasks.workunit.client.1.smithi043.stdout:3/923: fdatasync d4/f67 0 2024-09-16T23:49:09.796 INFO:tasks.workunit.client.1.smithi043.stdout:9/923: write d4/d5/d16/fa0 [1005750,20444] 0 2024-09-16T23:49:09.796 INFO:tasks.workunit.client.1.smithi043.stdout:4/977: write d8/d15/d20/f8d [87089,33324] 0 2024-09-16T23:49:09.798 INFO:tasks.workunit.client.1.smithi043.stdout:3/924: dread d4/de/d1f/d2c/db7/dc3/f89 [0,4194304] 0 2024-09-16T23:49:09.799 INFO:tasks.workunit.client.1.smithi043.stdout:4/978: chown d8/d25/d39/ca6 76 1 2024-09-16T23:49:09.799 INFO:tasks.workunit.client.1.smithi043.stdout:9/924: read d4/d28/d5f/d86/f96 [5240480,46372] 0 2024-09-16T23:49:09.800 INFO:tasks.workunit.client.1.smithi043.stdout:6/941: dread de/d1a/d80/fdd [4194304,4194304] 0 2024-09-16T23:49:09.800 INFO:tasks.workunit.client.1.smithi043.stdout:3/925: fsync d4/de/d40/df0/d88/f93 0 2024-09-16T23:49:09.801 INFO:tasks.workunit.client.1.smithi043.stdout:3/926: write d4/d30/f101 [393729,127394] 0 2024-09-16T23:49:09.802 INFO:tasks.workunit.client.1.smithi043.stdout:4/979: symlink d8/d25/d39/dd3/d100/d105/l132 0 2024-09-16T23:49:09.803 INFO:tasks.workunit.client.1.smithi043.stdout:9/925: creat d4/d21/d33/da2/d10a/f12f x:0 0 0 2024-09-16T23:49:09.803 INFO:tasks.workunit.client.1.smithi043.stdout:6/942: mkdir de/d23/d49/d145/d146 0 2024-09-16T23:49:09.805 INFO:tasks.workunit.client.1.smithi043.stdout:9/926: chown d4/d1c/d25/d2b/da1/df9/f5b 7 1 2024-09-16T23:49:09.805 INFO:tasks.workunit.client.1.smithi043.stdout:9/927: stat d4/f31 0 2024-09-16T23:49:09.808 INFO:tasks.workunit.client.1.smithi043.stdout:6/943: dread de/d1a/d80/fdd [4194304,4194304] 0 2024-09-16T23:49:09.808 INFO:tasks.workunit.client.1.smithi043.stdout:9/928: read d4/d1c/d25/d2b/d4c/da3/ff2 [5907221,103052] 0 2024-09-16T23:49:09.808 INFO:tasks.workunit.client.1.smithi043.stdout:4/980: creat d8/d15/d20/d2b/d23/f133 x:0 0 0 2024-09-16T23:49:09.810 INFO:tasks.workunit.client.0.smithi042.stdout:1/816: creat d0/d9/d90/d6b/d88/d63/f129 x:0 0 0 2024-09-16T23:49:09.811 INFO:tasks.workunit.client.0.smithi042.stdout:1/817: chown d0/d9/d90/d6b/d88/def/f106 63846 1 2024-09-16T23:49:09.811 INFO:tasks.workunit.client.1.smithi043.stdout:4/981: unlink d8/f68 0 2024-09-16T23:49:09.811 INFO:tasks.workunit.client.0.smithi042.stdout:1/818: stat d0/d17/d5e 0 2024-09-16T23:49:09.813 INFO:tasks.workunit.client.1.smithi043.stdout:6/944: mkdir de/d23/d34/d9b/d8b/d147 0 2024-09-16T23:49:09.814 INFO:tasks.workunit.client.0.smithi042.stdout:1/819: getdents d0/d9/d26/d8b/dec 0 2024-09-16T23:49:09.815 INFO:tasks.workunit.client.1.smithi043.stdout:4/982: creat d8/d15/d20/dfa/d73/f134 x:0 0 0 2024-09-16T23:49:09.816 INFO:tasks.workunit.client.1.smithi043.stdout:9/929: creat d4/d1c/d25/db1/d108/f130 x:0 0 0 2024-09-16T23:49:09.822 INFO:tasks.workunit.client.1.smithi043.stdout:3/927: dwrite d4/de/d40/f69 [0,4194304] 0 2024-09-16T23:49:09.826 INFO:tasks.workunit.client.1.smithi043.stdout:3/928: truncate d4/f1b 8701807 0 2024-09-16T23:49:09.828 INFO:tasks.workunit.client.1.smithi043.stdout:6/945: dwrite de/d138/dca/ffb [0,4194304] 0 2024-09-16T23:49:09.828 INFO:tasks.workunit.client.1.smithi043.stdout:3/929: chown d4/d30/f41 52563 1 2024-09-16T23:49:09.829 INFO:tasks.workunit.client.1.smithi043.stdout:4/983: link d8/d25/d39/d41/l77 d8/d15/d9b/dbf/l135 0 2024-09-16T23:49:09.831 INFO:tasks.workunit.client.1.smithi043.stdout:3/930: dread - d4/de/d5d/def/f113 zero size 2024-09-16T23:49:09.832 INFO:tasks.workunit.client.1.smithi043.stdout:9/930: dwrite d4/d1c/d25/d2b/d4c/f72 [0,4194304] 0 2024-09-16T23:49:09.833 INFO:tasks.workunit.client.1.smithi043.stdout:6/946: creat de/d23/d34/d9b/d8b/dbd/f148 x:0 0 0 2024-09-16T23:49:09.833 INFO:tasks.workunit.client.0.smithi042.stdout:1/820: dread d0/d17/d79/f7c [0,4194304] 0 2024-09-16T23:49:09.834 INFO:tasks.workunit.client.1.smithi043.stdout:4/984: mknod d8/d15/d20/d2b/dc8/c136 0 2024-09-16T23:49:09.835 INFO:tasks.workunit.client.1.smithi043.stdout:3/931: mknod d4/d30/c11f 0 2024-09-16T23:49:09.837 INFO:tasks.workunit.client.1.smithi043.stdout:9/931: symlink d4/d21/d33/da2/d10a/l131 0 2024-09-16T23:49:09.839 INFO:tasks.workunit.client.0.smithi042.stdout:1/821: creat d0/d9/d90/d6b/d88/d32/f12a x:0 0 0 2024-09-16T23:49:09.843 INFO:tasks.workunit.client.1.smithi043.stdout:9/932: rename d4/ld to d4/d21/d33/d10e/l132 0 2024-09-16T23:49:09.846 INFO:tasks.workunit.client.0.smithi042.stdout:1/822: write d0/d9/d26/f84 [95645,36613] 0 2024-09-16T23:49:09.852 INFO:tasks.workunit.client.1.smithi043.stdout:4/985: write d8/d25/d39/d41/d57/f129 [190390,73876] 0 2024-09-16T23:49:09.854 INFO:tasks.workunit.client.0.smithi042.stdout:1/823: dwrite d0/d9/d90/d6b/d88/d32/d2f/fa0 [0,4194304] 0 2024-09-16T23:49:09.855 INFO:tasks.workunit.client.1.smithi043.stdout:3/932: dwrite d4/de/d40/f80 [4194304,4194304] 0 2024-09-16T23:49:09.857 INFO:tasks.workunit.client.1.smithi043.stdout:3/933: chown d4/d30/c11f 1485364 1 2024-09-16T23:49:09.863 INFO:tasks.workunit.client.1.smithi043.stdout:9/933: dread d4/d1c/d25/d2b/fa5 [0,4194304] 0 2024-09-16T23:49:09.868 INFO:tasks.workunit.client.1.smithi043.stdout:6/947: getdents de/d23/d63/d7f 0 2024-09-16T23:49:09.872 INFO:tasks.workunit.client.1.smithi043.stdout:9/934: mknod d4/d1c/d25/d2b/da1/c133 0 2024-09-16T23:49:09.872 INFO:tasks.workunit.client.1.smithi043.stdout:6/948: fsync de/d23/d34/d66/fac 0 2024-09-16T23:49:09.878 INFO:tasks.workunit.client.1.smithi043.stdout:6/949: rmdir de/d1a/d11a 0 2024-09-16T23:49:09.881 INFO:tasks.workunit.client.1.smithi043.stdout:4/986: dread d8/d25/f4f [0,4194304] 0 2024-09-16T23:49:09.883 INFO:tasks.workunit.client.1.smithi043.stdout:4/987: fsync d8/d25/f5b 0 2024-09-16T23:49:09.887 INFO:tasks.workunit.client.1.smithi043.stdout:9/935: dread d4/d1c/d25/d2b/da1/df9/d45/f6a [0,4194304] 0 2024-09-16T23:49:09.887 INFO:tasks.workunit.client.1.smithi043.stdout:6/950: dwrite de/d23/d34/d9b/d8b/fbb [0,4194304] 0 2024-09-16T23:49:09.887 INFO:tasks.workunit.client.1.smithi043.stdout:4/988: chown d8/d25/d39/dd3/fa9 185 1 2024-09-16T23:49:09.889 INFO:tasks.workunit.client.1.smithi043.stdout:6/951: chown de/d23/d34/d9b/d5a/da3/cf9 248818 1 2024-09-16T23:49:09.891 INFO:tasks.workunit.client.1.smithi043.stdout:3/934: write d4/de/f58 [382400,38347] 0 2024-09-16T23:49:09.893 INFO:tasks.workunit.client.1.smithi043.stdout:9/936: rename d4/d1c/d25/db1/f123 to d4/d1c/d25/d2b/d4c/da3/f134 0 2024-09-16T23:49:09.897 INFO:tasks.workunit.client.1.smithi043.stdout:4/989: dwrite d8/d15/d20/dfa/d5e/fff [0,4194304] 0 2024-09-16T23:49:09.900 INFO:tasks.workunit.client.1.smithi043.stdout:9/937: truncate d4/d1c/d25/d2b/da1/df9/ffa 942438 0 2024-09-16T23:49:09.901 INFO:tasks.workunit.client.1.smithi043.stdout:6/952: dwrite de/d23/d34/d9b/d5a/dfe/f11c [8388608,4194304] 0 2024-09-16T23:49:09.903 INFO:tasks.workunit.client.1.smithi043.stdout:3/935: creat d4/de/d1f/f120 x:0 0 0 2024-09-16T23:49:09.905 INFO:tasks.workunit.client.1.smithi043.stdout:3/936: chown d4/de/d1f/d2c/l2d 463326001 1 2024-09-16T23:49:09.907 INFO:tasks.workunit.client.1.smithi043.stdout:4/990: rename d8/d15/d20/d2b/d22/f130 to d8/d15/d20/dfa/d73/f137 0 2024-09-16T23:49:09.913 INFO:tasks.workunit.client.1.smithi043.stdout:9/938: creat d4/d1c/d25/f135 x:0 0 0 2024-09-16T23:49:09.915 INFO:tasks.workunit.client.1.smithi043.stdout:6/953: getdents de/d23/d34/d66/d70 0 2024-09-16T23:49:09.915 INFO:tasks.workunit.client.1.smithi043.stdout:6/954: chown de/d1a 0 1 2024-09-16T23:49:09.917 INFO:tasks.workunit.client.1.smithi043.stdout:9/939: creat d4/d28/dcf/f136 x:0 0 0 2024-09-16T23:49:09.918 INFO:tasks.workunit.client.1.smithi043.stdout:4/991: link d8/d25/d39/d41/f60 d8/d25/d39/dd3/d100/d108/f138 0 2024-09-16T23:49:09.920 INFO:tasks.workunit.client.1.smithi043.stdout:6/955: rename de/d23/d34/d9b/d5a/da3/dfc to de/d23/d34/d66/d121/d149 0 2024-09-16T23:49:09.922 INFO:tasks.workunit.client.1.smithi043.stdout:9/940: rename d4/d28/d5f/d86 to d4/d21/d33/da2/d10a/d137 0 2024-09-16T23:49:09.923 INFO:tasks.workunit.client.1.smithi043.stdout:4/992: mkdir d8/d25/d39/dd3/d139 0 2024-09-16T23:49:09.924 INFO:tasks.workunit.client.1.smithi043.stdout:4/993: read d8/f5d [244412,13833] 0 2024-09-16T23:49:09.924 INFO:tasks.workunit.client.1.smithi043.stdout:9/941: creat d4/d1c/d6b/f138 x:0 0 0 2024-09-16T23:49:09.925 INFO:tasks.workunit.client.1.smithi043.stdout:3/937: write d4/de/d40/d4d/fbd [4366016,118324] 0 2024-09-16T23:49:09.927 INFO:tasks.workunit.client.1.smithi043.stdout:6/956: getdents de/d23/d63 0 2024-09-16T23:49:09.928 INFO:tasks.workunit.client.1.smithi043.stdout:9/942: mknod d4/d21/d33/da2/d10a/d137/c139 0 2024-09-16T23:49:09.930 INFO:tasks.workunit.client.1.smithi043.stdout:3/938: unlink d4/de/d40/df0/d88/db1/dbb/lbe 0 2024-09-16T23:49:09.931 INFO:tasks.workunit.client.1.smithi043.stdout:6/957: rename f2 to de/d23/d49/dc9/d124/f14a 0 2024-09-16T23:49:09.932 INFO:tasks.workunit.client.1.smithi043.stdout:9/943: mknod d4/d21/d2a/c13a 0 2024-09-16T23:49:09.934 INFO:tasks.workunit.client.1.smithi043.stdout:6/958: creat de/d23/d49/d145/d146/f14b x:0 0 0 2024-09-16T23:49:09.936 INFO:tasks.workunit.client.1.smithi043.stdout:9/944: rename d4/d1c/d25/d2b/fa5 to d4/d1c/d6b/f13b 0 2024-09-16T23:49:09.937 INFO:tasks.workunit.client.1.smithi043.stdout:4/994: dread d8/d25/d39/d41/f60 [0,4194304] 0 2024-09-16T23:49:09.940 INFO:tasks.workunit.client.1.smithi043.stdout:4/995: fsync d8/d25/d39/d58/fc0 0 2024-09-16T23:49:09.942 INFO:tasks.workunit.client.1.smithi043.stdout:9/945: creat d4/d1c/d25/d2b/d4c/dd2/de5/f13c x:0 0 0 2024-09-16T23:49:09.945 INFO:tasks.workunit.client.1.smithi043.stdout:8/865: sync 2024-09-16T23:49:09.946 INFO:tasks.workunit.client.1.smithi043.stdout:4/996: creat d8/d15/f13a x:0 0 0 2024-09-16T23:49:09.947 INFO:tasks.workunit.client.1.smithi043.stdout:3/939: dwrite d4/de/d1f/d2c/fc4 [0,4194304] 0 2024-09-16T23:49:09.949 INFO:tasks.workunit.client.1.smithi043.stdout:6/959: truncate de/d23/d34/d9b/d5a/f90 906420 0 2024-09-16T23:49:09.950 INFO:tasks.workunit.client.1.smithi043.stdout:3/940: truncate d4/de/d35/d10c/f10e 865561 0 2024-09-16T23:49:09.950 INFO:tasks.workunit.client.1.smithi043.stdout:6/960: fsync de/d1a/d28/f84 0 2024-09-16T23:49:09.952 INFO:tasks.workunit.client.1.smithi043.stdout:4/997: fdatasync d8/d15/d9b/d8b/ff8 0 2024-09-16T23:49:09.953 INFO:tasks.workunit.client.1.smithi043.stdout:3/941: stat d4/d30/ldf 0 2024-09-16T23:49:09.955 INFO:tasks.workunit.client.1.smithi043.stdout:3/942: fdatasync d4/de/d1f/d2c/db7/dc6/fee 0 2024-09-16T23:49:09.960 INFO:tasks.workunit.client.1.smithi043.stdout:3/943: mknod d4/de/d40/d4d/c121 0 2024-09-16T23:49:09.960 INFO:tasks.workunit.client.1.smithi043.stdout:9/946: write d4/d21/d35/d4b/f80 [872309,22415] 0 2024-09-16T23:49:09.962 INFO:tasks.workunit.client.0.smithi042.stdout:7/804: sync 2024-09-16T23:49:09.962 INFO:tasks.workunit.client.0.smithi042.stdout:0/749: sync 2024-09-16T23:49:09.964 INFO:tasks.workunit.client.0.smithi042.stdout:0/750: rmdir d0/df/d23/d33/d51 39 2024-09-16T23:49:09.964 INFO:tasks.workunit.client.1.smithi043.stdout:4/998: dwrite d8/d15/d20/d2b/dc8/fe3 [0,4194304] 0 2024-09-16T23:49:09.964 INFO:tasks.workunit.client.0.smithi042.stdout:0/751: chown d0/df/d23/c9b 14744 1 2024-09-16T23:49:09.965 INFO:tasks.workunit.client.0.smithi042.stdout:7/805: truncate d3/d1f/d49/d78/dbe/fda 232473 0 2024-09-16T23:49:09.967 INFO:tasks.workunit.client.1.smithi043.stdout:9/947: creat d4/d28/dcf/dd0/f13d x:0 0 0 2024-09-16T23:49:09.967 INFO:tasks.workunit.client.0.smithi042.stdout:0/752: rmdir d0/df/d13/dcb/dcc 39 2024-09-16T23:49:09.969 INFO:tasks.workunit.client.1.smithi043.stdout:9/948: stat d4/d1c/f20 0 2024-09-16T23:49:09.971 INFO:tasks.workunit.client.0.smithi042.stdout:7/806: creat d3/f11b x:0 0 0 2024-09-16T23:49:09.972 INFO:tasks.workunit.client.0.smithi042.stdout:0/753: mkdir d0/df/d13/dcb/dcc/df8 0 2024-09-16T23:49:09.974 INFO:tasks.workunit.client.0.smithi042.stdout:7/807: mkdir d3/d1f/d49/d76/d11c 0 2024-09-16T23:49:09.975 INFO:tasks.workunit.client.1.smithi043.stdout:8/866: dwrite de/d18/d1f/d2f/d39/d7b/fba [0,4194304] 0 2024-09-16T23:49:09.979 INFO:tasks.workunit.client.1.smithi043.stdout:6/961: dwrite de/d23/d34/d9b/d5a/f90 [0,4194304] 0 2024-09-16T23:49:09.985 INFO:tasks.workunit.client.1.smithi043.stdout:4/999: dwrite d8/d15/d20/d2b/d54/f91 [0,4194304] 0 2024-09-16T23:49:09.986 INFO:tasks.workunit.client.1.smithi043.stdout:6/962: symlink de/d23/dc3/l14c 0 2024-09-16T23:49:09.987 INFO:tasks.workunit.client.1.smithi043.stdout:3/944: dwrite d4/de/d40/d4d/fda [0,4194304] 0 2024-09-16T23:49:09.989 INFO:tasks.workunit.client.0.smithi042.stdout:0/754: truncate d0/df/d23/d33/d56/da9/f92 458649 0 2024-09-16T23:49:09.994 INFO:tasks.workunit.client.1.smithi043.stdout:8/867: mknod de/d18/d1f/d2f/d39/d7b/d83/dbb/dce/de4/c103 0 2024-09-16T23:49:09.995 INFO:tasks.workunit.client.0.smithi042.stdout:0/755: creat d0/df/d13/ff9 x:0 0 0 2024-09-16T23:49:09.997 INFO:tasks.workunit.client.1.smithi043.stdout:3/945: creat d4/de/dea/f122 x:0 0 0 2024-09-16T23:49:09.997 INFO:tasks.workunit.client.1.smithi043.stdout:6/963: dwrite fd [0,4194304] 0 2024-09-16T23:49:09.998 INFO:tasks.workunit.client.0.smithi042.stdout:7/808: dwrite d3/d98/da0/d41/f8b [4194304,4194304] 0 2024-09-16T23:49:09.999 INFO:tasks.workunit.client.1.smithi043.stdout:3/946: fsync d4/de/d1f/d34/fa8 0 2024-09-16T23:49:10.000 INFO:tasks.workunit.client.1.smithi043.stdout:8/868: unlink de/d18/d1f/d2f/d39/d7b/c87 0 2024-09-16T23:49:10.001 INFO:tasks.workunit.client.0.smithi042.stdout:0/756: creat d0/df/d13/ffa x:0 0 0 2024-09-16T23:49:10.004 INFO:tasks.workunit.client.0.smithi042.stdout:0/757: stat d0/df/d13/d2d/d35/c4d 0 2024-09-16T23:49:10.004 INFO:tasks.workunit.client.0.smithi042.stdout:7/809: readlink d3/d98/da0/d24/l4c 0 2024-09-16T23:49:10.004 INFO:tasks.workunit.client.1.smithi043.stdout:9/949: truncate d4/d1c/d25/d2b/da1/df9/f5b 2547824 0 2024-09-16T23:49:10.005 INFO:tasks.workunit.client.0.smithi042.stdout:0/758: chown d0/d15/de4 807 1 2024-09-16T23:49:10.005 INFO:tasks.workunit.client.0.smithi042.stdout:0/759: fdatasync d0/df/d13/d49/fd0 0 2024-09-16T23:49:10.006 INFO:tasks.workunit.client.1.smithi043.stdout:6/964: dwrite de/d138/fc8 [0,4194304] 0 2024-09-16T23:49:10.011 INFO:tasks.workunit.client.0.smithi042.stdout:0/760: creat d0/df/df7/ffb x:0 0 0 2024-09-16T23:49:10.012 INFO:tasks.workunit.client.1.smithi043.stdout:9/950: creat d4/d1c/d25/d2b/f13e x:0 0 0 2024-09-16T23:49:10.012 INFO:tasks.workunit.client.1.smithi043.stdout:2/833: sync 2024-09-16T23:49:10.012 INFO:tasks.workunit.client.1.smithi043.stdout:0/993: sync 2024-09-16T23:49:10.013 INFO:tasks.workunit.client.1.smithi043.stdout:7/899: sync 2024-09-16T23:49:10.013 INFO:tasks.workunit.client.0.smithi042.stdout:0/761: dread - d0/df/df7/ffb zero size 2024-09-16T23:49:10.014 INFO:tasks.workunit.client.1.smithi043.stdout:6/965: rename de/d23/d34/d9b/d8b/dbd/c136 to de/d23/d34/d9b/d8b/de9/c14d 0 2024-09-16T23:49:10.016 INFO:tasks.workunit.client.1.smithi043.stdout:7/900: readlink d4/d27/d41/d1b/l5c 0 2024-09-16T23:49:10.016 INFO:tasks.workunit.client.1.smithi043.stdout:3/947: creat d4/d30/f123 x:0 0 0 2024-09-16T23:49:10.017 INFO:tasks.workunit.client.0.smithi042.stdout:7/810: dread d3/d1f/d49/d78/dbe/fca [0,4194304] 0 2024-09-16T23:49:10.020 INFO:tasks.workunit.client.0.smithi042.stdout:0/762: symlink d0/df/d23/d33/d56/dc5/lfc 0 2024-09-16T23:49:10.021 INFO:tasks.workunit.client.0.smithi042.stdout:7/811: rmdir d3/d1f 39 2024-09-16T23:49:10.024 INFO:tasks.workunit.client.0.smithi042.stdout:0/763: symlink d0/df/d13/d2d/d35/d9f/lfd 0 2024-09-16T23:49:10.025 INFO:tasks.workunit.client.0.smithi042.stdout:7/812: mknod d3/d1f/d49/d76/d101/d64/d92/c11d 0 2024-09-16T23:49:10.025 INFO:tasks.workunit.client.0.smithi042.stdout:0/764: write d0/df/d13/d49/fd0 [556643,148] 0 2024-09-16T23:49:10.026 INFO:tasks.workunit.client.1.smithi043.stdout:3/948: mknod d4/de/c124 0 2024-09-16T23:49:10.026 INFO:tasks.workunit.client.1.smithi043.stdout:8/869: write de/fd0 [3122832,63526] 0 2024-09-16T23:49:10.027 INFO:tasks.workunit.client.1.smithi043.stdout:6/966: dwrite de/d1a/d28/ff1 [0,4194304] 0 2024-09-16T23:49:10.028 INFO:tasks.workunit.client.0.smithi042.stdout:7/813: symlink d3/l11e 0 2024-09-16T23:49:10.030 INFO:tasks.workunit.client.1.smithi043.stdout:7/901: rename d4/db/d11/d15/d29/c5a to d4/db/da6/c129 0 2024-09-16T23:49:10.030 INFO:tasks.workunit.client.1.smithi043.stdout:3/949: chown d4/de/d1f/dfc 0 1 2024-09-16T23:49:10.031 INFO:tasks.workunit.client.1.smithi043.stdout:2/834: truncate d8/d11/d16/fa6 613459 0 2024-09-16T23:49:10.035 INFO:tasks.workunit.client.1.smithi043.stdout:2/835: chown d8/d11/d16 24406933 1 2024-09-16T23:49:10.036 INFO:tasks.workunit.client.1.smithi043.stdout:0/994: dwrite d0/d2/df/d1a/d46/d39/daa/db7/f108 [0,4194304] 0 2024-09-16T23:49:10.037 INFO:tasks.workunit.client.1.smithi043.stdout:8/870: creat de/d18/d1f/d2f/d39/d4d/f104 x:0 0 0 2024-09-16T23:49:10.039 INFO:tasks.workunit.client.1.smithi043.stdout:7/902: mknod d4/db/d11/d15/dda/d121/c12a 0 2024-09-16T23:49:10.040 INFO:tasks.workunit.client.1.smithi043.stdout:3/950: dread d4/d30/f101 [0,4194304] 0 2024-09-16T23:49:10.041 INFO:tasks.workunit.client.0.smithi042.stdout:7/814: dread d3/d1f/d49/d76/d19/ddc/dcd/fcb [0,4194304] 0 2024-09-16T23:49:10.042 INFO:tasks.workunit.client.0.smithi042.stdout:7/815: dread - d3/d1f/d49/d78/f102 zero size 2024-09-16T23:49:10.043 INFO:tasks.workunit.client.1.smithi043.stdout:9/951: dwrite d4/d28/d49/f73 [0,4194304] 0 2024-09-16T23:49:10.043 INFO:tasks.workunit.client.1.smithi043.stdout:2/836: mkdir d8/d5e/d83/d3a/d77/da9/d119 0 2024-09-16T23:49:10.043 INFO:tasks.workunit.client.1.smithi043.stdout:7/903: fsync d4/d27/d41/f8d 0 2024-09-16T23:49:10.044 INFO:tasks.workunit.client.0.smithi042.stdout:2/713: sync 2024-09-16T23:49:10.044 INFO:tasks.workunit.client.0.smithi042.stdout:8/621: sync 2024-09-16T23:49:10.045 INFO:tasks.workunit.client.1.smithi043.stdout:6/967: dread de/d23/d49/dc9/d141/f117 [0,4194304] 0 2024-09-16T23:49:10.053 INFO:tasks.workunit.client.0.smithi042.stdout:0/765: dwrite d0/df/d23/d3e/f86 [0,4194304] 0 2024-09-16T23:49:10.053 INFO:tasks.workunit.client.0.smithi042.stdout:7/816: write d3/d98/f8c [3181388,114655] 0 2024-09-16T23:49:10.054 INFO:tasks.workunit.client.0.smithi042.stdout:7/817: fdatasync d3/d3f/f90 0 2024-09-16T23:49:10.055 INFO:tasks.workunit.client.0.smithi042.stdout:8/622: creat da/d19/d1d/d30/d3f/d6d/fc1 x:0 0 0 2024-09-16T23:49:10.056 INFO:tasks.workunit.client.1.smithi043.stdout:3/951: rmdir d4 39 2024-09-16T23:49:10.058 INFO:tasks.workunit.client.1.smithi043.stdout:9/952: rmdir d4/d21/d33 39 2024-09-16T23:49:10.061 INFO:tasks.workunit.client.1.smithi043.stdout:9/953: write d4/d1c/d25/d2b/d4c/dd2/f118 [524259,47333] 0 2024-09-16T23:49:10.061 INFO:tasks.workunit.client.0.smithi042.stdout:8/623: mkdir da/d19/d1d/d30/d3f/dc2 0 2024-09-16T23:49:10.063 INFO:tasks.workunit.client.1.smithi043.stdout:2/837: dread d8/d11/d16/f28 [0,4194304] 0 2024-09-16T23:49:10.066 INFO:tasks.workunit.client.0.smithi042.stdout:2/714: truncate d0/f3a 5982928 0 2024-09-16T23:49:10.067 INFO:tasks.workunit.client.1.smithi043.stdout:8/871: mkdir de/d18/d1f/d2f/d39/d7b/dd8/d105 0 2024-09-16T23:49:10.067 INFO:tasks.workunit.client.1.smithi043.stdout:6/968: dread - de/d1a/d80/fd0 zero size 2024-09-16T23:49:10.068 INFO:tasks.workunit.client.0.smithi042.stdout:8/624: mknod da/d19/d1d/d30/d49/cc3 0 2024-09-16T23:49:10.069 INFO:tasks.workunit.client.1.smithi043.stdout:0/995: creat d0/d2/df/d1a/d46/d39/d83/d138/f153 x:0 0 0 2024-09-16T23:49:10.069 INFO:tasks.workunit.client.0.smithi042.stdout:2/715: write d0/da/d1b/d3b/d44/fb3 [2066239,91440] 0 2024-09-16T23:49:10.072 INFO:tasks.workunit.client.1.smithi043.stdout:9/954: creat d4/d1c/d25/d2b/d4c/da3/dca/dfe/f13f x:0 0 0 2024-09-16T23:49:10.074 INFO:tasks.workunit.client.0.smithi042.stdout:0/766: dwrite d0/d15/f3c [0,4194304] 0 2024-09-16T23:49:10.074 INFO:tasks.workunit.client.1.smithi043.stdout:6/969: dread de/f67 [0,4194304] 0 2024-09-16T23:49:10.074 INFO:tasks.workunit.client.1.smithi043.stdout:9/955: write d4/d21/d35/d4b/f80 [1593374,58420] 0 2024-09-16T23:49:10.074 INFO:tasks.workunit.client.1.smithi043.stdout:7/904: write d4/d27/d41/d1b/d95/fb4 [1479077,9609] 0 2024-09-16T23:49:10.076 INFO:tasks.workunit.client.0.smithi042.stdout:0/767: chown d0/df/d23/c47 0 1 2024-09-16T23:49:10.077 INFO:tasks.workunit.client.1.smithi043.stdout:6/970: fdatasync de/d1a/d39/f78 0 2024-09-16T23:49:10.077 INFO:tasks.workunit.client.1.smithi043.stdout:0/996: truncate d0/d2/df/da7/fe5 860146 0 2024-09-16T23:49:10.077 INFO:tasks.workunit.client.1.smithi043.stdout:9/956: dread - d4/d1c/fe3 zero size 2024-09-16T23:49:10.080 INFO:tasks.workunit.client.1.smithi043.stdout:2/838: read d8/d5e/d83/fa7 [29520,48828] 0 2024-09-16T23:49:10.080 INFO:tasks.workunit.client.1.smithi043.stdout:6/971: fdatasync de/d1a/d80/f10f 0 2024-09-16T23:49:10.080 INFO:tasks.workunit.client.1.smithi043.stdout:3/952: fdatasync d4/de/d1f/f79 0 2024-09-16T23:49:10.081 INFO:tasks.workunit.client.0.smithi042.stdout:2/716: write d0/da/d1b/d3b/f56 [583150,39579] 0 2024-09-16T23:49:10.082 INFO:tasks.workunit.client.1.smithi043.stdout:7/905: dread - d4/db/d11/f111 zero size 2024-09-16T23:49:10.082 INFO:tasks.workunit.client.0.smithi042.stdout:2/717: fsync d0/da/d88/fdf 0 2024-09-16T23:49:10.083 INFO:tasks.workunit.client.1.smithi043.stdout:2/839: chown d8/d4b/d4f/d7c/cd2 432978 1 2024-09-16T23:49:10.085 INFO:tasks.workunit.client.1.smithi043.stdout:2/840: stat d8/d35/dc0 0 2024-09-16T23:49:10.086 INFO:tasks.workunit.client.0.smithi042.stdout:2/718: unlink d0/da/d17/f28 0 2024-09-16T23:49:10.087 INFO:tasks.workunit.client.0.smithi042.stdout:8/625: dread da/d19/d2c/f36 [0,4194304] 0 2024-09-16T23:49:10.089 INFO:tasks.workunit.client.1.smithi043.stdout:2/841: read - d8/d5e/d83/d3a/d77/f108 zero size 2024-09-16T23:49:10.090 INFO:tasks.workunit.client.0.smithi042.stdout:0/768: dwrite d0/df/d23/d33/d51/d79/fb0 [0,4194304] 0 2024-09-16T23:49:10.091 INFO:tasks.workunit.client.1.smithi043.stdout:8/872: rename de/d18/d1f/d2f to de/d18/d27/d41/d5a/d106 0 2024-09-16T23:49:10.091 INFO:tasks.workunit.client.1.smithi043.stdout:7/906: dwrite d4/db/d11/d15/dbe/f105 [0,4194304] 0 2024-09-16T23:49:10.093 INFO:tasks.workunit.client.1.smithi043.stdout:3/953: rmdir d4/de/d1f/d2c/db7/dc6 39 2024-09-16T23:49:10.093 INFO:tasks.workunit.client.1.smithi043.stdout:6/972: dwrite de/d23/d34/d9b/d50/fa8 [0,4194304] 0 2024-09-16T23:49:10.097 INFO:tasks.workunit.client.0.smithi042.stdout:8/626: mkdir da/d19/d1d/d30/d3f/dc2/dc4 0 2024-09-16T23:49:10.097 INFO:tasks.workunit.client.1.smithi043.stdout:2/842: mknod d8/d1f/c11a 0 2024-09-16T23:49:10.098 INFO:tasks.workunit.client.1.smithi043.stdout:2/843: fsync d8/d35/f37 0 2024-09-16T23:49:10.100 INFO:tasks.workunit.client.1.smithi043.stdout:7/907: creat d4/db/d11/d21/d31/d65/f12b x:0 0 0 2024-09-16T23:49:10.101 INFO:tasks.workunit.client.1.smithi043.stdout:7/908: readlink d4/l69 0 2024-09-16T23:49:10.104 INFO:tasks.workunit.client.1.smithi043.stdout:6/973: mkdir de/d23/d63/d7f/d14e 0 2024-09-16T23:49:10.105 INFO:tasks.workunit.client.0.smithi042.stdout:8/627: creat da/d19/d1d/d30/d3f/dc2/dc4/fc5 x:0 0 0 2024-09-16T23:49:10.107 INFO:tasks.workunit.client.1.smithi043.stdout:2/844: unlink d8/d1d/d20/d23/d38/f69 0 2024-09-16T23:49:10.115 INFO:tasks.workunit.client.1.smithi043.stdout:7/909: symlink d4/d27/l12c 0 2024-09-16T23:49:10.117 INFO:tasks.workunit.client.0.smithi042.stdout:8/628: link da/d4b/f72 da/d19/d1d/d5a/fc6 0 2024-09-16T23:49:10.118 INFO:tasks.workunit.client.1.smithi043.stdout:8/873: dread de/d18/d1f/f2d [0,4194304] 0 2024-09-16T23:49:10.118 INFO:tasks.workunit.client.1.smithi043.stdout:3/954: dwrite d4/de/d40/f45 [4194304,4194304] 0 2024-09-16T23:49:10.120 INFO:tasks.workunit.client.1.smithi043.stdout:7/910: creat d4/db/d11/d15/d29/da7/f12d x:0 0 0 2024-09-16T23:49:10.121 INFO:tasks.workunit.client.0.smithi042.stdout:8/629: mkdir da/d4b/d74/dc7 0 2024-09-16T23:49:10.123 INFO:tasks.workunit.client.1.smithi043.stdout:8/874: read de/d18/d27/d41/d5a/d106/d39/d4d/f5d [533378,78038] 0 2024-09-16T23:49:10.123 INFO:tasks.workunit.client.1.smithi043.stdout:2/845: getdents d8/d1f 0 2024-09-16T23:49:10.126 INFO:tasks.workunit.client.1.smithi043.stdout:6/974: dwrite de/d23/d34/d66/d70/f76 [0,4194304] 0 2024-09-16T23:49:10.126 INFO:tasks.workunit.client.1.smithi043.stdout:2/846: stat d8/d1d/d20/d23 0 2024-09-16T23:49:10.128 INFO:tasks.workunit.client.1.smithi043.stdout:2/847: dread - d8/d5e/d83/d3a/d77/dea/fee zero size 2024-09-16T23:49:10.130 INFO:tasks.workunit.client.1.smithi043.stdout:6/975: write de/d23/d34/d9b/d8b/feb [3946650,68186] 0 2024-09-16T23:49:10.132 INFO:tasks.workunit.client.1.smithi043.stdout:8/875: symlink de/d18/d27/d41/d5a/d106/d39/l107 0 2024-09-16T23:49:10.133 INFO:tasks.workunit.client.1.smithi043.stdout:2/848: creat d8/d1d/d20/d23/d38/da2/f11b x:0 0 0 2024-09-16T23:49:10.134 INFO:tasks.workunit.client.1.smithi043.stdout:7/911: link d4/db/d11/d15/fd9 d4/db/d11/d117/d85/f12e 0 2024-09-16T23:49:10.134 INFO:tasks.workunit.client.1.smithi043.stdout:6/976: symlink de/d23/d34/d66/d9d/l14f 0 2024-09-16T23:49:10.135 INFO:tasks.workunit.client.0.smithi042.stdout:0/769: creat d0/df/d23/d3e/d8f/ffe x:0 0 0 2024-09-16T23:49:10.136 INFO:tasks.workunit.client.1.smithi043.stdout:6/977: chown de/d23/d49/c134 36778 1 2024-09-16T23:49:10.137 INFO:tasks.workunit.client.0.smithi042.stdout:0/770: write d0/df/d23/d33/d56/da9/fdd [639976,112802] 0 2024-09-16T23:49:10.138 INFO:tasks.workunit.client.1.smithi043.stdout:2/849: rmdir d8/d4b/d5f 39 2024-09-16T23:49:10.139 INFO:tasks.workunit.client.1.smithi043.stdout:3/955: rename d4/de/l61 to d4/de/d40/df0/d88/db1/l125 0 2024-09-16T23:49:10.139 INFO:tasks.workunit.client.0.smithi042.stdout:2/719: rename d0/da/d88/dae/f75 to d0/d30/d43/ff1 0 2024-09-16T23:49:10.140 INFO:tasks.workunit.client.1.smithi043.stdout:8/876: unlink de/d18/d27/d41/d5a/d106/d39/d7b/d83/da5/le8 0 2024-09-16T23:49:10.141 INFO:tasks.workunit.client.1.smithi043.stdout:7/912: creat d4/db/d11/d117/d85/d8b/d118/f12f x:0 0 0 2024-09-16T23:49:10.142 INFO:tasks.workunit.client.0.smithi042.stdout:2/720: mkdir d0/da/d1b/d3b/d44/df2 0 2024-09-16T23:49:10.142 INFO:tasks.workunit.client.1.smithi043.stdout:7/913: read - d4/db/d11/d21/d31/dcb/ff0 zero size 2024-09-16T23:49:10.143 INFO:tasks.workunit.client.1.smithi043.stdout:6/978: creat de/d23/d49/dc9/d124/f150 x:0 0 0 2024-09-16T23:49:10.143 INFO:tasks.workunit.client.0.smithi042.stdout:2/721: read d0/d30/d43/fac [1478296,123176] 0 2024-09-16T23:49:10.144 INFO:tasks.workunit.client.1.smithi043.stdout:6/979: dread - de/d1a/d28/d13d/f13f zero size 2024-09-16T23:49:10.146 INFO:tasks.workunit.client.1.smithi043.stdout:6/980: read de/d23/d49/dc9/d141/f117 [540608,9673] 0 2024-09-16T23:49:10.147 INFO:tasks.workunit.client.0.smithi042.stdout:8/630: write da/d19/d1d/d5a/d99/f9a [3659961,2034] 0 2024-09-16T23:49:10.149 INFO:tasks.workunit.client.1.smithi043.stdout:8/877: creat de/d18/d27/f108 x:0 0 0 2024-09-16T23:49:10.150 INFO:tasks.workunit.client.0.smithi042.stdout:8/631: stat da/d19/d1d/c6e 0 2024-09-16T23:49:10.150 INFO:tasks.workunit.client.1.smithi043.stdout:7/914: dwrite d4/f11a [0,4194304] 0 2024-09-16T23:49:10.152 INFO:tasks.workunit.client.1.smithi043.stdout:2/850: mknod d8/d4b/d5f/d63/db6/c11c 0 2024-09-16T23:49:10.154 INFO:tasks.workunit.client.1.smithi043.stdout:6/981: mkdir de/d23/d34/d66/d9d/df3/d151 0 2024-09-16T23:49:10.154 INFO:tasks.workunit.client.0.smithi042.stdout:8/632: chown da/d19/d1d/f3c 390 1 2024-09-16T23:49:10.158 INFO:tasks.workunit.client.1.smithi043.stdout:2/851: creat d8/dec/f11d x:0 0 0 2024-09-16T23:49:10.160 INFO:tasks.workunit.client.0.smithi042.stdout:2/722: dwrite d0/d30/d43/f62 [0,4194304] 0 2024-09-16T23:49:10.160 INFO:tasks.workunit.client.1.smithi043.stdout:7/915: dwrite d4/db/f104 [0,4194304] 0 2024-09-16T23:49:10.160 INFO:tasks.workunit.client.0.smithi042.stdout:8/633: chown da/d4b/f65 235 1 2024-09-16T23:49:10.164 INFO:tasks.workunit.client.1.smithi043.stdout:6/982: rename de/d23/d34/d9b/d5a/dfe/d10d/f13e to de/d23/d34/d66/f152 0 2024-09-16T23:49:10.169 INFO:tasks.workunit.client.1.smithi043.stdout:7/916: creat d4/db/d11/d15/d29/d7a/dcd/f130 x:0 0 0 2024-09-16T23:49:10.170 INFO:tasks.workunit.client.1.smithi043.stdout:2/852: fdatasync d8/f1a 0 2024-09-16T23:49:10.172 INFO:tasks.workunit.client.0.smithi042.stdout:8/634: unlink da/d4b/f72 0 2024-09-16T23:49:10.174 INFO:tasks.workunit.client.1.smithi043.stdout:8/878: getdents de/d18/d1f/d25/d65 0 2024-09-16T23:49:10.176 INFO:tasks.workunit.client.1.smithi043.stdout:6/983: rename de/d23/d34/d9b/d5a/l8c to de/d23/d63/l153 0 2024-09-16T23:49:10.182 INFO:tasks.workunit.client.1.smithi043.stdout:2/853: write d8/d1d/d20/d23/d38/da2/fe9 [541127,129814] 0 2024-09-16T23:49:10.182 INFO:tasks.workunit.client.1.smithi043.stdout:8/879: dwrite de/d18/d27/d41/d5a/d106/d39/d7b/d83/fc0 [0,4194304] 0 2024-09-16T23:49:10.185 INFO:tasks.workunit.client.1.smithi043.stdout:7/917: symlink d4/db/d11/d117/d85/d8b/df7/l131 0 2024-09-16T23:49:10.192 INFO:tasks.workunit.client.0.smithi042.stdout:2/723: dwrite d0/da/d17/f1c [4194304,4194304] 0 2024-09-16T23:49:10.195 INFO:tasks.workunit.client.1.smithi043.stdout:2/854: truncate d8/d4b/d4f/d56/f95 1784084 0 2024-09-16T23:49:10.196 INFO:tasks.workunit.client.0.smithi042.stdout:8/635: dwrite da/d19/d1d/d5a/fc6 [0,4194304] 0 2024-09-16T23:49:10.202 INFO:tasks.workunit.client.1.smithi043.stdout:7/918: unlink d4/db/d43/c46 0 2024-09-16T23:49:10.202 INFO:tasks.workunit.client.1.smithi043.stdout:8/880: read de/d18/d27/d41/d5a/d37/f99 [3205792,4415] 0 2024-09-16T23:49:10.202 INFO:tasks.workunit.client.1.smithi043.stdout:6/984: rmdir de/d23/d63/d7f/d14e 0 2024-09-16T23:49:10.203 INFO:tasks.workunit.client.0.smithi042.stdout:2/724: dwrite d0/da/d1b/d3b/d44/f90 [0,4194304] 0 2024-09-16T23:49:10.208 INFO:tasks.workunit.client.0.smithi042.stdout:2/725: mkdir d0/da/d1b/d3b/d44/d54/df3 0 2024-09-16T23:49:10.209 INFO:tasks.workunit.client.0.smithi042.stdout:2/726: read - d0/da/d17/d3e/d86/fd7 zero size 2024-09-16T23:49:10.210 INFO:tasks.workunit.client.0.smithi042.stdout:2/727: readlink d0/da/d1b/l1d 0 2024-09-16T23:49:10.211 INFO:tasks.workunit.client.0.smithi042.stdout:6/787: sync 2024-09-16T23:49:10.211 INFO:tasks.workunit.client.0.smithi042.stdout:9/604: sync 2024-09-16T23:49:10.211 INFO:tasks.workunit.client.0.smithi042.stdout:1/824: sync 2024-09-16T23:49:10.211 INFO:tasks.workunit.client.0.smithi042.stdout:5/887: sync 2024-09-16T23:49:10.212 INFO:tasks.workunit.client.0.smithi042.stdout:5/888: chown d9/d2c 7616 1 2024-09-16T23:49:10.213 INFO:tasks.workunit.client.1.smithi043.stdout:7/919: rename d4/d27/d41/d1b/d95/fec to d4/db/f132 0 2024-09-16T23:49:10.213 INFO:tasks.workunit.client.0.smithi042.stdout:2/728: mkdir d0/d26/df4 0 2024-09-16T23:49:10.214 INFO:tasks.workunit.client.0.smithi042.stdout:5/889: chown d9/db/d19/d21/d85/da5/f94 671 1 2024-09-16T23:49:10.216 INFO:tasks.workunit.client.0.smithi042.stdout:8/636: write da/d19/d1d/f3c [361859,115903] 0 2024-09-16T23:49:10.216 INFO:tasks.workunit.client.0.smithi042.stdout:1/825: creat d0/d9/d4e/f12b x:0 0 0 2024-09-16T23:49:10.218 INFO:tasks.workunit.client.1.smithi043.stdout:7/920: mkdir d4/d27/d133 0 2024-09-16T23:49:10.218 INFO:tasks.workunit.client.0.smithi042.stdout:8/637: fsync da/d4b/fa9 0 2024-09-16T23:49:10.219 INFO:tasks.workunit.client.0.smithi042.stdout:1/826: read d0/d9/d4e/fa4 [1673131,48153] 0 2024-09-16T23:49:10.221 INFO:tasks.workunit.client.0.smithi042.stdout:5/890: truncate d9/d4a/d89/f9f 2277100 0 2024-09-16T23:49:10.222 INFO:tasks.workunit.client.1.smithi043.stdout:7/921: dread - d4/db/d11/d21/d31/d65/f12b zero size 2024-09-16T23:49:10.222 INFO:tasks.workunit.client.0.smithi042.stdout:2/729: fdatasync d0/f95 0 2024-09-16T23:49:10.223 INFO:tasks.workunit.client.0.smithi042.stdout:5/891: dread - d9/db/d1b/d31/d46/d107/f10f zero size 2024-09-16T23:49:10.225 INFO:tasks.workunit.client.0.smithi042.stdout:8/638: getdents da/d19/d1d/d30/d49/d58 0 2024-09-16T23:49:10.226 INFO:tasks.workunit.client.0.smithi042.stdout:1/827: fsync d0/d9/d90/d66/f7f 0 2024-09-16T23:49:10.226 INFO:tasks.workunit.client.1.smithi043.stdout:6/985: dwrite de/d1a/d39/fa0 [0,4194304] 0 2024-09-16T23:49:10.226 INFO:tasks.workunit.client.0.smithi042.stdout:6/788: dread d8/d70/d81/dd/f19 [0,4194304] 0 2024-09-16T23:49:10.227 INFO:tasks.workunit.client.1.smithi043.stdout:6/986: stat de/d23 0 2024-09-16T23:49:10.228 INFO:tasks.workunit.client.1.smithi043.stdout:8/881: dwrite de/d70/d3d/fa3 [0,4194304] 0 2024-09-16T23:49:10.229 INFO:tasks.workunit.client.0.smithi042.stdout:5/892: mkdir d9/db/d1b/d31/d46/d58/d9d/d134 0 2024-09-16T23:49:10.229 INFO:tasks.workunit.client.0.smithi042.stdout:8/639: mkdir da/d19/d2c/dc8 0 2024-09-16T23:49:10.231 INFO:tasks.workunit.client.1.smithi043.stdout:8/882: dread - de/d18/d27/d41/d5a/d106/d39/d7b/d83/fe5 zero size 2024-09-16T23:49:10.232 INFO:tasks.workunit.client.0.smithi042.stdout:2/730: dwrite d0/d98/fe9 [0,4194304] 0 2024-09-16T23:49:10.233 INFO:tasks.workunit.client.0.smithi042.stdout:2/731: chown d0/d5d 1957631 1 2024-09-16T23:49:10.233 INFO:tasks.workunit.client.1.smithi043.stdout:6/987: symlink de/d23/d34/d66/d9d/l154 0 2024-09-16T23:49:10.237 INFO:tasks.workunit.client.0.smithi042.stdout:6/789: mkdir d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/dfb 0 2024-09-16T23:49:10.240 INFO:tasks.workunit.client.1.smithi043.stdout:8/883: fsync de/d18/d27/f94 0 2024-09-16T23:49:10.240 INFO:tasks.workunit.client.1.smithi043.stdout:7/922: link d4/l69 d4/db/d11/d21/d31/dcb/df8/l134 0 2024-09-16T23:49:10.241 INFO:tasks.workunit.client.0.smithi042.stdout:5/893: readlink d9/db/d19/d21/d85/dec/d112/d61/d43/lbd 0 2024-09-16T23:49:10.243 INFO:tasks.workunit.client.0.smithi042.stdout:2/732: stat d0/da/d17/f37 0 2024-09-16T23:49:10.246 INFO:tasks.workunit.client.0.smithi042.stdout:6/790: rename d8/cef to d8/d70/d81/dd/d1b/d67/d97/cfc 0 2024-09-16T23:49:10.247 INFO:tasks.workunit.client.1.smithi043.stdout:6/988: dwrite de/d23/d34/d9b/d5a/f90 [0,4194304] 0 2024-09-16T23:49:10.249 INFO:tasks.workunit.client.0.smithi042.stdout:5/894: dwrite d9/d2c/d121/f12e [0,4194304] 0 2024-09-16T23:49:10.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/605: dwrite d4/d12/d16/d68/f72 [0,4194304] 0 2024-09-16T23:49:10.253 INFO:tasks.workunit.client.1.smithi043.stdout:7/923: dwrite d4/db/d11/d15/d29/d7a/dcd/ff1 [0,4194304] 0 2024-09-16T23:49:10.256 INFO:tasks.workunit.client.0.smithi042.stdout:5/895: mknod d9/db/d19/c135 0 2024-09-16T23:49:10.257 INFO:tasks.workunit.client.0.smithi042.stdout:8/640: write da/d19/f39 [1391250,98711] 0 2024-09-16T23:49:10.260 INFO:tasks.workunit.client.0.smithi042.stdout:6/791: truncate d8/d70/d81/dd/d2c/d57/f5d 2448558 0 2024-09-16T23:49:10.261 INFO:tasks.workunit.client.1.smithi043.stdout:7/924: rename d4/db/d11/d15/d29/fbd to d4/d27/d8a/d93/f135 0 2024-09-16T23:49:10.266 INFO:tasks.workunit.client.1.smithi043.stdout:8/884: write de/d70/fa0 [1125846,83684] 0 2024-09-16T23:49:10.266 INFO:tasks.workunit.client.1.smithi043.stdout:6/989: write de/d1a/d39/ff2 [882587,47789] 0 2024-09-16T23:49:10.271 INFO:tasks.workunit.client.0.smithi042.stdout:1/828: dwrite d0/d17/dd4/f127 [0,4194304] 0 2024-09-16T23:49:10.272 INFO:tasks.workunit.client.0.smithi042.stdout:6/792: chown d8/d70/d81/dd/d1b/d34/d39/d5e/f7c 103 1 2024-09-16T23:49:10.273 INFO:tasks.workunit.client.1.smithi043.stdout:7/925: creat d4/db/d11/d21/f136 x:0 0 0 2024-09-16T23:49:10.276 INFO:tasks.workunit.client.0.smithi042.stdout:9/606: dread d4/d12/d2d/d37/d59/f96 [0,4194304] 0 2024-09-16T23:49:10.279 INFO:tasks.workunit.client.1.smithi043.stdout:6/990: getdents de/d23/d34/d66/d121 0 2024-09-16T23:49:10.282 INFO:tasks.workunit.client.0.smithi042.stdout:8/641: dwrite da/d19/d1d/f89 [0,4194304] 0 2024-09-16T23:49:10.283 INFO:tasks.workunit.client.0.smithi042.stdout:6/793: dread d8/d70/d81/dd/d2c/d57/f60 [0,4194304] 0 2024-09-16T23:49:10.283 INFO:tasks.workunit.client.1.smithi043.stdout:7/926: dwrite d4/db/d11/d21/d31/d65/f12b [0,4194304] 0 2024-09-16T23:49:10.284 INFO:tasks.workunit.client.0.smithi042.stdout:5/896: write d9/db/d11/f66 [1231734,117886] 0 2024-09-16T23:49:10.285 INFO:tasks.workunit.client.1.smithi043.stdout:6/991: rename de/dbf/f131 to de/d23/d63/f155 0 2024-09-16T23:49:10.286 INFO:tasks.workunit.client.0.smithi042.stdout:9/607: rmdir d4 39 2024-09-16T23:49:10.286 INFO:tasks.workunit.client.0.smithi042.stdout:8/642: mknod da/d19/d2c/d56/cc9 0 2024-09-16T23:49:10.287 INFO:tasks.workunit.client.0.smithi042.stdout:6/794: readlink d8/d70/d81/dd/d1b/d2d/da1/db3/led 0 2024-09-16T23:49:10.289 INFO:tasks.workunit.client.0.smithi042.stdout:9/608: readlink d4/d12/d2d/d37/l4c 0 2024-09-16T23:49:10.290 INFO:tasks.workunit.client.0.smithi042.stdout:6/795: readlink d8/d70/d81/dd/d1b/d50/da2/lcb 0 2024-09-16T23:49:10.293 INFO:tasks.workunit.client.1.smithi043.stdout:6/992: dwrite de/d23/d34/d66/d9d/df3/f143 [0,4194304] 0 2024-09-16T23:49:10.295 INFO:tasks.workunit.client.1.smithi043.stdout:7/927: creat d4/db/f137 x:0 0 0 2024-09-16T23:49:10.295 INFO:tasks.workunit.client.0.smithi042.stdout:9/609: creat d4/d12/d2d/d51/fbe x:0 0 0 2024-09-16T23:49:10.296 INFO:tasks.workunit.client.1.smithi043.stdout:6/993: fdatasync de/d23/dc3/fd1 0 2024-09-16T23:49:10.297 INFO:tasks.workunit.client.1.smithi043.stdout:7/928: stat d4/d27/d41/l6e 0 2024-09-16T23:49:10.298 INFO:tasks.workunit.client.0.smithi042.stdout:6/796: truncate d8/d70/d81/dd/d1b/f9d 420403 0 2024-09-16T23:49:10.299 INFO:tasks.workunit.client.0.smithi042.stdout:9/610: fdatasync d4/d12/d2d/d37/d59/f96 0 2024-09-16T23:49:10.299 INFO:tasks.workunit.client.1.smithi043.stdout:8/885: dread de/d18/d27/d41/d5a/d37/f69 [0,4194304] 0 2024-09-16T23:49:10.300 INFO:tasks.workunit.client.0.smithi042.stdout:1/829: dread d0/d9/d90/d66/de2/f111 [4194304,4194304] 0 2024-09-16T23:49:10.300 INFO:tasks.workunit.client.1.smithi043.stdout:7/929: write d4/db/d11/d21/fef [470262,68954] 0 2024-09-16T23:49:10.302 INFO:tasks.workunit.client.0.smithi042.stdout:6/797: readlink d8/d70/d81/dd/d2c/d57/dc5/ld4 0 2024-09-16T23:49:10.305 INFO:tasks.workunit.client.0.smithi042.stdout:1/830: creat d0/d9/dba/f12c x:0 0 0 2024-09-16T23:49:10.309 INFO:tasks.workunit.client.0.smithi042.stdout:6/798: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/dee/ffd x:0 0 0 2024-09-16T23:49:10.309 INFO:tasks.workunit.client.0.smithi042.stdout:9/611: dwrite d4/d10/f15 [0,4194304] 0 2024-09-16T23:49:10.309 INFO:tasks.workunit.client.1.smithi043.stdout:8/886: creat de/d18/d27/d41/d5a/f109 x:0 0 0 2024-09-16T23:49:10.309 INFO:tasks.workunit.client.0.smithi042.stdout:1/831: truncate d0/d17/f125 702560 0 2024-09-16T23:49:10.310 INFO:tasks.workunit.client.1.smithi043.stdout:8/887: chown de/d18/d27/l58 5981 1 2024-09-16T23:49:10.311 INFO:tasks.workunit.client.0.smithi042.stdout:9/612: fsync d4/d12/d2d/d37/fa4 0 2024-09-16T23:49:10.314 INFO:tasks.workunit.client.0.smithi042.stdout:1/832: symlink d0/d17/dd4/l12d 0 2024-09-16T23:49:10.314 INFO:tasks.workunit.client.0.smithi042.stdout:6/799: truncate d8/d70/d81/dd/d1b/d50/fb8 261978 0 2024-09-16T23:49:10.314 INFO:tasks.workunit.client.0.smithi042.stdout:9/613: chown d4/d12/d60/d97/fa0 631282 1 2024-09-16T23:49:10.315 INFO:tasks.workunit.client.0.smithi042.stdout:1/833: chown d0/d9/f9b 24176586 1 2024-09-16T23:49:10.317 INFO:tasks.workunit.client.0.smithi042.stdout:6/800: dread - d8/d70/d81/dd/d1b/d50/d96/da0/dd0/ff0 zero size 2024-09-16T23:49:10.317 INFO:tasks.workunit.client.0.smithi042.stdout:5/897: dread d9/db/d19/f52 [0,4194304] 0 2024-09-16T23:49:10.319 INFO:tasks.workunit.client.0.smithi042.stdout:9/614: creat d4/d12/d2d/d37/d59/fbf x:0 0 0 2024-09-16T23:49:10.319 INFO:tasks.workunit.client.0.smithi042.stdout:1/834: truncate d0/d17/fb8 285645 0 2024-09-16T23:49:10.320 INFO:tasks.workunit.client.1.smithi043.stdout:8/888: rmdir de/d18/d27/d41/d5a/d106/d39/d4d/dfe 0 2024-09-16T23:49:10.320 INFO:tasks.workunit.client.0.smithi042.stdout:5/898: chown d9/db/d19/d21/d41/d7d/fbf 85533928 1 2024-09-16T23:49:10.321 INFO:tasks.workunit.client.0.smithi042.stdout:6/801: chown d8/d70/d81/dd/d2c/d57/f60 32675300 1 2024-09-16T23:49:10.321 INFO:tasks.workunit.client.0.smithi042.stdout:8/643: dread da/d2f/f38 [0,4194304] 0 2024-09-16T23:49:10.322 INFO:tasks.workunit.client.1.smithi043.stdout:7/930: dwrite d4/d27/dbc/fe5 [0,4194304] 0 2024-09-16T23:49:10.323 INFO:tasks.workunit.client.1.smithi043.stdout:6/994: dread de/d23/d49/f4c [0,4194304] 0 2024-09-16T23:49:10.323 INFO:tasks.workunit.client.0.smithi042.stdout:8/644: read - da/d19/d2c/d56/d97/dad/fbf zero size 2024-09-16T23:49:10.324 INFO:tasks.workunit.client.0.smithi042.stdout:9/615: mknod d4/d12/d2d/d37/d3a/dad/d8e/cc0 0 2024-09-16T23:49:10.325 INFO:tasks.workunit.client.0.smithi042.stdout:5/899: read d9/db/d19/d21/d85/dec/d112/d61/f7c [3715648,67751] 0 2024-09-16T23:49:10.325 INFO:tasks.workunit.client.1.smithi043.stdout:7/931: read d4/db/fb2 [104551,24572] 0 2024-09-16T23:49:10.325 INFO:tasks.workunit.client.0.smithi042.stdout:1/835: creat d0/d9/f12e x:0 0 0 2024-09-16T23:49:10.327 INFO:tasks.workunit.client.0.smithi042.stdout:6/802: symlink d8/d70/d81/dd/dcd/de1/lfe 0 2024-09-16T23:49:10.328 INFO:tasks.workunit.client.1.smithi043.stdout:8/889: rename de/d70 to de/d18/d27/d8e/df3/d10a 0 2024-09-16T23:49:10.329 INFO:tasks.workunit.client.0.smithi042.stdout:1/836: symlink d0/ddb/ddd/l12f 0 2024-09-16T23:49:10.331 INFO:tasks.workunit.client.1.smithi043.stdout:6/995: stat de/d23/d34/d9b/d8b/dbd/c130 0 2024-09-16T23:49:10.332 INFO:tasks.workunit.client.1.smithi043.stdout:6/996: write de/dbf/fe4 [1219536,71598] 0 2024-09-16T23:49:10.333 INFO:tasks.workunit.client.0.smithi042.stdout:5/900: creat d9/db/de/d10/f136 x:0 0 0 2024-09-16T23:49:10.333 INFO:tasks.workunit.client.0.smithi042.stdout:9/616: dwrite d4/d10/faa [0,4194304] 0 2024-09-16T23:49:10.334 INFO:tasks.workunit.client.1.smithi043.stdout:8/890: creat de/d18/d27/d8e/df3/f10b x:0 0 0 2024-09-16T23:49:10.334 INFO:tasks.workunit.client.0.smithi042.stdout:6/803: symlink d8/d70/d81/dd/d1b/d67/d97/lff 0 2024-09-16T23:49:10.335 INFO:tasks.workunit.client.0.smithi042.stdout:9/617: write d4/d10/f4a [1958976,8966] 0 2024-09-16T23:49:10.340 INFO:tasks.workunit.client.1.smithi043.stdout:6/997: fsync de/d1a/fad 0 2024-09-16T23:49:10.341 INFO:tasks.workunit.client.1.smithi043.stdout:7/932: link d4/db/c5d d4/db/d11/d117/d85/d8b/c138 0 2024-09-16T23:49:10.343 INFO:tasks.workunit.client.0.smithi042.stdout:8/645: dread da/f17 [0,4194304] 0 2024-09-16T23:49:10.345 INFO:tasks.workunit.client.1.smithi043.stdout:6/998: fsync de/d1a/f20 0 2024-09-16T23:49:10.346 INFO:tasks.workunit.client.0.smithi042.stdout:5/901: fsync d9/f131 0 2024-09-16T23:49:10.346 INFO:tasks.workunit.client.0.smithi042.stdout:8/646: read - da/d19/d1d/f6b zero size 2024-09-16T23:49:10.346 INFO:tasks.workunit.client.1.smithi043.stdout:6/999: dread - de/f104 zero size 2024-09-16T23:49:10.347 INFO:tasks.workunit.client.1.smithi043.stdout:7/933: creat d4/dc0/f139 x:0 0 0 2024-09-16T23:49:10.348 INFO:tasks.workunit.client.0.smithi042.stdout:5/902: stat d9/db/de/d92/db5/c109 0 2024-09-16T23:49:10.349 INFO:tasks.workunit.client.1.smithi043.stdout:8/891: dwrite de/d18/d27/d41/d5a/d106/d39/d7b/d83/da5/ff0 [0,4194304] 0 2024-09-16T23:49:10.349 INFO:tasks.workunit.client.0.smithi042.stdout:6/804: truncate d8/d70/d81/dd/f19 173494 0 2024-09-16T23:49:10.351 INFO:tasks.workunit.client.1.smithi043.stdout:8/892: chown de/d18/d1f/d25/f3b 14275 1 2024-09-16T23:49:10.354 INFO:tasks.workunit.client.1.smithi043.stdout:8/893: creat de/d18/d27/d41/d5a/da9/f10c x:0 0 0 2024-09-16T23:49:10.355 INFO:tasks.workunit.client.0.smithi042.stdout:5/903: unlink d9/db/de/lf8 0 2024-09-16T23:49:10.356 INFO:tasks.workunit.client.0.smithi042.stdout:6/805: dread d8/d70/d81/dd/d1b/dbf/dde/f6e [0,4194304] 0 2024-09-16T23:49:10.357 INFO:tasks.workunit.client.0.smithi042.stdout:9/618: link d4/d10/f57 d4/d12/d2d/fc1 0 2024-09-16T23:49:10.359 INFO:tasks.workunit.client.1.smithi043.stdout:8/894: creat de/d18/d2b/df4/f10d x:0 0 0 2024-09-16T23:49:10.359 INFO:tasks.workunit.client.0.smithi042.stdout:6/806: stat d8/d70/d81/d90 0 2024-09-16T23:49:10.361 INFO:tasks.workunit.client.1.smithi043.stdout:8/895: write de/d18/d27/d41/d5a/f109 [200298,50580] 0 2024-09-16T23:49:10.361 INFO:tasks.workunit.client.0.smithi042.stdout:9/619: write d4/d12/f1e [7716800,47575] 0 2024-09-16T23:49:10.362 INFO:tasks.workunit.client.1.smithi043.stdout:7/934: rmdir d4/db/d11/d117/d123 0 2024-09-16T23:49:10.362 INFO:tasks.workunit.client.0.smithi042.stdout:5/904: getdents d9/db/d1b/da4 0 2024-09-16T23:49:10.363 INFO:tasks.workunit.client.0.smithi042.stdout:9/620: write d4/d10/f30 [3823683,8679] 0 2024-09-16T23:49:10.368 INFO:tasks.workunit.client.0.smithi042.stdout:8/647: write da/f83 [4304781,127838] 0 2024-09-16T23:49:10.368 INFO:tasks.workunit.client.0.smithi042.stdout:5/905: rmdir d9/db/de/d5e 39 2024-09-16T23:49:10.368 INFO:tasks.workunit.client.0.smithi042.stdout:6/807: rename d8/d70/d81/dd/d1b/dbf/cf4 to d8/d70/d81/dd/d1b/d67/d97/c100 0 2024-09-16T23:49:10.369 INFO:tasks.workunit.client.1.smithi043.stdout:7/935: dread d4/d27/d41/d1b/f90 [0,4194304] 0 2024-09-16T23:49:10.369 INFO:tasks.workunit.client.1.smithi043.stdout:8/896: dwrite de/d18/d27/d41/d5a/f26 [4194304,4194304] 0 2024-09-16T23:49:10.369 INFO:tasks.workunit.client.1.smithi043.stdout:7/936: fdatasync d4/d27/dbc/fe9 0 2024-09-16T23:49:10.370 INFO:tasks.workunit.client.0.smithi042.stdout:8/648: dread da/d19/d2c/f2d [0,4194304] 0 2024-09-16T23:49:10.371 INFO:tasks.workunit.client.0.smithi042.stdout:1/837: dwrite d0/d17/f36 [0,4194304] 0 2024-09-16T23:49:10.373 INFO:tasks.workunit.client.0.smithi042.stdout:5/906: fsync d9/db/de/d10/f1a 0 2024-09-16T23:49:10.379 INFO:tasks.workunit.client.1.smithi043.stdout:7/937: dwrite d4/db/d11/fc6 [0,4194304] 0 2024-09-16T23:49:10.383 INFO:tasks.workunit.client.1.smithi043.stdout:7/938: rmdir d4/db/d11/d117/d85/d8b/d118 39 2024-09-16T23:49:10.386 INFO:tasks.workunit.client.0.smithi042.stdout:5/907: symlink d9/d12f/l137 0 2024-09-16T23:49:10.388 INFO:tasks.workunit.client.1.smithi043.stdout:8/897: dread de/d18/d27/f75 [0,4194304] 0 2024-09-16T23:49:10.391 INFO:tasks.workunit.client.1.smithi043.stdout:7/939: dwrite d4/db/f137 [0,4194304] 0 2024-09-16T23:49:10.393 INFO:tasks.workunit.client.0.smithi042.stdout:9/621: rename d4/d12/d2d/d37 to d4/d12/d2d/d94/dc2 0 2024-09-16T23:49:10.397 INFO:tasks.workunit.client.1.smithi043.stdout:8/898: rename de/d18/d27/d8e/df3/d10a/d3d/fcf to de/d18/d2b/df4/f10e 0 2024-09-16T23:49:10.397 INFO:tasks.workunit.client.1.smithi043.stdout:8/899: chown de/d18/d27/d41/d5a/da9 3267 1 2024-09-16T23:49:10.400 INFO:tasks.workunit.client.0.smithi042.stdout:6/808: getdents d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b 0 2024-09-16T23:49:10.401 INFO:tasks.workunit.client.0.smithi042.stdout:8/649: rename da/d4b/c5d to da/cca 0 2024-09-16T23:49:10.402 INFO:tasks.workunit.client.0.smithi042.stdout:6/809: write d8/d70/d81/dd/d1b/d67/d97/dc9/fe9 [638840,46684] 0 2024-09-16T23:49:10.402 INFO:tasks.workunit.client.1.smithi043.stdout:8/900: mknod de/d18/d1f/d25/d82/c10f 0 2024-09-16T23:49:10.402 INFO:tasks.workunit.client.0.smithi042.stdout:1/838: truncate d0/d9/d90/d6b/d88/d32/d2f/de3/ffd 2280884 0 2024-09-16T23:49:10.403 INFO:tasks.workunit.client.1.smithi043.stdout:7/940: dwrite d4/db/d11/d117/ffa [0,4194304] 0 2024-09-16T23:49:10.404 INFO:tasks.workunit.client.0.smithi042.stdout:6/810: chown d8/d70/d81/dd/d1b/d50/da2/cf1 3135 1 2024-09-16T23:49:10.405 INFO:tasks.workunit.client.0.smithi042.stdout:6/811: fdatasync d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/dee/ffd 0 2024-09-16T23:49:10.411 INFO:tasks.workunit.client.1.smithi043.stdout:8/901: link de/d18/d1f/d25/f3b de/d18/d2b/df4/f110 0 2024-09-16T23:49:10.411 INFO:tasks.workunit.client.0.smithi042.stdout:5/908: write d9/db/d1b/d31/d46/d58/f9a [5196243,4952] 0 2024-09-16T23:49:10.411 INFO:tasks.workunit.client.0.smithi042.stdout:9/622: write d4/d10/f3f [3422517,117604] 0 2024-09-16T23:49:10.415 INFO:tasks.workunit.client.1.smithi043.stdout:8/902: mknod de/d18/d27/d41/d5a/dc8/deb/c111 0 2024-09-16T23:49:10.417 INFO:tasks.workunit.client.0.smithi042.stdout:1/839: write d0/d9/d90/d6b/d88/d63/fa8 [721777,84666] 0 2024-09-16T23:49:10.418 INFO:tasks.workunit.client.0.smithi042.stdout:5/909: read d9/db/d19/d21/d41/d7d/f7e [984912,86475] 0 2024-09-16T23:49:10.418 INFO:tasks.workunit.client.0.smithi042.stdout:6/812: symlink d8/d70/d81/d90/l101 0 2024-09-16T23:49:10.419 INFO:tasks.workunit.client.0.smithi042.stdout:8/650: dwrite da/d19/f1e [4194304,4194304] 0 2024-09-16T23:49:10.420 INFO:tasks.workunit.client.1.smithi043.stdout:8/903: rename de/d18/d27/d8e/df3/d10a/cff to de/d18/d27/d41/d5a/d106/d39/d7b/c112 0 2024-09-16T23:49:10.421 INFO:tasks.workunit.client.0.smithi042.stdout:1/840: chown d0/d9/fb9 2 1 2024-09-16T23:49:10.423 INFO:tasks.workunit.client.0.smithi042.stdout:5/910: creat d9/d2c/da9/f138 x:0 0 0 2024-09-16T23:49:10.423 INFO:tasks.workunit.client.1.smithi043.stdout:7/941: dread d4/d27/d41/d1b/d66/f8c [0,4194304] 0 2024-09-16T23:49:10.424 INFO:tasks.workunit.client.0.smithi042.stdout:1/841: chown d0/d9/dba/l115 13 1 2024-09-16T23:49:10.424 INFO:tasks.workunit.client.1.smithi043.stdout:8/904: symlink de/d18/d27/d41/d5a/d106/d39/d7b/d83/dbb/l113 0 2024-09-16T23:49:10.425 INFO:tasks.workunit.client.0.smithi042.stdout:9/623: dwrite d4/d12/d2d/d94/dc2/d28/d5e/d83/fb0 [0,4194304] 0 2024-09-16T23:49:10.426 INFO:tasks.workunit.client.1.smithi043.stdout:8/905: dread - de/d18/d27/d41/fee zero size 2024-09-16T23:49:10.426 INFO:tasks.workunit.client.0.smithi042.stdout:6/813: fsync d8/d70/d81/dd/d1b/dbf/dde/f51 0 2024-09-16T23:49:10.427 INFO:tasks.workunit.client.0.smithi042.stdout:5/911: rmdir d9/db/d19/d21 39 2024-09-16T23:49:10.427 INFO:tasks.workunit.client.1.smithi043.stdout:7/942: dread d4/d27/dbc/f10f [0,4194304] 0 2024-09-16T23:49:10.428 INFO:tasks.workunit.client.0.smithi042.stdout:7/818: sync 2024-09-16T23:49:10.428 INFO:tasks.workunit.client.0.smithi042.stdout:0/771: sync 2024-09-16T23:49:10.429 INFO:tasks.workunit.client.1.smithi043.stdout:8/906: chown de/d18/c1c 0 1 2024-09-16T23:49:10.430 INFO:tasks.workunit.client.0.smithi042.stdout:8/651: getdents da/d19/d1d/d30/d49 0 2024-09-16T23:49:10.430 INFO:tasks.workunit.client.0.smithi042.stdout:1/842: mkdir d0/d9/d90/d6b/d130 0 2024-09-16T23:49:10.431 INFO:tasks.workunit.client.1.smithi043.stdout:7/943: rmdir d4/db/d11/d15/d29/da7 39 2024-09-16T23:49:10.432 INFO:tasks.workunit.client.1.smithi043.stdout:8/907: readlink de/d18/l4e 0 2024-09-16T23:49:10.433 INFO:tasks.workunit.client.0.smithi042.stdout:8/652: truncate da/d19/d2c/d56/fae 5141494 0 2024-09-16T23:49:10.433 INFO:tasks.workunit.client.0.smithi042.stdout:6/814: rmdir d8/d70/d81/dd/d1b/dbf/dde 39 2024-09-16T23:49:10.434 INFO:tasks.workunit.client.0.smithi042.stdout:9/624: dread d4/d10/f6b [0,4194304] 0 2024-09-16T23:49:10.435 INFO:tasks.workunit.client.1.smithi043.stdout:7/944: mkdir d4/db/d11/d15/dbe/d13a 0 2024-09-16T23:49:10.436 INFO:tasks.workunit.client.0.smithi042.stdout:0/772: unlink d0/f78 0 2024-09-16T23:49:10.438 INFO:tasks.workunit.client.1.smithi043.stdout:7/945: readlink d4/d27/d8a/d93/l9e 0 2024-09-16T23:49:10.441 INFO:tasks.workunit.client.1.smithi043.stdout:7/946: creat d4/db/d11/d21/d31/d33/f13b x:0 0 0 2024-09-16T23:49:10.442 INFO:tasks.workunit.client.0.smithi042.stdout:7/819: creat d3/d98/da0/d24/d11a/f11f x:0 0 0 2024-09-16T23:49:10.442 INFO:tasks.workunit.client.0.smithi042.stdout:6/815: write d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/f2e [7638039,85697] 0 2024-09-16T23:49:10.442 INFO:tasks.workunit.client.0.smithi042.stdout:0/773: symlink d0/df/d13/d49/d70/da1/lff 0 2024-09-16T23:49:10.442 INFO:tasks.workunit.client.0.smithi042.stdout:9/625: truncate d4/d12/d60/f78 751603 0 2024-09-16T23:49:10.444 INFO:tasks.workunit.client.1.smithi043.stdout:7/947: creat d4/db/da6/f13c x:0 0 0 2024-09-16T23:49:10.444 INFO:tasks.workunit.client.0.smithi042.stdout:1/843: unlink d0/d9/d26/d8b/lf0 0 2024-09-16T23:49:10.445 INFO:tasks.workunit.client.0.smithi042.stdout:5/912: dread d9/db/d19/d21/d41/d7d/f12b [0,4194304] 0 2024-09-16T23:49:10.445 INFO:tasks.workunit.client.1.smithi043.stdout:8/908: write de/d18/d27/d41/d5a/f13 [185186,14445] 0 2024-09-16T23:49:10.446 INFO:tasks.workunit.client.1.smithi043.stdout:8/909: chown de/d18/d27/l58 6 1 2024-09-16T23:49:10.447 INFO:tasks.workunit.client.0.smithi042.stdout:8/653: write da/d19/d1d/d30/f41 [934576,29712] 0 2024-09-16T23:49:10.447 INFO:tasks.workunit.client.0.smithi042.stdout:1/844: readlink d0/d17/dd4/l12d 0 2024-09-16T23:49:10.448 INFO:tasks.workunit.client.1.smithi043.stdout:7/948: fdatasync d4/db/d11/d21/d31/fb8 0 2024-09-16T23:49:10.448 INFO:tasks.workunit.client.0.smithi042.stdout:7/820: dread - d3/d1f/d49/d76/d19/ff1 zero size 2024-09-16T23:49:10.448 INFO:tasks.workunit.client.0.smithi042.stdout:0/774: creat d0/df/d23/d33/d56/dc5/dcd/f100 x:0 0 0 2024-09-16T23:49:10.449 INFO:tasks.workunit.client.1.smithi043.stdout:8/910: write de/d18/d27/d41/d5a/d106/d39/d7b/d83/fc0 [2583536,49093] 0 2024-09-16T23:49:10.450 INFO:tasks.workunit.client.0.smithi042.stdout:9/626: creat d4/d12/d2d/d94/dc2/d41/fc3 x:0 0 0 2024-09-16T23:49:10.451 INFO:tasks.workunit.client.1.smithi043.stdout:8/911: read de/d18/d27/d41/d5a/d106/d39/d7b/d83/fc0 [4089906,100862] 0 2024-09-16T23:49:10.453 INFO:tasks.workunit.client.0.smithi042.stdout:1/845: write d0/d9/d90/d6b/d88/dc2/f120 [895923,76206] 0 2024-09-16T23:49:10.453 INFO:tasks.workunit.client.0.smithi042.stdout:6/816: truncate d8/d70/d81/dd/d1b/f93 685721 0 2024-09-16T23:49:10.456 INFO:tasks.workunit.client.0.smithi042.stdout:0/775: symlink d0/df/d13/d2d/d35/d9f/l101 0 2024-09-16T23:49:10.456 INFO:tasks.workunit.client.0.smithi042.stdout:0/776: stat d0/df/lc8 0 2024-09-16T23:49:10.457 INFO:tasks.workunit.client.0.smithi042.stdout:9/627: mkdir d4/d12/d77/dc4 0 2024-09-16T23:49:10.457 INFO:tasks.workunit.client.0.smithi042.stdout:0/777: chown d0/df/d23/d33/d56/da9 1665422369 1 2024-09-16T23:49:10.458 INFO:tasks.workunit.client.0.smithi042.stdout:8/654: rmdir da/d19/d1d/d30/d49/db2/dbe 0 2024-09-16T23:49:10.459 INFO:tasks.workunit.client.0.smithi042.stdout:8/655: mknod da/d19/d2c/d56/ccb 0 2024-09-16T23:49:10.461 INFO:tasks.workunit.client.0.smithi042.stdout:8/656: creat da/d19/fcc x:0 0 0 2024-09-16T23:49:10.462 INFO:tasks.workunit.client.0.smithi042.stdout:7/821: rename d3/d1f/d49/d76/d101/d5b/ld4 to d3/d1f/d49/d76/d101/da5/de3/l120 0 2024-09-16T23:49:10.464 INFO:tasks.workunit.client.1.smithi043.stdout:8/912: dread de/d18/d27/f85 [0,4194304] 0 2024-09-16T23:49:10.464 INFO:tasks.workunit.client.0.smithi042.stdout:9/628: symlink d4/d12/d2d/d94/dc2/d28/d5e/d83/db3/lc5 0 2024-09-16T23:49:10.465 INFO:tasks.workunit.client.1.smithi043.stdout:8/913: chown de/d18/d27/d41/d5a/d106/d39/d7b/d83/dbb/cd9 44844 1 2024-09-16T23:49:10.466 INFO:tasks.workunit.client.0.smithi042.stdout:8/657: link da/d19/d1d/d30/f41 da/d19/d1d/d30/d3f/dc2/fcd 0 2024-09-16T23:49:10.466 INFO:tasks.workunit.client.0.smithi042.stdout:1/846: link d0/d17/ff7 d0/d17/dc7/f131 0 2024-09-16T23:49:10.469 INFO:tasks.workunit.client.0.smithi042.stdout:7/822: getdents d3/d1f/d4f/d119 0 2024-09-16T23:49:10.469 INFO:tasks.workunit.client.0.smithi042.stdout:5/913: dwrite d9/db/d1b/d31/d46/d58/fbb [0,4194304] 0 2024-09-16T23:49:10.469 INFO:tasks.workunit.client.0.smithi042.stdout:1/847: chown d0/d9/d26 22375 1 2024-09-16T23:49:10.470 INFO:tasks.workunit.client.0.smithi042.stdout:9/629: creat d4/d12/d16/fc6 x:0 0 0 2024-09-16T23:49:10.471 INFO:tasks.workunit.client.0.smithi042.stdout:1/848: chown d0/c5a 96863 1 2024-09-16T23:49:10.471 INFO:tasks.workunit.client.0.smithi042.stdout:7/823: chown d3/fe 11 1 2024-09-16T23:49:10.472 INFO:tasks.workunit.client.0.smithi042.stdout:9/630: stat d4/d12/d2d/d94/dc2/d41/cb4 0 2024-09-16T23:49:10.473 INFO:tasks.workunit.client.0.smithi042.stdout:5/914: write d9/db/d19/d21/d41/fab [4968950,85781] 0 2024-09-16T23:49:10.474 INFO:tasks.workunit.client.0.smithi042.stdout:1/849: symlink d0/d17/d79/d92/l132 0 2024-09-16T23:49:10.474 INFO:tasks.workunit.client.0.smithi042.stdout:6/817: creat d8/d70/d81/dd/d1b/f102 x:0 0 0 2024-09-16T23:49:10.476 INFO:tasks.workunit.client.0.smithi042.stdout:1/850: dread - d0/d9/dba/fd1 zero size 2024-09-16T23:49:10.479 INFO:tasks.workunit.client.0.smithi042.stdout:1/851: truncate d0/d9/d26/f84 788148 0 2024-09-16T23:49:10.479 INFO:tasks.workunit.client.0.smithi042.stdout:9/631: read - d4/d12/d2d/d94/dc2/d41/d92/da7/f89 zero size 2024-09-16T23:49:10.484 INFO:tasks.workunit.client.0.smithi042.stdout:7/824: dread d3/d1f/d49/d76/d19/d63/f75 [0,4194304] 0 2024-09-16T23:49:10.485 INFO:tasks.workunit.client.0.smithi042.stdout:9/632: dread d4/d12/d2d/d94/dc2/d3a/d50/f99 [0,4194304] 0 2024-09-16T23:49:10.488 INFO:tasks.workunit.client.0.smithi042.stdout:9/633: mknod d4/d12/d2d/d94/dc2/cc7 0 2024-09-16T23:49:10.489 INFO:tasks.workunit.client.0.smithi042.stdout:7/825: creat d3/d1f/d49/d76/f121 x:0 0 0 2024-09-16T23:49:10.491 INFO:tasks.workunit.client.0.smithi042.stdout:6/818: dwrite d8/d70/d81/dd/f11 [0,4194304] 0 2024-09-16T23:49:10.492 INFO:tasks.workunit.client.0.smithi042.stdout:9/634: unlink d4/d12/d2d/d94/dc2/d28/f31 0 2024-09-16T23:49:10.493 INFO:tasks.workunit.client.0.smithi042.stdout:7/826: dread d3/d1f/d49/d78/dbe/fca [0,4194304] 0 2024-09-16T23:49:10.493 INFO:tasks.workunit.client.0.smithi042.stdout:5/915: dread d9/db/f25 [4194304,4194304] 0 2024-09-16T23:49:10.496 INFO:tasks.workunit.client.0.smithi042.stdout:7/827: truncate d3/f84 3183223 0 2024-09-16T23:49:10.499 INFO:tasks.workunit.client.0.smithi042.stdout:7/828: creat d3/d3f/f122 x:0 0 0 2024-09-16T23:49:10.500 INFO:tasks.workunit.client.0.smithi042.stdout:9/635: rename d4/d12/d2d/d94/dc2/d3a/dad/d8e/da1/fb2 to d4/d12/d16/fc8 0 2024-09-16T23:49:10.501 INFO:tasks.workunit.client.0.smithi042.stdout:6/819: dread d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d99/fd9 [0,4194304] 0 2024-09-16T23:49:10.504 INFO:tasks.workunit.client.0.smithi042.stdout:5/916: dwrite d9/db/d1b/d31/f7a [0,4194304] 0 2024-09-16T23:49:10.508 INFO:tasks.workunit.client.0.smithi042.stdout:5/917: creat d9/db/d1b/d79/f139 x:0 0 0 2024-09-16T23:49:10.508 INFO:tasks.workunit.client.1.smithi043.stdout:8/914: mkdir de/d18/d2b/d114 0 2024-09-16T23:49:10.509 INFO:tasks.workunit.client.1.smithi043.stdout:8/915: chown de/d18/d27/d41/d5a/d106/l67 491248 1 2024-09-16T23:49:10.510 INFO:tasks.workunit.client.0.smithi042.stdout:9/636: dread d4/d12/d2d/d94/dc2/d59/f96 [0,4194304] 0 2024-09-16T23:49:10.511 INFO:tasks.workunit.client.0.smithi042.stdout:8/658: symlink da/d19/d1d/d30/d3f/dc2/lce 0 2024-09-16T23:49:10.511 INFO:tasks.workunit.client.0.smithi042.stdout:1/852: getdents d0/d9/d90/d66/de2 0 2024-09-16T23:49:10.512 INFO:tasks.workunit.client.0.smithi042.stdout:6/820: link d8/c53 d8/d70/d81/dd/d1b/dbf/dde/c103 0 2024-09-16T23:49:10.512 INFO:tasks.workunit.client.1.smithi043.stdout:8/916: mknod de/d18/d27/d8e/c115 0 2024-09-16T23:49:10.515 INFO:tasks.workunit.client.0.smithi042.stdout:5/918: creat d9/db/de/d5e/f13a x:0 0 0 2024-09-16T23:49:10.516 INFO:tasks.workunit.client.0.smithi042.stdout:7/829: dwrite d3/fa [0,4194304] 0 2024-09-16T23:49:10.517 INFO:tasks.workunit.client.0.smithi042.stdout:1/853: chown d0/d9/d90/d6b/d8e/de5 14 1 2024-09-16T23:49:10.521 INFO:tasks.workunit.client.0.smithi042.stdout:9/637: dread d4/d12/d2d/d94/dc2/d28/d5e/d83/fb0 [0,4194304] 0 2024-09-16T23:49:10.528 INFO:tasks.workunit.client.0.smithi042.stdout:5/919: creat d9/db/de/d10/f13b x:0 0 0 2024-09-16T23:49:10.529 INFO:tasks.workunit.client.0.smithi042.stdout:9/638: readlink d4/d12/d2d/d94/dc2/d28/d5e/d83/lbb 0 2024-09-16T23:49:10.529 INFO:tasks.workunit.client.0.smithi042.stdout:1/854: write d0/d9/d90/d6b/d88/def/f104 [605523,41296] 0 2024-09-16T23:49:10.531 INFO:tasks.workunit.client.1.smithi043.stdout:8/917: dread de/d18/d27/f94 [0,4194304] 0 2024-09-16T23:49:10.531 INFO:tasks.workunit.client.1.smithi043.stdout:8/918: fdatasync de/f17 0 2024-09-16T23:49:10.532 INFO:tasks.workunit.client.0.smithi042.stdout:6/821: dread d8/d70/d81/dd/d1b/d2d/da1/fc1 [0,4194304] 0 2024-09-16T23:49:10.532 INFO:tasks.workunit.client.1.smithi043.stdout:8/919: chown de/d18/d27/d41/d5a/d106/d39/d4d/l5e 295369571 1 2024-09-16T23:49:10.534 INFO:tasks.workunit.client.0.smithi042.stdout:1/855: mknod d0/d17/d79/c133 0 2024-09-16T23:49:10.534 INFO:tasks.workunit.client.0.smithi042.stdout:9/639: mkdir d4/d12/d2d/dc9 0 2024-09-16T23:49:10.536 INFO:tasks.workunit.client.1.smithi043.stdout:8/920: symlink de/d18/d2b/l116 0 2024-09-16T23:49:10.537 INFO:tasks.workunit.client.0.smithi042.stdout:5/920: truncate d9/db/d19/ff6 2033270 0 2024-09-16T23:49:10.537 INFO:tasks.workunit.client.0.smithi042.stdout:7/830: dwrite d3/d1f/d49/d76/d19/d63/f75 [0,4194304] 0 2024-09-16T23:49:10.538 INFO:tasks.workunit.client.0.smithi042.stdout:6/822: write d8/d70/d81/dd/d1b/dbf/dde/f79 [519984,60364] 0 2024-09-16T23:49:10.542 INFO:tasks.workunit.client.0.smithi042.stdout:1/856: write d0/d9/d26/fd2 [93184,32754] 0 2024-09-16T23:49:10.542 INFO:tasks.workunit.client.1.smithi043.stdout:8/921: rename de/d18/d2b/da1/fe2 to de/d18/d27/d41/d5a/dc8/deb/f117 0 2024-09-16T23:49:10.544 INFO:tasks.workunit.client.0.smithi042.stdout:5/921: unlink d9/db/d1b/d79/fb7 0 2024-09-16T23:49:10.545 INFO:tasks.workunit.client.0.smithi042.stdout:8/659: dread da/d19/d1d/f63 [0,4194304] 0 2024-09-16T23:49:10.546 INFO:tasks.workunit.client.0.smithi042.stdout:8/660: write da/f14 [2287651,5618] 0 2024-09-16T23:49:10.548 INFO:tasks.workunit.client.0.smithi042.stdout:9/640: dread d4/d12/d2d/d94/dc2/d3a/f5c [0,4194304] 0 2024-09-16T23:49:10.549 INFO:tasks.workunit.client.0.smithi042.stdout:5/922: mkdir d9/db/d1b/da4/de2/d13c 0 2024-09-16T23:49:10.550 INFO:tasks.workunit.client.1.smithi043.stdout:8/922: dwrite de/d18/d27/d41/d5a/f109 [0,4194304] 0 2024-09-16T23:49:10.552 INFO:tasks.workunit.client.0.smithi042.stdout:9/641: chown d4/d12/d2d/d94/dc2/d28/d5e/d83/db3/lc5 1186 1 2024-09-16T23:49:10.555 INFO:tasks.workunit.client.0.smithi042.stdout:5/923: dread d9/db/d1b/d31/f7a [0,4194304] 0 2024-09-16T23:49:10.556 INFO:tasks.workunit.client.1.smithi043.stdout:8/923: mknod de/d18/d27/d41/d5a/d106/d39/d7b/d83/da5/c118 0 2024-09-16T23:49:10.559 INFO:tasks.workunit.client.0.smithi042.stdout:9/642: creat d4/d12/d2d/d94/dc2/d3a/dad/fca x:0 0 0 2024-09-16T23:49:10.562 INFO:tasks.workunit.client.0.smithi042.stdout:5/924: symlink d9/db/d1b/d31/d6c/d100/l13d 0 2024-09-16T23:49:10.565 INFO:tasks.workunit.client.0.smithi042.stdout:6/823: dwrite d8/d70/d81/dd/d1b/d34/d39/fa9 [0,4194304] 0 2024-09-16T23:49:10.565 INFO:tasks.workunit.client.0.smithi042.stdout:8/661: truncate da/d19/f39 1249259 0 2024-09-16T23:49:10.567 INFO:tasks.workunit.client.0.smithi042.stdout:5/925: mknod d9/db/d1b/d31/d6c/c13e 0 2024-09-16T23:49:10.569 INFO:tasks.workunit.client.0.smithi042.stdout:1/857: dwrite d0/d9/d90/d6b/d88/d32/d2f/de3/fff [0,4194304] 0 2024-09-16T23:49:10.570 INFO:tasks.workunit.client.0.smithi042.stdout:8/662: symlink da/d4b/lcf 0 2024-09-16T23:49:10.572 INFO:tasks.workunit.client.0.smithi042.stdout:1/858: chown d0/d17/d18/f73 12197 1 2024-09-16T23:49:10.574 INFO:tasks.workunit.client.0.smithi042.stdout:9/643: dwrite d4/d12/d16/f6d [0,4194304] 0 2024-09-16T23:49:10.575 INFO:tasks.workunit.client.0.smithi042.stdout:6/824: symlink d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/l104 0 2024-09-16T23:49:10.577 INFO:tasks.workunit.client.0.smithi042.stdout:8/663: write da/d19/d1d/d5a/fc6 [1947448,105901] 0 2024-09-16T23:49:10.584 INFO:tasks.workunit.client.0.smithi042.stdout:6/825: creat d8/d70/d81/dd/d1b/d50/d96/da0/dd7/f105 x:0 0 0 2024-09-16T23:49:10.584 INFO:tasks.workunit.client.0.smithi042.stdout:5/926: dwrite d9/f131 [0,4194304] 0 2024-09-16T23:49:10.584 INFO:tasks.workunit.client.0.smithi042.stdout:8/664: mkdir da/d19/d2c/d56/d97/dad/dd0 0 2024-09-16T23:49:10.586 INFO:tasks.workunit.client.0.smithi042.stdout:2/733: sync 2024-09-16T23:49:10.586 INFO:tasks.workunit.client.0.smithi042.stdout:1/859: write d0/d17/d5e/f74 [181138,115007] 0 2024-09-16T23:49:10.590 INFO:tasks.workunit.client.0.smithi042.stdout:8/665: mknod da/cd1 0 2024-09-16T23:49:10.590 INFO:tasks.workunit.client.0.smithi042.stdout:2/734: unlink d0/d30/d43/fea 0 2024-09-16T23:49:10.594 INFO:tasks.workunit.client.0.smithi042.stdout:1/860: mkdir d0/d9/d134 0 2024-09-16T23:49:10.596 INFO:tasks.workunit.client.0.smithi042.stdout:2/735: fdatasync d0/f92 0 2024-09-16T23:49:10.602 INFO:tasks.workunit.client.0.smithi042.stdout:6/826: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/fdb [0,4194304] 0 2024-09-16T23:49:10.605 INFO:tasks.workunit.client.0.smithi042.stdout:5/927: dwrite d9/db/d19/d21/d41/fab [0,4194304] 0 2024-09-16T23:49:10.606 INFO:tasks.workunit.client.0.smithi042.stdout:2/736: getdents d0 0 2024-09-16T23:49:10.613 INFO:tasks.workunit.client.0.smithi042.stdout:5/928: creat d9/db/d1b/da4/f13f x:0 0 0 2024-09-16T23:49:10.613 INFO:tasks.workunit.client.0.smithi042.stdout:9/644: dread d4/d12/d16/d68/f9f [0,4194304] 0 2024-09-16T23:49:10.616 INFO:tasks.workunit.client.0.smithi042.stdout:2/737: creat d0/da/d88/ff5 x:0 0 0 2024-09-16T23:49:10.616 INFO:tasks.workunit.client.0.smithi042.stdout:8/666: dread da/d19/d1d/d30/d3f/f61 [0,4194304] 0 2024-09-16T23:49:10.619 INFO:tasks.workunit.client.0.smithi042.stdout:1/861: dwrite d0/d9/d26/d8b/f103 [0,4194304] 0 2024-09-16T23:49:10.619 INFO:tasks.workunit.client.0.smithi042.stdout:5/929: creat d9/db/de/d10/d2f/f140 x:0 0 0 2024-09-16T23:49:10.622 INFO:tasks.workunit.client.0.smithi042.stdout:5/930: creat d9/d63/f141 x:0 0 0 2024-09-16T23:49:10.625 INFO:tasks.workunit.client.0.smithi042.stdout:6/827: write d8/d70/d81/dd/d1b/f9d [1151539,95900] 0 2024-09-16T23:49:10.625 INFO:tasks.workunit.client.0.smithi042.stdout:9/645: dwrite d4/d12/d2d/d94/dc2/d59/f91 [0,4194304] 0 2024-09-16T23:49:10.626 INFO:tasks.workunit.client.0.smithi042.stdout:6/828: chown d8/f37 1795078 1 2024-09-16T23:49:10.631 INFO:tasks.workunit.client.0.smithi042.stdout:2/738: write d0/d30/d43/f89 [859853,45368] 0 2024-09-16T23:49:10.636 INFO:tasks.workunit.client.0.smithi042.stdout:5/931: write d9/db/d19/d21/d85/da5/f94 [978126,94655] 0 2024-09-16T23:49:10.639 INFO:tasks.workunit.client.0.smithi042.stdout:8/667: dwrite da/d19/d1d/d30/f41 [0,4194304] 0 2024-09-16T23:49:10.642 INFO:tasks.workunit.client.0.smithi042.stdout:9/646: dwrite d4/d10/f1b [0,4194304] 0 2024-09-16T23:49:10.644 INFO:tasks.workunit.client.0.smithi042.stdout:8/668: readlink da/lb 0 2024-09-16T23:49:10.646 INFO:tasks.workunit.client.0.smithi042.stdout:5/932: getdents d9/db/de 0 2024-09-16T23:49:10.647 INFO:tasks.workunit.client.0.smithi042.stdout:6/829: dwrite d8/d70/d81/dd/d2c/d57/f77 [4194304,4194304] 0 2024-09-16T23:49:10.655 INFO:tasks.workunit.client.0.smithi042.stdout:8/669: creat da/d19/d2c/d56/d97/fd2 x:0 0 0 2024-09-16T23:49:10.655 INFO:tasks.workunit.client.0.smithi042.stdout:6/830: truncate d8/d70/d81/dd/d2c/d57/f94 1232942 0 2024-09-16T23:49:10.657 INFO:tasks.workunit.client.0.smithi042.stdout:5/933: write d9/db/de/d5e/fcb [4834958,128486] 0 2024-09-16T23:49:10.658 INFO:tasks.workunit.client.0.smithi042.stdout:9/647: dwrite d4/d12/d16/d68/f9f [4194304,4194304] 0 2024-09-16T23:49:10.661 INFO:tasks.workunit.client.0.smithi042.stdout:6/831: creat d8/d70/d81/dd/d1b/d50/d96/f106 x:0 0 0 2024-09-16T23:49:10.661 INFO:tasks.workunit.client.0.smithi042.stdout:8/670: creat da/d19/d1d/d30/d3f/dc2/fd3 x:0 0 0 2024-09-16T23:49:10.662 INFO:tasks.workunit.client.0.smithi042.stdout:9/648: unlink d4/d12/d2d/d94/dc2/d41/f49 0 2024-09-16T23:49:10.663 INFO:tasks.workunit.client.0.smithi042.stdout:5/934: rmdir d9/db/d19/d21/d85/dec/d112 39 2024-09-16T23:49:10.664 INFO:tasks.workunit.client.0.smithi042.stdout:8/671: mkdir da/d19/d2c/d56/dd4 0 2024-09-16T23:49:10.669 INFO:tasks.workunit.client.0.smithi042.stdout:5/935: getdents d9/d11a/d12c 0 2024-09-16T23:49:10.669 INFO:tasks.workunit.client.0.smithi042.stdout:9/649: dread d4/d12/d2d/d94/dc2/d28/d5e/d83/fb0 [0,4194304] 0 2024-09-16T23:49:10.671 INFO:tasks.workunit.client.0.smithi042.stdout:5/936: stat d9/db/d1b/d31/d6c/c82 0 2024-09-16T23:49:10.674 INFO:tasks.workunit.client.0.smithi042.stdout:8/672: write da/d19/d1d/d5a/faf [1120143,41225] 0 2024-09-16T23:49:10.674 INFO:tasks.workunit.client.0.smithi042.stdout:6/832: dwrite d8/d70/d81/dd/d2c/d57/fca [0,4194304] 0 2024-09-16T23:49:10.675 INFO:tasks.workunit.client.0.smithi042.stdout:9/650: dread d4/d12/d2d/d94/dc2/d3a/f5c [0,4194304] 0 2024-09-16T23:49:10.676 INFO:tasks.workunit.client.0.smithi042.stdout:8/673: stat da/d19/la3 0 2024-09-16T23:49:10.677 INFO:tasks.workunit.client.0.smithi042.stdout:5/937: mknod d9/db/d1b/d31/d46/d107/c142 0 2024-09-16T23:49:10.681 INFO:tasks.workunit.client.0.smithi042.stdout:5/938: creat d9/db/de/d10/d2f/f143 x:0 0 0 2024-09-16T23:49:10.682 INFO:tasks.workunit.client.0.smithi042.stdout:8/674: truncate da/d19/d2c/d56/d97/dad/fbf 332832 0 2024-09-16T23:49:10.683 INFO:tasks.workunit.client.0.smithi042.stdout:6/833: link d8/d70/d81/dd/d2c/d57/dc5/ld4 d8/d70/d81/dd/d1b/d2d/l107 0 2024-09-16T23:49:10.684 INFO:tasks.workunit.client.0.smithi042.stdout:5/939: truncate d9/db/de/d92/fa1 539445 0 2024-09-16T23:49:10.686 INFO:tasks.workunit.client.0.smithi042.stdout:6/834: chown d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b/c8d 105 1 2024-09-16T23:49:10.689 INFO:tasks.workunit.client.0.smithi042.stdout:8/675: getdents da 0 2024-09-16T23:49:10.693 INFO:tasks.workunit.client.0.smithi042.stdout:9/651: dwrite d4/d10/d24/f35 [0,4194304] 0 2024-09-16T23:49:10.693 INFO:tasks.workunit.client.0.smithi042.stdout:6/835: creat d8/d70/d81/dd/d2c/d57/f108 x:0 0 0 2024-09-16T23:49:10.695 INFO:tasks.workunit.client.0.smithi042.stdout:5/940: dread d9/db/d19/d21/d85/dec/d112/d61/fd9 [0,4194304] 0 2024-09-16T23:49:10.696 INFO:tasks.workunit.client.0.smithi042.stdout:9/652: readlink d4/d12/d2d/d94/dc2/d41/l64 0 2024-09-16T23:49:10.698 INFO:tasks.workunit.client.0.smithi042.stdout:8/676: dwrite da/f83 [0,4194304] 0 2024-09-16T23:49:10.699 INFO:tasks.workunit.client.0.smithi042.stdout:6/836: symlink d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/dfb/l109 0 2024-09-16T23:49:10.699 INFO:tasks.workunit.client.0.smithi042.stdout:8/677: read da/d19/d2c/f2d [14027,113268] 0 2024-09-16T23:49:10.700 INFO:tasks.workunit.client.0.smithi042.stdout:9/653: mknod d4/d12/d2d/ccb 0 2024-09-16T23:49:10.703 INFO:tasks.workunit.client.0.smithi042.stdout:6/837: mkdir d8/d70/d81/dd/dcd/d10a 0 2024-09-16T23:49:10.705 INFO:tasks.workunit.client.0.smithi042.stdout:9/654: chown d4/d12/d2d/d94/dc2/c2b 5689 1 2024-09-16T23:49:10.706 INFO:tasks.workunit.client.0.smithi042.stdout:9/655: readlink d4/d10/d4b/l62 0 2024-09-16T23:49:10.706 INFO:tasks.workunit.client.0.smithi042.stdout:9/656: chown d4/d10 1736552279 1 2024-09-16T23:49:10.708 INFO:tasks.workunit.client.0.smithi042.stdout:5/941: dread d9/db/d1b/d31/d6c/f93 [0,4194304] 0 2024-09-16T23:49:10.708 INFO:tasks.workunit.client.0.smithi042.stdout:8/678: write da/f96 [252567,19893] 0 2024-09-16T23:49:10.708 INFO:tasks.workunit.client.0.smithi042.stdout:9/657: chown d4/d12/f36 2 1 2024-09-16T23:49:10.709 INFO:tasks.workunit.client.0.smithi042.stdout:8/679: chown da/d19/d2c/d56/d97/dad/dd0 4007881 1 2024-09-16T23:49:10.711 INFO:tasks.workunit.client.0.smithi042.stdout:9/658: write d4/d12/d2d/d94/dc2/d3a/f48 [2438225,36874] 0 2024-09-16T23:49:10.715 INFO:tasks.workunit.client.0.smithi042.stdout:9/659: creat d4/d12/d2d/d94/dc2/d41/d92/fcc x:0 0 0 2024-09-16T23:49:10.717 INFO:tasks.workunit.client.0.smithi042.stdout:9/660: read d4/d12/d16/f6d [436235,99803] 0 2024-09-16T23:49:10.718 INFO:tasks.workunit.client.0.smithi042.stdout:8/680: dwrite da/d19/d1d/d30/d49/f7b [0,4194304] 0 2024-09-16T23:49:10.720 INFO:tasks.workunit.client.0.smithi042.stdout:6/838: rename d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d7b to d8/d70/d81/dd/d1b/d34/d39/d5e/d10b 0 2024-09-16T23:49:10.725 INFO:tasks.workunit.client.0.smithi042.stdout:5/942: rename d9/d4a/d89/f8a to d9/db/de/d92/db5/f144 0 2024-09-16T23:49:10.726 INFO:tasks.workunit.client.1.smithi043.stdout:3/956: sync 2024-09-16T23:49:10.726 INFO:tasks.workunit.client.1.smithi043.stdout:2/855: sync 2024-09-16T23:49:10.726 INFO:tasks.workunit.client.1.smithi043.stdout:9/957: sync 2024-09-16T23:49:10.727 INFO:tasks.workunit.client.1.smithi043.stdout:0/997: sync 2024-09-16T23:49:10.730 INFO:tasks.workunit.client.0.smithi042.stdout:9/661: dwrite d4/d12/d16/d68/f74 [0,4194304] 0 2024-09-16T23:49:10.730 INFO:tasks.workunit.client.0.smithi042.stdout:6/839: dread d8/d70/d81/dd/d2c/d57/f60 [0,4194304] 0 2024-09-16T23:49:10.731 INFO:tasks.workunit.client.1.smithi043.stdout:9/958: symlink d4/d5/d64/df8/d100/l140 0 2024-09-16T23:49:10.732 INFO:tasks.workunit.client.1.smithi043.stdout:9/959: chown d4/d1c/d25/d2b/d110/dbf/f105 1 1 2024-09-16T23:49:10.734 INFO:tasks.workunit.client.1.smithi043.stdout:2/856: symlink d8/d5e/d83/d3a/d77/dea/l11e 0 2024-09-16T23:49:10.735 INFO:tasks.workunit.client.1.smithi043.stdout:3/957: truncate d4/d30/fcc 395261 0 2024-09-16T23:49:10.738 INFO:tasks.workunit.client.0.smithi042.stdout:6/840: dread - d8/d70/d81/dd/d1b/d50/d96/da0/dd0/ff0 zero size 2024-09-16T23:49:10.739 INFO:tasks.workunit.client.1.smithi043.stdout:0/998: creat d0/d2/df/d1a/d49/d80/d11e/f154 x:0 0 0 2024-09-16T23:49:10.739 INFO:tasks.workunit.client.0.smithi042.stdout:5/943: dwrite d9/db/de/d92/daf/fe0 [0,4194304] 0 2024-09-16T23:49:10.740 INFO:tasks.workunit.client.1.smithi043.stdout:0/999: chown d0/d2/df/d1a/d3b/c3f 4 1 2024-09-16T23:49:10.741 INFO:tasks.workunit.client.1.smithi043.stdout:9/960: rename d4/d1c/d25/db1/cf5 to d4/d21/d35/d4b/c141 0 2024-09-16T23:49:10.742 INFO:tasks.workunit.client.0.smithi042.stdout:8/681: dread da/d19/d1d/d30/d3f/d6d/f7e [0,4194304] 0 2024-09-16T23:49:10.742 INFO:tasks.workunit.client.1.smithi043.stdout:9/961: dread - d4/d1c/fb5 zero size 2024-09-16T23:49:10.743 INFO:tasks.workunit.client.1.smithi043.stdout:3/958: mknod d4/de/d1f/c126 0 2024-09-16T23:49:10.744 INFO:tasks.workunit.client.0.smithi042.stdout:5/944: creat d9/d2c/d121/f145 x:0 0 0 2024-09-16T23:49:10.745 INFO:tasks.workunit.client.1.smithi043.stdout:2/857: creat d8/d4b/d5f/f11f x:0 0 0 2024-09-16T23:49:10.746 INFO:tasks.workunit.client.0.smithi042.stdout:9/662: dwrite d4/d12/d2d/d94/dc2/f6a [0,4194304] 0 2024-09-16T23:49:10.747 INFO:tasks.workunit.client.1.smithi043.stdout:3/959: mknod d4/de/d1f/d2c/db7/dc3/dd3/c127 0 2024-09-16T23:49:10.748 INFO:tasks.workunit.client.1.smithi043.stdout:9/962: creat d4/d21/d33/da2/d10a/f142 x:0 0 0 2024-09-16T23:49:10.749 INFO:tasks.workunit.client.1.smithi043.stdout:2/858: mknod d8/d4b/d4f/d7c/c120 0 2024-09-16T23:49:10.750 INFO:tasks.workunit.client.0.smithi042.stdout:5/945: write d9/db/d19/ff6 [1999793,37824] 0 2024-09-16T23:49:10.750 INFO:tasks.workunit.client.1.smithi043.stdout:9/963: readlink d4/d21/d33/da2/d10a/l131 0 2024-09-16T23:49:10.752 INFO:tasks.workunit.client.0.smithi042.stdout:9/663: creat d4/d12/d2d/fcd x:0 0 0 2024-09-16T23:49:10.755 INFO:tasks.workunit.client.1.smithi043.stdout:2/859: write d8/d1d/d20/d23/d38/da2/fdd [19026,85102] 0 2024-09-16T23:49:10.755 INFO:tasks.workunit.client.0.smithi042.stdout:5/946: mknod d9/db/de/d92/daf/d128/c146 0 2024-09-16T23:49:10.755 INFO:tasks.workunit.client.1.smithi043.stdout:3/960: dwrite d4/de/f58 [0,4194304] 0 2024-09-16T23:49:10.756 INFO:tasks.workunit.client.1.smithi043.stdout:9/964: creat d4/d1c/d25/d2b/d4c/f143 x:0 0 0 2024-09-16T23:49:10.757 INFO:tasks.workunit.client.0.smithi042.stdout:6/841: rename d8/d70/d81/d90 to d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d10c 0 2024-09-16T23:49:10.758 INFO:tasks.workunit.client.1.smithi043.stdout:9/965: readlink d4/d1c/d25/d2b/d4c/l78 0 2024-09-16T23:49:10.758 INFO:tasks.workunit.client.1.smithi043.stdout:2/860: dread - d8/d35/d9f/fdc zero size 2024-09-16T23:49:10.761 INFO:tasks.workunit.client.1.smithi043.stdout:2/861: write d8/d5e/d83/d9e/fef [4720965,75433] 0 2024-09-16T23:49:10.762 INFO:tasks.workunit.client.1.smithi043.stdout:9/966: creat d4/d28/d5f/f144 x:0 0 0 2024-09-16T23:49:10.763 INFO:tasks.workunit.client.0.smithi042.stdout:9/664: dwrite d4/d12/d2d/d94/dc2/d41/d92/fcc [0,4194304] 0 2024-09-16T23:49:10.766 INFO:tasks.workunit.client.0.smithi042.stdout:5/947: link d9/db/d19/d21/d85/dec/d112/d61/f5d d9/db/d19/d21/d85/dec/d112/d61/dcc/f147 0 2024-09-16T23:49:10.768 INFO:tasks.workunit.client.1.smithi043.stdout:2/862: dwrite d8/d5e/fe0 [0,4194304] 0 2024-09-16T23:49:10.769 INFO:tasks.workunit.client.0.smithi042.stdout:6/842: dwrite d8/d70/d81/dd/d1b/dbf/dde/d4f/f5b [0,4194304] 0 2024-09-16T23:49:10.771 INFO:tasks.workunit.client.1.smithi043.stdout:2/863: stat d8/d4b 0 2024-09-16T23:49:10.773 INFO:tasks.workunit.client.0.smithi042.stdout:5/948: creat d9/db/de/d10/d2f/f148 x:0 0 0 2024-09-16T23:49:10.778 INFO:tasks.workunit.client.0.smithi042.stdout:9/665: rename d4/d12/d2d/d94/dc2/d3a/dad to d4/d12/d2d/dc9/dce 0 2024-09-16T23:49:10.779 INFO:tasks.workunit.client.0.smithi042.stdout:6/843: dwrite d8/d70/d81/dd/d1b/d67/d97/dc9/fe9 [0,4194304] 0 2024-09-16T23:49:10.781 INFO:tasks.workunit.client.1.smithi043.stdout:2/864: dread d8/d4b/f5b [0,4194304] 0 2024-09-16T23:49:10.784 INFO:tasks.workunit.client.0.smithi042.stdout:0/778: sync 2024-09-16T23:49:10.784 INFO:tasks.workunit.client.1.smithi043.stdout:2/865: symlink d8/d1d/d20/l121 0 2024-09-16T23:49:10.785 INFO:tasks.workunit.client.0.smithi042.stdout:5/949: symlink d9/db/de/l149 0 2024-09-16T23:49:10.786 INFO:tasks.workunit.client.1.smithi043.stdout:9/967: dread d4/d21/d2a/d2c/f69 [4194304,4194304] 0 2024-09-16T23:49:10.787 INFO:tasks.workunit.client.1.smithi043.stdout:2/866: write d8/d11/d16/f19 [504788,29861] 0 2024-09-16T23:49:10.789 INFO:tasks.workunit.client.0.smithi042.stdout:0/779: chown d0/df/d13/d49/c5e 55 1 2024-09-16T23:49:10.789 INFO:tasks.workunit.client.0.smithi042.stdout:5/950: chown d9/db/d19/d21/d41/d7d/c86 5686438 1 2024-09-16T23:49:10.790 INFO:tasks.workunit.client.0.smithi042.stdout:5/951: stat d9/db/d1b/da4/de2/d13c 0 2024-09-16T23:49:10.791 INFO:tasks.workunit.client.1.smithi043.stdout:9/968: truncate d4/d1c/d25/d2b/da1/df9/f5b 1608628 0 2024-09-16T23:49:10.792 INFO:tasks.workunit.client.1.smithi043.stdout:9/969: chown d4/d1c/fe3 556717 1 2024-09-16T23:49:10.793 INFO:tasks.workunit.client.1.smithi043.stdout:9/970: write d4/d21/d2a/d2c/d7b/f111 [418727,67013] 0 2024-09-16T23:49:10.794 INFO:tasks.workunit.client.0.smithi042.stdout:6/844: rmdir d8/d70/d81/dd/dcd/d10a 0 2024-09-16T23:49:10.795 INFO:tasks.workunit.client.0.smithi042.stdout:0/780: creat d0/df/f102 x:0 0 0 2024-09-16T23:49:10.797 INFO:tasks.workunit.client.0.smithi042.stdout:6/845: write d8/d70/d81/dd/d2c/d57/f60 [14040,91432] 0 2024-09-16T23:49:10.797 INFO:tasks.workunit.client.1.smithi043.stdout:9/971: rmdir d4/d28/dcf/dd0 39 2024-09-16T23:49:10.798 INFO:tasks.workunit.client.1.smithi043.stdout:9/972: readlink d4/d84/d103/l40 0 2024-09-16T23:49:10.799 INFO:tasks.workunit.client.0.smithi042.stdout:0/781: creat d0/df/d23/d3e/f103 x:0 0 0 2024-09-16T23:49:10.802 INFO:tasks.workunit.client.0.smithi042.stdout:6/846: unlink d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/c6c 0 2024-09-16T23:49:10.804 INFO:tasks.workunit.client.0.smithi042.stdout:9/666: write d4/d10/f1f [2773329,42032] 0 2024-09-16T23:49:10.804 INFO:tasks.workunit.client.0.smithi042.stdout:5/952: write d9/db/d19/d21/d41/d7d/fbf [150877,16900] 0 2024-09-16T23:49:10.806 INFO:tasks.workunit.client.0.smithi042.stdout:9/667: chown d4/d12/d2d/dc9/dce 25 1 2024-09-16T23:49:10.807 INFO:tasks.workunit.client.1.smithi043.stdout:9/973: fsync d4/d28/ffc 0 2024-09-16T23:49:10.807 INFO:tasks.workunit.client.1.smithi043.stdout:2/867: write d8/d1d/d20/d23/d38/f66 [1307746,30076] 0 2024-09-16T23:49:10.807 INFO:tasks.workunit.client.0.smithi042.stdout:0/782: creat d0/f104 x:0 0 0 2024-09-16T23:49:10.807 INFO:tasks.workunit.client.1.smithi043.stdout:9/974: fdatasync d4/d21/d33/d95/fc9 0 2024-09-16T23:49:10.810 INFO:tasks.workunit.client.0.smithi042.stdout:5/953: mknod d9/db/d1b/c14a 0 2024-09-16T23:49:10.811 INFO:tasks.workunit.client.1.smithi043.stdout:9/975: rmdir d4/d1c/d25/d2b/d4c/dd2/de5 39 2024-09-16T23:49:10.813 INFO:tasks.workunit.client.0.smithi042.stdout:6/847: dwrite d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/fdd [0,4194304] 0 2024-09-16T23:49:10.814 INFO:tasks.workunit.client.1.smithi043.stdout:9/976: creat d4/d1c/d25/d2b/d4c/da3/dca/f145 x:0 0 0 2024-09-16T23:49:10.815 INFO:tasks.workunit.client.0.smithi042.stdout:5/954: write d9/db/d19/d21/d41/d7d/fbf [1043961,4489] 0 2024-09-16T23:49:10.815 INFO:tasks.workunit.client.1.smithi043.stdout:9/977: write d4/d1c/d25/db1/fe1 [2528143,116740] 0 2024-09-16T23:49:10.816 INFO:tasks.workunit.client.1.smithi043.stdout:2/868: write d8/d1d/d20/d23/d38/f47 [80827,73524] 0 2024-09-16T23:49:10.818 INFO:tasks.workunit.client.1.smithi043.stdout:9/978: write d4/d1c/d25/d2b/d4c/f72 [4627710,28650] 0 2024-09-16T23:49:10.818 INFO:tasks.workunit.client.1.smithi043.stdout:2/869: write d8/d1f/f6e [728070,83807] 0 2024-09-16T23:49:10.820 INFO:tasks.workunit.client.1.smithi043.stdout:2/870: write d8/d4b/d4f/d7c/f88 [4250172,71723] 0 2024-09-16T23:49:10.821 INFO:tasks.workunit.client.0.smithi042.stdout:0/783: dwrite d0/df/d23/d3e/d8f/fb6 [0,4194304] 0 2024-09-16T23:49:10.822 INFO:tasks.workunit.client.0.smithi042.stdout:6/848: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d10c/c10d 0 2024-09-16T23:49:10.823 INFO:tasks.workunit.client.1.smithi043.stdout:9/979: mknod d4/d28/d5f/c146 0 2024-09-16T23:49:10.824 INFO:tasks.workunit.client.0.smithi042.stdout:5/955: mknod d9/db/d1b/d31/d6c/d100/c14b 0 2024-09-16T23:49:10.825 INFO:tasks.workunit.client.1.smithi043.stdout:9/980: mknod d4/d28/d5f/d91/c147 0 2024-09-16T23:49:10.827 INFO:tasks.workunit.client.0.smithi042.stdout:9/668: truncate d4/d12/d16/d68/f72 3196124 0 2024-09-16T23:49:10.830 INFO:tasks.workunit.client.1.smithi043.stdout:9/981: creat d4/d1c/d25/d2b/d110/dbf/f148 x:0 0 0 2024-09-16T23:49:10.835 INFO:tasks.workunit.client.0.smithi042.stdout:0/784: dwrite d0/df/d13/d49/d70/ff6 [0,4194304] 0 2024-09-16T23:49:10.836 INFO:tasks.workunit.client.0.smithi042.stdout:6/849: dwrite d8/d70/d81/dd/d1b/d50/d96/f106 [0,4194304] 0 2024-09-16T23:49:10.838 INFO:tasks.workunit.client.0.smithi042.stdout:9/669: creat d4/d10/d24/fcf x:0 0 0 2024-09-16T23:49:10.840 INFO:tasks.workunit.client.1.smithi043.stdout:9/982: dwrite d4/d21/f89 [4194304,4194304] 0 2024-09-16T23:49:10.840 INFO:tasks.workunit.client.1.smithi043.stdout:2/871: dwrite d8/d1d/d20/d23/fb8 [0,4194304] 0 2024-09-16T23:49:10.840 INFO:tasks.workunit.client.0.smithi042.stdout:6/850: fdatasync d8/d70/d81/dd/d2c/f10 0 2024-09-16T23:49:10.841 INFO:tasks.workunit.client.0.smithi042.stdout:9/670: write d4/d10/faa [1082841,91964] 0 2024-09-16T23:49:10.845 INFO:tasks.workunit.client.1.smithi043.stdout:2/872: symlink d8/d1d/d20/d23/d38/da2/l122 0 2024-09-16T23:49:10.854 INFO:tasks.workunit.client.0.smithi042.stdout:5/956: link d9/db/d1b/d31/d6c/d100/l13d d9/db/de/d10/dc8/l14c 0 2024-09-16T23:49:10.854 INFO:tasks.workunit.client.0.smithi042.stdout:0/785: creat d0/df/d23/d3e/d8f/dda/f105 x:0 0 0 2024-09-16T23:49:10.858 INFO:tasks.workunit.client.0.smithi042.stdout:0/786: chown d0/df/d23/f55 3 1 2024-09-16T23:49:10.859 INFO:tasks.workunit.client.0.smithi042.stdout:0/787: chown d0/df/c12 532418308 1 2024-09-16T23:49:10.861 INFO:tasks.workunit.client.0.smithi042.stdout:6/851: dwrite d8/d70/d81/dd/d1b/d50/d96/f106 [4194304,4194304] 0 2024-09-16T23:49:10.862 INFO:tasks.workunit.client.0.smithi042.stdout:9/671: creat d4/d12/d2d/d94/dc2/d3a/fd0 x:0 0 0 2024-09-16T23:49:10.864 INFO:tasks.workunit.client.0.smithi042.stdout:0/788: write d0/df/d23/d33/d56/da9/f92 [2872546,119975] 0 2024-09-16T23:49:10.866 INFO:tasks.workunit.client.0.smithi042.stdout:0/789: write d0/df/d13/d2d/d35/d9f/fdc [452304,85978] 0 2024-09-16T23:49:10.872 INFO:tasks.workunit.client.0.smithi042.stdout:5/957: write d9/db/d19/d21/d41/fd0 [704490,44889] 0 2024-09-16T23:49:10.873 INFO:tasks.workunit.client.1.smithi043.stdout:2/873: dread d8/f43 [0,4194304] 0 2024-09-16T23:49:10.873 INFO:tasks.workunit.client.0.smithi042.stdout:9/672: dwrite d4/d12/d16/d68/f74 [0,4194304] 0 2024-09-16T23:49:10.874 INFO:tasks.workunit.client.1.smithi043.stdout:9/983: truncate d4/d21/f89 664194 0 2024-09-16T23:49:10.874 INFO:tasks.workunit.client.1.smithi043.stdout:7/949: sync 2024-09-16T23:49:10.875 INFO:tasks.workunit.client.0.smithi042.stdout:6/852: getdents d8/d70/d81/dd 0 2024-09-16T23:49:10.875 INFO:tasks.workunit.client.0.smithi042.stdout:0/790: dread d0/df/d23/d33/d56/da9/fdd [0,4194304] 0 2024-09-16T23:49:10.876 INFO:tasks.workunit.client.1.smithi043.stdout:7/950: chown d4/db/d11/d117/f11c 1033 1 2024-09-16T23:49:10.876 INFO:tasks.workunit.client.0.smithi042.stdout:9/673: fsync d4/d12/d60/d97/fa0 0 2024-09-16T23:49:10.876 INFO:tasks.workunit.client.0.smithi042.stdout:5/958: stat d9/db/d19/d21/l10c 0 2024-09-16T23:49:10.877 INFO:tasks.workunit.client.0.smithi042.stdout:5/959: fsync d9/db/f28 0 2024-09-16T23:49:10.878 INFO:tasks.workunit.client.0.smithi042.stdout:0/791: dread d0/df/d23/f55 [0,4194304] 0 2024-09-16T23:49:10.879 INFO:tasks.workunit.client.0.smithi042.stdout:5/960: readlink d9/db/l35 0 2024-09-16T23:49:10.881 INFO:tasks.workunit.client.0.smithi042.stdout:5/961: chown d9/db/d1b/d110 589852424 1 2024-09-16T23:49:10.881 INFO:tasks.workunit.client.0.smithi042.stdout:0/792: chown d0/df/d23/d33/d56/da9 1131505 1 2024-09-16T23:49:10.888 INFO:tasks.workunit.client.0.smithi042.stdout:5/962: mkdir d9/db/de/d5e/d10e/d14d 0 2024-09-16T23:49:10.888 INFO:tasks.workunit.client.0.smithi042.stdout:0/793: mknod d0/df/d23/c106 0 2024-09-16T23:49:10.889 INFO:tasks.workunit.client.0.smithi042.stdout:5/963: write d9/db/d19/d21/d41/fab [641391,62301] 0 2024-09-16T23:49:10.894 INFO:tasks.workunit.client.0.smithi042.stdout:6/853: creat d8/d70/d81/dd/d1b/d50/f10e x:0 0 0 2024-09-16T23:49:10.896 INFO:tasks.workunit.client.0.smithi042.stdout:0/794: write d0/df/d13/fe7 [1087,122574] 0 2024-09-16T23:49:10.897 INFO:tasks.workunit.client.0.smithi042.stdout:9/674: dread d4/d10/f4a [0,4194304] 0 2024-09-16T23:49:10.900 INFO:tasks.workunit.client.1.smithi043.stdout:7/951: dread d4/db/d50/fa9 [0,4194304] 0 2024-09-16T23:49:10.905 INFO:tasks.workunit.client.0.smithi042.stdout:6/854: write d8/d70/d81/dd/d1b/d50/f98 [2013896,58936] 0 2024-09-16T23:49:10.906 INFO:tasks.workunit.client.1.smithi043.stdout:7/952: rmdir d4/d27/d41/d1b 39 2024-09-16T23:49:10.911 INFO:tasks.workunit.client.0.smithi042.stdout:9/675: dwrite d4/d12/d2d/d51/fbe [0,4194304] 0 2024-09-16T23:49:10.911 INFO:tasks.workunit.client.0.smithi042.stdout:0/795: dwrite d0/df/f18 [0,4194304] 0 2024-09-16T23:49:10.911 INFO:tasks.workunit.client.1.smithi043.stdout:9/984: rename d4/d21/d2a/d2c/c7c to d4/d1c/d25/c149 0 2024-09-16T23:49:10.913 INFO:tasks.workunit.client.0.smithi042.stdout:5/964: dread d9/d2c/d7b/fa2 [0,4194304] 0 2024-09-16T23:49:10.914 INFO:tasks.workunit.client.1.smithi043.stdout:2/874: dwrite d8/d1d/d20/d26/f60 [4194304,4194304] 0 2024-09-16T23:49:10.916 INFO:tasks.workunit.client.1.smithi043.stdout:2/875: write d8/d1d/d20/d23/d38/da2/f11b [824190,52611] 0 2024-09-16T23:49:10.917 INFO:tasks.workunit.client.1.smithi043.stdout:2/876: write d8/d1d/d20/d26/f60 [4539330,117931] 0 2024-09-16T23:49:10.918 INFO:tasks.workunit.client.1.smithi043.stdout:9/985: fdatasync d4/d21/d2a/d2c/f9e 0 2024-09-16T23:49:10.919 INFO:tasks.workunit.client.0.smithi042.stdout:9/676: unlink d4/c4d 0 2024-09-16T23:49:10.920 INFO:tasks.workunit.client.1.smithi043.stdout:9/986: chown d4/d21/f89 1 1 2024-09-16T23:49:10.921 INFO:tasks.workunit.client.1.smithi043.stdout:9/987: readlink d4/d21/d35/dd4/l127 0 2024-09-16T23:49:10.921 INFO:tasks.workunit.client.0.smithi042.stdout:9/677: dread - d4/d12/d2d/dc9/dce/fca zero size 2024-09-16T23:49:10.922 INFO:tasks.workunit.client.0.smithi042.stdout:9/678: chown d4/d12/d60 43215 1 2024-09-16T23:49:10.926 INFO:tasks.workunit.client.0.smithi042.stdout:6/855: getdents d8/d70/d81/dd/d1b/d50/d96 0 2024-09-16T23:49:10.927 INFO:tasks.workunit.client.1.smithi043.stdout:8/924: sync 2024-09-16T23:49:10.928 INFO:tasks.workunit.client.0.smithi042.stdout:9/679: mkdir d4/d12/d77/dd1 0 2024-09-16T23:49:10.928 INFO:tasks.workunit.client.1.smithi043.stdout:9/988: symlink d4/d28/d5f/db6/l14a 0 2024-09-16T23:49:10.931 INFO:tasks.workunit.client.1.smithi043.stdout:7/953: dread d4/db/d11/d117/d85/d8b/fe7 [0,4194304] 0 2024-09-16T23:49:10.932 INFO:tasks.workunit.client.1.smithi043.stdout:2/877: creat d8/d1d/d25/f123 x:0 0 0 2024-09-16T23:49:10.933 INFO:tasks.workunit.client.1.smithi043.stdout:9/989: dread d4/d21/d33/d95/fc0 [0,4194304] 0 2024-09-16T23:49:10.934 INFO:tasks.workunit.client.1.smithi043.stdout:8/925: creat de/d18/d27/d41/d5a/d106/d39/d7b/d83/dbb/dce/de4/f119 x:0 0 0 2024-09-16T23:49:10.935 INFO:tasks.workunit.client.0.smithi042.stdout:9/680: unlink d4/d12/d2d/d94/dc2/f6a 0 2024-09-16T23:49:10.936 INFO:tasks.workunit.client.0.smithi042.stdout:5/965: getdents d9/db/de/d92/daf 0 2024-09-16T23:49:10.938 INFO:tasks.workunit.client.0.smithi042.stdout:9/681: chown d4/d12/d2d/d94/dc2/l4c 2987 1 2024-09-16T23:49:10.939 INFO:tasks.workunit.client.1.smithi043.stdout:9/990: dread - d4/d28/dcf/dd0/f13d zero size 2024-09-16T23:49:10.939 INFO:tasks.workunit.client.0.smithi042.stdout:0/796: dwrite d0/d48/f9c [0,4194304] 0 2024-09-16T23:49:10.939 INFO:tasks.workunit.client.0.smithi042.stdout:5/966: fdatasync d9/db/d19/d21/f2b 0 2024-09-16T23:49:10.941 INFO:tasks.workunit.client.0.smithi042.stdout:5/967: dread - d9/db/d1b/f97 zero size 2024-09-16T23:49:10.941 INFO:tasks.workunit.client.1.smithi043.stdout:8/926: symlink de/d18/d27/d41/d5a/d106/d39/d7b/d83/dbb/dce/de4/l11a 0 2024-09-16T23:49:10.944 INFO:tasks.workunit.client.1.smithi043.stdout:8/927: write de/d18/d27/d41/d5a/f13 [1759722,2930] 0 2024-09-16T23:49:10.944 INFO:tasks.workunit.client.1.smithi043.stdout:9/991: mkdir d4/d1c/d25/d2b/d110/dbf/d14b 0 2024-09-16T23:49:10.945 INFO:tasks.workunit.client.0.smithi042.stdout:7/831: sync 2024-09-16T23:49:10.949 INFO:tasks.workunit.client.0.smithi042.stdout:6/856: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/fdb 2659497 0 2024-09-16T23:49:10.950 INFO:tasks.workunit.client.1.smithi043.stdout:8/928: dread de/d18/d1f/d25/d65/d7f/f8c [0,4194304] 0 2024-09-16T23:49:10.951 INFO:tasks.workunit.client.1.smithi043.stdout:9/992: rename d4/d1c/d6b/f138 to d4/d21/d124/d128/f14c 0 2024-09-16T23:49:10.951 INFO:tasks.workunit.client.0.smithi042.stdout:0/797: rename d0/df/d23/f55 to d0/df/d23/f107 0 2024-09-16T23:49:10.952 INFO:tasks.workunit.client.1.smithi043.stdout:9/993: chown d4/d28/d49/f73 31748598 1 2024-09-16T23:49:10.952 INFO:tasks.workunit.client.0.smithi042.stdout:5/968: rmdir d9/db/de/d5e/d10e/d14d 0 2024-09-16T23:49:10.954 INFO:tasks.workunit.client.1.smithi043.stdout:8/929: dread - de/fc5 zero size 2024-09-16T23:49:10.954 INFO:tasks.workunit.client.0.smithi042.stdout:0/798: dread - d0/df/d13/d49/d70/da1/fea zero size 2024-09-16T23:49:10.957 INFO:tasks.workunit.client.0.smithi042.stdout:6/857: rename d8/d70/d81/dd/d1b/d50/fb9 to d8/d70/d81/dd/d1b/d50/d96/da0/dd0/f10f 0 2024-09-16T23:49:10.957 INFO:tasks.workunit.client.0.smithi042.stdout:9/682: dread d4/d12/d2d/d94/dc2/d28/f65 [8388608,4194304] 0 2024-09-16T23:49:10.957 INFO:tasks.workunit.client.1.smithi043.stdout:9/994: dwrite d4/d21/d33/d95/f9b [0,4194304] 0 2024-09-16T23:49:10.958 INFO:tasks.workunit.client.1.smithi043.stdout:8/930: creat de/d18/d27/d41/d5a/d106/d39/d7b/dd8/f11b x:0 0 0 2024-09-16T23:49:10.959 INFO:tasks.workunit.client.0.smithi042.stdout:5/969: chown d9/d11a/d12c/l132 27002727 1 2024-09-16T23:49:10.959 INFO:tasks.workunit.client.0.smithi042.stdout:7/832: dwrite d3/d1f/f3d [4194304,4194304] 0 2024-09-16T23:49:10.959 INFO:tasks.workunit.client.0.smithi042.stdout:0/799: truncate d0/df/d23/d33/d56/f61 4937947 0 2024-09-16T23:49:10.967 INFO:tasks.workunit.client.0.smithi042.stdout:5/970: unlink d9/db/d1b/d31/d46/d58/ff3 0 2024-09-16T23:49:10.967 INFO:tasks.workunit.client.0.smithi042.stdout:0/800: rmdir d0/df/d13/d49/d70/da1 39 2024-09-16T23:49:10.970 INFO:tasks.workunit.client.0.smithi042.stdout:5/971: write d9/db/d19/d21/d85/da5/f88 [464598,43874] 0 2024-09-16T23:49:10.970 INFO:tasks.workunit.client.0.smithi042.stdout:6/858: dwrite d8/d70/d81/dd/d1b/d50/f10e [0,4194304] 0 2024-09-16T23:49:10.971 INFO:tasks.workunit.client.0.smithi042.stdout:7/833: mknod d3/d98/da0/d24/d28/c123 0 2024-09-16T23:49:10.973 INFO:tasks.workunit.client.0.smithi042.stdout:1/862: sync 2024-09-16T23:49:10.973 INFO:tasks.workunit.client.0.smithi042.stdout:8/682: sync 2024-09-16T23:49:10.973 INFO:tasks.workunit.client.0.smithi042.stdout:2/739: sync 2024-09-16T23:49:10.973 INFO:tasks.workunit.client.0.smithi042.stdout:7/834: readlink d3/d1f/d49/d76/d19/d63/laa 0 2024-09-16T23:49:10.974 INFO:tasks.workunit.client.1.smithi043.stdout:9/995: truncate d4/d21/d35/d4b/f80 301710 0 2024-09-16T23:49:10.975 INFO:tasks.workunit.client.0.smithi042.stdout:9/683: rename d4/d12/d2d/d94/dc2/d28/l9e to d4/d10/d24/ld2 0 2024-09-16T23:49:10.978 INFO:tasks.workunit.client.0.smithi042.stdout:8/683: truncate da/d19/d1d/d5a/f79 3026366 0 2024-09-16T23:49:10.979 INFO:tasks.workunit.client.0.smithi042.stdout:9/684: truncate d4/d12/d2d/d94/dc2/d3a/fd0 523287 0 2024-09-16T23:49:10.981 INFO:tasks.workunit.client.0.smithi042.stdout:9/685: read d4/d10/f4a [2198383,69898] 0 2024-09-16T23:49:10.981 INFO:tasks.workunit.client.0.smithi042.stdout:6/859: mknod d8/d70/d81/dd/d1b/d34/d39/d5e/c110 0 2024-09-16T23:49:10.982 INFO:tasks.workunit.client.1.smithi043.stdout:2/878: dwrite d8/d35/fb9 [0,4194304] 0 2024-09-16T23:49:10.983 INFO:tasks.workunit.client.1.smithi043.stdout:7/954: dwrite d4/db/f5e [0,4194304] 0 2024-09-16T23:49:10.984 INFO:tasks.workunit.client.0.smithi042.stdout:0/801: creat d0/d15/de4/ddb/f108 x:0 0 0 2024-09-16T23:49:10.985 INFO:tasks.workunit.client.1.smithi043.stdout:9/996: fdatasync d4/d1c/d25/f62 0 2024-09-16T23:49:10.985 INFO:tasks.workunit.client.0.smithi042.stdout:8/684: dwrite da/d19/f1e [4194304,4194304] 0 2024-09-16T23:49:10.986 INFO:tasks.workunit.client.1.smithi043.stdout:8/931: dwrite de/d18/d1f/d25/d65/f8a [0,4194304] 0 2024-09-16T23:49:10.988 INFO:tasks.workunit.client.1.smithi043.stdout:9/997: write d4/d1c/d25/d2b/d4c/f143 [70257,126704] 0 2024-09-16T23:49:10.989 INFO:tasks.workunit.client.1.smithi043.stdout:3/961: sync 2024-09-16T23:49:10.993 INFO:tasks.workunit.client.0.smithi042.stdout:0/802: write d0/df/d13/dd6/fd8 [779519,105063] 0 2024-09-16T23:49:10.995 INFO:tasks.workunit.client.0.smithi042.stdout:7/835: creat d3/d98/da0/d24/d28/d9b/f124 x:0 0 0 2024-09-16T23:49:10.996 INFO:tasks.workunit.client.0.smithi042.stdout:1/863: mknod d0/d9/d90/d6b/d8e/df8/c135 0 2024-09-16T23:49:10.997 INFO:tasks.workunit.client.1.smithi043.stdout:8/932: mknod de/d18/d27/d41/c11c 0 2024-09-16T23:49:10.998 INFO:tasks.workunit.client.0.smithi042.stdout:6/860: dwrite d8/d70/d81/dd/d2c/ff3 [0,4194304] 0 2024-09-16T23:49:11.003 INFO:tasks.workunit.client.1.smithi043.stdout:7/955: dwrite d4/db/d11/d21/d31/d65/fdf [0,4194304] 0 2024-09-16T23:49:11.004 INFO:tasks.workunit.client.0.smithi042.stdout:8/685: fdatasync da/d19/d1d/f62 0 2024-09-16T23:49:11.005 INFO:tasks.workunit.client.1.smithi043.stdout:3/962: dwrite d4/f19 [0,4194304] 0 2024-09-16T23:49:11.007 INFO:tasks.workunit.client.0.smithi042.stdout:9/686: creat d4/d12/d77/dc4/fd3 x:0 0 0 2024-09-16T23:49:11.008 INFO:tasks.workunit.client.1.smithi043.stdout:2/879: creat d8/d5e/d83/d9e/dab/dd3/f124 x:0 0 0 2024-09-16T23:49:11.009 INFO:tasks.workunit.client.1.smithi043.stdout:9/998: symlink d4/d21/d35/l14d 0 2024-09-16T23:49:11.010 INFO:tasks.workunit.client.0.smithi042.stdout:7/836: mkdir d3/d1f/d49/d78/dbe/dc8/d125 0 2024-09-16T23:49:11.011 INFO:tasks.workunit.client.0.smithi042.stdout:0/803: dwrite d0/df/d13/d2d/d35/d9f/fb5 [0,4194304] 0 2024-09-16T23:49:11.013 INFO:tasks.workunit.client.0.smithi042.stdout:5/972: write d9/db/d19/fdb [1602885,90179] 0 2024-09-16T23:49:11.013 INFO:tasks.workunit.client.0.smithi042.stdout:6/861: mkdir d8/d70/d81/dd/d1b/d67/d97/d111 0 2024-09-16T23:49:11.015 INFO:tasks.workunit.client.0.smithi042.stdout:7/837: truncate d3/f9c 801981 0 2024-09-16T23:49:11.015 INFO:tasks.workunit.client.0.smithi042.stdout:1/864: creat d0/db2/f136 x:0 0 0 2024-09-16T23:49:11.016 INFO:tasks.workunit.client.1.smithi043.stdout:2/880: creat d8/d4b/d4f/d7c/f125 x:0 0 0 2024-09-16T23:49:11.017 INFO:tasks.workunit.client.0.smithi042.stdout:6/862: fsync d8/d70/d81/dd/d2c/d57/fca 0 2024-09-16T23:49:11.017 INFO:tasks.workunit.client.0.smithi042.stdout:9/687: chown d4/d12/d2d/d94/dc2/d3a/d50/c53 134 1 2024-09-16T23:49:11.020 INFO:tasks.workunit.client.0.smithi042.stdout:1/865: write d0/d9/d90/d6b/d88/dc2/f126 [360249,103972] 0 2024-09-16T23:49:11.023 INFO:tasks.workunit.client.0.smithi042.stdout:1/866: readlink d0/d17/l10e 0 2024-09-16T23:49:11.023 INFO:tasks.workunit.client.0.smithi042.stdout:5/973: creat d9/db/d1b/d31/d46/d58/d9d/f14e x:0 0 0 2024-09-16T23:49:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:10 smithi042 ceph-mon[27914]: pgmap v14: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 140 MiB/s rd, 354 MiB/s wr, 722 op/s 2024-09-16T23:49:11.024 INFO:tasks.workunit.client.0.smithi042.stdout:2/740: write d0/f22 [1807523,77629] 0 2024-09-16T23:49:11.024 INFO:tasks.workunit.client.0.smithi042.stdout:1/867: write d0/d17/d5e/f74 [881469,28814] 0 2024-09-16T23:49:11.025 INFO:tasks.workunit.client.0.smithi042.stdout:8/686: read da/d19/d1d/f2b [81199,42051] 0 2024-09-16T23:49:11.025 INFO:tasks.workunit.client.1.smithi043.stdout:7/956: link d4/db/l80 d4/d27/d41/l13d 0 2024-09-16T23:49:11.026 INFO:tasks.workunit.client.1.smithi043.stdout:9/999: dwrite d4/d1c/ddc/fef [0,4194304] 0 2024-09-16T23:49:11.028 INFO:tasks.workunit.client.0.smithi042.stdout:6/863: mkdir d8/d70/d81/dd/d1b/d67/d97/d112 0 2024-09-16T23:49:11.028 INFO:tasks.workunit.client.0.smithi042.stdout:9/688: chown d4/ld 1453908 1 2024-09-16T23:49:11.029 INFO:tasks.workunit.client.0.smithi042.stdout:2/741: truncate d0/da/d1b/d3b/d44/d54/fb1 244652 0 2024-09-16T23:49:11.032 INFO:tasks.workunit.client.0.smithi042.stdout:8/687: mknod da/d19/db6/cd5 0 2024-09-16T23:49:11.032 INFO:tasks.workunit.client.0.smithi042.stdout:9/689: chown d4/d12/d77 246049 1 2024-09-16T23:49:11.037 INFO:tasks.workunit.client.0.smithi042.stdout:6/864: readlink d8/d70/d81/dd/d1b/d34/d39/l92 0 2024-09-16T23:49:11.037 INFO:tasks.workunit.client.0.smithi042.stdout:8/688: mkdir da/d4b/d74/dd6 0 2024-09-16T23:49:11.038 INFO:tasks.workunit.client.1.smithi043.stdout:3/963: dwrite d4/fb8 [4194304,4194304] 0 2024-09-16T23:49:11.038 INFO:tasks.workunit.client.0.smithi042.stdout:9/690: write d4/d12/d2d/d94/dc2/d41/d92/fcc [3197785,3828] 0 2024-09-16T23:49:11.039 INFO:tasks.workunit.client.1.smithi043.stdout:8/933: write de/d18/f4c [3911811,75847] 0 2024-09-16T23:49:11.039 INFO:tasks.workunit.client.0.smithi042.stdout:8/689: stat da/d19/d1d/d30/l94 0 2024-09-16T23:49:11.043 INFO:tasks.workunit.client.1.smithi043.stdout:3/964: creat d4/de/d35/d10c/f128 x:0 0 0 2024-09-16T23:49:11.044 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:10 smithi043 ceph-mon[35191]: pgmap v14: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 140 MiB/s rd, 354 MiB/s wr, 722 op/s 2024-09-16T23:49:11.045 INFO:tasks.workunit.client.1.smithi043.stdout:2/881: write d8/d1d/d20/d26/f7d [1241373,84252] 0 2024-09-16T23:49:11.048 INFO:tasks.workunit.client.0.smithi042.stdout:1/868: unlink d0/d9/d90/d6b/d88/d32/d2f/de3/ffd 0 2024-09-16T23:49:11.049 INFO:tasks.workunit.client.0.smithi042.stdout:6/865: dread d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f80 [0,4194304] 0 2024-09-16T23:49:11.050 INFO:tasks.workunit.client.0.smithi042.stdout:0/804: dread d0/f7 [0,4194304] 0 2024-09-16T23:49:11.050 INFO:tasks.workunit.client.0.smithi042.stdout:2/742: mkdir d0/da/d17/d5c/df6 0 2024-09-16T23:49:11.050 INFO:tasks.workunit.client.1.smithi043.stdout:2/882: unlink d8/d4b/d5f/l73 0 2024-09-16T23:49:11.050 INFO:tasks.workunit.client.1.smithi043.stdout:7/957: dread d4/db/d28/f97 [0,4194304] 0 2024-09-16T23:49:11.050 INFO:tasks.workunit.client.0.smithi042.stdout:9/691: dwrite d4/d10/f3f [0,4194304] 0 2024-09-16T23:49:11.051 INFO:tasks.workunit.client.1.smithi043.stdout:2/883: chown d8/d5e/ffd 2227 1 2024-09-16T23:49:11.053 INFO:tasks.workunit.client.0.smithi042.stdout:7/838: write d3/d98/f71 [1155395,121190] 0 2024-09-16T23:49:11.053 INFO:tasks.workunit.client.0.smithi042.stdout:9/692: dread - d4/d10/d24/fcf zero size 2024-09-16T23:49:11.054 INFO:tasks.workunit.client.0.smithi042.stdout:2/743: fdatasync d0/f87 0 2024-09-16T23:49:11.056 INFO:tasks.workunit.client.0.smithi042.stdout:2/744: stat d0/d30/f55 0 2024-09-16T23:49:11.057 INFO:tasks.workunit.client.1.smithi043.stdout:2/884: mkdir d8/d4b/d5f/d63/db6/d126 0 2024-09-16T23:49:11.059 INFO:tasks.workunit.client.0.smithi042.stdout:0/805: mknod d0/df/d13/d2d/c109 0 2024-09-16T23:49:11.060 INFO:tasks.workunit.client.1.smithi043.stdout:7/958: symlink d4/d27/d41/l13e 0 2024-09-16T23:49:11.063 INFO:tasks.workunit.client.0.smithi042.stdout:9/693: chown d4/fb 1 1 2024-09-16T23:49:11.063 INFO:tasks.workunit.client.0.smithi042.stdout:1/869: dwrite d0/d9/d26/daa/dea/fee [0,4194304] 0 2024-09-16T23:49:11.065 INFO:tasks.workunit.client.0.smithi042.stdout:0/806: truncate d0/d15/f85 970955 0 2024-09-16T23:49:11.066 INFO:tasks.workunit.client.1.smithi043.stdout:3/965: dread d4/de/d1f/d2c/f32 [0,4194304] 0 2024-09-16T23:49:11.067 INFO:tasks.workunit.client.1.smithi043.stdout:3/966: chown d4/d30/fe9 4115639 1 2024-09-16T23:49:11.069 INFO:tasks.workunit.client.0.smithi042.stdout:5/974: dwrite d9/db/d19/d21/d41/d7d/f7e [0,4194304] 0 2024-09-16T23:49:11.071 INFO:tasks.workunit.client.1.smithi043.stdout:3/967: creat d4/de/d5d/def/f129 x:0 0 0 2024-09-16T23:49:11.072 INFO:tasks.workunit.client.0.smithi042.stdout:7/839: mkdir d3/d1f/d49/d76/d101/d64/d92/dfb/d126 0 2024-09-16T23:49:11.073 INFO:tasks.workunit.client.0.smithi042.stdout:2/745: mkdir d0/da/d1b/d3b/d44/dec/df7 0 2024-09-16T23:49:11.076 INFO:tasks.workunit.client.0.smithi042.stdout:6/866: creat d8/d70/d81/f113 x:0 0 0 2024-09-16T23:49:11.077 INFO:tasks.workunit.client.1.smithi043.stdout:7/959: write d4/d27/f6c [1898274,28760] 0 2024-09-16T23:49:11.079 INFO:tasks.workunit.client.0.smithi042.stdout:1/870: link d0/d9/d4e/f12b d0/d9/d90/d6b/d88/def/f137 0 2024-09-16T23:49:11.080 INFO:tasks.workunit.client.0.smithi042.stdout:1/871: dread - d0/d9a/f11c zero size 2024-09-16T23:49:11.082 INFO:tasks.workunit.client.0.smithi042.stdout:9/694: creat d4/d12/d60/fd4 x:0 0 0 2024-09-16T23:49:11.084 INFO:tasks.workunit.client.0.smithi042.stdout:6/867: creat d8/d70/d81/dd/d1b/d50/da2/f114 x:0 0 0 2024-09-16T23:49:11.085 INFO:tasks.workunit.client.1.smithi043.stdout:2/885: dwrite d8/d1d/d20/d26/f97 [0,4194304] 0 2024-09-16T23:49:11.087 INFO:tasks.workunit.client.1.smithi043.stdout:3/968: dwrite d4/de/d1f/d2c/f4a [0,4194304] 0 2024-09-16T23:49:11.087 INFO:tasks.workunit.client.0.smithi042.stdout:6/868: dread - d8/d70/d81/dd/d1b/d50/d96/da0/dd0/ff0 zero size 2024-09-16T23:49:11.089 INFO:tasks.workunit.client.0.smithi042.stdout:2/746: dread d0/d78/d8b/fd0 [0,4194304] 0 2024-09-16T23:49:11.089 INFO:tasks.workunit.client.0.smithi042.stdout:5/975: write d9/db/de/d92/fa1 [39374,41025] 0 2024-09-16T23:49:11.090 INFO:tasks.workunit.client.0.smithi042.stdout:0/807: getdents d0/df/d13/d49 0 2024-09-16T23:49:11.091 INFO:tasks.workunit.client.0.smithi042.stdout:0/808: chown d0/f60 27821228 1 2024-09-16T23:49:11.094 INFO:tasks.workunit.client.0.smithi042.stdout:1/872: dwrite d0/d3b/f82 [4194304,4194304] 0 2024-09-16T23:49:11.094 INFO:tasks.workunit.client.0.smithi042.stdout:9/695: creat d4/d12/d77/fd5 x:0 0 0 2024-09-16T23:49:11.095 INFO:tasks.workunit.client.1.smithi043.stdout:2/886: dwrite d8/d5e/ffd [0,4194304] 0 2024-09-16T23:49:11.097 INFO:tasks.workunit.client.0.smithi042.stdout:6/869: truncate d8/d70/d81/dd/d2c/d57/f108 421050 0 2024-09-16T23:49:11.100 INFO:tasks.workunit.client.0.smithi042.stdout:9/696: write d4/d12/d16/d68/f9f [1040654,34373] 0 2024-09-16T23:49:11.101 INFO:tasks.workunit.client.1.smithi043.stdout:3/969: link d4/de/d1f/d2c/c110 d4/de/d5d/def/c12a 0 2024-09-16T23:49:11.102 INFO:tasks.workunit.client.0.smithi042.stdout:5/976: unlink d9/db/d19/d21/d85/dec/d112/d61/f4e 0 2024-09-16T23:49:11.103 INFO:tasks.workunit.client.0.smithi042.stdout:0/809: truncate d0/df/d13/d49/d70/fb3 3250958 0 2024-09-16T23:49:11.108 INFO:tasks.workunit.client.1.smithi043.stdout:3/970: mknod d4/de/d1f/d2c/c12b 0 2024-09-16T23:49:11.109 INFO:tasks.workunit.client.0.smithi042.stdout:0/810: write d0/d48/fb1 [4164923,33163] 0 2024-09-16T23:49:11.110 INFO:tasks.workunit.client.0.smithi042.stdout:1/873: write d0/d9/d90/d6b/d88/dc2/f120 [343055,90731] 0 2024-09-16T23:49:11.111 INFO:tasks.workunit.client.0.smithi042.stdout:9/697: fdatasync d4/d10/f1b 0 2024-09-16T23:49:11.111 INFO:tasks.workunit.client.1.smithi043.stdout:7/960: dwrite d4/db/d11/d15/dda/fdd [0,4194304] 0 2024-09-16T23:49:11.114 INFO:tasks.workunit.client.0.smithi042.stdout:2/747: dwrite d0/d30/dcb/de5/fb5 [0,4194304] 0 2024-09-16T23:49:11.115 INFO:tasks.workunit.client.1.smithi043.stdout:7/961: chown d4/db/d11/d21/d31/d65/l98 3213 1 2024-09-16T23:49:11.116 INFO:tasks.workunit.client.0.smithi042.stdout:6/870: creat d8/d70/d81/dd/d2c/d57/f115 x:0 0 0 2024-09-16T23:49:11.118 INFO:tasks.workunit.client.1.smithi043.stdout:2/887: truncate d8/d1d/d20/d26/f3f 372890 0 2024-09-16T23:49:11.119 INFO:tasks.workunit.client.0.smithi042.stdout:6/871: write d8/d70/d81/dd/d2c/f10 [2258279,65339] 0 2024-09-16T23:49:11.119 INFO:tasks.workunit.client.1.smithi043.stdout:7/962: write d4/db/d11/d117/ffa [1573986,22756] 0 2024-09-16T23:49:11.120 INFO:tasks.workunit.client.1.smithi043.stdout:3/971: rename d4/d30/l63 to d4/d30/l12c 0 2024-09-16T23:49:11.122 INFO:tasks.workunit.client.1.smithi043.stdout:7/963: chown d4/db/d11/d15/dda 739112 1 2024-09-16T23:49:11.123 INFO:tasks.workunit.client.0.smithi042.stdout:9/698: mknod d4/d12/d77/dd1/cd6 0 2024-09-16T23:49:11.123 INFO:tasks.workunit.client.1.smithi043.stdout:2/888: creat d8/d5e/d83/d9e/dab/f127 x:0 0 0 2024-09-16T23:49:11.126 INFO:tasks.workunit.client.0.smithi042.stdout:1/874: dwrite d0/d9/d90/d6b/d88/d32/d2f/de3/f10c [0,4194304] 0 2024-09-16T23:49:11.126 INFO:tasks.workunit.client.1.smithi043.stdout:2/889: chown d8/d5e/d83/d3a/f72 16074949 1 2024-09-16T23:49:11.127 INFO:tasks.workunit.client.0.smithi042.stdout:5/977: rmdir d9/db/d1b/d31/d46/d58/dfa 0 2024-09-16T23:49:11.129 INFO:tasks.workunit.client.0.smithi042.stdout:6/872: dread d8/d70/d81/dd/d1b/dbf/dde/f51 [0,4194304] 0 2024-09-16T23:49:11.132 INFO:tasks.workunit.client.1.smithi043.stdout:7/964: dwrite d4/db/d11/fcf [0,4194304] 0 2024-09-16T23:49:11.133 INFO:tasks.workunit.client.0.smithi042.stdout:9/699: dwrite d4/d12/d2d/d94/dc2/d28/f90 [0,4194304] 0 2024-09-16T23:49:11.134 INFO:tasks.workunit.client.0.smithi042.stdout:9/700: chown d4/d12/d2d/d51 11462 1 2024-09-16T23:49:11.135 INFO:tasks.workunit.client.1.smithi043.stdout:2/890: truncate d8/d1d/d20/d23/d38/f66 4696511 0 2024-09-16T23:49:11.136 INFO:tasks.workunit.client.0.smithi042.stdout:0/811: rename d0/df/d13/d49/d70/c72 to d0/c10a 0 2024-09-16T23:49:11.137 INFO:tasks.workunit.client.0.smithi042.stdout:0/812: chown d0/df/d13/f34 4339180 1 2024-09-16T23:49:11.139 INFO:tasks.workunit.client.1.smithi043.stdout:7/965: mknod d4/dc0/d122/c13f 0 2024-09-16T23:49:11.142 INFO:tasks.workunit.client.0.smithi042.stdout:1/875: creat d0/d9/d26/daa/f138 x:0 0 0 2024-09-16T23:49:11.154 INFO:tasks.workunit.client.1.smithi043.stdout:3/972: write d4/de/d1f/d2c/db7/dc3/fe0 [120616,41615] 0 2024-09-16T23:49:11.156 INFO:tasks.workunit.client.0.smithi042.stdout:1/876: creat d0/d17/dc7/f139 x:0 0 0 2024-09-16T23:49:11.164 INFO:tasks.workunit.client.1.smithi043.stdout:7/966: dread d4/db/f18 [0,4194304] 0 2024-09-16T23:49:11.167 INFO:tasks.workunit.client.1.smithi043.stdout:2/891: dwrite d8/f1a [4194304,4194304] 0 2024-09-16T23:49:11.168 INFO:tasks.workunit.client.0.smithi042.stdout:0/813: symlink d0/df/d23/d33/d51/l10b 0 2024-09-16T23:49:11.168 INFO:tasks.workunit.client.0.smithi042.stdout:1/877: mkdir d0/d17/d79/d92/d13a 0 2024-09-16T23:49:11.170 INFO:tasks.workunit.client.0.smithi042.stdout:1/878: write d0/d9/d90/d6b/d88/f30 [1902977,128529] 0 2024-09-16T23:49:11.172 INFO:tasks.workunit.client.0.smithi042.stdout:5/978: write d9/db/d1b/d31/d6c/f93 [1839528,112983] 0 2024-09-16T23:49:11.173 INFO:tasks.workunit.client.0.smithi042.stdout:5/979: stat d9/db/d1b/d31/d46/d58/d120/c130 0 2024-09-16T23:49:11.177 INFO:tasks.workunit.client.0.smithi042.stdout:5/980: dread d9/d2c/d7b/fa2 [0,4194304] 0 2024-09-16T23:49:11.204 INFO:tasks.workunit.client.0.smithi042.stdout:6/873: link d8/d38/c6d d8/d38/c116 0 2024-09-16T23:49:11.210 INFO:tasks.workunit.client.0.smithi042.stdout:6/874: fsync d8/d70/d81/dd/d2c/f13 0 2024-09-16T23:49:11.212 INFO:tasks.workunit.client.0.smithi042.stdout:5/981: rename d9/c12a to d9/c14f 0 2024-09-16T23:49:11.214 INFO:tasks.workunit.client.0.smithi042.stdout:1/879: getdents d0/d9a 0 2024-09-16T23:49:11.215 INFO:tasks.workunit.client.0.smithi042.stdout:6/875: creat d8/d70/d81/dd/d1b/d50/d96/da0/f117 x:0 0 0 2024-09-16T23:49:11.216 INFO:tasks.workunit.client.0.smithi042.stdout:5/982: symlink d9/d63/l150 0 2024-09-16T23:49:11.216 INFO:tasks.workunit.client.0.smithi042.stdout:6/876: write d8/d70/d81/dd/d1b/d50/f98 [2264979,66706] 0 2024-09-16T23:49:11.217 INFO:tasks.workunit.client.0.smithi042.stdout:1/880: creat d0/d9/d4e/f13b x:0 0 0 2024-09-16T23:49:11.220 INFO:tasks.workunit.client.0.smithi042.stdout:5/983: read d9/db/d1b/f56 [1397285,78476] 0 2024-09-16T23:49:11.220 INFO:tasks.workunit.client.0.smithi042.stdout:6/877: fsync d8/d70/d81/dd/d1b/dbf/dde/d4f/d72/d99/fd9 0 2024-09-16T23:49:11.224 INFO:tasks.workunit.client.0.smithi042.stdout:6/878: mkdir d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d118 0 2024-09-16T23:49:11.226 INFO:tasks.workunit.client.0.smithi042.stdout:6/879: write d8/d70/d81/dd/d1b/dbf/dde/f6e [48750,20160] 0 2024-09-16T23:49:11.227 INFO:tasks.workunit.client.0.smithi042.stdout:6/880: write d8/d70/d81/dd/f11 [988801,103196] 0 2024-09-16T23:49:11.227 INFO:tasks.workunit.client.0.smithi042.stdout:1/881: link d0/d9/d90/d6b/d88/d63/cab d0/d17/d5e/c13c 0 2024-09-16T23:49:11.228 INFO:tasks.workunit.client.0.smithi042.stdout:1/882: readlink d0/d9a/lfa 0 2024-09-16T23:49:11.231 INFO:tasks.workunit.client.0.smithi042.stdout:1/883: unlink d0/f6 0 2024-09-16T23:49:11.231 INFO:tasks.workunit.client.0.smithi042.stdout:6/881: rename d8/d70/d81/dd/d2c/l88 to d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d10c/d9f/l119 0 2024-09-16T23:49:11.232 INFO:tasks.workunit.client.0.smithi042.stdout:1/884: chown d0/d9/d90/d6b/d88/d32/c54 29 1 2024-09-16T23:49:11.249 INFO:tasks.workunit.client.0.smithi042.stdout:6/882: write d8/d70/d81/dd/d1b/dbf/dde/d4f/f6b [631782,109781] 0 2024-09-16T23:49:11.249 INFO:tasks.workunit.client.0.smithi042.stdout:1/885: write d0/d9/d90/d6b/d88/d32/f33 [311777,81539] 0 2024-09-16T23:49:11.251 INFO:tasks.workunit.client.0.smithi042.stdout:0/814: dwrite d0/df/d23/f107 [0,4194304] 0 2024-09-16T23:49:11.254 INFO:tasks.workunit.client.0.smithi042.stdout:5/984: dwrite d9/db/d1b/da4/ffc [0,4194304] 0 2024-09-16T23:49:11.255 INFO:tasks.workunit.client.0.smithi042.stdout:5/985: dread - d9/db/de/d10/d2f/f140 zero size 2024-09-16T23:49:11.256 INFO:tasks.workunit.client.0.smithi042.stdout:1/886: creat d0/d17/d5e/d11f/f13d x:0 0 0 2024-09-16T23:49:11.261 INFO:tasks.workunit.client.0.smithi042.stdout:5/986: symlink d9/db/d1b/d31/d46/d58/d120/l151 0 2024-09-16T23:49:11.265 INFO:tasks.workunit.client.0.smithi042.stdout:5/987: mknod d9/db/d19/d21/d85/c152 0 2024-09-16T23:49:11.265 INFO:tasks.workunit.client.1.smithi043.stdout:7/967: truncate d4/db/d11/d117/d85/d8b/faf 3315938 0 2024-09-16T23:49:11.266 INFO:tasks.workunit.client.1.smithi043.stdout:7/968: fdatasync d4/db/d11/d15/d29/d7a/dcd/ff1 0 2024-09-16T23:49:11.270 INFO:tasks.workunit.client.0.smithi042.stdout:0/815: dread d0/df/d23/d33/d56/da9/fbc [0,4194304] 0 2024-09-16T23:49:11.270 INFO:tasks.workunit.client.0.smithi042.stdout:5/988: rename d9/d11a/d12c to d9/db/d19/d153 0 2024-09-16T23:49:11.271 INFO:tasks.workunit.client.0.smithi042.stdout:0/816: write d0/df/d13/f97 [3137182,102043] 0 2024-09-16T23:49:11.273 INFO:tasks.workunit.client.1.smithi043.stdout:7/969: dwrite d4/d27/f6c [0,4194304] 0 2024-09-16T23:49:11.274 INFO:tasks.workunit.client.1.smithi043.stdout:7/970: read - d4/db/da6/f13c zero size 2024-09-16T23:49:11.275 INFO:tasks.workunit.client.1.smithi043.stdout:7/971: stat d4/db/d11/d21/d31/d33/f113 0 2024-09-16T23:49:11.276 INFO:tasks.workunit.client.0.smithi042.stdout:6/883: creat d8/d70/d81/dd/d1b/d34/f11a x:0 0 0 2024-09-16T23:49:11.276 INFO:tasks.workunit.client.0.smithi042.stdout:5/989: mknod d9/db/d1b/d110/c154 0 2024-09-16T23:49:11.278 INFO:tasks.workunit.client.0.smithi042.stdout:1/887: write d0/d17/fa7 [928319,85738] 0 2024-09-16T23:49:11.278 INFO:tasks.workunit.client.1.smithi043.stdout:7/972: write d4/db/d28/f97 [4091266,29674] 0 2024-09-16T23:49:11.282 INFO:tasks.workunit.client.0.smithi042.stdout:6/884: dread d8/d70/d81/dd/d1b/dbf/dde/d4f/f6b [0,4194304] 0 2024-09-16T23:49:11.283 INFO:tasks.workunit.client.0.smithi042.stdout:1/888: symlink d0/d17/d79/l13e 0 2024-09-16T23:49:11.283 INFO:tasks.workunit.client.1.smithi043.stdout:7/973: mknod d4/db/d11/d117/d85/d8b/d118/c140 0 2024-09-16T23:49:11.284 INFO:tasks.workunit.client.0.smithi042.stdout:0/817: write d0/df/f19 [32594,70679] 0 2024-09-16T23:49:11.285 INFO:tasks.workunit.client.0.smithi042.stdout:5/990: dwrite d9/db/de/d10/f20 [0,4194304] 0 2024-09-16T23:49:11.287 INFO:tasks.workunit.client.0.smithi042.stdout:6/885: read d8/d70/d81/dd/dcd/fac [428628,24991] 0 2024-09-16T23:49:11.287 INFO:tasks.workunit.client.0.smithi042.stdout:1/889: creat d0/d17/d79/f13f x:0 0 0 2024-09-16T23:49:11.291 INFO:tasks.workunit.client.1.smithi043.stdout:7/974: dwrite d4/db/d11/d117/d85/d8b/d118/f12f [0,4194304] 0 2024-09-16T23:49:11.292 INFO:tasks.workunit.client.0.smithi042.stdout:0/818: rmdir d0/df/d23/d33/d56/dc5/dcd 39 2024-09-16T23:49:11.295 INFO:tasks.workunit.client.0.smithi042.stdout:1/890: creat d0/d9/d90/d6b/d8e/df8/f140 x:0 0 0 2024-09-16T23:49:11.296 INFO:tasks.workunit.client.0.smithi042.stdout:0/819: creat d0/df/ddf/f10c x:0 0 0 2024-09-16T23:49:11.301 INFO:tasks.workunit.client.0.smithi042.stdout:0/820: symlink d0/df/d13/dcb/l10d 0 2024-09-16T23:49:11.302 INFO:tasks.workunit.client.0.smithi042.stdout:6/886: mknod d8/d70/d81/dd/d1b/d67/c11b 0 2024-09-16T23:49:11.307 INFO:tasks.workunit.client.0.smithi042.stdout:5/991: dwrite d9/db/d19/d21/d41/d7d/f116 [0,4194304] 0 2024-09-16T23:49:11.308 INFO:tasks.workunit.client.0.smithi042.stdout:6/887: truncate d8/d70/d81/dd/d2c/d57/fbc 149284 0 2024-09-16T23:49:11.310 INFO:tasks.workunit.client.0.smithi042.stdout:1/891: dwrite d0/d3b/d8d/fcc [0,4194304] 0 2024-09-16T23:49:11.313 INFO:tasks.workunit.client.0.smithi042.stdout:5/992: dread d9/db/fb6 [0,4194304] 0 2024-09-16T23:49:11.314 INFO:tasks.workunit.client.0.smithi042.stdout:1/892: creat d0/ddb/ddd/f141 x:0 0 0 2024-09-16T23:49:11.314 INFO:tasks.workunit.client.0.smithi042.stdout:1/893: dread - d0/d9a/fcf zero size 2024-09-16T23:49:11.315 INFO:tasks.workunit.client.0.smithi042.stdout:6/888: mkdir d8/d70/d11c 0 2024-09-16T23:49:11.316 INFO:tasks.workunit.client.0.smithi042.stdout:5/993: symlink d9/db/d19/l155 0 2024-09-16T23:49:11.317 INFO:tasks.workunit.client.0.smithi042.stdout:5/994: read - d9/d2c/da9/f138 zero size 2024-09-16T23:49:11.319 INFO:tasks.workunit.client.0.smithi042.stdout:5/995: mknod d9/db/d1b/c156 0 2024-09-16T23:49:11.325 INFO:tasks.workunit.client.0.smithi042.stdout:1/894: dwrite d0/d9/d90/d6b/d88/d32/d2f/fb0 [0,4194304] 0 2024-09-16T23:49:11.327 INFO:tasks.workunit.client.0.smithi042.stdout:1/895: creat d0/d9/d90/f142 x:0 0 0 2024-09-16T23:49:11.329 INFO:tasks.workunit.client.0.smithi042.stdout:5/996: dread d9/db/d11/f40 [0,4194304] 0 2024-09-16T23:49:11.329 INFO:tasks.workunit.client.0.smithi042.stdout:1/896: chown d0/d17/d5e/c1a 17663 1 2024-09-16T23:49:11.334 INFO:tasks.workunit.client.0.smithi042.stdout:5/997: write d9/db/d19/d21/ff4 [889739,82258] 0 2024-09-16T23:49:11.338 INFO:tasks.workunit.client.0.smithi042.stdout:1/897: dwrite d0/d17/d18/f73 [0,4194304] 0 2024-09-16T23:49:11.338 INFO:tasks.workunit.client.0.smithi042.stdout:1/898: stat d0/d72/f110 0 2024-09-16T23:49:11.339 INFO:tasks.workunit.client.0.smithi042.stdout:1/899: chown d0/d9a 1941379 1 2024-09-16T23:49:11.339 INFO:tasks.workunit.client.0.smithi042.stdout:5/998: rename d9/d2c/d7b/fd2 to d9/db/d19/d21/d41/d7d/f157 0 2024-09-16T23:49:11.340 INFO:tasks.workunit.client.0.smithi042.stdout:5/999: stat d9/db/d1b/f8c 0 2024-09-16T23:49:11.341 INFO:tasks.workunit.client.0.smithi042.stdout:1/900: creat d0/d9/d26/d8b/dec/f143 x:0 0 0 2024-09-16T23:49:11.363 INFO:tasks.workunit.client.1.smithi043.stdout:7/975: dwrite d4/f62 [0,4194304] 0 2024-09-16T23:49:11.366 INFO:tasks.workunit.client.0.smithi042.stdout:1/901: dwrite d0/d9/d90/d66/f7f [0,4194304] 0 2024-09-16T23:49:11.366 INFO:tasks.workunit.client.1.smithi043.stdout:7/976: dread - d4/fdc zero size 2024-09-16T23:49:11.369 INFO:tasks.workunit.client.1.smithi043.stdout:7/977: symlink d4/d27/d8a/l141 0 2024-09-16T23:49:11.374 INFO:tasks.workunit.client.1.smithi043.stdout:7/978: symlink d4/db/l142 0 2024-09-16T23:49:11.375 INFO:tasks.workunit.client.0.smithi042.stdout:1/902: dwrite d0/d9/d90/d6b/d88/f30 [0,4194304] 0 2024-09-16T23:49:11.378 INFO:tasks.workunit.client.1.smithi043.stdout:7/979: fdatasync d4/db/d11/d21/d31/fb5 0 2024-09-16T23:49:11.378 INFO:tasks.workunit.client.0.smithi042.stdout:1/903: truncate d0/d9/d26/daa/dea/fee 4554748 0 2024-09-16T23:49:11.381 INFO:tasks.workunit.client.1.smithi043.stdout:7/980: creat d4/db/d11/d15/dd2/f143 x:0 0 0 2024-09-16T23:49:11.383 INFO:tasks.workunit.client.1.smithi043.stdout:7/981: read d4/db/d11/d21/d31/d65/fdf [1561387,58903] 0 2024-09-16T23:49:11.386 INFO:tasks.workunit.client.0.smithi042.stdout:1/904: link d0/d9/d90/d43/l49 d0/d9/d90/d66/l144 0 2024-09-16T23:49:11.387 INFO:tasks.workunit.client.1.smithi043.stdout:7/982: creat d4/db/d11/d21/d94/f144 x:0 0 0 2024-09-16T23:49:11.388 INFO:tasks.workunit.client.0.smithi042.stdout:1/905: truncate d0/d9/d90/fdc 413994 0 2024-09-16T23:49:11.390 INFO:tasks.workunit.client.1.smithi043.stdout:7/983: mknod d4/d27/d8a/d93/c145 0 2024-09-16T23:49:11.393 INFO:tasks.workunit.client.1.smithi043.stdout:7/984: truncate d4/db/d11/d21/d31/d33/f55 2883354 0 2024-09-16T23:49:11.394 INFO:tasks.workunit.client.0.smithi042.stdout:1/906: write d0/d9/f1c [766840,17165] 0 2024-09-16T23:49:11.397 INFO:tasks.workunit.client.0.smithi042.stdout:1/907: unlink d0/d17/f20 0 2024-09-16T23:49:11.398 INFO:tasks.workunit.client.1.smithi043.stdout:7/985: creat d4/db/d11/d15/d29/f146 x:0 0 0 2024-09-16T23:49:11.398 INFO:tasks.workunit.client.0.smithi042.stdout:1/908: rmdir d0/d9/d26/dc6 39 2024-09-16T23:49:11.399 INFO:tasks.workunit.client.1.smithi043.stdout:7/986: fdatasync d4/db/d11/d15/d29/d7a/dcd/f130 0 2024-09-16T23:49:11.399 INFO:tasks.workunit.client.1.smithi043.stdout:7/987: chown d4/db/d28/d10c/f119 13419 1 2024-09-16T23:49:11.399 INFO:tasks.workunit.client.0.smithi042.stdout:1/909: chown d0/d9/d90/d6b/d8e/df8/c135 951 1 2024-09-16T23:49:11.401 INFO:tasks.workunit.client.0.smithi042.stdout:1/910: write d0/d9/d90/d6b/d88/dc2/f126 [458666,130302] 0 2024-09-16T23:49:11.404 INFO:tasks.workunit.client.0.smithi042.stdout:1/911: mknod d0/d17/df1/c145 0 2024-09-16T23:49:11.405 INFO:tasks.workunit.client.0.smithi042.stdout:1/912: write d0/d9/d26/f84 [1702673,64362] 0 2024-09-16T23:49:11.407 INFO:tasks.workunit.client.1.smithi043.stdout:7/988: dwrite d4/d27/d41/d1b/d95/ff6 [0,4194304] 0 2024-09-16T23:49:11.411 INFO:tasks.workunit.client.1.smithi043.stdout:7/989: mkdir d4/db/d11/d15/dd2/d147 0 2024-09-16T23:49:11.412 INFO:tasks.workunit.client.1.smithi043.stdout:7/990: fdatasync d4/f11a 0 2024-09-16T23:49:11.414 INFO:tasks.workunit.client.0.smithi042.stdout:1/913: dwrite d0/d72/f110 [0,4194304] 0 2024-09-16T23:49:11.419 INFO:tasks.workunit.client.0.smithi042.stdout:1/914: mkdir d0/d17/dd4/d146 0 2024-09-16T23:49:11.419 INFO:tasks.workunit.client.1.smithi043.stdout:7/991: mkdir d4/db/d11/d15/d29/d148 0 2024-09-16T23:49:11.420 INFO:tasks.workunit.client.0.smithi042.stdout:1/915: read d0/d3b/d8d/fcc [2242,29633] 0 2024-09-16T23:49:11.423 INFO:tasks.workunit.client.1.smithi043.stdout:7/992: truncate d4/db/d11/d15/fd9 2759802 0 2024-09-16T23:49:11.424 INFO:tasks.workunit.client.0.smithi042.stdout:1/916: symlink d0/d9/d90/d6b/d88/d32/d2f/l147 0 2024-09-16T23:49:11.425 INFO:tasks.workunit.client.1.smithi043.stdout:7/993: fdatasync d4/db/d11/d21/d31/f82 0 2024-09-16T23:49:11.427 INFO:tasks.workunit.client.0.smithi042.stdout:1/917: mkdir d0/d9/d90/d66/de2/d148 0 2024-09-16T23:49:11.427 INFO:tasks.workunit.client.1.smithi043.stdout:7/994: creat d4/d27/d41/d110/f149 x:0 0 0 2024-09-16T23:49:11.427 INFO:tasks.workunit.client.1.smithi043.stdout:8/934: sync 2024-09-16T23:49:11.432 INFO:tasks.workunit.client.1.smithi043.stdout:7/995: link d4/db/d11/fcf d4/db/d28/d10c/f14a 0 2024-09-16T23:49:11.432 INFO:tasks.workunit.client.0.smithi042.stdout:1/918: dwrite d0/d9/d90/d6b/d88/f61 [0,4194304] 0 2024-09-16T23:49:11.433 INFO:tasks.workunit.client.1.smithi043.stdout:8/935: creat de/d18/d27/d41/d5a/d106/d39/d7b/dd8/d105/f11d x:0 0 0 2024-09-16T23:49:11.437 INFO:tasks.workunit.client.1.smithi043.stdout:8/936: mkdir de/d18/d27/d41/d5a/d37/d6a/d11e 0 2024-09-16T23:49:11.438 INFO:tasks.workunit.client.0.smithi042.stdout:1/919: rename d0/d17/d79/fa2 to d0/d17/d79/d92/d13a/f149 0 2024-09-16T23:49:11.439 INFO:tasks.workunit.client.0.smithi042.stdout:1/920: write d0/d17/d79/f13f [219282,85980] 0 2024-09-16T23:49:11.443 INFO:tasks.workunit.client.0.smithi042.stdout:1/921: fsync d0/d9/d90/d6b/d88/def/f137 0 2024-09-16T23:49:11.446 INFO:tasks.workunit.client.0.smithi042.stdout:1/922: symlink d0/d17/dd4/d146/l14a 0 2024-09-16T23:49:11.446 INFO:tasks.workunit.client.1.smithi043.stdout:8/937: link de/f90 de/d18/d27/d41/d5a/d106/d39/d7b/dd8/d105/f11f 0 2024-09-16T23:49:11.450 INFO:tasks.workunit.client.1.smithi043.stdout:8/938: creat de/d18/d27/d8e/f120 x:0 0 0 2024-09-16T23:49:11.450 INFO:tasks.workunit.client.0.smithi042.stdout:1/923: creat d0/d9/d26/f14b x:0 0 0 2024-09-16T23:49:11.451 INFO:tasks.workunit.client.0.smithi042.stdout:1/924: write d0/d9/d4e/f13b [990139,102031] 0 2024-09-16T23:49:11.452 INFO:tasks.workunit.client.1.smithi043.stdout:8/939: write de/f17 [3150791,75447] 0 2024-09-16T23:49:11.454 INFO:tasks.workunit.client.1.smithi043.stdout:7/996: dread d4/db/d11/d15/fd9 [0,4194304] 0 2024-09-16T23:49:11.457 INFO:tasks.workunit.client.1.smithi043.stdout:8/940: creat de/d18/d27/d8e/df3/d10a/d3d/f121 x:0 0 0 2024-09-16T23:49:11.460 INFO:tasks.workunit.client.1.smithi043.stdout:8/941: mknod de/d18/d27/d41/d5a/d106/d39/d7b/df6/c122 0 2024-09-16T23:49:11.460 INFO:tasks.workunit.client.1.smithi043.stdout:8/942: chown de/f17 0 1 2024-09-16T23:49:11.460 INFO:tasks.workunit.client.1.smithi043.stdout:8/943: stat de/d18/d1f/ld5 0 2024-09-16T23:49:11.461 INFO:tasks.workunit.client.1.smithi043.stdout:7/997: creat d4/db/d11/d15/f14b x:0 0 0 2024-09-16T23:49:11.462 INFO:tasks.workunit.client.1.smithi043.stdout:8/944: read de/d18/d27/d41/d5a/d106/d39/d7b/d83/da5/ff0 [2827031,82781] 0 2024-09-16T23:49:11.462 INFO:tasks.workunit.client.1.smithi043.stdout:3/973: sync 2024-09-16T23:49:11.463 INFO:tasks.workunit.client.0.smithi042.stdout:1/925: dwrite d0/d9/d90/d6b/d88/f2a [0,4194304] 0 2024-09-16T23:49:11.463 INFO:tasks.workunit.client.0.smithi042.stdout:8/690: sync 2024-09-16T23:49:11.464 INFO:tasks.workunit.client.1.smithi043.stdout:3/974: chown d4/de/d1f/c11c 1916013928 1 2024-09-16T23:49:11.468 INFO:tasks.workunit.client.0.smithi042.stdout:8/691: mkdir da/d19/d2c/dc8/dd7 0 2024-09-16T23:49:11.470 INFO:tasks.workunit.client.1.smithi043.stdout:8/945: unlink de/d18/d27/d41/d5a/d37/l66 0 2024-09-16T23:49:11.471 INFO:tasks.workunit.client.0.smithi042.stdout:1/926: write d0/d9/d90/d6b/d88/f76 [2946626,102819] 0 2024-09-16T23:49:11.472 INFO:tasks.workunit.client.1.smithi043.stdout:3/975: dwrite d4/de/d40/df0/d88/db1/dbb/fe4 [0,4194304] 0 2024-09-16T23:49:11.473 INFO:tasks.workunit.client.1.smithi043.stdout:3/976: read - d4/d30/f107 zero size 2024-09-16T23:49:11.474 INFO:tasks.workunit.client.0.smithi042.stdout:8/692: creat da/d19/d1d/d30/d49/db2/fd8 x:0 0 0 2024-09-16T23:49:11.478 INFO:tasks.workunit.client.1.smithi043.stdout:8/946: creat de/d18/d27/d41/d5a/d106/d39/f123 x:0 0 0 2024-09-16T23:49:11.479 INFO:tasks.workunit.client.0.smithi042.stdout:1/927: dwrite d0/db2/f136 [0,4194304] 0 2024-09-16T23:49:11.482 INFO:tasks.workunit.client.0.smithi042.stdout:8/693: creat da/d19/d1d/d30/d49/fd9 x:0 0 0 2024-09-16T23:49:11.482 INFO:tasks.workunit.client.0.smithi042.stdout:1/928: dread - d0/d9a/fcf zero size 2024-09-16T23:49:11.483 INFO:tasks.workunit.client.1.smithi043.stdout:7/998: dwrite d4/db/f14 [0,4194304] 0 2024-09-16T23:49:11.484 INFO:tasks.workunit.client.0.smithi042.stdout:1/929: chown d0/d9/d90/d66/de2/f119 0 1 2024-09-16T23:49:11.485 INFO:tasks.workunit.client.1.smithi043.stdout:7/999: fsync d4/d27/d41/d1b/d75/fde 0 2024-09-16T23:49:11.485 INFO:tasks.workunit.client.0.smithi042.stdout:1/930: truncate d0/d9/f12e 747754 0 2024-09-16T23:49:11.488 INFO:tasks.workunit.client.1.smithi043.stdout:3/977: rename d4/de/d35/d10c/f128 to d4/de/d40/df0/d88/db1/f12d 0 2024-09-16T23:49:11.489 INFO:tasks.workunit.client.1.smithi043.stdout:8/947: mknod de/d18/d27/df7/c124 0 2024-09-16T23:49:11.492 INFO:tasks.workunit.client.1.smithi043.stdout:3/978: read d4/de/d1f/d2c/fc4 [1446426,39030] 0 2024-09-16T23:49:11.493 INFO:tasks.workunit.client.0.smithi042.stdout:1/931: link d0/d9/fd9 d0/d9/d134/f14c 0 2024-09-16T23:49:11.496 INFO:tasks.workunit.client.1.smithi043.stdout:3/979: creat d4/de/d40/df0/d88/db1/dbb/f12e x:0 0 0 2024-09-16T23:49:11.500 INFO:tasks.workunit.client.1.smithi043.stdout:3/980: truncate d4/de/d1f/d34/f74 2886642 0 2024-09-16T23:49:11.501 INFO:tasks.workunit.client.0.smithi042.stdout:8/694: truncate da/f2a 1203627 0 2024-09-16T23:49:11.501 INFO:tasks.workunit.client.1.smithi043.stdout:3/981: write d4/de/d40/d4d/fbd [1744044,56505] 0 2024-09-16T23:49:11.504 INFO:tasks.workunit.client.1.smithi043.stdout:3/982: chown d4/de/d1f/d34/l92 28 1 2024-09-16T23:49:11.507 INFO:tasks.workunit.client.1.smithi043.stdout:8/948: dread de/d18/d27/d41/d5a/d106/d39/d7b/d83/f84 [0,4194304] 0 2024-09-16T23:49:11.509 INFO:tasks.workunit.client.1.smithi043.stdout:8/949: chown de/d18/d1f/d25/lb7 7 1 2024-09-16T23:49:11.510 INFO:tasks.workunit.client.1.smithi043.stdout:3/983: creat d4/de/d40/df0/d88/db1/f12f x:0 0 0 2024-09-16T23:49:11.510 INFO:tasks.workunit.client.0.smithi042.stdout:8/695: write da/d19/d1d/f62 [2477070,92492] 0 2024-09-16T23:49:11.511 INFO:tasks.workunit.client.1.smithi043.stdout:3/984: chown d4/de/d1f/d2c/f90 59482516 1 2024-09-16T23:49:11.511 INFO:tasks.workunit.client.0.smithi042.stdout:8/696: fsync da/d4b/fa9 0 2024-09-16T23:49:11.512 INFO:tasks.workunit.client.1.smithi043.stdout:3/985: dread d4/de/f51 [0,4194304] 0 2024-09-16T23:49:11.513 INFO:tasks.workunit.client.1.smithi043.stdout:3/986: chown d4/de/d35/d10c/f10e 224644 1 2024-09-16T23:49:11.514 INFO:tasks.workunit.client.1.smithi043.stdout:8/950: creat de/d18/d2b/f125 x:0 0 0 2024-09-16T23:49:11.521 INFO:tasks.workunit.client.0.smithi042.stdout:8/697: truncate da/d19/d2c/d56/fae 4183481 0 2024-09-16T23:49:11.523 INFO:tasks.workunit.client.0.smithi042.stdout:8/698: dread da/d19/d1d/f32 [0,4194304] 0 2024-09-16T23:49:11.525 INFO:tasks.workunit.client.1.smithi043.stdout:3/987: dread d4/de/d1f/d34/fad [0,4194304] 0 2024-09-16T23:49:11.526 INFO:tasks.workunit.client.1.smithi043.stdout:8/951: dread de/d18/d27/d41/d5a/d37/f99 [0,4194304] 0 2024-09-16T23:49:11.526 INFO:tasks.workunit.client.0.smithi042.stdout:8/699: link da/d2f/c9c da/d19/d2c/d56/dd4/cda 0 2024-09-16T23:49:11.528 INFO:tasks.workunit.client.0.smithi042.stdout:8/700: write da/d19/d1d/d30/d3f/dc2/fcd [4000043,71387] 0 2024-09-16T23:49:11.540 INFO:tasks.workunit.client.1.smithi043.stdout:3/988: dwrite d4/d30/fcc [0,4194304] 0 2024-09-16T23:49:11.543 INFO:tasks.workunit.client.0.smithi042.stdout:8/701: dread da/d19/d1d/d5a/d99/f9a [0,4194304] 0 2024-09-16T23:49:11.556 INFO:tasks.workunit.client.1.smithi043.stdout:3/989: dwrite d4/de/d1f/d34/fdd [0,4194304] 0 2024-09-16T23:49:11.557 INFO:tasks.workunit.client.0.smithi042.stdout:8/702: dwrite da/d19/d2c/d56/d97/dad/fbf [0,4194304] 0 2024-09-16T23:49:11.560 INFO:tasks.workunit.client.1.smithi043.stdout:3/990: symlink d4/de/dea/l130 0 2024-09-16T23:49:11.560 INFO:tasks.workunit.client.0.smithi042.stdout:8/703: dread - da/d19/fcc zero size 2024-09-16T23:49:11.562 INFO:tasks.workunit.client.1.smithi043.stdout:3/991: mkdir d4/de/d5d/d131 0 2024-09-16T23:49:11.564 INFO:tasks.workunit.client.0.smithi042.stdout:8/704: symlink da/d19/d1d/d5a/ldb 0 2024-09-16T23:49:11.570 INFO:tasks.workunit.client.1.smithi043.stdout:3/992: creat d4/de/d40/df0/d88/f132 x:0 0 0 2024-09-16T23:49:11.570 INFO:tasks.workunit.client.1.smithi043.stdout:3/993: dread - d4/de/d35/f112 zero size 2024-09-16T23:49:11.575 INFO:tasks.workunit.client.0.smithi042.stdout:8/705: link da/d19/d1d/d30/d49/f85 da/d19/d1d/d30/d3f/d6d/fdc 0 2024-09-16T23:49:11.575 INFO:tasks.workunit.client.1.smithi043.stdout:3/994: dwrite d4/de/d40/d4d/fda [0,4194304] 0 2024-09-16T23:49:11.576 INFO:tasks.workunit.client.0.smithi042.stdout:8/706: stat da/cd1 0 2024-09-16T23:49:11.577 INFO:tasks.workunit.client.1.smithi043.stdout:3/995: read d4/de/d1f/d2c/db7/dc3/fe0 [789019,43549] 0 2024-09-16T23:49:11.579 INFO:tasks.workunit.client.0.smithi042.stdout:8/707: symlink da/d19/d2c/d56/d97/ldd 0 2024-09-16T23:49:11.584 INFO:tasks.workunit.client.1.smithi043.stdout:3/996: dread d4/de/d1f/f3e [0,4194304] 0 2024-09-16T23:49:11.586 INFO:tasks.workunit.client.0.smithi042.stdout:8/708: write da/f90 [765647,64743] 0 2024-09-16T23:49:11.586 INFO:tasks.workunit.client.1.smithi043.stdout:3/997: mknod d4/de/d40/d4d/c133 0 2024-09-16T23:49:11.588 INFO:tasks.workunit.client.0.smithi042.stdout:8/709: mknod da/d4b/d74/cde 0 2024-09-16T23:49:11.592 INFO:tasks.workunit.client.0.smithi042.stdout:8/710: stat da/d19/d1d/c27 0 2024-09-16T23:49:11.592 INFO:tasks.workunit.client.0.smithi042.stdout:8/711: chown da/d19/d2c/d56/d97/ca0 34134792 1 2024-09-16T23:49:11.593 INFO:tasks.workunit.client.1.smithi043.stdout:3/998: truncate d4/de/d40/df0/f5c 585002 0 2024-09-16T23:49:11.594 INFO:tasks.workunit.client.0.smithi042.stdout:8/712: mkdir da/d19/d1d/d5a/d99/ddf 0 2024-09-16T23:49:11.596 INFO:tasks.workunit.client.0.smithi042.stdout:8/713: dread da/d19/d1d/d30/d3f/d6d/f76 [0,4194304] 0 2024-09-16T23:49:11.596 INFO:tasks.workunit.client.0.smithi042.stdout:8/714: chown da/d19/f1a 2500164 1 2024-09-16T23:49:11.599 INFO:tasks.workunit.client.0.smithi042.stdout:8/715: symlink da/d4b/le0 0 2024-09-16T23:49:11.600 INFO:tasks.workunit.client.1.smithi043.stdout:3/999: dwrite d4/de/d40/fbf [0,4194304] 0 2024-09-16T23:49:11.602 INFO:tasks.workunit.client.0.smithi042.stdout:8/716: dread da/f90 [0,4194304] 0 2024-09-16T23:49:11.606 INFO:tasks.workunit.client.0.smithi042.stdout:8/717: unlink da/d19/f1f 0 2024-09-16T23:49:11.667 INFO:tasks.workunit.client.0.smithi042.stdout:2/748: sync 2024-09-16T23:49:11.667 INFO:tasks.workunit.client.0.smithi042.stdout:9/701: sync 2024-09-16T23:49:11.667 INFO:tasks.workunit.client.0.smithi042.stdout:0/821: sync 2024-09-16T23:49:11.667 INFO:tasks.workunit.client.0.smithi042.stdout:6/889: sync 2024-09-16T23:49:11.667 INFO:tasks.workunit.client.0.smithi042.stdout:7/840: sync 2024-09-16T23:49:11.668 INFO:tasks.workunit.client.0.smithi042.stdout:7/841: chown d3/d3f/cc1 0 1 2024-09-16T23:49:11.669 INFO:tasks.workunit.client.0.smithi042.stdout:7/842: readlink d3/d1f/l50 0 2024-09-16T23:49:11.669 INFO:tasks.workunit.client.0.smithi042.stdout:6/890: mkdir d8/d70/d84/d11d 0 2024-09-16T23:49:11.670 INFO:tasks.workunit.client.0.smithi042.stdout:0/822: symlink d0/df/d23/d33/d56/l10e 0 2024-09-16T23:49:11.676 INFO:tasks.workunit.client.0.smithi042.stdout:7/843: dwrite d3/d1f/d49/d76/f25 [0,4194304] 0 2024-09-16T23:49:11.677 INFO:tasks.workunit.client.0.smithi042.stdout:7/844: chown d3/d98/da0/d24/d11a/f11f 855654670 1 2024-09-16T23:49:11.678 INFO:tasks.workunit.client.1.smithi043.stdout:2/892: sync 2024-09-16T23:49:11.680 INFO:tasks.workunit.client.1.smithi043.stdout:2/893: readlink d8/d1d/lf1 0 2024-09-16T23:49:11.681 INFO:tasks.workunit.client.1.smithi043.stdout:2/894: chown d8/d4b/d4f/d7c/c8a 44528 1 2024-09-16T23:49:11.684 INFO:tasks.workunit.client.0.smithi042.stdout:0/823: mkdir d0/d10f 0 2024-09-16T23:49:11.686 INFO:tasks.workunit.client.1.smithi043.stdout:2/895: unlink d8/d4b/d4f/d56/f10b 0 2024-09-16T23:49:11.687 INFO:tasks.workunit.client.0.smithi042.stdout:9/702: rename d4/d10/l13 to d4/d12/d60/ld7 0 2024-09-16T23:49:11.688 INFO:tasks.workunit.client.1.smithi043.stdout:2/896: readlink d8/d1d/d20/d23/d38/le3 0 2024-09-16T23:49:11.688 INFO:tasks.workunit.client.0.smithi042.stdout:0/824: dread d0/df/d23/d33/d56/da9/fdd [0,4194304] 0 2024-09-16T23:49:11.688 INFO:tasks.workunit.client.0.smithi042.stdout:2/749: link d0/da/d17/d3e/d86/cb9 d0/da/d1b/d3b/d44/df2/cf8 0 2024-09-16T23:49:11.689 INFO:tasks.workunit.client.1.smithi043.stdout:2/897: dread - d8/d4b/d5f/f11f zero size 2024-09-16T23:49:11.689 INFO:tasks.workunit.client.0.smithi042.stdout:6/891: dwrite d8/d70/d81/dd/d1b/d50/d96/da0/dd0/f10f [0,4194304] 0 2024-09-16T23:49:11.691 INFO:tasks.workunit.client.1.smithi043.stdout:2/898: symlink d8/d11/dca/l128 0 2024-09-16T23:49:11.692 INFO:tasks.workunit.client.0.smithi042.stdout:7/845: truncate d3/d98/da0/d24/f10a 558742 0 2024-09-16T23:49:11.694 INFO:tasks.workunit.client.1.smithi043.stdout:2/899: rmdir d8/d35/d9f 39 2024-09-16T23:49:11.694 INFO:tasks.workunit.client.0.smithi042.stdout:2/750: creat d0/d30/d43/df0/ff9 x:0 0 0 2024-09-16T23:49:11.696 INFO:tasks.workunit.client.0.smithi042.stdout:0/825: unlink d0/d15/de4/ddb/le0 0 2024-09-16T23:49:11.696 INFO:tasks.workunit.client.1.smithi043.stdout:2/900: mknod d8/d4b/d5f/d8e/c129 0 2024-09-16T23:49:11.699 INFO:tasks.workunit.client.0.smithi042.stdout:9/703: rename d4/d12/d2d/d94/dc2/d59/f88 to d4/d12/d2d/d94/dc2/d3a/d50/fd8 0 2024-09-16T23:49:11.699 INFO:tasks.workunit.client.1.smithi043.stdout:2/901: read d8/d11/d16/f19 [92896,35464] 0 2024-09-16T23:49:11.701 INFO:tasks.workunit.client.0.smithi042.stdout:2/751: symlink d0/da/d17/d3e/lfa 0 2024-09-16T23:49:11.702 INFO:tasks.workunit.client.0.smithi042.stdout:6/892: unlink d8/d70/d81/dd/d2c/d57/f94 0 2024-09-16T23:49:11.703 INFO:tasks.workunit.client.1.smithi043.stdout:2/902: rmdir d8/d5e 39 2024-09-16T23:49:11.704 INFO:tasks.workunit.client.1.smithi043.stdout:2/903: dread - d8/d4b/d4f/d7c/f125 zero size 2024-09-16T23:49:11.704 INFO:tasks.workunit.client.0.smithi042.stdout:6/893: truncate d8/d70/d81/dd/d2c/d57/f115 907143 0 2024-09-16T23:49:11.705 INFO:tasks.workunit.client.0.smithi042.stdout:9/704: unlink d4/d10/f30 0 2024-09-16T23:49:11.706 INFO:tasks.workunit.client.0.smithi042.stdout:7/846: symlink d3/d98/da0/d24/dc3/l127 0 2024-09-16T23:49:11.707 INFO:tasks.workunit.client.0.smithi042.stdout:2/752: fdatasync d0/da/d17/d5c/fce 0 2024-09-16T23:49:11.710 INFO:tasks.workunit.client.0.smithi042.stdout:6/894: mkdir d8/d38/d11e 0 2024-09-16T23:49:11.713 INFO:tasks.workunit.client.0.smithi042.stdout:7/847: rmdir d3/d3f 39 2024-09-16T23:49:11.716 INFO:tasks.workunit.client.0.smithi042.stdout:2/753: rmdir d0/da/d88/dae 39 2024-09-16T23:49:11.718 INFO:tasks.workunit.client.0.smithi042.stdout:0/826: rename d0/df/c8a to d0/df/d13/c110 0 2024-09-16T23:49:11.719 INFO:tasks.workunit.client.1.smithi043.stdout:2/904: dread d8/d4b/d5f/f61 [4194304,4194304] 0 2024-09-16T23:49:11.719 INFO:tasks.workunit.client.0.smithi042.stdout:9/705: creat d4/d12/d2d/d51/db9/fd9 x:0 0 0 2024-09-16T23:49:11.720 INFO:tasks.workunit.client.1.smithi043.stdout:2/905: write d8/f1a [47760,19781] 0 2024-09-16T23:49:11.721 INFO:tasks.workunit.client.0.smithi042.stdout:7/848: symlink d3/d1f/d49/d76/d101/d64/d92/l128 0 2024-09-16T23:49:11.723 INFO:tasks.workunit.client.1.smithi043.stdout:2/906: read d8/d4b/d5f/d8e/de8/feb [1854619,26780] 0 2024-09-16T23:49:11.723 INFO:tasks.workunit.client.0.smithi042.stdout:2/754: symlink d0/da/d17/d3e/d86/d96/lfb 0 2024-09-16T23:49:11.724 INFO:tasks.workunit.client.0.smithi042.stdout:0/827: dwrite d0/d15/de4/ddb/f108 [0,4194304] 0 2024-09-16T23:49:11.724 INFO:tasks.workunit.client.0.smithi042.stdout:0/828: chown d0/df 196 1 2024-09-16T23:49:11.725 INFO:tasks.workunit.client.1.smithi043.stdout:2/907: symlink d8/d4b/d5f/d63/db6/d126/l12a 0 2024-09-16T23:49:11.728 INFO:tasks.workunit.client.0.smithi042.stdout:0/829: truncate d0/df/d23/d3e/d8f/dda/f105 594809 0 2024-09-16T23:49:11.729 INFO:tasks.workunit.client.0.smithi042.stdout:7/849: write d3/d3f/f10e [2919682,107230] 0 2024-09-16T23:49:11.730 INFO:tasks.workunit.client.1.smithi043.stdout:8/952: sync 2024-09-16T23:49:11.731 INFO:tasks.workunit.client.1.smithi043.stdout:8/953: chown de/d18/d27/d41/d5a/d106/l64 705486 1 2024-09-16T23:49:11.731 INFO:tasks.workunit.client.0.smithi042.stdout:6/895: rename d8/d70/d81/dd/d1b/dbf/dde/d4f to d8/d70/d81/dd/d1b/dbf/d11f 0 2024-09-16T23:49:11.732 INFO:tasks.workunit.client.1.smithi043.stdout:2/908: getdents d8/d5e/d83/d9e 0 2024-09-16T23:49:11.732 INFO:tasks.workunit.client.0.smithi042.stdout:6/896: readlink d8/d70/d81/dd/d1b/d67/lc6 0 2024-09-16T23:49:11.733 INFO:tasks.workunit.client.1.smithi043.stdout:8/954: dread - de/d18/d27/d41/d5a/dc8/deb/f117 zero size 2024-09-16T23:49:11.733 INFO:tasks.workunit.client.0.smithi042.stdout:6/897: chown d8/d70/d81/dd/d1b/d2d/c65 805471845 1 2024-09-16T23:49:11.734 INFO:tasks.workunit.client.1.smithi043.stdout:8/955: read - de/d18/d2b/f125 zero size 2024-09-16T23:49:11.735 INFO:tasks.workunit.client.1.smithi043.stdout:8/956: chown de/d18/d27/l6d 14795487 1 2024-09-16T23:49:11.735 INFO:tasks.workunit.client.1.smithi043.stdout:2/909: mknod d8/d35/d9f/c12b 0 2024-09-16T23:49:11.736 INFO:tasks.workunit.client.1.smithi043.stdout:2/910: chown d8/d1d/d20/d23/l71 920 1 2024-09-16T23:49:11.739 INFO:tasks.workunit.client.0.smithi042.stdout:9/706: rename d4/d12/d77/c84 to d4/d12/d2d/dc9/dce/d8e/cda 0 2024-09-16T23:49:11.746 INFO:tasks.workunit.client.0.smithi042.stdout:2/755: link d0/c1e d0/da/d1b/d3b/d44/d54/df3/cfc 0 2024-09-16T23:49:11.746 INFO:tasks.workunit.client.0.smithi042.stdout:9/707: rmdir d4/d12/d2d/d94 39 2024-09-16T23:49:11.748 INFO:tasks.workunit.client.0.smithi042.stdout:9/708: fdatasync d4/d12/d2d/d51/db9/fd9 0 2024-09-16T23:49:11.748 INFO:tasks.workunit.client.0.smithi042.stdout:7/850: getdents d3/d98/da0/d41 0 2024-09-16T23:49:11.749 INFO:tasks.workunit.client.0.smithi042.stdout:6/898: link d8/d70/d81/dd/d1b/dbf/dde/c103 d8/d70/d81/dd/d1b/d67/d97/d112/c120 0 2024-09-16T23:49:11.750 INFO:tasks.workunit.client.0.smithi042.stdout:0/830: dwrite d0/df/d23/d3e/f54 [0,4194304] 0 2024-09-16T23:49:11.750 INFO:tasks.workunit.client.1.smithi043.stdout:2/911: dread d8/d4b/d5f/d63/f99 [0,4194304] 0 2024-09-16T23:49:11.751 INFO:tasks.workunit.client.0.smithi042.stdout:2/756: symlink d0/da/d17/lfd 0 2024-09-16T23:49:11.751 INFO:tasks.workunit.client.0.smithi042.stdout:6/899: write d8/d70/d81/f113 [292049,98867] 0 2024-09-16T23:49:11.752 INFO:tasks.workunit.client.1.smithi043.stdout:2/912: write d8/d11/dca/f115 [241696,116670] 0 2024-09-16T23:49:11.753 INFO:tasks.workunit.client.1.smithi043.stdout:2/913: dread - d8/dec/f11d zero size 2024-09-16T23:49:11.754 INFO:tasks.workunit.client.0.smithi042.stdout:2/757: write d0/da/d1b/d3b/d44/d54/fb1 [640311,61945] 0 2024-09-16T23:49:11.756 INFO:tasks.workunit.client.1.smithi043.stdout:2/914: write d8/d5e/d83/d9e/dab/fed [1905200,101453] 0 2024-09-16T23:49:11.757 INFO:tasks.workunit.client.0.smithi042.stdout:0/831: unlink d0/df/d23/d33/d51/l10b 0 2024-09-16T23:49:11.758 INFO:tasks.workunit.client.0.smithi042.stdout:0/832: stat d0/df/d13/d49/l4f 0 2024-09-16T23:49:11.759 INFO:tasks.workunit.client.0.smithi042.stdout:9/709: chown d4/d12/d2d/dc9/dce/d8e/cda 0 1 2024-09-16T23:49:11.759 INFO:tasks.workunit.client.0.smithi042.stdout:9/710: fsync d4/d10/d24/f35 0 2024-09-16T23:49:11.762 INFO:tasks.workunit.client.1.smithi043.stdout:2/915: rename d8/d5e/d83/d9e/dab/le1 to d8/d5e/d83/d3a/d77/da9/l12c 0 2024-09-16T23:49:11.763 INFO:tasks.workunit.client.1.smithi043.stdout:2/916: fsync d8/d5e/d83/d9e/fef 0 2024-09-16T23:49:11.764 INFO:tasks.workunit.client.1.smithi043.stdout:2/917: fsync d8/d1d/d20/d23/d38/da2/fe9 0 2024-09-16T23:49:11.765 INFO:tasks.workunit.client.1.smithi043.stdout:2/918: chown d8/d4b 449906260 1 2024-09-16T23:49:11.766 INFO:tasks.workunit.client.0.smithi042.stdout:2/758: rename d0/d30/f55 to d0/d26/df4/ffe 0 2024-09-16T23:49:11.768 INFO:tasks.workunit.client.0.smithi042.stdout:7/851: link d3/d98/fed d3/d1f/d49/d76/d101/d64/d92/dfb/f129 0 2024-09-16T23:49:11.770 INFO:tasks.workunit.client.0.smithi042.stdout:6/900: dread d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f73 [4194304,4194304] 0 2024-09-16T23:49:11.773 INFO:tasks.workunit.client.0.smithi042.stdout:7/852: symlink d3/d1f/d49/d76/d101/d64/d92/l12a 0 2024-09-16T23:49:11.773 INFO:tasks.workunit.client.0.smithi042.stdout:2/759: mkdir d0/da/d88/dae/dff 0 2024-09-16T23:49:11.774 INFO:tasks.workunit.client.0.smithi042.stdout:6/901: write d8/d70/d81/dd/d1b/f102 [983073,104915] 0 2024-09-16T23:49:11.775 INFO:tasks.workunit.client.0.smithi042.stdout:0/833: dwrite d0/df/f10 [0,4194304] 0 2024-09-16T23:49:11.776 INFO:tasks.workunit.client.0.smithi042.stdout:9/711: dread d4/d10/d24/f3b [0,4194304] 0 2024-09-16T23:49:11.778 INFO:tasks.workunit.client.0.smithi042.stdout:9/712: truncate d4/d12/d2d/fcd 286476 0 2024-09-16T23:49:11.779 INFO:tasks.workunit.client.0.smithi042.stdout:7/853: rmdir d3/d1f/d49/d78/dbe/dc8 39 2024-09-16T23:49:11.780 INFO:tasks.workunit.client.1.smithi043.stdout:2/919: dwrite d8/d4b/d5f/d8e/de8/feb [0,4194304] 0 2024-09-16T23:49:11.784 INFO:tasks.workunit.client.1.smithi043.stdout:2/920: fsync d8/d4b/fcc 0 2024-09-16T23:49:11.785 INFO:tasks.workunit.client.0.smithi042.stdout:6/902: rename d8/d70/d81/dd/d1b/d34/d39/d5e/c110 to d8/d70/d81/dd/d1b/d2d/c121 0 2024-09-16T23:49:11.786 INFO:tasks.workunit.client.0.smithi042.stdout:9/713: dwrite d4/d10/f57 [4194304,4194304] 0 2024-09-16T23:49:11.788 INFO:tasks.workunit.client.1.smithi043.stdout:2/921: creat d8/d11/d111/f12d x:0 0 0 2024-09-16T23:49:11.790 INFO:tasks.workunit.client.1.smithi043.stdout:2/922: chown d8/d11/f6f 0 1 2024-09-16T23:49:11.791 INFO:tasks.workunit.client.0.smithi042.stdout:8/718: sync 2024-09-16T23:49:11.792 INFO:tasks.workunit.client.0.smithi042.stdout:1/932: sync 2024-09-16T23:49:11.792 INFO:tasks.workunit.client.0.smithi042.stdout:8/719: chown da/d19/d1d/f62 31 1 2024-09-16T23:49:11.792 INFO:tasks.workunit.client.0.smithi042.stdout:1/933: chown d0/f4a 15436881 1 2024-09-16T23:49:11.793 INFO:tasks.workunit.client.1.smithi043.stdout:2/923: write d8/d4b/d5f/d8e/de8/feb [3168694,61643] 0 2024-09-16T23:49:11.797 INFO:tasks.workunit.client.0.smithi042.stdout:1/934: stat d0/d9/d26/daa/f101 0 2024-09-16T23:49:11.797 INFO:tasks.workunit.client.0.smithi042.stdout:6/903: mknod d8/d70/d84/c122 0 2024-09-16T23:49:11.797 INFO:tasks.workunit.client.1.smithi043.stdout:2/924: mknod d8/d5e/d83/d3a/d77/dea/c12e 0 2024-09-16T23:49:11.798 INFO:tasks.workunit.client.1.smithi043.stdout:2/925: chown d8/d4b/d5f/d8e/f8c 1 1 2024-09-16T23:49:11.798 INFO:tasks.workunit.client.0.smithi042.stdout:9/714: symlink d4/d12/d2d/dc9/ldb 0 2024-09-16T23:49:11.799 INFO:tasks.workunit.client.0.smithi042.stdout:8/720: getdents da/d19/d1d/d5a 0 2024-09-16T23:49:11.799 INFO:tasks.workunit.client.0.smithi042.stdout:9/715: fsync d4/d12/f1e 0 2024-09-16T23:49:11.800 INFO:tasks.workunit.client.0.smithi042.stdout:0/834: creat d0/df/d13/dcb/f111 x:0 0 0 2024-09-16T23:49:11.800 INFO:tasks.workunit.client.1.smithi043.stdout:2/926: rmdir d8/d5e/d83/d3a/d77/da9/d10a 39 2024-09-16T23:49:11.801 INFO:tasks.workunit.client.0.smithi042.stdout:8/721: read - da/d19/d1d/d30/d3f/dc2/dc4/fc5 zero size 2024-09-16T23:49:11.804 INFO:tasks.workunit.client.1.smithi043.stdout:2/927: symlink d8/d4b/d5f/d63/db6/d126/l12f 0 2024-09-16T23:49:11.807 INFO:tasks.workunit.client.0.smithi042.stdout:9/716: creat d4/d12/d77/dd1/fdc x:0 0 0 2024-09-16T23:49:11.807 INFO:tasks.workunit.client.0.smithi042.stdout:0/835: dread d0/f37 [0,4194304] 0 2024-09-16T23:49:11.808 INFO:tasks.workunit.client.0.smithi042.stdout:2/760: dwrite d0/da/f2c [4194304,4194304] 0 2024-09-16T23:49:11.810 INFO:tasks.workunit.client.0.smithi042.stdout:8/722: write da/d19/d1d/d5a/f8e [4045223,32964] 0 2024-09-16T23:49:11.814 INFO:tasks.workunit.client.0.smithi042.stdout:7/854: write d3/d1f/d49/d76/d19/ddc/dcd/db5/fd9 [3689,12981] 0 2024-09-16T23:49:11.814 INFO:tasks.workunit.client.0.smithi042.stdout:2/761: creat d0/d98/f100 x:0 0 0 2024-09-16T23:49:11.816 INFO:tasks.workunit.client.0.smithi042.stdout:1/935: creat d0/f14d x:0 0 0 2024-09-16T23:49:11.817 INFO:tasks.workunit.client.1.smithi043.stdout:2/928: dwrite d8/d11/d16/fa6 [0,4194304] 0 2024-09-16T23:49:11.818 INFO:tasks.workunit.client.0.smithi042.stdout:7/855: fsync d3/fe 0 2024-09-16T23:49:11.819 INFO:tasks.workunit.client.0.smithi042.stdout:7/856: write d3/d98/da0/d24/f10a [324811,31375] 0 2024-09-16T23:49:11.822 INFO:tasks.workunit.client.0.smithi042.stdout:6/904: getdents d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d10c 0 2024-09-16T23:49:11.823 INFO:tasks.workunit.client.0.smithi042.stdout:1/936: fdatasync d0/d9/d90/d6b/d88/f75 0 2024-09-16T23:49:11.824 INFO:tasks.workunit.client.1.smithi043.stdout:2/929: rename d8/d1d/d20/d23/d38/f47 to d8/d4b/d5f/d8e/de8/f130 0 2024-09-16T23:49:11.824 INFO:tasks.workunit.client.0.smithi042.stdout:6/905: chown d8/d70/d81/dd/l14 814039 1 2024-09-16T23:49:11.826 INFO:tasks.workunit.client.0.smithi042.stdout:6/906: read - d8/d70/d81/dd/d1b/d34/f11a zero size 2024-09-16T23:49:11.828 INFO:tasks.workunit.client.0.smithi042.stdout:7/857: mkdir d3/d1f/d49/d76/d101/da5/de3/d12b 0 2024-09-16T23:49:11.828 INFO:tasks.workunit.client.0.smithi042.stdout:7/858: readlink d3/d1f/d49/d76/l2f 0 2024-09-16T23:49:11.830 INFO:tasks.workunit.client.1.smithi043.stdout:2/930: unlink d8/d4b/d4f/lb3 0 2024-09-16T23:49:11.830 INFO:tasks.workunit.client.0.smithi042.stdout:0/836: link d0/df/d13/dd6/cde d0/df/d23/d33/d56/da9/c112 0 2024-09-16T23:49:11.830 INFO:tasks.workunit.client.0.smithi042.stdout:0/837: dread - d0/df/d13/ffa zero size 2024-09-16T23:49:11.837 INFO:tasks.workunit.client.0.smithi042.stdout:1/937: rename d0/d9/d4e/fe6 to d0/d9/d90/d6b/f14e 0 2024-09-16T23:49:11.837 INFO:tasks.workunit.client.0.smithi042.stdout:6/907: dwrite d8/d70/d81/dd/d1b/dbf/d11f/f6b [0,4194304] 0 2024-09-16T23:49:11.839 INFO:tasks.workunit.client.0.smithi042.stdout:9/717: truncate d4/d10/f4a 2988890 0 2024-09-16T23:49:11.840 INFO:tasks.workunit.client.0.smithi042.stdout:0/838: truncate d0/df/d13/d49/d70/fb3 479819 0 2024-09-16T23:49:11.842 INFO:tasks.workunit.client.1.smithi043.stdout:2/931: dwrite d8/d4b/d5f/d63/f76 [0,4194304] 0 2024-09-16T23:49:11.842 INFO:tasks.workunit.client.0.smithi042.stdout:7/859: link d3/d98/da0/d41/cb2 d3/d1f/c12c 0 2024-09-16T23:49:11.843 INFO:tasks.workunit.client.0.smithi042.stdout:1/938: truncate d0/d9/d90/d6b/d88/d32/d2f/f50 4555399 0 2024-09-16T23:49:11.847 INFO:tasks.workunit.client.1.smithi043.stdout:2/932: creat d8/d4b/d5f/f131 x:0 0 0 2024-09-16T23:49:11.847 INFO:tasks.workunit.client.0.smithi042.stdout:0/839: symlink d0/df/d23/d33/d56/l113 0 2024-09-16T23:49:11.847 INFO:tasks.workunit.client.1.smithi043.stdout:2/933: chown d8/d1f/l98 16 1 2024-09-16T23:49:11.850 INFO:tasks.workunit.client.1.smithi043.stdout:2/934: fdatasync d8/d11/d16/fb7 0 2024-09-16T23:49:11.851 INFO:tasks.workunit.client.1.smithi043.stdout:2/935: fdatasync d8/d4b/d4f/d7c/fad 0 2024-09-16T23:49:11.853 INFO:tasks.workunit.client.0.smithi042.stdout:6/908: unlink d8/d70/d81/dd/d1b/d34/f64 0 2024-09-16T23:49:11.854 INFO:tasks.workunit.client.0.smithi042.stdout:6/909: fdatasync d8/d70/d81/dd/d1b/d50/d96/f106 0 2024-09-16T23:49:11.855 INFO:tasks.workunit.client.1.smithi043.stdout:2/936: rename d8/d5e/d83/d9e/dab/fed to d8/d5e/d83/d9e/f132 0 2024-09-16T23:49:11.857 INFO:tasks.workunit.client.1.smithi043.stdout:2/937: fdatasync d8/d1d/d20/d26/f97 0 2024-09-16T23:49:11.859 INFO:tasks.workunit.client.0.smithi042.stdout:6/910: read d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f78 [247136,60511] 0 2024-09-16T23:49:11.863 INFO:tasks.workunit.client.1.smithi043.stdout:2/938: fdatasync d8/d11/f6f 0 2024-09-16T23:49:11.864 INFO:tasks.workunit.client.0.smithi042.stdout:9/718: creat d4/d12/d2d/fdd x:0 0 0 2024-09-16T23:49:11.868 INFO:tasks.workunit.client.0.smithi042.stdout:6/911: symlink d8/d70/d81/dd/d1b/d67/d97/l123 0 2024-09-16T23:49:11.869 INFO:tasks.workunit.client.0.smithi042.stdout:6/912: fsync d8/d70/d81/dd/d1b/f102 0 2024-09-16T23:49:11.870 INFO:tasks.workunit.client.0.smithi042.stdout:7/860: creat d3/d1f/d49/d76/d19/ddc/f12d x:0 0 0 2024-09-16T23:49:11.871 INFO:tasks.workunit.client.0.smithi042.stdout:6/913: fdatasync d8/d70/d81/dd/d2c/f10 0 2024-09-16T23:49:11.874 INFO:tasks.workunit.client.0.smithi042.stdout:0/840: rename d0/df/d13/d49/d70/da1/lbe to d0/d10f/l114 0 2024-09-16T23:49:11.877 INFO:tasks.workunit.client.0.smithi042.stdout:7/861: chown d3/d98/da0/d24/c33 212422 1 2024-09-16T23:49:11.880 INFO:tasks.workunit.client.0.smithi042.stdout:0/841: mknod d0/d15/de4/ddb/c115 0 2024-09-16T23:49:11.881 INFO:tasks.workunit.client.0.smithi042.stdout:0/842: fdatasync d0/df/d23/d33/d56/da9/f92 0 2024-09-16T23:49:11.885 INFO:tasks.workunit.client.0.smithi042.stdout:7/862: getdents d3/d1f/d49/d76/d101/da5/de3 0 2024-09-16T23:49:11.889 INFO:tasks.workunit.client.0.smithi042.stdout:7/863: creat d3/d1f/f12e x:0 0 0 2024-09-16T23:49:11.892 INFO:tasks.workunit.client.0.smithi042.stdout:7/864: truncate d3/d1f/d49/d76/d101/f83 882048 0 2024-09-16T23:49:11.895 INFO:tasks.workunit.client.0.smithi042.stdout:0/843: dread d0/d48/fb1 [0,4194304] 0 2024-09-16T23:49:11.896 INFO:tasks.workunit.client.0.smithi042.stdout:7/865: mknod d3/d1f/d49/d76/d19/ddc/dcd/db5/dc4/c12f 0 2024-09-16T23:49:11.897 INFO:tasks.workunit.client.0.smithi042.stdout:7/866: readlink d3/d98/da0/lf6 0 2024-09-16T23:49:11.897 INFO:tasks.workunit.client.1.smithi043.stdout:2/939: mkdir d8/d4b/d4f/d56/d133 0 2024-09-16T23:49:11.898 INFO:tasks.workunit.client.0.smithi042.stdout:0/844: fdatasync d0/df/d23/d33/d56/da9/fdd 0 2024-09-16T23:49:11.898 INFO:tasks.workunit.client.0.smithi042.stdout:0/845: fdatasync d0/d15/fe8 0 2024-09-16T23:49:11.899 INFO:tasks.workunit.client.0.smithi042.stdout:0/846: fdatasync d0/df/d13/ff9 0 2024-09-16T23:49:11.907 INFO:tasks.workunit.client.0.smithi042.stdout:7/867: dwrite d3/d1f/d49/d76/d19/ddc/dcd/fcb [0,4194304] 0 2024-09-16T23:49:11.907 INFO:tasks.workunit.client.0.smithi042.stdout:0/847: rename d0/df/d13/dd6/cde to d0/df/d13/d49/d70/da1/c116 0 2024-09-16T23:49:11.910 INFO:tasks.workunit.client.1.smithi043.stdout:2/940: dread d8/d4b/d5f/d8e/f8c [0,4194304] 0 2024-09-16T23:49:11.911 INFO:tasks.workunit.client.0.smithi042.stdout:0/848: symlink d0/df/d13/dd6/l117 0 2024-09-16T23:49:11.912 INFO:tasks.workunit.client.0.smithi042.stdout:7/868: dread d3/d98/da0/d41/f62 [0,4194304] 0 2024-09-16T23:49:11.912 INFO:tasks.workunit.client.1.smithi043.stdout:2/941: write d8/d5e/ffd [2733036,34857] 0 2024-09-16T23:49:11.913 INFO:tasks.workunit.client.0.smithi042.stdout:0/849: readlink d0/df/d13/dcb/dcc/lee 0 2024-09-16T23:49:11.915 INFO:tasks.workunit.client.0.smithi042.stdout:7/869: read d3/fe [635767,89979] 0 2024-09-16T23:49:11.915 INFO:tasks.workunit.client.0.smithi042.stdout:0/850: dread - d0/df/d23/d33/d56/dc5/ff4 zero size 2024-09-16T23:49:11.916 INFO:tasks.workunit.client.0.smithi042.stdout:0/851: write d0/df/d23/d33/d51/fd4 [2979786,11759] 0 2024-09-16T23:49:11.918 INFO:tasks.workunit.client.0.smithi042.stdout:0/852: write d0/df/d13/dcb/f111 [309861,125653] 0 2024-09-16T23:49:11.920 INFO:tasks.workunit.client.0.smithi042.stdout:0/853: dread - d0/df/d13/d49/d70/da1/fea zero size 2024-09-16T23:49:11.924 INFO:tasks.workunit.client.1.smithi043.stdout:8/957: sync 2024-09-16T23:49:11.927 INFO:tasks.workunit.client.0.smithi042.stdout:0/854: dwrite d0/df/d23/d33/d51/ff5 [0,4194304] 0 2024-09-16T23:49:11.936 INFO:tasks.workunit.client.0.smithi042.stdout:7/870: dread d3/d1f/d49/d76/d19/ddc/dcd/db5/dc4/fc9 [0,4194304] 0 2024-09-16T23:49:11.936 INFO:tasks.workunit.client.0.smithi042.stdout:0/855: dwrite d0/df/d23/d3e/d8f/fb6 [4194304,4194304] 0 2024-09-16T23:49:11.937 INFO:tasks.workunit.client.1.smithi043.stdout:2/942: dwrite d8/d5e/f87 [0,4194304] 0 2024-09-16T23:49:11.937 INFO:tasks.workunit.client.0.smithi042.stdout:7/871: stat d3/fd1 0 2024-09-16T23:49:11.937 INFO:tasks.workunit.client.0.smithi042.stdout:7/872: fdatasync d3/d98/da0/d24/d28/d9b/f108 0 2024-09-16T23:49:11.938 INFO:tasks.workunit.client.1.smithi043.stdout:8/958: dread de/fb0 [4194304,4194304] 0 2024-09-16T23:49:11.939 INFO:tasks.workunit.client.1.smithi043.stdout:2/943: fdatasync d8/d1d/d20/d26/fb5 0 2024-09-16T23:49:11.939 INFO:tasks.workunit.client.1.smithi043.stdout:8/959: readlink de/d18/d2b/l9f 0 2024-09-16T23:49:11.941 INFO:tasks.workunit.client.1.smithi043.stdout:8/960: write de/d18/d1f/d25/fb9 [4526286,89570] 0 2024-09-16T23:49:11.942 INFO:tasks.workunit.client.0.smithi042.stdout:7/873: dread d3/d98/da0/d24/fbd [0,4194304] 0 2024-09-16T23:49:11.943 INFO:tasks.workunit.client.1.smithi043.stdout:2/944: rmdir d8/d5e/d83 39 2024-09-16T23:49:11.944 INFO:tasks.workunit.client.0.smithi042.stdout:7/874: chown d3/d1f/d49/d76/d19/fe0 188621 1 2024-09-16T23:49:11.945 INFO:tasks.workunit.client.0.smithi042.stdout:0/856: creat d0/df/d13/df1/f118 x:0 0 0 2024-09-16T23:49:11.947 INFO:tasks.workunit.client.0.smithi042.stdout:7/875: symlink d3/d98/da0/d24/d11a/l130 0 2024-09-16T23:49:11.948 INFO:tasks.workunit.client.0.smithi042.stdout:0/857: truncate d0/df/d23/d3e/d8f/ffe 350315 0 2024-09-16T23:49:11.950 INFO:tasks.workunit.client.1.smithi043.stdout:8/961: getdents de/d18/d1f/d25 0 2024-09-16T23:49:11.951 INFO:tasks.workunit.client.0.smithi042.stdout:0/858: creat d0/df/d13/d2d/f119 x:0 0 0 2024-09-16T23:49:11.952 INFO:tasks.workunit.client.0.smithi042.stdout:0/859: readlink d0/df/d13/d49/l6f 0 2024-09-16T23:49:11.952 INFO:tasks.workunit.client.1.smithi043.stdout:2/945: creat d8/d5e/d83/d3a/f134 x:0 0 0 2024-09-16T23:49:11.953 INFO:tasks.workunit.client.1.smithi043.stdout:8/962: dread - de/d18/d27/d41/d5a/d37/d6a/f73 zero size 2024-09-16T23:49:11.953 INFO:tasks.workunit.client.1.smithi043.stdout:8/963: readlink de/d18/d27/l6d 0 2024-09-16T23:49:11.955 INFO:tasks.workunit.client.1.smithi043.stdout:2/946: symlink d8/d1d/l135 0 2024-09-16T23:49:11.956 INFO:tasks.workunit.client.1.smithi043.stdout:2/947: write d8/d11/d16/fa6 [2530719,17703] 0 2024-09-16T23:49:11.956 INFO:tasks.workunit.client.0.smithi042.stdout:7/876: write d3/d98/f89 [2706793,42697] 0 2024-09-16T23:49:11.957 INFO:tasks.workunit.client.1.smithi043.stdout:8/964: truncate de/d18/d27/d8e/df3/d10a/d3d/f121 299187 0 2024-09-16T23:49:11.958 INFO:tasks.workunit.client.0.smithi042.stdout:0/860: dwrite d0/df/d23/d33/d56/dc5/fc6 [0,4194304] 0 2024-09-16T23:49:11.962 INFO:tasks.workunit.client.0.smithi042.stdout:0/861: truncate d0/df/d23/d33/f5c 651578 0 2024-09-16T23:49:11.965 INFO:tasks.workunit.client.1.smithi043.stdout:2/948: dwrite d8/d1d/d20/d23/fb8 [4194304,4194304] 0 2024-09-16T23:49:11.970 INFO:tasks.workunit.client.0.smithi042.stdout:0/862: link d0/df/d23/d33/d56/f61 d0/f11a 0 2024-09-16T23:49:11.975 INFO:tasks.workunit.client.0.smithi042.stdout:0/863: mknod d0/df/d13/dcb/dcc/df8/c11b 0 2024-09-16T23:49:11.976 INFO:tasks.workunit.client.0.smithi042.stdout:7/877: dwrite d3/d1f/d49/d76/d19/ff1 [0,4194304] 0 2024-09-16T23:49:11.976 INFO:tasks.workunit.client.1.smithi043.stdout:2/949: dread d8/d5e/d83/d3a/d77/f9b [0,4194304] 0 2024-09-16T23:49:11.978 INFO:tasks.workunit.client.1.smithi043.stdout:2/950: truncate d8/d4b/d5f/d8e/fbb 4650655 0 2024-09-16T23:49:11.982 INFO:tasks.workunit.client.0.smithi042.stdout:0/864: dwrite d0/df/d13/d49/ff3 [0,4194304] 0 2024-09-16T23:49:11.983 INFO:tasks.workunit.client.0.smithi042.stdout:0/865: chown d0/c5f 0 1 2024-09-16T23:49:11.984 INFO:tasks.workunit.client.1.smithi043.stdout:2/951: rename d8/d1d/lbc to d8/d5e/d83/l136 0 2024-09-16T23:49:11.988 INFO:tasks.workunit.client.1.smithi043.stdout:2/952: symlink d8/d1d/d20/d23/dfc/l137 0 2024-09-16T23:49:11.988 INFO:tasks.workunit.client.0.smithi042.stdout:0/866: creat d0/d48/daf/f11c x:0 0 0 2024-09-16T23:49:11.989 INFO:tasks.workunit.client.0.smithi042.stdout:0/867: chown d0/df/d13/d49/d70 8237 1 2024-09-16T23:49:11.990 INFO:tasks.workunit.client.1.smithi043.stdout:2/953: write d8/d4b/d5f/d8e/fbb [1411973,86608] 0 2024-09-16T23:49:12.001 INFO:tasks.workunit.client.0.smithi042.stdout:0/868: write d0/df/d13/d49/fd0 [391854,85436] 0 2024-09-16T23:49:12.002 INFO:tasks.workunit.client.1.smithi043.stdout:2/954: rename d8/d1d/d20/d26/c7a to d8/d5e/d83/d3a/d77/da9/c138 0 2024-09-16T23:49:12.005 INFO:tasks.workunit.client.0.smithi042.stdout:0/869: creat d0/f11d x:0 0 0 2024-09-16T23:49:12.005 INFO:tasks.workunit.client.1.smithi043.stdout:2/955: mknod d8/d4b/d5f/d63/db6/c139 0 2024-09-16T23:49:12.007 INFO:tasks.workunit.client.1.smithi043.stdout:2/956: symlink d8/d4b/d5f/l13a 0 2024-09-16T23:49:12.009 INFO:tasks.workunit.client.1.smithi043.stdout:2/957: truncate d8/d4b/d5f/d8e/f8c 4616603 0 2024-09-16T23:49:12.010 INFO:tasks.workunit.client.1.smithi043.stdout:2/958: dread - d8/d5e/d83/d9e/fa5 zero size 2024-09-16T23:49:12.012 INFO:tasks.workunit.client.1.smithi043.stdout:2/959: write d8/d1d/d25/f123 [394727,83855] 0 2024-09-16T23:49:12.016 INFO:tasks.workunit.client.0.smithi042.stdout:0/870: dread d0/f9 [4194304,4194304] 0 2024-09-16T23:49:12.019 INFO:tasks.workunit.client.0.smithi042.stdout:0/871: rename d0/df/d23/d33/d56/da9/dc3 to d0/df/d23/d3e/d8f/dda/d11e 0 2024-09-16T23:49:12.019 INFO:tasks.workunit.client.0.smithi042.stdout:0/872: chown d0/d15/cad 7582873 1 2024-09-16T23:49:12.021 INFO:tasks.workunit.client.0.smithi042.stdout:0/873: creat d0/df/d13/d2d/f11f x:0 0 0 2024-09-16T23:49:12.022 INFO:tasks.workunit.client.0.smithi042.stdout:0/874: fdatasync d0/f37 0 2024-09-16T23:49:12.023 INFO:tasks.workunit.client.0.smithi042.stdout:0/875: chown d0/df/d23/d3e/c40 338600407 1 2024-09-16T23:49:12.024 INFO:tasks.workunit.client.0.smithi042.stdout:0/876: write d0/df/f8c [675862,81109] 0 2024-09-16T23:49:12.026 INFO:tasks.workunit.client.0.smithi042.stdout:0/877: unlink d0/d15/l32 0 2024-09-16T23:49:12.029 INFO:tasks.workunit.client.0.smithi042.stdout:0/878: creat d0/df/d23/d33/d51/f120 x:0 0 0 2024-09-16T23:49:12.030 INFO:tasks.workunit.client.0.smithi042.stdout:0/879: write d0/df/d23/d33/d51/ff5 [3450531,86229] 0 2024-09-16T23:49:12.036 INFO:tasks.workunit.client.0.smithi042.stdout:0/880: dwrite d0/df/d23/d33/d51/d79/fb0 [4194304,4194304] 0 2024-09-16T23:49:12.039 INFO:tasks.workunit.client.0.smithi042.stdout:0/881: read d0/df/d13/f97 [3884433,28275] 0 2024-09-16T23:49:12.040 INFO:tasks.workunit.client.0.smithi042.stdout:0/882: write d0/f4 [5303373,70448] 0 2024-09-16T23:49:12.049 INFO:tasks.workunit.client.0.smithi042.stdout:0/883: truncate d0/df/f19 1777132 0 2024-09-16T23:49:12.051 INFO:tasks.workunit.client.0.smithi042.stdout:0/884: truncate d0/df/df7/ffb 525158 0 2024-09-16T23:49:12.054 INFO:tasks.workunit.client.0.smithi042.stdout:0/885: link d0/df/d23/d33/d51/f120 d0/df/d13/d2d/d35/d9f/f121 0 2024-09-16T23:49:12.057 INFO:tasks.workunit.client.0.smithi042.stdout:0/886: creat d0/df/d23/d33/d56/dc5/dcd/f122 x:0 0 0 2024-09-16T23:49:12.065 INFO:tasks.workunit.client.0.smithi042.stdout:0/887: dread d0/df/d23/f31 [0,4194304] 0 2024-09-16T23:49:12.067 INFO:tasks.workunit.client.0.smithi042.stdout:0/888: creat d0/df/d23/d3e/d8f/dda/d11e/f123 x:0 0 0 2024-09-16T23:49:12.068 INFO:tasks.workunit.client.0.smithi042.stdout:0/889: readlink d0/df/d23/d33/d56/l10e 0 2024-09-16T23:49:12.069 INFO:tasks.workunit.client.0.smithi042.stdout:0/890: write d0/d48/daf/f11c [182994,23041] 0 2024-09-16T23:49:12.073 INFO:tasks.workunit.client.0.smithi042.stdout:0/891: link d0/df/d13/f74 d0/d10f/f124 0 2024-09-16T23:49:12.077 INFO:tasks.workunit.client.0.smithi042.stdout:0/892: link d0/df/d23/d3e/d8f/la3 d0/df/d13/dcb/l125 0 2024-09-16T23:49:12.078 INFO:tasks.workunit.client.0.smithi042.stdout:0/893: fsync d0/df/d13/df1/f118 0 2024-09-16T23:49:12.078 INFO:tasks.workunit.client.0.smithi042.stdout:0/894: fdatasync d0/df/d23/d33/d51/fd4 0 2024-09-16T23:49:12.083 INFO:tasks.workunit.client.0.smithi042.stdout:0/895: rename d0/df/d23/d33/d51/d79/fb0 to d0/df/d23/d33/f126 0 2024-09-16T23:49:12.084 INFO:tasks.workunit.client.0.smithi042.stdout:0/896: mkdir d0/df/df7/d127 0 2024-09-16T23:49:12.090 INFO:tasks.workunit.client.0.smithi042.stdout:0/897: getdents d0/df/d23/d3e 0 2024-09-16T23:49:12.092 INFO:tasks.workunit.client.0.smithi042.stdout:0/898: mkdir d0/df/d13/d2d/d35/d9f/d128 0 2024-09-16T23:49:12.094 INFO:tasks.workunit.client.0.smithi042.stdout:0/899: symlink d0/d15/l129 0 2024-09-16T23:49:12.096 INFO:tasks.workunit.client.0.smithi042.stdout:0/900: mknod d0/df/df7/d127/c12a 0 2024-09-16T23:49:12.096 INFO:tasks.workunit.client.0.smithi042.stdout:0/901: fsync d0/df/d13/d2d/d35/f82 0 2024-09-16T23:49:12.098 INFO:tasks.workunit.client.0.smithi042.stdout:0/902: mknod d0/df/d23/d33/d56/da9/c12b 0 2024-09-16T23:49:12.100 INFO:tasks.workunit.client.0.smithi042.stdout:0/903: creat d0/df/f12c x:0 0 0 2024-09-16T23:49:12.101 INFO:tasks.workunit.client.0.smithi042.stdout:0/904: chown d0/d15 869 1 2024-09-16T23:49:12.101 INFO:tasks.workunit.client.0.smithi042.stdout:0/905: stat d0/d48/fb1 0 2024-09-16T23:49:12.102 INFO:tasks.workunit.client.0.smithi042.stdout:0/906: dread - d0/df/d23/d33/d56/dc5/dcd/f100 zero size 2024-09-16T23:49:12.103 INFO:tasks.workunit.client.0.smithi042.stdout:0/907: creat d0/df/df7/d127/f12d x:0 0 0 2024-09-16T23:49:12.109 INFO:tasks.workunit.client.0.smithi042.stdout:0/908: dwrite d0/df/d13/dd6/fd8 [0,4194304] 0 2024-09-16T23:49:12.114 INFO:tasks.workunit.client.0.smithi042.stdout:0/909: link d0/df/d23/d33/d56/l10e d0/df/ddf/l12e 0 2024-09-16T23:49:12.115 INFO:tasks.workunit.client.0.smithi042.stdout:0/910: write d0/df/f8c [1036201,83379] 0 2024-09-16T23:49:12.123 INFO:tasks.workunit.client.0.smithi042.stdout:0/911: dwrite d0/df/d23/d3e/d8f/fe1 [0,4194304] 0 2024-09-16T23:49:12.126 INFO:tasks.workunit.client.0.smithi042.stdout:0/912: mkdir d0/df/d23/d3e/d8f/dda/d12f 0 2024-09-16T23:49:12.133 INFO:tasks.workunit.client.0.smithi042.stdout:0/913: link d0/df/d13/d49/d70/da1/lb7 d0/df/d23/d33/d56/dc5/l130 0 2024-09-16T23:49:12.134 INFO:tasks.workunit.client.0.smithi042.stdout:0/914: write d0/df/d23/f107 [965166,93444] 0 2024-09-16T23:49:12.135 INFO:tasks.workunit.client.0.smithi042.stdout:0/915: dread - d0/df/d13/d2d/d35/d9f/f121 zero size 2024-09-16T23:49:12.136 INFO:tasks.workunit.client.0.smithi042.stdout:0/916: fsync d0/df/d23/d33/d56/da9/fdd 0 2024-09-16T23:49:12.175 INFO:tasks.workunit.client.0.smithi042.stdout:0/917: write d0/df/d23/d33/d56/da9/f93 [765305,43463] 0 2024-09-16T23:49:12.191 INFO:tasks.workunit.client.1.smithi043.stdout:8/965: sync 2024-09-16T23:49:12.193 INFO:tasks.workunit.client.1.smithi043.stdout:8/966: read de/d18/d27/d8e/df3/d10a/f76 [773681,51279] 0 2024-09-16T23:49:12.194 INFO:tasks.workunit.client.1.smithi043.stdout:8/967: truncate de/d18/d1f/f59 3930089 0 2024-09-16T23:49:12.196 INFO:tasks.workunit.client.1.smithi043.stdout:8/968: truncate de/d18/d27/d41/d5a/f109 4382410 0 2024-09-16T23:49:12.209 INFO:tasks.workunit.client.0.smithi042.stdout:0/918: rename d0/df/ddf/f10c to d0/df/d13/dcb/dcc/f131 0 2024-09-16T23:49:12.211 INFO:tasks.workunit.client.0.smithi042.stdout:0/919: symlink d0/d48/daf/l132 0 2024-09-16T23:49:12.213 INFO:tasks.workunit.client.0.smithi042.stdout:0/920: readlink d0/df/ddf/l12e 0 2024-09-16T23:49:12.215 INFO:tasks.workunit.client.0.smithi042.stdout:0/921: symlink d0/df/d23/d3e/d8f/dda/l133 0 2024-09-16T23:49:12.217 INFO:tasks.workunit.client.0.smithi042.stdout:0/922: truncate d0/df/d13/d2d/d35/d9f/fdc 768750 0 2024-09-16T23:49:12.222 INFO:tasks.workunit.client.0.smithi042.stdout:0/923: rename d0/df/d13/d49/d70/da1/fea to d0/df/d13/dcb/f134 0 2024-09-16T23:49:12.223 INFO:tasks.workunit.client.0.smithi042.stdout:0/924: truncate d0/df/d23/d33/d51/ff5 5058222 0 2024-09-16T23:49:12.224 INFO:tasks.workunit.client.0.smithi042.stdout:0/925: read - d0/df/d23/d33/d56/dc5/ff4 zero size 2024-09-16T23:49:12.228 INFO:tasks.workunit.client.0.smithi042.stdout:0/926: creat d0/d15/de4/f135 x:0 0 0 2024-09-16T23:49:12.230 INFO:tasks.workunit.client.0.smithi042.stdout:0/927: mknod d0/df/df7/d127/c136 0 2024-09-16T23:49:12.234 INFO:tasks.workunit.client.0.smithi042.stdout:0/928: getdents d0/d48 0 2024-09-16T23:49:12.238 INFO:tasks.workunit.client.0.smithi042.stdout:0/929: creat d0/d15/de4/f137 x:0 0 0 2024-09-16T23:49:12.239 INFO:tasks.workunit.client.1.smithi043.stdout:8/969: dread de/f90 [0,4194304] 0 2024-09-16T23:49:12.240 INFO:tasks.workunit.client.1.smithi043.stdout:8/970: fdatasync de/f23 0 2024-09-16T23:49:12.241 INFO:tasks.workunit.client.0.smithi042.stdout:0/930: creat d0/df/d23/f138 x:0 0 0 2024-09-16T23:49:12.242 INFO:tasks.workunit.client.0.smithi042.stdout:0/931: read - d0/df/d13/fa0 zero size 2024-09-16T23:49:12.248 INFO:tasks.workunit.client.0.smithi042.stdout:0/932: dwrite d0/df/fb2 [0,4194304] 0 2024-09-16T23:49:12.250 INFO:tasks.workunit.client.0.smithi042.stdout:0/933: read - d0/df/d13/d2d/d35/d9f/fab zero size 2024-09-16T23:49:12.252 INFO:tasks.workunit.client.0.smithi042.stdout:0/934: write d0/df/d13/d49/d70/ff6 [5196877,32345] 0 2024-09-16T23:49:12.253 INFO:tasks.workunit.client.0.smithi042.stdout:0/935: fdatasync d0/f4 0 2024-09-16T23:49:12.254 INFO:tasks.workunit.client.0.smithi042.stdout:0/936: dread - d0/df/d23/d33/d56/dc5/ff4 zero size 2024-09-16T23:49:12.257 INFO:tasks.workunit.client.0.smithi042.stdout:0/937: mkdir d0/df/d13/d49/d70/d139 0 2024-09-16T23:49:12.263 INFO:tasks.workunit.client.0.smithi042.stdout:0/938: dwrite d0/df/df7/ffb [0,4194304] 0 2024-09-16T23:49:12.266 INFO:tasks.workunit.client.0.smithi042.stdout:0/939: truncate d0/df/d23/d33/d56/da9/fbc 4518672 0 2024-09-16T23:49:12.272 INFO:tasks.workunit.client.0.smithi042.stdout:8/723: sync 2024-09-16T23:49:12.273 INFO:tasks.workunit.client.0.smithi042.stdout:0/940: write d0/d15/de4/ddb/fa2 [830660,127909] 0 2024-09-16T23:49:12.274 INFO:tasks.workunit.client.0.smithi042.stdout:8/724: creat da/d4b/d74/fe1 x:0 0 0 2024-09-16T23:49:12.275 INFO:tasks.workunit.client.0.smithi042.stdout:0/941: write d0/df/d13/d49/fd0 [105781,64529] 0 2024-09-16T23:49:12.275 INFO:tasks.workunit.client.0.smithi042.stdout:8/725: fdatasync da/d19/fcc 0 2024-09-16T23:49:12.278 INFO:tasks.workunit.client.0.smithi042.stdout:8/726: symlink da/d19/d1d/d30/d3f/le2 0 2024-09-16T23:49:12.278 INFO:tasks.workunit.client.0.smithi042.stdout:0/942: mknod d0/df/d13/d2d/d35/d9f/c13a 0 2024-09-16T23:49:12.279 INFO:tasks.workunit.client.0.smithi042.stdout:8/727: chown da/d19/d2c/d56/d97/f9d 37231 1 2024-09-16T23:49:12.280 INFO:tasks.workunit.client.0.smithi042.stdout:0/943: creat d0/df/d23/d3e/d8f/dda/d12f/f13b x:0 0 0 2024-09-16T23:49:12.282 INFO:tasks.workunit.client.0.smithi042.stdout:8/728: rename da/d19/d1d/d30/d3f/dc2/dc4 to da/d19/d1d/d30/d3f/dc2/de3 0 2024-09-16T23:49:12.284 INFO:tasks.workunit.client.0.smithi042.stdout:0/944: rename d0/df/d13/f34 to d0/df/d23/d3e/d8f/dda/f13c 0 2024-09-16T23:49:12.286 INFO:tasks.workunit.client.0.smithi042.stdout:8/729: link da/d19/l21 da/d4b/d74/dc7/le4 0 2024-09-16T23:49:12.286 INFO:tasks.workunit.client.0.smithi042.stdout:8/730: readlink da/d19/d2c/db0/lba 0 2024-09-16T23:49:12.293 INFO:tasks.workunit.client.0.smithi042.stdout:0/945: dread d0/df/d23/d33/d56/dc5/fe9 [0,4194304] 0 2024-09-16T23:49:12.296 INFO:tasks.workunit.client.0.smithi042.stdout:8/731: dwrite da/f2a [0,4194304] 0 2024-09-16T23:49:12.298 INFO:tasks.workunit.client.0.smithi042.stdout:0/946: truncate d0/df/d13/d49/d70/f98 70428 0 2024-09-16T23:49:12.300 INFO:tasks.workunit.client.0.smithi042.stdout:7/878: sync 2024-09-16T23:49:12.300 INFO:tasks.workunit.client.0.smithi042.stdout:9/719: sync 2024-09-16T23:49:12.300 INFO:tasks.workunit.client.0.smithi042.stdout:6/914: sync 2024-09-16T23:49:12.300 INFO:tasks.workunit.client.0.smithi042.stdout:1/939: sync 2024-09-16T23:49:12.301 INFO:tasks.workunit.client.0.smithi042.stdout:2/762: sync 2024-09-16T23:49:12.302 INFO:tasks.workunit.client.0.smithi042.stdout:0/947: symlink d0/df/df7/d127/l13d 0 2024-09-16T23:49:12.304 INFO:tasks.workunit.client.0.smithi042.stdout:8/732: dread da/d19/d2c/d56/d97/dad/fbf [0,4194304] 0 2024-09-16T23:49:12.305 INFO:tasks.workunit.client.0.smithi042.stdout:0/948: read d0/df/d13/d49/d70/ff6 [2306360,51224] 0 2024-09-16T23:49:12.306 INFO:tasks.workunit.client.0.smithi042.stdout:2/763: creat d0/da/d1b/d3b/d44/dec/f101 x:0 0 0 2024-09-16T23:49:12.306 INFO:tasks.workunit.client.0.smithi042.stdout:7/879: dread - d3/d1f/d49/d76/d101/da5/f106 zero size 2024-09-16T23:49:12.307 INFO:tasks.workunit.client.0.smithi042.stdout:8/733: creat da/d19/d2c/dc8/fe5 x:0 0 0 2024-09-16T23:49:12.308 INFO:tasks.workunit.client.0.smithi042.stdout:7/880: write d3/d1f/d49/f56 [1794942,64535] 0 2024-09-16T23:49:12.309 INFO:tasks.workunit.client.0.smithi042.stdout:7/881: fsync d3/d98/f89 0 2024-09-16T23:49:12.310 INFO:tasks.workunit.client.0.smithi042.stdout:1/940: creat d0/d9/f14f x:0 0 0 2024-09-16T23:49:12.311 INFO:tasks.workunit.client.0.smithi042.stdout:7/882: write d3/d98/f71 [890048,58243] 0 2024-09-16T23:49:12.311 INFO:tasks.workunit.client.0.smithi042.stdout:1/941: stat d0/d9/d4e/f13b 0 2024-09-16T23:49:12.311 INFO:tasks.workunit.client.0.smithi042.stdout:2/764: symlink d0/d8d/da0/l102 0 2024-09-16T23:49:12.312 INFO:tasks.workunit.client.0.smithi042.stdout:8/734: symlink da/d19/d1d/d30/d49/d58/le6 0 2024-09-16T23:49:12.314 INFO:tasks.workunit.client.0.smithi042.stdout:7/883: chown d3/d98/da0/d41/f103 19 1 2024-09-16T23:49:12.314 INFO:tasks.workunit.client.0.smithi042.stdout:0/949: creat d0/df/d23/d33/d56/dc5/dcd/dd1/f13e x:0 0 0 2024-09-16T23:49:12.315 INFO:tasks.workunit.client.0.smithi042.stdout:6/915: link d8/d70/d81/dd/d1b/d34/d39/d5e/ff9 d8/d70/d81/dd/d1b/d50/da2/f124 0 2024-09-16T23:49:12.316 INFO:tasks.workunit.client.0.smithi042.stdout:2/765: read d0/da/d1b/d48/db7/fd2 [1304128,67814] 0 2024-09-16T23:49:12.318 INFO:tasks.workunit.client.0.smithi042.stdout:8/735: truncate da/d19/d1d/f50 1542447 0 2024-09-16T23:49:12.321 INFO:tasks.workunit.client.0.smithi042.stdout:1/942: dwrite d0/d9a/fb7 [0,4194304] 0 2024-09-16T23:49:12.323 INFO:tasks.workunit.client.0.smithi042.stdout:9/720: getdents d4/d10 0 2024-09-16T23:49:12.325 INFO:tasks.workunit.client.0.smithi042.stdout:8/736: unlink da/d4b/lb9 0 2024-09-16T23:49:12.326 INFO:tasks.workunit.client.0.smithi042.stdout:2/766: creat d0/d98/f103 x:0 0 0 2024-09-16T23:49:12.329 INFO:tasks.workunit.client.0.smithi042.stdout:7/884: mknod d3/d1f/d49/d78/dbe/dc8/c131 0 2024-09-16T23:49:12.330 INFO:tasks.workunit.client.0.smithi042.stdout:8/737: creat da/d19/d1d/d30/d49/d58/d6a/fe7 x:0 0 0 2024-09-16T23:49:12.331 INFO:tasks.workunit.client.0.smithi042.stdout:1/943: creat d0/d17/d79/d118/f150 x:0 0 0 2024-09-16T23:49:12.334 INFO:tasks.workunit.client.0.smithi042.stdout:6/916: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f95 739978 0 2024-09-16T23:49:12.339 INFO:tasks.workunit.client.0.smithi042.stdout:8/738: dwrite da/d4b/fa9 [0,4194304] 0 2024-09-16T23:49:12.340 INFO:tasks.workunit.client.0.smithi042.stdout:2/767: mkdir d0/da/d17/d104 0 2024-09-16T23:49:12.343 INFO:tasks.workunit.client.0.smithi042.stdout:0/950: getdents d0/df/d23/d33/d56/dc5 0 2024-09-16T23:49:12.351 INFO:tasks.workunit.client.0.smithi042.stdout:1/944: dwrite d0/d9/d90/d66/de2/f119 [0,4194304] 0 2024-09-16T23:49:12.351 INFO:tasks.workunit.client.0.smithi042.stdout:2/768: write d0/fc6 [529497,44559] 0 2024-09-16T23:49:12.351 INFO:tasks.workunit.client.0.smithi042.stdout:9/721: dwrite d4/d12/d2d/d94/dc2/d28/f90 [0,4194304] 0 2024-09-16T23:49:12.357 INFO:tasks.workunit.client.0.smithi042.stdout:7/885: dread d3/d1f/f23 [0,4194304] 0 2024-09-16T23:49:12.358 INFO:tasks.workunit.client.0.smithi042.stdout:1/945: read d0/d3b/fe0 [2655974,58204] 0 2024-09-16T23:49:12.359 INFO:tasks.workunit.client.0.smithi042.stdout:6/917: dwrite d8/d70/d81/dd/d2c/f31 [0,4194304] 0 2024-09-16T23:49:12.365 INFO:tasks.workunit.client.0.smithi042.stdout:9/722: creat d4/d12/d2d/dc9/fde x:0 0 0 2024-09-16T23:49:12.370 INFO:tasks.workunit.client.0.smithi042.stdout:0/951: dwrite d0/df/df7/d127/f12d [0,4194304] 0 2024-09-16T23:49:12.373 INFO:tasks.workunit.client.0.smithi042.stdout:7/886: symlink d3/d1f/d49/d78/dbe/l132 0 2024-09-16T23:49:12.375 INFO:tasks.workunit.client.0.smithi042.stdout:8/739: dread da/d19/d2c/f36 [0,4194304] 0 2024-09-16T23:49:12.375 INFO:tasks.workunit.client.0.smithi042.stdout:0/952: fsync d0/df/df7/ffb 0 2024-09-16T23:49:12.380 INFO:tasks.workunit.client.0.smithi042.stdout:2/769: dread d0/d30/d43/f62 [0,4194304] 0 2024-09-16T23:49:12.382 INFO:tasks.workunit.client.1.smithi043.stdout:2/960: sync 2024-09-16T23:49:12.383 INFO:tasks.workunit.client.0.smithi042.stdout:6/918: rename d8/d70/d81/dd/d1b/c8e to d8/d70/d81/dd/d1b/d2d/da1/c125 0 2024-09-16T23:49:12.384 INFO:tasks.workunit.client.1.smithi043.stdout:2/961: readlink d8/d4b/d5f/d63/laa 0 2024-09-16T23:49:12.385 INFO:tasks.workunit.client.0.smithi042.stdout:8/740: truncate da/d19/d1d/d30/d49/fd9 223290 0 2024-09-16T23:49:12.386 INFO:tasks.workunit.client.0.smithi042.stdout:1/946: dread d0/d72/f65 [0,4194304] 0 2024-09-16T23:49:12.388 INFO:tasks.workunit.client.0.smithi042.stdout:7/887: rename d3/d98/f7d to d3/d98/da0/d24/dc3/f133 0 2024-09-16T23:49:12.388 INFO:tasks.workunit.client.0.smithi042.stdout:9/723: dwrite d4/d12/f36 [0,4194304] 0 2024-09-16T23:49:12.393 INFO:tasks.workunit.client.0.smithi042.stdout:0/953: dwrite d0/df/d23/d33/d56/da9/f92 [0,4194304] 0 2024-09-16T23:49:12.396 INFO:tasks.workunit.client.0.smithi042.stdout:6/919: chown d8/d70/d81/dd/d1b/d34/d39/d5e/d10b/cf5 128507 1 2024-09-16T23:49:12.396 INFO:tasks.workunit.client.0.smithi042.stdout:8/741: mknod da/d19/d1d/d30/d49/d58/ce8 0 2024-09-16T23:49:12.396 INFO:tasks.workunit.client.0.smithi042.stdout:7/888: mknod d3/d1f/d49/d76/d101/d64/d92/dfb/c134 0 2024-09-16T23:49:12.399 INFO:tasks.workunit.client.0.smithi042.stdout:6/920: read d8/d70/d81/f23 [2482372,103370] 0 2024-09-16T23:49:12.399 INFO:tasks.workunit.client.1.smithi043.stdout:2/962: truncate d8/d1d/d20/d23/fb8 7789704 0 2024-09-16T23:49:12.403 INFO:tasks.workunit.client.0.smithi042.stdout:1/947: write d0/d9/d90/f3a [4416306,109839] 0 2024-09-16T23:49:12.403 INFO:tasks.workunit.client.1.smithi043.stdout:2/963: symlink d8/d11/d111/l13b 0 2024-09-16T23:49:12.403 INFO:tasks.workunit.client.1.smithi043.stdout:2/964: readlink d8/d1f/l3e 0 2024-09-16T23:49:12.406 INFO:tasks.workunit.client.1.smithi043.stdout:2/965: fdatasync d8/d4b/fd6 0 2024-09-16T23:49:12.406 INFO:tasks.workunit.client.0.smithi042.stdout:1/948: fdatasync d0/d72/f110 0 2024-09-16T23:49:12.407 INFO:tasks.workunit.client.0.smithi042.stdout:8/742: write da/faa [1796080,43167] 0 2024-09-16T23:49:12.412 INFO:tasks.workunit.client.0.smithi042.stdout:2/770: dwrite d0/da/d17/d5c/fce [0,4194304] 0 2024-09-16T23:49:12.413 INFO:tasks.workunit.client.1.smithi043.stdout:2/966: symlink d8/d1d/d25/l13c 0 2024-09-16T23:49:12.413 INFO:tasks.workunit.client.1.smithi043.stdout:2/967: fdatasync d8/d4b/d4f/d7c/f88 0 2024-09-16T23:49:12.413 INFO:tasks.workunit.client.1.smithi043.stdout:2/968: readlink d8/d5e/d83/d3a/d77/da9/lcf 0 2024-09-16T23:49:12.414 INFO:tasks.workunit.client.0.smithi042.stdout:1/949: write d0/d17/d79/d118/f150 [37137,88442] 0 2024-09-16T23:49:12.416 INFO:tasks.workunit.client.0.smithi042.stdout:2/771: stat d0/da/d1b/d3b/d44/d54/l67 0 2024-09-16T23:49:12.418 INFO:tasks.workunit.client.0.smithi042.stdout:0/954: link d0/df/d13/d49/c95 d0/df/d23/d3e/c13f 0 2024-09-16T23:49:12.419 INFO:tasks.workunit.client.0.smithi042.stdout:9/724: dread d4/d12/d2d/d51/fbe [0,4194304] 0 2024-09-16T23:49:12.421 INFO:tasks.workunit.client.1.smithi043.stdout:2/969: dwrite d8/d35/f36 [0,4194304] 0 2024-09-16T23:49:12.422 INFO:tasks.workunit.client.0.smithi042.stdout:0/955: chown d0/df/d23/d3e/d8f/dda/l133 15088 1 2024-09-16T23:49:12.424 INFO:tasks.workunit.client.0.smithi042.stdout:2/772: unlink d0/da/d1b/c7e 0 2024-09-16T23:49:12.424 INFO:tasks.workunit.client.1.smithi043.stdout:2/970: truncate d8/d5e/fac 857998 0 2024-09-16T23:49:12.425 INFO:tasks.workunit.client.0.smithi042.stdout:1/950: dread d0/d9/d90/d6b/d88/f30 [0,4194304] 0 2024-09-16T23:49:12.426 INFO:tasks.workunit.client.1.smithi043.stdout:2/971: symlink d8/dec/l13d 0 2024-09-16T23:49:12.427 INFO:tasks.workunit.client.0.smithi042.stdout:0/956: creat d0/df/d13/d2d/d35/f140 x:0 0 0 2024-09-16T23:49:12.430 INFO:tasks.workunit.client.1.smithi043.stdout:2/972: truncate d8/d5e/d83/d3a/d77/fe6 366994 0 2024-09-16T23:49:12.430 INFO:tasks.workunit.client.0.smithi042.stdout:7/889: dwrite d3/d1f/d49/d76/d101/da5/f106 [0,4194304] 0 2024-09-16T23:49:12.437 INFO:tasks.workunit.client.1.smithi043.stdout:2/973: getdents d8/d1d/d20/d23/d38 0 2024-09-16T23:49:12.437 INFO:tasks.workunit.client.0.smithi042.stdout:0/957: fdatasync d0/df/d23/d33/d56/da9/fdd 0 2024-09-16T23:49:12.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/773: getdents d0/d30 0 2024-09-16T23:49:12.440 INFO:tasks.workunit.client.1.smithi043.stdout:2/974: dread d8/d5e/d83/d3a/d77/f9b [0,4194304] 0 2024-09-16T23:49:12.441 INFO:tasks.workunit.client.0.smithi042.stdout:7/890: symlink d3/d3f/d114/l135 0 2024-09-16T23:49:12.441 INFO:tasks.workunit.client.1.smithi043.stdout:8/971: sync 2024-09-16T23:49:12.443 INFO:tasks.workunit.client.0.smithi042.stdout:8/743: dwrite da/d19/d1d/d30/d49/f85 [0,4194304] 0 2024-09-16T23:49:12.445 INFO:tasks.workunit.client.0.smithi042.stdout:9/725: dwrite d4/d12/d2d/d94/dc2/d3a/d50/f99 [0,4194304] 0 2024-09-16T23:49:12.445 INFO:tasks.workunit.client.0.smithi042.stdout:1/951: truncate d0/d17/dc7/f131 859832 0 2024-09-16T23:49:12.447 INFO:tasks.workunit.client.0.smithi042.stdout:0/958: symlink d0/df/l141 0 2024-09-16T23:49:12.447 INFO:tasks.workunit.client.0.smithi042.stdout:6/921: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/f47 [0,4194304] 0 2024-09-16T23:49:12.450 INFO:tasks.workunit.client.0.smithi042.stdout:9/726: dread - d4/d12/d2d/dc9/dce/fca zero size 2024-09-16T23:49:12.451 INFO:tasks.workunit.client.1.smithi043.stdout:8/972: dwrite de/d18/d27/d41/d5a/f13 [0,4194304] 0 2024-09-16T23:49:12.452 INFO:tasks.workunit.client.0.smithi042.stdout:0/959: chown d0/df/d23/d33/d51/d79/fc2 66 1 2024-09-16T23:49:12.456 INFO:tasks.workunit.client.0.smithi042.stdout:1/952: symlink d0/d17/d79/d118/l151 0 2024-09-16T23:49:12.457 INFO:tasks.workunit.client.0.smithi042.stdout:1/953: stat d0/d9a/fbd 0 2024-09-16T23:49:12.459 INFO:tasks.workunit.client.1.smithi043.stdout:8/973: unlink de/d18/d27/d41/d5a/d106/d39/d7b/d83/da5/c118 0 2024-09-16T23:49:12.459 INFO:tasks.workunit.client.0.smithi042.stdout:9/727: creat d4/d12/d2d/d94/dc2/d41/d92/fdf x:0 0 0 2024-09-16T23:49:12.462 INFO:tasks.workunit.client.0.smithi042.stdout:6/922: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/dee/f126 x:0 0 0 2024-09-16T23:49:12.464 INFO:tasks.workunit.client.1.smithi043.stdout:8/974: mkdir de/d18/d27/d41/d5a/dc8/deb/d126 0 2024-09-16T23:49:12.465 INFO:tasks.workunit.client.1.smithi043.stdout:2/975: write d8/d1d/d20/d23/fb8 [1723485,70096] 0 2024-09-16T23:49:12.466 INFO:tasks.workunit.client.0.smithi042.stdout:0/960: symlink d0/df/d23/d33/d56/da9/l142 0 2024-09-16T23:49:12.468 INFO:tasks.workunit.client.0.smithi042.stdout:7/891: rename d3/d1f/d49/f56 to d3/f136 0 2024-09-16T23:49:12.469 INFO:tasks.workunit.client.0.smithi042.stdout:2/774: write d0/d5d/f79 [2113340,52392] 0 2024-09-16T23:49:12.471 INFO:tasks.workunit.client.0.smithi042.stdout:6/923: rmdir d8/d70/d81/dd/d1b/d34/d39/d5e/d10b 39 2024-09-16T23:49:12.471 INFO:tasks.workunit.client.1.smithi043.stdout:2/976: unlink d8/d11/dca/f115 0 2024-09-16T23:49:12.473 INFO:tasks.workunit.client.0.smithi042.stdout:0/961: fdatasync d0/df/d13/d49/d70/da1/fa7 0 2024-09-16T23:49:12.474 INFO:tasks.workunit.client.0.smithi042.stdout:2/775: rmdir d0/da/d17/d3e/d86 39 2024-09-16T23:49:12.475 INFO:tasks.workunit.client.0.smithi042.stdout:8/744: truncate da/d19/d1d/d30/d3f/dc2/fcd 438188 0 2024-09-16T23:49:12.475 INFO:tasks.workunit.client.1.smithi043.stdout:2/977: mkdir d8/d5e/d83/d9e/dab/dd3/d13e 0 2024-09-16T23:49:12.478 INFO:tasks.workunit.client.0.smithi042.stdout:0/962: chown d0/df/d13/d2d/d35/fa5 9346069 1 2024-09-16T23:49:12.479 INFO:tasks.workunit.client.0.smithi042.stdout:1/954: dwrite d0/d3b/fe0 [0,4194304] 0 2024-09-16T23:49:12.479 INFO:tasks.workunit.client.0.smithi042.stdout:2/776: creat d0/d78/f105 x:0 0 0 2024-09-16T23:49:12.480 INFO:tasks.workunit.client.0.smithi042.stdout:0/963: fdatasync d0/df/f102 0 2024-09-16T23:49:12.481 INFO:tasks.workunit.client.0.smithi042.stdout:9/728: rename d4/d12/d16/d68/f71 to d4/d10/d24/fe0 0 2024-09-16T23:49:12.482 INFO:tasks.workunit.client.1.smithi043.stdout:2/978: link d8/d1d/d20/d23/d38/l50 d8/d35/l13f 0 2024-09-16T23:49:12.484 INFO:tasks.workunit.client.1.smithi043.stdout:8/975: dread de/d18/d27/d41/d5a/f56 [0,4194304] 0 2024-09-16T23:49:12.485 INFO:tasks.workunit.client.1.smithi043.stdout:8/976: fdatasync de/fd0 0 2024-09-16T23:49:12.486 INFO:tasks.workunit.client.1.smithi043.stdout:2/979: mkdir d8/d5e/d83/d3a/d77/d140 0 2024-09-16T23:49:12.486 INFO:tasks.workunit.client.0.smithi042.stdout:1/955: creat d0/d3b/f152 x:0 0 0 2024-09-16T23:49:12.486 INFO:tasks.workunit.client.0.smithi042.stdout:8/745: dread da/d19/f39 [0,4194304] 0 2024-09-16T23:49:12.488 INFO:tasks.workunit.client.0.smithi042.stdout:6/924: rename d8/d70/d81/dd/d1b/d2d/da1/db3/led to d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d10c/l127 0 2024-09-16T23:49:12.489 INFO:tasks.workunit.client.0.smithi042.stdout:8/746: mkdir da/d19/d1d/d30/d49/d58/de9 0 2024-09-16T23:49:12.490 INFO:tasks.workunit.client.0.smithi042.stdout:7/892: link d3/d98/fce d3/d1f/f137 0 2024-09-16T23:49:12.490 INFO:tasks.workunit.client.1.smithi043.stdout:2/980: mkdir d8/d5e/d83/d3a/d77/da9/d119/d141 0 2024-09-16T23:49:12.491 INFO:tasks.workunit.client.0.smithi042.stdout:7/893: chown d3/d1f/d49/d76/d19/ddc/f12d 5 1 2024-09-16T23:49:12.492 INFO:tasks.workunit.client.1.smithi043.stdout:2/981: chown d8/d4b/d5f/d63/db6/d126/l12f 194274 1 2024-09-16T23:49:12.494 INFO:tasks.workunit.client.1.smithi043.stdout:2/982: truncate d8/d5e/d83/d3a/f134 860629 0 2024-09-16T23:49:12.496 INFO:tasks.workunit.client.0.smithi042.stdout:6/925: chown d8/l4d 341696 1 2024-09-16T23:49:12.497 INFO:tasks.workunit.client.0.smithi042.stdout:2/777: symlink d0/da/d1b/d3b/d44/d54/df3/l106 0 2024-09-16T23:49:12.500 INFO:tasks.workunit.client.1.smithi043.stdout:8/977: dread de/f71 [4194304,4194304] 0 2024-09-16T23:49:12.501 INFO:tasks.workunit.client.0.smithi042.stdout:0/964: rename d0/df/d23/d33/d51 to d0/df/d23/d33/d56/dc5/d143 0 2024-09-16T23:49:12.502 INFO:tasks.workunit.client.1.smithi043.stdout:2/983: dwrite d8/d1d/d25/f123 [0,4194304] 0 2024-09-16T23:49:12.507 INFO:tasks.workunit.client.0.smithi042.stdout:8/747: dread da/d19/d1d/d30/d3f/dc2/fcd [0,4194304] 0 2024-09-16T23:49:12.508 INFO:tasks.workunit.client.0.smithi042.stdout:7/894: rmdir d3 39 2024-09-16T23:49:12.510 INFO:tasks.workunit.client.1.smithi043.stdout:2/984: dread d8/d1d/d20/d23/d38/da2/f11b [0,4194304] 0 2024-09-16T23:49:12.510 INFO:tasks.workunit.client.0.smithi042.stdout:9/729: truncate d4/d12/d2d/d94/dc2/f70 1612801 0 2024-09-16T23:49:12.512 INFO:tasks.workunit.client.0.smithi042.stdout:1/956: mknod d0/d9/d26/c153 0 2024-09-16T23:49:12.512 INFO:tasks.workunit.client.0.smithi042.stdout:8/748: read - da/d19/d80/f8f zero size 2024-09-16T23:49:12.513 INFO:tasks.workunit.client.0.smithi042.stdout:2/778: creat d0/d8d/da0/f107 x:0 0 0 2024-09-16T23:49:12.514 INFO:tasks.workunit.client.0.smithi042.stdout:2/779: fsync d0/da/f2c 0 2024-09-16T23:49:12.514 INFO:tasks.workunit.client.0.smithi042.stdout:0/965: rmdir d0/df/df7 39 2024-09-16T23:49:12.515 INFO:tasks.workunit.client.1.smithi043.stdout:8/978: rename de/d18/d27/d8e/df3/d10a/d3d/daf/df5 to de/d18/d27/d41/d5a/d127 0 2024-09-16T23:49:12.516 INFO:tasks.workunit.client.0.smithi042.stdout:2/780: stat d0/da/d1b/d3b 0 2024-09-16T23:49:12.517 INFO:tasks.workunit.client.0.smithi042.stdout:0/966: readlink d0/df/d23/d33/d56/da9/l87 0 2024-09-16T23:49:12.519 INFO:tasks.workunit.client.0.smithi042.stdout:1/957: mkdir d0/d17/dd4/d154 0 2024-09-16T23:49:12.519 INFO:tasks.workunit.client.0.smithi042.stdout:7/895: chown d3/d1f/d49/d76/d101/d64/d92/ca8 38640166 1 2024-09-16T23:49:12.520 INFO:tasks.workunit.client.1.smithi043.stdout:8/979: creat de/d18/d27/d41/d5a/d106/d39/d4d/f128 x:0 0 0 2024-09-16T23:49:12.522 INFO:tasks.workunit.client.0.smithi042.stdout:6/926: creat d8/d70/d81/dd/d2c/d57/f128 x:0 0 0 2024-09-16T23:49:12.522 INFO:tasks.workunit.client.0.smithi042.stdout:2/781: mknod d0/da/d1b/d48/db7/c108 0 2024-09-16T23:49:12.524 INFO:tasks.workunit.client.1.smithi043.stdout:2/985: rename d8/d11/d16/f19 to d8/d4b/f142 0 2024-09-16T23:49:12.525 INFO:tasks.workunit.client.0.smithi042.stdout:8/749: dwrite da/d19/d1d/d30/d3f/d6d/fdc [0,4194304] 0 2024-09-16T23:49:12.525 INFO:tasks.workunit.client.1.smithi043.stdout:2/986: stat d8/d4b/d5f/d63/f76 0 2024-09-16T23:49:12.527 INFO:tasks.workunit.client.0.smithi042.stdout:9/730: read d4/d12/d2d/d94/dc2/f70 [145522,98056] 0 2024-09-16T23:49:12.528 INFO:tasks.workunit.client.1.smithi043.stdout:8/980: rename de/d18/d27/d41/d5a/d106/l67 to de/d18/d27/d41/d5a/dc8/deb/d126/l129 0 2024-09-16T23:49:12.530 INFO:tasks.workunit.client.1.smithi043.stdout:2/987: creat d8/d1d/d20/d26/f143 x:0 0 0 2024-09-16T23:49:12.530 INFO:tasks.workunit.client.1.smithi043.stdout:8/981: dread - de/d18/d27/d41/d5a/d106/d39/d7b/dd8/ffb zero size 2024-09-16T23:49:12.533 INFO:tasks.workunit.client.0.smithi042.stdout:0/967: creat d0/d15/de4/f144 x:0 0 0 2024-09-16T23:49:12.535 INFO:tasks.workunit.client.0.smithi042.stdout:2/782: mknod d0/d30/d43/c109 0 2024-09-16T23:49:12.536 INFO:tasks.workunit.client.0.smithi042.stdout:9/731: mkdir d4/d10/d24/de1 0 2024-09-16T23:49:12.536 INFO:tasks.workunit.client.0.smithi042.stdout:0/968: write d0/d48/f9c [3896447,114684] 0 2024-09-16T23:49:12.537 INFO:tasks.workunit.client.1.smithi043.stdout:8/982: rename de/d18/d27/d41/d5a/c33 to de/d18/d27/d41/d5a/d37/d6a/c12a 0 2024-09-16T23:49:12.541 INFO:tasks.workunit.client.0.smithi042.stdout:7/896: dwrite d3/d1f/d49/d78/dbe/dc8/fcc [0,4194304] 0 2024-09-16T23:49:12.542 INFO:tasks.workunit.client.0.smithi042.stdout:9/732: fsync d4/d12/d2d/d94/dc2/d28/f65 0 2024-09-16T23:49:12.545 INFO:tasks.workunit.client.1.smithi043.stdout:8/983: symlink de/d18/d1f/d25/d65/l12b 0 2024-09-16T23:49:12.546 INFO:tasks.workunit.client.0.smithi042.stdout:6/927: dread d8/d70/d81/f23 [0,4194304] 0 2024-09-16T23:49:12.546 INFO:tasks.workunit.client.0.smithi042.stdout:9/733: fsync d4/d10/d24/fe0 0 2024-09-16T23:49:12.548 INFO:tasks.workunit.client.0.smithi042.stdout:7/897: getdents d3/d1f/d49/d78/db9 0 2024-09-16T23:49:12.548 INFO:tasks.workunit.client.0.smithi042.stdout:8/750: write da/d4b/f87 [3054509,97470] 0 2024-09-16T23:49:12.548 INFO:tasks.workunit.client.1.smithi043.stdout:2/988: truncate d8/fc 971849 0 2024-09-16T23:49:12.551 INFO:tasks.workunit.client.0.smithi042.stdout:1/958: rename d0/d9/d90/d66/de2 to d0/d9/d90/d66/d155 0 2024-09-16T23:49:12.551 INFO:tasks.workunit.client.0.smithi042.stdout:7/898: chown d3/d3f/f90 100537212 1 2024-09-16T23:49:12.554 INFO:tasks.workunit.client.0.smithi042.stdout:1/959: chown d0/d9/d26/daa/dea 4 1 2024-09-16T23:49:12.556 INFO:tasks.workunit.client.0.smithi042.stdout:7/899: creat d3/d1f/d49/d78/dbe/dc8/f138 x:0 0 0 2024-09-16T23:49:12.558 INFO:tasks.workunit.client.0.smithi042.stdout:6/928: link d8/d70/d81/dd/d1b/d34/l35 d8/d70/d81/dd/d1b/d34/d39/d5e/l129 0 2024-09-16T23:49:12.559 INFO:tasks.workunit.client.1.smithi043.stdout:2/989: write d8/d5e/d83/d3a/d77/f9b [1927125,20114] 0 2024-09-16T23:49:12.561 INFO:tasks.workunit.client.0.smithi042.stdout:9/734: dwrite d4/f5 [0,4194304] 0 2024-09-16T23:49:12.562 INFO:tasks.workunit.client.0.smithi042.stdout:6/929: chown d8/d70/d81 72402 1 2024-09-16T23:49:12.563 INFO:tasks.workunit.client.1.smithi043.stdout:2/990: read d8/d5e/d83/d9e/fef [4638430,67009] 0 2024-09-16T23:49:12.564 INFO:tasks.workunit.client.0.smithi042.stdout:0/969: dread d0/f4 [0,4194304] 0 2024-09-16T23:49:12.565 INFO:tasks.workunit.client.1.smithi043.stdout:8/984: dwrite de/d18/d27/d41/d5a/d106/d39/d4d/fbf [0,4194304] 0 2024-09-16T23:49:12.568 INFO:tasks.workunit.client.0.smithi042.stdout:1/960: rename d0/d9/d26/f14b to d0/ddb/ddd/f156 0 2024-09-16T23:49:12.573 INFO:tasks.workunit.client.1.smithi043.stdout:8/985: symlink de/d18/d27/d41/d5a/d106/d39/d4d/l12c 0 2024-09-16T23:49:12.573 INFO:tasks.workunit.client.0.smithi042.stdout:6/930: creat d8/d70/d81/dd/d1b/d2d/f12a x:0 0 0 2024-09-16T23:49:12.575 INFO:tasks.workunit.client.1.smithi043.stdout:2/991: mkdir d8/d4b/d4f/d56/d133/d144 0 2024-09-16T23:49:12.575 INFO:tasks.workunit.client.0.smithi042.stdout:8/751: dwrite da/d19/d2c/f36 [0,4194304] 0 2024-09-16T23:49:12.576 INFO:tasks.workunit.client.0.smithi042.stdout:2/783: dwrite d0/f95 [0,4194304] 0 2024-09-16T23:49:12.579 INFO:tasks.workunit.client.0.smithi042.stdout:1/961: mknod d0/d17/d79/d92/c157 0 2024-09-16T23:49:12.580 INFO:tasks.workunit.client.1.smithi043.stdout:8/986: truncate de/d18/d27/d41/d5a/f1b 1536159 0 2024-09-16T23:49:12.580 INFO:tasks.workunit.client.1.smithi043.stdout:2/992: symlink d8/d5e/d83/d9e/dab/d114/l145 0 2024-09-16T23:49:12.583 INFO:tasks.workunit.client.0.smithi042.stdout:7/900: dwrite d3/d1f/d49/d76/d19/ff1 [4194304,4194304] 0 2024-09-16T23:49:12.585 INFO:tasks.workunit.client.1.smithi043.stdout:2/993: chown d8/d4b/d5f/d8e/de8/f130 117505139 1 2024-09-16T23:49:12.586 INFO:tasks.workunit.client.0.smithi042.stdout:2/784: mknod d0/d30/c10a 0 2024-09-16T23:49:12.587 INFO:tasks.workunit.client.0.smithi042.stdout:7/901: write d3/f9c [271421,91441] 0 2024-09-16T23:49:12.589 INFO:tasks.workunit.client.0.smithi042.stdout:8/752: mknod da/d19/d2c/d56/cea 0 2024-09-16T23:49:12.590 INFO:tasks.workunit.client.0.smithi042.stdout:0/970: dwrite d0/df/d23/d3e/faa [0,4194304] 0 2024-09-16T23:49:12.591 INFO:tasks.workunit.client.1.smithi043.stdout:8/987: dwrite de/d18/d27/d41/d5a/d106/d39/d7b/fba [0,4194304] 0 2024-09-16T23:49:12.591 INFO:tasks.workunit.client.1.smithi043.stdout:2/994: creat d8/d1d/d20/d23/d38/f146 x:0 0 0 2024-09-16T23:49:12.592 INFO:tasks.workunit.client.0.smithi042.stdout:7/902: chown d3/d1f/d49/d76/d19/ddc/dcd/ffd 0 1 2024-09-16T23:49:12.593 INFO:tasks.workunit.client.0.smithi042.stdout:6/931: readlink d8/d70/d81/dd/d1b/d34/d39/d5e/d10b/l104 0 2024-09-16T23:49:12.593 INFO:tasks.workunit.client.0.smithi042.stdout:2/785: truncate d0/da/d17/f52 1430597 0 2024-09-16T23:49:12.596 INFO:tasks.workunit.client.1.smithi043.stdout:8/988: write de/d18/d27/d8e/df3/d10a/fa0 [523457,23354] 0 2024-09-16T23:49:12.597 INFO:tasks.workunit.client.0.smithi042.stdout:1/962: link d0/d17/dd4/f127 d0/d17/dc7/f158 0 2024-09-16T23:49:12.599 INFO:tasks.workunit.client.0.smithi042.stdout:1/963: dread - d0/d9/dba/fc5 zero size 2024-09-16T23:49:12.602 INFO:tasks.workunit.client.0.smithi042.stdout:6/932: read d8/d70/d81/dd/d1b/d34/d39/fa9 [586925,74330] 0 2024-09-16T23:49:12.602 INFO:tasks.workunit.client.1.smithi043.stdout:2/995: dwrite d8/d35/fde [0,4194304] 0 2024-09-16T23:49:12.604 INFO:tasks.workunit.client.0.smithi042.stdout:8/753: write da/d2f/f5e [2140026,59962] 0 2024-09-16T23:49:12.607 INFO:tasks.workunit.client.0.smithi042.stdout:7/903: creat d3/d1f/d49/d76/d19/ddc/dcd/db5/dc4/f139 x:0 0 0 2024-09-16T23:49:12.607 INFO:tasks.workunit.client.1.smithi043.stdout:8/989: creat de/d18/d2b/f12d x:0 0 0 2024-09-16T23:49:12.607 INFO:tasks.workunit.client.1.smithi043.stdout:2/996: rmdir d8/d5e/d83/ddb 39 2024-09-16T23:49:12.609 INFO:tasks.workunit.client.1.smithi043.stdout:2/997: readlink d8/l7f 0 2024-09-16T23:49:12.611 INFO:tasks.workunit.client.0.smithi042.stdout:0/971: dwrite d0/df/d23/d33/d56/f5d [0,4194304] 0 2024-09-16T23:49:12.611 INFO:tasks.workunit.client.1.smithi043.stdout:8/990: stat de/d18/d2b/df4/f110 0 2024-09-16T23:49:12.611 INFO:tasks.workunit.client.0.smithi042.stdout:6/933: mknod d8/d70/c12b 0 2024-09-16T23:49:12.617 INFO:tasks.workunit.client.0.smithi042.stdout:8/754: rename da/d19/d1d/f32 to da/d19/d2c/d56/d97/dad/dd0/feb 0 2024-09-16T23:49:12.622 INFO:tasks.workunit.client.0.smithi042.stdout:8/755: mkdir da/d19/d1d/d30/d49/d58/d6a/dec 0 2024-09-16T23:49:12.625 INFO:tasks.workunit.client.0.smithi042.stdout:2/786: creat d0/d30/d43/d77/f10b x:0 0 0 2024-09-16T23:49:12.627 INFO:tasks.workunit.client.1.smithi043.stdout:2/998: truncate d8/d35/f36 723754 0 2024-09-16T23:49:12.628 INFO:tasks.workunit.client.0.smithi042.stdout:2/787: fdatasync d0/da/d88/fe8 0 2024-09-16T23:49:12.629 INFO:tasks.workunit.client.0.smithi042.stdout:2/788: chown d0/da/d17/d5c/fce 6456 1 2024-09-16T23:49:12.630 INFO:tasks.workunit.client.0.smithi042.stdout:2/789: write d0/da/d88/ff5 [829053,13947] 0 2024-09-16T23:49:12.630 INFO:tasks.workunit.client.0.smithi042.stdout:7/904: rename d3/d1f/d49/d76/d19/ddc/dcd to d3/d1f/d49/d78/dbe/dc8/d125/d13a 0 2024-09-16T23:49:12.631 INFO:tasks.workunit.client.1.smithi043.stdout:8/991: truncate de/d18/d27/d41/f50 3932651 0 2024-09-16T23:49:12.632 INFO:tasks.workunit.client.0.smithi042.stdout:8/756: truncate da/d19/d1d/f62 1736376 0 2024-09-16T23:49:12.633 INFO:tasks.workunit.client.0.smithi042.stdout:6/934: truncate d8/d70/d81/dd/d1b/dbf/d11f/d72/f2e 7784045 0 2024-09-16T23:49:12.634 INFO:tasks.workunit.client.0.smithi042.stdout:8/757: write da/d19/d1d/d30/d49/f85 [83848,23884] 0 2024-09-16T23:49:12.635 INFO:tasks.workunit.client.1.smithi043.stdout:2/999: dwrite d8/d4b/d5f/d8e/de8/feb [0,4194304] 0 2024-09-16T23:49:12.636 INFO:tasks.workunit.client.1.smithi043.stdout:8/992: mknod de/d18/d27/d41/d5a/d106/d39/d4d/c12e 0 2024-09-16T23:49:12.636 INFO:tasks.workunit.client.0.smithi042.stdout:2/790: rmdir d0/da/d1b/d3b/d44 39 2024-09-16T23:49:12.638 INFO:tasks.workunit.client.0.smithi042.stdout:7/905: dread d3/d1f/d49/d78/dbe/fda [0,4194304] 0 2024-09-16T23:49:12.640 INFO:tasks.workunit.client.0.smithi042.stdout:0/972: rename d0/df/d13/c6c to d0/df/d23/d33/d56/dc5/c145 0 2024-09-16T23:49:12.643 INFO:tasks.workunit.client.0.smithi042.stdout:6/935: dwrite d8/d70/d81/dd/d1b/d2d/f12a [0,4194304] 0 2024-09-16T23:49:12.645 INFO:tasks.workunit.client.0.smithi042.stdout:8/758: dwrite da/d19/d2c/dc8/fe5 [0,4194304] 0 2024-09-16T23:49:12.650 INFO:tasks.workunit.client.0.smithi042.stdout:6/936: write d8/d70/d81/dd/d2c/d57/f128 [480656,6032] 0 2024-09-16T23:49:12.653 INFO:tasks.workunit.client.1.smithi043.stdout:8/993: dread de/d18/d1f/d25/fb9 [4194304,4194304] 0 2024-09-16T23:49:12.656 INFO:tasks.workunit.client.0.smithi042.stdout:2/791: dwrite d0/d30/d43/f62 [0,4194304] 0 2024-09-16T23:49:12.657 INFO:tasks.workunit.client.1.smithi043.stdout:8/994: read de/d18/d1f/d25/f6b [1906375,119866] 0 2024-09-16T23:49:12.657 INFO:tasks.workunit.client.0.smithi042.stdout:0/973: dwrite d0/df/d13/ff9 [0,4194304] 0 2024-09-16T23:49:12.661 INFO:tasks.workunit.client.0.smithi042.stdout:6/937: write d8/d70/d81/dd/d1b/d50/d96/da0/dd7/f105 [108205,72962] 0 2024-09-16T23:49:12.662 INFO:tasks.workunit.client.0.smithi042.stdout:8/759: dread da/d19/d2c/f36 [0,4194304] 0 2024-09-16T23:49:12.665 INFO:tasks.workunit.client.0.smithi042.stdout:2/792: mkdir d0/d5d/d10c 0 2024-09-16T23:49:12.666 INFO:tasks.workunit.client.1.smithi043.stdout:8/995: dread de/d18/d27/d8e/f80 [4194304,4194304] 0 2024-09-16T23:49:12.671 INFO:tasks.workunit.client.1.smithi043.stdout:8/996: mknod de/c12f 0 2024-09-16T23:49:12.678 INFO:tasks.workunit.client.0.smithi042.stdout:2/793: getdents d0/da/d17 0 2024-09-16T23:49:12.680 INFO:tasks.workunit.client.0.smithi042.stdout:2/794: symlink d0/d26/l10d 0 2024-09-16T23:49:12.683 INFO:tasks.workunit.client.1.smithi043.stdout:8/997: dwrite de/d18/d27/d41/d5a/dc8/deb/f117 [0,4194304] 0 2024-09-16T23:49:12.687 INFO:tasks.workunit.client.0.smithi042.stdout:0/974: truncate d0/df/d23/f107 3553638 0 2024-09-16T23:49:12.689 INFO:tasks.workunit.client.1.smithi043.stdout:8/998: dread de/d18/d27/d41/d5a/d106/d39/d7b/d83/f84 [0,4194304] 0 2024-09-16T23:49:12.689 INFO:tasks.workunit.client.0.smithi042.stdout:0/975: write d0/df/d23/d33/d56/f5d [544879,98920] 0 2024-09-16T23:49:12.694 INFO:tasks.workunit.client.0.smithi042.stdout:8/760: dread da/f71 [0,4194304] 0 2024-09-16T23:49:12.697 INFO:tasks.workunit.client.1.smithi043.stdout:8/999: dwrite de/d18/d27/d41/d5a/d106/d39/d7b/df6/ff8 [0,4194304] 0 2024-09-16T23:49:12.698 INFO:tasks.workunit.client.0.smithi042.stdout:2/795: dwrite d0/f87 [0,4194304] 0 2024-09-16T23:49:12.698 INFO:tasks.workunit.client.0.smithi042.stdout:6/938: dwrite d8/d70/d81/dd/d2c/f31 [0,4194304] 0 2024-09-16T23:49:12.699 INFO:tasks.workunit.client.0.smithi042.stdout:0/976: mknod d0/df/d13/dcb/c146 0 2024-09-16T23:49:12.700 INFO:tasks.workunit.client.0.smithi042.stdout:0/977: dread - d0/f104 zero size 2024-09-16T23:49:12.702 INFO:tasks.workunit.client.0.smithi042.stdout:8/761: link da/d4b/d74/c86 da/d19/d1d/d5a/d99/ced 0 2024-09-16T23:49:12.702 INFO:tasks.workunit.client.0.smithi042.stdout:6/939: rmdir d8/d70/d81/dd/d1b/dbf 39 2024-09-16T23:49:12.705 INFO:tasks.workunit.client.1.smithi043.stderr:+ rm -rf -- ./tmp.FE3mQPppf0 2024-09-16T23:49:12.707 INFO:tasks.workunit.client.0.smithi042.stdout:8/762: mknod da/d19/d1d/d5a/d99/ddf/cee 0 2024-09-16T23:49:12.710 INFO:tasks.workunit.client.0.smithi042.stdout:8/763: mkdir da/d4b/def 0 2024-09-16T23:49:12.711 INFO:tasks.workunit.client.0.smithi042.stdout:0/978: rename d0/df/d13/d2d/d35/f140 to d0/d15/de4/f147 0 2024-09-16T23:49:12.713 INFO:tasks.workunit.client.0.smithi042.stdout:8/764: mkdir da/d4b/d74/df0 0 2024-09-16T23:49:12.714 INFO:tasks.workunit.client.0.smithi042.stdout:0/979: symlink d0/df/d13/dcb/dcc/df8/l148 0 2024-09-16T23:49:12.715 INFO:tasks.workunit.client.0.smithi042.stdout:0/980: readlink d0/df/d23/d33/d56/l10e 0 2024-09-16T23:49:12.718 INFO:tasks.workunit.client.0.smithi042.stdout:8/765: getdents da/d19/d2c/dc8 0 2024-09-16T23:49:12.719 INFO:tasks.workunit.client.0.smithi042.stdout:0/981: chown d0/df/d23/d33/f5c 719134 1 2024-09-16T23:49:12.720 INFO:tasks.workunit.client.0.smithi042.stdout:6/940: dread d8/d70/d81/dd/d1b/d50/f98 [0,4194304] 0 2024-09-16T23:49:12.723 INFO:tasks.workunit.client.0.smithi042.stdout:0/982: unlink d0/d15/de4/f147 0 2024-09-16T23:49:12.723 INFO:tasks.workunit.client.0.smithi042.stdout:8/766: dread da/f71 [0,4194304] 0 2024-09-16T23:49:12.724 INFO:tasks.workunit.client.0.smithi042.stdout:2/796: dread d0/da/d17/d5c/fa8 [4194304,4194304] 0 2024-09-16T23:49:12.727 INFO:tasks.workunit.client.0.smithi042.stdout:8/767: mknod da/d4b/d74/dd6/cf1 0 2024-09-16T23:49:12.727 INFO:tasks.workunit.client.0.smithi042.stdout:6/941: rename d8/d70/d81/dd/d1b/d2d/da1/db3 to d8/d70/d81/dd/d1b/d34/d39/d12c 0 2024-09-16T23:49:12.728 INFO:tasks.workunit.client.0.smithi042.stdout:0/983: creat d0/df/d23/d3e/d8f/dda/d11e/f149 x:0 0 0 2024-09-16T23:49:12.729 INFO:tasks.workunit.client.0.smithi042.stdout:6/942: read d8/d70/d81/dd/d2c/d57/f128 [301608,16511] 0 2024-09-16T23:49:12.732 INFO:tasks.workunit.client.0.smithi042.stdout:8/768: write da/d19/d1d/f62 [1884046,53994] 0 2024-09-16T23:49:12.732 INFO:tasks.workunit.client.0.smithi042.stdout:2/797: rename d0/da/d17/d5c/l74 to d0/da/d1b/d3b/d44/l10e 0 2024-09-16T23:49:12.735 INFO:tasks.workunit.client.0.smithi042.stdout:8/769: creat da/d19/d2c/db0/ff2 x:0 0 0 2024-09-16T23:49:12.735 INFO:tasks.workunit.client.0.smithi042.stdout:0/984: rename d0/df/f10 to d0/df/d23/d3e/d8f/f14a 0 2024-09-16T23:49:12.736 INFO:tasks.workunit.client.0.smithi042.stdout:0/985: readlink d0/df/d23/l46 0 2024-09-16T23:49:12.737 INFO:tasks.workunit.client.0.smithi042.stdout:0/986: write d0/df/d23/d33/d56/da9/fdd [1548112,22263] 0 2024-09-16T23:49:12.739 INFO:tasks.workunit.client.0.smithi042.stdout:2/798: rename d0/da/d88/fa4 to d0/da/d88/f10f 0 2024-09-16T23:49:12.739 INFO:tasks.workunit.client.0.smithi042.stdout:8/770: fdatasync da/d19/d2c/f66 0 2024-09-16T23:49:12.741 INFO:tasks.workunit.client.0.smithi042.stdout:8/771: dread - da/d19/d80/f8f zero size 2024-09-16T23:49:12.743 INFO:tasks.workunit.client.0.smithi042.stdout:2/799: mknod d0/d30/c110 0 2024-09-16T23:49:12.745 INFO:tasks.workunit.client.0.smithi042.stdout:0/987: link d0/df/d23/d3e/f54 d0/df/d13/dd6/f14b 0 2024-09-16T23:49:12.746 INFO:tasks.workunit.client.0.smithi042.stdout:0/988: fdatasync d0/df/d13/df1/f118 0 2024-09-16T23:49:12.746 INFO:tasks.workunit.client.0.smithi042.stdout:2/800: read - d0/d5d/f82 zero size 2024-09-16T23:49:12.748 INFO:tasks.workunit.client.0.smithi042.stdout:2/801: rmdir d0/d26/df4 39 2024-09-16T23:49:12.748 INFO:tasks.workunit.client.0.smithi042.stdout:6/943: dread d8/d70/d81/dd/d1b/dbf/d11f/d72/f32 [0,4194304] 0 2024-09-16T23:49:12.750 INFO:tasks.workunit.client.0.smithi042.stdout:0/989: creat d0/df/d23/d3e/f14c x:0 0 0 2024-09-16T23:49:12.753 INFO:tasks.workunit.client.0.smithi042.stdout:2/802: mknod d0/da/d1b/d3b/c111 0 2024-09-16T23:49:12.755 INFO:tasks.workunit.client.0.smithi042.stdout:6/944: symlink d8/d70/d81/dd/d2c/d57/l12d 0 2024-09-16T23:49:12.757 INFO:tasks.workunit.client.0.smithi042.stdout:0/990: dwrite d0/df/d13/dcb/f111 [0,4194304] 0 2024-09-16T23:49:12.757 INFO:tasks.workunit.client.0.smithi042.stdout:2/803: creat d0/da/d1b/d3b/d44/d54/f112 x:0 0 0 2024-09-16T23:49:12.763 INFO:tasks.workunit.client.0.smithi042.stdout:2/804: rename d0/d26/f9f to d0/da/d88/dae/dff/f113 0 2024-09-16T23:49:12.771 INFO:tasks.workunit.client.0.smithi042.stdout:2/805: dwrite d0/da/d88/fe8 [0,4194304] 0 2024-09-16T23:49:12.775 INFO:tasks.workunit.client.0.smithi042.stdout:0/991: rename d0/df/d23/d33/d56/dc5/dcd to d0/d15/de4/d14d 0 2024-09-16T23:49:12.778 INFO:tasks.workunit.client.0.smithi042.stdout:2/806: fdatasync d0/d30/f38 0 2024-09-16T23:49:12.779 INFO:tasks.workunit.client.0.smithi042.stdout:6/945: rmdir d8/d70/d81/dd/d2c 39 2024-09-16T23:49:12.782 INFO:tasks.workunit.client.0.smithi042.stdout:2/807: stat d0/c72 0 2024-09-16T23:49:12.782 INFO:tasks.workunit.client.0.smithi042.stdout:2/808: stat d0/d8d 0 2024-09-16T23:49:12.784 INFO:tasks.workunit.client.0.smithi042.stdout:0/992: link d0/df/d23/d33/d56/da9/f92 d0/df/d13/d2d/d35/d9f/f14e 0 2024-09-16T23:49:12.789 INFO:tasks.workunit.client.0.smithi042.stdout:2/809: symlink d0/da/d17/d5c/df6/l114 0 2024-09-16T23:49:12.790 INFO:tasks.workunit.client.0.smithi042.stdout:6/946: chown d8/d70/d81/dd/d1b/d67/d97/cfc 2817544 1 2024-09-16T23:49:12.791 INFO:tasks.workunit.client.0.smithi042.stdout:0/993: dwrite d0/df/d13/d49/ff3 [0,4194304] 0 2024-09-16T23:49:12.794 INFO:tasks.workunit.client.0.smithi042.stdout:0/994: fsync d0/df/df7/d127/f12d 0 2024-09-16T23:49:12.796 INFO:tasks.workunit.client.0.smithi042.stdout:2/810: rename d0/da/d1b/d3b/d44/fb3 to d0/da/d1b/d3b/d44/f115 0 2024-09-16T23:49:12.800 INFO:tasks.workunit.client.0.smithi042.stdout:2/811: fsync d0/d30/d43/f62 0 2024-09-16T23:49:12.801 INFO:tasks.workunit.client.0.smithi042.stdout:6/947: link d8/d70/d81/dd/d1b/d50/f98 d8/d70/d81/dd/d1b/d50/d96/da0/dd7/f12e 0 2024-09-16T23:49:12.803 INFO:tasks.workunit.client.0.smithi042.stdout:6/948: stat d8/d70/d81/dd/d1b/dbf/dde/f7f 0 2024-09-16T23:49:12.804 INFO:tasks.workunit.client.0.smithi042.stdout:2/812: rmdir d0/d30/dcb 39 2024-09-16T23:49:12.805 INFO:tasks.workunit.client.0.smithi042.stdout:0/995: dwrite d0/df/d23/d33/d56/dc5/fc6 [0,4194304] 0 2024-09-16T23:49:12.806 INFO:tasks.workunit.client.0.smithi042.stdout:2/813: write d0/da/fcc [392049,125897] 0 2024-09-16T23:49:12.808 INFO:tasks.workunit.client.0.smithi042.stdout:6/949: symlink d8/d70/d81/dd/d1b/d34/d39/db4/l12f 0 2024-09-16T23:49:12.815 INFO:tasks.workunit.client.0.smithi042.stdout:6/950: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/dee/f130 x:0 0 0 2024-09-16T23:49:12.815 INFO:tasks.workunit.client.0.smithi042.stdout:2/814: dwrite d0/da/d88/fe8 [4194304,4194304] 0 2024-09-16T23:49:12.825 INFO:tasks.workunit.client.0.smithi042.stdout:0/996: link d0/d15/c80 d0/df/d23/d33/d56/dc5/d143/c14f 0 2024-09-16T23:49:12.825 INFO:tasks.workunit.client.0.smithi042.stdout:2/815: dwrite d0/d30/d43/d77/f10b [0,4194304] 0 2024-09-16T23:49:12.825 INFO:tasks.workunit.client.0.smithi042.stdout:6/951: dread d8/d70/d81/dd/d2c/d57/f128 [0,4194304] 0 2024-09-16T23:49:12.829 INFO:tasks.workunit.client.0.smithi042.stdout:6/952: mkdir d8/d70/d81/dd/d2c/d131 0 2024-09-16T23:49:12.831 INFO:tasks.workunit.client.0.smithi042.stdout:0/997: dread d0/df/d23/d33/f126 [4194304,4194304] 0 2024-09-16T23:49:12.834 INFO:tasks.workunit.client.0.smithi042.stdout:6/953: symlink d8/d70/d81/dd/d1b/d34/d39/l132 0 2024-09-16T23:49:12.836 INFO:tasks.workunit.client.0.smithi042.stdout:2/816: getdents d0/da/d1b/d3b/d44/d54 0 2024-09-16T23:49:12.836 INFO:tasks.workunit.client.0.smithi042.stdout:2/817: dread - d0/d30/d43/df0/ff9 zero size 2024-09-16T23:49:12.837 INFO:tasks.workunit.client.0.smithi042.stdout:6/954: mknod d8/d70/d81/dd/d1b/d50/d96/c133 0 2024-09-16T23:49:12.839 INFO:tasks.workunit.client.0.smithi042.stdout:2/818: fdatasync d0/d26/f4e 0 2024-09-16T23:49:12.842 INFO:tasks.workunit.client.0.smithi042.stdout:6/955: symlink d8/d70/d81/dd/d1b/d50/d96/da0/dd0/l134 0 2024-09-16T23:49:12.844 INFO:tasks.workunit.client.0.smithi042.stdout:2/819: chown d0/da/l36 3 1 2024-09-16T23:49:12.845 INFO:tasks.workunit.client.0.smithi042.stdout:0/998: link d0/df/d13/d2d/c84 d0/c150 0 2024-09-16T23:49:12.845 INFO:tasks.workunit.client.0.smithi042.stdout:0/999: readlink d0/d15/de4/le3 0 2024-09-16T23:49:12.845 INFO:tasks.workunit.client.0.smithi042.stdout:2/820: dread - d0/da/d1b/d48/fc5 zero size 2024-09-16T23:49:12.846 INFO:tasks.workunit.client.0.smithi042.stdout:2/821: truncate d0/d8d/da0/fd6 1017497 0 2024-09-16T23:49:12.848 INFO:tasks.workunit.client.0.smithi042.stdout:6/956: link d8/d70/d81/dd/d1b/dbf/d11f/d72/d99/le7 d8/d70/d81/dd/d1b/d34/l135 0 2024-09-16T23:49:12.850 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:12 smithi043 ceph-mon[35191]: pgmap v15: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 95 MiB/s rd, 234 MiB/s wr, 490 op/s 2024-09-16T23:49:12.850 INFO:tasks.workunit.client.0.smithi042.stdout:6/957: write d8/d70/d81/dd/d1b/d50/da2/f114 [944721,79692] 0 2024-09-16T23:49:12.850 INFO:tasks.workunit.client.0.smithi042.stdout:2/822: creat d0/da/d17/d3e/f116 x:0 0 0 2024-09-16T23:49:12.852 INFO:tasks.workunit.client.0.smithi042.stdout:2/823: read d0/da/d1b/d3b/d44/f90 [4039184,41128] 0 2024-09-16T23:49:12.854 INFO:tasks.workunit.client.0.smithi042.stdout:2/824: dread - d0/d30/dcb/de5/fda zero size 2024-09-16T23:49:12.857 INFO:tasks.workunit.client.0.smithi042.stdout:6/958: link d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/d10c/lc0 d8/d70/d81/dd/d1b/d34/d39/l136 0 2024-09-16T23:49:12.861 INFO:tasks.workunit.client.0.smithi042.stdout:2/825: dwrite d0/da/d17/d3e/fef [0,4194304] 0 2024-09-16T23:49:12.866 INFO:tasks.workunit.client.0.smithi042.stdout:2/826: chown d0/da/d17/d3e/d86/d96/lfb 7 1 2024-09-16T23:49:12.869 INFO:tasks.workunit.client.0.smithi042.stdout:2/827: creat d0/da/d1b/d3b/d44/dec/f117 x:0 0 0 2024-09-16T23:49:12.871 INFO:tasks.workunit.client.0.smithi042.stdout:6/959: dread d8/d70/d81/dd/d1b/d2d/da1/fc1 [0,4194304] 0 2024-09-16T23:49:12.874 INFO:tasks.workunit.client.0.smithi042.stdout:2/828: getdents d0/d30/d43/d77 0 2024-09-16T23:49:12.877 INFO:tasks.workunit.client.0.smithi042.stdout:2/829: creat d0/da/d17/d3e/d86/d96/f118 x:0 0 0 2024-09-16T23:49:12.877 INFO:tasks.workunit.client.0.smithi042.stdout:2/830: readlink d0/da/d17/lfd 0 2024-09-16T23:49:12.878 INFO:tasks.workunit.client.0.smithi042.stdout:6/960: creat d8/d70/d81/dd/d1b/dbf/d11f/d72/f137 x:0 0 0 2024-09-16T23:49:12.881 INFO:tasks.workunit.client.0.smithi042.stdout:2/831: creat d0/da/d17/d3e/d86/f119 x:0 0 0 2024-09-16T23:49:12.884 INFO:tasks.workunit.client.0.smithi042.stdout:6/961: link d8/d70/d81/dd/d1b/dbf/dde/f6e d8/d70/d81/dd/d1b/dbf/d11f/d72/f138 0 2024-09-16T23:49:12.886 INFO:tasks.workunit.client.0.smithi042.stdout:2/832: dwrite d0/da/d88/ff5 [0,4194304] 0 2024-09-16T23:49:12.891 INFO:tasks.workunit.client.0.smithi042.stdout:6/962: rename d8/d70/d81/dd/d1b/dbf/d11f/d72/f138 to d8/d70/d81/dd/d2c/d57/f139 0 2024-09-16T23:49:12.894 INFO:tasks.workunit.client.0.smithi042.stdout:6/963: creat d8/d70/d81/dd/d1b/d34/d39/d5e/d5a/f13a x:0 0 0 2024-09-16T23:49:12.897 INFO:tasks.workunit.client.0.smithi042.stdout:6/964: creat d8/d70/d81/dd/d1b/d34/d39/d12c/f13b x:0 0 0 2024-09-16T23:49:12.898 INFO:tasks.workunit.client.0.smithi042.stdout:6/965: fsync d8/d70/d81/dd/d1b/f102 0 2024-09-16T23:49:12.901 INFO:tasks.workunit.client.0.smithi042.stdout:6/966: mkdir d8/d70/d81/dd/dcd/d13c 0 2024-09-16T23:49:12.911 INFO:tasks.workunit.client.0.smithi042.stdout:6/967: write d8/d70/d81/dd/f5f [2527586,30282] 0 2024-09-16T23:49:12.912 INFO:tasks.workunit.client.0.smithi042.stdout:6/968: rmdir d8/d70/d81/dd/d1b/dbf 39 2024-09-16T23:49:12.916 INFO:tasks.workunit.client.0.smithi042.stdout:6/969: creat d8/d70/d81/dd/d1b/d67/d97/d111/f13d x:0 0 0 2024-09-16T23:49:12.922 INFO:tasks.workunit.client.0.smithi042.stdout:6/970: dwrite d8/d70/d81/dd/dcd/fec [0,4194304] 0 2024-09-16T23:49:12.924 INFO:tasks.workunit.client.0.smithi042.stdout:6/971: fdatasync d8/d70/d81/dd/d2c/d57/f128 0 2024-09-16T23:49:12.937 INFO:tasks.workunit.client.0.smithi042.stdout:6/972: dwrite d8/d70/d81/dd/d1b/dbf/d11f/d72/f32 [4194304,4194304] 0 2024-09-16T23:49:12.948 INFO:tasks.workunit.client.0.smithi042.stdout:6/973: dwrite d8/d70/d81/dd/d1b/d34/d39/d5e/ff9 [0,4194304] 0 2024-09-16T23:49:12.955 INFO:tasks.workunit.client.0.smithi042.stdout:6/974: truncate d8/d70/d81/dd/d1b/f93 1296141 0 2024-09-16T23:49:12.961 INFO:tasks.workunit.client.0.smithi042.stdout:6/975: rename d8/d70/d81/dd/d1b/d34/d39/cfa to d8/d70/d81/dd/d1b/d67/c13e 0 2024-09-16T23:49:12.962 INFO:tasks.workunit.client.0.smithi042.stdout:6/976: write d8/d70/d81/dd/d2c/d57/fca [2477927,14795] 0 2024-09-16T23:49:12.964 INFO:tasks.workunit.client.0.smithi042.stdout:6/977: chown d8/d70/d81/dd/d1b/dbf 274 1 2024-09-16T23:49:12.969 INFO:tasks.workunit.client.0.smithi042.stdout:6/978: rename d8/d70/d81/dd/d1b/d50/d96/da0/dd7/f105 to d8/d70/d81/dd/d1b/d50/d96/da0/dd0/f13f 0 2024-09-16T23:49:12.970 INFO:tasks.workunit.client.0.smithi042.stdout:6/979: chown d8/d70/d81/dd/d2c/d57/f139 58776 1 2024-09-16T23:49:12.978 INFO:tasks.workunit.client.0.smithi042.stdout:6/980: truncate d8/d70/d81/dd/d1b/d50/da2/f114 873706 0 2024-09-16T23:49:12.990 INFO:tasks.workunit.client.0.smithi042.stdout:6/981: dread d8/d70/d81/dd/f5f [0,4194304] 0 2024-09-16T23:49:12.992 INFO:tasks.workunit.client.0.smithi042.stdout:6/982: mknod d8/d70/d81/dd/d1b/d67/dc7/c140 0 2024-09-16T23:49:12.996 INFO:tasks.workunit.client.0.smithi042.stdout:6/983: truncate d8/d70/d81/dd/d1b/d34/d39/d5e/d4c/f78 4010848 0 2024-09-16T23:49:13.012 INFO:tasks.workunit.client.0.smithi042.stdout:6/984: unlink d8/d70/d81/dd/dcd/fac 0 2024-09-16T23:49:13.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:12 smithi042 ceph-mon[27914]: pgmap v15: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 95 MiB/s rd, 234 MiB/s wr, 490 op/s 2024-09-16T23:49:13.026 INFO:tasks.workunit.client.0.smithi042.stdout:6/985: truncate d8/d70/d81/dd/d1b/d50/da2/f114 1288258 0 2024-09-16T23:49:13.030 INFO:tasks.workunit.client.0.smithi042.stdout:6/986: dread d8/d70/d81/dd/d1b/d2d/f41 [0,4194304] 0 2024-09-16T23:49:13.032 INFO:tasks.workunit.client.0.smithi042.stdout:6/987: symlink d8/d70/d81/dd/dcd/de1/l141 0 2024-09-16T23:49:13.037 INFO:tasks.workunit.client.0.smithi042.stdout:6/988: link d8/d70/d81/dd/d1b/d50/d96/da0/f117 d8/d70/d81/dd/d2c/d57/dc5/f142 0 2024-09-16T23:49:13.038 INFO:tasks.workunit.client.0.smithi042.stdout:6/989: chown d8/d70/d81/dd/d1b/d34/c4a 168094719 1 2024-09-16T23:49:13.040 INFO:tasks.workunit.client.0.smithi042.stdout:6/990: mknod d8/d70/d81/dd/d1b/dbf/dde/c143 0 2024-09-16T23:49:13.042 INFO:tasks.workunit.client.0.smithi042.stdout:6/991: creat d8/d70/d81/dd/d2c/d57/dc5/f144 x:0 0 0 2024-09-16T23:49:13.043 INFO:tasks.workunit.client.0.smithi042.stdout:6/992: chown d8/d70/d81/dd/l14 1401345373 1 2024-09-16T23:49:13.045 INFO:tasks.workunit.client.0.smithi042.stdout:6/993: fsync d8/d70/d81/dd/d1b/f93 0 2024-09-16T23:49:13.049 INFO:tasks.workunit.client.0.smithi042.stdout:6/994: mknod d8/d70/d81/dd/d1b/dbf/d11f/d72/c145 0 2024-09-16T23:49:13.054 INFO:tasks.workunit.client.0.smithi042.stdout:6/995: creat d8/d70/d81/dd/f146 x:0 0 0 2024-09-16T23:49:13.057 INFO:tasks.workunit.client.0.smithi042.stdout:6/996: rename d8/d70/d81/dd/d1b/d34/d39/l132 to d8/d70/d81/dd/d1b/d50/d96/da0/l147 0 2024-09-16T23:49:13.065 INFO:tasks.workunit.client.0.smithi042.stdout:6/997: truncate d8/d70/d81/dd/d2c/d57/fca 1786514 0 2024-09-16T23:49:13.072 INFO:tasks.workunit.client.0.smithi042.stdout:6/998: dwrite d8/d70/d81/dd/f22 [0,4194304] 0 2024-09-16T23:49:13.074 INFO:tasks.workunit.client.0.smithi042.stdout:6/999: chown d8/d70/d81/dd/d1b/f9d 131 1 2024-09-16T23:49:13.222 INFO:tasks.workunit.client.0.smithi042.stdout:9/735: sync 2024-09-16T23:49:13.223 INFO:tasks.workunit.client.0.smithi042.stdout:2/833: sync 2024-09-16T23:49:13.223 INFO:tasks.workunit.client.0.smithi042.stdout:1/964: sync 2024-09-16T23:49:13.223 INFO:tasks.workunit.client.0.smithi042.stdout:7/906: sync 2024-09-16T23:49:13.223 INFO:tasks.workunit.client.0.smithi042.stdout:8/772: sync 2024-09-16T23:49:13.225 INFO:tasks.workunit.client.0.smithi042.stdout:8/773: write da/d19/d1d/d30/d3f/d6d/fdc [291971,88437] 0 2024-09-16T23:49:13.226 INFO:tasks.workunit.client.0.smithi042.stdout:9/736: rmdir d4/d12/d2d/d94/dc2/d41/d92 39 2024-09-16T23:49:13.228 INFO:tasks.workunit.client.0.smithi042.stdout:1/965: dread d0/d9/f1c [0,4194304] 0 2024-09-16T23:49:13.232 INFO:tasks.workunit.client.0.smithi042.stdout:9/737: mkdir d4/d12/d2d/d94/dc2/de2 0 2024-09-16T23:49:13.239 INFO:tasks.workunit.client.0.smithi042.stdout:9/738: creat d4/d12/d16/d68/fe3 x:0 0 0 2024-09-16T23:49:13.240 INFO:tasks.workunit.client.0.smithi042.stdout:8/774: dwrite da/d19/d1d/d30/d49/d58/d6a/f8b [0,4194304] 0 2024-09-16T23:49:13.246 INFO:tasks.workunit.client.0.smithi042.stdout:9/739: link d4/d12/d16/d68/fe3 d4/d12/d2d/d51/db9/fe4 0 2024-09-16T23:49:13.249 INFO:tasks.workunit.client.0.smithi042.stdout:8/775: dwrite da/faa [4194304,4194304] 0 2024-09-16T23:49:13.254 INFO:tasks.workunit.client.0.smithi042.stdout:2/834: write d0/da/d17/d3e/d86/d96/f9b [620681,37874] 0 2024-09-16T23:49:13.260 INFO:tasks.workunit.client.0.smithi042.stdout:9/740: dwrite d4/d12/d2d/fc1 [4194304,4194304] 0 2024-09-16T23:49:13.264 INFO:tasks.workunit.client.0.smithi042.stdout:8/776: rename da/d19/d1d/d5a/d99/ced to da/d19/d1d/d30/d3f/d6d/cf3 0 2024-09-16T23:49:13.267 INFO:tasks.workunit.client.0.smithi042.stdout:8/777: readlink da/d19/l28 0 2024-09-16T23:49:13.268 INFO:tasks.workunit.client.0.smithi042.stdout:7/907: dwrite d3/d98/fed [0,4194304] 0 2024-09-16T23:49:13.270 INFO:tasks.workunit.client.0.smithi042.stdout:1/966: dwrite d0/d17/dc7/f131 [0,4194304] 0 2024-09-16T23:49:13.272 INFO:tasks.workunit.client.0.smithi042.stdout:2/835: dread d0/d30/d43/f62 [0,4194304] 0 2024-09-16T23:49:13.277 INFO:tasks.workunit.client.0.smithi042.stdout:2/836: dread d0/da/fcc [0,4194304] 0 2024-09-16T23:49:13.282 INFO:tasks.workunit.client.0.smithi042.stdout:1/967: mkdir d0/d17/d18/d159 0 2024-09-16T23:49:13.283 INFO:tasks.workunit.client.0.smithi042.stdout:9/741: link d4/d12/d2d/dc9/dce/c5f d4/d12/d2d/d94/dc2/ce5 0 2024-09-16T23:49:13.284 INFO:tasks.workunit.client.0.smithi042.stdout:1/968: readlink d0/ddb/ddd/l12f 0 2024-09-16T23:49:13.286 INFO:tasks.workunit.client.0.smithi042.stdout:2/837: dwrite d0/da/d17/d3e/fed [0,4194304] 0 2024-09-16T23:49:13.288 INFO:tasks.workunit.client.0.smithi042.stdout:1/969: fsync d0/d9/dba/fd1 0 2024-09-16T23:49:13.289 INFO:tasks.workunit.client.0.smithi042.stdout:9/742: readlink d4/d12/d2d/d94/dc2/d41/d92/da7/lbd 0 2024-09-16T23:49:13.292 INFO:tasks.workunit.client.0.smithi042.stdout:2/838: symlink d0/d30/d43/df0/l11a 0 2024-09-16T23:49:13.292 INFO:tasks.workunit.client.0.smithi042.stdout:8/778: truncate da/d19/f3b 6323962 0 2024-09-16T23:49:13.296 INFO:tasks.workunit.client.0.smithi042.stdout:9/743: rename d4/d12/d2d/d94/dc2/l4c to d4/d12/d77/dd1/le6 0 2024-09-16T23:49:13.297 INFO:tasks.workunit.client.0.smithi042.stdout:1/970: rmdir d0/d9/d26/dc6 39 2024-09-16T23:49:13.298 INFO:tasks.workunit.client.0.smithi042.stdout:7/908: write d3/d98/da0/d24/f59 [4114013,109764] 0 2024-09-16T23:49:13.300 INFO:tasks.workunit.client.0.smithi042.stdout:1/971: fsync d0/d9/fb9 0 2024-09-16T23:49:13.300 INFO:tasks.workunit.client.0.smithi042.stdout:7/909: read d3/d98/fed [2722183,112669] 0 2024-09-16T23:49:13.302 INFO:tasks.workunit.client.0.smithi042.stdout:2/839: rename d0/d26/l10d to d0/l11b 0 2024-09-16T23:49:13.303 INFO:tasks.workunit.client.0.smithi042.stdout:8/779: dwrite da/d19/d1d/d30/f41 [0,4194304] 0 2024-09-16T23:49:13.305 INFO:tasks.workunit.client.0.smithi042.stdout:8/780: readlink da/d19/d1d/d5a/ldb 0 2024-09-16T23:49:13.306 INFO:tasks.workunit.client.0.smithi042.stdout:1/972: creat d0/d3b/d8d/f15a x:0 0 0 2024-09-16T23:49:13.307 INFO:tasks.workunit.client.0.smithi042.stdout:7/910: creat d3/d1f/d49/d78/dbe/dc8/d125/f13b x:0 0 0 2024-09-16T23:49:13.308 INFO:tasks.workunit.client.0.smithi042.stdout:2/840: unlink d0/da/d1b/l25 0 2024-09-16T23:49:13.311 INFO:tasks.workunit.client.0.smithi042.stdout:9/744: creat d4/d12/d16/d68/fe7 x:0 0 0 2024-09-16T23:49:13.315 INFO:tasks.workunit.client.0.smithi042.stdout:1/973: rename d0/d9/dba/fc5 to d0/d9/d90/d6b/d8e/df8/f15b 0 2024-09-16T23:49:13.316 INFO:tasks.workunit.client.0.smithi042.stdout:2/841: readlink d0/l11b 0 2024-09-16T23:49:13.316 INFO:tasks.workunit.client.0.smithi042.stdout:8/781: getdents da/d19/d2c/db0 0 2024-09-16T23:49:13.317 INFO:tasks.workunit.client.0.smithi042.stdout:8/782: stat da/d4b/le0 0 2024-09-16T23:49:13.318 INFO:tasks.workunit.client.0.smithi042.stdout:9/745: creat d4/d12/d16/da9/fe8 x:0 0 0 2024-09-16T23:49:13.319 INFO:tasks.workunit.client.0.smithi042.stdout:7/911: mkdir d3/d1f/d49/d76/d101/da5/de3/d12b/d13c 0 2024-09-16T23:49:13.321 INFO:tasks.workunit.client.0.smithi042.stdout:8/783: creat da/d2f/ff4 x:0 0 0 2024-09-16T23:49:13.322 INFO:tasks.workunit.client.0.smithi042.stdout:2/842: link d0/da/d17/f42 d0/da/d1b/d3b/d44/f11c 0 2024-09-16T23:49:13.324 INFO:tasks.workunit.client.0.smithi042.stdout:2/843: fsync d0/f2b 0 2024-09-16T23:49:13.324 INFO:tasks.workunit.client.0.smithi042.stdout:7/912: stat d3/d1f/d49/d78/dbe/dc8/d125/d13a/db5 0 2024-09-16T23:49:13.328 INFO:tasks.workunit.client.0.smithi042.stdout:7/913: fsync d3/fe 0 2024-09-16T23:49:13.328 INFO:tasks.workunit.client.0.smithi042.stdout:9/746: dwrite d4/d10/f57 [0,4194304] 0 2024-09-16T23:49:13.329 INFO:tasks.workunit.client.0.smithi042.stdout:2/844: truncate d0/d30/dcb/de5/fc0 457575 0 2024-09-16T23:49:13.331 INFO:tasks.workunit.client.0.smithi042.stdout:9/747: write d4/d12/d2d/d94/dc2/d41/d92/fdf [76359,34755] 0 2024-09-16T23:49:13.333 INFO:tasks.workunit.client.0.smithi042.stdout:1/974: dread d0/d9/d90/d66/f105 [0,4194304] 0 2024-09-16T23:49:13.333 INFO:tasks.workunit.client.0.smithi042.stdout:9/748: dread - d4/d12/d60/d97/fa0 zero size 2024-09-16T23:49:13.333 INFO:tasks.workunit.client.0.smithi042.stdout:2/845: chown d0/d30/d43/ff1 7 1 2024-09-16T23:49:13.334 INFO:tasks.workunit.client.0.smithi042.stdout:8/784: write da/d19/d1d/d30/d3f/f75 [606552,95059] 0 2024-09-16T23:49:13.334 INFO:tasks.workunit.client.0.smithi042.stdout:1/975: chown d0/d17/f21 42372159 1 2024-09-16T23:49:13.335 INFO:tasks.workunit.client.0.smithi042.stdout:7/914: mknod d3/d1f/d49/d76/d19/c13d 0 2024-09-16T23:49:13.338 INFO:tasks.workunit.client.0.smithi042.stdout:2/846: rmdir d0/d30/d43 39 2024-09-16T23:49:13.340 INFO:tasks.workunit.client.0.smithi042.stdout:9/749: chown d4/d12/d16/d68/f72 5793379 1 2024-09-16T23:49:13.341 INFO:tasks.workunit.client.0.smithi042.stdout:2/847: fdatasync d0/da/d17/d3e/d86/d96/f9b 0 2024-09-16T23:49:13.342 INFO:tasks.workunit.client.0.smithi042.stdout:1/976: symlink d0/d9/d26/daa/dea/l15c 0 2024-09-16T23:49:13.344 INFO:tasks.workunit.client.0.smithi042.stdout:7/915: mkdir d3/d1f/d49/d76/d19/ddc/d13e 0 2024-09-16T23:49:13.345 INFO:tasks.workunit.client.0.smithi042.stdout:8/785: dwrite da/d19/d1d/d30/d3f/dc2/de3/fc5 [0,4194304] 0 2024-09-16T23:49:13.346 INFO:tasks.workunit.client.0.smithi042.stdout:9/750: creat d4/d12/d2d/d51/db9/fe9 x:0 0 0 2024-09-16T23:49:13.346 INFO:tasks.workunit.client.0.smithi042.stdout:7/916: write d3/f30 [2912604,78102] 0 2024-09-16T23:49:13.348 INFO:tasks.workunit.client.0.smithi042.stdout:1/977: fdatasync d0/d72/f65 0 2024-09-16T23:49:13.348 INFO:tasks.workunit.client.0.smithi042.stdout:9/751: dread - d4/d12/d60/fd4 zero size 2024-09-16T23:49:13.348 INFO:tasks.workunit.client.0.smithi042.stdout:2/848: mkdir d0/da/d17/d3e/d86/d96/d11d 0 2024-09-16T23:49:13.349 INFO:tasks.workunit.client.0.smithi042.stdout:8/786: mknod da/d19/db6/cf5 0 2024-09-16T23:49:13.350 INFO:tasks.workunit.client.0.smithi042.stdout:7/917: rmdir d3/d1f/d49/d76/d101/d5b 39 2024-09-16T23:49:13.354 INFO:tasks.workunit.client.0.smithi042.stdout:9/752: symlink d4/d12/d16/d68/lea 0 2024-09-16T23:49:13.356 INFO:tasks.workunit.client.0.smithi042.stdout:8/787: dread da/d19/d1d/d30/d3f/dc2/fcd [0,4194304] 0 2024-09-16T23:49:13.358 INFO:tasks.workunit.client.0.smithi042.stdout:1/978: truncate d0/d17/dd4/f127 1024897 0 2024-09-16T23:49:13.359 INFO:tasks.workunit.client.0.smithi042.stdout:7/918: mknod d3/d1f/d49/d78/db9/c13f 0 2024-09-16T23:49:13.363 INFO:tasks.workunit.client.0.smithi042.stdout:9/753: dread d4/d12/d2d/d94/dc2/d3a/f5b [0,4194304] 0 2024-09-16T23:49:13.364 INFO:tasks.workunit.client.0.smithi042.stdout:8/788: write da/d19/d1d/d30/d3f/dc2/fcd [341547,66975] 0 2024-09-16T23:49:13.368 INFO:tasks.workunit.client.0.smithi042.stdout:7/919: creat d3/d1f/d49/d76/d19/ddc/d13e/f140 x:0 0 0 2024-09-16T23:49:13.369 INFO:tasks.workunit.client.0.smithi042.stdout:2/849: write d0/da/d88/dae/f6b [887855,66492] 0 2024-09-16T23:49:13.377 INFO:tasks.workunit.client.0.smithi042.stdout:8/789: dwrite da/d19/d1d/d30/d3f/f61 [0,4194304] 0 2024-09-16T23:49:13.378 INFO:tasks.workunit.client.0.smithi042.stdout:2/850: rename d0/da/d1b/d3b/d44/d54/f5f to d0/da/d17/d3e/d86/f11e 0 2024-09-16T23:49:13.378 INFO:tasks.workunit.client.0.smithi042.stdout:1/979: dread d0/d9/d90/d6b/d88/d32/d2f/de3/f10c [0,4194304] 0 2024-09-16T23:49:13.383 INFO:tasks.workunit.client.0.smithi042.stdout:7/920: dwrite d3/f11b [0,4194304] 0 2024-09-16T23:49:13.383 INFO:tasks.workunit.client.0.smithi042.stdout:1/980: fdatasync d0/d17/f21 0 2024-09-16T23:49:13.385 INFO:tasks.workunit.client.0.smithi042.stdout:2/851: mkdir d0/da/d17/d5c/df6/d11f 0 2024-09-16T23:49:13.388 INFO:tasks.workunit.client.0.smithi042.stdout:8/790: mkdir da/d19/d2c/db0/df6 0 2024-09-16T23:49:13.389 INFO:tasks.workunit.client.0.smithi042.stdout:2/852: write d0/da/d1b/d3b/d44/dec/f101 [37030,39514] 0 2024-09-16T23:49:13.390 INFO:tasks.workunit.client.0.smithi042.stdout:7/921: read d3/d1f/d49/d76/d19/ff1 [3538910,13074] 0 2024-09-16T23:49:13.391 INFO:tasks.workunit.client.0.smithi042.stdout:1/981: mkdir d0/d9/d26/daa/dea/d15d 0 2024-09-16T23:49:13.392 INFO:tasks.workunit.client.0.smithi042.stdout:8/791: read - da/d19/d1d/d30/d49/db2/fd8 zero size 2024-09-16T23:49:13.393 INFO:tasks.workunit.client.0.smithi042.stdout:1/982: dread - d0/d9/d90/d6b/d8e/df8/f140 zero size 2024-09-16T23:49:13.393 INFO:tasks.workunit.client.0.smithi042.stdout:7/922: symlink d3/d1f/d49/l141 0 2024-09-16T23:49:13.396 INFO:tasks.workunit.client.0.smithi042.stdout:8/792: symlink da/d19/d1d/d30/d49/d58/de9/lf7 0 2024-09-16T23:49:13.397 INFO:tasks.workunit.client.0.smithi042.stdout:7/923: mkdir d3/d98/d142 0 2024-09-16T23:49:13.398 INFO:tasks.workunit.client.0.smithi042.stdout:2/853: link d0/d26/f2f d0/d5d/d10c/f120 0 2024-09-16T23:49:13.399 INFO:tasks.workunit.client.0.smithi042.stdout:8/793: truncate da/d19/d2c/d56/d97/dad/fb5 850276 0 2024-09-16T23:49:13.400 INFO:tasks.workunit.client.0.smithi042.stdout:1/983: creat d0/d9/d134/f15e x:0 0 0 2024-09-16T23:49:13.401 INFO:tasks.workunit.client.0.smithi042.stdout:7/924: stat d3/d1f/d49/d76/d101/d5b/db8 0 2024-09-16T23:49:13.402 INFO:tasks.workunit.client.0.smithi042.stdout:8/794: creat da/d19/d1d/d30/d3f/dc2/de3/ff8 x:0 0 0 2024-09-16T23:49:13.402 INFO:tasks.workunit.client.0.smithi042.stdout:8/795: stat da/d19/d80 0 2024-09-16T23:49:13.403 INFO:tasks.workunit.client.0.smithi042.stdout:1/984: creat d0/db2/f15f x:0 0 0 2024-09-16T23:49:13.405 INFO:tasks.workunit.client.0.smithi042.stdout:1/985: chown d0/d9/d90/d6b/d8e 68 1 2024-09-16T23:49:13.406 INFO:tasks.workunit.client.0.smithi042.stdout:2/854: creat d0/da/d1b/f121 x:0 0 0 2024-09-16T23:49:13.406 INFO:tasks.workunit.client.0.smithi042.stdout:8/796: dread da/d19/d2c/f4e [0,4194304] 0 2024-09-16T23:49:13.408 INFO:tasks.workunit.client.0.smithi042.stdout:1/986: write d0/d9/d90/d66/f7f [4082839,75710] 0 2024-09-16T23:49:13.408 INFO:tasks.workunit.client.0.smithi042.stdout:2/855: rmdir d0/da/d88/dae/dff 39 2024-09-16T23:49:13.409 INFO:tasks.workunit.client.0.smithi042.stdout:7/925: dwrite d3/fa [4194304,4194304] 0 2024-09-16T23:49:13.413 INFO:tasks.workunit.client.0.smithi042.stdout:2/856: unlink d0/da/d88/l8f 0 2024-09-16T23:49:13.414 INFO:tasks.workunit.client.0.smithi042.stdout:7/926: write d3/d1f/d49/d78/dbe/dc8/d125/f13b [222075,18314] 0 2024-09-16T23:49:13.415 INFO:tasks.workunit.client.0.smithi042.stdout:2/857: chown d0/da/d88/dae/fee 1 1 2024-09-16T23:49:13.416 INFO:tasks.workunit.client.0.smithi042.stdout:1/987: symlink d0/d9/d90/d6b/d88/d32/d2f/l160 0 2024-09-16T23:49:13.416 INFO:tasks.workunit.client.0.smithi042.stdout:1/988: readlink d0/d9/lb4 0 2024-09-16T23:49:13.420 INFO:tasks.workunit.client.0.smithi042.stdout:1/989: creat d0/d9/d90/d6b/d130/f161 x:0 0 0 2024-09-16T23:49:13.421 INFO:tasks.workunit.client.0.smithi042.stdout:1/990: fsync d0/d3b/d8d/f15a 0 2024-09-16T23:49:13.423 INFO:tasks.workunit.client.0.smithi042.stdout:2/858: dwrite d0/da/fcc [0,4194304] 0 2024-09-16T23:49:13.424 INFO:tasks.workunit.client.0.smithi042.stdout:1/991: fsync d0/d9/d90/d6b/d88/d32/f33 0 2024-09-16T23:49:13.430 INFO:tasks.workunit.client.0.smithi042.stdout:8/797: dwrite da/f71 [0,4194304] 0 2024-09-16T23:49:13.432 INFO:tasks.workunit.client.0.smithi042.stdout:8/798: write da/d4b/f87 [3996642,95101] 0 2024-09-16T23:49:13.436 INFO:tasks.workunit.client.0.smithi042.stdout:1/992: rename d0/d9/d4e/f13b to d0/f162 0 2024-09-16T23:49:13.437 INFO:tasks.workunit.client.0.smithi042.stdout:8/799: link da/d19/d2c/c3d da/d19/d1d/d30/d49/d58/cf9 0 2024-09-16T23:49:13.438 INFO:tasks.workunit.client.0.smithi042.stdout:8/800: chown da/d19/d1d/d30/d49/d58/f5b 317 1 2024-09-16T23:49:13.438 INFO:tasks.workunit.client.0.smithi042.stdout:2/859: rename d0/da/d1b/d3b/ccf to d0/d30/dcb/de5/c122 0 2024-09-16T23:49:13.441 INFO:tasks.workunit.client.0.smithi042.stdout:1/993: unlink d0/ddb/ddd/l12f 0 2024-09-16T23:49:13.446 INFO:tasks.workunit.client.0.smithi042.stdout:8/801: rename da/d19/d1d/d30/c55 to da/d19/d1d/d5a/d99/ddf/cfa 0 2024-09-16T23:49:13.447 INFO:tasks.workunit.client.0.smithi042.stdout:2/860: dwrite d0/da/d88/fe8 [0,4194304] 0 2024-09-16T23:49:13.447 INFO:tasks.workunit.client.0.smithi042.stdout:2/861: chown d0/d78 234152720 1 2024-09-16T23:49:13.448 INFO:tasks.workunit.client.0.smithi042.stdout:8/802: fdatasync da/f24 0 2024-09-16T23:49:13.449 INFO:tasks.workunit.client.0.smithi042.stdout:8/803: write da/f14 [3737251,30280] 0 2024-09-16T23:49:13.456 INFO:tasks.workunit.client.0.smithi042.stdout:1/994: dwrite d0/d17/f2d [0,4194304] 0 2024-09-16T23:49:13.457 INFO:tasks.workunit.client.0.smithi042.stdout:8/804: dread da/d19/f7a [0,4194304] 0 2024-09-16T23:49:13.461 INFO:tasks.workunit.client.0.smithi042.stdout:1/995: truncate d0/d17/d5e/d11f/f13d 950381 0 2024-09-16T23:49:13.461 INFO:tasks.workunit.client.0.smithi042.stdout:1/996: dread - d0/d9/d90/d6b/d88/def/f137 zero size 2024-09-16T23:49:13.461 INFO:tasks.workunit.client.0.smithi042.stdout:8/805: read - da/d19/d2c/f82 zero size 2024-09-16T23:49:13.470 INFO:tasks.workunit.client.0.smithi042.stdout:8/806: rmdir da/d19/d2c/db0/df6 0 2024-09-16T23:49:13.474 INFO:tasks.workunit.client.0.smithi042.stdout:8/807: unlink da/d19/d1d/d30/f45 0 2024-09-16T23:49:13.475 INFO:tasks.workunit.client.0.smithi042.stdout:1/997: dread d0/d9/d90/d6b/d88/f76 [0,4194304] 0 2024-09-16T23:49:13.476 INFO:tasks.workunit.client.0.smithi042.stdout:8/808: creat da/d4b/d74/ffb x:0 0 0 2024-09-16T23:49:13.478 INFO:tasks.workunit.client.0.smithi042.stdout:8/809: write da/d19/d1d/d30/d3f/dc2/fcd [1820287,56112] 0 2024-09-16T23:49:13.484 INFO:tasks.workunit.client.0.smithi042.stdout:8/810: dwrite da/d2f/ff4 [0,4194304] 0 2024-09-16T23:49:13.492 INFO:tasks.workunit.client.0.smithi042.stdout:1/998: dread d0/d9/d90/d6b/d88/f75 [0,4194304] 0 2024-09-16T23:49:13.494 INFO:tasks.workunit.client.0.smithi042.stdout:1/999: dread - d0/db2/f15f zero size 2024-09-16T23:49:13.494 INFO:tasks.workunit.client.0.smithi042.stdout:8/811: dwrite da/d19/d1d/d5a/faf [0,4194304] 0 2024-09-16T23:49:13.508 INFO:tasks.workunit.client.0.smithi042.stdout:8/812: dread da/d19/f1a [0,4194304] 0 2024-09-16T23:49:13.509 INFO:tasks.workunit.client.0.smithi042.stdout:8/813: chown da/d19/d1d/d5a/ldb 1 1 2024-09-16T23:49:13.513 INFO:tasks.workunit.client.0.smithi042.stdout:8/814: fdatasync da/d19/d1d/d30/d49/f69 0 2024-09-16T23:49:13.516 INFO:tasks.workunit.client.0.smithi042.stdout:8/815: mknod da/d19/d2c/d56/d97/dad/cfc 0 2024-09-16T23:49:13.517 INFO:tasks.workunit.client.0.smithi042.stdout:8/816: chown da/d19/d1d/d30/d49/d58/de9 3 1 2024-09-16T23:49:13.520 INFO:tasks.workunit.client.0.smithi042.stdout:8/817: symlink da/d4b/d74/dd6/lfd 0 2024-09-16T23:49:13.520 INFO:tasks.workunit.client.0.smithi042.stdout:8/818: truncate da/d4b/f87 5173314 0 2024-09-16T23:49:13.522 INFO:tasks.workunit.client.0.smithi042.stdout:8/819: mknod da/d2f/cfe 0 2024-09-16T23:49:13.527 INFO:tasks.workunit.client.0.smithi042.stdout:8/820: dwrite da/d19/f1e [0,4194304] 0 2024-09-16T23:49:13.529 INFO:tasks.workunit.client.0.smithi042.stdout:8/821: dread - da/d19/d1d/d30/d3f/dc2/de3/ff8 zero size 2024-09-16T23:49:13.530 INFO:tasks.workunit.client.0.smithi042.stdout:8/822: fdatasync da/d19/d1d/d30/d49/d58/d6a/f8b 0 2024-09-16T23:49:13.536 INFO:tasks.workunit.client.0.smithi042.stdout:8/823: rename da/d19/d1d/fb3 to da/d2f/fff 0 2024-09-16T23:49:13.539 INFO:tasks.workunit.client.0.smithi042.stdout:8/824: unlink da/d19/d2c/d56/d97/ldd 0 2024-09-16T23:49:13.546 INFO:tasks.workunit.client.0.smithi042.stdout:8/825: dread da/d19/d1d/f62 [0,4194304] 0 2024-09-16T23:49:13.549 INFO:tasks.workunit.client.0.smithi042.stdout:8/826: write da/d19/d1d/f63 [2225628,102905] 0 2024-09-16T23:49:13.551 INFO:tasks.workunit.client.0.smithi042.stdout:8/827: fsync da/d19/d1d/d30/d49/f7b 0 2024-09-16T23:49:13.553 INFO:tasks.workunit.client.0.smithi042.stdout:8/828: stat da/d19/d1d/d30/d49/cc3 0 2024-09-16T23:49:13.556 INFO:tasks.workunit.client.0.smithi042.stdout:8/829: dread da/d19/d2c/f67 [0,4194304] 0 2024-09-16T23:49:13.613 INFO:tasks.workunit.client.0.smithi042.stdout:8/830: truncate da/d19/f7a 692882 0 2024-09-16T23:49:13.614 INFO:tasks.workunit.client.0.smithi042.stdout:8/831: chown da/d19/d2c/d56/d97 59235 1 2024-09-16T23:49:13.615 INFO:tasks.workunit.client.0.smithi042.stdout:8/832: read da/d2f/ff4 [2420623,111479] 0 2024-09-16T23:49:13.616 INFO:tasks.workunit.client.0.smithi042.stdout:8/833: truncate da/d19/fcc 858108 0 2024-09-16T23:49:13.618 INFO:tasks.workunit.client.0.smithi042.stdout:8/834: mkdir da/d19/d1d/d30/d49/d58/d100 0 2024-09-16T23:49:13.622 INFO:tasks.workunit.client.0.smithi042.stdout:8/835: creat da/d19/d80/f101 x:0 0 0 2024-09-16T23:49:13.623 INFO:tasks.workunit.client.0.smithi042.stdout:8/836: dread - da/d19/d1d/d30/d49/d58/d6a/fe7 zero size 2024-09-16T23:49:13.625 INFO:tasks.workunit.client.0.smithi042.stdout:8/837: truncate da/d4b/f65 809302 0 2024-09-16T23:49:13.639 INFO:tasks.workunit.client.0.smithi042.stdout:8/838: dread da/d19/d1d/f89 [0,4194304] 0 2024-09-16T23:49:13.641 INFO:tasks.workunit.client.0.smithi042.stdout:8/839: symlink da/d19/d1d/d5a/d99/l102 0 2024-09-16T23:49:13.644 INFO:tasks.workunit.client.0.smithi042.stdout:8/840: mknod da/d19/d1d/d30/d49/d58/d6a/dec/c103 0 2024-09-16T23:49:13.658 INFO:tasks.workunit.client.0.smithi042.stdout:8/841: dread da/d19/d80/fa6 [0,4194304] 0 2024-09-16T23:49:13.658 INFO:tasks.workunit.client.0.smithi042.stdout:8/842: dread - da/d19/d2c/f82 zero size 2024-09-16T23:49:13.661 INFO:tasks.workunit.client.0.smithi042.stdout:8/843: read da/d4b/f65 [654332,34146] 0 2024-09-16T23:49:13.662 INFO:tasks.workunit.client.0.smithi042.stdout:8/844: chown da/d4b/f87 12 1 2024-09-16T23:49:13.666 INFO:tasks.workunit.client.0.smithi042.stdout:8/845: getdents da/d19/d2c/d56/dd4 0 2024-09-16T23:49:13.667 INFO:tasks.workunit.client.0.smithi042.stdout:8/846: write da/d19/d1d/d5a/f79 [3149136,4213] 0 2024-09-16T23:49:13.671 INFO:tasks.workunit.client.0.smithi042.stdout:8/847: dread da/f17 [0,4194304] 0 2024-09-16T23:49:13.676 INFO:tasks.workunit.client.0.smithi042.stdout:8/848: mkdir da/d4b/d74/dc7/d104 0 2024-09-16T23:49:13.678 INFO:tasks.workunit.client.0.smithi042.stdout:8/849: mknod da/d19/d1d/d5a/c105 0 2024-09-16T23:49:13.680 INFO:tasks.workunit.client.0.smithi042.stdout:8/850: dread da/d19/d1d/f62 [0,4194304] 0 2024-09-16T23:49:13.684 INFO:tasks.workunit.client.0.smithi042.stdout:8/851: rename da/f24 to da/d4b/d74/dc7/d104/f106 0 2024-09-16T23:49:13.698 INFO:tasks.workunit.client.0.smithi042.stdout:8/852: dwrite da/d2f/fbc [0,4194304] 0 2024-09-16T23:49:13.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/853: unlink da/d4b/d74/cde 0 2024-09-16T23:49:13.707 INFO:tasks.workunit.client.0.smithi042.stdout:8/854: creat da/d4b/d74/f107 x:0 0 0 2024-09-16T23:49:13.713 INFO:tasks.workunit.client.0.smithi042.stdout:8/855: creat da/d19/d1d/d30/d3f/d6d/f108 x:0 0 0 2024-09-16T23:49:13.716 INFO:tasks.workunit.client.0.smithi042.stdout:8/856: creat da/d4b/d74/f109 x:0 0 0 2024-09-16T23:49:13.716 INFO:tasks.workunit.client.0.smithi042.stdout:8/857: stat da/d19/d2c/f36 0 2024-09-16T23:49:13.718 INFO:tasks.workunit.client.0.smithi042.stdout:8/858: write da/d19/d1d/d30/d49/d58/d6a/f8b [549544,13200] 0 2024-09-16T23:49:13.730 INFO:tasks.workunit.client.0.smithi042.stdout:8/859: dread da/d19/f6f [0,4194304] 0 2024-09-16T23:49:13.737 INFO:tasks.workunit.client.0.smithi042.stdout:8/860: dwrite da/d4b/f65 [0,4194304] 0 2024-09-16T23:49:13.746 INFO:tasks.workunit.client.0.smithi042.stdout:8/861: write da/d19/d80/fa6 [3723503,67376] 0 2024-09-16T23:49:13.759 INFO:tasks.workunit.client.0.smithi042.stdout:8/862: write da/d19/f7a [698274,110974] 0 2024-09-16T23:49:13.763 INFO:tasks.workunit.client.0.smithi042.stdout:9/754: sync 2024-09-16T23:49:13.768 INFO:tasks.workunit.client.0.smithi042.stdout:9/755: write d4/d12/d16/f6d [647449,49816] 0 2024-09-16T23:49:13.769 INFO:tasks.workunit.client.0.smithi042.stdout:8/863: dwrite da/d19/d2c/d56/fae [0,4194304] 0 2024-09-16T23:49:13.776 INFO:tasks.workunit.client.0.smithi042.stdout:9/756: dwrite d4/d12/d16/da9/fe8 [0,4194304] 0 2024-09-16T23:49:13.781 INFO:tasks.workunit.client.0.smithi042.stdout:9/757: fdatasync d4/d12/d2d/d94/dc2/d3a/f5c 0 2024-09-16T23:49:13.782 INFO:tasks.workunit.client.0.smithi042.stdout:9/758: chown d4/d12/d77/fd5 1013539 1 2024-09-16T23:49:13.783 INFO:tasks.workunit.client.0.smithi042.stdout:8/864: dwrite da/d19/d80/f8f [0,4194304] 0 2024-09-16T23:49:13.785 INFO:tasks.workunit.client.0.smithi042.stdout:9/759: read d4/d12/d2d/d94/dc2/d59/f91 [818125,75480] 0 2024-09-16T23:49:13.793 INFO:tasks.workunit.client.0.smithi042.stdout:8/865: symlink da/d19/d1d/d5a/d99/l10a 0 2024-09-16T23:49:13.793 INFO:tasks.workunit.client.0.smithi042.stdout:9/760: rename d4/d12/d2d/d94/dc2/d28/la2 to d4/d12/leb 0 2024-09-16T23:49:13.794 INFO:tasks.workunit.client.0.smithi042.stdout:8/866: fsync da/d19/d1d/d30/d3f/d6d/fc1 0 2024-09-16T23:49:13.795 INFO:tasks.workunit.client.0.smithi042.stdout:9/761: truncate d4/d12/d2d/d94/dc2/fa4 792324 0 2024-09-16T23:49:13.796 INFO:tasks.workunit.client.0.smithi042.stdout:9/762: dread - d4/d12/d2d/dc9/fde zero size 2024-09-16T23:49:13.797 INFO:tasks.workunit.client.0.smithi042.stdout:9/763: stat d4/d10/d24/fe0 0 2024-09-16T23:49:13.802 INFO:tasks.workunit.client.0.smithi042.stdout:8/867: dwrite da/d19/d1d/f89 [4194304,4194304] 0 2024-09-16T23:49:13.803 INFO:tasks.workunit.client.0.smithi042.stdout:8/868: dread - da/d4b/d74/ffb zero size 2024-09-16T23:49:13.806 INFO:tasks.workunit.client.0.smithi042.stdout:8/869: write da/f47 [4344653,124107] 0 2024-09-16T23:49:13.809 INFO:tasks.workunit.client.0.smithi042.stdout:9/764: truncate d4/d12/d2d/d94/dc2/f70 1182045 0 2024-09-16T23:49:13.810 INFO:tasks.workunit.client.0.smithi042.stdout:8/870: mknod da/d19/d2c/c10b 0 2024-09-16T23:49:13.812 INFO:tasks.workunit.client.0.smithi042.stdout:9/765: creat d4/d10/d24/de1/fec x:0 0 0 2024-09-16T23:49:13.814 INFO:tasks.workunit.client.0.smithi042.stdout:8/871: dread da/d19/d1d/d30/f41 [0,4194304] 0 2024-09-16T23:49:13.818 INFO:tasks.workunit.client.0.smithi042.stdout:9/766: rename d4/d12/d2d/d94/dc2/d28/l42 to d4/d12/d2d/d94/dc2/d28/d5e/d83/db3/led 0 2024-09-16T23:49:13.819 INFO:tasks.workunit.client.0.smithi042.stdout:8/872: getdents da/d4b/d74/dd6 0 2024-09-16T23:49:13.819 INFO:tasks.workunit.client.0.smithi042.stdout:9/767: chown d4/d12/d2d/fcd 1397740618 1 2024-09-16T23:49:13.820 INFO:tasks.workunit.client.0.smithi042.stdout:9/768: write d4/d10/f6b [1774345,17948] 0 2024-09-16T23:49:13.821 INFO:tasks.workunit.client.0.smithi042.stdout:8/873: symlink da/d2f/l10c 0 2024-09-16T23:49:13.821 INFO:tasks.workunit.client.0.smithi042.stdout:9/769: chown d4/d12/d16/f6d 17 1 2024-09-16T23:49:13.826 INFO:tasks.workunit.client.0.smithi042.stdout:8/874: dread da/f96 [0,4194304] 0 2024-09-16T23:49:13.834 INFO:tasks.workunit.client.0.smithi042.stdout:9/770: dread d4/d10/faa [0,4194304] 0 2024-09-16T23:49:13.836 INFO:tasks.workunit.client.0.smithi042.stdout:9/771: creat d4/d12/d77/fee x:0 0 0 2024-09-16T23:49:13.837 INFO:tasks.workunit.client.0.smithi042.stdout:7/927: sync 2024-09-16T23:49:13.837 INFO:tasks.workunit.client.0.smithi042.stdout:2/862: sync 2024-09-16T23:49:13.838 INFO:tasks.workunit.client.0.smithi042.stdout:9/772: chown d4/d12/d2d/d94/dc2/d28/c6c 8033 1 2024-09-16T23:49:13.841 INFO:tasks.workunit.client.0.smithi042.stdout:8/875: dwrite da/d4b/d74/dc7/d104/f106 [4194304,4194304] 0 2024-09-16T23:49:13.845 INFO:tasks.workunit.client.0.smithi042.stdout:7/928: creat d3/d98/da0/f143 x:0 0 0 2024-09-16T23:49:13.846 INFO:tasks.workunit.client.0.smithi042.stdout:8/876: symlink da/d19/d2c/d56/d97/dab/l10d 0 2024-09-16T23:49:13.847 INFO:tasks.workunit.client.0.smithi042.stdout:8/877: fdatasync da/d19/d2c/dc8/fe5 0 2024-09-16T23:49:13.848 INFO:tasks.workunit.client.0.smithi042.stdout:2/863: write d0/f1a [3200183,125109] 0 2024-09-16T23:49:13.851 INFO:tasks.workunit.client.0.smithi042.stdout:7/929: mkdir d3/d98/da0/d24/d28/d144 0 2024-09-16T23:49:13.852 INFO:tasks.workunit.client.0.smithi042.stdout:2/864: creat d0/da/d17/d3e/d86/d96/f123 x:0 0 0 2024-09-16T23:49:13.852 INFO:tasks.workunit.client.0.smithi042.stdout:7/930: chown d3/d1f/cde 12090197 1 2024-09-16T23:49:13.854 INFO:tasks.workunit.client.0.smithi042.stdout:8/878: mknod da/c10e 0 2024-09-16T23:49:13.857 INFO:tasks.workunit.client.0.smithi042.stdout:7/931: symlink d3/d1f/d49/d76/d101/d64/d92/dfb/d126/l145 0 2024-09-16T23:49:13.859 INFO:tasks.workunit.client.0.smithi042.stdout:9/773: dread d4/d10/f1f [0,4194304] 0 2024-09-16T23:49:13.859 INFO:tasks.workunit.client.0.smithi042.stdout:9/774: readlink d4/d12/d16/l3e 0 2024-09-16T23:49:13.860 INFO:tasks.workunit.client.0.smithi042.stdout:7/932: rmdir d3/d1f/d49/d76/d101/d64/d92 39 2024-09-16T23:49:13.862 INFO:tasks.workunit.client.0.smithi042.stdout:7/933: dread - d3/d1f/d49/d76/d19/ddc/d13e/f140 zero size 2024-09-16T23:49:13.862 INFO:tasks.workunit.client.0.smithi042.stdout:7/934: stat d3/d1f/d4f/l99 0 2024-09-16T23:49:13.863 INFO:tasks.workunit.client.0.smithi042.stdout:9/775: mkdir d4/d12/d60/def 0 2024-09-16T23:49:13.863 INFO:tasks.workunit.client.0.smithi042.stdout:8/879: creat da/d19/d2c/f10f x:0 0 0 2024-09-16T23:49:13.865 INFO:tasks.workunit.client.0.smithi042.stdout:7/935: mkdir d3/d98/da0/d24/d11a/d146 0 2024-09-16T23:49:13.867 INFO:tasks.workunit.client.0.smithi042.stdout:8/880: creat da/d19/d1d/d30/d3f/d6d/f110 x:0 0 0 2024-09-16T23:49:13.869 INFO:tasks.workunit.client.0.smithi042.stdout:8/881: read - da/d19/d1d/d30/d3f/d6d/fc1 zero size 2024-09-16T23:49:13.870 INFO:tasks.workunit.client.0.smithi042.stdout:7/936: dread d3/d1f/d49/d78/dbe/dc8/d125/d13a/db5/fd9 [0,4194304] 0 2024-09-16T23:49:13.870 INFO:tasks.workunit.client.0.smithi042.stdout:7/937: read d3/fa [7815685,47605] 0 2024-09-16T23:49:13.873 INFO:tasks.workunit.client.0.smithi042.stdout:7/938: creat d3/d1f/d49/d76/d19/ddc/d13e/f147 x:0 0 0 2024-09-16T23:49:13.874 INFO:tasks.workunit.client.0.smithi042.stdout:8/882: creat da/d19/d2c/dc8/f111 x:0 0 0 2024-09-16T23:49:13.882 INFO:tasks.workunit.client.0.smithi042.stdout:9/776: dwrite d4/d12/d2d/d94/dc2/d3a/d50/fd8 [0,4194304] 0 2024-09-16T23:49:13.884 INFO:tasks.workunit.client.0.smithi042.stdout:8/883: rename da/d2f/l10c to da/d19/d80/l112 0 2024-09-16T23:49:13.885 INFO:tasks.workunit.client.0.smithi042.stdout:9/777: dread d4/d12/d2d/d94/dc2/d3a/fd0 [0,4194304] 0 2024-09-16T23:49:13.889 INFO:tasks.workunit.client.0.smithi042.stdout:9/778: rmdir d4/d12/d2d/d94/dc2/de2 0 2024-09-16T23:49:13.891 INFO:tasks.workunit.client.0.smithi042.stdout:9/779: symlink d4/lf0 0 2024-09-16T23:49:13.892 INFO:tasks.workunit.client.0.smithi042.stdout:8/884: link da/d19/d2c/dc8/fe5 da/d19/d1d/d30/d49/d58/f113 0 2024-09-16T23:49:13.893 INFO:tasks.workunit.client.0.smithi042.stdout:9/780: truncate d4/d10/f6b 4517995 0 2024-09-16T23:49:13.896 INFO:tasks.workunit.client.0.smithi042.stdout:9/781: rmdir d4/d12/d60/def 0 2024-09-16T23:49:13.899 INFO:tasks.workunit.client.0.smithi042.stdout:9/782: mkdir d4/d12/d16/df1 0 2024-09-16T23:49:13.901 INFO:tasks.workunit.client.0.smithi042.stdout:8/885: dwrite da/d19/d1d/d30/d3f/d6d/f110 [0,4194304] 0 2024-09-16T23:49:13.905 INFO:tasks.workunit.client.0.smithi042.stdout:9/783: rmdir d4/d12/d2d/d94/dc2 39 2024-09-16T23:49:13.914 INFO:tasks.workunit.client.0.smithi042.stdout:9/784: dwrite d4/d12/d2d/d94/dc2/d59/f96 [4194304,4194304] 0 2024-09-16T23:49:13.921 INFO:tasks.workunit.client.0.smithi042.stdout:9/785: creat d4/d12/d2d/dc9/dce/d8e/da1/ff2 x:0 0 0 2024-09-16T23:49:13.923 INFO:tasks.workunit.client.0.smithi042.stdout:9/786: chown d4/d12/d2d/d94/dc2/d59 323979 1 2024-09-16T23:49:13.930 INFO:tasks.workunit.client.0.smithi042.stdout:9/787: dwrite d4/d12/d2d/d94/dc2/d3a/f48 [0,4194304] 0 2024-09-16T23:49:13.934 INFO:tasks.workunit.client.0.smithi042.stdout:9/788: truncate d4/d12/d2d/d94/dc2/d3a/f5b 702100 0 2024-09-16T23:49:13.940 INFO:tasks.workunit.client.0.smithi042.stdout:9/789: creat d4/d12/d2d/d94/dc2/d41/d92/da7/ff3 x:0 0 0 2024-09-16T23:49:13.942 INFO:tasks.workunit.client.0.smithi042.stdout:9/790: write d4/d12/d2d/d51/fbe [2474645,33261] 0 2024-09-16T23:49:13.945 INFO:tasks.workunit.client.0.smithi042.stdout:9/791: mkdir d4/d10/d4b/df4 0 2024-09-16T23:49:14.198 INFO:tasks.workunit.client.0.smithi042.stdout:2/865: sync 2024-09-16T23:49:14.202 INFO:tasks.workunit.client.0.smithi042.stdout:2/866: dread d0/da/d17/d3e/d86/d96/f9b [0,4194304] 0 2024-09-16T23:49:14.203 INFO:tasks.workunit.client.0.smithi042.stdout:2/867: stat d0/da/d17/d3e/lb4 0 2024-09-16T23:49:14.204 INFO:tasks.workunit.client.0.smithi042.stdout:2/868: write d0/da/d17/d3e/fef [867258,7117] 0 2024-09-16T23:49:14.206 INFO:tasks.workunit.client.0.smithi042.stdout:2/869: mknod d0/d98/c124 0 2024-09-16T23:49:14.220 INFO:tasks.workunit.client.0.smithi042.stdout:2/870: dread d0/da/d1b/d48/f58 [0,4194304] 0 2024-09-16T23:49:14.221 INFO:tasks.workunit.client.0.smithi042.stdout:2/871: rmdir d0/da/d1b/d3b 39 2024-09-16T23:49:14.222 INFO:tasks.workunit.client.0.smithi042.stdout:2/872: fdatasync d0/da/f2c 0 2024-09-16T23:49:14.222 INFO:tasks.workunit.client.0.smithi042.stdout:2/873: readlink d0/l24 0 2024-09-16T23:49:14.224 INFO:tasks.workunit.client.0.smithi042.stdout:2/874: chown d0/d30/d43/f89 0 1 2024-09-16T23:49:14.237 INFO:tasks.workunit.client.0.smithi042.stdout:2/875: dread d0/d26/f2f [0,4194304] 0 2024-09-16T23:49:14.238 INFO:tasks.workunit.client.0.smithi042.stdout:2/876: stat d0/c8 0 2024-09-16T23:49:14.240 INFO:tasks.workunit.client.0.smithi042.stdout:2/877: truncate d0/da/d1b/d3b/d44/dec/f101 686039 0 2024-09-16T23:49:14.244 INFO:tasks.workunit.client.0.smithi042.stdout:2/878: link d0/d98/lb6 d0/da/d17/d5c/l125 0 2024-09-16T23:49:14.246 INFO:tasks.workunit.client.0.smithi042.stdout:2/879: stat d0/da/d17/d3e/f76 0 2024-09-16T23:49:14.252 INFO:tasks.workunit.client.0.smithi042.stdout:2/880: dwrite d0/da/d17/d3e/f116 [0,4194304] 0 2024-09-16T23:49:14.256 INFO:tasks.workunit.client.0.smithi042.stdout:2/881: dread - d0/d30/dcb/de5/fc4 zero size 2024-09-16T23:49:14.256 INFO:tasks.workunit.client.0.smithi042.stdout:2/882: write d0/da/d1b/d3b/d44/d54/f112 [107308,70703] 0 2024-09-16T23:49:14.264 INFO:tasks.workunit.client.0.smithi042.stdout:2/883: dread d0/da/d17/f59 [0,4194304] 0 2024-09-16T23:49:14.265 INFO:tasks.workunit.client.0.smithi042.stdout:2/884: fdatasync d0/f22 0 2024-09-16T23:49:14.273 INFO:tasks.workunit.client.0.smithi042.stdout:2/885: write d0/da/d17/f2d [2682453,56673] 0 2024-09-16T23:49:14.276 INFO:tasks.workunit.client.0.smithi042.stdout:2/886: fsync d0/da/d17/d3e/d86/f11e 0 2024-09-16T23:49:14.281 INFO:tasks.workunit.client.0.smithi042.stdout:2/887: link d0/da/d1b/f121 d0/da/d1b/f126 0 2024-09-16T23:49:14.281 INFO:tasks.workunit.client.0.smithi042.stdout:8/886: sync 2024-09-16T23:49:14.281 INFO:tasks.workunit.client.0.smithi042.stdout:7/939: sync 2024-09-16T23:49:14.281 INFO:tasks.workunit.client.0.smithi042.stdout:9/792: sync 2024-09-16T23:49:14.283 INFO:tasks.workunit.client.0.smithi042.stdout:9/793: read d4/d10/f57 [235102,4813] 0 2024-09-16T23:49:14.285 INFO:tasks.workunit.client.0.smithi042.stdout:9/794: write d4/d12/d2d/fcd [387083,17208] 0 2024-09-16T23:49:14.286 INFO:tasks.workunit.client.0.smithi042.stdout:2/888: write d0/d5d/f82 [540493,86648] 0 2024-09-16T23:49:14.288 INFO:tasks.workunit.client.0.smithi042.stdout:7/940: mknod d3/d1f/d49/d76/d19/c148 0 2024-09-16T23:49:14.291 INFO:tasks.workunit.client.0.smithi042.stdout:8/887: write da/d19/d1d/f50 [327150,70224] 0 2024-09-16T23:49:14.292 INFO:tasks.workunit.client.0.smithi042.stdout:7/941: creat d3/d3f/d114/f149 x:0 0 0 2024-09-16T23:49:14.292 INFO:tasks.workunit.client.0.smithi042.stdout:8/888: fsync da/d2f/fbc 0 2024-09-16T23:49:14.297 INFO:tasks.workunit.client.0.smithi042.stdout:9/795: dwrite d4/d10/faa [0,4194304] 0 2024-09-16T23:49:14.299 INFO:tasks.workunit.client.0.smithi042.stdout:8/889: mkdir da/d19/d1d/d5a/d114 0 2024-09-16T23:49:14.301 INFO:tasks.workunit.client.0.smithi042.stdout:9/796: fdatasync d4/d12/d2d/d94/dc2/d59/f96 0 2024-09-16T23:49:14.301 INFO:tasks.workunit.client.0.smithi042.stdout:2/889: dwrite d0/da/d1b/d48/db7/fd2 [0,4194304] 0 2024-09-16T23:49:14.301 INFO:tasks.workunit.client.0.smithi042.stdout:7/942: write d3/d1f/d49/d76/d19/d63/f70 [2043841,84969] 0 2024-09-16T23:49:14.303 INFO:tasks.workunit.client.0.smithi042.stdout:2/890: chown d0/d30/dcb/de5/fc0 66809358 1 2024-09-16T23:49:14.304 INFO:tasks.workunit.client.0.smithi042.stdout:8/890: mkdir da/d19/d2c/dc8/d115 0 2024-09-16T23:49:14.306 INFO:tasks.workunit.client.0.smithi042.stdout:7/943: mknod d3/d1f/d49/d76/d19/d63/c14a 0 2024-09-16T23:49:14.307 INFO:tasks.workunit.client.0.smithi042.stdout:2/891: symlink d0/da/d17/d5c/df6/l127 0 2024-09-16T23:49:14.308 INFO:tasks.workunit.client.0.smithi042.stdout:2/892: chown d0/da/d17/f42 5866972 1 2024-09-16T23:49:14.309 INFO:tasks.workunit.client.0.smithi042.stdout:2/893: fdatasync d0/d30/d43/f62 0 2024-09-16T23:49:14.310 INFO:tasks.workunit.client.0.smithi042.stdout:2/894: chown d0/da/d88/cab 1 1 2024-09-16T23:49:14.311 INFO:tasks.workunit.client.0.smithi042.stdout:8/891: rename da/d19/d1d/d5a/d99 to da/d19/d2c/d56/d116 0 2024-09-16T23:49:14.313 INFO:tasks.workunit.client.0.smithi042.stdout:7/944: getdents d3/d1f/d49/d76/d11c 0 2024-09-16T23:49:14.313 INFO:tasks.workunit.client.0.smithi042.stdout:9/797: link d4/d12/d2d/d94/dc2/d41/d92/da7/f89 d4/d12/d2d/dc9/dce/d8e/ff5 0 2024-09-16T23:49:14.314 INFO:tasks.workunit.client.0.smithi042.stdout:8/892: write da/f83 [3002656,117841] 0 2024-09-16T23:49:14.316 INFO:tasks.workunit.client.0.smithi042.stdout:8/893: read da/d2f/f38 [3108438,83670] 0 2024-09-16T23:49:14.320 INFO:tasks.workunit.client.0.smithi042.stdout:8/894: mknod da/d2f/c117 0 2024-09-16T23:49:14.320 INFO:tasks.workunit.client.0.smithi042.stdout:2/895: write d0/da/d1b/d48/db7/fd5 [709637,13006] 0 2024-09-16T23:49:14.327 INFO:tasks.workunit.client.0.smithi042.stdout:2/896: dread d0/d8d/da0/fd6 [0,4194304] 0 2024-09-16T23:49:14.329 INFO:tasks.workunit.client.0.smithi042.stdout:8/895: rmdir da/d4b/d74/dc7 39 2024-09-16T23:49:14.331 INFO:tasks.workunit.client.0.smithi042.stdout:7/945: dwrite d3/d3f/fe5 [0,4194304] 0 2024-09-16T23:49:14.331 INFO:tasks.workunit.client.0.smithi042.stdout:9/798: dwrite d4/d12/d2d/d94/dc2/d41/d92/da7/f8b [0,4194304] 0 2024-09-16T23:49:14.334 INFO:tasks.workunit.client.0.smithi042.stdout:8/896: rmdir da/d4b/d74/dd6 39 2024-09-16T23:49:14.335 INFO:tasks.workunit.client.0.smithi042.stdout:2/897: symlink d0/da/d1b/d3b/d44/d54/df3/l128 0 2024-09-16T23:49:14.337 INFO:tasks.workunit.client.0.smithi042.stdout:9/799: unlink d4/d12/d77/dc4/fd3 0 2024-09-16T23:49:14.339 INFO:tasks.workunit.client.0.smithi042.stdout:7/946: creat d3/d1f/d49/d78/dbe/dc8/d125/d13a/dcf/f14b x:0 0 0 2024-09-16T23:49:14.343 INFO:tasks.workunit.client.0.smithi042.stdout:8/897: dwrite da/d19/d2c/d56/d97/dad/fbf [4194304,4194304] 0 2024-09-16T23:49:14.344 INFO:tasks.workunit.client.0.smithi042.stdout:9/800: read - d4/d12/d2d/d94/dc2/d41/d92/da7/f89 zero size 2024-09-16T23:49:14.346 INFO:tasks.workunit.client.0.smithi042.stdout:2/898: mkdir d0/da/d88/d129 0 2024-09-16T23:49:14.346 INFO:tasks.workunit.client.0.smithi042.stdout:9/801: write d4/d12/d2d/d94/dc2/d59/f96 [9363845,102127] 0 2024-09-16T23:49:14.348 INFO:tasks.workunit.client.0.smithi042.stdout:7/947: creat d3/d98/da0/f14c x:0 0 0 2024-09-16T23:49:14.352 INFO:tasks.workunit.client.0.smithi042.stdout:7/948: mknod d3/d1f/d49/d76/d19/d63/c14d 0 2024-09-16T23:49:14.353 INFO:tasks.workunit.client.0.smithi042.stdout:8/898: rename da/d19/d1d/d30/d3f/d6d/f7e to da/d19/d1d/d30/d49/d58/f118 0 2024-09-16T23:49:14.354 INFO:tasks.workunit.client.0.smithi042.stdout:2/899: mkdir d0/d30/dcb/d12a 0 2024-09-16T23:49:14.358 INFO:tasks.workunit.client.0.smithi042.stdout:7/949: rename d3/f84 to d3/d1f/d49/d78/db9/f14e 0 2024-09-16T23:49:14.358 INFO:tasks.workunit.client.0.smithi042.stdout:9/802: link d4/d12/l98 d4/d12/d2d/d94/dc2/d3a/d50/lf6 0 2024-09-16T23:49:14.360 INFO:tasks.workunit.client.0.smithi042.stdout:2/900: mkdir d0/d30/d43/d77/d12b 0 2024-09-16T23:49:14.362 INFO:tasks.workunit.client.0.smithi042.stdout:2/901: fdatasync d0/da/d17/d3e/f116 0 2024-09-16T23:49:14.363 INFO:tasks.workunit.client.0.smithi042.stdout:7/950: symlink d3/d1f/d49/d76/l14f 0 2024-09-16T23:49:14.365 INFO:tasks.workunit.client.0.smithi042.stdout:9/803: link d4/d12/d2d/d94/dc2/d41/fc3 d4/d12/d77/dc4/ff7 0 2024-09-16T23:49:14.366 INFO:tasks.workunit.client.0.smithi042.stdout:9/804: readlink d4/d12/d2d/d94/dc2/d3a/d50/l55 0 2024-09-16T23:49:14.367 INFO:tasks.workunit.client.0.smithi042.stdout:9/805: stat d4/lf0 0 2024-09-16T23:49:14.368 INFO:tasks.workunit.client.0.smithi042.stdout:8/899: write da/d19/d1d/d30/d3f/d6d/f76 [1172263,12701] 0 2024-09-16T23:49:14.372 INFO:tasks.workunit.client.0.smithi042.stdout:7/951: creat d3/d1f/d49/d76/d101/d64/d92/dfb/d126/f150 x:0 0 0 2024-09-16T23:49:14.372 INFO:tasks.workunit.client.0.smithi042.stdout:8/900: chown da/d4b/d74/c86 42 1 2024-09-16T23:49:14.374 INFO:tasks.workunit.client.0.smithi042.stdout:8/901: write da/d19/f7a [83553,63284] 0 2024-09-16T23:49:14.375 INFO:tasks.workunit.client.0.smithi042.stdout:9/806: dwrite d4/d12/d2d/d94/dc2/d3a/d50/f99 [4194304,4194304] 0 2024-09-16T23:49:14.377 INFO:tasks.workunit.client.0.smithi042.stdout:7/952: creat d3/d1f/d49/d76/d19/ddc/d13e/f151 x:0 0 0 2024-09-16T23:49:14.378 INFO:tasks.workunit.client.0.smithi042.stdout:2/902: write d0/f3a [2318762,99984] 0 2024-09-16T23:49:14.383 INFO:tasks.workunit.client.0.smithi042.stdout:7/953: mkdir d3/d98/da0/d24/dc3/d152 0 2024-09-16T23:49:14.385 INFO:tasks.workunit.client.0.smithi042.stdout:8/902: dwrite da/d19/d2c/d56/fae [0,4194304] 0 2024-09-16T23:49:14.385 INFO:tasks.workunit.client.0.smithi042.stdout:7/954: readlink d3/l3a 0 2024-09-16T23:49:14.386 INFO:tasks.workunit.client.0.smithi042.stdout:8/903: stat da/d19/d2c/dc8/dd7 0 2024-09-16T23:49:14.388 INFO:tasks.workunit.client.0.smithi042.stdout:9/807: rename d4/d12/d2d/d94/dc2/d41/d92/da7/lbd to d4/d12/d2d/d94/lf8 0 2024-09-16T23:49:14.394 INFO:tasks.workunit.client.0.smithi042.stdout:8/904: creat da/d19/d1d/d30/d49/d58/f119 x:0 0 0 2024-09-16T23:49:14.405 INFO:tasks.workunit.client.0.smithi042.stdout:2/903: dread d0/da/f1f [0,4194304] 0 2024-09-16T23:49:14.405 INFO:tasks.workunit.client.0.smithi042.stdout:2/904: chown d0/d26/ldb 38 1 2024-09-16T23:49:14.408 INFO:tasks.workunit.client.0.smithi042.stdout:8/905: write da/d19/d2c/f82 [1019009,109026] 0 2024-09-16T23:49:14.413 INFO:tasks.workunit.client.0.smithi042.stdout:8/906: creat da/d19/f11a x:0 0 0 2024-09-16T23:49:14.416 INFO:tasks.workunit.client.0.smithi042.stdout:8/907: mknod da/d19/d80/c11b 0 2024-09-16T23:49:14.430 INFO:tasks.workunit.client.0.smithi042.stdout:8/908: write da/d19/d2c/f66 [1080743,10731] 0 2024-09-16T23:49:14.433 INFO:tasks.workunit.client.0.smithi042.stdout:8/909: read - da/d4b/d74/f109 zero size 2024-09-16T23:49:14.433 INFO:tasks.workunit.client.0.smithi042.stdout:7/955: mkdir d3/d1f/d49/d76/d101/d64/d92/d153 0 2024-09-16T23:49:14.434 INFO:tasks.workunit.client.0.smithi042.stdout:9/808: symlink d4/d12/d2d/d94/dc2/d28/d5e/d83/db3/lf9 0 2024-09-16T23:49:14.437 INFO:tasks.workunit.client.0.smithi042.stdout:7/956: write d3/d98/da0/d24/d11a/f11f [293480,94100] 0 2024-09-16T23:49:14.440 INFO:tasks.workunit.client.0.smithi042.stdout:2/905: creat d0/da/d17/d5c/f12c x:0 0 0 2024-09-16T23:49:14.440 INFO:tasks.workunit.client.0.smithi042.stdout:9/809: truncate d4/d12/d2d/d94/dc2/fa4 902944 0 2024-09-16T23:49:14.441 INFO:tasks.workunit.client.0.smithi042.stdout:7/957: chown d3/d1f/d49/d76/d101/d5b/ffc 858741539 1 2024-09-16T23:49:14.442 INFO:tasks.workunit.client.0.smithi042.stdout:8/910: dwrite da/d19/d1d/d30/d49/db2/fd8 [0,4194304] 0 2024-09-16T23:49:14.443 INFO:tasks.workunit.client.0.smithi042.stdout:7/958: chown d3/d1f/ff8 11953684 1 2024-09-16T23:49:14.444 INFO:tasks.workunit.client.0.smithi042.stdout:9/810: write d4/d12/d2d/d94/dc2/fa4 [1855594,114514] 0 2024-09-16T23:49:14.446 INFO:tasks.workunit.client.0.smithi042.stdout:9/811: fsync d4/d10/f15 0 2024-09-16T23:49:14.447 INFO:tasks.workunit.client.0.smithi042.stdout:8/911: stat da/d19/d2c/d56/d116/ddf/cfa 0 2024-09-16T23:49:14.454 INFO:tasks.workunit.client.0.smithi042.stdout:7/959: dwrite d3/d1f/d49/d78/dbe/dc8/d125/f13b [0,4194304] 0 2024-09-16T23:49:14.455 INFO:tasks.workunit.client.0.smithi042.stdout:8/912: write da/d2f/ff4 [2335654,8402] 0 2024-09-16T23:49:14.455 INFO:tasks.workunit.client.0.smithi042.stdout:7/960: readlink d3/l3a 0 2024-09-16T23:49:14.460 INFO:tasks.workunit.client.0.smithi042.stdout:8/913: write da/d19/d1d/d30/d49/d58/d6a/f8b [2428160,60664] 0 2024-09-16T23:49:14.463 INFO:tasks.workunit.client.0.smithi042.stdout:9/812: creat d4/d12/d2d/d94/dc2/d41/d92/ffa x:0 0 0 2024-09-16T23:49:14.469 INFO:tasks.workunit.client.0.smithi042.stdout:7/961: creat d3/d1f/d49/d76/d101/da5/de3/d12b/d13c/f154 x:0 0 0 2024-09-16T23:49:14.471 INFO:tasks.workunit.client.0.smithi042.stdout:9/813: creat d4/d12/d16/d68/ffb x:0 0 0 2024-09-16T23:49:14.472 INFO:tasks.workunit.client.0.smithi042.stdout:9/814: chown d4/d12/d2d/fdd 19550733 1 2024-09-16T23:49:14.472 INFO:tasks.workunit.client.0.smithi042.stdout:7/962: creat d3/d1f/d49/d76/d101/d64/d92/dfb/d126/f155 x:0 0 0 2024-09-16T23:49:14.475 INFO:tasks.workunit.client.0.smithi042.stdout:7/963: fsync d3/d1f/d49/d76/d101/fd2 0 2024-09-16T23:49:14.480 INFO:tasks.workunit.client.0.smithi042.stdout:7/964: unlink d3/d3f/d114/l135 0 2024-09-16T23:49:14.480 INFO:tasks.workunit.client.0.smithi042.stdout:7/965: write d3/d1f/f3d [6375018,19337] 0 2024-09-16T23:49:14.482 INFO:tasks.workunit.client.0.smithi042.stdout:7/966: dread - d3/d1f/d49/d76/d19/ddc/d13e/f140 zero size 2024-09-16T23:49:14.484 INFO:tasks.workunit.client.0.smithi042.stdout:7/967: truncate d3/d1f/d49/d78/db9/f14e 4800493 0 2024-09-16T23:49:14.486 INFO:tasks.workunit.client.0.smithi042.stdout:7/968: write d3/d98/da0/d24/d28/d9b/f108 [633705,81676] 0 2024-09-16T23:49:14.492 INFO:tasks.workunit.client.0.smithi042.stdout:7/969: link d3/d1f/d49/d76/d101/da5/cef d3/d1f/d49/d76/d101/d64/d92/c156 0 2024-09-16T23:49:14.495 INFO:tasks.workunit.client.0.smithi042.stdout:7/970: dread d3/f9c [0,4194304] 0 2024-09-16T23:49:14.498 INFO:tasks.workunit.client.0.smithi042.stdout:7/971: mknod d3/d1f/d49/d76/d101/d64/d92/dfb/c157 0 2024-09-16T23:49:14.500 INFO:tasks.workunit.client.0.smithi042.stdout:9/815: creat d4/ffc x:0 0 0 2024-09-16T23:49:14.502 INFO:tasks.workunit.client.0.smithi042.stdout:7/972: rename d3/d1f/d49/d78/f102 to d3/d1f/d49/d78/dbe/dc8/d125/d13a/db5/f158 0 2024-09-16T23:49:14.503 INFO:tasks.workunit.client.0.smithi042.stdout:9/816: mkdir d4/d10/d4b/dfd 0 2024-09-16T23:49:14.506 INFO:tasks.workunit.client.0.smithi042.stdout:7/973: dread d3/d98/da0/d24/d28/d9b/f108 [0,4194304] 0 2024-09-16T23:49:14.506 INFO:tasks.workunit.client.0.smithi042.stdout:2/906: write d0/da/d1b/f29 [7840215,67655] 0 2024-09-16T23:49:14.510 INFO:tasks.workunit.client.0.smithi042.stdout:9/817: getdents d4/d12/d16/da9 0 2024-09-16T23:49:14.512 INFO:tasks.workunit.client.0.smithi042.stdout:2/907: rename d0/d26/f7c to d0/d5d/f12d 0 2024-09-16T23:49:14.513 INFO:tasks.workunit.client.0.smithi042.stdout:7/974: dwrite d3/d98/f8c [0,4194304] 0 2024-09-16T23:49:14.514 INFO:tasks.workunit.client.0.smithi042.stdout:9/818: mkdir d4/d12/d16/dfe 0 2024-09-16T23:49:14.515 INFO:tasks.workunit.client.0.smithi042.stdout:2/908: write d0/d30/d43/f63 [4376496,21843] 0 2024-09-16T23:49:14.516 INFO:tasks.workunit.client.0.smithi042.stdout:2/909: fsync d0/d30/dcb/de5/fb5 0 2024-09-16T23:49:14.520 INFO:tasks.workunit.client.0.smithi042.stdout:2/910: rename d0/d78/f105 to d0/d30/d43/d77/d12b/f12e 0 2024-09-16T23:49:14.524 INFO:tasks.workunit.client.0.smithi042.stdout:2/911: dread d0/d5d/f82 [0,4194304] 0 2024-09-16T23:49:14.527 INFO:tasks.workunit.client.0.smithi042.stdout:2/912: unlink d0/da/d17/d3e/d86/lad 0 2024-09-16T23:49:14.531 INFO:tasks.workunit.client.0.smithi042.stdout:2/913: dread d0/da/d1b/d3b/d44/dec/f101 [0,4194304] 0 2024-09-16T23:49:14.533 INFO:tasks.workunit.client.0.smithi042.stdout:2/914: creat d0/d78/d8b/f12f x:0 0 0 2024-09-16T23:49:14.534 INFO:tasks.workunit.client.0.smithi042.stdout:2/915: chown d0/da/d1b/d3b/d44/df2 189 1 2024-09-16T23:49:14.535 INFO:tasks.workunit.client.0.smithi042.stdout:2/916: fdatasync d0/d30/d43/f62 0 2024-09-16T23:49:14.535 INFO:tasks.workunit.client.0.smithi042.stdout:7/975: write d3/d1f/d49/d78/dbe/fc7 [1840177,72501] 0 2024-09-16T23:49:14.536 INFO:tasks.workunit.client.0.smithi042.stdout:7/976: chown d3/d1f/d49/d76/d101/da5 31087167 1 2024-09-16T23:49:14.537 INFO:tasks.workunit.client.0.smithi042.stdout:7/977: chown d3/d3f/f5e 58127946 1 2024-09-16T23:49:14.538 INFO:tasks.workunit.client.0.smithi042.stdout:7/978: chown d3/d1f/d49/d76/d101/d5b 1 1 2024-09-16T23:49:14.539 INFO:tasks.workunit.client.0.smithi042.stdout:7/979: dread - d3/d98/da0/d24/dc3/fdf zero size 2024-09-16T23:49:14.539 INFO:tasks.workunit.client.0.smithi042.stdout:7/980: chown d3/d1f/d4f/l88 11626444 1 2024-09-16T23:49:14.550 INFO:tasks.workunit.client.0.smithi042.stdout:7/981: dwrite d3/d98/da0/d41/f45 [0,4194304] 0 2024-09-16T23:49:14.555 INFO:tasks.workunit.client.0.smithi042.stdout:7/982: dread d3/f11b [0,4194304] 0 2024-09-16T23:49:14.561 INFO:tasks.workunit.client.0.smithi042.stdout:7/983: truncate d3/f30 2740666 0 2024-09-16T23:49:14.562 INFO:tasks.workunit.client.0.smithi042.stdout:7/984: creat d3/d1f/d49/f159 x:0 0 0 2024-09-16T23:49:14.563 INFO:tasks.workunit.client.0.smithi042.stdout:7/985: chown d3/d98/da0/lf6 40324203 1 2024-09-16T23:49:14.566 INFO:tasks.workunit.client.0.smithi042.stdout:7/986: creat d3/d1f/d49/d76/d101/d64/d92/f15a x:0 0 0 2024-09-16T23:49:14.567 INFO:tasks.workunit.client.0.smithi042.stdout:7/987: dread - d3/d1f/d49/d76/d19/ddc/d13e/f140 zero size 2024-09-16T23:49:14.568 INFO:tasks.workunit.client.0.smithi042.stdout:7/988: write d3/d3f/fe5 [4052388,29432] 0 2024-09-16T23:49:14.583 INFO:tasks.workunit.client.0.smithi042.stdout:7/989: dread d3/d1f/d49/d78/dbe/dc8/fcc [0,4194304] 0 2024-09-16T23:49:14.585 INFO:tasks.workunit.client.0.smithi042.stdout:7/990: fdatasync d3/d1f/d49/d78/f7c 0 2024-09-16T23:49:14.587 INFO:tasks.workunit.client.0.smithi042.stdout:7/991: mknod d3/d1f/d49/d78/c15b 0 2024-09-16T23:49:14.589 INFO:tasks.workunit.client.0.smithi042.stdout:7/992: symlink d3/d1f/d49/d76/d101/d64/d92/d153/l15c 0 2024-09-16T23:49:14.599 INFO:tasks.workunit.client.0.smithi042.stdout:7/993: dwrite d3/d1f/d49/d76/d101/da5/f104 [0,4194304] 0 2024-09-16T23:49:15.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:14 smithi042 ceph-mon[27914]: pgmap v16: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 95 MiB/s rd, 234 MiB/s wr, 490 op/s 2024-09-16T23:49:15.100 INFO:tasks.workunit.client.0.smithi042.stdout:9/819: sync 2024-09-16T23:49:15.100 INFO:tasks.workunit.client.0.smithi042.stdout:8/914: sync 2024-09-16T23:49:15.101 INFO:tasks.workunit.client.0.smithi042.stdout:9/820: readlink d4/ld 0 2024-09-16T23:49:15.104 INFO:tasks.workunit.client.0.smithi042.stdout:8/915: rmdir da/d19/d1d/d30/d49/db2 39 2024-09-16T23:49:15.105 INFO:tasks.workunit.client.0.smithi042.stdout:8/916: stat da/d4b/le0 0 2024-09-16T23:49:15.111 INFO:tasks.workunit.client.0.smithi042.stdout:8/917: dwrite da/d19/d1d/d5a/f8e [0,4194304] 0 2024-09-16T23:49:15.118 INFO:tasks.workunit.client.0.smithi042.stdout:8/918: creat da/d4b/d74/dc7/f11c x:0 0 0 2024-09-16T23:49:15.119 INFO:tasks.workunit.client.0.smithi042.stdout:8/919: stat da/d19/d2c/d56/d97/dab 0 2024-09-16T23:49:15.126 INFO:tasks.workunit.client.0.smithi042.stdout:8/920: dwrite da/d19/d1d/d30/d3f/d6d/f108 [0,4194304] 0 2024-09-16T23:49:15.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:14 smithi043 ceph-mon[35191]: pgmap v16: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 95 MiB/s rd, 234 MiB/s wr, 490 op/s 2024-09-16T23:49:15.130 INFO:tasks.workunit.client.0.smithi042.stdout:8/921: creat da/d2f/f11d x:0 0 0 2024-09-16T23:49:15.134 INFO:tasks.workunit.client.0.smithi042.stdout:8/922: mknod da/d19/d1d/d30/d3f/c11e 0 2024-09-16T23:49:15.140 INFO:tasks.workunit.client.0.smithi042.stdout:8/923: dwrite da/d4b/d74/f107 [0,4194304] 0 2024-09-16T23:49:15.148 INFO:tasks.workunit.client.0.smithi042.stdout:8/924: mkdir da/d4b/def/d11f 0 2024-09-16T23:49:15.153 INFO:tasks.workunit.client.0.smithi042.stdout:8/925: dread da/d19/d1d/d30/d3f/dc2/fcd [0,4194304] 0 2024-09-16T23:49:15.159 INFO:tasks.workunit.client.0.smithi042.stdout:8/926: dwrite da/d19/d80/fa6 [0,4194304] 0 2024-09-16T23:49:15.179 INFO:tasks.workunit.client.0.smithi042.stdout:8/927: dwrite da/f17 [0,4194304] 0 2024-09-16T23:49:15.192 INFO:tasks.workunit.client.0.smithi042.stdout:8/928: write da/d19/d2c/d56/f98 [879960,5608] 0 2024-09-16T23:49:15.197 INFO:tasks.workunit.client.0.smithi042.stdout:8/929: fdatasync da/d19/d2c/d56/d97/f9d 0 2024-09-16T23:49:15.198 INFO:tasks.workunit.client.0.smithi042.stdout:8/930: write da/d4b/d74/f107 [1747960,93064] 0 2024-09-16T23:49:15.199 INFO:tasks.workunit.client.0.smithi042.stdout:8/931: stat da/d19/d1d/f89 0 2024-09-16T23:49:15.200 INFO:tasks.workunit.client.0.smithi042.stdout:8/932: chown da/d19/d1d/d30/d3f 22577 1 2024-09-16T23:49:15.204 INFO:tasks.workunit.client.0.smithi042.stdout:8/933: dread - da/d19/d1d/d30/d49/d58/d6a/fe7 zero size 2024-09-16T23:49:15.208 INFO:tasks.workunit.client.0.smithi042.stdout:8/934: fdatasync da/d19/d1d/d30/d3f/f7f 0 2024-09-16T23:49:15.214 INFO:tasks.workunit.client.0.smithi042.stdout:8/935: dwrite da/d19/d1d/d5a/fc6 [0,4194304] 0 2024-09-16T23:49:15.220 INFO:tasks.workunit.client.0.smithi042.stdout:8/936: unlink da/faa 0 2024-09-16T23:49:15.222 INFO:tasks.workunit.client.0.smithi042.stdout:8/937: write da/d4b/f87 [4140063,121137] 0 2024-09-16T23:49:15.226 INFO:tasks.workunit.client.0.smithi042.stdout:8/938: mkdir da/d4b/d74/dc7/d120 0 2024-09-16T23:49:15.229 INFO:tasks.workunit.client.0.smithi042.stdout:8/939: creat da/d19/d2c/d56/d97/dad/dd0/f121 x:0 0 0 2024-09-16T23:49:15.259 INFO:tasks.workunit.client.0.smithi042.stdout:7/994: sync 2024-09-16T23:49:15.260 INFO:tasks.workunit.client.0.smithi042.stdout:2/917: sync 2024-09-16T23:49:15.260 INFO:tasks.workunit.client.0.smithi042.stdout:2/918: chown d0/da/d17/d3e/d86/f11e 48113247 1 2024-09-16T23:49:15.275 INFO:tasks.workunit.client.0.smithi042.stdout:7/995: dread d3/d1f/d49/d76/d101/d64/d92/dfb/f129 [0,4194304] 0 2024-09-16T23:49:15.276 INFO:tasks.workunit.client.0.smithi042.stdout:2/919: dread d0/d26/f50 [0,4194304] 0 2024-09-16T23:49:15.276 INFO:tasks.workunit.client.0.smithi042.stdout:2/920: chown d0/d30/dcb 20 1 2024-09-16T23:49:15.280 INFO:tasks.workunit.client.0.smithi042.stdout:2/921: getdents d0/da/d88 0 2024-09-16T23:49:15.288 INFO:tasks.workunit.client.0.smithi042.stdout:2/922: write d0/da/d17/f42 [4682054,25223] 0 2024-09-16T23:49:15.294 INFO:tasks.workunit.client.0.smithi042.stdout:7/996: dwrite d3/d1f/d49/d78/dbe/dc8/d125/d13a/dcf/fb1 [4194304,4194304] 0 2024-09-16T23:49:15.297 INFO:tasks.workunit.client.0.smithi042.stdout:7/997: fdatasync d3/d98/da0/d24/d28/f3e 0 2024-09-16T23:49:15.298 INFO:tasks.workunit.client.0.smithi042.stdout:2/923: dwrite d0/da/d17/f37 [0,4194304] 0 2024-09-16T23:49:15.302 INFO:tasks.workunit.client.0.smithi042.stdout:7/998: mkdir d3/d1f/d4f/d119/d15d 0 2024-09-16T23:49:15.305 INFO:tasks.workunit.client.0.smithi042.stdout:2/924: truncate d0/da/d17/d3e/f116 2933697 0 2024-09-16T23:49:15.307 INFO:tasks.workunit.client.0.smithi042.stdout:2/925: write d0/da/d17/d3e/d86/d96/f123 [517672,79694] 0 2024-09-16T23:49:15.320 INFO:tasks.workunit.client.0.smithi042.stdout:7/999: dwrite d3/d1f/f137 [4194304,4194304] 0 2024-09-16T23:49:15.322 INFO:tasks.workunit.client.0.smithi042.stdout:2/926: dread d0/fc6 [0,4194304] 0 2024-09-16T23:49:15.324 INFO:tasks.workunit.client.0.smithi042.stdout:2/927: fdatasync d0/d30/fbb 0 2024-09-16T23:49:15.326 INFO:tasks.workunit.client.0.smithi042.stdout:2/928: fsync d0/da/d17/d5c/fce 0 2024-09-16T23:49:15.338 INFO:tasks.workunit.client.0.smithi042.stdout:2/929: dwrite d0/da/d17/d3e/f91 [4194304,4194304] 0 2024-09-16T23:49:15.342 INFO:tasks.workunit.client.0.smithi042.stdout:2/930: creat d0/da/d17/d3e/d86/d96/d11d/f130 x:0 0 0 2024-09-16T23:49:15.345 INFO:tasks.workunit.client.0.smithi042.stdout:2/931: symlink d0/da/d1b/d3b/d44/d54/l131 0 2024-09-16T23:49:15.350 INFO:tasks.workunit.client.0.smithi042.stdout:2/932: write d0/d30/fbb [4595848,76455] 0 2024-09-16T23:49:15.354 INFO:tasks.workunit.client.0.smithi042.stdout:2/933: unlink d0/d30/dcb/de5/c122 0 2024-09-16T23:49:15.355 INFO:tasks.workunit.client.0.smithi042.stdout:2/934: stat d0/da/d17/f42 0 2024-09-16T23:49:15.466 INFO:tasks.workunit.client.0.smithi042.stdout:9/821: sync 2024-09-16T23:49:15.472 INFO:tasks.workunit.client.0.smithi042.stdout:9/822: link d4/d12/d2d/d94/dc2/fa4 d4/d12/d2d/dc9/fff 0 2024-09-16T23:49:15.476 INFO:tasks.workunit.client.0.smithi042.stdout:9/823: dread d4/d12/d2d/d94/dc2/d41/d92/da7/f8b [0,4194304] 0 2024-09-16T23:49:15.487 INFO:tasks.workunit.client.0.smithi042.stdout:9/824: dwrite d4/d12/d60/f78 [0,4194304] 0 2024-09-16T23:49:15.498 INFO:tasks.workunit.client.0.smithi042.stdout:9/825: dwrite d4/d12/d2d/d94/dc2/f70 [0,4194304] 0 2024-09-16T23:49:15.501 INFO:tasks.workunit.client.0.smithi042.stdout:9/826: chown d4/d12/l56 509387 1 2024-09-16T23:49:15.502 INFO:tasks.workunit.client.0.smithi042.stdout:9/827: chown d4/d12/d77/fee 1 1 2024-09-16T23:49:15.504 INFO:tasks.workunit.client.0.smithi042.stdout:9/828: symlink d4/d12/d16/dfe/l100 0 2024-09-16T23:49:15.506 INFO:tasks.workunit.client.0.smithi042.stdout:9/829: mknod d4/d12/d16/da9/c101 0 2024-09-16T23:49:15.507 INFO:tasks.workunit.client.0.smithi042.stdout:9/830: fsync d4/d12/d2d/d94/dc2/d3a/f5c 0 2024-09-16T23:49:15.509 INFO:tasks.workunit.client.0.smithi042.stdout:9/831: creat d4/d12/d16/da9/f102 x:0 0 0 2024-09-16T23:49:15.511 INFO:tasks.workunit.client.0.smithi042.stdout:9/832: read d4/fb [1433889,51296] 0 2024-09-16T23:49:15.511 INFO:tasks.workunit.client.0.smithi042.stdout:9/833: read - d4/d12/d77/dd1/fdc zero size 2024-09-16T23:49:15.514 INFO:tasks.workunit.client.0.smithi042.stdout:9/834: creat d4/d10/d24/f103 x:0 0 0 2024-09-16T23:49:15.515 INFO:tasks.workunit.client.0.smithi042.stdout:9/835: stat d4/d10/d24/f3b 0 2024-09-16T23:49:15.515 INFO:tasks.workunit.client.0.smithi042.stdout:9/836: fdatasync d4/d12/d2d/dc9/fde 0 2024-09-16T23:49:15.516 INFO:tasks.workunit.client.0.smithi042.stdout:9/837: chown d4/d12/d2d/d94/dc2/d3a/f5c 226142272 1 2024-09-16T23:49:15.521 INFO:tasks.workunit.client.0.smithi042.stdout:8/940: sync 2024-09-16T23:49:15.526 INFO:tasks.workunit.client.0.smithi042.stdout:9/838: rmdir d4 39 2024-09-16T23:49:15.530 INFO:tasks.workunit.client.0.smithi042.stdout:9/839: creat d4/d12/f104 x:0 0 0 2024-09-16T23:49:15.532 INFO:tasks.workunit.client.0.smithi042.stdout:9/840: mknod d4/d10/d24/de1/c105 0 2024-09-16T23:49:15.532 INFO:tasks.workunit.client.0.smithi042.stdout:9/841: chown d4/d12/d16/df1 0 1 2024-09-16T23:49:15.534 INFO:tasks.workunit.client.0.smithi042.stdout:9/842: write d4/d12/d2d/d94/dc2/d3a/d50/f99 [5545519,95612] 0 2024-09-16T23:49:15.539 INFO:tasks.workunit.client.0.smithi042.stdout:9/843: write d4/fb [2487492,32567] 0 2024-09-16T23:49:15.541 INFO:tasks.workunit.client.0.smithi042.stdout:9/844: mkdir d4/d12/d2d/dc9/dce/d106 0 2024-09-16T23:49:15.547 INFO:tasks.workunit.client.0.smithi042.stdout:9/845: dwrite d4/d12/d60/f78 [0,4194304] 0 2024-09-16T23:49:15.548 INFO:tasks.workunit.client.0.smithi042.stdout:8/941: truncate da/d19/d2c/d56/f98 941000 0 2024-09-16T23:49:15.553 INFO:tasks.workunit.client.0.smithi042.stdout:8/942: mknod da/d19/d1d/d30/d3f/d6d/c122 0 2024-09-16T23:49:15.556 INFO:tasks.workunit.client.0.smithi042.stdout:9/846: dwrite d4/d12/d16/da9/fe8 [4194304,4194304] 0 2024-09-16T23:49:15.559 INFO:tasks.workunit.client.0.smithi042.stdout:9/847: creat d4/d12/d16/dfe/f107 x:0 0 0 2024-09-16T23:49:15.567 INFO:tasks.workunit.client.0.smithi042.stdout:9/848: rename d4/d12/d2d/d94/dc2/d3a/f5b to d4/d12/d2d/d94/dc2/d28/d5e/f108 0 2024-09-16T23:49:15.567 INFO:tasks.workunit.client.0.smithi042.stdout:8/943: dwrite da/d19/d2c/d56/d97/dad/dd0/f121 [0,4194304] 0 2024-09-16T23:49:15.569 INFO:tasks.workunit.client.0.smithi042.stdout:9/849: rmdir d4/d12/d2d/d51 39 2024-09-16T23:49:15.570 INFO:tasks.workunit.client.0.smithi042.stdout:9/850: dread - d4/d10/d24/f103 zero size 2024-09-16T23:49:15.574 INFO:tasks.workunit.client.0.smithi042.stdout:9/851: creat d4/d12/d2d/d94/f109 x:0 0 0 2024-09-16T23:49:15.576 INFO:tasks.workunit.client.0.smithi042.stdout:9/852: creat d4/d12/d77/f10a x:0 0 0 2024-09-16T23:49:15.578 INFO:tasks.workunit.client.0.smithi042.stdout:9/853: truncate d4/d12/d2d/d94/dc2/d59/f96 9592028 0 2024-09-16T23:49:15.579 INFO:tasks.workunit.client.0.smithi042.stdout:8/944: truncate da/d19/d1d/d30/d3f/d6d/f110 3500421 0 2024-09-16T23:49:15.583 INFO:tasks.workunit.client.0.smithi042.stdout:9/854: rename d4/d12/d2d/d94/dc2/d28/d5e/d83/db3/lf9 to d4/d12/l10b 0 2024-09-16T23:49:15.583 INFO:tasks.workunit.client.0.smithi042.stdout:8/945: rmdir da/d19/d2c/dc8/d115 0 2024-09-16T23:49:15.584 INFO:tasks.workunit.client.0.smithi042.stdout:9/855: dread - d4/d12/d16/dfe/f107 zero size 2024-09-16T23:49:15.586 INFO:tasks.workunit.client.0.smithi042.stdout:9/856: chown d4/d12/d2d/d51/db9/fd9 775931 1 2024-09-16T23:49:15.587 INFO:tasks.workunit.client.0.smithi042.stdout:8/946: mkdir da/d19/d1d/d30/d3f/dc2/d123 0 2024-09-16T23:49:15.591 INFO:tasks.workunit.client.0.smithi042.stdout:9/857: rename d4/d10/d24/fcf to d4/d12/d60/d97/f10c 0 2024-09-16T23:49:15.591 INFO:tasks.workunit.client.0.smithi042.stdout:8/947: creat da/d19/d1d/f124 x:0 0 0 2024-09-16T23:49:15.591 INFO:tasks.workunit.client.0.smithi042.stdout:8/948: chown da/d19/d2c 474845 1 2024-09-16T23:49:15.593 INFO:tasks.workunit.client.0.smithi042.stdout:9/858: chown d4/d12/d2d/d94/dc2/d28/d5e/d83/db3/lc5 11 1 2024-09-16T23:49:15.594 INFO:tasks.workunit.client.0.smithi042.stdout:8/949: stat da/f2a 0 2024-09-16T23:49:15.595 INFO:tasks.workunit.client.0.smithi042.stdout:9/859: mkdir d4/d10/d4b/d10d 0 2024-09-16T23:49:15.595 INFO:tasks.workunit.client.0.smithi042.stdout:8/950: write da/d2f/f5e [294372,46008] 0 2024-09-16T23:49:15.596 INFO:tasks.workunit.client.0.smithi042.stdout:8/951: dread - da/d19/d1d/f6b zero size 2024-09-16T23:49:15.600 INFO:tasks.workunit.client.0.smithi042.stdout:9/860: rename d4/d12/d2d/c47 to d4/d12/d60/d97/c10e 0 2024-09-16T23:49:15.600 INFO:tasks.workunit.client.0.smithi042.stdout:2/935: sync 2024-09-16T23:49:15.601 INFO:tasks.workunit.client.0.smithi042.stdout:9/861: readlink d4/d12/l98 0 2024-09-16T23:49:15.602 INFO:tasks.workunit.client.0.smithi042.stdout:2/936: chown d0/d30/d43/c7b 7411520 1 2024-09-16T23:49:15.603 INFO:tasks.workunit.client.0.smithi042.stdout:9/862: stat d4/d12/d16/da9/c79 0 2024-09-16T23:49:15.604 INFO:tasks.workunit.client.0.smithi042.stdout:8/952: dwrite da/d19/d2c/d56/d97/dad/fb5 [0,4194304] 0 2024-09-16T23:49:15.604 INFO:tasks.workunit.client.0.smithi042.stdout:2/937: truncate d0/da/d17/f59 4966826 0 2024-09-16T23:49:15.606 INFO:tasks.workunit.client.0.smithi042.stdout:9/863: truncate d4/d10/d24/de1/fec 122825 0 2024-09-16T23:49:15.607 INFO:tasks.workunit.client.0.smithi042.stdout:2/938: symlink d0/d78/l132 0 2024-09-16T23:49:15.615 INFO:tasks.workunit.client.0.smithi042.stdout:2/939: dwrite d0/da/f2c [0,4194304] 0 2024-09-16T23:49:15.622 INFO:tasks.workunit.client.0.smithi042.stdout:8/953: dwrite da/d19/d2c/f2d [0,4194304] 0 2024-09-16T23:49:15.624 INFO:tasks.workunit.client.0.smithi042.stdout:9/864: truncate d4/d10/d24/f3b 1324624 0 2024-09-16T23:49:15.627 INFO:tasks.workunit.client.0.smithi042.stdout:9/865: symlink d4/d12/d16/da9/l10f 0 2024-09-16T23:49:15.629 INFO:tasks.workunit.client.0.smithi042.stdout:8/954: mknod da/d19/d1d/d30/d3f/dc2/d123/c125 0 2024-09-16T23:49:15.629 INFO:tasks.workunit.client.0.smithi042.stdout:9/866: mknod d4/d10/d4b/df4/c110 0 2024-09-16T23:49:15.631 INFO:tasks.workunit.client.0.smithi042.stdout:9/867: stat d4/d12/f36 0 2024-09-16T23:49:15.635 INFO:tasks.workunit.client.0.smithi042.stdout:9/868: mknod d4/d10/d4b/d10d/c111 0 2024-09-16T23:49:15.636 INFO:tasks.workunit.client.0.smithi042.stdout:2/940: dwrite d0/d30/dcb/de5/fc4 [0,4194304] 0 2024-09-16T23:49:15.640 INFO:tasks.workunit.client.0.smithi042.stdout:9/869: readlink d4/d12/leb 0 2024-09-16T23:49:15.640 INFO:tasks.workunit.client.0.smithi042.stdout:8/955: dwrite da/d19/d1d/d30/d49/d58/f118 [0,4194304] 0 2024-09-16T23:49:15.641 INFO:tasks.workunit.client.0.smithi042.stdout:9/870: chown d4/d12/d16/dfe/f107 544 1 2024-09-16T23:49:15.643 INFO:tasks.workunit.client.0.smithi042.stdout:9/871: chown d4/d10/d4b/df4 76 1 2024-09-16T23:49:15.649 INFO:tasks.workunit.client.0.smithi042.stdout:9/872: mknod d4/d12/d2d/d94/dc2/d28/d5e/d83/c112 0 2024-09-16T23:49:15.650 INFO:tasks.workunit.client.0.smithi042.stdout:2/941: dwrite d0/d5d/f82 [0,4194304] 0 2024-09-16T23:49:15.653 INFO:tasks.workunit.client.0.smithi042.stdout:2/942: creat d0/da/d17/d5c/df6/f133 x:0 0 0 2024-09-16T23:49:15.653 INFO:tasks.workunit.client.0.smithi042.stdout:9/873: truncate d4/d10/d24/f3b 6595 0 2024-09-16T23:49:15.656 INFO:tasks.workunit.client.0.smithi042.stdout:2/943: symlink d0/da/d17/d3e/d86/d96/l134 0 2024-09-16T23:49:15.658 INFO:tasks.workunit.client.0.smithi042.stdout:9/874: mknod d4/d12/d2d/d94/dc2/c113 0 2024-09-16T23:49:15.659 INFO:tasks.workunit.client.0.smithi042.stdout:2/944: symlink d0/da/d1b/d3b/d44/dec/l135 0 2024-09-16T23:49:15.660 INFO:tasks.workunit.client.0.smithi042.stdout:8/956: truncate da/f71 3842532 0 2024-09-16T23:49:15.660 INFO:tasks.workunit.client.0.smithi042.stdout:9/875: mkdir d4/d12/d114 0 2024-09-16T23:49:15.662 INFO:tasks.workunit.client.0.smithi042.stdout:2/945: symlink d0/da/d17/d3e/d86/l136 0 2024-09-16T23:49:15.663 INFO:tasks.workunit.client.0.smithi042.stdout:9/876: symlink d4/d12/d2d/d94/dc2/d28/l115 0 2024-09-16T23:49:15.664 INFO:tasks.workunit.client.0.smithi042.stdout:8/957: creat da/d19/d1d/d30/d3f/dc2/de3/f126 x:0 0 0 2024-09-16T23:49:15.666 INFO:tasks.workunit.client.0.smithi042.stdout:2/946: mknod d0/d5d/d10c/c137 0 2024-09-16T23:49:15.666 INFO:tasks.workunit.client.0.smithi042.stdout:8/958: fsync da/d19/d1d/d30/d49/d58/f119 0 2024-09-16T23:49:15.668 INFO:tasks.workunit.client.0.smithi042.stdout:8/959: readlink da/d19/d2c/db0/lba 0 2024-09-16T23:49:15.672 INFO:tasks.workunit.client.0.smithi042.stdout:9/877: dwrite d4/d12/d2d/d94/dc2/d59/fbf [0,4194304] 0 2024-09-16T23:49:15.677 INFO:tasks.workunit.client.0.smithi042.stdout:9/878: chown d4/d12/d2d/d94/dc2/d3a/d50/fd8 17 1 2024-09-16T23:49:15.679 INFO:tasks.workunit.client.0.smithi042.stdout:2/947: dwrite d0/da/d17/d3e/fed [0,4194304] 0 2024-09-16T23:49:15.689 INFO:tasks.workunit.client.0.smithi042.stdout:2/948: rename d0/da/d17/d3e/d86/d96/f123 to d0/d8d/f138 0 2024-09-16T23:49:15.690 INFO:tasks.workunit.client.0.smithi042.stdout:9/879: dwrite d4/d12/d16/da9/fe8 [4194304,4194304] 0 2024-09-16T23:49:15.690 INFO:tasks.workunit.client.0.smithi042.stdout:8/960: getdents da/d19/d2c/d56/d97 0 2024-09-16T23:49:15.690 INFO:tasks.workunit.client.0.smithi042.stdout:9/880: stat d4/d12/d16/dfe/l100 0 2024-09-16T23:49:15.690 INFO:tasks.workunit.client.0.smithi042.stdout:2/949: chown d0/da/d1b/d3b/d44/d54/df3/cfc 27868314 1 2024-09-16T23:49:15.692 INFO:tasks.workunit.client.0.smithi042.stdout:2/950: write d0/d30/dcb/de5/fda [810860,45030] 0 2024-09-16T23:49:15.696 INFO:tasks.workunit.client.0.smithi042.stdout:8/961: dwrite da/d19/d1d/d30/d3f/d6d/f108 [0,4194304] 0 2024-09-16T23:49:15.701 INFO:tasks.workunit.client.0.smithi042.stdout:8/962: mknod da/d19/d1d/d30/c127 0 2024-09-16T23:49:15.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/963: symlink da/d4b/d74/l128 0 2024-09-16T23:49:15.705 INFO:tasks.workunit.client.0.smithi042.stdout:8/964: readlink da/d4b/lcf 0 2024-09-16T23:49:15.709 INFO:tasks.workunit.client.0.smithi042.stdout:8/965: creat da/d19/d1d/d30/d3f/f129 x:0 0 0 2024-09-16T23:49:15.715 INFO:tasks.workunit.client.0.smithi042.stdout:8/966: link da/d19/d2c/d56/ccb da/d19/d1d/d30/d49/db2/c12a 0 2024-09-16T23:49:15.717 INFO:tasks.workunit.client.0.smithi042.stdout:8/967: creat da/d19/d2c/d56/f12b x:0 0 0 2024-09-16T23:49:15.722 INFO:tasks.workunit.client.0.smithi042.stdout:8/968: link da/f17 da/d19/d2c/db0/f12c 0 2024-09-16T23:49:15.722 INFO:tasks.workunit.client.0.smithi042.stdout:8/969: fdatasync da/d4b/d74/f107 0 2024-09-16T23:49:15.728 INFO:tasks.workunit.client.0.smithi042.stdout:8/970: dwrite da/d19/d1d/d30/d3f/f61 [0,4194304] 0 2024-09-16T23:49:15.730 INFO:tasks.workunit.client.0.smithi042.stdout:8/971: readlink da/d19/d1d/d30/d49/d58/le6 0 2024-09-16T23:49:15.733 INFO:tasks.workunit.client.0.smithi042.stdout:8/972: symlink da/d19/d2c/dc8/l12d 0 2024-09-16T23:49:15.748 INFO:tasks.workunit.client.0.smithi042.stdout:8/973: dread da/d2f/fbc [0,4194304] 0 2024-09-16T23:49:15.750 INFO:tasks.workunit.client.0.smithi042.stdout:8/974: dread - da/d19/d1d/d30/d3f/dc2/fd3 zero size 2024-09-16T23:49:15.750 INFO:tasks.workunit.client.0.smithi042.stdout:8/975: chown da/d19/d2c/f4e 55425 1 2024-09-16T23:49:16.192 INFO:tasks.workunit.client.0.smithi042.stdout:8/976: sync 2024-09-16T23:49:16.193 INFO:tasks.workunit.client.0.smithi042.stdout:9/881: sync 2024-09-16T23:49:16.193 INFO:tasks.workunit.client.0.smithi042.stdout:2/951: sync 2024-09-16T23:49:16.248 INFO:tasks.workunit.client.0.smithi042.stdout:9/882: creat d4/d10/d4b/f116 x:0 0 0 2024-09-16T23:49:16.250 INFO:tasks.workunit.client.0.smithi042.stdout:9/883: write d4/d12/f63 [72559,113508] 0 2024-09-16T23:49:16.251 INFO:tasks.workunit.client.0.smithi042.stdout:2/952: rename d0/da/d1b/d48/db7/c108 to d0/d30/d43/c139 0 2024-09-16T23:49:16.252 INFO:tasks.workunit.client.0.smithi042.stdout:9/884: symlink d4/d10/d4b/dfd/l117 0 2024-09-16T23:49:16.253 INFO:tasks.workunit.client.0.smithi042.stdout:9/885: dread - d4/d12/f104 zero size 2024-09-16T23:49:16.255 INFO:tasks.workunit.client.0.smithi042.stdout:9/886: creat d4/d10/d4b/dfd/f118 x:0 0 0 2024-09-16T23:49:16.257 INFO:tasks.workunit.client.0.smithi042.stdout:2/953: rename d0/d98/lb6 to d0/da/d88/dae/dff/l13a 0 2024-09-16T23:49:16.258 INFO:tasks.workunit.client.0.smithi042.stdout:9/887: mknod d4/d12/d2d/d94/dc2/d41/d92/da7/c119 0 2024-09-16T23:49:16.258 INFO:tasks.workunit.client.0.smithi042.stdout:2/954: readlink d0/da/d1b/d3b/d44/d54/l131 0 2024-09-16T23:49:16.260 INFO:tasks.workunit.client.0.smithi042.stdout:2/955: read d0/da/d1b/d3b/d44/f115 [1038573,33398] 0 2024-09-16T23:49:16.260 INFO:tasks.workunit.client.0.smithi042.stdout:9/888: creat d4/d12/d2d/d94/dc2/d28/d5e/d83/f11a x:0 0 0 2024-09-16T23:49:16.266 INFO:tasks.workunit.client.0.smithi042.stdout:9/889: rename d4/d10/d4b/d10d to d4/d12/d2d/d94/d11b 0 2024-09-16T23:49:16.266 INFO:tasks.workunit.client.0.smithi042.stdout:9/890: chown d4/d12 238 1 2024-09-16T23:49:16.267 INFO:tasks.workunit.client.0.smithi042.stdout:2/956: truncate d0/da/d17/d3e/d86/fd7 530523 0 2024-09-16T23:49:16.269 INFO:tasks.workunit.client.0.smithi042.stdout:2/957: mknod d0/d78/d8b/c13b 0 2024-09-16T23:49:16.270 INFO:tasks.workunit.client.0.smithi042.stdout:2/958: fsync d0/d78/d8b/f12f 0 2024-09-16T23:49:16.271 INFO:tasks.workunit.client.0.smithi042.stdout:2/959: write d0/da/d17/d3e/fef [2850080,79261] 0 2024-09-16T23:49:16.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:16.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:16.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi043 to 2241M 2024-09-16T23:49:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:15 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:16.275 INFO:tasks.workunit.client.0.smithi042.stdout:2/960: rename d0/fc6 to d0/da/d88/d129/f13c 0 2024-09-16T23:49:16.277 INFO:tasks.workunit.client.0.smithi042.stdout:2/961: creat d0/da/d1b/d3b/d44/dec/f13d x:0 0 0 2024-09-16T23:49:16.279 INFO:tasks.workunit.client.0.smithi042.stdout:2/962: chown d0/da/d17/d3e/d86/cb9 56 1 2024-09-16T23:49:16.280 INFO:tasks.workunit.client.0.smithi042.stdout:2/963: stat d0/da/d1b/d3b/d44/d54/fb1 0 2024-09-16T23:49:16.281 INFO:tasks.workunit.client.0.smithi042.stdout:2/964: chown d0/da/d1b/d3b/d44/d54/df3/cfc 1177332 1 2024-09-16T23:49:16.300 INFO:tasks.workunit.client.0.smithi042.stdout:9/891: write d4/d12/d2d/fc1 [5144222,43129] 0 2024-09-16T23:49:16.300 INFO:tasks.workunit.client.0.smithi042.stdout:9/892: dread - d4/d12/d77/f10a zero size 2024-09-16T23:49:16.301 INFO:tasks.workunit.client.0.smithi042.stdout:9/893: symlink d4/d12/d77/l11c 0 2024-09-16T23:49:16.302 INFO:tasks.workunit.client.0.smithi042.stdout:9/894: stat d4/d12/d2d/d94/dc2/d3a/d50/cae 0 2024-09-16T23:49:16.304 INFO:tasks.workunit.client.0.smithi042.stdout:9/895: write d4/d12/d2d/d94/dc2/d3a/f48 [325316,102709] 0 2024-09-16T23:49:16.306 INFO:tasks.workunit.client.0.smithi042.stdout:2/965: dwrite d0/da/d88/dae/fee [0,4194304] 0 2024-09-16T23:49:16.308 INFO:tasks.workunit.client.0.smithi042.stdout:9/896: mknod d4/d12/d2d/d94/dc2/d28/c11d 0 2024-09-16T23:49:16.309 INFO:tasks.workunit.client.0.smithi042.stdout:2/966: creat d0/d78/d8b/f13e x:0 0 0 2024-09-16T23:49:16.310 INFO:tasks.workunit.client.0.smithi042.stdout:9/897: rmdir d4/d12/d2d/d94/dc2/d3a/d50 39 2024-09-16T23:49:16.311 INFO:tasks.workunit.client.0.smithi042.stdout:2/967: symlink d0/d30/dcb/d12a/l13f 0 2024-09-16T23:49:16.313 INFO:tasks.workunit.client.0.smithi042.stdout:2/968: symlink d0/d5d/d10c/l140 0 2024-09-16T23:49:16.315 INFO:tasks.workunit.client.0.smithi042.stdout:2/969: mkdir d0/d8d/da0/d141 0 2024-09-16T23:49:16.318 INFO:tasks.workunit.client.0.smithi042.stdout:9/898: dread d4/d12/d2d/d94/dc2/d3a/f5c [0,4194304] 0 2024-09-16T23:49:16.322 INFO:tasks.workunit.client.0.smithi042.stdout:2/970: dwrite d0/da/d17/d3e/d86/f119 [0,4194304] 0 2024-09-16T23:49:16.323 INFO:tasks.workunit.client.0.smithi042.stdout:9/899: rename d4/d12/d2d/d51/db9/fe4 to d4/d12/d60/d97/f11e 0 2024-09-16T23:49:16.326 INFO:tasks.workunit.client.0.smithi042.stdout:9/900: truncate d4/d12/d60/d97/fa0 1015721 0 2024-09-16T23:49:16.328 INFO:tasks.workunit.client.0.smithi042.stdout:9/901: mknod d4/d12/d77/dc4/c11f 0 2024-09-16T23:49:16.331 INFO:tasks.workunit.client.0.smithi042.stdout:9/902: stat d4/d12/d77/dd1/le6 0 2024-09-16T23:49:16.332 INFO:tasks.workunit.client.0.smithi042.stdout:2/971: write d0/d30/d43/f89 [1278674,58351] 0 2024-09-16T23:49:16.332 INFO:tasks.workunit.client.0.smithi042.stdout:9/903: fdatasync d4/d12/d60/f78 0 2024-09-16T23:49:16.334 INFO:tasks.workunit.client.0.smithi042.stdout:2/972: write d0/da/d17/d3e/d86/d96/d11d/f130 [900382,78597] 0 2024-09-16T23:49:16.335 INFO:tasks.workunit.client.0.smithi042.stdout:9/904: fsync d4/d12/d77/dc4/ff7 0 2024-09-16T23:49:16.337 INFO:tasks.workunit.client.0.smithi042.stdout:9/905: write d4/d12/d60/f78 [4414122,73245] 0 2024-09-16T23:49:16.339 INFO:tasks.workunit.client.0.smithi042.stdout:2/973: creat d0/d30/f142 x:0 0 0 2024-09-16T23:49:16.341 INFO:tasks.workunit.client.0.smithi042.stdout:2/974: symlink d0/d5d/l143 0 2024-09-16T23:49:16.343 INFO:tasks.workunit.client.0.smithi042.stdout:2/975: write d0/da/d88/fe8 [9109296,96503] 0 2024-09-16T23:49:16.344 INFO:tasks.workunit.client.0.smithi042.stdout:9/906: dwrite d4/d10/d24/de1/fec [0,4194304] 0 2024-09-16T23:49:16.348 INFO:tasks.workunit.client.0.smithi042.stdout:9/907: creat d4/d12/d2d/d94/dc2/d28/d5e/d83/f120 x:0 0 0 2024-09-16T23:49:16.349 INFO:tasks.workunit.client.0.smithi042.stdout:2/976: symlink d0/da/d17/d104/l144 0 2024-09-16T23:49:16.350 INFO:tasks.workunit.client.0.smithi042.stdout:2/977: read - d0/da/d17/d5c/df6/f133 zero size 2024-09-16T23:49:16.353 INFO:tasks.workunit.client.0.smithi042.stdout:9/908: dread d4/d12/d2d/d94/dc2/d41/d92/da7/f8b [0,4194304] 0 2024-09-16T23:49:16.357 INFO:tasks.workunit.client.0.smithi042.stdout:9/909: mkdir d4/d12/d2d/d94/d121 0 2024-09-16T23:49:16.358 INFO:tasks.workunit.client.0.smithi042.stdout:9/910: write d4/d12/d2d/dc9/dce/d8e/da1/ff2 [36146,116670] 0 2024-09-16T23:49:16.368 INFO:tasks.workunit.client.0.smithi042.stdout:2/978: dwrite d0/d8d/da0/fd6 [0,4194304] 0 2024-09-16T23:49:16.371 INFO:tasks.workunit.client.0.smithi042.stdout:2/979: mkdir d0/d26/d145 0 2024-09-16T23:49:16.372 INFO:tasks.workunit.client.0.smithi042.stdout:2/980: chown d0/d78 197 1 2024-09-16T23:49:16.374 INFO:tasks.workunit.client.0.smithi042.stdout:2/981: mknod d0/d30/dcb/c146 0 2024-09-16T23:49:16.375 INFO:tasks.workunit.client.0.smithi042.stdout:9/911: dread d4/d12/d2d/d94/dc2/d41/d92/fcc [0,4194304] 0 2024-09-16T23:49:16.376 INFO:tasks.workunit.client.0.smithi042.stdout:2/982: chown d0/da/d1b/d3b/d44/d54/f112 2522 1 2024-09-16T23:49:16.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:16.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi043 to 2241M 2024-09-16T23:49:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:15 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:16.378 INFO:tasks.workunit.client.0.smithi042.stdout:9/912: read d4/d12/d2d/d94/dc2/d28/f65 [4249289,113347] 0 2024-09-16T23:49:16.380 INFO:tasks.workunit.client.0.smithi042.stdout:9/913: creat d4/d10/f122 x:0 0 0 2024-09-16T23:49:16.383 INFO:tasks.workunit.client.0.smithi042.stdout:9/914: creat d4/d12/d2d/d94/dc2/d28/d5e/d83/db3/f123 x:0 0 0 2024-09-16T23:49:16.385 INFO:tasks.workunit.client.0.smithi042.stdout:9/915: creat d4/d12/d2d/d94/dc2/d28/d5e/f124 x:0 0 0 2024-09-16T23:49:16.389 INFO:tasks.workunit.client.0.smithi042.stdout:9/916: truncate d4/d10/d24/f3b 705693 0 2024-09-16T23:49:16.392 INFO:tasks.workunit.client.0.smithi042.stdout:2/983: dread d0/da/d17/d3e/d86/fa1 [0,4194304] 0 2024-09-16T23:49:16.393 INFO:tasks.workunit.client.0.smithi042.stdout:9/917: rename d4/d12/d2d/d51/l8d to d4/d12/d2d/d94/dc2/d3a/d50/l125 0 2024-09-16T23:49:16.394 INFO:tasks.workunit.client.0.smithi042.stdout:9/918: stat d4/d12/d2d/d51/db9/fd9 0 2024-09-16T23:49:16.394 INFO:tasks.workunit.client.0.smithi042.stdout:2/984: dread - d0/da/d17/d3e/f69 zero size 2024-09-16T23:49:16.395 INFO:tasks.workunit.client.0.smithi042.stdout:2/985: chown d0/da/d1b/d3b/l61 48 1 2024-09-16T23:49:16.405 INFO:tasks.workunit.client.0.smithi042.stdout:9/919: dwrite d4/d12/d2d/dc9/fff [0,4194304] 0 2024-09-16T23:49:16.407 INFO:tasks.workunit.client.0.smithi042.stdout:2/986: dread d0/d30/f38 [0,4194304] 0 2024-09-16T23:49:16.409 INFO:tasks.workunit.client.0.smithi042.stdout:9/920: symlink d4/d12/d2d/l126 0 2024-09-16T23:49:16.413 INFO:tasks.workunit.client.0.smithi042.stdout:2/987: rename d0/da/c10 to d0/da/d88/c147 0 2024-09-16T23:49:16.414 INFO:tasks.workunit.client.0.smithi042.stdout:9/921: dread d4/d12/d2d/dc9/fff [0,4194304] 0 2024-09-16T23:49:16.416 INFO:tasks.workunit.client.0.smithi042.stdout:2/988: creat d0/da/d1b/d48/f148 x:0 0 0 2024-09-16T23:49:16.418 INFO:tasks.workunit.client.0.smithi042.stdout:2/989: symlink d0/da/d17/d5c/l149 0 2024-09-16T23:49:16.421 INFO:tasks.workunit.client.0.smithi042.stdout:9/922: write d4/d12/d77/dc4/ff7 [119150,66909] 0 2024-09-16T23:49:16.422 INFO:tasks.workunit.client.0.smithi042.stdout:9/923: readlink d4/d12/d2d/dc9/dce/l8f 0 2024-09-16T23:49:16.424 INFO:tasks.workunit.client.0.smithi042.stdout:2/990: rename d0/d30/dcb/c146 to d0/da/d1b/c14a 0 2024-09-16T23:49:16.424 INFO:tasks.workunit.client.0.smithi042.stdout:9/924: write d4/f5 [3718837,128826] 0 2024-09-16T23:49:16.425 INFO:tasks.workunit.client.0.smithi042.stdout:2/991: stat d0/da/d1b/d3b/d44/d54/df3/cfc 0 2024-09-16T23:49:16.430 INFO:tasks.workunit.client.0.smithi042.stdout:8/977: sync 2024-09-16T23:49:16.433 INFO:tasks.workunit.client.0.smithi042.stdout:2/992: dread d0/d30/d43/d77/fbc [0,4194304] 0 2024-09-16T23:49:16.434 INFO:tasks.workunit.client.0.smithi042.stdout:8/978: creat da/d19/d1d/d5a/f12e x:0 0 0 2024-09-16T23:49:16.437 INFO:tasks.workunit.client.0.smithi042.stdout:8/979: creat da/d19/d1d/d5a/f12f x:0 0 0 2024-09-16T23:49:16.438 INFO:tasks.workunit.client.0.smithi042.stdout:2/993: link d0/da/d17/la6 d0/da/d17/l14b 0 2024-09-16T23:49:16.440 INFO:tasks.workunit.client.0.smithi042.stdout:9/925: dread d4/d10/f3f [0,4194304] 0 2024-09-16T23:49:16.447 INFO:tasks.workunit.client.0.smithi042.stdout:8/980: dwrite da/f83 [0,4194304] 0 2024-09-16T23:49:16.448 INFO:tasks.workunit.client.0.smithi042.stdout:2/994: mkdir d0/da/d17/d5c/df6/d11f/d14c 0 2024-09-16T23:49:16.449 INFO:tasks.workunit.client.0.smithi042.stdout:9/926: dread d4/d12/d2d/d94/dc2/f70 [0,4194304] 0 2024-09-16T23:49:16.450 INFO:tasks.workunit.client.0.smithi042.stdout:8/981: symlink da/d4b/def/l130 0 2024-09-16T23:49:16.455 INFO:tasks.workunit.client.0.smithi042.stdout:8/982: truncate da/d19/d1d/f2b 1691267 0 2024-09-16T23:49:16.455 INFO:tasks.workunit.client.0.smithi042.stdout:9/927: symlink d4/d12/d2d/d94/dc2/d41/l127 0 2024-09-16T23:49:16.457 INFO:tasks.workunit.client.0.smithi042.stdout:2/995: write d0/d30/d43/ff1 [177661,31147] 0 2024-09-16T23:49:16.458 INFO:tasks.workunit.client.0.smithi042.stdout:9/928: mkdir d4/d12/d2d/d51/db9/d128 0 2024-09-16T23:49:16.460 INFO:tasks.workunit.client.0.smithi042.stdout:8/983: mknod da/d19/d1d/d30/d49/d58/d100/c131 0 2024-09-16T23:49:16.462 INFO:tasks.workunit.client.0.smithi042.stdout:2/996: truncate d0/da/d17/fbe 192165 0 2024-09-16T23:49:16.462 INFO:tasks.workunit.client.0.smithi042.stdout:8/984: dread - da/d19/d1d/d30/d3f/dc2/de3/f126 zero size 2024-09-16T23:49:16.463 INFO:tasks.workunit.client.0.smithi042.stdout:9/929: link d4/d12/d2d/dc9/dce/fca d4/d10/d4b/dfd/f129 0 2024-09-16T23:49:16.464 INFO:tasks.workunit.client.0.smithi042.stdout:8/985: write da/d4b/d74/f109 [71226,56926] 0 2024-09-16T23:49:16.464 INFO:tasks.workunit.client.0.smithi042.stdout:2/997: symlink d0/d30/d43/df0/l14d 0 2024-09-16T23:49:16.465 INFO:tasks.workunit.client.0.smithi042.stdout:9/930: fsync d4/d10/d24/de1/fec 0 2024-09-16T23:49:16.466 INFO:tasks.workunit.client.0.smithi042.stdout:2/998: readlink d0/da/d17/d5c/df6/l127 0 2024-09-16T23:49:16.468 INFO:tasks.workunit.client.0.smithi042.stdout:9/931: dread - d4/d12/d2d/dc9/dce/d8e/ff5 zero size 2024-09-16T23:49:16.469 INFO:tasks.workunit.client.0.smithi042.stdout:2/999: mknod d0/da/d1b/d3b/d44/dec/c14e 0 2024-09-16T23:49:16.470 INFO:tasks.workunit.client.0.smithi042.stdout:9/932: chown d4/d10/f4a 14 1 2024-09-16T23:49:16.474 INFO:tasks.workunit.client.0.smithi042.stdout:8/986: dwrite da/d19/d1d/d30/d49/d58/d6a/f8b [0,4194304] 0 2024-09-16T23:49:16.474 INFO:tasks.workunit.client.0.smithi042.stdout:9/933: write d4/d12/d2d/dc9/fff [744551,85709] 0 2024-09-16T23:49:16.475 INFO:tasks.workunit.client.0.smithi042.stdout:9/934: chown d4/d12/d2d/ccb 4358709 1 2024-09-16T23:49:16.481 INFO:tasks.workunit.client.0.smithi042.stdout:9/935: write d4/d12/d2d/dc9/dce/d8e/ff5 [553271,6444] 0 2024-09-16T23:49:16.482 INFO:tasks.workunit.client.0.smithi042.stdout:8/987: dread da/f96 [0,4194304] 0 2024-09-16T23:49:16.484 INFO:tasks.workunit.client.0.smithi042.stdout:9/936: mknod d4/d12/d2d/d94/c12a 0 2024-09-16T23:49:16.485 INFO:tasks.workunit.client.0.smithi042.stdout:8/988: symlink da/d19/d1d/d30/d49/db2/l132 0 2024-09-16T23:49:16.486 INFO:tasks.workunit.client.0.smithi042.stdout:8/989: readlink da/d4b/lcf 0 2024-09-16T23:49:16.486 INFO:tasks.workunit.client.0.smithi042.stdout:9/937: read d4/d12/d60/f78 [1558206,34224] 0 2024-09-16T23:49:16.489 INFO:tasks.workunit.client.0.smithi042.stdout:8/990: write da/f96 [808389,39873] 0 2024-09-16T23:49:16.498 INFO:tasks.workunit.client.0.smithi042.stdout:8/991: dwrite da/d19/f11a [0,4194304] 0 2024-09-16T23:49:16.502 INFO:tasks.workunit.client.0.smithi042.stdout:9/938: dread d4/d12/d2d/d94/dc2/d3a/f48 [0,4194304] 0 2024-09-16T23:49:16.505 INFO:tasks.workunit.client.0.smithi042.stdout:8/992: dwrite da/d2f/fbc [0,4194304] 0 2024-09-16T23:49:16.520 INFO:tasks.workunit.client.0.smithi042.stdout:8/993: dread da/d19/d1d/f2b [0,4194304] 0 2024-09-16T23:49:16.523 INFO:tasks.workunit.client.0.smithi042.stdout:8/994: mknod da/d19/d2c/db0/c133 0 2024-09-16T23:49:16.527 INFO:tasks.workunit.client.0.smithi042.stdout:8/995: dread da/d19/f23 [0,4194304] 0 2024-09-16T23:49:16.533 INFO:tasks.workunit.client.0.smithi042.stdout:8/996: dwrite da/d19/d2c/db0/ff2 [0,4194304] 0 2024-09-16T23:49:16.535 INFO:tasks.workunit.client.0.smithi042.stdout:8/997: write da/d19/d1d/d5a/f12f [387291,99498] 0 2024-09-16T23:49:16.546 INFO:tasks.workunit.client.0.smithi042.stdout:8/998: write da/d19/f23 [643960,69220] 0 2024-09-16T23:49:16.551 INFO:tasks.workunit.client.0.smithi042.stdout:8/999: creat da/d19/d1d/d5a/f134 x:0 0 0 2024-09-16T23:49:16.741 INFO:tasks.workunit.client.0.smithi042.stdout:9/939: sync 2024-09-16T23:49:16.749 INFO:tasks.workunit.client.0.smithi042.stdout:9/940: dread d4/d10/f1f [4194304,4194304] 0 2024-09-16T23:49:16.755 INFO:tasks.workunit.client.0.smithi042.stdout:9/941: dwrite d4/d12/d2d/d94/dc2/d59/fbf [4194304,4194304] 0 2024-09-16T23:49:16.756 INFO:tasks.workunit.client.0.smithi042.stdout:9/942: readlink d4/d12/d77/dd1/le6 0 2024-09-16T23:49:16.759 INFO:tasks.workunit.client.0.smithi042.stdout:9/943: mkdir d4/d10/d4b/dfd/d12b 0 2024-09-16T23:49:16.762 INFO:tasks.workunit.client.0.smithi042.stdout:9/944: mkdir d4/d12/d2d/d94/dc2/d59/d12c 0 2024-09-16T23:49:16.765 INFO:tasks.workunit.client.0.smithi042.stdout:9/945: creat d4/d12/d2d/d94/d121/f12d x:0 0 0 2024-09-16T23:49:16.766 INFO:tasks.workunit.client.0.smithi042.stdout:9/946: write d4/d10/f6b [3465215,125824] 0 2024-09-16T23:49:16.768 INFO:tasks.workunit.client.0.smithi042.stdout:9/947: write d4/d10/d4b/dfd/f118 [111424,8545] 0 2024-09-16T23:49:16.768 INFO:tasks.workunit.client.0.smithi042.stdout:9/948: fsync d4/d12/d77/f10a 0 2024-09-16T23:49:16.775 INFO:tasks.workunit.client.0.smithi042.stdout:9/949: dwrite d4/d12/d60/fd4 [0,4194304] 0 2024-09-16T23:49:16.777 INFO:tasks.workunit.client.0.smithi042.stdout:9/950: chown d4/d12/d2d/d94/dc2/c2b 242822 1 2024-09-16T23:49:16.778 INFO:tasks.workunit.client.0.smithi042.stdout:9/951: rmdir d4/d12/d16/d68 39 2024-09-16T23:49:16.779 INFO:tasks.workunit.client.0.smithi042.stdout:9/952: chown d4/d10/d24/c8a 1522 1 2024-09-16T23:49:16.782 INFO:tasks.workunit.client.0.smithi042.stdout:9/953: mkdir d4/d12/d2d/d94/dc2/d28/d5e/d12e 0 2024-09-16T23:49:16.787 INFO:tasks.workunit.client.0.smithi042.stdout:9/954: getdents d4/d12/d2d/d94/d121 0 2024-09-16T23:49:16.790 INFO:tasks.workunit.client.0.smithi042.stdout:9/955: getdents d4/d10/d4b 0 2024-09-16T23:49:16.791 INFO:tasks.workunit.client.0.smithi042.stdout:9/956: stat d4/d12/d2d/dc9/ldb 0 2024-09-16T23:49:16.793 INFO:tasks.workunit.client.0.smithi042.stdout:9/957: write d4/d12/d2d/d51/fbe [879076,27288] 0 2024-09-16T23:49:16.955 INFO:tasks.workunit.client.0.smithi042.stdout:9/958: sync 2024-09-16T23:49:16.956 INFO:tasks.workunit.client.0.smithi042.stdout:9/959: write d4/d10/f122 [536342,25030] 0 2024-09-16T23:49:16.963 INFO:tasks.workunit.client.0.smithi042.stdout:9/960: dwrite d4/d12/d16/fc8 [0,4194304] 0 2024-09-16T23:49:16.971 INFO:tasks.workunit.client.0.smithi042.stdout:9/961: dwrite d4/d10/d24/f103 [0,4194304] 0 2024-09-16T23:49:16.973 INFO:tasks.workunit.client.0.smithi042.stdout:9/962: truncate d4/d12/d77/dc4/ff7 257803 0 2024-09-16T23:49:16.978 INFO:tasks.workunit.client.0.smithi042.stdout:9/963: creat d4/d10/d24/f12f x:0 0 0 2024-09-16T23:49:16.980 INFO:tasks.workunit.client.0.smithi042.stdout:9/964: write d4/d12/d2d/dc9/dce/d8e/da1/ff2 [1179624,73168] 0 2024-09-16T23:49:16.981 INFO:tasks.workunit.client.0.smithi042.stdout:9/965: write d4/d10/f122 [403840,77506] 0 2024-09-16T23:49:16.984 INFO:tasks.workunit.client.0.smithi042.stdout:9/966: fdatasync d4/d12/d2d/d94/dc2/d3a/fd0 0 2024-09-16T23:49:16.990 INFO:tasks.workunit.client.0.smithi042.stdout:9/967: dwrite d4/d12/f63 [0,4194304] 0 2024-09-16T23:49:16.991 INFO:tasks.workunit.client.0.smithi042.stdout:9/968: readlink d4/d10/d4b/l95 0 2024-09-16T23:49:16.995 INFO:tasks.workunit.client.0.smithi042.stdout:9/969: mkdir d4/d12/d2d/d94/d130 0 2024-09-16T23:49:16.998 INFO:tasks.workunit.client.0.smithi042.stdout:9/970: creat d4/d12/d2d/d94/dc2/d28/d5e/d12e/f131 x:0 0 0 2024-09-16T23:49:17.005 INFO:tasks.workunit.client.0.smithi042.stdout:9/971: rename d4/d12/d16/da9/fe8 to d4/d12/d16/d68/f132 0 2024-09-16T23:49:17.006 INFO:tasks.workunit.client.0.smithi042.stdout:9/972: dread - d4/d10/d4b/dfd/f129 zero size 2024-09-16T23:49:17.010 INFO:tasks.workunit.client.0.smithi042.stdout:9/973: rename d4/d12/d16/df1 to d4/d10/d24/de1/d133 0 2024-09-16T23:49:17.017 INFO:tasks.workunit.client.0.smithi042.stdout:9/974: dwrite d4/d10/f15 [0,4194304] 0 2024-09-16T23:49:17.019 INFO:tasks.workunit.client.0.smithi042.stdout:9/975: fsync d4/d12/d2d/d94/dc2/f70 0 2024-09-16T23:49:17.022 INFO:tasks.workunit.client.0.smithi042.stdout:9/976: creat d4/d12/d77/dd1/f134 x:0 0 0 2024-09-16T23:49:17.029 INFO:tasks.workunit.client.0.smithi042.stdout:9/977: dwrite d4/d12/d60/fd4 [0,4194304] 0 2024-09-16T23:49:17.033 INFO:tasks.workunit.client.0.smithi042.stdout:9/978: creat d4/d12/d2d/d94/dc2/d28/d5e/f135 x:0 0 0 2024-09-16T23:49:17.038 INFO:tasks.workunit.client.0.smithi042.stdout:9/979: rename d4/d12/d2d/d94/f109 to d4/d10/d4b/dfd/d12b/f136 0 2024-09-16T23:49:17.039 INFO:tasks.workunit.client.0.smithi042.stdout:9/980: write d4/d10/d24/de1/fec [792820,105835] 0 2024-09-16T23:49:17.040 INFO:tasks.workunit.client.0.smithi042.stdout:9/981: stat d4/d12/d2d/d51/db9 0 2024-09-16T23:49:17.040 INFO:tasks.workunit.client.0.smithi042.stdout:9/982: stat d4/d12/d2d/dc9/ldb 0 2024-09-16T23:49:17.042 INFO:tasks.workunit.client.0.smithi042.stdout:9/983: symlink d4/d12/d60/d97/l137 0 2024-09-16T23:49:17.199 INFO:tasks.workunit.client.0.smithi042.stdout:9/984: sync 2024-09-16T23:49:17.260 INFO:tasks.workunit.client.0.smithi042.stdout:9/985: dwrite d4/d12/d2d/dc9/dce/fca [0,4194304] 0 2024-09-16T23:49:17.265 INFO:tasks.workunit.client.0.smithi042.stdout:9/986: creat d4/f138 x:0 0 0 2024-09-16T23:49:17.271 INFO:tasks.workunit.client.0.smithi042.stdout:9/987: getdents d4/d12 0 2024-09-16T23:49:17.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:16 smithi042 ceph-mon[27914]: pgmap v17: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 132 MiB/s rd, 338 MiB/s wr, 709 op/s 2024-09-16T23:49:17.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:16 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:17.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:16 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:17.273 INFO:tasks.workunit.client.0.smithi042.stdout:9/988: write d4/d12/d2d/d94/dc2/d28/d5e/d83/f11a [281876,37867] 0 2024-09-16T23:49:17.277 INFO:tasks.workunit.client.0.smithi042.stdout:9/989: truncate d4/d12/d2d/d94/dc2/d28/f87 1498995 0 2024-09-16T23:49:17.279 INFO:tasks.workunit.client.0.smithi042.stdout:9/990: creat d4/d12/d2d/d94/dc2/d59/d12c/f139 x:0 0 0 2024-09-16T23:49:17.281 INFO:tasks.workunit.client.0.smithi042.stdout:9/991: creat d4/d12/d2d/d94/d121/f13a x:0 0 0 2024-09-16T23:49:17.282 INFO:tasks.workunit.client.0.smithi042.stdout:9/992: stat d4/d10/f6b 0 2024-09-16T23:49:17.283 INFO:tasks.workunit.client.0.smithi042.stdout:9/993: chown d4/d12/d2d/dc9/dce/d8e/da1 6510303 1 2024-09-16T23:49:17.285 INFO:tasks.workunit.client.0.smithi042.stdout:9/994: mknod d4/d12/d2d/dc9/dce/d106/c13b 0 2024-09-16T23:49:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:16 smithi043 ceph-mon[35191]: pgmap v17: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 132 MiB/s rd, 338 MiB/s wr, 709 op/s 2024-09-16T23:49:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:16 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:17.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:16 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:17.457 INFO:tasks.workunit.client.0.smithi042.stdout:9/995: sync 2024-09-16T23:49:17.458 INFO:tasks.workunit.client.0.smithi042.stdout:9/996: write d4/d12/d2d/d51/fbe [1375520,114155] 0 2024-09-16T23:49:17.460 INFO:tasks.workunit.client.0.smithi042.stdout:9/997: mkdir d4/d12/d77/dc4/d13c 0 2024-09-16T23:49:17.462 INFO:tasks.workunit.client.0.smithi042.stdout:9/998: fsync d4/d12/d2d/d94/dc2/d41/d92/fcc 0 2024-09-16T23:49:17.464 INFO:tasks.workunit.client.0.smithi042.stdout:9/999: chown d4/d12/d2d/d94/dc2/d28/c7c 568235 1 2024-09-16T23:49:17.469 INFO:tasks.workunit.client.0.smithi042.stderr:+ rm -rf -- ./tmp.0BFvYjQHmP 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: Adjusting osd_memory_target on smithi042 to 1217M 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: pgmap v18: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 85 MiB/s rd, 221 MiB/s wr, 469 op/s 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:49:18.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:18 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: Adjusting osd_memory_target on smithi042 to 1217M 2024-09-16T23:49:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: pgmap v18: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 85 MiB/s rd, 221 MiB/s wr, 469 op/s 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:49:19.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:18 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:19.750 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-09-16T23:49:19.751 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:19 smithi042 ceph-mon[27914]: Reconfiguring daemon prometheus.smithi042 on smithi042 2024-09-16T23:49:19.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-09-16T23:49:19.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:19 smithi043 ceph-mon[35191]: Reconfiguring daemon prometheus.smithi042 on smithi042 2024-09-16T23:49:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 -- 172.21.15.42:0/3022158166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c100e10 msgr2=0x7fc60c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 --2- 172.21.15.42:0/3022158166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c100e10 0x7fc60c101270 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc600009910 tx=0x7fc600009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 -- 172.21.15.42:0/3022158166 shutdown_connections 2024-09-16T23:49:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 --2- 172.21.15.42:0/3022158166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c100e10 0x7fc60c101270 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 --2- 172.21.15.42:0/3022158166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c0ffc10 0x7fc60c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 -- 172.21.15.42:0/3022158166 >> 172.21.15.42:0/3022158166 conn(0x7fc60c0fb190 msgr2=0x7fc60c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 -- 172.21.15.42:0/3022158166 shutdown_connections 2024-09-16T23:49:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.361+0000 7fc6124a7700 1 -- 172.21.15.42:0/3022158166 wait complete. 2024-09-16T23:49:20.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.362+0000 7fc6124a7700 1 Processor -- start 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.362+0000 7fc6124a7700 1 -- start start 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc6124a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c0ffc10 0x7fc60c0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc6124a7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c100e10 0x7fc60c19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc6124a7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c19b7d0 con 0x7fc60c0ffc10 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc6124a7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c19b910 con 0x7fc60c100e10 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc60bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c0ffc10 0x7fc60c0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc60b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c100e10 0x7fc60c19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc60bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c0ffc10 0x7fc60c0690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39058/0 (socket says 172.21.15.42:39058) 2024-09-16T23:49:20.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.363+0000 7fc60bfff700 1 -- 172.21.15.42:0/2552772853 learned_addr learned my addr 172.21.15.42:0/2552772853 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc60bfff700 1 -- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c100e10 msgr2=0x7fc60c19b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc60bfff700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c100e10 0x7fc60c19b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc60bfff700 1 -- 172.21.15.42:0/2552772853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000095f0 con 0x7fc60c0ffc10 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc60b7fe700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c100e10 0x7fc60c19b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc60bfff700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c0ffc10 0x7fc60c0690e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00e8f0 tx=0x7fc5fc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc6097fa700 1 -- 172.21.15.42:0/2552772853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc009790 con 0x7fc60c0ffc10 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c195030 con 0x7fc60c0ffc10 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc6097fa700 1 -- 172.21.15.42:0/2552772853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc5fc00c810 con 0x7fc60c0ffc10 2024-09-16T23:49:20.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc6097fa700 1 -- 172.21.15.42:0/2552772853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc003d20 con 0x7fc60c0ffc10 2024-09-16T23:49:20.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.364+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c195580 con 0x7fc60c0ffc10 2024-09-16T23:49:20.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.365+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c108700 con 0x7fc60c0ffc10 2024-09-16T23:49:20.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.366+0000 7fc6097fa700 1 -- 172.21.15.42:0/2552772853 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 91460+0+0 (secure 0 0 0) 0x7fc5fc0041f0 con 0x7fc60c0ffc10 2024-09-16T23:49:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.366+0000 7fc6097fa700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc5f406d530 0x7fc5f406f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.366+0000 7fc6097fa700 1 -- 172.21.15.42:0/2552772853 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc5fc014070 con 0x7fc60c0ffc10 2024-09-16T23:49:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.366+0000 7fc60b7fe700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc5f406d530 0x7fc5f406f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:20.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.367+0000 7fc60b7fe700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc5f406d530 0x7fc5f406f9f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc60c069660 tx=0x7fc600005fd0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:20.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.372+0000 7fc6097fa700 1 -- 172.21.15.42:0/2552772853 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fc5fc05a250 con 0x7fc60c0ffc10 2024-09-16T23:49:20.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.709+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc60c105760 con 0x7fc5f406d530 2024-09-16T23:49:20.707 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:20 smithi042 ceph-mon[27914]: pgmap v19: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 92 MiB/s rd, 244 MiB/s wr, 577 op/s 2024-09-16T23:49:20.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.710+0000 7fc6097fa700 1 -- 172.21.15.42:0/2552772853 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fc60c105760 con 0x7fc5f406d530 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc5f406d530 msgr2=0x7fc5f406f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc5f406d530 0x7fc5f406f9f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc60c069660 tx=0x7fc600005fd0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c0ffc10 msgr2=0x7fc60c0690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c0ffc10 0x7fc60c0690e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00e8f0 tx=0x7fc5fc00ec00 comp rx=0 tx=0).stop 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 shutdown_connections 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc5f406d530 0x7fc5f406f9f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc60c0ffc10 0x7fc60c0690e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 --2- 172.21.15.42:0/2552772853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c100e10 0x7fc60c19b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 >> 172.21.15.42:0/2552772853 conn(0x7fc60c0fb190 msgr2=0x7fc60c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 shutdown_connections 2024-09-16T23:49:20.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.713+0000 7fc6124a7700 1 -- 172.21.15.42:0/2552772853 wait complete. 2024-09-16T23:49:20.722 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:49:20.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 -- 172.21.15.42:0/4249211469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c100cb0 msgr2=0x7fc25c1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 --2- 172.21.15.42:0/4249211469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c100cb0 0x7fc25c1010d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc25400b0a0 tx=0x7fc25400b3b0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 -- 172.21.15.42:0/4249211469 shutdown_connections 2024-09-16T23:49:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 --2- 172.21.15.42:0/4249211469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc25c101eb0 0x7fc25c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 --2- 172.21.15.42:0/4249211469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c100cb0 0x7fc25c1010d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 -- 172.21.15.42:0/4249211469 >> 172.21.15.42:0/4249211469 conn(0x7fc25c0fc230 msgr2=0x7fc25c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 -- 172.21.15.42:0/4249211469 shutdown_connections 2024-09-16T23:49:20.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 -- 172.21.15.42:0/4249211469 wait complete. 2024-09-16T23:49:20.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.842+0000 7fc261eb2700 1 Processor -- start 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc261eb2700 1 -- start start 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc261eb2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc25c100cb0 0x7fc25c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc261eb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c101eb0 0x7fc25c196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc261eb2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c197080 con 0x7fc25c101eb0 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc261eb2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c1971c0 con 0x7fc25c100cb0 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc25affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c101eb0 0x7fc25c196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc25affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c101eb0 0x7fc25c196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39088/0 (socket says 172.21.15.42:39088) 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc25b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc25c100cb0 0x7fc25c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.843+0000 7fc25affd700 1 -- 172.21.15.42:0/1133284624 learned_addr learned my addr 172.21.15.42:0/1133284624 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc25affd700 1 -- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc25c100cb0 msgr2=0x7fc25c196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:20.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc25affd700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc25c100cb0 0x7fc25c196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc25affd700 1 -- 172.21.15.42:0/1133284624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254009d00 con 0x7fc25c101eb0 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc25b7fe700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc25c100cb0 0x7fc25c196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc25affd700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c101eb0 0x7fc25c196a60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00e920 tx=0x7fc24c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc258ff9700 1 -- 172.21.15.42:0/1133284624 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc24c00c9d0 con 0x7fc25c101eb0 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc258ff9700 1 -- 172.21.15.42:0/1133284624 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc24c010430 con 0x7fc25c101eb0 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c19bc70 con 0x7fc25c101eb0 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc258ff9700 1 -- 172.21.15.42:0/1133284624 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc24c018650 con 0x7fc25c101eb0 2024-09-16T23:49:20.843 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.844+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c19c1c0 con 0x7fc25c101eb0 2024-09-16T23:49:20.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.845+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c109810 con 0x7fc25c101eb0 2024-09-16T23:49:20.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.846+0000 7fc258ff9700 1 -- 172.21.15.42:0/1133284624 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 91460+0+0 (secure 0 0 0) 0x7fc24c00cb30 con 0x7fc25c101eb0 2024-09-16T23:49:20.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.847+0000 7fc258ff9700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc24806d520 0x7fc24806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:20.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.847+0000 7fc258ff9700 1 -- 172.21.15.42:0/1133284624 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc24c014070 con 0x7fc25c101eb0 2024-09-16T23:49:20.844 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.847+0000 7fc25b7fe700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc24806d520 0x7fc24806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:20.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.848+0000 7fc25b7fe700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc24806d520 0x7fc24806f9e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc254009fd0 tx=0x7fc254006000 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:20.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:20.853+0000 7fc258ff9700 1 -- 172.21.15.42:0/1133284624 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fc24c0539f0 con 0x7fc25c101eb0 2024-09-16T23:49:20.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:20 smithi043 ceph-mon[35191]: pgmap v19: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 92 MiB/s rd, 244 MiB/s wr, 577 op/s 2024-09-16T23:49:21.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.213+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc25c106800 con 0x7fc24806d520 2024-09-16T23:49:21.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.215+0000 7fc258ff9700 1 -- 172.21.15.42:0/1133284624 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fc25c106800 con 0x7fc24806d520 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc24806d520 msgr2=0x7fc24806f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc24806d520 0x7fc24806f9e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc254009fd0 tx=0x7fc254006000 comp rx=0 tx=0).stop 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c101eb0 msgr2=0x7fc25c196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c101eb0 0x7fc25c196a60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc24c00e920 tx=0x7fc24c00ec30 comp rx=0 tx=0).stop 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 shutdown_connections 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7fc24806d520 0x7fc24806f9e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc25c100cb0 0x7fc25c196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 --2- 172.21.15.42:0/1133284624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc25c101eb0 0x7fc25c196a60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 >> 172.21.15.42:0/1133284624 conn(0x7fc25c0fc230 msgr2=0x7fc25c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 shutdown_connections 2024-09-16T23:49:21.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.218+0000 7fc261eb2700 1 -- 172.21.15.42:0/1133284624 wait complete. 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 -- 172.21.15.42:0/4092745143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420101740 msgr2=0x7f9420103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 --2- 172.21.15.42:0/4092745143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420101740 0x7f9420103b30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f941c009910 tx=0x7f941c009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 -- 172.21.15.42:0/4092745143 shutdown_connections 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 --2- 172.21.15.42:0/4092745143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9420104070 0x7f9420106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 --2- 172.21.15.42:0/4092745143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420101740 0x7f9420103b30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 -- 172.21.15.42:0/4092745143 >> 172.21.15.42:0/4092745143 conn(0x7f94200fb150 msgr2=0x7f94200fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 -- 172.21.15.42:0/4092745143 shutdown_connections 2024-09-16T23:49:21.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.370+0000 7f9427b28700 1 -- 172.21.15.42:0/4092745143 wait complete. 2024-09-16T23:49:21.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.371+0000 7f9427b28700 1 Processor -- start 2024-09-16T23:49:21.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.371+0000 7f9427b28700 1 -- start start 2024-09-16T23:49:21.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.371+0000 7f9427b28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9420101740 0x7f94201010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:21.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.371+0000 7f9427b28700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420104070 0x7f94200ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:21.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.371+0000 7f9427b28700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942019a0f0 con 0x7f9420101740 2024-09-16T23:49:21.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.371+0000 7f9427b28700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942019a230 con 0x7f9420104070 2024-09-16T23:49:21.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.371+0000 7f94250c3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420104070 0x7f94200ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f94258c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9420101740 0x7f94201010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f94250c3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420104070 0x7f94200ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:52566/0 (socket says 172.21.15.42:52566) 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f94250c3700 1 -- 172.21.15.42:0/749444276 learned_addr learned my addr 172.21.15.42:0/749444276 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f94250c3700 1 -- 172.21.15.42:0/749444276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9420101740 msgr2=0x7f94201010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f94250c3700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9420101740 0x7f94201010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f94250c3700 1 -- 172.21.15.42:0/749444276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0095f0 con 0x7f9420104070 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f94250c3700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420104070 0x7f94200ff6f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f941000e940 tx=0x7f941000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f9416ffd700 1 -- 172.21.15.42:0/749444276 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f941000cba0 con 0x7f9420104070 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94200ffc30 con 0x7f9420104070 2024-09-16T23:49:21.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f9416ffd700 1 -- 172.21.15.42:0/749444276 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f941000ee90 con 0x7f9420104070 2024-09-16T23:49:21.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f9416ffd700 1 -- 172.21.15.42:0/749444276 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9410005650 con 0x7f9420104070 2024-09-16T23:49:21.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.372+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9420100150 con 0x7f9420104070 2024-09-16T23:49:21.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.374+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94201003d0 con 0x7f9420104070 2024-09-16T23:49:21.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.374+0000 7f9416ffd700 1 -- 172.21.15.42:0/749444276 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 91460+0+0 (secure 0 0 0) 0x7f9410004750 con 0x7f9420104070 2024-09-16T23:49:21.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.375+0000 7f9416ffd700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f940c06d530 0x7f940c06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:21.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.375+0000 7f9416ffd700 1 -- 172.21.15.42:0/749444276 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9410014070 con 0x7f9420104070 2024-09-16T23:49:21.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.375+0000 7f94258c4700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f940c06d530 0x7f940c06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:21.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.375+0000 7f94258c4700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f940c06d530 0x7f940c06f9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f941c009fd0 tx=0x7f941c005940 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:21.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.381+0000 7f9416ffd700 1 -- 172.21.15.42:0/749444276 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f941005a280 con 0x7f9420104070 2024-09-16T23:49:21.619 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:21 smithi042 ceph-mon[27914]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:21.620 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:21 smithi042 ceph-mon[27914]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:21.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.696+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9420104ce0 con 0x7f940c06d530 2024-09-16T23:49:21.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.709+0000 7f9416ffd700 1 -- 172.21.15.42:0/749444276 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f9420104ce0 con 0x7f940c06d530 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (7m) 4s ago 8m 25.4M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (9m) 4s ago 9m 7675k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (7m) 17s ago 7m 7570k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (8m) 4s ago 8m 6660k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (7m) 17s ago 7m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (7m) 4s ago 8m 83.0M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:49:21.707 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (3m) 4s ago 3m 144M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (3m) 4s ago 3m 13.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (3m) 17s ago 3m 132M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (3m) 17s ago 2m 14.7M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:9283,8765,8443 running (10m) 4s ago 10m 191M - 18.2.1 5be31c24972a cd8abd4f468e 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (60s) 17s ago 7m 483M - 19.1.1-330-gc2ddf972 ebb0615149de 50141468f451 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (10m) 4s ago 10m 49.3M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (7m) 17s ago 7m 43.2M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:49:21.708 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (8m) 4s ago 8m 16.7M - 1.5.0 0da6a335fe13 7c84e9bf6c1c 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (7m) 17s ago 7m 16.6M - 1.5.0 0da6a335fe13 82d41c3cc1ee 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 4s ago 6m 132M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (6m) 4s ago 6m 157M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (5m) 4s ago 5m 138M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (5m) 17s ago 5m 164M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (5m) 17s ago 5m 158M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (4m) 17s ago 4m 178M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:49:21.709 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (6m) 4s ago 8m 60.7M - 2.43.0 a07b618ecd1d 8ebaffdf2443 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f940c06d530 msgr2=0x7f940c06f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f940c06d530 0x7f940c06f9f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f941c009fd0 tx=0x7f941c005940 comp rx=0 tx=0).stop 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420104070 msgr2=0x7f94200ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420104070 0x7f94200ff6f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f941000e940 tx=0x7f941000ec50 comp rx=0 tx=0).stop 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 shutdown_connections 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f940c06d530 0x7f940c06f9f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9420101740 0x7f94201010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 --2- 172.21.15.42:0/749444276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9420104070 0x7f94200ff6f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 >> 172.21.15.42:0/749444276 conn(0x7f94200fb150 msgr2=0x7f94200fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 shutdown_connections 2024-09-16T23:49:21.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.712+0000 7f9427b28700 1 -- 172.21.15.42:0/749444276 wait complete. 2024-09-16T23:49:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 -- 172.21.15.42:0/3890369485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 msgr2=0x7f4d6c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 --2- 172.21.15.42:0/3890369485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 0x7f4d6c101270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009910 tx=0x7f4d5c009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 -- 172.21.15.42:0/3890369485 shutdown_connections 2024-09-16T23:49:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 --2- 172.21.15.42:0/3890369485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 0x7f4d6c101270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 --2- 172.21.15.42:0/3890369485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d6c0ffc10 0x7f4d6c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 -- 172.21.15.42:0/3890369485 >> 172.21.15.42:0/3890369485 conn(0x7f4d6c0fb190 msgr2=0x7f4d6c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 -- 172.21.15.42:0/3890369485 shutdown_connections 2024-09-16T23:49:21.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.862+0000 7f4d7327f700 1 -- 172.21.15.42:0/3890369485 wait complete. 2024-09-16T23:49:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.863+0000 7f4d7327f700 1 Processor -- start 2024-09-16T23:49:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.863+0000 7f4d7327f700 1 -- start start 2024-09-16T23:49:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.863+0000 7f4d7327f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d6c0ffc10 0x7f4d6c196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.863+0000 7f4d7327f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 0x7f4d6c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:21.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.863+0000 7f4d7327f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c196fe0 con 0x7f4d6c0ffc10 2024-09-16T23:49:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.863+0000 7f4d7327f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c197120 con 0x7f4d6c100e10 2024-09-16T23:49:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7101b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d6c0ffc10 0x7f4d6c196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7101b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d6c0ffc10 0x7f4d6c196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39130/0 (socket says 172.21.15.42:39130) 2024-09-16T23:49:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7101b700 1 -- 172.21.15.42:0/319393531 learned_addr learned my addr 172.21.15.42:0/319393531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7081a700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 0x7f4d6c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7081a700 1 -- 172.21.15.42:0/319393531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d6c0ffc10 msgr2=0x7f4d6c196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7081a700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d6c0ffc10 0x7f4d6c196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7081a700 1 -- 172.21.15.42:0/319393531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c0095f0 con 0x7f4d6c100e10 2024-09-16T23:49:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.864+0000 7f4d7081a700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 0x7f4d6c1969c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009910 tx=0x7f4d5c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.865+0000 7f4d627fc700 1 -- 172.21.15.42:0/319393531 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d5c01d070 con 0x7f4d6c100e10 2024-09-16T23:49:21.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.865+0000 7f4d627fc700 1 -- 172.21.15.42:0/319393531 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4d5c005dc0 con 0x7f4d6c100e10 2024-09-16T23:49:21.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.865+0000 7f4d627fc700 1 -- 172.21.15.42:0/319393531 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d5c017630 con 0x7f4d6c100e10 2024-09-16T23:49:21.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.865+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c19bb70 con 0x7f4d6c100e10 2024-09-16T23:49:21.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.865+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c19c060 con 0x7f4d6c100e10 2024-09-16T23:49:21.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.866+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c108700 con 0x7f4d6c100e10 2024-09-16T23:49:21.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.867+0000 7f4d627fc700 1 -- 172.21.15.42:0/319393531 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 25) v1 ==== 91460+0+0 (secure 0 0 0) 0x7f4d5c0038a0 con 0x7f4d6c100e10 2024-09-16T23:49:21.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.867+0000 7f4d627fc700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f4d5806d530 0x7f4d5806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:21.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.867+0000 7f4d627fc700 1 -- 172.21.15.42:0/319393531 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4d5c08e0d0 con 0x7f4d6c100e10 2024-09-16T23:49:21.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.867+0000 7f4d7101b700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f4d5806d530 0x7f4d5806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:21.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.868+0000 7f4d7101b700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f4d5806d530 0x7f4d5806f9f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005950 tx=0x7f4d68009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:21.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:21.872+0000 7f4d627fc700 1 -- 172.21.15.42:0/319393531 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f4d5c05b280 con 0x7f4d6c100e10 2024-09-16T23:49:21.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:21 smithi043 ceph-mon[35191]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:21.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:21 smithi043 ceph-mon[35191]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:22.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.381+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4d6c1088f0 con 0x7f4d6c100e10 2024-09-16T23:49:22.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.382+0000 7f4d627fc700 1 -- 172.21.15.42:0/319393531 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7f4d5c05ae10 con 0x7f4d6c100e10 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:49:22.380 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:49:22.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:22.381 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:49:22.381 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-09-16T23:49:22.381 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:49:22.381 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:49:22.381 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:49:22.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.385+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f4d5806d530 msgr2=0x7f4d5806f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:22.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.385+0000 7f4d7327f700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f4d5806d530 0x7f4d5806f9f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005950 tx=0x7f4d68009260 comp rx=0 tx=0).stop 2024-09-16T23:49:22.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.385+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 msgr2=0x7f4d6c1969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:22.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.385+0000 7f4d7327f700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 0x7f4d6c1969c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009910 tx=0x7f4d5c00f690 comp rx=0 tx=0).stop 2024-09-16T23:49:22.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.386+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 shutdown_connections 2024-09-16T23:49:22.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.386+0000 7f4d7327f700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f4d5806d530 0x7f4d5806f9f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.386+0000 7f4d7327f700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d6c0ffc10 0x7f4d6c196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.386+0000 7f4d7327f700 1 --2- 172.21.15.42:0/319393531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d6c100e10 0x7f4d6c1969c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.386+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 >> 172.21.15.42:0/319393531 conn(0x7f4d6c0fb190 msgr2=0x7f4d6c068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:22.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.386+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 shutdown_connections 2024-09-16T23:49:22.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.386+0000 7f4d7327f700 1 -- 172.21.15.42:0/319393531 wait complete. 2024-09-16T23:49:22.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.559+0000 7f64b71a5700 1 -- 172.21.15.42:0/1876522718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00ff5f0 msgr2=0x7f64b00ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:22.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.559+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1876522718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00ff5f0 0x7f64b00ffa10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009910 tx=0x7f64a4009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:22.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.559+0000 7f64b71a5700 1 -- 172.21.15.42:0/1876522718 shutdown_connections 2024-09-16T23:49:22.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.559+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1876522718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64b00fff50 0x7f64b01003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.559+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1876522718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00ff5f0 0x7f64b00ffa10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.559+0000 7f64b71a5700 1 -- 172.21.15.42:0/1876522718 >> 172.21.15.42:0/1876522718 conn(0x7f64b00fb190 msgr2=0x7f64b00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:22.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.560+0000 7f64b71a5700 1 -- 172.21.15.42:0/1876522718 shutdown_connections 2024-09-16T23:49:22.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.560+0000 7f64b71a5700 1 -- 172.21.15.42:0/1876522718 wait complete. 2024-09-16T23:49:22.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.560+0000 7f64b71a5700 1 Processor -- start 2024-09-16T23:49:22.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.560+0000 7f64b71a5700 1 -- start start 2024-09-16T23:49:22.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.560+0000 7f64b71a5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64b00ff5f0 0x7f64b0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:22.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.560+0000 7f64b71a5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00fff50 0x7f64b0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64b71a5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b01991f0 con 0x7f64b00fff50 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64b71a5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b0199330 con 0x7f64b00ff5f0 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00fff50 0x7f64b0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00fff50 0x7f64b0198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39146/0 (socket says 172.21.15.42:39146) 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64affff700 1 -- 172.21.15.42:0/1398983000 learned_addr learned my addr 172.21.15.42:0/1398983000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64b4f41700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64b00ff5f0 0x7f64b0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64affff700 1 -- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64b00ff5f0 msgr2=0x7f64b0198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:22.562 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64affff700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64b00ff5f0 0x7f64b0198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64affff700 1 -- 172.21.15.42:0/1398983000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a40095f0 con 0x7f64b00fff50 2024-09-16T23:49:22.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64b4f41700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64b00ff5f0 0x7f64b0198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:49:22.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.561+0000 7f64affff700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00fff50 0x7f64b0198bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f649c00e920 tx=0x7f649c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:22.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.562+0000 7f64adffb700 1 -- 172.21.15.42:0/1398983000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649c00c9d0 con 0x7f64b00fff50 2024-09-16T23:49:22.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.562+0000 7f64adffb700 1 -- 172.21.15.42:0/1398983000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f649c010430 con 0x7f64b00fff50 2024-09-16T23:49:22.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.562+0000 7f64adffb700 1 -- 172.21.15.42:0/1398983000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649c018650 con 0x7f64b00fff50 2024-09-16T23:49:22.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.562+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b019dde0 con 0x7f64b00fff50 2024-09-16T23:49:22.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.562+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b019e300 con 0x7f64b00fff50 2024-09-16T23:49:22.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.564+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b010b9f0 con 0x7f64b00fff50 2024-09-16T23:49:22.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.564+0000 7f64adffb700 1 -- 172.21.15.42:0/1398983000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 91460+0+0 (secure 0 0 0) 0x7f649c0105a0 con 0x7f64b00fff50 2024-09-16T23:49:22.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.565+0000 7f64adffb700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f64a006d530 0x7f64a006f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:22.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.565+0000 7f64adffb700 1 -- 172.21.15.42:0/1398983000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f649c014070 con 0x7f64b00fff50 2024-09-16T23:49:22.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.566+0000 7f64b4f41700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f64a006d530 0x7f64a006f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:22.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.567+0000 7f64b4f41700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f64a006d530 0x7f64a006f9f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009fd0 tx=0x7f64a4005940 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:22.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.572+0000 7f64adffb700 1 -- 172.21.15.42:0/1398983000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f649c0539f0 con 0x7f64b00fff50 2024-09-16T23:49:22.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:22 smithi042 ceph-mon[27914]: pgmap v20: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 44 MiB/s rd, 127 MiB/s wr, 326 op/s 2024-09-16T23:49:22.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:22 smithi042 ceph-mon[27914]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:22.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:22 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/319393531' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:49:22.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.926+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64b0109340 con 0x7f64a006d530 2024-09-16T23:49:22.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.927+0000 7f64adffb700 1 -- 172.21.15.42:0/1398983000 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f64b0109340 con 0x7f64a006d530 2024-09-16T23:49:22.924 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:49:22.924 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:49:22.924 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:49:22.925 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:49:22.925 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-09-16T23:49:22.925 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "1/2 daemons upgraded", 2024-09-16T23:49:22.925 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "", 2024-09-16T23:49:22.925 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:49:22.925 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:49:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f64a006d530 msgr2=0x7f64a006f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f64a006d530 0x7f64a006f9f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f64a4009fd0 tx=0x7f64a4005940 comp rx=0 tx=0).stop 2024-09-16T23:49:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00fff50 msgr2=0x7f64b0198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00fff50 0x7f64b0198bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f649c00e920 tx=0x7f649c00ec30 comp rx=0 tx=0).stop 2024-09-16T23:49:22.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 shutdown_connections 2024-09-16T23:49:22.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f64a006d530 0x7f64a006f9f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64b00ff5f0 0x7f64b0198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 --2- 172.21.15.42:0/1398983000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64b00fff50 0x7f64b0198bd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:22.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.929+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 >> 172.21.15.42:0/1398983000 conn(0x7f64b00fb190 msgr2=0x7f64b0107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:22.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.930+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 shutdown_connections 2024-09-16T23:49:22.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:22.930+0000 7f64b71a5700 1 -- 172.21.15.42:0/1398983000 wait complete. 2024-09-16T23:49:22.975 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:22 smithi043 ceph-mon[35191]: pgmap v20: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 44 MiB/s rd, 127 MiB/s wr, 326 op/s 2024-09-16T23:49:22.975 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:22 smithi043 ceph-mon[35191]: from='client.24405 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:22.975 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:22 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/319393531' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:49:23.953 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:23 smithi042 ceph-mon[27914]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:24.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:23 smithi043 ceph-mon[35191]: from='client.14642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:24.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:24 smithi042 ceph-mon[27914]: pgmap v21: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 44 MiB/s rd, 127 MiB/s wr, 326 op/s 2024-09-16T23:49:25.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:24 smithi043 ceph-mon[35191]: pgmap v21: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 44 MiB/s rd, 127 MiB/s wr, 326 op/s 2024-09-16T23:49:25.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:25 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:25.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:25 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:25.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:25 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:49:25.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:25 smithi042 ceph-mon[27914]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:49:25.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:25 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:25.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:25 smithi042 ceph-mon[27914]: Upgrade: Need to upgrade myself (mgr.smithi043.xveyou) 2024-09-16T23:49:25.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:25 smithi042 ceph-mon[27914]: Upgrade: Need to upgrade myself (mgr.smithi043.xveyou) 2024-09-16T23:49:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:25 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:25 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:25 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:49:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:25 smithi043 ceph-mon[35191]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:49:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:25 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:25 smithi043 ceph-mon[35191]: Upgrade: Need to upgrade myself (mgr.smithi043.xveyou) 2024-09-16T23:49:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:25 smithi043 ceph-mon[35191]: Upgrade: Need to upgrade myself (mgr.smithi043.xveyou) 2024-09-16T23:49:26.684 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T23:49:26.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-16T23:49:27.019 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:26 smithi042 ceph-mon[27914]: pgmap v22: 65 pgs: 65 active+clean; 2.3 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 44 MiB/s rd, 128 MiB/s wr, 434 op/s 2024-09-16T23:49:27.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:26 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:27.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:26 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:49:27.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:26 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:49:27.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:26 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:49:27.020 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:26 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:49:27.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:26 smithi043 ceph-mon[35191]: pgmap v22: 65 pgs: 65 active+clean; 2.3 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 44 MiB/s rd, 128 MiB/s wr, 434 op/s 2024-09-16T23:49:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:26 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:26 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:49:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:26 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:49:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:26 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:49:27.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:26 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:49:27.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:27 smithi042 ceph-mon[27914]: Upgrade: Updating mgr.smithi042.zrkumz 2024-09-16T23:49:27.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:27 smithi042 ceph-mon[27914]: Deploying daemon mgr.smithi042.zrkumz on smithi042 2024-09-16T23:49:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:27 smithi043 ceph-mon[35191]: Upgrade: Updating mgr.smithi042.zrkumz 2024-09-16T23:49:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:27 smithi043 ceph-mon[35191]: Deploying daemon mgr.smithi042.zrkumz on smithi042 2024-09-16T23:49:28.867 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:28 smithi042 ceph-mon[27914]: pgmap v23: 65 pgs: 65 active+clean; 2.3 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 6.8 MiB/s rd, 24 MiB/s wr, 215 op/s 2024-09-16T23:49:29.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:28 smithi043 ceph-mon[35191]: pgmap v23: 65 pgs: 65 active+clean; 2.3 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 6.8 MiB/s rd, 24 MiB/s wr, 215 op/s 2024-09-16T23:49:31.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:30 smithi042 ceph-mon[27914]: pgmap v24: 65 pgs: 65 active+clean; 895 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 320 op/s 2024-09-16T23:49:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:30 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:30 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:30 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:31.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:30 smithi043 ceph-mon[35191]: pgmap v24: 65 pgs: 65 active+clean; 895 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 6.8 MiB/s rd, 25 MiB/s wr, 320 op/s 2024-09-16T23:49:31.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:30 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:31.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:30 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:31.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:30 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:31.265 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T23:49:31.266 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T23:49:33.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:32 smithi042 ceph-mon[27914]: pgmap v25: 65 pgs: 65 active+clean; 895 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 213 op/s 2024-09-16T23:49:33.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:32 smithi043 ceph-mon[35191]: pgmap v25: 65 pgs: 65 active+clean; 895 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 213 op/s 2024-09-16T23:49:34.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:33 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:34.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:33 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:34.894 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:34 smithi042 ceph-mon[27914]: pgmap v26: 65 pgs: 65 active+clean; 895 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 213 op/s 2024-09-16T23:49:35.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:34 smithi043 ceph-mon[35191]: pgmap v26: 65 pgs: 65 active+clean; 895 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 213 op/s 2024-09-16T23:49:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:36 smithi042 ceph-mon[27914]: pgmap v27: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-09-16T23:49:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:36 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:36 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:36 smithi043 ceph-mon[35191]: pgmap v27: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 2.4 MiB/s wr, 298 op/s 2024-09-16T23:49:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:36 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:36 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: Standby manager daemon smithi042.zrkumz restarted 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: Standby manager daemon smithi042.zrkumz started 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/crt"}]: dispatch 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/key"}]: dispatch 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:37 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:38.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: Standby manager daemon smithi042.zrkumz restarted 2024-09-16T23:49:38.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: Standby manager daemon smithi042.zrkumz started 2024-09-16T23:49:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/crt"}]: dispatch 2024-09-16T23:49:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:49:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi042.zrkumz/key"}]: dispatch 2024-09-16T23:49:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:49:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:37 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:38 smithi042 ceph-mon[27914]: pgmap v28: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-09-16T23:49:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:38 smithi042 ceph-mon[27914]: mgrmap e26: smithi043.xveyou(active, since 50s), standbys: smithi042.zrkumz 2024-09-16T23:49:39.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:38 smithi043 ceph-mon[35191]: pgmap v28: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-09-16T23:49:39.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:38 smithi043 ceph-mon[35191]: mgrmap e26: smithi043.xveyou(active, since 50s), standbys: smithi042.zrkumz 2024-09-16T23:49:41.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:40 smithi042 ceph-mon[27914]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-09-16T23:49:41.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:40 smithi043 ceph-mon[35191]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-09-16T23:49:42.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:42 smithi043 ceph-mon[35191]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-09-16T23:49:43.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:42 smithi042 ceph-mon[27914]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-09-16T23:49:45.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:44 smithi042 ceph-mon[27914]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-09-16T23:49:45.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:44 smithi043 ceph-mon[35191]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-09-16T23:49:47.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:46 smithi042 ceph-mon[27914]: pgmap v32: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-16T23:49:47.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:46 smithi043 ceph-mon[35191]: pgmap v32: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 61 KiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-16T23:49:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:48 smithi043 ceph-mon[35191]: pgmap v33: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-09-16T23:49:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:48 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:49.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:48 smithi042 ceph-mon[27914]: pgmap v33: 65 pgs: 65 active+clean; 310 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-09-16T23:49:49.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:48 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:50 smithi042 ceph-mon[27914]: pgmap v34: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64 KiB/s rd, 3.1 MiB/s wr, 189 op/s 2024-09-16T23:49:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:50 smithi043 ceph-mon[35191]: pgmap v34: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 64 KiB/s rd, 3.1 MiB/s wr, 189 op/s 2024-09-16T23:49:53.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:52 smithi042 ceph-mon[27914]: pgmap v35: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.0 MiB/s wr, 127 op/s 2024-09-16T23:49:53.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2592230231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e34100e10 msgr2=0x7f9e34101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2592230231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e34100e10 0x7f9e34101270 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009910 tx=0x7f9e24009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:53.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2592230231 shutdown_connections 2024-09-16T23:49:53.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2592230231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e34100e10 0x7f9e34101270 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2592230231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e340ffc10 0x7f9e34100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2592230231 >> 172.21.15.42:0/2592230231 conn(0x7f9e340fb190 msgr2=0x7f9e340fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:53.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2592230231 shutdown_connections 2024-09-16T23:49:53.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.053+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2592230231 wait complete. 2024-09-16T23:49:53.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e3bbca700 1 Processor -- start 2024-09-16T23:49:53.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e3bbca700 1 -- start start 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e3bbca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e340ffc10 0x7f9e34196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e3bbca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e34100e10 0x7f9e341969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e3bbca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34196fe0 con 0x7f9e340ffc10 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e3bbca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e34197120 con 0x7f9e34100e10 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e39966700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e340ffc10 0x7f9e34196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e39966700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e340ffc10 0x7f9e34196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35606/0 (socket says 172.21.15.42:35606) 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e39966700 1 -- 172.21.15.42:0/2834648147 learned_addr learned my addr 172.21.15.42:0/2834648147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:53.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.054+0000 7f9e39165700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e34100e10 0x7f9e341969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e39966700 1 -- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e34100e10 msgr2=0x7f9e341969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e39966700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e34100e10 0x7f9e341969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e39966700 1 -- 172.21.15.42:0/2834648147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240095f0 con 0x7f9e340ffc10 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e39966700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e340ffc10 0x7f9e34196480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d900 tx=0x7f9e3000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e2affd700 1 -- 172.21.15.42:0/2834648147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e3000b920 con 0x7f9e340ffc10 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e2affd700 1 -- 172.21.15.42:0/2834648147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9e30010460 con 0x7f9e340ffc10 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3419bbd0 con 0x7f9e340ffc10 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.055+0000 7f9e2affd700 1 -- 172.21.15.42:0/2834648147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e3000f590 con 0x7f9e340ffc10 2024-09-16T23:49:53.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.056+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3419c120 con 0x7f9e340ffc10 2024-09-16T23:49:53.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.057+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e34108700 con 0x7f9e340ffc10 2024-09-16T23:49:53.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.058+0000 7f9e2affd700 1 -- 172.21.15.42:0/2834648147 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 92506+0+0 (secure 0 0 0) 0x7f9e300105d0 con 0x7f9e340ffc10 2024-09-16T23:49:53.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.058+0000 7f9e2affd700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f9e2006e440 0x7f9e20070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:53.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.058+0000 7f9e2affd700 1 -- 172.21.15.42:0/2834648147 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9e3008dd30 con 0x7f9e340ffc10 2024-09-16T23:49:53.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.059+0000 7f9e39165700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f9e2006e440 0x7f9e20070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:53.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.059+0000 7f9e39165700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f9e2006e440 0x7f9e20070900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b5c0 tx=0x7f9e24005ee0 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:53.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.064+0000 7f9e2affd700 1 -- 172.21.15.42:0/2834648147 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f9e3005aee0 con 0x7f9e340ffc10 2024-09-16T23:49:53.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:52 smithi043 ceph-mon[35191]: pgmap v35: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.0 MiB/s wr, 127 op/s 2024-09-16T23:49:53.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.376+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e34068c90 con 0x7f9e2006e440 2024-09-16T23:49:53.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.378+0000 7f9e2affd700 1 -- 172.21.15.42:0/2834648147 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f9e34068c90 con 0x7f9e2006e440 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f9e2006e440 msgr2=0x7f9e20070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f9e2006e440 0x7f9e20070900 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400b5c0 tx=0x7f9e24005ee0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e340ffc10 msgr2=0x7f9e34196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e340ffc10 0x7f9e34196480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000d900 tx=0x7f9e3000dc10 comp rx=0 tx=0).stop 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 shutdown_connections 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f9e2006e440 0x7f9e20070900 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e340ffc10 0x7f9e34196480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 --2- 172.21.15.42:0/2834648147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e34100e10 0x7f9e341969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 >> 172.21.15.42:0/2834648147 conn(0x7f9e340fb190 msgr2=0x7f9e34068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:53.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 shutdown_connections 2024-09-16T23:49:53.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.380+0000 7f9e3bbca700 1 -- 172.21.15.42:0/2834648147 wait complete. 2024-09-16T23:49:53.390 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:49:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.544+0000 7facd032d700 1 -- 172.21.15.42:0/1969630469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 msgr2=0x7facc8107d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.544+0000 7facd032d700 1 --2- 172.21.15.42:0/1969630469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc8107d90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7facb8009910 tx=0x7facb8009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.546+0000 7facd032d700 1 -- 172.21.15.42:0/1969630469 shutdown_connections 2024-09-16T23:49:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.546+0000 7facd032d700 1 --2- 172.21.15.42:0/1969630469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 0x7facc81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.546+0000 7facd032d700 1 --2- 172.21.15.42:0/1969630469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc8107d90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.546+0000 7facd032d700 1 -- 172.21.15.42:0/1969630469 >> 172.21.15.42:0/1969630469 conn(0x7facc806da40 msgr2=0x7facc806fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.546+0000 7facd032d700 1 -- 172.21.15.42:0/1969630469 shutdown_connections 2024-09-16T23:49:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.546+0000 7facd032d700 1 -- 172.21.15.42:0/1969630469 wait complete. 2024-09-16T23:49:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.547+0000 7facd032d700 1 Processor -- start 2024-09-16T23:49:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.547+0000 7facd032d700 1 -- start start 2024-09-16T23:49:53.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.547+0000 7facd032d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc81a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.547+0000 7facd032d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 0x7facc81a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.547+0000 7facd032d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc81a1a00 con 0x7facc8072b20 2024-09-16T23:49:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.547+0000 7facd032d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc81a1b40 con 0x7facc8108360 2024-09-16T23:49:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.548+0000 7facce0c9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc81a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:53.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.548+0000 7faccd8c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 0x7facc81a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.548+0000 7facce0c9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc81a0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35624/0 (socket says 172.21.15.42:35624) 2024-09-16T23:49:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.548+0000 7facce0c9700 1 -- 172.21.15.42:0/1717599461 learned_addr learned my addr 172.21.15.42:0/1717599461 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.549+0000 7facce0c9700 1 -- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 msgr2=0x7facc81a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.548+0000 7faccd8c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 0x7facc81a13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:47168/0 (socket says 172.21.15.42:47168) 2024-09-16T23:49:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.549+0000 7facce0c9700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 0x7facc81a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.550+0000 7facce0c9700 1 -- 172.21.15.42:0/1717599461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80095f0 con 0x7facc8072b20 2024-09-16T23:49:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.550+0000 7faccd8c8700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 0x7facc81a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:49:53.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.550+0000 7facce0c9700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc81a0ea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7facb8000c00 tx=0x7facb800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:53.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.551+0000 7facbf7fe700 1 -- 172.21.15.42:0/1717599461 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb801d070 con 0x7facc8072b20 2024-09-16T23:49:53.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.551+0000 7facbf7fe700 1 -- 172.21.15.42:0/1717599461 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7facb8005dc0 con 0x7facc8072b20 2024-09-16T23:49:53.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.551+0000 7facbf7fe700 1 -- 172.21.15.42:0/1717599461 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facb801d070 con 0x7facc8072b20 2024-09-16T23:49:53.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.551+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc81a6590 con 0x7facc8072b20 2024-09-16T23:49:53.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.551+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc81a69e0 con 0x7facc8072b20 2024-09-16T23:49:53.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.552+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc8114200 con 0x7facc8072b20 2024-09-16T23:49:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.554+0000 7facbf7fe700 1 -- 172.21.15.42:0/1717599461 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 26) v1 ==== 92506+0+0 (secure 0 0 0) 0x7facb8003680 con 0x7facc8072b20 2024-09-16T23:49:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.555+0000 7facbf7fe700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7facb406e530 0x7facb40709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:53.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.555+0000 7facbf7fe700 1 -- 172.21.15.42:0/1717599461 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7facb808f210 con 0x7facc8072b20 2024-09-16T23:49:53.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.555+0000 7faccd8c8700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7facb406e530 0x7facb40709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:53.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.556+0000 7faccd8c8700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7facb406e530 0x7facb40709f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7facc4009be0 tx=0x7facc4009260 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.561+0000 7facbf7fe700 1 -- 172.21.15.42:0/1717599461 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7facb805c320 con 0x7facc8072b20 2024-09-16T23:49:53.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.884+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7facc81a70e0 con 0x7facb406e530 2024-09-16T23:49:53.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.885+0000 7facbf7fe700 1 -- 172.21.15.42:0/1717599461 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7facc81a70e0 con 0x7facb406e530 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7facb406e530 msgr2=0x7facb40709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7facb406e530 0x7facb40709f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7facc4009be0 tx=0x7facc4009260 comp rx=0 tx=0).stop 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 msgr2=0x7facc81a0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc81a0ea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7facb8000c00 tx=0x7facb800f740 comp rx=0 tx=0).stop 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 shutdown_connections 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7facb406e530 0x7facb40709f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7facc8072b20 0x7facc81a0ea0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 --2- 172.21.15.42:0/1717599461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7facc8108360 0x7facc81a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:53.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 >> 172.21.15.42:0/1717599461 conn(0x7facc806da40 msgr2=0x7facc810d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:53.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 shutdown_connections 2024-09-16T23:49:53.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:53.887+0000 7facd032d700 1 -- 172.21.15.42:0/1717599461 wait complete. 2024-09-16T23:49:54.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 -- 172.21.15.42:0/2695216921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66101017d0 msgr2=0x7f6610103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 --2- 172.21.15.42:0/2695216921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66101017d0 0x7f6610103bc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6600009870 tx=0x7f6600009b80 comp rx=0 tx=0).stop 2024-09-16T23:49:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 -- 172.21.15.42:0/2695216921 shutdown_connections 2024-09-16T23:49:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 --2- 172.21.15.42:0/2695216921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6610104100 0x7f66101064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 --2- 172.21.15.42:0/2695216921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66101017d0 0x7f6610103bc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 -- 172.21.15.42:0/2695216921 >> 172.21.15.42:0/2695216921 conn(0x7f66100fb120 msgr2=0x7f66100fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 -- 172.21.15.42:0/2695216921 shutdown_connections 2024-09-16T23:49:54.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.014+0000 7f6615f78700 1 -- 172.21.15.42:0/2695216921 wait complete. 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f6615f78700 1 Processor -- start 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f6615f78700 1 -- start start 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f6615f78700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66101017d0 0x7f6610196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f6615f78700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6610104100 0x7f66101969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f6615f78700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6610196fc0 con 0x7f66101017d0 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f6615f78700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6610197100 con 0x7f6610104100 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f660effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6610104100 0x7f66101969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f660effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6610104100 0x7f66101969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:47182/0 (socket says 172.21.15.42:47182) 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f660effd700 1 -- 172.21.15.42:0/3577819157 learned_addr learned my addr 172.21.15.42:0/3577819157 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.015+0000 7f660f7fe700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66101017d0 0x7f6610196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:54.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660effd700 1 -- 172.21.15.42:0/3577819157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66101017d0 msgr2=0x7f6610196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660effd700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66101017d0 0x7f6610196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660effd700 1 -- 172.21.15.42:0/3577819157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040095f0 con 0x7f6610104100 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660f7fe700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66101017d0 0x7f6610196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660effd700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6610104100 0x7f66101969a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f660400eb90 tx=0x7f660400ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660cff9700 1 -- 172.21.15.42:0/3577819157 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f660400cd70 con 0x7f6610104100 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660cff9700 1 -- 172.21.15.42:0/3577819157 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66040044d0 con 0x7f6610104100 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6600009520 con 0x7f6610104100 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f660cff9700 1 -- 172.21.15.42:0/3577819157 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6604010660 con 0x7f6610104100 2024-09-16T23:49:54.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.016+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661019bf70 con 0x7f6610104100 2024-09-16T23:49:54.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.017+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6610109810 con 0x7f6610104100 2024-09-16T23:49:54.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.018+0000 7f660cff9700 1 -- 172.21.15.42:0/3577819157 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) v1 ==== 92506+0+0 (secure 0 0 0) 0x7f6604003f40 con 0x7f6610104100 2024-09-16T23:49:54.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.018+0000 7f660cff9700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f65f806e440 0x7f65f8070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:54.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.018+0000 7f660cff9700 1 -- 172.21.15.42:0/3577819157 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6604014070 con 0x7f6610104100 2024-09-16T23:49:54.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.019+0000 7f660f7fe700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f65f806e440 0x7f65f8070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:54.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.019+0000 7f660f7fe700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f65f806e440 0x7f65f8070900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6600009f50 tx=0x7f6600019040 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:54.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.026+0000 7f660cff9700 1 -- 172.21.15.42:0/3577819157 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f660405ba50 con 0x7f6610104100 2024-09-16T23:49:54.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.369+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 --> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6610104d70 con 0x7f65f806e440 2024-09-16T23:49:54.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.381+0000 7f660cff9700 1 -- 172.21.15.42:0/3577819157 <== mgr.24351 v2:172.21.15.43:6828/36745564 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f6610104d70 con 0x7f65f806e440 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (7m) 17s ago 9m 25.4M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (9m) 17s ago 9m 7730k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (8m) 50s ago 8m 7570k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (9m) 17s ago 9m 6660k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (8m) 50s ago 8m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (7m) 17s ago 9m 83.9M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (3m) 17s ago 3m 172M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (3m) 17s ago 3m 13.7M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (3m) 50s ago 3m 132M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:49:54.380 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (3m) 50s ago 3m 14.7M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (24s) 17s ago 10m 66.1M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (93s) 50s ago 8m 483M - 19.1.1-330-gc2ddf972 ebb0615149de 50141468f451 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (10m) 17s ago 10m 50.6M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (8m) 50s ago 8m 43.2M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (9m) 17s ago 9m 16.5M - 1.5.0 0da6a335fe13 7c84e9bf6c1c 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (8m) 50s ago 8m 16.6M - 1.5.0 0da6a335fe13 82d41c3cc1ee 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 17s ago 7m 169M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (6m) 17s ago 6m 170M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (6m) 17s ago 6m 141M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (6m) 50s ago 6m 164M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (5m) 50s ago 5m 158M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (5m) 50s ago 5m 178M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:49:54.381 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (30s) 17s ago 8m 40.1M - 2.43.0 a07b618ecd1d f830d9898bef 2024-09-16T23:49:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f65f806e440 msgr2=0x7f65f8070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f65f806e440 0x7f65f8070900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6600009f50 tx=0x7f6600019040 comp rx=0 tx=0).stop 2024-09-16T23:49:54.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6610104100 msgr2=0x7f66101969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6610104100 0x7f66101969a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f660400eb90 tx=0x7f660400ef50 comp rx=0 tx=0).stop 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 shutdown_connections 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f65f806e440 0x7f65f8070900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66101017d0 0x7f6610196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 --2- 172.21.15.42:0/3577819157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6610104100 0x7f66101969a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 >> 172.21.15.42:0/3577819157 conn(0x7f66100fb120 msgr2=0x7f66100fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 shutdown_connections 2024-09-16T23:49:54.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.384+0000 7f6615f78700 1 -- 172.21.15.42:0/3577819157 wait complete. 2024-09-16T23:49:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.550+0000 7f352a777700 1 -- 172.21.15.42:0/2569269708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 msgr2=0x7f3524100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.550+0000 7f352a777700 1 --2- 172.21.15.42:0/2569269708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 0x7f3524100980 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3518009910 tx=0x7f3518009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:54.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.551+0000 7f352a777700 1 -- 172.21.15.42:0/2569269708 shutdown_connections 2024-09-16T23:49:54.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.551+0000 7f352a777700 1 --2- 172.21.15.42:0/2569269708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 0x7f3524100980 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.551+0000 7f352a777700 1 --2- 172.21.15.42:0/2569269708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 0x7f35240ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.551+0000 7f352a777700 1 -- 172.21.15.42:0/2569269708 >> 172.21.15.42:0/2569269708 conn(0x7f35240fa8c0 msgr2=0x7f35240fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:54.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.551+0000 7f352a777700 1 -- 172.21.15.42:0/2569269708 shutdown_connections 2024-09-16T23:49:54.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.552+0000 7f352a777700 1 -- 172.21.15.42:0/2569269708 wait complete. 2024-09-16T23:49:54.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.553+0000 7f352a777700 1 Processor -- start 2024-09-16T23:49:54.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.553+0000 7f352a777700 1 -- start start 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f352a777700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 0x7f3524071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f352a777700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 0x7f3524072410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f352a777700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352406e9c0 con 0x7f3524100520 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f352a777700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352406eb30 con 0x7f35240ff320 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f3523fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 0x7f3524071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f3523fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 0x7f3524071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:47200/0 (socket says 172.21.15.42:47200) 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f3523fff700 1 -- 172.21.15.42:0/2164810192 learned_addr learned my addr 172.21.15.42:0/2164810192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f35237fe700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 0x7f3524072410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f3523fff700 1 -- 172.21.15.42:0/2164810192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 msgr2=0x7f3524072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f3523fff700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 0x7f3524072410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.554+0000 7f3523fff700 1 -- 172.21.15.42:0/2164810192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180095f0 con 0x7f35240ff320 2024-09-16T23:49:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.555+0000 7f3523fff700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 0x7f3524071ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f351401fcc0 tx=0x7f351401ae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.555+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f351401a7a0 con 0x7f35240ff320 2024-09-16T23:49:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.555+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f35140044d0 con 0x7f35240ff320 2024-09-16T23:49:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.555+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3514021440 con 0x7f35240ff320 2024-09-16T23:49:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.556+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352406ee10 con 0x7f35240ff320 2024-09-16T23:49:54.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.556+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352406f300 con 0x7f35240ff320 2024-09-16T23:49:54.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.557+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3524189c00 con 0x7f35240ff320 2024-09-16T23:49:54.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.559+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 26) v1 ==== 92506+0+0 (secure 0 0 0) 0x7f351401dc20 con 0x7f35240ff320 2024-09-16T23:49:54.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.559+0000 7f35217fa700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f350c06e4d0 0x7f350c070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:54.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.559+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3514025070 con 0x7f35240ff320 2024-09-16T23:49:54.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.560+0000 7f35237fe700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f350c06e4d0 0x7f350c070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:54.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.562+0000 7f35237fe700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f350c06e4d0 0x7f350c070990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3518006010 tx=0x7f3518009f70 comp rx=0 tx=0).ready entity=mgr.24351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:54.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.566+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f351406c880 con 0x7f35240ff320 2024-09-16T23:49:54.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.573+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 27) v1 ==== 46123+0+0 (secure 0 0 0) 0x7f3514069120 con 0x7f35240ff320 2024-09-16T23:49:54.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.574+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f350c06e4d0 msgr2=0x7f350c070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.574+0000 7f35217fa700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f350c06e4d0 0x7f350c070990 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3518006010 tx=0x7f3518009f70 comp rx=0 tx=0).stop 2024-09-16T23:49:54.867 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:54.867 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: pgmap v36: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.0 MiB/s wr, 127 op/s 2024-09-16T23:49:54.867 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr fail", "who": "smithi043.xveyou"}]: dispatch 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr fail", "who": "smithi043.xveyou"}]: dispatch 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: osdmap e48: 6 total, 6 up, 6 in 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd='[{"prefix": "mgr fail", "who": "smithi043.xveyou"}]': finished 2024-09-16T23:49:54.868 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: mgrmap e27: smithi042.zrkumz(active, starting, since 0.00925494s) 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:49:54.869 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: pgmap v36: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 42 KiB/s rd, 2.0 MiB/s wr, 127 op/s 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:54.870 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 172.21.15.43:0/2114889420' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr fail", "who": "smithi043.xveyou"}]: dispatch 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "mgr fail", "who": "smithi043.xveyou"}]: dispatch 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: osdmap e48: 6 total, 6 up, 6 in 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.24351 ' entity='mgr.smithi043.xveyou' cmd='[{"prefix": "mgr fail", "who": "smithi043.xveyou"}]': finished 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: mgrmap e27: smithi042.zrkumz(active, starting, since 0.00925494s) 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:49:54.871 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:49:54.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:49:54.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:49:54.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.930+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f352402cf20 con 0x7f35240ff320 2024-09-16T23:49:54.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.930+0000 7f35217fa700 1 -- 172.21.15.42:0/2164810192 <== mon.1 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f351406c880 con 0x7f35240ff320 2024-09-16T23:49:54.929 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:49:54.929 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:49:54.929 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:49:54.930 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-09-16T23:49:54.931 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:49:54.931 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:49:54.931 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 msgr2=0x7f3524071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 0x7f3524071ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f351401fcc0 tx=0x7f351401ae60 comp rx=0 tx=0).stop 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 shutdown_connections 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:6828/36745564,v1:172.21.15.43:6829/36745564] conn(0x7f350c06e4d0 0x7f350c070990 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35240ff320 0x7f3524071ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 --2- 172.21.15.42:0/2164810192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3524100520 0x7f3524072410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 >> 172.21.15.42:0/2164810192 conn(0x7f35240fa8c0 msgr2=0x7f35240fb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:54.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 shutdown_connections 2024-09-16T23:49:54.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:54.933+0000 7f352a777700 1 -- 172.21.15.42:0/2164810192 wait complete. 2024-09-16T23:49:55.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.104+0000 7efd205a3700 1 -- 172.21.15.42:0/1654023719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd180ffc10 msgr2=0x7efd18100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:55.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.104+0000 7efd205a3700 1 --2- 172.21.15.42:0/1654023719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd180ffc10 0x7efd18100030 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd14009910 tx=0x7efd14009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:55.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.104+0000 7efd205a3700 1 -- 172.21.15.42:0/1654023719 shutdown_connections 2024-09-16T23:49:55.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.104+0000 7efd205a3700 1 --2- 172.21.15.42:0/1654023719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18100e10 0x7efd18101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:55.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.104+0000 7efd205a3700 1 --2- 172.21.15.42:0/1654023719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd180ffc10 0x7efd18100030 secure :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd14009910 tx=0x7efd14009c20 comp rx=0 tx=0).stop 2024-09-16T23:49:55.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.104+0000 7efd205a3700 1 -- 172.21.15.42:0/1654023719 >> 172.21.15.42:0/1654023719 conn(0x7efd180fb190 msgr2=0x7efd180fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:55.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.105+0000 7efd205a3700 1 -- 172.21.15.42:0/1654023719 shutdown_connections 2024-09-16T23:49:55.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.105+0000 7efd205a3700 1 -- 172.21.15.42:0/1654023719 wait complete. 2024-09-16T23:49:55.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.106+0000 7efd205a3700 1 Processor -- start 2024-09-16T23:49:55.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.106+0000 7efd205a3700 1 -- start start 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd205a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd18100e10 0x7efd18195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd205a3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18195b70 0x7efd1819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd205a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd18196080 con 0x7efd18100e10 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd1db3e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18195b70 0x7efd1819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd1db3e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18195b70 0x7efd1819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:47334/0 (socket says 172.21.15.42:47334) 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd1e33f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd18100e10 0x7efd18195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd1db3e700 1 -- 172.21.15.42:0/3392144062 learned_addr learned my addr 172.21.15.42:0/3392144062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:49:55.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.107+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd181961f0 con 0x7efd18195b70 2024-09-16T23:49:55.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.108+0000 7efd1db3e700 1 -- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd18100e10 msgr2=0x7efd18195630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:55.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.108+0000 7efd1db3e700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd18100e10 0x7efd18195630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.108+0000 7efd1db3e700 1 -- 172.21.15.42:0/3392144062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd140095f0 con 0x7efd18195b70 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.108+0000 7efd1db3e700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18195b70 0x7efd1819abe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e920 tx=0x7efd0800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.109+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd0800cba0 con 0x7efd18195b70 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.109+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd180685e0 con 0x7efd18195b70 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.109+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd18068b30 con 0x7efd18195b70 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.109+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd0800cd00 con 0x7efd18195b70 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.109+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd08018770 con 0x7efd18195b70 2024-09-16T23:49:55.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.110+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 27) v1 ==== 46123+0+0 (secure 0 0 0) 0x7efd080188d0 con 0x7efd18195b70 2024-09-16T23:49:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.110+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7efd08014070 con 0x7efd18195b70 2024-09-16T23:49:55.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.111+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd18108700 con 0x7efd18195b70 2024-09-16T23:49:55.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.118+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7efd0801fdf0 con 0x7efd18195b70 2024-09-16T23:49:55.742 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:55 smithi042 ceph-mon[27914]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:49:55.742 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:55 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2164810192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:49:55.742 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:55.743 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:55.743 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:49:55.743 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:49:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.743+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 28) v1 ==== 46544+0+0 (secure 0 0 0) 0x7efd08018c20 con 0x7efd18195b70 2024-09-16T23:49:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.743+0000 7efd0f7fe700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7efd0403e4e0 0x7efd040408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:49:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.743+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd04040e10 con 0x7efd0403e4e0 2024-09-16T23:49:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.747+0000 7efd1e33f700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7efd0403e4e0 0x7efd040408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:49:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.749+0000 7efd1e33f700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7efd0403e4e0 0x7efd040408d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd14000c00 tx=0x7efd14009f70 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:49:55.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.751+0000 7efd0f7fe700 1 -- 172.21.15.42:0/3392144062 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7efd04040e10 con 0x7efd0403e4e0 2024-09-16T23:49:55.750 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:49:55.750 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:49:55.750 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:49:55.750 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:49:55.750 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:49:55.750 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:49:55.751 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:49:55.751 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "2/2 daemons upgraded", 2024-09-16T23:49:55.751 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "", 2024-09-16T23:49:55.751 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:49:55.751 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:49:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7efd0403e4e0 msgr2=0x7efd040408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7efd0403e4e0 0x7efd040408d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd14000c00 tx=0x7efd14009f70 comp rx=0 tx=0).stop 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18195b70 msgr2=0x7efd1819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18195b70 0x7efd1819abe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e920 tx=0x7efd0800ece0 comp rx=0 tx=0).stop 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 shutdown_connections 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd18100e10 0x7efd18195630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7efd0403e4e0 0x7efd040408d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 --2- 172.21.15.42:0/3392144062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd18195b70 0x7efd1819abe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 >> 172.21.15.42:0/3392144062 conn(0x7efd180fb190 msgr2=0x7efd18104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.755+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 shutdown_connections 2024-09-16T23:49:55.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:49:55.756+0000 7efd205a3700 1 -- 172.21.15.42:0/3392144062 wait complete. 2024-09-16T23:49:56.046 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:55 smithi043 ceph-mon[35191]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:49:56.046 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:55 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2164810192' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:49:56.046 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:49:56.046 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:49:56.046 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:49:56.046 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:49:57.003 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:56 smithi042 ceph-mon[27914]: mgrmap e28: smithi042.zrkumz(active, since 1.17496s) 2024-09-16T23:49:57.003 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:56 smithi042 ceph-mon[27914]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:57.003 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:56 smithi042 ceph-mon[27914]: pgmap v3: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:49:57.003 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:56 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:49:57.004 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:56 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:49:57.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:56 smithi043 ceph-mon[35191]: mgrmap e28: smithi042.zrkumz(active, since 1.17496s) 2024-09-16T23:49:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:56 smithi043 ceph-mon[35191]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:49:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:56 smithi043 ceph-mon[35191]: pgmap v3: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:49:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:56 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:49:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:56 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:49:58.019 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:57 smithi042 ceph-mon[27914]: [16/Sep/2024:23:49:57] ENGINE Bus STARTING 2024-09-16T23:49:58.019 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:57 smithi042 ceph-mon[27914]: [16/Sep/2024:23:49:57] ENGINE Serving on http://172.21.15.42:8765 2024-09-16T23:49:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:57 smithi043 ceph-mon[35191]: [16/Sep/2024:23:49:57] ENGINE Bus STARTING 2024-09-16T23:49:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:57 smithi043 ceph-mon[35191]: [16/Sep/2024:23:49:57] ENGINE Serving on http://172.21.15.42:8765 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:58 smithi043 ceph-mon[35191]: [16/Sep/2024:23:49:57] ENGINE Serving on https://172.21.15.42:7150 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:58 smithi043 ceph-mon[35191]: [16/Sep/2024:23:49:57] ENGINE Bus STARTED 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:58 smithi043 ceph-mon[35191]: [16/Sep/2024:23:49:57] ENGINE Error in HTTPServer.serve 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Traceback (most recent call last): 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._run(expiration_interval) 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T23:49:58.988 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: s = self.context.wrap_socket( 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: return self.sslsocket_class._create( 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self.do_handshake() 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._sslobj.do_handshake() 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T23:49:58.989 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:58 smithi043 ceph-mon[35191]: mgrmap e29: smithi042.zrkumz(active, since 3s) 2024-09-16T23:49:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:58 smithi042 ceph-mon[27914]: [16/Sep/2024:23:49:57] ENGINE Serving on https://172.21.15.42:7150 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:58 smithi042 ceph-mon[27914]: [16/Sep/2024:23:49:57] ENGINE Bus STARTED 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:58 smithi042 ceph-mon[27914]: [16/Sep/2024:23:49:57] ENGINE Error in HTTPServer.serve 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Traceback (most recent call last): 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._connections.run(self.expiration_interval) 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._run(expiration_interval) 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: s = self.context.wrap_socket( 2024-09-16T23:49:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T23:49:59.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: return self.sslsocket_class._create( 2024-09-16T23:49:59.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T23:49:59.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self.do_handshake() 2024-09-16T23:49:59.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T23:49:59.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._sslobj.do_handshake() 2024-09-16T23:49:59.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T23:49:59.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:58 smithi042 ceph-mon[27914]: mgrmap e29: smithi042.zrkumz(active, since 3s) 2024-09-16T23:49:59.799 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:49:59 smithi042 ceph-mon[27914]: pgmap v5: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:49:59.969 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:49:59 smithi043 ceph-mon[35191]: pgmap v5: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:01.245 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:00 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:01.245 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:00 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:01.245 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:00 smithi042 ceph-mon[27914]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:50:01.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:00 smithi042 ceph-mon[27914]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:50:01.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:00 smithi042 ceph-mon[27914]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:50:01.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:00 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:01.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:00 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:01.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:00 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:00 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:00 smithi043 ceph-mon[35191]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:50:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:00 smithi043 ceph-mon[35191]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:50:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:00 smithi043 ceph-mon[35191]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:50:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:00 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:00 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:02.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:01 smithi042 ceph-mon[27914]: pgmap v6: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:01 smithi042 ceph-mon[27914]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:50:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:01 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:50:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:01 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:50:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:01 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:50:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:01 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:50:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:01 smithi043 ceph-mon[35191]: pgmap v6: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:01 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:50:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:01 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:50:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:01 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:50:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:01 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:50:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:01 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/692305612' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:50:03.191 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:02 smithi042 ceph-mon[27914]: mgrmap e30: smithi042.zrkumz(active, since 7s), standbys: smithi043.xveyou 2024-09-16T23:50:03.191 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:02 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:50:03.191 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:02 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:03.191 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:02 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:03.191 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:02 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:03.192 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:02 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:02 smithi043 ceph-mon[35191]: mgrmap e30: smithi042.zrkumz(active, since 7s), standbys: smithi043.xveyou 2024-09-16T23:50:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:02 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:50:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:02 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:02 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:02 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:02 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:04.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:03 smithi043 ceph-mon[35191]: pgmap v7: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 67 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-09-16T23:50:04.245 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:03 smithi042 ceph-mon[27914]: pgmap v7: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 67 KiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-09-16T23:50:06.266 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:05 smithi042 ceph-mon[27914]: pgmap v8: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-09-16T23:50:06.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:05 smithi043 ceph-mon[35191]: pgmap v8: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.4 MiB/s wr, 102 op/s 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: pgmap v9: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:50:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:07 smithi043 ceph-mon[35191]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:50:08.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: pgmap v9: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60 KiB/s rd, 1.8 MiB/s wr, 157 op/s 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:50:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:07 smithi042 ceph-mon[27914]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:50:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:08 smithi042 ceph-mon[27914]: Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:08 smithi043 ceph-mon[35191]: Reconfiguring prometheus.smithi042 (dependencies changed)... 2024-09-16T23:50:10.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:09 smithi042 ceph-mon[27914]: Reconfiguring daemon prometheus.smithi042 on smithi042 2024-09-16T23:50:10.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:09 smithi042 ceph-mon[27914]: pgmap v10: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-09-16T23:50:10.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:09 smithi043 ceph-mon[35191]: Reconfiguring daemon prometheus.smithi042 on smithi042 2024-09-16T23:50:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:09 smithi043 ceph-mon[35191]: pgmap v10: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-09-16T23:50:11.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:10 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:11.371 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:10 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:12.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:11 smithi042 ceph-mon[27914]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-09-16T23:50:12.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:11 smithi043 ceph-mon[35191]: pgmap v11: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 141 op/s 2024-09-16T23:50:14.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:13 smithi042 ceph-mon[27914]: pgmap v12: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-16T23:50:14.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:13 smithi043 ceph-mon[35191]: pgmap v12: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s rd, 2.3 MiB/s wr, 203 op/s 2024-09-16T23:50:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:15 smithi043 ceph-mon[35191]: pgmap v13: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-16T23:50:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:15 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:15 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:15 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:50:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:15 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:16.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:15 smithi042 ceph-mon[27914]: pgmap v13: 65 pgs: 65 active+clean; 327 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-09-16T23:50:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:15 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:15 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:15 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:50:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:15 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:17.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:16 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:50:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:16 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:50:18.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:17 smithi043 ceph-mon[35191]: pgmap v14: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-16T23:50:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:17 smithi043 ceph-mon[35191]: Upgrade: Updating mgr.smithi043.xveyou 2024-09-16T23:50:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:17 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:50:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:17 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:50:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:17 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:17 smithi043 ceph-mon[35191]: Deploying daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:50:18.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:17 smithi042 ceph-mon[27914]: pgmap v14: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-09-16T23:50:18.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:17 smithi042 ceph-mon[27914]: Upgrade: Updating mgr.smithi043.xveyou 2024-09-16T23:50:18.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:17 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:50:18.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:17 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:50:18.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:17 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:18.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:17 smithi042 ceph-mon[27914]: Deploying daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:50:20.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:19 smithi042 ceph-mon[27914]: pgmap v15: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-16T23:50:20.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:19 smithi043 ceph-mon[35191]: pgmap v15: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-16T23:50:22.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:21 smithi042 ceph-mon[27914]: pgmap v16: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-16T23:50:22.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:21 smithi043 ceph-mon[35191]: pgmap v16: 65 pgs: 65 active+clean; 329 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 118 op/s 2024-09-16T23:50:24.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:23 smithi042 ceph-mon[27914]: pgmap v17: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-16T23:50:24.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:23 smithi043 ceph-mon[35191]: pgmap v17: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-16T23:50:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:25 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:25.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:25 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 -- 172.21.15.42:0/4096381594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 msgr2=0x7f93641064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 --2- 172.21.15.42:0/4096381594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 0x7f93641064c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9358009910 tx=0x7f9358009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 -- 172.21.15.42:0/4096381594 shutdown_connections 2024-09-16T23:50:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 --2- 172.21.15.42:0/4096381594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 0x7f93641064c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:25.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 --2- 172.21.15.42:0/4096381594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 0x7f9364103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 -- 172.21.15.42:0/4096381594 >> 172.21.15.42:0/4096381594 conn(0x7f93640fb190 msgr2=0x7f93640fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 -- 172.21.15.42:0/4096381594 shutdown_connections 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.903+0000 7f9369281700 1 -- 172.21.15.42:0/4096381594 wait complete. 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.904+0000 7f9369281700 1 Processor -- start 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.904+0000 7f9369281700 1 -- start start 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.904+0000 7f9369281700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 0x7f9364196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.904+0000 7f9369281700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 0x7f9364196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.904+0000 7f9369281700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9364196fb0 con 0x7f93641040d0 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.904+0000 7f9369281700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93641970f0 con 0x7f93641017a0 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f9362ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 0x7f9364196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f93627fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 0x7f9364196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:25.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f9362ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 0x7f9364196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:49080/0 (socket says 172.21.15.42:49080) 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f9362ffd700 1 -- 172.21.15.42:0/168553062 learned_addr learned my addr 172.21.15.42:0/168553062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f9362ffd700 1 -- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 msgr2=0x7f9364196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f9362ffd700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 0x7f9364196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f9362ffd700 1 -- 172.21.15.42:0/168553062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580095f0 con 0x7f93641017a0 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f93627fc700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 0x7f9364196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.905+0000 7f9362ffd700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 0x7f9364196450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f935400e920 tx=0x7f935400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.906+0000 7f934bfff700 1 -- 172.21.15.42:0/168553062 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935400c9d0 con 0x7f93641017a0 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.906+0000 7f934bfff700 1 -- 172.21.15.42:0/168553062 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9354010430 con 0x7f93641017a0 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.906+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936419bba0 con 0x7f93641017a0 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.906+0000 7f934bfff700 1 -- 172.21.15.42:0/168553062 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9354018650 con 0x7f93641017a0 2024-09-16T23:50:25.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.906+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936419c0f0 con 0x7f93641017a0 2024-09-16T23:50:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.907+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9364190610 con 0x7f93641017a0 2024-09-16T23:50:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.908+0000 7f934bfff700 1 -- 172.21.15.42:0/168553062 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 30) v1 ==== 92506+0+0 (secure 0 0 0) 0x7f93540105a0 con 0x7f93641017a0 2024-09-16T23:50:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.908+0000 7f934bfff700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f934c06e440 0x7f934c070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.908+0000 7f934bfff700 1 -- 172.21.15.42:0/168553062 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f9354014070 con 0x7f93641017a0 2024-09-16T23:50:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.908+0000 7f93627fc700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f934c06e440 0x7f934c070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:25.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.909+0000 7f93627fc700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f934c06e440 0x7f934c070900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9358009fd0 tx=0x7f9358005940 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:25.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:25.916+0000 7f934bfff700 1 -- 172.21.15.42:0/168553062 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f93540539f0 con 0x7f93641017a0 2024-09-16T23:50:26.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.254+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93640611d0 con 0x7f934c06e440 2024-09-16T23:50:26.252 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:26 smithi042 ceph-mon[27914]: pgmap v18: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-09-16T23:50:26.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.255+0000 7f934bfff700 1 -- 172.21.15.42:0/168553062 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f93640611d0 con 0x7f934c06e440 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f934c06e440 msgr2=0x7f934c070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f934c06e440 0x7f934c070900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9358009fd0 tx=0x7f9358005940 comp rx=0 tx=0).stop 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 msgr2=0x7f9364196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 0x7f9364196450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f935400e920 tx=0x7f935400ec30 comp rx=0 tx=0).stop 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 shutdown_connections 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f934c06e440 0x7f934c070900 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93641017a0 0x7f9364196450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 --2- 172.21.15.42:0/168553062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93641040d0 0x7f9364196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 >> 172.21.15.42:0/168553062 conn(0x7f93640fb190 msgr2=0x7f93640fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 shutdown_connections 2024-09-16T23:50:26.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.258+0000 7f9369281700 1 -- 172.21.15.42:0/168553062 wait complete. 2024-09-16T23:50:26.266 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:50:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3372326844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481040d0 msgr2=0x7fd5481064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3372326844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481040d0 0x7fd5481064c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd538009910 tx=0x7fd538009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3372326844 shutdown_connections 2024-09-16T23:50:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3372326844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481040d0 0x7fd5481064c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3372326844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5481017a0 0x7fd548103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3372326844 >> 172.21.15.42:0/3372326844 conn(0x7fd5480fb190 msgr2=0x7fd5480fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:26.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3372326844 shutdown_connections 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.378+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3372326844 wait complete. 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54fd0f700 1 Processor -- start 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54fd0f700 1 -- start start 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54fd0f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481017a0 0x7fd548196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54fd0f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5481040d0 0x7fd548196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54fd0f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd548196fb0 con 0x7fd5481017a0 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54fd0f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5481970f0 con 0x7fd5481040d0 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54daab700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481017a0 0x7fd548196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54daab700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481017a0 0x7fd548196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46342/0 (socket says 172.21.15.42:46342) 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54daab700 1 -- 172.21.15.42:0/3044905040 learned_addr learned my addr 172.21.15.42:0/3044905040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54d2aa700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5481040d0 0x7fd548196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54daab700 1 -- 172.21.15.42:0/3044905040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5481040d0 msgr2=0x7fd548196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54daab700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5481040d0 0x7fd548196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.379+0000 7fd54daab700 1 -- 172.21.15.42:0/3044905040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380095f0 con 0x7fd5481017a0 2024-09-16T23:50:26.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.380+0000 7fd54daab700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481017a0 0x7fd548196450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e920 tx=0x7fd54400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:26.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.380+0000 7fd53effd700 1 -- 172.21.15.42:0/3044905040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54400c9d0 con 0x7fd5481017a0 2024-09-16T23:50:26.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.380+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54819bba0 con 0x7fd5481017a0 2024-09-16T23:50:26.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.380+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54819c0f0 con 0x7fd5481017a0 2024-09-16T23:50:26.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.380+0000 7fd53effd700 1 -- 172.21.15.42:0/3044905040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd544010430 con 0x7fd5481017a0 2024-09-16T23:50:26.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.380+0000 7fd53effd700 1 -- 172.21.15.42:0/3044905040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd544018650 con 0x7fd5481017a0 2024-09-16T23:50:26.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:26 smithi043 ceph-mon[35191]: pgmap v18: 65 pgs: 65 active+clean; 324 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-09-16T23:50:26.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.381+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c005320 con 0x7fd5481017a0 2024-09-16T23:50:26.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.382+0000 7fd53effd700 1 -- 172.21.15.42:0/3044905040 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) v1 ==== 92506+0+0 (secure 0 0 0) 0x7fd544010a60 con 0x7fd5481017a0 2024-09-16T23:50:26.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.383+0000 7fd53effd700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd53406e430 0x7fd5340708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:26.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.383+0000 7fd53effd700 1 -- 172.21.15.42:0/3044905040 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd544014070 con 0x7fd5481017a0 2024-09-16T23:50:26.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.383+0000 7fd54d2aa700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd53406e430 0x7fd5340708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:26.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.383+0000 7fd54d2aa700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd53406e430 0x7fd5340708f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b6f0 tx=0x7fd538005a90 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:26.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.392+0000 7fd53effd700 1 -- 172.21.15.42:0/3044905040 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fd5440539f0 con 0x7fd5481017a0 2024-09-16T23:50:26.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.817+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd52c000bf0 con 0x7fd53406e430 2024-09-16T23:50:26.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.818+0000 7fd53effd700 1 -- 172.21.15.42:0/3044905040 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fd52c000bf0 con 0x7fd53406e430 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd53406e430 msgr2=0x7fd5340708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd53406e430 0x7fd5340708f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b6f0 tx=0x7fd538005a90 comp rx=0 tx=0).stop 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481017a0 msgr2=0x7fd548196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481017a0 0x7fd548196450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e920 tx=0x7fd54400ec30 comp rx=0 tx=0).stop 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 shutdown_connections 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5481017a0 0x7fd548196450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd53406e430 0x7fd5340708f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 --2- 172.21.15.42:0/3044905040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5481040d0 0x7fd548196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 >> 172.21.15.42:0/3044905040 conn(0x7fd5480fb190 msgr2=0x7fd5480fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:26.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 shutdown_connections 2024-09-16T23:50:26.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.821+0000 7fd54fd0f700 1 -- 172.21.15.42:0/3044905040 wait complete. 2024-09-16T23:50:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 -- 172.21.15.42:0/2213306962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 msgr2=0x7f09600fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 --2- 172.21.15.42:0/2213306962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09600fffc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f095c009910 tx=0x7f095c009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 -- 172.21.15.42:0/2213306962 shutdown_connections 2024-09-16T23:50:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 --2- 172.21.15.42:0/2213306962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960100da0 0x7f0960101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 --2- 172.21.15.42:0/2213306962 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09600fffc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 -- 172.21.15.42:0/2213306962 >> 172.21.15.42:0/2213306962 conn(0x7f09600fb120 msgr2=0x7f09600fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:26.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 -- 172.21.15.42:0/2213306962 shutdown_connections 2024-09-16T23:50:26.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.960+0000 7f0967bd7700 1 -- 172.21.15.42:0/2213306962 wait complete. 2024-09-16T23:50:26.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.961+0000 7f0967bd7700 1 Processor -- start 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.961+0000 7f0967bd7700 1 -- start start 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.961+0000 7f0967bd7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09601964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.961+0000 7f0967bd7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960100da0 0x7f09601969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.961+0000 7f0967bd7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0960197010 con 0x7f09600ffba0 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.961+0000 7f0967bd7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0960197150 con 0x7f0960100da0 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965973700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09601964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965973700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09601964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46348/0 (socket says 172.21.15.42:46348) 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965172700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960100da0 0x7f09601969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965973700 1 -- 172.21.15.42:0/1250838246 learned_addr learned my addr 172.21.15.42:0/1250838246 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:26.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965973700 1 -- 172.21.15.42:0/1250838246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960100da0 msgr2=0x7f09601969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965973700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960100da0 0x7f09601969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965973700 1 -- 172.21.15.42:0/1250838246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0095f0 con 0x7f09600ffba0 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965172700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960100da0 0x7f09601969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0965973700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09601964b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f095c000c00 tx=0x7f095c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0956ffd700 1 -- 172.21.15.42:0/1250838246 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f095c01d070 con 0x7f09600ffba0 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.962+0000 7f0956ffd700 1 -- 172.21.15.42:0/1250838246 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f095c005dc0 con 0x7f09600ffba0 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.963+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096019bba0 con 0x7f09600ffba0 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.963+0000 7f0956ffd700 1 -- 172.21.15.42:0/1250838246 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f095c022470 con 0x7f09600ffba0 2024-09-16T23:50:26.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.963+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096019c090 con 0x7f09600ffba0 2024-09-16T23:50:26.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.964+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0960108700 con 0x7f09600ffba0 2024-09-16T23:50:26.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.964+0000 7f0956ffd700 1 -- 172.21.15.42:0/1250838246 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) v1 ==== 92506+0+0 (secure 0 0 0) 0x7f095c00fa70 con 0x7f09600ffba0 2024-09-16T23:50:26.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.965+0000 7f0956ffd700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f094c06e440 0x7f094c070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.965+0000 7f0956ffd700 1 -- 172.21.15.42:0/1250838246 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f095c08f760 con 0x7f09600ffba0 2024-09-16T23:50:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.965+0000 7f0965172700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f094c06e440 0x7f094c070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:26.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.966+0000 7f0965172700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f094c06e440 0x7f094c070900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0950011490 tx=0x7f0950009260 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:26.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:26.971+0000 7f0956ffd700 1 -- 172.21.15.42:0/1250838246 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f095c05c7c0 con 0x7f09600ffba0 2024-09-16T23:50:27.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.218+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0960069520 con 0x7f094c06e440 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.235+0000 7f0956ffd700 1 -- 172.21.15.42:0/1250838246 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f0960069520 con 0x7f094c06e440 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (8m) 25s ago 9m 25.5M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (10m) 25s ago 10m 7763k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (9m) 27s ago 9m 7658k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (10m) 25s ago 10m 6660k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (8m) 27s ago 8m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:50:27.233 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (8m) 25s ago 9m 84.4M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (4m) 25s ago 4m 158M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (4m) 25s ago 4m 13.9M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (4m) 27s ago 4m 179M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (4m) 27s ago 4m 15.2M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (57s) 25s ago 11m 475M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (2m) 27s ago 8m 105M - 19.1.1-330-gc2ddf972 ebb0615149de 50141468f451 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (11m) 25s ago 11m 53.1M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (8m) 27s ago 8m 46.7M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (10m) 25s ago 9m 16.4M - 1.5.0 0da6a335fe13 7c84e9bf6c1c 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (8m) 27s ago 8m 16.5M - 1.5.0 0da6a335fe13 82d41c3cc1ee 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 25s ago 7m 188M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (7m) 25s ago 7m 185M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (7m) 25s ago 7m 150M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (6m) 27s ago 6m 221M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:50:27.234 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (6m) 27s ago 6m 195M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:50:27.235 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (5m) 27s ago 5m 190M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:50:27.235 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 starting - - - - 2024-09-16T23:50:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f094c06e440 msgr2=0x7f094c070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f094c06e440 0x7f094c070900 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0950011490 tx=0x7f0950009260 comp rx=0 tx=0).stop 2024-09-16T23:50:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 msgr2=0x7f09601964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09601964b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f095c000c00 tx=0x7f095c00f690 comp rx=0 tx=0).stop 2024-09-16T23:50:27.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 shutdown_connections 2024-09-16T23:50:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f09600ffba0 0x7f09601964b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f094c06e440 0x7f094c070900 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 --2- 172.21.15.42:0/1250838246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0960100da0 0x7f09601969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 >> 172.21.15.42:0/1250838246 conn(0x7f09600fb120 msgr2=0x7f0960068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 shutdown_connections 2024-09-16T23:50:27.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.238+0000 7f0967bd7700 1 -- 172.21.15.42:0/1250838246 wait complete. 2024-09-16T23:50:27.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 -- 172.21.15.42:0/3777780705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef80ffbb0 msgr2=0x7fcef80fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 --2- 172.21.15.42:0/3777780705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef80ffbb0 0x7fcef80fffd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009910 tx=0x7fcee8009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 -- 172.21.15.42:0/3777780705 shutdown_connections 2024-09-16T23:50:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 --2- 172.21.15.42:0/3777780705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef8100db0 0x7fcef8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 --2- 172.21.15.42:0/3777780705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef80ffbb0 0x7fcef80fffd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 -- 172.21.15.42:0/3777780705 >> 172.21.15.42:0/3777780705 conn(0x7fcef80fb150 msgr2=0x7fcef80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 -- 172.21.15.42:0/3777780705 shutdown_connections 2024-09-16T23:50:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.387+0000 7fcefd668700 1 -- 172.21.15.42:0/3777780705 wait complete. 2024-09-16T23:50:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.388+0000 7fcefd668700 1 Processor -- start 2024-09-16T23:50:27.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.388+0000 7fcefd668700 1 -- start start 2024-09-16T23:50:27.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.388+0000 7fcefd668700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef80ffbb0 0x7fcef81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:27.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.388+0000 7fcefd668700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef8100db0 0x7fcef8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:27.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.388+0000 7fcefd668700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8199240 con 0x7fcef8100db0 2024-09-16T23:50:27.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.388+0000 7fcefd668700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8199380 con 0x7fcef80ffbb0 2024-09-16T23:50:27.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef8100db0 0x7fcef8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef80ffbb0 0x7fcef81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef8100db0 0x7fcef8198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46372/0 (socket says 172.21.15.42:46372) 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef67fc700 1 -- 172.21.15.42:0/3782327490 learned_addr learned my addr 172.21.15.42:0/3782327490 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef67fc700 1 -- 172.21.15.42:0/3782327490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef80ffbb0 msgr2=0x7fcef81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef67fc700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef80ffbb0 0x7fcef81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef67fc700 1 -- 172.21.15.42:0/3782327490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee80095f0 con 0x7fcef8100db0 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcef67fc700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef8100db0 0x7fcef8198c20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e920 tx=0x7fceec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcedffff700 1 -- 172.21.15.42:0/3782327490 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec00c9d0 con 0x7fcef8100db0 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.389+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef819de30 con 0x7fcef8100db0 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.390+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef819e350 con 0x7fcef8100db0 2024-09-16T23:50:27.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.390+0000 7fcedffff700 1 -- 172.21.15.42:0/3782327490 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fceec010430 con 0x7fcef8100db0 2024-09-16T23:50:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.390+0000 7fcedffff700 1 -- 172.21.15.42:0/3782327490 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceec018650 con 0x7fcef8100db0 2024-09-16T23:50:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.391+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee4005320 con 0x7fcef8100db0 2024-09-16T23:50:27.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.392+0000 7fcedffff700 1 -- 172.21.15.42:0/3782327490 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 30) v1 ==== 92506+0+0 (secure 0 0 0) 0x7fceec00cb30 con 0x7fcef8100db0 2024-09-16T23:50:27.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.392+0000 7fcedffff700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcee006e430 0x7fcee00708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:27.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.393+0000 7fcedffff700 1 -- 172.21.15.42:0/3782327490 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fceec014070 con 0x7fcef8100db0 2024-09-16T23:50:27.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.393+0000 7fcef6ffd700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcee006e430 0x7fcee00708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:27.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.393+0000 7fcef6ffd700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcee006e430 0x7fcee00708f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fd0 tx=0x7fcee8005a90 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:27.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.402+0000 7fcedffff700 1 -- 172.21.15.42:0/3782327490 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fceec0539f0 con 0x7fcef8100db0 2024-09-16T23:50:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.827+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcee4005cc0 con 0x7fcef8100db0 2024-09-16T23:50:27.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.828+0000 7fcedffff700 1 -- 172.21.15.42:0/3782327490 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fceec05b500 con 0x7fcef8100db0 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:50:27.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:50:27.827 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:50:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcee006e430 msgr2=0x7fcee00708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcee006e430 0x7fcee00708f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fd0 tx=0x7fcee8005a90 comp rx=0 tx=0).stop 2024-09-16T23:50:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef8100db0 msgr2=0x7fcef8198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef8100db0 0x7fcef8198c20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e920 tx=0x7fceec00ec30 comp rx=0 tx=0).stop 2024-09-16T23:50:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 shutdown_connections 2024-09-16T23:50:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcee006e430 0x7fcee00708f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef80ffbb0 0x7fcef81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 --2- 172.21.15.42:0/3782327490 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcef8100db0 0x7fcef8198c20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 >> 172.21.15.42:0/3782327490 conn(0x7fcef80fb150 msgr2=0x7fcef8068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:27.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 shutdown_connections 2024-09-16T23:50:27.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.831+0000 7fcefd668700 1 -- 172.21.15.42:0/3782327490 wait complete. 2024-09-16T23:50:27.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/698474160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c880ffc10 msgr2=0x7f2c88100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/698474160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c880ffc10 0x7f2c88100030 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009910 tx=0x7f2c78009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:27.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/698474160 shutdown_connections 2024-09-16T23:50:27.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/698474160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c88100e10 0x7f2c88101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/698474160 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c880ffc10 0x7f2c88100030 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/698474160 >> 172.21.15.42:0/698474160 conn(0x7f2c880fb190 msgr2=0x7f2c880fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:27.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/698474160 shutdown_connections 2024-09-16T23:50:27.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.959+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/698474160 wait complete. 2024-09-16T23:50:27.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.960+0000 7f2c8ce3a700 1 Processor -- start 2024-09-16T23:50:27.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.960+0000 7f2c8ce3a700 1 -- start start 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.960+0000 7f2c8ce3a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c880ffc10 0x7f2c880690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.960+0000 7f2c8ce3a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c88100e10 0x7f2c8819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.960+0000 7f2c8ce3a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8819b800 con 0x7f2c88100e10 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.960+0000 7f2c8ce3a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8819b940 con 0x7f2c880ffc10 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c880ffc10 0x7f2c880690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c880ffc10 0x7f2c880690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:49152/0 (socket says 172.21.15.42:49152) 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8659c700 1 -- 172.21.15.42:0/4075096776 learned_addr learned my addr 172.21.15.42:0/4075096776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c85d9b700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c88100e10 0x7f2c8819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8659c700 1 -- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c88100e10 msgr2=0x7f2c8819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8659c700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c88100e10 0x7f2c8819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8659c700 1 -- 172.21.15.42:0/4075096776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c780095f0 con 0x7f2c880ffc10 2024-09-16T23:50:27.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8659c700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c880ffc10 0x7f2c880690e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800bdd0 tx=0x7f2c7800be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c777fe700 1 -- 172.21.15.42:0/4075096776 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c7801d070 con 0x7f2c880ffc10 2024-09-16T23:50:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c777fe700 1 -- 172.21.15.42:0/4075096776 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2c7800f460 con 0x7f2c880ffc10 2024-09-16T23:50:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8819bbc0 con 0x7f2c880ffc10 2024-09-16T23:50:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.961+0000 7f2c777fe700 1 -- 172.21.15.42:0/4075096776 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c78021640 con 0x7f2c880ffc10 2024-09-16T23:50:27.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.962+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8819c0b0 con 0x7f2c880ffc10 2024-09-16T23:50:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.963+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c88108700 con 0x7f2c880ffc10 2024-09-16T23:50:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.963+0000 7f2c777fe700 1 -- 172.21.15.42:0/4075096776 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 30) v1 ==== 92506+0+0 (secure 0 0 0) 0x7f2c78004af0 con 0x7f2c880ffc10 2024-09-16T23:50:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.964+0000 7f2c777fe700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2c7006e430 0x7f2c700708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.964+0000 7f2c777fe700 1 -- 172.21.15.42:0/4075096776 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f2c7808f610 con 0x7f2c880ffc10 2024-09-16T23:50:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.964+0000 7f2c85d9b700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2c7006e430 0x7f2c700708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:27.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.964+0000 7f2c85d9b700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2c7006e430 0x7f2c700708f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2c88069660 tx=0x7f2c7c00a300 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:27.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:27.968+0000 7f2c777fe700 1 -- 172.21.15.42:0/4075096776 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f2c7805c670 con 0x7f2c880ffc10 2024-09-16T23:50:28.235 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-09-16T23:50:28.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-16T23:50:28.259 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:28 smithi043 ceph-mon[35191]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:28.259 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:28 smithi043 ceph-mon[35191]: pgmap v19: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.8 MiB/s wr, 238 op/s 2024-09-16T23:50:28.259 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:28 smithi043 ceph-mon[35191]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:28.259 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:28 smithi043 ceph-mon[35191]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:28.259 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:28 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3782327490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:28.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:28 smithi042 ceph-mon[27914]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:28.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:28 smithi042 ceph-mon[27914]: pgmap v19: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 1.8 MiB/s wr, 238 op/s 2024-09-16T23:50:28.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:28 smithi042 ceph-mon[27914]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:28 smithi042 ceph-mon[27914]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:28.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:28 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3782327490' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:28.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.284+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c88105760 con 0x7f2c7006e430 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.285+0000 7f2c777fe700 1 -- 172.21.15.42:0/4075096776 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f2c88105760 con 0x7f2c7006e430 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:50:28.283 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "2/2 daemons upgraded", 2024-09-16T23:50:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T23:50:28.284 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:50:28.284 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:50:28.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2c7006e430 msgr2=0x7f2c700708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:28.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2c7006e430 0x7f2c700708f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2c88069660 tx=0x7f2c7c00a300 comp rx=0 tx=0).stop 2024-09-16T23:50:28.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c880ffc10 msgr2=0x7f2c880690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:28.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c880ffc10 0x7f2c880690e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800bdd0 tx=0x7f2c7800be00 comp rx=0 tx=0).stop 2024-09-16T23:50:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 shutdown_connections 2024-09-16T23:50:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2c7006e430 0x7f2c700708f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c880ffc10 0x7f2c880690e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 --2- 172.21.15.42:0/4075096776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c88100e10 0x7f2c8819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 >> 172.21.15.42:0/4075096776 conn(0x7f2c880fb190 msgr2=0x7f2c88104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:28.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 shutdown_connections 2024-09-16T23:50:28.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:28.288+0000 7f2c8ce3a700 1 -- 172.21.15.42:0/4075096776 wait complete. 2024-09-16T23:50:28.591 DEBUG:teuthology.parallel:result is None 2024-09-16T23:50:29.900 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-16T23:50:29.900 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T23:50:30.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:30 smithi042 ceph-mon[27914]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:30.213 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:30 smithi042 ceph-mon[27914]: pgmap v20: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-09-16T23:50:30.214 DEBUG:teuthology.parallel:result is None 2024-09-16T23:50:30.214 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T23:50:30.258 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T23:50:30.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-16T23:50:30.259 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:30 smithi043 ceph-mon[35191]: from='client.24477 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:50:30.260 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:30 smithi043 ceph-mon[35191]: pgmap v20: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-09-16T23:50:30.297 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-16T23:50:30.297 DEBUG:teuthology.parallel:result is None 2024-09-16T23:50:31.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:31 smithi043 ceph-mon[35191]: pgmap v21: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-09-16T23:50:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:31 smithi042 ceph-mon[27914]: pgmap v21: 65 pgs: 65 active+clean; 326 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-09-16T23:50:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:33.738 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:33 smithi042 ceph-mon[27914]: pgmap v22: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.7 MiB/s wr, 269 op/s 2024-09-16T23:50:33.970 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:33 smithi043 ceph-mon[35191]: pgmap v22: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 43 KiB/s rd, 1.7 MiB/s wr, 269 op/s 2024-09-16T23:50:35.515 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:35.515 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:35.515 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:35 smithi043 ceph-mon[35191]: pgmap v23: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-09-16T23:50:35.515 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:35.515 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:35 smithi042 ceph-mon[27914]: pgmap v23: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-09-16T23:50:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:36 smithi042 ceph-mon[27914]: Standby manager daemon smithi043.xveyou restarted 2024-09-16T23:50:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:36 smithi042 ceph-mon[27914]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:50:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:36 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:50:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:36 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:50:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:36 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:50:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:36 smithi042 ceph-mon[27914]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:50:36.548 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:36 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou restarted 2024-09-16T23:50:36.549 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:36 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:50:36.549 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:36 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:50:36.549 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:36 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:50:36.549 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:36 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:50:36.549 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:36 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/2712779327' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:50:37.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:37 smithi043 ceph-mon[35191]: mgrmap e31: smithi042.zrkumz(active, since 41s), standbys: smithi043.xveyou 2024-09-16T23:50:37.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:37 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:37.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:37 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:37.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:37 smithi043 ceph-mon[35191]: pgmap v24: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-09-16T23:50:37.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:37 smithi042 ceph-mon[27914]: mgrmap e31: smithi042.zrkumz(active, since 41s), standbys: smithi043.xveyou 2024-09-16T23:50:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:37 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:37 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:37 smithi042 ceph-mon[27914]: pgmap v24: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 183 op/s 2024-09-16T23:50:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all mgr 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.zrkumz"}]: dispatch 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.zrkumz"}]': finished 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi043.xveyou"}]: dispatch 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi043.xveyou"}]': finished 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all rgw 2024-09-16T23:50:39.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all iscsi 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all nfs 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all nvmeof 2024-09-16T23:50:39.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all mgr 2024-09-16T23:50:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.zrkumz"}]: dispatch 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi042.zrkumz"}]': finished 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi043.xveyou"}]: dispatch 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi043.xveyou"}]': finished 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all rgw 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-16T23:50:39.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all iscsi 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all nfs 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all nvmeof 2024-09-16T23:50:39.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:40.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:40 smithi042 ceph-mon[27914]: pgmap v25: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 549 KiB/s wr, 91 op/s 2024-09-16T23:50:40.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:40 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:40.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:40 smithi043 ceph-mon[35191]: pgmap v25: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 549 KiB/s wr, 91 op/s 2024-09-16T23:50:40.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:40 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:41.489 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:41 smithi042 ceph-mon[27914]: Upgrade: Updating node-exporter.smithi042 (1/2) 2024-09-16T23:50:41.490 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:41 smithi042 ceph-mon[27914]: Deploying daemon node-exporter.smithi042 on smithi042 2024-09-16T23:50:41.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:41 smithi043 ceph-mon[35191]: Upgrade: Updating node-exporter.smithi042 (1/2) 2024-09-16T23:50:41.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:41 smithi043 ceph-mon[35191]: Deploying daemon node-exporter.smithi042 on smithi042 2024-09-16T23:50:42.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:42 smithi042 ceph-mon[27914]: pgmap v26: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 549 KiB/s wr, 91 op/s 2024-09-16T23:50:42.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:42 smithi043 ceph-mon[35191]: pgmap v26: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 549 KiB/s wr, 91 op/s 2024-09-16T23:50:43.650 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:43 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:43.651 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:43 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:43.651 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:43 smithi043 ceph-mon[35191]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 549 KiB/s wr, 91 op/s 2024-09-16T23:50:43.664 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:43 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:43.664 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:43 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:43.664 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:43 smithi042 ceph-mon[27914]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 549 KiB/s wr, 91 op/s 2024-09-16T23:50:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:44 smithi043 ceph-mon[35191]: Upgrade: Updating node-exporter.smithi043 (2/2) 2024-09-16T23:50:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:44 smithi043 ceph-mon[35191]: Deploying daemon node-exporter.smithi043 on smithi043 2024-09-16T23:50:44.672 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:44 smithi042 ceph-mon[27914]: Upgrade: Updating node-exporter.smithi043 (2/2) 2024-09-16T23:50:44.672 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:44 smithi042 ceph-mon[27914]: Deploying daemon node-exporter.smithi043 on smithi043 2024-09-16T23:50:45.687 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:45 smithi043 ceph-mon[35191]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 7.2 KiB/s wr, 3 op/s 2024-09-16T23:50:45.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:45 smithi042 ceph-mon[27914]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 7.2 KiB/s wr, 3 op/s 2024-09-16T23:50:47.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:47 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:47.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:47 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:47.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:47 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:47.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:47 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:47.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:47 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:47.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:47 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:48.292 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:48 smithi042 ceph-mon[27914]: pgmap v29: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 7.2 KiB/s wr, 3 op/s 2024-09-16T23:50:48.302 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:48 smithi043 ceph-mon[35191]: pgmap v29: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 7.2 KiB/s wr, 3 op/s 2024-09-16T23:50:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:49 smithi042 ceph-mon[27914]: pgmap v30: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:49 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:49 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:49 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:49 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:50.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:49 smithi043 ceph-mon[35191]: pgmap v30: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:49 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:49 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:49 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:49 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.258 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:52 smithi043 ceph-mon[35191]: pgmap v31: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:52.258 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:52 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.258 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:52 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.258 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:52 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.258 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:52 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.262 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:52 smithi042 ceph-mon[27914]: pgmap v31: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:52.262 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:52 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.262 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:52 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.262 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:52 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:52.262 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:52 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:53 smithi042 ceph-mon[27914]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:53 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:53 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:53 smithi043 ceph-mon[35191]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:53 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:53 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:55.731 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:55 smithi042 ceph-mon[27914]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:55.731 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:55.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:55 smithi043 ceph-mon[35191]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:55.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:50:57.954 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:57.954 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:57.954 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:57.955 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:57 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:57 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:50:58.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 -- 172.21.15.42:0/432297492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 msgr2=0x7fd814102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/432297492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814102300 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd804009910 tx=0x7fd804009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 -- 172.21.15.42:0/432297492 shutdown_connections 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/432297492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814102300 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/432297492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd814100ca0 0x7fd8141010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 -- 172.21.15.42:0/432297492 >> 172.21.15.42:0/432297492 conn(0x7fd8140fc260 msgr2=0x7fd8140fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 -- 172.21.15.42:0/432297492 shutdown_connections 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.399+0000 7fd81c0f7700 1 -- 172.21.15.42:0/432297492 wait complete. 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd81c0f7700 1 Processor -- start 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd81c0f7700 1 -- start start 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd81c0f7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd814100ca0 0x7fd814196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:58.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd81c0f7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd81c0f7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd814197080 con 0x7fd814101ea0 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd81c0f7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141971c0 con 0x7fd814100ca0 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd819692700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd819692700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44936/0 (socket says 172.21.15.42:44936) 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd819692700 1 -- 172.21.15.42:0/462899028 learned_addr learned my addr 172.21.15.42:0/462899028 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd819e93700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd814100ca0 0x7fd814196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.400+0000 7fd819692700 1 -- 172.21.15.42:0/462899028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd814100ca0 msgr2=0x7fd814196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd819692700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd814100ca0 0x7fd814196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd819692700 1 -- 172.21.15.42:0/462899028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040095f0 con 0x7fd814101ea0 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd819e93700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd814100ca0 0x7fd814196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd819692700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814196a60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd804006010 tx=0x7fd80400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd80affd700 1 -- 172.21.15.42:0/462899028 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd80401d070 con 0x7fd814101ea0 2024-09-16T23:50:58.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81419bc10 con 0x7fd814101ea0 2024-09-16T23:50:58.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd80affd700 1 -- 172.21.15.42:0/462899028 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd80400f460 con 0x7fd814101ea0 2024-09-16T23:50:58.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd80affd700 1 -- 172.21.15.42:0/462899028 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd804021640 con 0x7fd814101ea0 2024-09-16T23:50:58.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.401+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81419c100 con 0x7fd814101ea0 2024-09-16T23:50:58.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.402+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8141098a0 con 0x7fd814101ea0 2024-09-16T23:50:58.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.403+0000 7fd80affd700 1 -- 172.21.15.42:0/462899028 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 92506+0+0 (secure 0 0 0) 0x7fd80400fab0 con 0x7fd814101ea0 2024-09-16T23:50:58.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.403+0000 7fd80affd700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd80006e440 0x7fd800070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:58.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.403+0000 7fd80affd700 1 -- 172.21.15.42:0/462899028 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd80408f7e0 con 0x7fd814101ea0 2024-09-16T23:50:58.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.403+0000 7fd819e93700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd80006e440 0x7fd800070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:58.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.404+0000 7fd819e93700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd80006e440 0x7fd800070900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd814101d00 tx=0x7fd810009570 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:58.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.408+0000 7fd80affd700 1 -- 172.21.15.42:0/462899028 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fd80405c7c0 con 0x7fd814101ea0 2024-09-16T23:50:58.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.638+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8141067f0 con 0x7fd80006e440 2024-09-16T23:50:58.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.639+0000 7fd80affd700 1 -- 172.21.15.42:0/462899028 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fd8141067f0 con 0x7fd80006e440 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd80006e440 msgr2=0x7fd800070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd80006e440 0x7fd800070900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd814101d00 tx=0x7fd810009570 comp rx=0 tx=0).stop 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 msgr2=0x7fd814196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814196a60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd804006010 tx=0x7fd80400be20 comp rx=0 tx=0).stop 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 shutdown_connections 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fd80006e440 0x7fd800070900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd814100ca0 0x7fd814196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 --2- 172.21.15.42:0/462899028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101ea0 0x7fd814196a60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.641+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 >> 172.21.15.42:0/462899028 conn(0x7fd8140fc260 msgr2=0x7fd8141050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:58.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.642+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 shutdown_connections 2024-09-16T23:50:58.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.642+0000 7fd81c0f7700 1 -- 172.21.15.42:0/462899028 wait complete. 2024-09-16T23:50:58.649 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/1409218416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb40ffbc0 msgr2=0x7fcfb40fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/1409218416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb40ffbc0 0x7fcfb40fffe0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009910 tx=0x7fcfa4009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/1409218416 shutdown_connections 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/1409218416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfb4100dc0 0x7fcfb4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/1409218416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb40ffbc0 0x7fcfb40fffe0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/1409218416 >> 172.21.15.42:0/1409218416 conn(0x7fcfb40fb160 msgr2=0x7fcfb40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/1409218416 shutdown_connections 2024-09-16T23:50:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.750+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/1409218416 wait complete. 2024-09-16T23:50:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb8bb4700 1 Processor -- start 2024-09-16T23:50:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb8bb4700 1 -- start start 2024-09-16T23:50:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb8bb4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfb40ffbc0 0x7fcfb4069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb8bb4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb4100dc0 0x7fcfb419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb8bb4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb4197430 con 0x7fcfb4100dc0 2024-09-16T23:50:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb8bb4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb419df00 con 0x7fcfb40ffbc0 2024-09-16T23:50:58.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb4100dc0 0x7fcfb419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb4100dc0 0x7fcfb419d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44962/0 (socket says 172.21.15.42:44962) 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.751+0000 7fcfb1d9b700 1 -- 172.21.15.42:0/2095121797 learned_addr learned my addr 172.21.15.42:0/2095121797 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfb259c700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfb40ffbc0 0x7fcfb4069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfb1d9b700 1 -- 172.21.15.42:0/2095121797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfb40ffbc0 msgr2=0x7fcfb4069970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfb1d9b700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfb40ffbc0 0x7fcfb4069970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfb1d9b700 1 -- 172.21.15.42:0/2095121797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40095f0 con 0x7fcfb4100dc0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfb1d9b700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb4100dc0 0x7fcfb419d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b4e0 tx=0x7fcfa800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfa37fe700 1 -- 172.21.15.42:0/2095121797 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa8011800 con 0x7fcfb4100dc0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfa37fe700 1 -- 172.21.15.42:0/2095121797 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcfa8011e10 con 0x7fcfb4100dc0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb419e1b0 con 0x7fcfb4100dc0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfa37fe700 1 -- 172.21.15.42:0/2095121797 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa800f2e0 con 0x7fcfb4100dc0 2024-09-16T23:50:58.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.752+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb419e700 con 0x7fcfb4100dc0 2024-09-16T23:50:58.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.753+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb418f5b0 con 0x7fcfb4100dc0 2024-09-16T23:50:58.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.754+0000 7fcfa37fe700 1 -- 172.21.15.42:0/2095121797 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 92506+0+0 (secure 0 0 0) 0x7fcfa8010380 con 0x7fcfb4100dc0 2024-09-16T23:50:58.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.754+0000 7fcfa37fe700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcf9c06e440 0x7fcf9c070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:58.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.754+0000 7fcfa37fe700 1 -- 172.21.15.42:0/2095121797 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fcfa808dea0 con 0x7fcfb4100dc0 2024-09-16T23:50:58.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.754+0000 7fcfb259c700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcf9c06e440 0x7fcf9c070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:58.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.755+0000 7fcfb259c700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcf9c06e440 0x7fcf9c070900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4006010 tx=0x7fcfa400b580 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:58.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.759+0000 7fcfa37fe700 1 -- 172.21.15.42:0/2095121797 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fcfa805af90 con 0x7fcfb4100dc0 2024-09-16T23:50:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.992+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfb402cfb0 con 0x7fcf9c06e440 2024-09-16T23:50:58.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.993+0000 7fcfa37fe700 1 -- 172.21.15.42:0/2095121797 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fcfb402cfb0 con 0x7fcf9c06e440 2024-09-16T23:50:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.995+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcf9c06e440 msgr2=0x7fcf9c070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.995+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcf9c06e440 0x7fcf9c070900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4006010 tx=0x7fcfa400b580 comp rx=0 tx=0).stop 2024-09-16T23:50:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.995+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb4100dc0 msgr2=0x7fcfb419d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.995+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb4100dc0 0x7fcfb419d9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b4e0 tx=0x7fcfa800b8a0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.996+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 shutdown_connections 2024-09-16T23:50:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.996+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fcf9c06e440 0x7fcf9c070900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.996+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfb40ffbc0 0x7fcfb4069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.996+0000 7fcfb8bb4700 1 --2- 172.21.15.42:0/2095121797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfb4100dc0 0x7fcfb419d9c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.996+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 >> 172.21.15.42:0/2095121797 conn(0x7fcfb40fb160 msgr2=0x7fcfb41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.996+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 shutdown_connections 2024-09-16T23:50:58.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:58.996+0000 7fcfb8bb4700 1 -- 172.21.15.42:0/2095121797 wait complete. 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 -- 172.21.15.42:0/375859360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 msgr2=0x7fb85c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 --2- 172.21.15.42:0/375859360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c101270 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb850009910 tx=0x7fb850009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 -- 172.21.15.42:0/375859360 shutdown_connections 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 --2- 172.21.15.42:0/375859360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c101270 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 --2- 172.21.15.42:0/375859360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb85c0ffc10 0x7fb85c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 -- 172.21.15.42:0/375859360 >> 172.21.15.42:0/375859360 conn(0x7fb85c0fb190 msgr2=0x7fb85c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 -- 172.21.15.42:0/375859360 shutdown_connections 2024-09-16T23:50:59.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.106+0000 7fb860d24700 1 -- 172.21.15.42:0/375859360 wait complete. 2024-09-16T23:50:59.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.107+0000 7fb860d24700 1 Processor -- start 2024-09-16T23:50:59.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.107+0000 7fb860d24700 1 -- start start 2024-09-16T23:50:59.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.107+0000 7fb860d24700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb85c0ffc10 0x7fb85c195440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.107+0000 7fb860d24700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.107+0000 7fb860d24700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c195fa0 con 0x7fb85c100e10 2024-09-16T23:50:59.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.107+0000 7fb860d24700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c1960e0 con 0x7fb85c0ffc10 2024-09-16T23:50:59.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb859d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb859d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44970/0 (socket says 172.21.15.42:44970) 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb859d9b700 1 -- 172.21.15.42:0/2270349053 learned_addr learned my addr 172.21.15.42:0/2270349053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb85a59c700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb85c0ffc10 0x7fb85c195440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb859d9b700 1 -- 172.21.15.42:0/2270349053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb85c0ffc10 msgr2=0x7fb85c195440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb859d9b700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb85c0ffc10 0x7fb85c195440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb859d9b700 1 -- 172.21.15.42:0/2270349053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500095f0 con 0x7fb85c100e10 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb859d9b700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c195980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b5c0 tx=0x7fb85000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb84b7fe700 1 -- 172.21.15.42:0/2270349053 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb85001d070 con 0x7fb85c100e10 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb84b7fe700 1 -- 172.21.15.42:0/2270349053 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb850022470 con 0x7fb85c100e10 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c19ab30 con 0x7fb85c100e10 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb84b7fe700 1 -- 172.21.15.42:0/2270349053 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb85000f690 con 0x7fb85c100e10 2024-09-16T23:50:59.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.108+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c19b020 con 0x7fb85c100e10 2024-09-16T23:50:59.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.109+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c1087a0 con 0x7fb85c100e10 2024-09-16T23:50:59.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.110+0000 7fb84b7fe700 1 -- 172.21.15.42:0/2270349053 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 92506+0+0 (secure 0 0 0) 0x7fb850004b70 con 0x7fb85c100e10 2024-09-16T23:50:59.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.110+0000 7fb84b7fe700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb84406e430 0x7fb8440708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.111+0000 7fb84b7fe700 1 -- 172.21.15.42:0/2270349053 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb85008f9d0 con 0x7fb85c100e10 2024-09-16T23:50:59.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.111+0000 7fb85a59c700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb84406e430 0x7fb8440708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.111+0000 7fb85a59c700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb84406e430 0x7fb8440708f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb84c006fd0 tx=0x7fb84c008040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:59.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.115+0000 7fb84b7fe700 1 -- 172.21.15.42:0/2270349053 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fb85005cae0 con 0x7fb85c100e10 2024-09-16T23:50:59.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.332+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb85c105760 con 0x7fb84406e430 2024-09-16T23:50:59.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.347+0000 7fb84b7fe700 1 -- 172.21.15.42:0/2270349053 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fb85c105760 con 0x7fb84406e430 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (8m) 8s ago 10m 25.6M - 0.25.0 c8568f914cd2 f1be95930461 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (10m) 8s ago 10m 7830k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (9m) 10s ago 9m 7726k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (10m) 8s ago 10m 6665k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (9m) 10s ago 9m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (8m) 8s ago 10m 83.0M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (4m) 8s ago 4m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (4m) 8s ago 4m 14.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (4m) 10s ago 4m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:50:59.345 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (4m) 10s ago 4m 15.7M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (89s) 8s ago 11m 494M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (29s) 10s ago 9m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (11m) 8s ago 11m 53.3M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (9m) 10s ago 9m 47.4M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (17s) 8s ago 10m 7964k - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (13s) 10s ago 9m 3334k - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (8m) 8s ago 8m 203M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (8m) 8s ago 8m 200M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (7m) 8s ago 7m 162M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (7m) 10s ago 7m 233M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (6m) 10s ago 6m 219M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (6m) 10s ago 6m 201M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:50:59.346 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (44s) 8s ago 9m 50.5M - 2.43.0 a07b618ecd1d 1b14cac28905 2024-09-16T23:50:59.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb84406e430 msgr2=0x7fb8440708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb84406e430 0x7fb8440708f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb84c006fd0 tx=0x7fb84c008040 comp rx=0 tx=0).stop 2024-09-16T23:50:59.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 msgr2=0x7fb85c195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c195980 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b5c0 tx=0x7fb85000be20 comp rx=0 tx=0).stop 2024-09-16T23:50:59.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 shutdown_connections 2024-09-16T23:50:59.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb84406e430 0x7fb8440708f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb85c0ffc10 0x7fb85c195440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 --2- 172.21.15.42:0/2270349053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb85c100e10 0x7fb85c195980 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 >> 172.21.15.42:0/2270349053 conn(0x7fb85c0fb190 msgr2=0x7fb85c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:59.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 shutdown_connections 2024-09-16T23:50:59.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.349+0000 7fb860d24700 1 -- 172.21.15.42:0/2270349053 wait complete. 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 -- 172.21.15.42:0/2121825081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03880ffbc0 msgr2=0x7f03880fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 --2- 172.21.15.42:0/2121825081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03880ffbc0 0x7f03880fffe0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f037c009910 tx=0x7f037c009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 -- 172.21.15.42:0/2121825081 shutdown_connections 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 --2- 172.21.15.42:0/2121825081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0388100dc0 0x7f0388101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 --2- 172.21.15.42:0/2121825081 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03880ffbc0 0x7f03880fffe0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 -- 172.21.15.42:0/2121825081 >> 172.21.15.42:0/2121825081 conn(0x7f03880fb160 msgr2=0x7f03880fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 -- 172.21.15.42:0/2121825081 shutdown_connections 2024-09-16T23:50:59.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.458+0000 7f038eeb2700 1 -- 172.21.15.42:0/2121825081 wait complete. 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.459+0000 7f038eeb2700 1 Processor -- start 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.459+0000 7f038eeb2700 1 -- start start 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.459+0000 7f038eeb2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03880ffbc0 0x7f0388103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.459+0000 7f038eeb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0388100dc0 0x7f03881021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.459+0000 7f038eeb2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03881026e0 con 0x7f0388100dc0 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.459+0000 7f038eeb2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0388102820 con 0x7f03880ffbc0 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0387fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0388100dc0 0x7f03881021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0387fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0388100dc0 0x7f03881021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44974/0 (socket says 172.21.15.42:44974) 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0387fff700 1 -- 172.21.15.42:0/3514201609 learned_addr learned my addr 172.21.15.42:0/3514201609 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f038cc4e700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03880ffbc0 0x7f0388103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0387fff700 1 -- 172.21.15.42:0/3514201609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03880ffbc0 msgr2=0x7f0388103b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0387fff700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03880ffbc0 0x7f0388103b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0387fff700 1 -- 172.21.15.42:0/3514201609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c0095f0 con 0x7f0388100dc0 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0387fff700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0388100dc0 0x7f03881021a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f037400d8d0 tx=0x7f037400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0385ffb700 1 -- 172.21.15.42:0/3514201609 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03740098a0 con 0x7f0388100dc0 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0385ffb700 1 -- 172.21.15.42:0/3514201609 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0374010460 con 0x7f0388100dc0 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0388102b00 con 0x7f0388100dc0 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.460+0000 7f0385ffb700 1 -- 172.21.15.42:0/3514201609 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f037400f5f0 con 0x7f0388100dc0 2024-09-16T23:50:59.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.461+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038819e850 con 0x7f0388100dc0 2024-09-16T23:50:59.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.462+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0388068af0 con 0x7f0388100dc0 2024-09-16T23:50:59.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.462+0000 7f0385ffb700 1 -- 172.21.15.42:0/3514201609 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 92506+0+0 (secure 0 0 0) 0x7f0374009a00 con 0x7f0388100dc0 2024-09-16T23:50:59.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.463+0000 7f0385ffb700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f037806e440 0x7f0378070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.463+0000 7f0385ffb700 1 -- 172.21.15.42:0/3514201609 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f037408e210 con 0x7f0388100dc0 2024-09-16T23:50:59.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.463+0000 7f038cc4e700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f037806e440 0x7f0378070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.463+0000 7f038cc4e700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f037806e440 0x7f0378070900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f037c006010 tx=0x7f037c005ec0 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:59.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.472+0000 7f0385ffb700 1 -- 172.21.15.42:0/3514201609 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f037405b1f0 con 0x7f0388100dc0 2024-09-16T23:50:59.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.772+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f038819e9e0 con 0x7f0388100dc0 2024-09-16T23:50:59.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.773+0000 7f0385ffb700 1 -- 172.21.15.42:0/3514201609 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f037405ad80 con 0x7f0388100dc0 2024-09-16T23:50:59.771 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:50:59.771 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:50:59.772 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:50:59.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:59.772 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:50:59.772 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:50:59.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:59.772 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:50:59.773 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:50:59.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:59.773 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:50:59.773 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:50:59.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:50:59.773 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:50:59.774 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-09-16T23:50:59.774 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:50:59.774 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:50:59.774 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:50:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f037806e440 msgr2=0x7f0378070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f037806e440 0x7f0378070900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f037c006010 tx=0x7f037c005ec0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0388100dc0 msgr2=0x7f03881021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0388100dc0 0x7f03881021a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f037400d8d0 tx=0x7f037400dbe0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 shutdown_connections 2024-09-16T23:50:59.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f037806e440 0x7f0378070900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03880ffbc0 0x7f0388103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 --2- 172.21.15.42:0/3514201609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0388100dc0 0x7f03881021a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 >> 172.21.15.42:0/3514201609 conn(0x7f03880fb160 msgr2=0x7f0388068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 shutdown_connections 2024-09-16T23:50:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.775+0000 7f038eeb2700 1 -- 172.21.15.42:0/3514201609 wait complete. 2024-09-16T23:50:59.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 -- 172.21.15.42:0/3943710531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 msgr2=0x7fb78c1064f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 --2- 172.21.15.42:0/3943710531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c1064f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb774009910 tx=0x7fb774009c20 comp rx=0 tx=0).stop 2024-09-16T23:50:59.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 -- 172.21.15.42:0/3943710531 shutdown_connections 2024-09-16T23:50:59.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 --2- 172.21.15.42:0/3943710531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c1064f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 --2- 172.21.15.42:0/3943710531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb78c1017d0 0x7fb78c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 -- 172.21.15.42:0/3943710531 >> 172.21.15.42:0/3943710531 conn(0x7fb78c0fb160 msgr2=0x7fb78c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:50:59.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 -- 172.21.15.42:0/3943710531 shutdown_connections 2024-09-16T23:50:59.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.886+0000 7fb791681700 1 -- 172.21.15.42:0/3943710531 wait complete. 2024-09-16T23:50:59.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb791681700 1 Processor -- start 2024-09-16T23:50:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb791681700 1 -- start start 2024-09-16T23:50:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb791681700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb78c1017d0 0x7fb78c1967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb791681700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb791681700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c197300 con 0x7fb78c104100 2024-09-16T23:50:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb791681700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c197440 con 0x7fb78c1017d0 2024-09-16T23:50:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb78a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb78a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44986/0 (socket says 172.21.15.42:44986) 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb78a7fc700 1 -- 172.21.15.42:0/2380575888 learned_addr learned my addr 172.21.15.42:0/2380575888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.887+0000 7fb78affd700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb78c1017d0 0x7fb78c1967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb78a7fc700 1 -- 172.21.15.42:0/2380575888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb78c1017d0 msgr2=0x7fb78c1967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb78a7fc700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb78c1017d0 0x7fb78c1967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb78a7fc700 1 -- 172.21.15.42:0/2380575888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740095f0 con 0x7fb78c104100 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb78a7fc700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c196ce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb774006010 tx=0x7fb77400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb783fff700 1 -- 172.21.15.42:0/2380575888 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77401d070 con 0x7fb78c104100 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c19be90 con 0x7fb78c104100 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb783fff700 1 -- 172.21.15.42:0/2380575888 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb77400f460 con 0x7fb78c104100 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb783fff700 1 -- 172.21.15.42:0/2380575888 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb774021640 con 0x7fb78c104100 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.888+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c19c350 con 0x7fb78c104100 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.889+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c190630 con 0x7fb78c104100 2024-09-16T23:50:59.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.890+0000 7fb783fff700 1 -- 172.21.15.42:0/2380575888 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 31) v1 ==== 92506+0+0 (secure 0 0 0) 0x7fb77400fab0 con 0x7fb78c104100 2024-09-16T23:50:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.890+0000 7fb783fff700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb77806e440 0x7fb778070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:50:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.890+0000 7fb783fff700 1 -- 172.21.15.42:0/2380575888 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb77408f6e0 con 0x7fb78c104100 2024-09-16T23:50:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.890+0000 7fb78affd700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb77806e440 0x7fb778070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:50:59.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.891+0000 7fb78affd700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb77806e440 0x7fb778070900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0060b0 tx=0x7fb77c00c040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:50:59.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:50:59.895+0000 7fb783fff700 1 -- 172.21.15.42:0/2380575888 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fb77405c6c0 con 0x7fb78c104100 2024-09-16T23:51:00.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:59 smithi042 ceph-mon[27914]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:00.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:59 smithi042 ceph-mon[27914]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:00.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:59 smithi042 ceph-mon[27914]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:00.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:50:59 smithi042 ceph-mon[27914]: Upgrade: Updating prometheus.smithi042 2024-09-16T23:51:00.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.129+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb78c02cf70 con 0x7fb77806e440 2024-09-16T23:51:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:59 smithi043 ceph-mon[35191]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:59 smithi043 ceph-mon[35191]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:59 smithi043 ceph-mon[35191]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:50:59 smithi043 ceph-mon[35191]: Upgrade: Updating prometheus.smithi042 2024-09-16T23:51:00.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.130+0000 7fb783fff700 1 -- 172.21.15.42:0/2380575888 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7fb78c02cf70 con 0x7fb77806e440 2024-09-16T23:51:00.128 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:51:00.128 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:51:00.128 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:51:00.128 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:51:00.128 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:51:00.129 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:51:00.129 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:51:00.129 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "2/2 daemons upgraded", 2024-09-16T23:51:00.129 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading prometheus daemons", 2024-09-16T23:51:00.129 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:51:00.129 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:51:00.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb77806e440 msgr2=0x7fb778070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:00.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb77806e440 0x7fb778070900 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0060b0 tx=0x7fb77c00c040 comp rx=0 tx=0).stop 2024-09-16T23:51:00.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 msgr2=0x7fb78c196ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c196ce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb774006010 tx=0x7fb77400bd70 comp rx=0 tx=0).stop 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 shutdown_connections 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb77806e440 0x7fb778070900 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb78c1017d0 0x7fb78c1967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 --2- 172.21.15.42:0/2380575888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb78c104100 0x7fb78c196ce0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 >> 172.21.15.42:0/2380575888 conn(0x7fb78c0fb160 msgr2=0x7fb78c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 shutdown_connections 2024-09-16T23:51:00.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:00.133+0000 7fb791681700 1 -- 172.21.15.42:0/2380575888 wait complete. 2024-09-16T23:51:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:00 smithi042 ceph-mon[27914]: Deploying daemon prometheus.smithi042 on smithi042 2024-09-16T23:51:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:00 smithi042 ceph-mon[27914]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:00 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/3514201609' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:00 smithi042 ceph-mon[27914]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:00 smithi043 ceph-mon[35191]: Deploying daemon prometheus.smithi042 on smithi042 2024-09-16T23:51:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:00 smithi043 ceph-mon[35191]: from='client.14726 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:00 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/3514201609' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:00 smithi043 ceph-mon[35191]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:01 smithi042 ceph-mon[27914]: pgmap v36: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:02.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:01 smithi043 ceph-mon[35191]: pgmap v36: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:03.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:03 smithi042 ceph-mon[27914]: pgmap v37: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:03 smithi043 ceph-mon[35191]: pgmap v37: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:06.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:05 smithi042 ceph-mon[27914]: pgmap v38: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:06.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:05 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:06.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:05 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:06.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:05 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:05 smithi043 ceph-mon[35191]: pgmap v38: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:05 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:05 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:05 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:08.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:07 smithi042 ceph-mon[27914]: pgmap v39: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:08.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:07 smithi043 ceph-mon[35191]: pgmap v39: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:10.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:09 smithi042 ceph-mon[27914]: pgmap v40: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:10.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:09 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:10.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:09 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:09 smithi043 ceph-mon[35191]: pgmap v40: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:09 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:09 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:11.177 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:10 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:11.177 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:10 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:11.177 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:10 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:11.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:10 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:11.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:10 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:11.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:10 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:12.190 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:11 smithi042 ceph-mon[27914]: pgmap v41: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:12.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:11 smithi043 ceph-mon[35191]: pgmap v41: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:14.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:13 smithi042 ceph-mon[27914]: pgmap v42: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:14.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:13 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:14.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:13 smithi043 ceph-mon[35191]: pgmap v42: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:14.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:13 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:15.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:14 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:15.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:15.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:14 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:16.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:15 smithi042 ceph-mon[27914]: pgmap v43: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:15 smithi043 ceph-mon[35191]: pgmap v43: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:17.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:16 smithi042 ceph-mon[27914]: Upgrade: Updating alertmanager.smithi042 2024-09-16T23:51:17.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:16 smithi042 ceph-mon[27914]: Deploying daemon alertmanager.smithi042 on smithi042 2024-09-16T23:51:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:16 smithi043 ceph-mon[35191]: Upgrade: Updating alertmanager.smithi042 2024-09-16T23:51:17.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:16 smithi043 ceph-mon[35191]: Deploying daemon alertmanager.smithi042 on smithi042 2024-09-16T23:51:18.200 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:17 smithi042 ceph-mon[27914]: pgmap v44: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:18.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:17 smithi043 ceph-mon[35191]: pgmap v44: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:20.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:19 smithi042 ceph-mon[27914]: pgmap v45: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:19 smithi043 ceph-mon[35191]: pgmap v45: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:21 smithi043 ceph-mon[35191]: pgmap v46: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:21 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:21 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:21 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:21 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:21 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:22.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:21 smithi042 ceph-mon[27914]: pgmap v46: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:21 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:21 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:21 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:21 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:21 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:24.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:23 smithi042 ceph-mon[27914]: pgmap v47: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:24.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:23 smithi043 ceph-mon[35191]: pgmap v47: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:26.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:25 smithi042 ceph-mon[27914]: pgmap v48: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:26.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:25 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:26.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:25 smithi043 ceph-mon[35191]: pgmap v48: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:26.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:25 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:27.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:26 smithi042 ceph-mon[27914]: mgrmap e32: smithi042.zrkumz(active, since 91s), standbys: smithi043.xveyou 2024-09-16T23:51:27.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:26 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:27.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:26 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:27.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:26 smithi043 ceph-mon[35191]: mgrmap e32: smithi042.zrkumz(active, since 91s), standbys: smithi043.xveyou 2024-09-16T23:51:27.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:26 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:27.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:26 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:28 smithi043 ceph-mon[35191]: pgmap v49: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:28 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:28 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:28.492 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:28 smithi042 ceph-mon[27914]: pgmap v49: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:28.492 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:28 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:28.492 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:28 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:30.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2568750225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a4100dc0 msgr2=0x7fa4a4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2568750225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a4100dc0 0x7fa4a4101220 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa490009910 tx=0x7fa490009c20 comp rx=0 tx=0).stop 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2568750225 shutdown_connections 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2568750225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a4100dc0 0x7fa4a4101220 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2568750225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a40ffbc0 0x7fa4a40fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2568750225 >> 172.21.15.42:0/2568750225 conn(0x7fa4a40fb160 msgr2=0x7fa4a40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2568750225 shutdown_connections 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2568750225 wait complete. 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.244+0000 7fa4aacc5700 1 Processor -- start 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4aacc5700 1 -- start start 2024-09-16T23:51:30.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4aacc5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a40ffbc0 0x7fa4a41967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4aacc5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a4100dc0 0x7fa4a4196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4aacc5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a4197320 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4aacc5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a4197460 con 0x7fa4a4100dc0 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a8a61700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a40ffbc0 0x7fa4a41967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a8a61700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a40ffbc0 0x7fa4a41967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45440/0 (socket says 172.21.15.42:45440) 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a8a61700 1 -- 172.21.15.42:0/2288378474 learned_addr learned my addr 172.21.15.42:0/2288378474 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a3fff700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a4100dc0 0x7fa4a4196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a8a61700 1 -- 172.21.15.42:0/2288378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a4100dc0 msgr2=0x7fa4a4196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a8a61700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a4100dc0 0x7fa4a4196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a8a61700 1 -- 172.21.15.42:0/2288378474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900095f0 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.245+0000 7fa4a8a61700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a40ffbc0 0x7fa4a41967c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d900 tx=0x7fa49800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.246+0000 7fa4a1ffb700 1 -- 172.21.15.42:0/2288378474 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4980041f0 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.246+0000 7fa4a1ffb700 1 -- 172.21.15.42:0/2288378474 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa498004350 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.246+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a419bf10 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.246+0000 7fa4a1ffb700 1 -- 172.21.15.42:0/2288378474 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa498003da0 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.246+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a419c3d0 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.247+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a4190630 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.247+0000 7fa4a1ffb700 1 -- 172.21.15.42:0/2288378474 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7fa498009730 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.248+0000 7fa4a1ffb700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa49406e300 0x7fa4940707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.248+0000 7fa4a1ffb700 1 -- 172.21.15.42:0/2288378474 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fa49808e1f0 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.248+0000 7fa4a3fff700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa49406e300 0x7fa4940707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.248+0000 7fa4a3fff700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa49406e300 0x7fa4940707c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa490005b40 tx=0x7fa490005a90 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:30.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.253+0000 7fa4a1ffb700 1 -- 172.21.15.42:0/2288378474 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fa49805b3f0 con 0x7fa4a40ffbc0 2024-09-16T23:51:30.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:30 smithi042 ceph-mon[27914]: pgmap v50: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:30.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:30 smithi043 ceph-mon[35191]: pgmap v50: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:30.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.481+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4a40611d0 con 0x7fa49406e300 2024-09-16T23:51:30.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.482+0000 7fa4a1ffb700 1 -- 172.21.15.42:0/2288378474 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fa4a40611d0 con 0x7fa49406e300 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa49406e300 msgr2=0x7fa4940707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa49406e300 0x7fa4940707c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa490005b40 tx=0x7fa490005a90 comp rx=0 tx=0).stop 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a40ffbc0 msgr2=0x7fa4a41967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a40ffbc0 0x7fa4a41967c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa49800d900 tx=0x7fa49800dcc0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 shutdown_connections 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4a40ffbc0 0x7fa4a41967c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa49406e300 0x7fa4940707c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 --2- 172.21.15.42:0/2288378474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a4100dc0 0x7fa4a4196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 >> 172.21.15.42:0/2288378474 conn(0x7fa4a40fb160 msgr2=0x7fa4a4104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 shutdown_connections 2024-09-16T23:51:30.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.484+0000 7fa4aacc5700 1 -- 172.21.15.42:0/2288378474 wait complete. 2024-09-16T23:51:30.493 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:51:30.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 -- 172.21.15.42:0/2712119118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80103f30 msgr2=0x7f7c80106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/2712119118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80103f30 0x7f7c80106320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009910 tx=0x7f7c74009c20 comp rx=0 tx=0).stop 2024-09-16T23:51:30.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 -- 172.21.15.42:0/2712119118 shutdown_connections 2024-09-16T23:51:30.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/2712119118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80103f30 0x7f7c80106320 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/2712119118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c80101600 0x7f7c801039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 -- 172.21.15.42:0/2712119118 >> 172.21.15.42:0/2712119118 conn(0x7f7c800fafd0 msgr2=0x7f7c800fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:30.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 -- 172.21.15.42:0/2712119118 shutdown_connections 2024-09-16T23:51:30.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 -- 172.21.15.42:0/2712119118 wait complete. 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.593+0000 7f7c87ef5700 1 Processor -- start 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c87ef5700 1 -- start start 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c87ef5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80101600 0x7f7c80100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c87ef5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c80103f30 0x7f7c800ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c87ef5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800ffaf0 con 0x7f7c80101600 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c87ef5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800ffc30 con 0x7f7c80103f30 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85c91700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80101600 0x7f7c80100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85c91700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80101600 0x7f7c80100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45456/0 (socket says 172.21.15.42:45456) 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85c91700 1 -- 172.21.15.42:0/3599270217 learned_addr learned my addr 172.21.15.42:0/3599270217 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85490700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c80103f30 0x7f7c800ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85c91700 1 -- 172.21.15.42:0/3599270217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c80103f30 msgr2=0x7f7c800ff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85c91700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c80103f30 0x7f7c800ff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85c91700 1 -- 172.21.15.42:0/3599270217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740095f0 con 0x7f7c80101600 2024-09-16T23:51:30.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.594+0000 7f7c85c91700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80101600 0x7f7c80100f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000ba50 tx=0x7f7c7000bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:30.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.595+0000 7f7c7effd700 1 -- 172.21.15.42:0/3599270217 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c7000c900 con 0x7f7c80101600 2024-09-16T23:51:30.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.595+0000 7f7c7effd700 1 -- 172.21.15.42:0/3599270217 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7c70014440 con 0x7f7c80101600 2024-09-16T23:51:30.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.595+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c800fff10 con 0x7f7c80101600 2024-09-16T23:51:30.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.595+0000 7f7c7effd700 1 -- 172.21.15.42:0/3599270217 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c700075a0 con 0x7f7c80101600 2024-09-16T23:51:30.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.595+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8019e3d0 con 0x7f7c80101600 2024-09-16T23:51:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.596+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c80190560 con 0x7f7c80101600 2024-09-16T23:51:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.596+0000 7f7c7effd700 1 -- 172.21.15.42:0/3599270217 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f7c700145b0 con 0x7f7c80101600 2024-09-16T23:51:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.597+0000 7f7c7effd700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f7c6806e300 0x7f7c680707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.597+0000 7f7c7effd700 1 -- 172.21.15.42:0/3599270217 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f7c70006e40 con 0x7f7c80101600 2024-09-16T23:51:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.597+0000 7f7c85490700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f7c6806e300 0x7f7c680707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.597+0000 7f7c85490700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f7c6806e300 0x7f7c680707c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c74006010 tx=0x7f7c7400b560 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:30.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.602+0000 7f7c7effd700 1 -- 172.21.15.42:0/3599270217 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f7c7005aa80 con 0x7f7c80101600 2024-09-16T23:51:30.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.832+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c800611d0 con 0x7f7c6806e300 2024-09-16T23:51:30.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.833+0000 7f7c7effd700 1 -- 172.21.15.42:0/3599270217 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f7c800611d0 con 0x7f7c6806e300 2024-09-16T23:51:30.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f7c6806e300 msgr2=0x7f7c680707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f7c6806e300 0x7f7c680707c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c74006010 tx=0x7f7c7400b560 comp rx=0 tx=0).stop 2024-09-16T23:51:30.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80101600 msgr2=0x7f7c80100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80101600 0x7f7c80100f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000ba50 tx=0x7f7c7000bd60 comp rx=0 tx=0).stop 2024-09-16T23:51:30.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 shutdown_connections 2024-09-16T23:51:30.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7c80101600 0x7f7c80100f60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f7c6806e300 0x7f7c680707c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 --2- 172.21.15.42:0/3599270217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c80103f30 0x7f7c800ff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 >> 172.21.15.42:0/3599270217 conn(0x7f7c800fafd0 msgr2=0x7f7c800fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:30.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 shutdown_connections 2024-09-16T23:51:30.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.835+0000 7f7c87ef5700 1 -- 172.21.15.42:0/3599270217 wait complete. 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 -- 172.21.15.42:0/2013157889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 msgr2=0x7f6ba4102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/2013157889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba4102310 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009910 tx=0x7f6b8c009c20 comp rx=0 tx=0).stop 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 -- 172.21.15.42:0/2013157889 shutdown_connections 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/2013157889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba4102310 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/2013157889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4100cb0 0x7f6ba41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 -- 172.21.15.42:0/2013157889 >> 172.21.15.42:0/2013157889 conn(0x7f6ba40fc230 msgr2=0x7f6ba40fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 -- 172.21.15.42:0/2013157889 shutdown_connections 2024-09-16T23:51:30.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.957+0000 7f6baa3a9700 1 -- 172.21.15.42:0/2013157889 wait complete. 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6baa3a9700 1 Processor -- start 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6baa3a9700 1 -- start start 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6baa3a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4100cb0 0x7f6ba41964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6baa3a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba41969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6baa3a9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4197010 con 0x7f6ba4101eb0 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6baa3a9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4197150 con 0x7f6ba4100cb0 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6ba37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba41969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6ba37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba41969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45466/0 (socket says 172.21.15.42:45466) 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6ba37fe700 1 -- 172.21.15.42:0/3943010401 learned_addr learned my addr 172.21.15.42:0/3943010401 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.958+0000 7f6ba3fff700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4100cb0 0x7f6ba41964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6ba37fe700 1 -- 172.21.15.42:0/3943010401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4100cb0 msgr2=0x7f6ba41964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6ba37fe700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4100cb0 0x7f6ba41964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6ba37fe700 1 -- 172.21.15.42:0/3943010401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8c0095f0 con 0x7f6ba4101eb0 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6ba37fe700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba41969f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c005b40 tx=0x7f6b8c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6ba17fa700 1 -- 172.21.15.42:0/3943010401 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b8c01d070 con 0x7f6ba4101eb0 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6ba17fa700 1 -- 172.21.15.42:0/3943010401 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b8c022470 con 0x7f6ba4101eb0 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba419bba0 con 0x7f6ba4101eb0 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6ba17fa700 1 -- 172.21.15.42:0/3943010401 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b8c00f690 con 0x7f6ba4101eb0 2024-09-16T23:51:30.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.959+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba419c090 con 0x7f6ba4101eb0 2024-09-16T23:51:30.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.960+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba4109810 con 0x7f6ba4101eb0 2024-09-16T23:51:30.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.961+0000 7f6ba17fa700 1 -- 172.21.15.42:0/3943010401 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f6b8c004af0 con 0x7f6ba4101eb0 2024-09-16T23:51:30.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.961+0000 7f6ba17fa700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f6b9006e300 0x7f6b900707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:30.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.961+0000 7f6ba17fa700 1 -- 172.21.15.42:0/3943010401 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f6b8c08f670 con 0x7f6ba4101eb0 2024-09-16T23:51:30.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.961+0000 7f6ba3fff700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f6b9006e300 0x7f6b900707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:30.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.962+0000 7f6ba3fff700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f6b9006e300 0x7f6b900707c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4101d10 tx=0x7f6b9400a380 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:30.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:30.966+0000 7f6ba17fa700 1 -- 172.21.15.42:0/3943010401 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f6b8c05c920 con 0x7f6ba4101eb0 2024-09-16T23:51:31.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.208+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ba4106800 con 0x7f6b9006e300 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.217+0000 7f6ba17fa700 1 -- 172.21.15.42:0/3943010401 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f6ba4106800 con 0x7f6b9006e300 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (10s) 4s ago 10m 15.1M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (11m) 4s ago 11m 7876k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (10m) 42s ago 10m 7726k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (11m) 4s ago 11m 6665k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (10m) 42s ago 10m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:51:31.215 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (9m) 4s ago 10m 84.0M - 9.4.7 954c08fa6188 f3ae051e67ef 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (5m) 4s ago 5m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (5m) 4s ago 5m 14.7M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (5m) 42s ago 5m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (5m) 42s ago 5m 15.7M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (2m) 4s ago 12m 496M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (61s) 42s ago 9m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (12m) 4s ago 12m 54.6M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (9m) 42s ago 9m 47.4M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (49s) 4s ago 11m 7805k - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (45s) 42s ago 9m 3334k - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (9m) 4s ago 9m 204M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (8m) 4s ago 8m 200M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (8m) 4s ago 8m 163M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:51:31.216 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (7m) 42s ago 7m 233M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:51:31.217 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (7m) 42s ago 7m 219M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:51:31.217 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (6m) 42s ago 6m 201M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:51:31.217 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (26s) 4s ago 10m 51.3M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:51:31.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.219+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f6b9006e300 msgr2=0x7f6b900707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.219+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f6b9006e300 0x7f6b900707c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4101d10 tx=0x7f6b9400a380 comp rx=0 tx=0).stop 2024-09-16T23:51:31.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.219+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 msgr2=0x7f6ba41969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.219+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba41969f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c005b40 tx=0x7f6b8c00bd70 comp rx=0 tx=0).stop 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.219+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 shutdown_connections 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.219+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f6b9006e300 0x7f6b900707c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.220+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4100cb0 0x7f6ba41964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.220+0000 7f6baa3a9700 1 --2- 172.21.15.42:0/3943010401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4101eb0 0x7f6ba41969f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.220+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 >> 172.21.15.42:0/3943010401 conn(0x7f6ba40fc230 msgr2=0x7f6ba41050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.220+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 shutdown_connections 2024-09-16T23:51:31.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.220+0000 7f6baa3a9700 1 -- 172.21.15.42:0/3943010401 wait complete. 2024-09-16T23:51:31.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 -- 172.21.15.42:0/3968875218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 msgr2=0x7f2fa8106520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 --2- 172.21.15.42:0/3968875218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa8106520 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009910 tx=0x7f2f98009c20 comp rx=0 tx=0).stop 2024-09-16T23:51:31.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 -- 172.21.15.42:0/3968875218 shutdown_connections 2024-09-16T23:51:31.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 --2- 172.21.15.42:0/3968875218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa8106520 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 --2- 172.21.15.42:0/3968875218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8101800 0x7f2fa8103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 -- 172.21.15.42:0/3968875218 >> 172.21.15.42:0/3968875218 conn(0x7f2fa80fb190 msgr2=0x7f2fa80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:31.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 -- 172.21.15.42:0/3968875218 shutdown_connections 2024-09-16T23:51:31.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.331+0000 7f2faef2d700 1 -- 172.21.15.42:0/3968875218 wait complete. 2024-09-16T23:51:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2faef2d700 1 Processor -- start 2024-09-16T23:51:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2faef2d700 1 -- start start 2024-09-16T23:51:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2faef2d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8101800 0x7f2fa8101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2faef2d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa80ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2faef2d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa819a150 con 0x7f2fa8104130 2024-09-16T23:51:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2faef2d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa819a290 con 0x7f2fa8101800 2024-09-16T23:51:31.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2f9ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa80ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2f9ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa80ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45478/0 (socket says 172.21.15.42:45478) 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.332+0000 7f2f9ffff700 1 -- 172.21.15.42:0/2717303351 learned_addr learned my addr 172.21.15.42:0/2717303351 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2faccc9700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8101800 0x7f2fa8101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2f9ffff700 1 -- 172.21.15.42:0/2717303351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8101800 msgr2=0x7f2fa8101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2f9ffff700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8101800 0x7f2fa8101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2f9ffff700 1 -- 172.21.15.42:0/2717303351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980095f0 con 0x7f2fa8104130 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2faccc9700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8101800 0x7f2fa8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2f9ffff700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa80ff7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2f98006010 tx=0x7f2f9800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2f9dffb700 1 -- 172.21.15.42:0/2717303351 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f9801d070 con 0x7f2fa8104130 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2f9dffb700 1 -- 172.21.15.42:0/2717303351 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2f9800f460 con 0x7f2fa8104130 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa80ffcf0 con 0x7f2fa8104130 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.333+0000 7f2f9dffb700 1 -- 172.21.15.42:0/2717303351 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f98021640 con 0x7f2fa8104130 2024-09-16T23:51:31.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.334+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa81001b0 con 0x7f2fa8104130 2024-09-16T23:51:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.335+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa8190690 con 0x7f2fa8104130 2024-09-16T23:51:31.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.336+0000 7f2f9dffb700 1 -- 172.21.15.42:0/2717303351 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f2f98004b20 con 0x7f2fa8104130 2024-09-16T23:51:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.336+0000 7f2f9dffb700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2f9006e300 0x7f2f900707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.337+0000 7f2f9dffb700 1 -- 172.21.15.42:0/2717303351 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f2f9808f4a0 con 0x7f2fa8104130 2024-09-16T23:51:31.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.337+0000 7f2faccc9700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2f9006e300 0x7f2f900707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:31.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.337+0000 7f2faccc9700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2f9006e300 0x7f2f900707c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007900 tx=0x7f2fa4008040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:31.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.341+0000 7f2f9dffb700 1 -- 172.21.15.42:0/2717303351 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f2f9805c720 con 0x7f2fa8104130 2024-09-16T23:51:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.649+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2fa802cd80 con 0x7f2fa8104130 2024-09-16T23:51:31.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.650+0000 7f2f9dffb700 1 -- 172.21.15.42:0/2717303351 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f2f9805c2b0 con 0x7f2fa8104130 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:51:31.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:51:31.649 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2f9006e300 msgr2=0x7f2f900707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2f9006e300 0x7f2f900707c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4007900 tx=0x7f2fa4008040 comp rx=0 tx=0).stop 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 msgr2=0x7f2fa80ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa80ff7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2f98006010 tx=0x7f2f9800be20 comp rx=0 tx=0).stop 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 shutdown_connections 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f2f9006e300 0x7f2f900707c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2fa8101800 0x7f2fa8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 --2- 172.21.15.42:0/2717303351 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fa8104130 0x7f2fa80ff7b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 >> 172.21.15.42:0/2717303351 conn(0x7f2fa80fb190 msgr2=0x7f2fa80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 shutdown_connections 2024-09-16T23:51:31.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.652+0000 7f2faef2d700 1 -- 172.21.15.42:0/2717303351 wait complete. 2024-09-16T23:51:31.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 -- 172.21.15.42:0/452161187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 msgr2=0x7f51941023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 --2- 172.21.15.42:0/452161187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51941023b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5184009910 tx=0x7f5184009c20 comp rx=0 tx=0).stop 2024-09-16T23:51:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 -- 172.21.15.42:0/452161187 shutdown_connections 2024-09-16T23:51:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 --2- 172.21.15.42:0/452161187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51941023b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 --2- 172.21.15.42:0/452161187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940fef70 0x7f51940ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 -- 172.21.15.42:0/452161187 >> 172.21.15.42:0/452161187 conn(0x7f51940fa8b0 msgr2=0x7f51940fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 -- 172.21.15.42:0/452161187 shutdown_connections 2024-09-16T23:51:31.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 -- 172.21.15.42:0/452161187 wait complete. 2024-09-16T23:51:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.762+0000 7f519b539700 1 Processor -- start 2024-09-16T23:51:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f519b539700 1 -- start start 2024-09-16T23:51:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f519b539700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940fef70 0x7f51941018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f519b539700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51940fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f5198ad4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51940fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:31.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f5198ad4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51940fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45498/0 (socket says 172.21.15.42:45498) 2024-09-16T23:51:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f5198ad4700 1 -- 172.21.15.42:0/3503873727 learned_addr learned my addr 172.21.15.42:0/3503873727 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:51:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5194100440 con 0x7f5194101f50 2024-09-16T23:51:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.763+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5194100580 con 0x7f51940fef70 2024-09-16T23:51:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f51992d5700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940fef70 0x7f51941018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f5198ad4700 1 -- 172.21.15.42:0/3503873727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940fef70 msgr2=0x7f51941018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f5198ad4700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940fef70 0x7f51941018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:31.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f5198ad4700 1 -- 172.21.15.42:0/3503873727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51840095f0 con 0x7f5194101f50 2024-09-16T23:51:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f51992d5700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940fef70 0x7f51941018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:51:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f5198ad4700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51940fff00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5184006010 tx=0x7f518400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f518a7fc700 1 -- 172.21.15.42:0/3503873727 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f518401d070 con 0x7f5194101f50 2024-09-16T23:51:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.764+0000 7f518a7fc700 1 -- 172.21.15.42:0/3503873727 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f518400f460 con 0x7f5194101f50 2024-09-16T23:51:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.765+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5194100800 con 0x7f5194101f50 2024-09-16T23:51:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.765+0000 7f518a7fc700 1 -- 172.21.15.42:0/3503873727 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5184021640 con 0x7f5194101f50 2024-09-16T23:51:31.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.765+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5194100cf0 con 0x7f5194101f50 2024-09-16T23:51:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.767+0000 7f518a7fc700 1 -- 172.21.15.42:0/3503873727 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f518400fab0 con 0x7f5194101f50 2024-09-16T23:51:31.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.767+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51941098b0 con 0x7f5194101f50 2024-09-16T23:51:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.768+0000 7f518a7fc700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f518006e300 0x7f51800707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:51:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.768+0000 7f518a7fc700 1 -- 172.21.15.42:0/3503873727 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f518408f470 con 0x7f5194101f50 2024-09-16T23:51:31.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.768+0000 7f51992d5700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f518006e300 0x7f51800707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:51:31.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.768+0000 7f51992d5700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f518006e300 0x7f51800707c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5194068c90 tx=0x7f5190008040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:51:31.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:31.772+0000 7f518a7fc700 1 -- 172.21.15.42:0/3503873727 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f518405c690 con 0x7f5194101f50 2024-09-16T23:51:32.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.006+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51941068a0 con 0x7f518006e300 2024-09-16T23:51:32.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.007+0000 7f518a7fc700 1 -- 172.21.15.42:0/3503873727 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f51941068a0 con 0x7f518006e300 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "2/2 daemons upgraded", 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading grafana daemons", 2024-09-16T23:51:32.005 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:51:32.006 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f518006e300 msgr2=0x7f51800707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f518006e300 0x7f51800707c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5194068c90 tx=0x7f5190008040 comp rx=0 tx=0).stop 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 msgr2=0x7f51940fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51940fff00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5184006010 tx=0x7f518400be20 comp rx=0 tx=0).stop 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 shutdown_connections 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f518006e300 0x7f51800707c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51940fef70 0x7f51941018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 --2- 172.21.15.42:0/3503873727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5194101f50 0x7f51940fff00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:51:32.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 >> 172.21.15.42:0/3503873727 conn(0x7f51940fa8b0 msgr2=0x7f5194105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:51:32.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 shutdown_connections 2024-09-16T23:51:32.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:51:32.009+0000 7f519b539700 1 -- 172.21.15.42:0/3503873727 wait complete. 2024-09-16T23:51:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: pgmap v51: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:32.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:31 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2717303351' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: pgmap v51: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:32.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:32.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:31 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2717303351' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:33.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:32 smithi042 ceph-mon[27914]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:33.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:32 smithi043 ceph-mon[35191]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:51:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:33 smithi042 ceph-mon[27914]: pgmap v52: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:33 smithi042 ceph-mon[27914]: Upgrade: Updating grafana.smithi042 2024-09-16T23:51:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:33 smithi042 ceph-mon[27914]: Deploying daemon grafana.smithi042 on smithi042 2024-09-16T23:51:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:33 smithi043 ceph-mon[35191]: pgmap v52: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:33 smithi043 ceph-mon[35191]: Upgrade: Updating grafana.smithi042 2024-09-16T23:51:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:33 smithi043 ceph-mon[35191]: Deploying daemon grafana.smithi042 on smithi042 2024-09-16T23:51:36.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:35 smithi042 ceph-mon[27914]: pgmap v53: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:36.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:35 smithi043 ceph-mon[35191]: pgmap v53: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:38.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:37 smithi042 ceph-mon[27914]: pgmap v54: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:38.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:37 smithi043 ceph-mon[35191]: pgmap v54: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:51:40.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:39 smithi043 ceph-mon[35191]: pgmap v55: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:40.146 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:39 smithi042 ceph-mon[27914]: pgmap v55: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:41.244 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:40 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:41.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:40 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:42.217 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:41 smithi042 ceph-mon[27914]: pgmap v56: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:42.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:41 smithi043 ceph-mon[35191]: pgmap v56: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:44.145 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:43 smithi042 ceph-mon[27914]: pgmap v57: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:44.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:43 smithi043 ceph-mon[35191]: pgmap v57: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:45.726 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:45 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:45.727 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:45 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:45.727 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:45 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:45.727 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:45 smithi042 ceph-mon[27914]: pgmap v58: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:45 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:45 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:45 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:45 smithi043 ceph-mon[35191]: pgmap v58: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:47.986 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:47 smithi042 ceph-mon[27914]: pgmap v59: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:47 smithi043 ceph-mon[35191]: pgmap v59: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:50.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:49 smithi042 ceph-mon[27914]: pgmap v60: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:49 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:49 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:49 smithi043 ceph-mon[35191]: pgmap v60: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:49 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:49 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:51.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:51 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:51.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:51 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:51 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:51 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:52.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:52 smithi042 ceph-mon[27914]: pgmap v61: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:52.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:52 smithi043 ceph-mon[35191]: pgmap v61: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:54.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: pgmap v62: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:54.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-16T23:51:54.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.527 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.527 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.527 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.527 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.527 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-16T23:51:54.527 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T23:51:54.527 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:54 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: pgmap v62: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-16T23:51:54.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.631 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.632 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.632 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.632 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.632 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-16T23:51:54.632 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-16T23:51:54.632 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T23:51:54.632 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:54 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:51:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: Upgrade: Finalizing container_image settings 2024-09-16T23:51:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: Upgrade: Complete! 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:55.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:55 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: Upgrade: Finalizing container_image settings 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: Upgrade: Complete! 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:51:55.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:55.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:51:55.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:55 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:51:56.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:56 smithi042 ceph-mon[27914]: pgmap v63: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:56.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:56 smithi043 ceph-mon[35191]: pgmap v63: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:58.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:51:58 smithi042 ceph-mon[27914]: pgmap v64: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:51:58.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:51:58 smithi043 ceph-mon[35191]: pgmap v64: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:00.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:00 smithi043 ceph-mon[35191]: pgmap v65: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:00.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:00 smithi042 ceph-mon[27914]: pgmap v65: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 -- 172.21.15.42:0/1485168107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 msgr2=0x7f87b40ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/1485168107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b40ff9b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009910 tx=0x7f87b0009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 -- 172.21.15.42:0/1485168107 shutdown_connections 2024-09-16T23:52:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/1485168107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b40ffef0 0x7f87b4100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/1485168107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b40ff9b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 -- 172.21.15.42:0/1485168107 >> 172.21.15.42:0/1485168107 conn(0x7f87b40fb190 msgr2=0x7f87b40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:02.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 -- 172.21.15.42:0/1485168107 shutdown_connections 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.132+0000 7f87bb4bc700 1 -- 172.21.15.42:0/1485168107 wait complete. 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87bb4bc700 1 Processor -- start 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87bb4bc700 1 -- start start 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87bb4bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87bb4bc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b40ffef0 0x7f87b4198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87bb4bc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b41992e0 con 0x7f87b40ff590 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87bb4bc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4199420 con 0x7f87b40ffef0 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87b9258700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87b9258700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b4198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44800/0 (socket says 172.21.15.42:44800) 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.133+0000 7f87b9258700 1 -- 172.21.15.42:0/960933142 learned_addr learned my addr 172.21.15.42:0/960933142 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:02.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87b8a57700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b40ffef0 0x7f87b4198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87b9258700 1 -- 172.21.15.42:0/960933142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b40ffef0 msgr2=0x7f87b4198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87b9258700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b40ffef0 0x7f87b4198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87b9258700 1 -- 172.21.15.42:0/960933142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00095f0 con 0x7f87b40ff590 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87b9258700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b4198780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005b40 tx=0x7f87b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87aa7fc700 1 -- 172.21.15.42:0/960933142 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b001d070 con 0x7f87b40ff590 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87aa7fc700 1 -- 172.21.15.42:0/960933142 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87b000f460 con 0x7f87b40ff590 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b419de70 con 0x7f87b40ff590 2024-09-16T23:52:02.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87aa7fc700 1 -- 172.21.15.42:0/960933142 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87b0021640 con 0x7f87b40ff590 2024-09-16T23:52:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.134+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b419e2e0 con 0x7f87b40ff590 2024-09-16T23:52:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.135+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b410ba70 con 0x7f87b40ff590 2024-09-16T23:52:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.136+0000 7f87aa7fc700 1 -- 172.21.15.42:0/960933142 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f87b000f5d0 con 0x7f87b40ff590 2024-09-16T23:52:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.136+0000 7f87aa7fc700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f87a006e300 0x7f87a00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.136+0000 7f87aa7fc700 1 -- 172.21.15.42:0/960933142 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f87b008f3d0 con 0x7f87b40ff590 2024-09-16T23:52:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.137+0000 7f87b8a57700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f87a006e300 0x7f87a00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:02.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.137+0000 7f87b8a57700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f87a006e300 0x7f87a00707c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009a10 tx=0x7f87a4009190 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:02.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.141+0000 7f87aa7fc700 1 -- 172.21.15.42:0/960933142 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f87b005c650 con 0x7f87b40ff590 2024-09-16T23:52:02.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.376+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87b41092e0 con 0x7f87a006e300 2024-09-16T23:52:02.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.377+0000 7f87aa7fc700 1 -- 172.21.15.42:0/960933142 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f87b41092e0 con 0x7f87a006e300 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f87a006e300 msgr2=0x7f87a00707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f87a006e300 0x7f87a00707c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009a10 tx=0x7f87a4009190 comp rx=0 tx=0).stop 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 msgr2=0x7f87b4198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b4198780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005b40 tx=0x7f87b000bd70 comp rx=0 tx=0).stop 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 shutdown_connections 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f87b40ff590 0x7f87b4198780 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f87a006e300 0x7f87a00707c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 --2- 172.21.15.42:0/960933142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b40ffef0 0x7f87b4198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:02.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 >> 172.21.15.42:0/960933142 conn(0x7f87b40fb190 msgr2=0x7f87b4107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:02.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 shutdown_connections 2024-09-16T23:52:02.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:02.379+0000 7f87bb4bc700 1 -- 172.21.15.42:0/960933142 wait complete. 2024-09-16T23:52:02.389 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:02 smithi042 ceph-mon[27914]: pgmap v66: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:02 smithi043 ceph-mon[35191]: pgmap v66: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:03.140 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-09-16T23:52:03.393 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:04.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:04 smithi042 ceph-mon[27914]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:04.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:04 smithi042 ceph-mon[27914]: pgmap v67: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:04 smithi043 ceph-mon[35191]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:04 smithi043 ceph-mon[35191]: pgmap v67: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:06.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:06 smithi042 ceph-mon[27914]: pgmap v68: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:06.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:06 smithi043 ceph-mon[35191]: pgmap v68: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 -- 172.21.15.42:0/2449885345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 msgr2=0x7fb3ac100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 --2- 172.21.15.42:0/2449885345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac100080 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009910 tx=0x7fb39c009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 -- 172.21.15.42:0/2449885345 shutdown_connections 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 --2- 172.21.15.42:0/2449885345 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac100080 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 --2- 172.21.15.42:0/2449885345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3ac105bd0 0x7fb3ac105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 -- 172.21.15.42:0/2449885345 >> 172.21.15.42:0/2449885345 conn(0x7fb3ac075670 msgr2=0x7fb3ac075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 -- 172.21.15.42:0/2449885345 shutdown_connections 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.576+0000 7fb3b343b700 1 -- 172.21.15.42:0/2449885345 wait complete. 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b343b700 1 Processor -- start 2024-09-16T23:52:07.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b343b700 1 -- start start 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b343b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b343b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3ac105bd0 0x7fb3ac19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b343b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac19f580 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b343b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac198a30 con 0x7fb3ac105bd0 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b11d7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b11d7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac19e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36302/0 (socket says 172.21.15.42:36302) 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b11d7700 1 -- 172.21.15.42:0/344408879 learned_addr learned my addr 172.21.15.42:0/344408879 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.577+0000 7fb3b09d6700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3ac105bd0 0x7fb3ac19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3b11d7700 1 -- 172.21.15.42:0/344408879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3ac105bd0 msgr2=0x7fb3ac19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3b11d7700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3ac105bd0 0x7fb3ac19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:07.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3b11d7700 1 -- 172.21.15.42:0/344408879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0095f0 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3b11d7700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac19e9b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b4b0 tx=0x7fb3a800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3a27fc700 1 -- 172.21.15.42:0/344408879 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a8011800 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3a27fc700 1 -- 172.21.15.42:0/344408879 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3a8011e20 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac198d10 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3a27fc700 1 -- 172.21.15.42:0/344408879 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a800f2e0 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.578+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac199260 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.580+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac04f2e0 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.580+0000 7fb3a27fc700 1 -- 172.21.15.42:0/344408879 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7fb3a8010380 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.580+0000 7fb3a27fc700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb39806e300 0x7fb3980707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.581+0000 7fb3a27fc700 1 -- 172.21.15.42:0/344408879 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb3a808dad0 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.581+0000 7fb3b09d6700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb39806e300 0x7fb3980707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:07.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.581+0000 7fb3b09d6700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb39806e300 0x7fb3980707c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac075f90 tx=0x7fb39c01a040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:07.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.585+0000 7fb3a27fc700 1 -- 172.21.15.42:0/344408879 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fb3a805ad50 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.889+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb3ac199530 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.889+0000 7fb3a27fc700 1 -- 172.21.15.42:0/344408879 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fb3a805a8e0 con 0x7fb3ac0ffc20 2024-09-16T23:52:07.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb39806e300 msgr2=0x7fb3980707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:07.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb39806e300 0x7fb3980707c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac075f90 tx=0x7fb39c01a040 comp rx=0 tx=0).stop 2024-09-16T23:52:07.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 msgr2=0x7fb3ac19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:07.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac19e9b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b4b0 tx=0x7fb3a800b7c0 comp rx=0 tx=0).stop 2024-09-16T23:52:07.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 shutdown_connections 2024-09-16T23:52:07.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3ac0ffc20 0x7fb3ac19e9b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb39806e300 0x7fb3980707c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 --2- 172.21.15.42:0/344408879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3ac105bd0 0x7fb3ac19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 >> 172.21.15.42:0/344408879 conn(0x7fb3ac075670 msgr2=0x7fb3ac0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 shutdown_connections 2024-09-16T23:52:07.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:07.891+0000 7fb3b343b700 1 -- 172.21.15.42:0/344408879 wait complete. 2024-09-16T23:52:07.900 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:52:08.511 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:08 smithi042 ceph-mon[27914]: pgmap v69: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:08.511 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:08 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/344408879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:08.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-09-16T23:52:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:08 smithi043 ceph-mon[35191]: pgmap v69: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:08 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/344408879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:08.743 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:10.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:10 smithi042 ceph-mon[27914]: pgmap v70: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:10.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:10 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 -- 172.21.15.42:0/1514511527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 msgr2=0x7fa9600ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1514511527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa9600ff7a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa948009910 tx=0x7fa948009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 -- 172.21.15.42:0/1514511527 shutdown_connections 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1514511527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa9600ff7a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1514511527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960105350 0x7fa960105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 -- 172.21.15.42:0/1514511527 >> 172.21.15.42:0/1514511527 conn(0x7fa960074e30 msgr2=0x7fa960075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 -- 172.21.15.42:0/1514511527 shutdown_connections 2024-09-16T23:52:10.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.352+0000 7fa964ebb700 1 -- 172.21.15.42:0/1514511527 wait complete. 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa964ebb700 1 Processor -- start 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa964ebb700 1 -- start start 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa964ebb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa960100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa964ebb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960105350 0x7fa960101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa964ebb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960101a40 con 0x7fa9600ff340 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa964ebb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960101b80 con 0x7fa960105350 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa95e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa960100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa95e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa960100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36318/0 (socket says 172.21.15.42:36318) 2024-09-16T23:52:10.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa95e59c700 1 -- 172.21.15.42:0/1739321335 learned_addr learned my addr 172.21.15.42:0/1739321335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.353+0000 7fa955d9b700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960105350 0x7fa960101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa95e59c700 1 -- 172.21.15.42:0/1739321335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960105350 msgr2=0x7fa960101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa95e59c700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960105350 0x7fa960101500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa95e59c700 1 -- 172.21.15.42:0/1739321335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480095f0 con 0x7fa9600ff340 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa95e59c700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa960100fc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa95000c930 tx=0x7fa95000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa957fff700 1 -- 172.21.15.42:0/1739321335 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa950007ad0 con 0x7fa9600ff340 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9601a2a20 con 0x7fa9600ff340 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa957fff700 1 -- 172.21.15.42:0/1739321335 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa95000ce80 con 0x7fa9600ff340 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa957fff700 1 -- 172.21.15.42:0/1739321335 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa950018630 con 0x7fa9600ff340 2024-09-16T23:52:10.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.354+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9601a2d80 con 0x7fa9600ff340 2024-09-16T23:52:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.355+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa960109860 con 0x7fa9600ff340 2024-09-16T23:52:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.356+0000 7fa957fff700 1 -- 172.21.15.42:0/1739321335 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7fa95000f450 con 0x7fa9600ff340 2024-09-16T23:52:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.356+0000 7fa957fff700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa94c06e300 0x7fa94c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.356+0000 7fa957fff700 1 -- 172.21.15.42:0/1739321335 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fa95008e0a0 con 0x7fa9600ff340 2024-09-16T23:52:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.356+0000 7fa955d9b700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa94c06e300 0x7fa94c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:10.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.357+0000 7fa955d9b700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa94c06e300 0x7fa94c0707c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9601027b0 tx=0x7fa94801a040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:10.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.361+0000 7fa957fff700 1 -- 172.21.15.42:0/1739321335 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fa950057770 con 0x7fa9600ff340 2024-09-16T23:52:10.430 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:10 smithi043 ceph-mon[35191]: pgmap v70: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:10.430 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:10 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:10.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.653+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa960109240 con 0x7fa9600ff340 2024-09-16T23:52:10.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.654+0000 7fa957fff700 1 -- 172.21.15.42:0/1739321335 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fa95005ae30 con 0x7fa9600ff340 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa94c06e300 msgr2=0x7fa94c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa94c06e300 0x7fa94c0707c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9601027b0 tx=0x7fa94801a040 comp rx=0 tx=0).stop 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 msgr2=0x7fa960100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa960100fc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa95000c930 tx=0x7fa95000cc40 comp rx=0 tx=0).stop 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 shutdown_connections 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9600ff340 0x7fa960100fc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fa94c06e300 0x7fa94c0707c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 --2- 172.21.15.42:0/1739321335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa960105350 0x7fa960101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 >> 172.21.15.42:0/1739321335 conn(0x7fa960074e30 msgr2=0x7fa960103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 shutdown_connections 2024-09-16T23:52:10.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:10.656+0000 7fa964ebb700 1 -- 172.21.15.42:0/1739321335 wait complete. 2024-09-16T23:52:10.664 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)" 2024-09-16T23:52:11.204 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:11 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/1739321335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:11.230 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-09-16T23:52:11.444 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:11.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:11 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/1739321335' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:12.462 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:12 smithi042 ceph-mon[27914]: pgmap v71: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:12 smithi043 ceph-mon[35191]: pgmap v71: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.030+0000 7f50aaeb7700 1 -- 172.21.15.42:0/3121925280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 msgr2=0x7f50a4105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.030+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/3121925280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a4105890 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f508c009910 tx=0x7f508c009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.030+0000 7f50aaeb7700 1 -- 172.21.15.42:0/3121925280 shutdown_connections 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.030+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/3121925280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a4105890 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.030+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/3121925280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4100e90 0x7f50a4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.031+0000 7f50aaeb7700 1 -- 172.21.15.42:0/3121925280 >> 172.21.15.42:0/3121925280 conn(0x7f50a40fc740 msgr2=0x7f50a40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.031+0000 7f50aaeb7700 1 -- 172.21.15.42:0/3121925280 shutdown_connections 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.031+0000 7f50aaeb7700 1 -- 172.21.15.42:0/3121925280 wait complete. 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.031+0000 7f50aaeb7700 1 Processor -- start 2024-09-16T23:52:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.031+0000 7f50aaeb7700 1 -- start start 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50aaeb7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4100e90 0x7f50a419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50aaeb7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50aaeb7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a419d2e0 con 0x7f50a4101840 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50aaeb7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4196790 con 0x7f50a4100e90 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57152/0 (socket says 172.21.15.42:57152) 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a3fff700 1 -- 172.21.15.42:0/87613120 learned_addr learned my addr 172.21.15.42:0/87613120 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a8c53700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4100e90 0x7f50a419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a3fff700 1 -- 172.21.15.42:0/87613120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4100e90 msgr2=0x7f50a419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:13.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a3fff700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4100e90 0x7f50a419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a3fff700 1 -- 172.21.15.42:0/87613120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0095f0 con 0x7f50a4101840 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a3fff700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a419cc50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f508c009fd0 tx=0x7f508c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a1ffb700 1 -- 172.21.15.42:0/87613120 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f508c01d070 con 0x7f50a4101840 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a4196a10 con 0x7f50a4101840 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a1ffb700 1 -- 172.21.15.42:0/87613120 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f508c022470 con 0x7f50a4101840 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.032+0000 7f50a1ffb700 1 -- 172.21.15.42:0/87613120 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f508c017650 con 0x7f50a4101840 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.033+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a4196ea0 con 0x7f50a4101840 2024-09-16T23:52:13.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.033+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a4109290 con 0x7f50a4101840 2024-09-16T23:52:13.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.034+0000 7f50a1ffb700 1 -- 172.21.15.42:0/87613120 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f508c0225e0 con 0x7f50a4101840 2024-09-16T23:52:13.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.035+0000 7f50a1ffb700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f509006e300 0x7f50900707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:13.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.035+0000 7f50a1ffb700 1 -- 172.21.15.42:0/87613120 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f508c08f6a0 con 0x7f50a4101840 2024-09-16T23:52:13.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.035+0000 7f50a8c53700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f509006e300 0x7f50900707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:13.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.035+0000 7f50a8c53700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f509006e300 0x7f50900707c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5094009c60 tx=0x7f5094009260 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:13.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.039+0000 7f50a1ffb700 1 -- 172.21.15.42:0/87613120 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f508c05c800 con 0x7f50a4101840 2024-09-16T23:52:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.336+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f50a404f2e0 con 0x7f50a4101840 2024-09-16T23:52:13.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.337+0000 7f50a1ffb700 1 -- 172.21.15.42:0/87613120 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f508c05c390 con 0x7f50a4101840 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f509006e300 msgr2=0x7f50900707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f509006e300 0x7f50900707c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5094009c60 tx=0x7f5094009260 comp rx=0 tx=0).stop 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 msgr2=0x7f50a419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a419cc50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f508c009fd0 tx=0x7f508c00f690 comp rx=0 tx=0).stop 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 shutdown_connections 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f509006e300 0x7f50900707c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50a4100e90 0x7f50a419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 --2- 172.21.15.42:0/87613120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a4101840 0x7f50a419cc50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:13.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 >> 172.21.15.42:0/87613120 conn(0x7f50a40fc740 msgr2=0x7f50a40fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 shutdown_connections 2024-09-16T23:52:13.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:13.339+0000 7f50aaeb7700 1 -- 172.21.15.42:0/87613120 wait complete. 2024-09-16T23:52:13.348 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:52:14.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-09-16T23:52:14.231 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:14.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:14 smithi042 ceph-mon[27914]: pgmap v72: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:14.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:14 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/87613120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:14 smithi043 ceph-mon[35191]: pgmap v72: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:14 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/87613120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:15.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 -- 172.21.15.42:0/2330852718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06440ffc20 msgr2=0x7f0644100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:15.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 --2- 172.21.15.42:0/2330852718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06440ffc20 0x7f0644100080 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0638009910 tx=0x7f0638009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:15.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 -- 172.21.15.42:0/2330852718 shutdown_connections 2024-09-16T23:52:15.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 --2- 172.21.15.42:0/2330852718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06440ffc20 0x7f0644100080 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:15.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 --2- 172.21.15.42:0/2330852718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0644105bd0 0x7f0644105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:15.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 -- 172.21.15.42:0/2330852718 >> 172.21.15.42:0/2330852718 conn(0x7f0644075670 msgr2=0x7f0644075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:15.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 -- 172.21.15.42:0/2330852718 shutdown_connections 2024-09-16T23:52:15.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.831+0000 7f0648feb700 1 -- 172.21.15.42:0/2330852718 wait complete. 2024-09-16T23:52:15.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.832+0000 7f0648feb700 1 Processor -- start 2024-09-16T23:52:15.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.832+0000 7f0648feb700 1 -- start start 2024-09-16T23:52:15.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.832+0000 7f0648feb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440ffc20 0x7f0644197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.832+0000 7f0648feb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0644105bd0 0x7f0644197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.832+0000 7f0648feb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0644198290 con 0x7f0644105bd0 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.832+0000 7f0648feb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441983d0 con 0x7f06440ffc20 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0641d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0644105bd0 0x7f0644197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0641d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0644105bd0 0x7f0644197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57176/0 (socket says 172.21.15.42:57176) 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0641d9b700 1 -- 172.21.15.42:0/3140369020 learned_addr learned my addr 172.21.15.42:0/3140369020 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f064259c700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440ffc20 0x7f0644197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0641d9b700 1 -- 172.21.15.42:0/3140369020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440ffc20 msgr2=0x7f0644197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0641d9b700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440ffc20 0x7f0644197730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0641d9b700 1 -- 172.21.15.42:0/3140369020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380095f0 con 0x7f0644105bd0 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0641d9b700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0644105bd0 0x7f0644197c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0638005f50 tx=0x7f063800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f06337fe700 1 -- 172.21.15.42:0/3140369020 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f063801d070 con 0x7f0644105bd0 2024-09-16T23:52:15.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06441a07a0 con 0x7f0644105bd0 2024-09-16T23:52:15.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f06337fe700 1 -- 172.21.15.42:0/3140369020 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f063800b610 con 0x7f0644105bd0 2024-09-16T23:52:15.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f06337fe700 1 -- 172.21.15.42:0/3140369020 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0638022470 con 0x7f0644105bd0 2024-09-16T23:52:15.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.833+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441a0bb0 con 0x7f0644105bd0 2024-09-16T23:52:15.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.834+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06441080d0 con 0x7f0644105bd0 2024-09-16T23:52:15.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.835+0000 7f06337fe700 1 -- 172.21.15.42:0/3140369020 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f063800faf0 con 0x7f0644105bd0 2024-09-16T23:52:15.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.836+0000 7f06337fe700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f062c06e300 0x7f062c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:15.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.836+0000 7f06337fe700 1 -- 172.21.15.42:0/3140369020 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f063808f490 con 0x7f0644105bd0 2024-09-16T23:52:15.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.836+0000 7f064259c700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f062c06e300 0x7f062c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:15.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.836+0000 7f064259c700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f062c06e300 0x7f062c0707c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0634009c60 tx=0x7f0634009260 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:15.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:15.840+0000 7f06337fe700 1 -- 172.21.15.42:0/3140369020 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f0638058b60 con 0x7f0644105bd0 2024-09-16T23:52:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:16.068+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "target": ["mon-mgr", ""]}) v1 -- 0x7f0644068c50 con 0x7f062c06e300 2024-09-16T23:52:16.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:16 smithi042 ceph-mon[27914]: pgmap v73: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:16.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:16 smithi043 ceph-mon[35191]: pgmap v73: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:17.376 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:17 smithi042 ceph-mon[27914]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:17.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:17 smithi043 ceph-mon[35191]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:17.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.926+0000 7f06337fe700 1 -- 172.21.15.42:0/3140369020 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5393 (secure 0 0 0) 0x7f0644068c50 con 0x7f062c06e300 2024-09-16T23:52:17.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.929+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f062c06e300 msgr2=0x7f062c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:17.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.929+0000 7f0648feb700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f062c06e300 0x7f062c0707c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0634009c60 tx=0x7f0634009260 comp rx=0 tx=0).stop 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.929+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0644105bd0 msgr2=0x7f0644197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.929+0000 7f0648feb700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0644105bd0 0x7f0644197c70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0638005f50 tx=0x7f063800f690 comp rx=0 tx=0).stop 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.930+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 shutdown_connections 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.930+0000 7f0648feb700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f062c06e300 0x7f062c0707c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.930+0000 7f0648feb700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06440ffc20 0x7f0644197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.930+0000 7f0648feb700 1 --2- 172.21.15.42:0/3140369020 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0644105bd0 0x7f0644197c70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.930+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 >> 172.21.15.42:0/3140369020 conn(0x7f0644075670 msgr2=0x7f06440fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.930+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 shutdown_connections 2024-09-16T23:52:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:17.930+0000 7f0648feb700 1 -- 172.21.15.42:0/3140369020 wait complete. 2024-09-16T23:52:17.940 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:52:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:18 smithi042 ceph-mon[27914]: pgmap v74: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:18 smithi043 ceph-mon[35191]: pgmap v74: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:18.657 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph orch ps' 2024-09-16T23:52:18.874 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:20.436 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:20 smithi042 ceph-mon[27914]: pgmap v75: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:20.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:20 smithi043 ceph-mon[35191]: pgmap v75: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:20.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.677+0000 7fb978a12700 1 -- 172.21.15.42:0/2406305690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 msgr2=0x7fb974100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:20.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.677+0000 7fb978a12700 1 --2- 172.21.15.42:0/2406305690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb974100180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb964009910 tx=0x7fb964009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:20.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.677+0000 7fb978a12700 1 -- 172.21.15.42:0/2406305690 shutdown_connections 2024-09-16T23:52:20.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.677+0000 7fb978a12700 1 --2- 172.21.15.42:0/2406305690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974100750 0x7fb9741047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:20.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.677+0000 7fb978a12700 1 --2- 172.21.15.42:0/2406305690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb974100180 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:20.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.677+0000 7fb978a12700 1 -- 172.21.15.42:0/2406305690 >> 172.21.15.42:0/2406305690 conn(0x7fb9740fb650 msgr2=0x7fb9740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:20.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.678+0000 7fb978a12700 1 -- 172.21.15.42:0/2406305690 shutdown_connections 2024-09-16T23:52:20.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.678+0000 7fb978a12700 1 -- 172.21.15.42:0/2406305690 wait complete. 2024-09-16T23:52:20.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.678+0000 7fb978a12700 1 Processor -- start 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.678+0000 7fb978a12700 1 -- start start 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb978a12700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb97419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb978a12700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974100750 0x7fb97419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb978a12700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb974196850 con 0x7fb9740ffda0 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb978a12700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9741969c0 con 0x7fb974100750 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb97259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb97419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb97259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb97419c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57194/0 (socket says 172.21.15.42:57194) 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb97259c700 1 -- 172.21.15.42:0/3990264947 learned_addr learned my addr 172.21.15.42:0/3990264947 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb971d9b700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974100750 0x7fb97419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb97259c700 1 -- 172.21.15.42:0/3990264947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974100750 msgr2=0x7fb97419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb97259c700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974100750 0x7fb97419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:20.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb97259c700 1 -- 172.21.15.42:0/3990264947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9640095f0 con 0x7fb9740ffda0 2024-09-16T23:52:20.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb97259c700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb97419c860 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb964006010 tx=0x7fb96400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:20.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.679+0000 7fb9637fe700 1 -- 172.21.15.42:0/3990264947 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb96401d070 con 0x7fb9740ffda0 2024-09-16T23:52:20.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.680+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb974196c40 con 0x7fb9740ffda0 2024-09-16T23:52:20.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.680+0000 7fb9637fe700 1 -- 172.21.15.42:0/3990264947 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb96400f460 con 0x7fb9740ffda0 2024-09-16T23:52:20.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.680+0000 7fb9637fe700 1 -- 172.21.15.42:0/3990264947 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb964017610 con 0x7fb9740ffda0 2024-09-16T23:52:20.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.680+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb974197130 con 0x7fb9740ffda0 2024-09-16T23:52:20.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.680+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97404f2e0 con 0x7fb9740ffda0 2024-09-16T23:52:20.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.681+0000 7fb9637fe700 1 -- 172.21.15.42:0/3990264947 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7fb96400f5d0 con 0x7fb9740ffda0 2024-09-16T23:52:20.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.682+0000 7fb9637fe700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb95c06e300 0x7fb95c0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:20.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.682+0000 7fb9637fe700 1 -- 172.21.15.42:0/3990264947 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb96408f650 con 0x7fb9740ffda0 2024-09-16T23:52:20.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.682+0000 7fb971d9b700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb95c06e300 0x7fb95c0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:20.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.682+0000 7fb971d9b700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb95c06e300 0x7fb95c0707c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb974198080 tx=0x7fb968009190 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:20.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.687+0000 7fb9637fe700 1 -- 172.21.15.42:0/3990264947 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fb964058d20 con 0x7fb9740ffda0 2024-09-16T23:52:20.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.903+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb974197e90 con 0x7fb95c06e300 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.912+0000 7fb9637fe700 1 -- 172.21.15.42:0/3990264947 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb974197e90 con 0x7fb95c06e300 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (60s) 31s ago 11m 15.1M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (12m) 31s ago 12m 7902k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (10m) 92s ago 10m 7726k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (11m) 31s ago 11m 6665k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (10m) 92s ago 10m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (36s) 31s ago 11m 46.1M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (6m) 31s ago 6m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:52:20.910 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (6m) 31s ago 6m 14.8M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (6m) 92s ago 6m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (5m) 92s ago 5m 15.7M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (2m) 31s ago 13m 504M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (110s) 92s ago 10m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (13m) 31s ago 13m 56.0M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (10m) 92s ago 10m 47.4M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (99s) 31s ago 11m 8430k - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (95s) 92s ago 10m 3334k - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:52:20.911 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (9m) 31s ago 9m 204M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:52:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (9m) 31s ago 9m 201M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:52:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (8m) 31s ago 8m 163M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:52:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (8m) 92s ago 8m 233M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:52:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (8m) 92s ago 8m 219M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:52:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (7m) 92s ago 7m 201M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:52:20.912 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (76s) 31s ago 11m 57.6M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb95c06e300 msgr2=0x7fb95c0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb95c06e300 0x7fb95c0707c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb974198080 tx=0x7fb968009190 comp rx=0 tx=0).stop 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 msgr2=0x7fb97419c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb97419c860 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb964006010 tx=0x7fb96400bd70 comp rx=0 tx=0).stop 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 shutdown_connections 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9740ffda0 0x7fb97419c860 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb95c06e300 0x7fb95c0707c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 --2- 172.21.15.42:0/3990264947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb974100750 0x7fb97419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.914+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 >> 172.21.15.42:0/3990264947 conn(0x7fb9740fb650 msgr2=0x7fb974104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.915+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 shutdown_connections 2024-09-16T23:52:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:20.915+0000 7fb978a12700 1 -- 172.21.15.42:0/3990264947 wait complete. 2024-09-16T23:52:21.544 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T23:52:21.545 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:52:21.545 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-09-16T23:52:21.752 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:22.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:22 smithi042 ceph-mon[27914]: pgmap v76: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:22.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:22 smithi042 ceph-mon[27914]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:22.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:22 smithi043 ceph-mon[35191]: pgmap v76: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:22.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:22 smithi043 ceph-mon[35191]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:23.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 -- 172.21.15.42:0/3950818567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 msgr2=0x7fb2ac102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:23.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/3950818567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac102280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b0a0 tx=0x7fb29c00b3b0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 -- 172.21.15.42:0/3950818567 shutdown_connections 2024-09-16T23:52:23.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/3950818567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac1027c0 0x7fb2ac10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/3950818567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac102280 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 -- 172.21.15.42:0/3950818567 >> 172.21.15.42:0/3950818567 conn(0x7fb2ac0fb650 msgr2=0x7fb2ac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 -- 172.21.15.42:0/3950818567 shutdown_connections 2024-09-16T23:52:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.465+0000 7fb2b1b47700 1 -- 172.21.15.42:0/3950818567 wait complete. 2024-09-16T23:52:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2b1b47700 1 Processor -- start 2024-09-16T23:52:23.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2b1b47700 1 -- start start 2024-09-16T23:52:23.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2b1b47700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:23.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2b1b47700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac1027c0 0x7fb2ac196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:23.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2b1b47700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac197560 con 0x7fb2ac101ea0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2b1b47700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac19b370 con 0x7fb2ac1027c0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2ab7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2ab7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38630/0 (socket says 172.21.15.42:38630) 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2ab7fe700 1 -- 172.21.15.42:0/348900197 learned_addr learned my addr 172.21.15.42:0/348900197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.466+0000 7fb2aaffd700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac1027c0 0x7fb2ac196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2ab7fe700 1 -- 172.21.15.42:0/348900197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac1027c0 msgr2=0x7fb2ac196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2ab7fe700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac1027c0 0x7fb2ac196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2ab7fe700 1 -- 172.21.15.42:0/348900197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c009d00 con 0x7fb2ac101ea0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2aaffd700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac1027c0 0x7fb2ac196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2ab7fe700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac196990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb29c000f80 tx=0x7fb29c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2a8ff9700 1 -- 172.21.15.42:0/348900197 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29c00e040 con 0x7fb2ac101ea0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2a8ff9700 1 -- 172.21.15.42:0/348900197 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb29c003e60 con 0x7fb2ac101ea0 2024-09-16T23:52:23.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac19b5f0 con 0x7fb2ac101ea0 2024-09-16T23:52:23.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2a8ff9700 1 -- 172.21.15.42:0/348900197 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29c01d910 con 0x7fb2ac101ea0 2024-09-16T23:52:23.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.467+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac19bae0 con 0x7fb2ac101ea0 2024-09-16T23:52:23.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.468+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac108380 con 0x7fb2ac101ea0 2024-09-16T23:52:23.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.469+0000 7fb2a8ff9700 1 -- 172.21.15.42:0/348900197 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7fb29c004820 con 0x7fb2ac101ea0 2024-09-16T23:52:23.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.469+0000 7fb2a8ff9700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb29406e300 0x7fb2940707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:23.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.469+0000 7fb2a8ff9700 1 -- 172.21.15.42:0/348900197 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb29c08f8b0 con 0x7fb2ac101ea0 2024-09-16T23:52:23.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.469+0000 7fb2aaffd700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb29406e300 0x7fb2940707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:23.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.470+0000 7fb2aaffd700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb29406e300 0x7fb2940707c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac197fb0 tx=0x7fb2a0008040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:23.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.474+0000 7fb2a8ff9700 1 -- 172.21.15.42:0/348900197 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fb29c05cb30 con 0x7fb2ac101ea0 2024-09-16T23:52:23.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.682+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fb2ac107d60 con 0x7fb2ac101ea0 2024-09-16T23:52:23.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.686+0000 7fb2a8ff9700 1 -- 172.21.15.42:0/348900197 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v45)=0 v45) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb29c05c6c0 con 0x7fb2ac101ea0 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb29406e300 msgr2=0x7fb2940707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb29406e300 0x7fb2940707c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac197fb0 tx=0x7fb2a0008040 comp rx=0 tx=0).stop 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 msgr2=0x7fb2ac196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac196990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb29c000f80 tx=0x7fb29c00cdd0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 shutdown_connections 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2ac101ea0 0x7fb2ac196990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fb29406e300 0x7fb2940707c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 --2- 172.21.15.42:0/348900197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2ac1027c0 0x7fb2ac196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:23.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 >> 172.21.15.42:0/348900197 conn(0x7fb2ac0fb650 msgr2=0x7fb2ac1054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:23.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 shutdown_connections 2024-09-16T23:52:23.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:23.688+0000 7fb2b1b47700 1 -- 172.21.15.42:0/348900197 wait complete. 2024-09-16T23:52:24.438 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T23:52:24.439 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:52:24.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-16T23:52:24.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:24 smithi042 ceph-mon[27914]: pgmap v77: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:24.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:24 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/348900197' entity='client.admin' 2024-09-16T23:52:24.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:24 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:24.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:24 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:52:24.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:24 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:52:24.464 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:24 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:24 smithi043 ceph-mon[35191]: pgmap v77: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:24 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/348900197' entity='client.admin' 2024-09-16T23:52:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:24 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:24 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:52:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:24 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:52:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:24 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:24.649 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:25.472 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:25 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:25 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:26.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 -- 172.21.15.42:0/1409808664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 msgr2=0x7f77f0073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:26.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 --2- 172.21.15.42:0/1409808664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f0073490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f77e000b0a0 tx=0x7f77e000b3b0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.349 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:26 smithi042 ceph-mon[27914]: pgmap v78: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:26.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 -- 172.21.15.42:0/1409808664 shutdown_connections 2024-09-16T23:52:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 --2- 172.21.15.42:0/1409808664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f00739d0 0x7f77f010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 --2- 172.21.15.42:0/1409808664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f0073490 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 -- 172.21.15.42:0/1409808664 >> 172.21.15.42:0/1409808664 conn(0x7f77f00fc740 msgr2=0x7f77f00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 -- 172.21.15.42:0/1409808664 shutdown_connections 2024-09-16T23:52:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.350+0000 7f77f5aca700 1 -- 172.21.15.42:0/1409808664 wait complete. 2024-09-16T23:52:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.351+0000 7f77f5aca700 1 Processor -- start 2024-09-16T23:52:26.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.351+0000 7f77f5aca700 1 -- start start 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.351+0000 7f77f5aca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.351+0000 7f77f5aca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f00739d0 0x7f77f019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.351+0000 7f77f5aca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f019f4e0 con 0x7f77f00730b0 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.351+0000 7f77f5aca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0198990 con 0x7f77f00739d0 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.351+0000 7f77ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f019e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38652/0 (socket says 172.21.15.42:38652) 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ef7fe700 1 -- 172.21.15.42:0/2757065564 learned_addr learned my addr 172.21.15.42:0/2757065564 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77eeffd700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f00739d0 0x7f77f019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ef7fe700 1 -- 172.21.15.42:0/2757065564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f00739d0 msgr2=0x7f77f019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ef7fe700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f00739d0 0x7f77f019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ef7fe700 1 -- 172.21.15.42:0/2757065564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0009d00 con 0x7f77f00730b0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ef7fe700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f019e910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000f80 tx=0x7f77e0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ecff9700 1 -- 172.21.15.42:0/2757065564 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e000e040 con 0x7f77f00730b0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ecff9700 1 -- 172.21.15.42:0/2757065564 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77e0007c00 con 0x7f77f00730b0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f0198c10 con 0x7f77f00730b0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77ecff9700 1 -- 172.21.15.42:0/2757065564 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77e001d740 con 0x7f77f00730b0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.352+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f0199100 con 0x7f77f00730b0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.353+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f010a6b0 con 0x7f77f00730b0 2024-09-16T23:52:26.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.354+0000 7f77ecff9700 1 -- 172.21.15.42:0/2757065564 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f77e0004040 con 0x7f77f00730b0 2024-09-16T23:52:26.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.354+0000 7f77ecff9700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f77dc06e300 0x7f77dc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:26.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.355+0000 7f77ecff9700 1 -- 172.21.15.42:0/2757065564 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f77e008f930 con 0x7f77f00730b0 2024-09-16T23:52:26.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.355+0000 7f77eeffd700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f77dc06e300 0x7f77dc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:26.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.355+0000 7f77eeffd700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f77dc06e300 0x7f77dc0707c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006fd0 tx=0x7f77d8008040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:26.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.359+0000 7f77ecff9700 1 -- 172.21.15.42:0/2757065564 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f77e005cb30 con 0x7f77f00730b0 2024-09-16T23:52:26.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.569+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f77f004f2e0 con 0x7f77f00730b0 2024-09-16T23:52:26.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.569+0000 7f77ecff9700 1 -- 172.21.15.42:0/2757065564 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v45)=0 v45) v1 ==== 155+0+0 (secure 0 0 0) 0x7f77e005c6c0 con 0x7f77f00730b0 2024-09-16T23:52:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f77dc06e300 msgr2=0x7f77dc0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f77dc06e300 0x7f77dc0707c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f77d8006fd0 tx=0x7f77d8008040 comp rx=0 tx=0).stop 2024-09-16T23:52:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 msgr2=0x7f77f019e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f019e910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f77e0000f80 tx=0x7f77e0003ce0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 shutdown_connections 2024-09-16T23:52:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77f00730b0 0x7f77f019e910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f77dc06e300 0x7f77dc0707c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 --2- 172.21.15.42:0/2757065564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77f00739d0 0x7f77f019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:26.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 >> 172.21.15.42:0/2757065564 conn(0x7f77f00fc740 msgr2=0x7f77f01077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:26.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 shutdown_connections 2024-09-16T23:52:26.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:26.571+0000 7f77f5aca700 1 -- 172.21.15.42:0/2757065564 wait complete. 2024-09-16T23:52:26.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:26 smithi043 ceph-mon[35191]: pgmap v78: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:27.222 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-16T23:52:27.474 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:28.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:28 smithi042 ceph-mon[27914]: pgmap v79: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:28.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:28 smithi043 ceph-mon[35191]: pgmap v79: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 -- 172.21.15.42:0/2382484396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 msgr2=0x7ff958105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/2382484396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958105fb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff948009910 tx=0x7ff948009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 -- 172.21.15.42:0/2382484396 shutdown_connections 2024-09-16T23:52:29.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/2382484396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9580ffc20 0x7ff958100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/2382484396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958105fb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 -- 172.21.15.42:0/2382484396 >> 172.21.15.42:0/2382484396 conn(0x7ff958075670 msgr2=0x7ff958075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 -- 172.21.15.42:0/2382484396 shutdown_connections 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.143+0000 7ff95cfc0700 1 -- 172.21.15.42:0/2382484396 wait complete. 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff95cfc0700 1 Processor -- start 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff95cfc0700 1 -- start start 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff95cfc0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9580ffc20 0x7ff9581977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff95cfc0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff95cfc0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958198340 con 0x7ff958105bd0 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff95cfc0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958198480 con 0x7ff9580ffc20 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff955d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff955d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38672/0 (socket says 172.21.15.42:38672) 2024-09-16T23:52:29.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.144+0000 7ff955d9b700 1 -- 172.21.15.42:0/232284533 learned_addr learned my addr 172.21.15.42:0/232284533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff955d9b700 1 -- 172.21.15.42:0/232284533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9580ffc20 msgr2=0x7ff9581977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff955d9b700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9580ffc20 0x7ff9581977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff955d9b700 1 -- 172.21.15.42:0/232284533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480095f0 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff955d9b700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958197d20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00b4b0 tx=0x7ff94c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff9477fe700 1 -- 172.21.15.42:0/232284533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94c011800 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff9477fe700 1 -- 172.21.15.42:0/232284533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff94c011e20 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9581a0850 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff9477fe700 1 -- 172.21.15.42:0/232284533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94c00f2e0 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.145+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9581a0d10 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.146+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9581080d0 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.146+0000 7ff9477fe700 1 -- 172.21.15.42:0/232284533 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7ff94c011960 con 0x7ff958105bd0 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.147+0000 7ff9477fe700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7ff94006e300 0x7ff9400707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:29.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.147+0000 7ff9477fe700 1 -- 172.21.15.42:0/232284533 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7ff94c08dc80 con 0x7ff958105bd0 2024-09-16T23:52:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.147+0000 7ff95659c700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7ff94006e300 0x7ff9400707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.147+0000 7ff95659c700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7ff94006e300 0x7ff9400707c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b5c0 tx=0x7ff948005960 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:29.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.152+0000 7ff9477fe700 1 -- 172.21.15.42:0/232284533 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7ff94c057350 con 0x7ff958105bd0 2024-09-16T23:52:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.357+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7ff95804f2e0 con 0x7ff958105bd0 2024-09-16T23:52:29.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.357+0000 7ff9477fe700 1 -- 172.21.15.42:0/232284533 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v45)=0 v45) v1 ==== 163+0+0 (secure 0 0 0) 0x7ff94c05aa10 con 0x7ff958105bd0 2024-09-16T23:52:29.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7ff94006e300 msgr2=0x7ff9400707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:29.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7ff94006e300 0x7ff9400707c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b5c0 tx=0x7ff948005960 comp rx=0 tx=0).stop 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 msgr2=0x7ff958197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958197d20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00b4b0 tx=0x7ff94c00b870 comp rx=0 tx=0).stop 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 shutdown_connections 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7ff94006e300 0x7ff9400707c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9580ffc20 0x7ff9581977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 --2- 172.21.15.42:0/232284533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff958105bd0 0x7ff958197d20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.359+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 >> 172.21.15.42:0/232284533 conn(0x7ff958075670 msgr2=0x7ff9580fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.360+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 shutdown_connections 2024-09-16T23:52:29.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:29.360+0000 7ff95cfc0700 1 -- 172.21.15.42:0/232284533 wait complete. 2024-09-16T23:52:29.998 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-16T23:52:30.212 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:30.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:30 smithi042 ceph-mon[27914]: pgmap v80: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:30.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:30 smithi043 ceph-mon[35191]: pgmap v80: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:31.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 -- 172.21.15.42:0/1186052013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c105bd0 msgr2=0x7f318c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:31.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 --2- 172.21.15.42:0/1186052013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c105bd0 0x7f318c105fb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f317c009910 tx=0x7f317c009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:31.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 -- 172.21.15.42:0/1186052013 shutdown_connections 2024-09-16T23:52:31.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 --2- 172.21.15.42:0/1186052013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318c0ffc20 0x7f318c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:31.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 --2- 172.21.15.42:0/1186052013 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c105bd0 0x7f318c105fb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:31.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 -- 172.21.15.42:0/1186052013 >> 172.21.15.42:0/1186052013 conn(0x7f318c075670 msgr2=0x7f318c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:31.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 -- 172.21.15.42:0/1186052013 shutdown_connections 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.905+0000 7f31927a4700 1 -- 172.21.15.42:0/1186052013 wait complete. 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f31927a4700 1 Processor -- start 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f31927a4700 1 -- start start 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f31927a4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c0ffc20 0x7f318c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f31927a4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318c105bd0 0x7f318c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f31927a4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c19f500 con 0x7f318c0ffc20 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f31927a4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c1989b0 con 0x7f318c105bd0 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f318bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c0ffc20 0x7f318c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f318bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c0ffc20 0x7f318c19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38686/0 (socket says 172.21.15.42:38686) 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.906+0000 7f318bfff700 1 -- 172.21.15.42:0/4057242586 learned_addr learned my addr 172.21.15.42:0/4057242586 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f3183fff700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318c105bd0 0x7f318c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:31.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f318bfff700 1 -- 172.21.15.42:0/4057242586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318c105bd0 msgr2=0x7f318c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f318bfff700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318c105bd0 0x7f318c19ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f318bfff700 1 -- 172.21.15.42:0/4057242586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c0095f0 con 0x7f318c0ffc20 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f318bfff700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c0ffc20 0x7f318c19e930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f317c006010 tx=0x7f317c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f3189ffb700 1 -- 172.21.15.42:0/4057242586 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c01d070 con 0x7f318c0ffc20 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f3189ffb700 1 -- 172.21.15.42:0/4057242586 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f317c004fb0 con 0x7f318c0ffc20 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c198c30 con 0x7f318c0ffc20 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f3189ffb700 1 -- 172.21.15.42:0/4057242586 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f317c022ea0 con 0x7f318c0ffc20 2024-09-16T23:52:31.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.907+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c199120 con 0x7f318c0ffc20 2024-09-16T23:52:31.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.908+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c04f2e0 con 0x7f318c0ffc20 2024-09-16T23:52:31.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.909+0000 7f3189ffb700 1 -- 172.21.15.42:0/4057242586 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f317c022470 con 0x7f318c0ffc20 2024-09-16T23:52:31.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.909+0000 7f3189ffb700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f317806e300 0x7f31780707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:31.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.909+0000 7f3189ffb700 1 -- 172.21.15.42:0/4057242586 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f317c08f540 con 0x7f318c0ffc20 2024-09-16T23:52:31.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.909+0000 7f3183fff700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f317806e300 0x7f31780707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:31.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.910+0000 7f3183fff700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f317806e300 0x7f31780707c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3174006fd0 tx=0x7f3174008040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:31.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:31.914+0000 7f3189ffb700 1 -- 172.21.15.42:0/4057242586 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f317c05c870 con 0x7f318c0ffc20 2024-09-16T23:52:32.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.123+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f318c199400 con 0x7f318c0ffc20 2024-09-16T23:52:32.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.123+0000 7f3189ffb700 1 -- 172.21.15.42:0/4057242586 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v45)=0 v45) v1 ==== 135+0+0 (secure 0 0 0) 0x7f317c022720 con 0x7f318c0ffc20 2024-09-16T23:52:32.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f317806e300 msgr2=0x7f31780707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:32.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f317806e300 0x7f31780707c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3174006fd0 tx=0x7f3174008040 comp rx=0 tx=0).stop 2024-09-16T23:52:32.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c0ffc20 msgr2=0x7f318c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:32.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c0ffc20 0x7f318c19e930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f317c006010 tx=0x7f317c00be20 comp rx=0 tx=0).stop 2024-09-16T23:52:32.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 shutdown_connections 2024-09-16T23:52:32.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f318c0ffc20 0x7f318c19e930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:32.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f317806e300 0x7f31780707c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:32.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 --2- 172.21.15.42:0/4057242586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318c105bd0 0x7f318c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:32.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.125+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 >> 172.21.15.42:0/4057242586 conn(0x7f318c075670 msgr2=0x7f318c068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:32.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.126+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 shutdown_connections 2024-09-16T23:52:32.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:32.126+0000 7f31927a4700 1 -- 172.21.15.42:0/4057242586 wait complete. 2024-09-16T23:52:32.505 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:32 smithi042 ceph-mon[27914]: pgmap v81: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:32 smithi043 ceph-mon[35191]: pgmap v81: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:32.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-16T23:52:33.045 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:34.397 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:34 smithi042 ceph-mon[27914]: pgmap v82: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 -- 172.21.15.42:0/1289584647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104102700 msgr2=0x7f010410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 --2- 172.21.15.42:0/1289584647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104102700 0x7f010410abf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009910 tx=0x7f00f4009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 -- 172.21.15.42:0/1289584647 shutdown_connections 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 --2- 172.21.15.42:0/1289584647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104102700 0x7f010410abf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 --2- 172.21.15.42:0/1289584647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0104101de0 0x7f01041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 -- 172.21.15.42:0/1289584647 >> 172.21.15.42:0/1289584647 conn(0x7f01040fb650 msgr2=0x7f01040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 -- 172.21.15.42:0/1289584647 shutdown_connections 2024-09-16T23:52:34.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 -- 172.21.15.42:0/1289584647 wait complete. 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 Processor -- start 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.623+0000 7f01095a7700 1 -- start start 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f01095a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104101de0 0x7f010419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f01095a7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0104102700 0x7f010419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f01095a7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010419d270 con 0x7f0104101de0 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f01095a7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104196720 con 0x7f0104102700 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f0102ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104101de0 0x7f010419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f0102ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104101de0 0x7f010419c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53422/0 (socket says 172.21.15.42:53422) 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f0102ffd700 1 -- 172.21.15.42:0/1159061707 learned_addr learned my addr 172.21.15.42:0/1159061707 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f01027fc700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0104102700 0x7f010419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f0102ffd700 1 -- 172.21.15.42:0/1159061707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0104102700 msgr2=0x7f010419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f0102ffd700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0104102700 0x7f010419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:34.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f0102ffd700 1 -- 172.21.15.42:0/1159061707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40095f0 con 0x7f0104101de0 2024-09-16T23:52:34.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f0102ffd700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104101de0 0x7f010419c6a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00d8d0 tx=0x7f00ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:34.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f00fbfff700 1 -- 172.21.15.42:0/1159061707 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00ec009940 con 0x7f0104101de0 2024-09-16T23:52:34.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0104196a00 con 0x7f0104101de0 2024-09-16T23:52:34.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.624+0000 7f00fbfff700 1 -- 172.21.15.42:0/1159061707 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00ec010460 con 0x7f0104101de0 2024-09-16T23:52:34.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.625+0000 7f00fbfff700 1 -- 172.21.15.42:0/1159061707 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00ec00f5f0 con 0x7f0104101de0 2024-09-16T23:52:34.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.625+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0104196f50 con 0x7f0104101de0 2024-09-16T23:52:34.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.626+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01041082f0 con 0x7f0104101de0 2024-09-16T23:52:34.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.626+0000 7f00fbfff700 1 -- 172.21.15.42:0/1159061707 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f00ec0105d0 con 0x7f0104101de0 2024-09-16T23:52:34.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.627+0000 7f00fbfff700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f00f006e300 0x7f00f00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:34.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.627+0000 7f00fbfff700 1 -- 172.21.15.42:0/1159061707 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f00ec08e0c0 con 0x7f0104101de0 2024-09-16T23:52:34.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.627+0000 7f01027fc700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f00f006e300 0x7f00f00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:34.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.627+0000 7f01027fc700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f00f006e300 0x7f00f00707c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0104197b90 tx=0x7f00f401a040 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:34.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:34 smithi043 ceph-mon[35191]: pgmap v82: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:34.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.631+0000 7f00fbfff700 1 -- 172.21.15.42:0/1159061707 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f00ec05b2c0 con 0x7f0104101de0 2024-09-16T23:52:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.862+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "target": ["mon-mgr", ""]}) v1 -- 0x7f0104197170 con 0x7f00f006e300 2024-09-16T23:52:34.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.868+0000 7f00fbfff700 1 -- 172.21.15.42:0/1159061707 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f0104197170 con 0x7f00f006e300 2024-09-16T23:52:34.865 INFO:teuthology.orchestra.run.smithi042.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:52:34.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.870+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f00f006e300 msgr2=0x7f00f00707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:34.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.870+0000 7f01095a7700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f00f006e300 0x7f00f00707c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0104197b90 tx=0x7f00f401a040 comp rx=0 tx=0).stop 2024-09-16T23:52:34.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.870+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104101de0 msgr2=0x7f010419c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:34.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.870+0000 7f01095a7700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104101de0 0x7f010419c6a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00d8d0 tx=0x7f00ec00dc90 comp rx=0 tx=0).stop 2024-09-16T23:52:34.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.871+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 shutdown_connections 2024-09-16T23:52:34.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.871+0000 7f01095a7700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0104101de0 0x7f010419c6a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:34.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.871+0000 7f01095a7700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f00f006e300 0x7f00f00707c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:34.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.871+0000 7f01095a7700 1 --2- 172.21.15.42:0/1159061707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0104102700 0x7f010419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:34.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.871+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 >> 172.21.15.42:0/1159061707 conn(0x7f01040fb650 msgr2=0x7f0104105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:34.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.871+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 shutdown_connections 2024-09-16T23:52:34.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:34.871+0000 7f01095a7700 1 -- 172.21.15.42:0/1159061707 wait complete. 2024-09-16T23:52:35.853 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T23:52:35.854 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-16T23:52:35.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-16T23:52:36.068 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: pgmap v83: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:35 smithi043 ceph-mon[35191]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:52:36.189 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: pgmap v83: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:36.189 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:36.189 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:52:36.189 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:36.190 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:36.190 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:52:36.190 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:52:36.190 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:36.190 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:35 smithi042 ceph-mon[27914]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-16T23:52:37.945 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:37 smithi042 ceph-mon[27914]: pgmap v84: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:37 smithi043 ceph-mon[35191]: pgmap v84: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:39.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 -- 172.21.15.42:0/3759584129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 msgr2=0x7fe110101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 --2- 172.21.15.42:0/3759584129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe110101270 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe10000b0a0 tx=0x7fe10000b3b0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 -- 172.21.15.42:0/3759584129 shutdown_connections 2024-09-16T23:52:39.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 --2- 172.21.15.42:0/3759584129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe110101840 0x7fe110105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 --2- 172.21.15.42:0/3759584129 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe110101270 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 -- 172.21.15.42:0/3759584129 >> 172.21.15.42:0/3759584129 conn(0x7fe1100fc740 msgr2=0x7fe1100feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:39.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 -- 172.21.15.42:0/3759584129 shutdown_connections 2024-09-16T23:52:39.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 -- 172.21.15.42:0/3759584129 wait complete. 2024-09-16T23:52:39.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.157+0000 7fe115fd2700 1 Processor -- start 2024-09-16T23:52:39.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe115fd2700 1 -- start start 2024-09-16T23:52:39.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe115fd2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe11010f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe115fd2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe110101840 0x7fe11010fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe115fd2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1101104a0 con 0x7fe110100e90 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe115fd2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110110610 con 0x7fe110101840 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe10f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe11010f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe10f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe11010f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53446/0 (socket says 172.21.15.42:53446) 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe10f7fe700 1 -- 172.21.15.42:0/1815831422 learned_addr learned my addr 172.21.15.42:0/1815831422 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe106ffd700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe110101840 0x7fe11010fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe10f7fe700 1 -- 172.21.15.42:0/1815831422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe110101840 msgr2=0x7fe11010fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe10f7fe700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe110101840 0x7fe11010fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe10f7fe700 1 -- 172.21.15.42:0/1815831422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100009d00 con 0x7fe110100e90 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.158+0000 7fe106ffd700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe110101840 0x7fe11010fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.159+0000 7fe10f7fe700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe11010f940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe100000f80 tx=0x7fe10000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.159+0000 7fe10d7fa700 1 -- 172.21.15.42:0/1815831422 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10000e040 con 0x7fe110100e90 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.159+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1101143f0 con 0x7fe110100e90 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.159+0000 7fe10d7fa700 1 -- 172.21.15.42:0/1815831422 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe100003de0 con 0x7fe110100e90 2024-09-16T23:52:39.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.159+0000 7fe10d7fa700 1 -- 172.21.15.42:0/1815831422 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10001d790 con 0x7fe110100e90 2024-09-16T23:52:39.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.159+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1101148e0 con 0x7fe110100e90 2024-09-16T23:52:39.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.160+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe110109230 con 0x7fe110100e90 2024-09-16T23:52:39.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.160+0000 7fe10d7fa700 1 -- 172.21.15.42:0/1815831422 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7fe100004040 con 0x7fe110100e90 2024-09-16T23:52:39.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.161+0000 7fe10d7fa700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fe0fc06e300 0x7fe0fc0707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.161+0000 7fe10d7fa700 1 -- 172.21.15.42:0/1815831422 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fe10008f8e0 con 0x7fe110100e90 2024-09-16T23:52:39.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.161+0000 7fe106ffd700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fe0fc06e300 0x7fe0fc0707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.161+0000 7fe106ffd700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fe0fc06e300 0x7fe0fc0707c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005950 tx=0x7fe0f80058e0 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:39.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.166+0000 7fe10d7fa700 1 -- 172.21.15.42:0/1815831422 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7fe100058fb0 con 0x7fe110100e90 2024-09-16T23:52:39.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.397+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe110110d10 con 0x7fe0fc06e300 2024-09-16T23:52:39.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.398+0000 7fe10d7fa700 1 -- 172.21.15.42:0/1815831422 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe110110d10 con 0x7fe0fc06e300 2024-09-16T23:52:39.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.400+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fe0fc06e300 msgr2=0x7fe0fc0707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.400+0000 7fe115fd2700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fe0fc06e300 0x7fe0fc0707c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005950 tx=0x7fe0f80058e0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.400+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 msgr2=0x7fe11010f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.400+0000 7fe115fd2700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe11010f940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe100000f80 tx=0x7fe10000cdd0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.401+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 shutdown_connections 2024-09-16T23:52:39.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.401+0000 7fe115fd2700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe110100e90 0x7fe11010f940 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.401+0000 7fe115fd2700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7fe0fc06e300 0x7fe0fc0707c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.401+0000 7fe115fd2700 1 --2- 172.21.15.42:0/1815831422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe110101840 0x7fe11010fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.401+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 >> 172.21.15.42:0/1815831422 conn(0x7fe1100fc740 msgr2=0x7fe1100fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:39.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.401+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 shutdown_connections 2024-09-16T23:52:39.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.401+0000 7fe115fd2700 1 -- 172.21.15.42:0/1815831422 wait complete. 2024-09-16T23:52:39.409 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:52:39.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 -- 172.21.15.42:0/3252910453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198100ea0 msgr2=0x7f7198101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 --2- 172.21.15.42:0/3252910453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198100ea0 0x7f7198101280 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7194009910 tx=0x7f7194009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 -- 172.21.15.42:0/3252910453 shutdown_connections 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 --2- 172.21.15.42:0/3252910453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7198101850 0x7f71981058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 --2- 172.21.15.42:0/3252910453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198100ea0 0x7f7198101280 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 -- 172.21.15.42:0/3252910453 >> 172.21.15.42:0/3252910453 conn(0x7f71980fc750 msgr2=0x7f71980feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 -- 172.21.15.42:0/3252910453 shutdown_connections 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.511+0000 7f71a0620700 1 -- 172.21.15.42:0/3252910453 wait complete. 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f71a0620700 1 Processor -- start 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f71a0620700 1 -- start start 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f71a0620700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7198100ea0 0x7f7198196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f71a0620700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198101850 0x7f7198196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f71a0620700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7198197560 con 0x7f7198101850 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f71a0620700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719819b370 con 0x7f7198100ea0 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f719dbbb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198101850 0x7f7198196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f719dbbb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198101850 0x7f7198196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53470/0 (socket says 172.21.15.42:53470) 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.512+0000 7f719dbbb700 1 -- 172.21.15.42:0/1334487657 learned_addr learned my addr 172.21.15.42:0/1334487657 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f719e3bc700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7198100ea0 0x7f7198196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f719dbbb700 1 -- 172.21.15.42:0/1334487657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7198100ea0 msgr2=0x7f7198196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f719dbbb700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7198100ea0 0x7f7198196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f719dbbb700 1 -- 172.21.15.42:0/1334487657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71940095f0 con 0x7f7198101850 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f719dbbb700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198101850 0x7f7198196ed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f718800b480 tx=0x7f718800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f718f7fe700 1 -- 172.21.15.42:0/1334487657 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71880118c0 con 0x7f7198101850 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719819b650 con 0x7f7198101850 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f718f7fe700 1 -- 172.21.15.42:0/1334487657 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7188010380 con 0x7f7198101850 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f718f7fe700 1 -- 172.21.15.42:0/1334487657 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f718800f4f0 con 0x7f7198101850 2024-09-16T23:52:39.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.513+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719819bba0 con 0x7f7198101850 2024-09-16T23:52:39.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.514+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7198109240 con 0x7f7198101850 2024-09-16T23:52:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.515+0000 7f718f7fe700 1 -- 172.21.15.42:0/1334487657 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f71880104f0 con 0x7f7198101850 2024-09-16T23:52:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.515+0000 7f718f7fe700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f718406e300 0x7f71840707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.515+0000 7f718f7fe700 1 -- 172.21.15.42:0/1334487657 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f718808dd00 con 0x7f7198101850 2024-09-16T23:52:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.515+0000 7f719e3bc700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f718406e300 0x7f71840707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.516+0000 7f719e3bc700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f718406e300 0x7f71840707c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f719400b5c0 tx=0x7f7194005960 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:39.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.521+0000 7f718f7fe700 1 -- 172.21.15.42:0/1334487657 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f7188091030 con 0x7f7198101850 2024-09-16T23:52:39.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.758+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f719819be20 con 0x7f718406e300 2024-09-16T23:52:39.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:39.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: Upgrade: Target is version 19.1.1-330-gc2ddf972 (squid) 2024-09-16T23:52:39.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3'] 2024-09-16T23:52:39.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:39.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:39.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: pgmap v85: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:39.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: Upgrade: Setting container_image for all mgr 2024-09-16T23:52:39.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:39.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:39 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T23:52:39.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.759+0000 7f718f7fe700 1 -- 172.21.15.42:0/1334487657 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f719819be20 con 0x7f718406e300 2024-09-16T23:52:39.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f718406e300 msgr2=0x7f71840707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f718406e300 0x7f71840707c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f719400b5c0 tx=0x7f7194005960 comp rx=0 tx=0).stop 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198101850 msgr2=0x7f7198196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198101850 0x7f7198196ed0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f718800b480 tx=0x7f718800b840 comp rx=0 tx=0).stop 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 shutdown_connections 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f718406e300 0x7f71840707c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7198100ea0 0x7f7198196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 --2- 172.21.15.42:0/1334487657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7198101850 0x7f7198196ed0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 >> 172.21.15.42:0/1334487657 conn(0x7f71980fc750 msgr2=0x7f71980fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 shutdown_connections 2024-09-16T23:52:39.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.762+0000 7f71a0620700 1 -- 172.21.15.42:0/1334487657 wait complete. 2024-09-16T23:52:39.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 -- 172.21.15.42:0/2243696477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 msgr2=0x7f4e281058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/2243696477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e281058a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009910 tx=0x7f4e1c009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:39.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 -- 172.21.15.42:0/2243696477 shutdown_connections 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/2243696477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e281058a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/2243696477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28100ea0 0x7f4e28101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 -- 172.21.15.42:0/2243696477 >> 172.21.15.42:0/2243696477 conn(0x7f4e280fc750 msgr2=0x7f4e280feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 -- 172.21.15.42:0/2243696477 shutdown_connections 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.873+0000 7f4e2cc27700 1 -- 172.21.15.42:0/2243696477 wait complete. 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e2cc27700 1 Processor -- start 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e2cc27700 1 -- start start 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e2cc27700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28100ea0 0x7f4e28073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e2cc27700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e280735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e2cc27700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28073b10 con 0x7f4e28101850 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e2cc27700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28073c50 con 0x7f4e28100ea0 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e280735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e280735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53498/0 (socket says 172.21.15.42:53498) 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e25d9b700 1 -- 172.21.15.42:0/1679905733 learned_addr learned my addr 172.21.15.42:0/1679905733 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e2659c700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28100ea0 0x7f4e28073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e25d9b700 1 -- 172.21.15.42:0/1679905733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28100ea0 msgr2=0x7f4e28073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e25d9b700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28100ea0 0x7f4e28073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e25d9b700 1 -- 172.21.15.42:0/1679905733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c0095f0 con 0x7f4e28101850 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.874+0000 7f4e25d9b700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e280735d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0092e0 tx=0x7f4e1c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.875+0000 7f4e177fe700 1 -- 172.21.15.42:0/1679905733 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e1c01d070 con 0x7f4e28101850 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.875+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e28073ed0 con 0x7f4e28101850 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.875+0000 7f4e177fe700 1 -- 172.21.15.42:0/1679905733 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e1c00f460 con 0x7f4e28101850 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.875+0000 7f4e177fe700 1 -- 172.21.15.42:0/1679905733 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e1c0175d0 con 0x7f4e28101850 2024-09-16T23:52:39.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.875+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e281a2e30 con 0x7f4e28101850 2024-09-16T23:52:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.876+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e281092a0 con 0x7f4e28101850 2024-09-16T23:52:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.876+0000 7f4e177fe700 1 -- 172.21.15.42:0/1679905733 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f4e1c00f5d0 con 0x7f4e28101850 2024-09-16T23:52:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.877+0000 7f4e177fe700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f4e1006e300 0x7f4e100707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.877+0000 7f4e177fe700 1 -- 172.21.15.42:0/1679905733 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f4e1c08f510 con 0x7f4e28101850 2024-09-16T23:52:39.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.877+0000 7f4e2659c700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f4e1006e300 0x7f4e100707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:39.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.877+0000 7f4e2659c700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f4e1006e300 0x7f4e100707c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e18005950 tx=0x7f4e1800a300 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:39.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: Upgrade: Target is version 19.1.1-330-gc2ddf972 (squid) 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3'] 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: pgmap v85: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: Upgrade: Setting container_image for all mgr 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:39 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T23:52:39.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:39.881+0000 7f4e177fe700 1 -- 172.21.15.42:0/1679905733 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f4e1c058be0 con 0x7f4e28101850 2024-09-16T23:52:40.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.105+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e28074630 con 0x7f4e1006e300 2024-09-16T23:52:40.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.114+0000 7f4e177fe700 1 -- 172.21.15.42:0/1679905733 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4e28074630 con 0x7f4e1006e300 2024-09-16T23:52:40.112 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:52:40.112 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (79s) 50s ago 12m 15.1M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:52:40.112 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (12m) 50s ago 12m 7902k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:52:40.112 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (11m) 111s ago 11m 7726k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:52:40.112 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (12m) 50s ago 12m 6665k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (11m) 111s ago 11m 6665k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (56s) 50s ago 11m 46.1M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (6m) 50s ago 6m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (6m) 50s ago 6m 14.8M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (6m) 111s ago 6m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (6m) 111s ago 6m 15.7M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (3m) 50s ago 13m 504M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (2m) 111s ago 11m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (13m) 50s ago 13m 56.0M 2048M 18.2.1 5be31c24972a 37006d2f6112 2024-09-16T23:52:40.113 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (10m) 111s ago 10m 47.4M 2048M 18.2.1 5be31c24972a 6770c421b06c 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (118s) 50s ago 12m 8430k - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (114s) 111s ago 11m 3334k - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (10m) 50s ago 10m 204M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (9m) 50s ago 9m 201M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (9m) 50s ago 9m 163M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (8m) 111s ago 8m 233M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (8m) 111s ago 8m 219M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (7m) 111s ago 7m 201M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:52:40.114 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (95s) 50s ago 11m 57.6M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f4e1006e300 msgr2=0x7f4e100707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f4e1006e300 0x7f4e100707c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4e18005950 tx=0x7f4e1800a300 comp rx=0 tx=0).stop 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 msgr2=0x7f4e280735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e280735d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0092e0 tx=0x7f4e1c00bd70 comp rx=0 tx=0).stop 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 shutdown_connections 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f4e1006e300 0x7f4e100707c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e28100ea0 0x7f4e28073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 --2- 172.21.15.42:0/1679905733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e28101850 0x7f4e280735d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.116+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 >> 172.21.15.42:0/1679905733 conn(0x7f4e280fc750 msgr2=0x7f4e280fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.117+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 shutdown_connections 2024-09-16T23:52:40.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.117+0000 7f4e2cc27700 1 -- 172.21.15.42:0/1679905733 wait complete. 2024-09-16T23:52:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 -- 172.21.15.42:0/3937536804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a340ffb40 msgr2=0x7f3a340fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 --2- 172.21.15.42:0/3937536804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a340ffb40 0x7f3a340fffc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009910 tx=0x7f3a1c009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 -- 172.21.15.42:0/3937536804 shutdown_connections 2024-09-16T23:52:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 --2- 172.21.15.42:0/3937536804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a340ffb40 0x7f3a340fffc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 --2- 172.21.15.42:0/3937536804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a34105bf0 0x7f3a34105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 -- 172.21.15.42:0/3937536804 >> 172.21.15.42:0/3937536804 conn(0x7f3a340fb610 msgr2=0x7f3a340fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 -- 172.21.15.42:0/3937536804 shutdown_connections 2024-09-16T23:52:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.224+0000 7f3a392b8700 1 -- 172.21.15.42:0/3937536804 wait complete. 2024-09-16T23:52:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a392b8700 1 Processor -- start 2024-09-16T23:52:40.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a392b8700 1 -- start start 2024-09-16T23:52:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a392b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a340ffb40 0x7f3a3419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a392b8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a34105bf0 0x7f3a3419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:40.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a392b8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3419d140 con 0x7f3a34105bf0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a392b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a34196680 con 0x7f3a340ffb40 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a2a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a34105bf0 0x7f3a3419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a2a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a34105bf0 0x7f3a3419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53528/0 (socket says 172.21.15.42:53528) 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a2a7fc700 1 -- 172.21.15.42:0/2473867123 learned_addr learned my addr 172.21.15.42:0/2473867123 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a32ffd700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a340ffb40 0x7f3a3419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a2a7fc700 1 -- 172.21.15.42:0/2473867123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a340ffb40 msgr2=0x7f3a3419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.225+0000 7f3a2a7fc700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a340ffb40 0x7f3a3419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.226+0000 7f3a2a7fc700 1 -- 172.21.15.42:0/2473867123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0095f0 con 0x7f3a34105bf0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.226+0000 7f3a2a7fc700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a34105bf0 0x7f3a3419cb40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.226+0000 7f3a30ff9700 1 -- 172.21.15.42:0/2473867123 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1c01d070 con 0x7f3a34105bf0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.226+0000 7f3a30ff9700 1 -- 172.21.15.42:0/2473867123 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a1c00f460 con 0x7f3a34105bf0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.226+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a34196900 con 0x7f3a34105bf0 2024-09-16T23:52:40.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.226+0000 7f3a30ff9700 1 -- 172.21.15.42:0/2473867123 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1c017630 con 0x7f3a34105bf0 2024-09-16T23:52:40.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.226+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a34196df0 con 0x7f3a34105bf0 2024-09-16T23:52:40.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.227+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a340679c0 con 0x7f3a34105bf0 2024-09-16T23:52:40.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.228+0000 7f3a30ff9700 1 -- 172.21.15.42:0/2473867123 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f3a1c00f5d0 con 0x7f3a34105bf0 2024-09-16T23:52:40.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.228+0000 7f3a30ff9700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f3a2006e300 0x7f3a200707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.228+0000 7f3a30ff9700 1 -- 172.21.15.42:0/2473867123 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f3a1c08f600 con 0x7f3a34105bf0 2024-09-16T23:52:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.228+0000 7f3a32ffd700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f3a2006e300 0x7f3a200707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:40.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.229+0000 7f3a32ffd700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f3a2006e300 0x7f3a200707c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400ac50 tx=0x7f3a2400a380 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:40.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.233+0000 7f3a30ff9700 1 -- 172.21.15.42:0/2473867123 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f3a1c05c800 con 0x7f3a34105bf0 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.532+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a3404f2e0 con 0x7f3a34105bf0 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.532+0000 7f3a30ff9700 1 -- 172.21.15.42:0/2473867123 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f3a1c05c390 con 0x7f3a34105bf0 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:52:40.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:52:40.531 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f3a2006e300 msgr2=0x7f3a200707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f3a2006e300 0x7f3a200707c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400ac50 tx=0x7f3a2400a380 comp rx=0 tx=0).stop 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a34105bf0 msgr2=0x7f3a3419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a34105bf0 0x7f3a3419cb40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c00bd70 comp rx=0 tx=0).stop 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 shutdown_connections 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f3a2006e300 0x7f3a200707c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a340ffb40 0x7f3a3419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 --2- 172.21.15.42:0/2473867123 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a34105bf0 0x7f3a3419cb40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 >> 172.21.15.42:0/2473867123 conn(0x7f3a340fb610 msgr2=0x7f3a34068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 shutdown_connections 2024-09-16T23:52:40.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.535+0000 7f3a392b8700 1 -- 172.21.15.42:0/2473867123 wait complete. 2024-09-16T23:52:40.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.645+0000 7f96341c7700 1 -- 172.21.15.42:0/1333476896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c105bf0 msgr2=0x7f962c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.645+0000 7f96341c7700 1 --2- 172.21.15.42:0/1333476896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c105bf0 0x7f962c105fd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9628009910 tx=0x7f9628009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:40.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.645+0000 7f96341c7700 1 -- 172.21.15.42:0/1333476896 shutdown_connections 2024-09-16T23:52:40.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.645+0000 7f96341c7700 1 --2- 172.21.15.42:0/1333476896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c0ffb40 0x7f962c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.645+0000 7f96341c7700 1 --2- 172.21.15.42:0/1333476896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c105bf0 0x7f962c105fd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.645+0000 7f96341c7700 1 -- 172.21.15.42:0/1333476896 >> 172.21.15.42:0/1333476896 conn(0x7f962c0fb610 msgr2=0x7f962c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:40.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.646+0000 7f96341c7700 1 -- 172.21.15.42:0/1333476896 shutdown_connections 2024-09-16T23:52:40.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.646+0000 7f96341c7700 1 -- 172.21.15.42:0/1333476896 wait complete. 2024-09-16T23:52:40.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.646+0000 7f96341c7700 1 Processor -- start 2024-09-16T23:52:40.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.646+0000 7f96341c7700 1 -- start start 2024-09-16T23:52:40.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f96341c7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c0ffb40 0x7f962c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:40.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f96341c7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c105bf0 0x7f962c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f96341c7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c102230 con 0x7f962c0ffb40 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f96341c7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c102370 con 0x7f962c105bf0 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631f63700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c0ffb40 0x7f962c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631f63700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c0ffb40 0x7f962c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53548/0 (socket says 172.21.15.42:53548) 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631f63700 1 -- 172.21.15.42:0/232101940 learned_addr learned my addr 172.21.15.42:0/232101940 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631762700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c105bf0 0x7f962c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631f63700 1 -- 172.21.15.42:0/232101940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c105bf0 msgr2=0x7f962c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631f63700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c105bf0 0x7f962c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631f63700 1 -- 172.21.15.42:0/232101940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96280095f0 con 0x7f962c0ffb40 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f9631f63700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c0ffb40 0x7f962c1017b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9628006010 tx=0x7f962800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.647+0000 7f961effd700 1 -- 172.21.15.42:0/232101940 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f962801d070 con 0x7f962c0ffb40 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.648+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c1025f0 con 0x7f962c0ffb40 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.648+0000 7f961effd700 1 -- 172.21.15.42:0/232101940 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f962800f460 con 0x7f962c0ffb40 2024-09-16T23:52:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.648+0000 7f961effd700 1 -- 172.21.15.42:0/232101940 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9628017610 con 0x7f962c0ffb40 2024-09-16T23:52:40.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.648+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c1a2c80 con 0x7f962c0ffb40 2024-09-16T23:52:40.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.649+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c04f2e0 con 0x7f962c0ffb40 2024-09-16T23:52:40.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.649+0000 7f961effd700 1 -- 172.21.15.42:0/232101940 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f962800f5d0 con 0x7f962c0ffb40 2024-09-16T23:52:40.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.650+0000 7f961effd700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f961806e300 0x7f96180707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:40.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.650+0000 7f961effd700 1 -- 172.21.15.42:0/232101940 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f962808f650 con 0x7f962c0ffb40 2024-09-16T23:52:40.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.650+0000 7f9631762700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f961806e300 0x7f96180707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:40.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.651+0000 7f9631762700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f961806e300 0x7f96180707c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f962c102fa0 tx=0x7f962000a380 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:40.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.655+0000 7f961effd700 1 -- 172.21.15.42:0/232101940 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f9628058d20 con 0x7f962c0ffb40 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:52:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:40 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/2473867123' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:40.780 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:40.780 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:52:40.780 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:40.780 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:52:40.780 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:52:40.780 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:52:40.780 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:40 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/2473867123' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:52:40.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.916+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f962c1a2fb0 con 0x7f962c0ffb40 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.916+0000 7f961effd700 1 -- 172.21.15.42:0/232101940 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f962805c3e0 con 0x7f962c0ffb40 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:e17 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 17 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:52:40.915 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:48:49.729605+0000 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:52:40.916 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:52:40.917 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:52:40.918 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:52:40.918 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:52:40.918 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:40.918 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f961806e300 msgr2=0x7f96180707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f961806e300 0x7f96180707c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f962c102fa0 tx=0x7f962000a380 comp rx=0 tx=0).stop 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c0ffb40 msgr2=0x7f962c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c0ffb40 0x7f962c1017b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9628006010 tx=0x7f962800bd70 comp rx=0 tx=0).stop 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 shutdown_connections 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f962c0ffb40 0x7f962c1017b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f961806e300 0x7f96180707c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 --2- 172.21.15.42:0/232101940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f962c105bf0 0x7f962c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 >> 172.21.15.42:0/232101940 conn(0x7f962c0fb610 msgr2=0x7f962c068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:40.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 shutdown_connections 2024-09-16T23:52:40.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:40.919+0000 7f96341c7700 1 -- 172.21.15.42:0/232101940 wait complete. 2024-09-16T23:52:40.920 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 -- 172.21.15.42:0/929788552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40fff50 msgr2=0x7f8dc4105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 --2- 172.21.15.42:0/929788552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40fff50 0x7f8dc4105840 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f8db4009910 tx=0x7f8db4009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 -- 172.21.15.42:0/929788552 shutdown_connections 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 --2- 172.21.15.42:0/929788552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40fff50 0x7f8dc4105840 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 --2- 172.21.15.42:0/929788552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc40ff5a0 0x7f8dc40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 -- 172.21.15.42:0/929788552 >> 172.21.15.42:0/929788552 conn(0x7f8dc4074e30 msgr2=0x7f8dc4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 -- 172.21.15.42:0/929788552 shutdown_connections 2024-09-16T23:52:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.031+0000 7f8dcc355700 1 -- 172.21.15.42:0/929788552 wait complete. 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dcc355700 1 Processor -- start 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dcc355700 1 -- start start 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dcc355700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40ff5a0 0x7f8dc419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dcc355700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc40fff50 0x7f8dc419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dcc355700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc419d180 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dcc355700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc41966c0 con 0x7f8dc40fff50 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dca0f1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40ff5a0 0x7f8dc419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:41.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dca0f1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40ff5a0 0x7f8dc419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53564/0 (socket says 172.21.15.42:53564) 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.032+0000 7f8dca0f1700 1 -- 172.21.15.42:0/1827852578 learned_addr learned my addr 172.21.15.42:0/1827852578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dca0f1700 1 -- 172.21.15.42:0/1827852578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc40fff50 msgr2=0x7f8dc419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dca0f1700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc40fff50 0x7f8dc419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dca0f1700 1 -- 172.21.15.42:0/1827852578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40095f0 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dca0f1700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40ff5a0 0x7f8dc419c640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c930 tx=0x7f8dc000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dbb7fe700 1 -- 172.21.15.42:0/1827852578 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc0007ad0 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dbb7fe700 1 -- 172.21.15.42:0/1827852578 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dc0007c30 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc41969a0 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dbb7fe700 1 -- 172.21.15.42:0/1827852578 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dc0018610 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.033+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc4196ef0 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.034+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc4109200 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.034+0000 7f8dbb7fe700 1 -- 172.21.15.42:0/1827852578 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7f8dc000f450 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.035+0000 7f8dbb7fe700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f8db006e300 0x7f8db00707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.035+0000 7f8dbb7fe700 1 -- 172.21.15.42:0/1827852578 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f8dc008e1c0 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.035+0000 7f8dc98f0700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f8db006e300 0x7f8db00707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:41.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.035+0000 7f8dc98f0700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f8db006e300 0x7f8db00707c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db400b5c0 tx=0x7f8db4005960 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:41.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.039+0000 7f8dbb7fe700 1 -- 172.21.15.42:0/1827852578 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7f8dc005b440 con 0x7f8dc40ff5a0 2024-09-16T23:52:41.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.281+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 --> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dc4197170 con 0x7f8db006e300 2024-09-16T23:52:41.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.282+0000 7f8dbb7fe700 1 -- 172.21.15.42:0/1827852578 <== mgr.14646 v2:172.21.15.42:6800/2643542629 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f8dc4197170 con 0x7f8db006e300 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "2/23 daemons upgraded", 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mon daemons", 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:52:41.280 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f8db006e300 msgr2=0x7f8db00707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f8db006e300 0x7f8db00707c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db400b5c0 tx=0x7f8db4005960 comp rx=0 tx=0).stop 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40ff5a0 msgr2=0x7f8dc419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40ff5a0 0x7f8dc419c640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c930 tx=0x7f8dc000ccf0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 shutdown_connections 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8dc40ff5a0 0x7f8dc419c640 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7f8db006e300 0x7f8db00707c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 --2- 172.21.15.42:0/1827852578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8dc40fff50 0x7f8dc419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 >> 172.21.15.42:0/1827852578 conn(0x7f8dc4074e30 msgr2=0x7f8dc40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:41.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 shutdown_connections 2024-09-16T23:52:41.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.284+0000 7f8dcc355700 1 -- 172.21.15.42:0/1827852578 wait complete. 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 -- 172.21.15.42:0/1436344053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30101850 msgr2=0x7eff301058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 --2- 172.21.15.42:0/1436344053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30101850 0x7eff301058a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009910 tx=0x7eff1c009c20 comp rx=0 tx=0).stop 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 -- 172.21.15.42:0/1436344053 shutdown_connections 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 --2- 172.21.15.42:0/1436344053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30101850 0x7eff301058a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 --2- 172.21.15.42:0/1436344053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff30100ea0 0x7eff30101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 -- 172.21.15.42:0/1436344053 >> 172.21.15.42:0/1436344053 conn(0x7eff300fc750 msgr2=0x7eff300feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 -- 172.21.15.42:0/1436344053 shutdown_connections 2024-09-16T23:52:41.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.392+0000 7eff36e43700 1 -- 172.21.15.42:0/1436344053 wait complete. 2024-09-16T23:52:41.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff36e43700 1 Processor -- start 2024-09-16T23:52:41.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff36e43700 1 -- start start 2024-09-16T23:52:41.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff36e43700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30100ea0 0x7eff30073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:41.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff36e43700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff30101850 0x7eff300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:41.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff36e43700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30073b10 con 0x7eff30100ea0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff36e43700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff30073c50 con 0x7eff30101850 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff34bdf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30100ea0 0x7eff30073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff34bdf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30100ea0 0x7eff30073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53590/0 (socket says 172.21.15.42:53590) 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff34bdf700 1 -- 172.21.15.42:0/1418944734 learned_addr learned my addr 172.21.15.42:0/1418944734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff2ffff700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff30101850 0x7eff300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff34bdf700 1 -- 172.21.15.42:0/1418944734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff30101850 msgr2=0x7eff300735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff34bdf700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff30101850 0x7eff300735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.393+0000 7eff34bdf700 1 -- 172.21.15.42:0/1418944734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c0095f0 con 0x7eff30100ea0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.394+0000 7eff34bdf700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30100ea0 0x7eff30073090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d8d0 tx=0x7eff2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.394+0000 7eff2dffb700 1 -- 172.21.15.42:0/1418944734 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff24009940 con 0x7eff30100ea0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.394+0000 7eff2dffb700 1 -- 172.21.15.42:0/1418944734 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff24010460 con 0x7eff30100ea0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.394+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff301a2b00 con 0x7eff30100ea0 2024-09-16T23:52:41.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.394+0000 7eff2dffb700 1 -- 172.21.15.42:0/1418944734 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2400f5f0 con 0x7eff30100ea0 2024-09-16T23:52:41.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.394+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff301a2f10 con 0x7eff30100ea0 2024-09-16T23:52:41.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.395+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff301092a0 con 0x7eff30100ea0 2024-09-16T23:52:41.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.395+0000 7eff2dffb700 1 -- 172.21.15.42:0/1418944734 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 32) v1 ==== 92404+0+0 (secure 0 0 0) 0x7eff240105d0 con 0x7eff30100ea0 2024-09-16T23:52:41.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.396+0000 7eff2dffb700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7eff2006e300 0x7eff200707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:52:41.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.396+0000 7eff2dffb700 1 -- 172.21.15.42:0/1418944734 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5898+0+0 (secure 0 0 0) 0x7eff2408dfb0 con 0x7eff30100ea0 2024-09-16T23:52:41.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.396+0000 7eff2ffff700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7eff2006e300 0x7eff200707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:52:41.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.397+0000 7eff2ffff700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7eff2006e300 0x7eff200707c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0059d0 tx=0x7eff1c005960 comp rx=0 tx=0).ready entity=mgr.14646 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:52:41.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.401+0000 7eff2dffb700 1 -- 172.21.15.42:0/1418944734 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186835 (secure 0 0 0) 0x7eff24057700 con 0x7eff30100ea0 2024-09-16T23:52:41.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.700+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7eff30108c80 con 0x7eff30100ea0 2024-09-16T23:52:41.698 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:41 smithi042 ceph-mon[27914]: Upgrade: Updating mon.smithi042 2024-09-16T23:52:41.698 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:41 smithi042 ceph-mon[27914]: Deploying daemon mon.smithi042 on smithi042 2024-09-16T23:52:41.698 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:41 smithi042 ceph-mon[27914]: pgmap v86: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:41.698 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:41 smithi042 ceph-mon[27914]: from='client.? 172.21.15.42:0/232101940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:52:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.700+0000 7eff2dffb700 1 -- 172.21.15.42:0/1418944734 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7eff2405adc0 con 0x7eff30100ea0 2024-09-16T23:52:41.698 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:52:41.699 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:52:41.699 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:52:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7eff2006e300 msgr2=0x7eff200707c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7eff2006e300 0x7eff200707c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff1c0059d0 tx=0x7eff1c005960 comp rx=0 tx=0).stop 2024-09-16T23:52:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30100ea0 msgr2=0x7eff30073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30100ea0 0x7eff30073090 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d8d0 tx=0x7eff2400dc90 comp rx=0 tx=0).stop 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 shutdown_connections 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7eff30100ea0 0x7eff30073090 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.42:6800/2643542629,v1:172.21.15.42:6801/2643542629] conn(0x7eff2006e300 0x7eff200707c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.702+0000 7eff36e43700 1 --2- 172.21.15.42:0/1418944734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff30101850 0x7eff300735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.703+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 >> 172.21.15.42:0/1418944734 conn(0x7eff300fc750 msgr2=0x7eff300fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.703+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 shutdown_connections 2024-09-16T23:52:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:52:41.703+0000 7eff36e43700 1 -- 172.21.15.42:0/1418944734 wait complete. 2024-09-16T23:52:41.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:41 smithi043 ceph-mon[35191]: Upgrade: Updating mon.smithi042 2024-09-16T23:52:41.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:41 smithi043 ceph-mon[35191]: Deploying daemon mon.smithi042 on smithi042 2024-09-16T23:52:41.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:41 smithi043 ceph-mon[35191]: pgmap v86: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:41.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:41 smithi043 ceph-mon[35191]: from='client.? 172.21.15.42:0/232101940' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:52:42.377 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:42 smithi042 systemd[1]: Stopping Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:52:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:42 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[27910]: 2024-09-16T23:52:42.378+0000 7ff781f9e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi042 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:52:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:42 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[27910]: 2024-09-16T23:52:42.378+0000 7ff781f9e700 -1 mon.smithi042@0(leader) e2 *** Got Signal Terminated *** 2024-09-16T23:52:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:42 smithi042 podman[110888]: 2024-09-16 23:52:42.55263872 +0000 UTC m=+0.288140361 container died 37006d2f61124c3b00092c5ff50edb0067221d792bab9f4099bdc71b932caa8a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-09-16T23:52:43.186 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:42 smithi042 podman[110888]: 2024-09-16 23:52:42.950513388 +0000 UTC m=+0.686015017 container cleanup 37006d2f61124c3b00092c5ff50edb0067221d792bab9f4099bdc71b932caa8a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-09-16T23:52:43.186 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:42 smithi042 bash[110888]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042 2024-09-16T23:52:43.507 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:43 smithi042 podman[110900]: 2024-09-16 23:52:43.189419421 +0000 UTC m=+0.636721557 container remove 37006d2f61124c3b00092c5ff50edb0067221d792bab9f4099bdc71b932caa8a (image=quay.io/ceph/ceph:v18.2.1, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T23:52:43.507 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:43 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service: Deactivated successfully. 2024-09-16T23:52:43.507 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:43 smithi042 systemd[1]: Stopped Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:52:43.507 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:43 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service: Consumed 15.046s CPU time. 2024-09-16T23:52:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:43 smithi042 systemd[1]: Starting Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:52:44.040 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:43 smithi042 podman[111037]: 2024-09-16 23:52:43.682510897 +0000 UTC m=+0.016536457 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:52:44.041 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:43 smithi042 podman[111037]: 2024-09-16 23:52:43.807665379 +0000 UTC m=+0.141690929 container create 7ace5d09425f790362c026295ed17ada57fa3cb56be7dbe2da6f3f77cbc73e61 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T23:52:44.041 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 podman[111037]: 2024-09-16 23:52:44.043669618 +0000 UTC m=+0.377695174 container init 7ace5d09425f790362c026295ed17ada57fa3cb56be7dbe2da6f3f77cbc73e61 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T23:52:44.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 podman[111037]: 2024-09-16 23:52:44.047079204 +0000 UTC m=+0.381104755 container start 7ace5d09425f790362c026295ed17ada57fa3cb56be7dbe2da6f3f77cbc73e61 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T23:52:44.319 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable), process ceph-mon, pid 2 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: pidfile_write: ignore empty --pid-file 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: load: jerasure load: lrc 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: RocksDB version: 7.9.2 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Git sha 0 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Compile date 2024-09-16 15:09:42 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: DB SUMMARY 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: DB Session ID: BN6DL9MBWOKH0YUQ7T2D 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: CURRENT file: CURRENT 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1517 Bytes 2024-09-16T23:52:44.320 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi042/store.db dir, Total Num: 1, files: 000032.sst 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi042/store.db: 000030.log size: 3902365 ; 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.error_if_exists: 0 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.create_if_missing: 0 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.paranoid_checks: 1 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.env: 0x55d708af9940 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.info_log: 0x55d709a2b8c0 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.statistics: (nil) 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.use_fsync: 0 2024-09-16T23:52:44.321 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_log_file_size: 0 2024-09-16T23:52:44.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T23:52:44.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T23:52:44.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T23:52:44.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T23:52:44.322 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.allow_fallocate: 1 2024-09-16T23:52:44.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T23:52:44.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T23:52:44.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.use_direct_reads: 0 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.db_log_dir: 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.wal_dir: 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T23:52:44.324 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.write_buffer_manager: 0x55d709a3adc0 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.unordered_write: 0 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T23:52:44.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.row_cache: None 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.wal_filter: None 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.two_write_queues: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.wal_compression: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.atomic_flush: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.log_readahead_size: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T23:52:44.326 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_background_jobs: 2 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_background_compactions: -1 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_subcompactions: 1 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T23:52:44.327 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_open_files: -1 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_background_flushes: -1 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Compression algorithms supported: 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kZSTD supported: 0 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kXpressCompression supported: 0 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kBZip2Compression supported: 0 2024-09-16T23:52:44.328 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kLZ4Compression supported: 1 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kZlibCompression supported: 1 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: kSnappyCompression supported: 1 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000015 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.merge_operator: 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_filter: None 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T23:52:44.329 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d709a2b520) 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: cache_index_and_filter_blocks: 1 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: pin_top_level_index_and_filter: 1 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: index_type: 0 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: data_block_index_type: 0 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: index_shortening: 1 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: checksum: 4 2024-09-16T23:52:44.330 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: no_block_cache: 0 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache: 0x55d709a1f1f0 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache_name: BinnedLRUCache 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache_options: 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: capacity : 536870912 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: num_shard_bits : 4 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: strict_capacity_limit : 0 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: high_pri_pool_ratio: 0.000 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_cache_compressed: (nil) 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: persistent_cache: (nil) 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_size: 4096 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_size_deviation: 10 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_restart_interval: 16 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: index_block_restart_interval: 1 2024-09-16T23:52:44.331 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: metadata_block_size: 4096 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: partition_filters: 0 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: use_delta_encoding: 1 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: filter_policy: bloomfilter 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: whole_key_filtering: 1 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: verify_compression: 0 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: read_amp_bytes_per_bit: 0 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: format_version: 5 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: enable_index_compression: 1 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: block_align: 0 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: max_auto_readahead_size: 262144 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: prepopulate_block_cache: 0 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: initial_auto_readahead_size: 8192 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T23:52:44.332 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression: NoCompression 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.num_levels: 7 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T23:52:44.333 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T23:52:44.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T23:52:44.335 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T23:52:44.336 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.inplace_update_support: 0 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.bloom_locality: 0 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.max_successive_merges: 0 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T23:52:44.337 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.ttl: 2592000 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.enable_blob_files: false 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.min_blob_size: 0 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T23:52:44.338 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi042/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12105, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: db6b78af-ad2b-4105-a874-aacbde5ad83e 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726530764074104, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726530764085964, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 3383840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12107, "largest_seqno": 15106, "table_properties": {"data_size": 3372156, "index_size": 7181, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3461, "raw_key_size": 30009, "raw_average_key_size": 22, "raw_value_size": 3345572, "raw_average_value_size": 2456, "num_data_blocks": 327, "num_entries": 1362, "num_filter_entries": 1362, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726530764, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "db6b78af-ad2b-4105-a874-aacbde5ad83e", "db_session_id": "BN6DL9MBWOKH0YUQ7T2D", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726530764086063, "job": 1, "event": "recovery_finished"} 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T23:52:44.339 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi042/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d709a4ce00 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: DB pointer 0x55d709b48000 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** DB Stats ** 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** Compaction Stats [default] ** 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: L0 1/0 3.23 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 582.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: L6 1/0 5.54 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Sum 2/0 8.76 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 582.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 582.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** Compaction Stats [default] ** 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 582.0 0.01 0.00 1 0.006 0 0 0.0 0.0 2024-09-16T23:52:44.341 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Cumulative compaction: 0.00 GB write, 222.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Interval compaction: 0.00 GB write, 222.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Block cache BinnedLRUCache@0x55d709a1f1f0#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-09-16T23:52:44.342 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: starting mon.smithi042 rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi042 fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???) e2 preinit fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???).mds e17 new map 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???).mds e17 print_map 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: e17 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: legacy client fscid: 1 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Filesystem 'cephfs' (1) 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: fs_name cephfs 2024-09-16T23:52:44.343 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: epoch 17 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: modified 2024-09-16T23:48:49.729605+0000 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: tableserver 0 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: root 0 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: session_timeout 60 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: session_autoclose 300 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: max_file_size 1099511627776 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: max_xattr_size 65536 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: required_client_features {} 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: last_failure 0 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: last_failure_osd_epoch 46 2024-09-16T23:52:44.344 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: max_mds 2 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: in 0,1 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: up {0=14478,1=24269} 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: failed 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: damaged 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: stopped 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: data_pools [3] 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: metadata_pool 2 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: inline_data enabled 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: balancer 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: bal_rank_mask -1 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: standby_count_wanted 1 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: qdb_cluster leader: 0 members: 2024-09-16T23:52:44.345 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: [mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: [mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Standby daemons: 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: [mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: [mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 ceph-mon[111056]: mon.smithi042@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 bash[111037]: 7ace5d09425f790362c026295ed17ada57fa3cb56be7dbe2da6f3f77cbc73e61 2024-09-16T23:52:44.346 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:44 smithi042 systemd[1]: Started Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:52:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:52:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:52:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: pgmap v87: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: pgmap v88: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: mon.smithi042 calling monitor election 2024-09-16T23:52:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: mon.smithi042 is new leader, mons smithi042,smithi043 in quorum (ranks 0,1) 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: monmap epoch 2 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: last_changed 2024-09-16T23:41:43.205311+0000 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: created 2024-09-16T23:38:57.273836+0000 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: min_mon_release 18 (reef) 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: election_strategy: 1 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.smithi042 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.smithi043 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: osdmap e48: 6 total, 6 up, 6 in 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: mgrmap e32: smithi042.zrkumz(active, since 2m), standbys: smithi043.xveyou 2024-09-16T23:52:45.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:52:45.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:52:45.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:52:45.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: from='mgr.14646 ' entity='' 2024-09-16T23:52:45.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:45 smithi043 ceph-mon[35191]: mgrmap e33: smithi042.zrkumz(active, since 2m), standbys: smithi043.xveyou 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: from='mgr.14646 172.21.15.42:0/3860944463' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: pgmap v87: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: pgmap v88: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: mon.smithi042 calling monitor election 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: mon.smithi042 is new leader, mons smithi042,smithi043 in quorum (ranks 0,1) 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: monmap epoch 2 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:52:45.891 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: last_changed 2024-09-16T23:41:43.205311+0000 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: created 2024-09-16T23:38:57.273836+0000 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: min_mon_release 18 (reef) 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: election_strategy: 1 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.smithi042 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.smithi043 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: osdmap e48: 6 total, 6 up, 6 in 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: mgrmap e32: smithi042.zrkumz(active, since 2m), standbys: smithi043.xveyou 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:52:45.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: from='mgr.14646 ' entity='' 2024-09-16T23:52:45.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:45 smithi042 ceph-mon[111056]: mgrmap e33: smithi042.zrkumz(active, since 2m), standbys: smithi043.xveyou 2024-09-16T23:52:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou restarted 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: Active manager daemon smithi042.zrkumz restarted 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: Activating manager daemon smithi042.zrkumz 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: mgrmap e34: smithi042.zrkumz(active, starting, since 0.0055829s), standbys: smithi043.xveyou 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:52:50.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:52:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:52:50.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:50 smithi043 ceph-mon[35191]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:52:50.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: Standby manager daemon smithi043.xveyou restarted 2024-09-16T23:52:50.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: Standby manager daemon smithi043.xveyou started 2024-09-16T23:52:50.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/crt"}]: dispatch 2024-09-16T23:52:50.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T23:52:50.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi043.xveyou/key"}]: dispatch 2024-09-16T23:52:50.898 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.? 172.21.15.43:0/3397185723' entity='mgr.smithi043.xveyou' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: Active manager daemon smithi042.zrkumz restarted 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: Activating manager daemon smithi042.zrkumz 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: mgrmap e34: smithi042.zrkumz(active, starting, since 0.0055829s), standbys: smithi043.xveyou 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi042.zrkumz", "id": "smithi042.zrkumz"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr metadata", "who": "smithi043.xveyou", "id": "smithi043.xveyou"}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:52:50.899 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T23:52:50.900 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:50 smithi042 ceph-mon[111056]: Manager daemon smithi042.zrkumz is now available 2024-09-16T23:52:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:51 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:51 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:51 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:52:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:51 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:52:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:51 smithi043 ceph-mon[35191]: mgrmap e35: smithi042.zrkumz(active, since 1.01059s), standbys: smithi043.xveyou 2024-09-16T23:52:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:51 smithi043 ceph-mon[35191]: pgmap v3: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:52:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:52:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/mirror_snapshot_schedule"}]: dispatch 2024-09-16T23:52:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi042.zrkumz/trash_purge_schedule"}]: dispatch 2024-09-16T23:52:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:51 smithi042 ceph-mon[111056]: mgrmap e35: smithi042.zrkumz(active, since 1.01059s), standbys: smithi043.xveyou 2024-09-16T23:52:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:51 smithi042 ceph-mon[111056]: pgmap v3: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:53.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:53 smithi043 ceph-mon[35191]: pgmap v4: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:53.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:53 smithi043 ceph-mon[35191]: mgrmap e36: smithi042.zrkumz(active, since 2s), standbys: smithi043.xveyou 2024-09-16T23:52:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:53 smithi042 ceph-mon[111056]: pgmap v4: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:53 smithi042 ceph-mon[111056]: mgrmap e36: smithi042.zrkumz(active, since 2s), standbys: smithi043.xveyou 2024-09-16T23:52:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: [16/Sep/2024:23:52:52] ENGINE Bus STARTING 2024-09-16T23:52:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: [16/Sep/2024:23:52:52] ENGINE Serving on http://172.21.15.42:8765 2024-09-16T23:52:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: [16/Sep/2024:23:52:53] ENGINE Serving on https://172.21.15.42:7150 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: [16/Sep/2024:23:52:53] ENGINE Bus STARTED 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: [16/Sep/2024:23:52:53] ENGINE Error in HTTPServer.serve 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Traceback (most recent call last): 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._connections.run(self.expiration_interval) 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._run(expiration_interval) 2024-09-16T23:52:54.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: s = self.context.wrap_socket( 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: return self.sslsocket_class._create( 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T23:52:54.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self.do_handshake() 2024-09-16T23:52:54.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T23:52:54.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: self._sslobj.do_handshake() 2024-09-16T23:52:54.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T23:52:54.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:54.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:54.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:54.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:54 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: [16/Sep/2024:23:52:52] ENGINE Bus STARTING 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: [16/Sep/2024:23:52:52] ENGINE Serving on http://172.21.15.42:8765 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: [16/Sep/2024:23:52:53] ENGINE Serving on https://172.21.15.42:7150 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: [16/Sep/2024:23:52:53] ENGINE Bus STARTED 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: [16/Sep/2024:23:52:53] ENGINE Error in HTTPServer.serve 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: Traceback (most recent call last): 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._connections.run(self.expiration_interval) 2024-09-16T23:52:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._run(expiration_interval) 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: s = self.context.wrap_socket( 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T23:52:55.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: return self.sslsocket_class._create( 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self.do_handshake() 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: self._sslobj.do_handshake() 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:55.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:55.026 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:55.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:55 smithi043 ceph-mon[35191]: pgmap v5: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:55.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:55 smithi043 ceph-mon[35191]: mgrmap e37: smithi042.zrkumz(active, since 4s), standbys: smithi043.xveyou 2024-09-16T23:52:56.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:55 smithi042 ceph-mon[111056]: pgmap v5: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:56.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:55 smithi042 ceph-mon[111056]: mgrmap e37: smithi042.zrkumz(active, since 4s), standbys: smithi043.xveyou 2024-09-16T23:52:56.974 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:56 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:56.974 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:56 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:56.974 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:56 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:56.974 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:56 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:56.974 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:56 smithi042 ceph-mon[111056]: pgmap v6: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:57.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:56 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:57.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:56 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:57.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:56 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:57.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:56 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:52:57.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:56 smithi043 ceph-mon[35191]: pgmap v6: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:52:58.965 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:52:58 smithi042 ceph-mon[111056]: pgmap v7: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-16T23:52:59.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:52:58 smithi043 ceph-mon[35191]: pgmap v7: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Detected new or changed devices on smithi042 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:53:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: pgmap v8: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Detected new or changed devices on smithi042 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi042:/etc/ceph/ceph.conf 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi043:/etc/ceph/ceph.conf 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T23:53:01.074 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: Updating smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: pgmap v8: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.075 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:01.076 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:00 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:02.007 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:01 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:02.008 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:01 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:02.008 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:01 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T23:53:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:02.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T23:53:03.340 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-mon[35191]: Upgrade: Updating mon.smithi043 2024-09-16T23:53:03.340 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:03.340 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:53:03.340 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:53:03.340 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-mon[35191]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:03.340 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-mon[35191]: Deploying daemon mon.smithi043 on smithi043 2024-09-16T23:53:03.340 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-mon[35191]: pgmap v9: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T23:53:03.502 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:03 smithi042 ceph-mon[111056]: Upgrade: Updating mon.smithi043 2024-09-16T23:53:03.503 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:03 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:03.503 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:03 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:53:03.503 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:03 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:53:03.503 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:03 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:03.503 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:03 smithi042 ceph-mon[111056]: Deploying daemon mon.smithi043 on smithi043 2024-09-16T23:53:03.503 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:03 smithi042 ceph-mon[111056]: pgmap v9: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T23:53:03.887 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 systemd[1]: Stopping Ceph mon.smithi043 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:53:04.362 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043[35182]: 2024-09-16T23:53:03.888+0000 7f366a0af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi043 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:53:04.363 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:03 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043[35182]: 2024-09-16T23:53:03.888+0000 7f366a0af700 -1 mon.smithi043@1(peon) e2 *** Got Signal Terminated *** 2024-09-16T23:53:04.363 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:04 smithi043 podman[81351]: 2024-09-16 23:53:04.05297696 +0000 UTC m=+0.270296179 container died 6770c421b06c1219e02d62a06fa0050930ad365e4878cd719eef5d0c7e4c4b34 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T23:53:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:04 smithi043 podman[81351]: 2024-09-16 23:53:04.471864486 +0000 UTC m=+0.689183708 container cleanup 6770c421b06c1219e02d62a06fa0050930ad365e4878cd719eef5d0c7e4c4b34 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1) 2024-09-16T23:53:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:04 smithi043 bash[81351]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043 2024-09-16T23:53:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:04 smithi043 podman[81364]: 2024-09-16 23:53:04.848377896 +0000 UTC m=+0.789243062 container remove 6770c421b06c1219e02d62a06fa0050930ad365e4878cd719eef5d0c7e4c4b34 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_CLEAN=True) 2024-09-16T23:53:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi043.service: Deactivated successfully. 2024-09-16T23:53:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 systemd[1]: Stopped Ceph mon.smithi043 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:53:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi043.service: Consumed 8.174s CPU time. 2024-09-16T23:53:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 systemd[1]: Starting Ceph mon.smithi043 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:53:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 podman[81491]: 2024-09-16 23:53:05.411716589 +0000 UTC m=+0.016688065 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:53:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 podman[81491]: 2024-09-16 23:53:05.54349672 +0000 UTC m=+0.148468177 container create a860b22006ae274b3a4a87469809c1722e0306795e61b6af0bd95b5e1648ed7d (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T23:53:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 podman[81491]: 2024-09-16 23:53:05.821507634 +0000 UTC m=+0.426479110 container init a860b22006ae274b3a4a87469809c1722e0306795e61b6af0bd95b5e1648ed7d (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T23:53:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 podman[81491]: 2024-09-16 23:53:05.824987033 +0000 UTC m=+0.429958490 container start a860b22006ae274b3a4a87469809c1722e0306795e61b6af0bd95b5e1648ed7d (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable), process ceph-mon, pid 2 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: pidfile_write: ignore empty --pid-file 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: load: jerasure load: lrc 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: RocksDB version: 7.9.2 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Git sha 0 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Compile date 2024-09-16 15:09:42 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: DB SUMMARY 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: DB Session ID: IR45SQVMBONJSQMZI4TX 2024-09-16T23:53:06.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: CURRENT file: CURRENT 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1411 Bytes 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi043/store.db dir, Total Num: 1, files: 000027.sst 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi043/store.db: 000025.log size: 8500038 ; 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.error_if_exists: 0 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.create_if_missing: 0 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.paranoid_checks: 1 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.env: 0x55bb62d81940 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T23:53:06.130 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.info_log: 0x55bb644bf8c0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.statistics: (nil) 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.use_fsync: 0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_log_file_size: 0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.allow_fallocate: 1 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.use_direct_reads: 0 2024-09-16T23:53:06.131 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T23:53:06.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T23:53:06.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.db_log_dir: 2024-09-16T23:53:06.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.wal_dir: 2024-09-16T23:53:06.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T23:53:06.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T23:53:06.132 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.write_buffer_manager: 0x55bb644cedc0 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T23:53:06.133 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.unordered_write: 0 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.row_cache: None 2024-09-16T23:53:06.134 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.wal_filter: None 2024-09-16T23:53:06.135 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.two_write_queues: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.wal_compression: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.atomic_flush: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.log_readahead_size: 0 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T23:53:06.136 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T23:53:06.137 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T23:53:06.137 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T23:53:06.137 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T23:53:06.137 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T23:53:06.137 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T23:53:06.137 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_background_jobs: 2 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_background_compactions: -1 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_subcompactions: 1 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T23:53:06.138 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_open_files: -1 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_background_flushes: -1 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Compression algorithms supported: 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kZSTD supported: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kXpressCompression supported: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kBZip2Compression supported: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T23:53:06.139 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kLZ4Compression supported: 1 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kZlibCompression supported: 1 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: kSnappyCompression supported: 1 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi043/store.db/MANIFEST-000010 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.merge_operator: 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_filter: None 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T23:53:06.140 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bb644bf520) 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: index_type: 0 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: data_block_index_type: 0 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: index_shortening: 1 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: checksum: 4 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: no_block_cache: 0 2024-09-16T23:53:06.141 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_cache: 0x55bb644b31f0 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_cache_options: 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: capacity : 536870912 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: num_shard_bits : 4 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: strict_capacity_limit : 0 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_cache_compressed: (nil) 2024-09-16T23:53:06.142 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: persistent_cache: (nil) 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_size: 4096 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_size_deviation: 10 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_restart_interval: 16 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: index_block_restart_interval: 1 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: metadata_block_size: 4096 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: partition_filters: 0 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: use_delta_encoding: 1 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: filter_policy: bloomfilter 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: whole_key_filtering: 1 2024-09-16T23:53:06.143 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: verify_compression: 0 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: format_version: 5 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: enable_index_compression: 1 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: block_align: 0 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: max_auto_readahead_size: 262144 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: prepopulate_block_cache: 0 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: initial_auto_readahead_size: 8192 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T23:53:06.144 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression: NoCompression 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.num_levels: 7 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T23:53:06.145 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T23:53:06.146 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T23:53:06.147 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T23:53:06.147 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T23:53:06.147 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T23:53:06.147 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T23:53:06.147 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T23:53:06.147 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T23:53:06.147 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T23:53:06.148 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T23:53:06.149 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.inplace_update_support: 0 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T23:53:06.150 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.bloom_locality: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.max_successive_merges: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.ttl: 2592000 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.enable_blob_files: false 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.min_blob_size: 0 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T23:53:06.151 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T23:53:06.152 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T23:53:06.152 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T23:53:06.152 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T23:53:06.152 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T23:53:06.152 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T23:53:06.152 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T23:53:06.152 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi043/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12132, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9e596109-1579-4284-914b-d1705added39 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726530785852263, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726530785872130, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 5037748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12137, "largest_seqno": 15925, "table_properties": {"data_size": 5024078, "index_size": 8655, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3973, "raw_key_size": 35012, "raw_average_key_size": 22, "raw_value_size": 4993052, "raw_average_value_size": 3174, "num_data_blocks": 391, "num_entries": 1573, "num_filter_entries": 1573, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726530785, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9e596109-1579-4284-914b-d1705added39", "db_session_id": "IR45SQVMBONJSQMZI4TX", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726530785872261, "job": 1, "event": "recovery_finished"} 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T23:53:06.153 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi043/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bb644e0e00 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: DB pointer 0x55bb645dc000 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ** DB Stats ** 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T23:53:06.154 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: L0 1/0 4.80 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 650.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: L6 1/0 5.54 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Sum 2/0 10.34 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 650.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 650.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ** Compaction Stats [default] ** 2024-09-16T23:53:06.155 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 650.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Cumulative compaction: 0.00 GB write, 210.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T23:53:06.156 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Interval compaction: 0.00 GB write, 210.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T23:53:06.157 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T23:53:06.157 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Block cache BinnedLRUCache@0x55bb644b31f0#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-16T23:53:06.157 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-09-16T23:53:06.157 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.157 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T23:53:06.157 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: starting mon.smithi043 rank 1 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi043 fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???) e2 preinit fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???).mds e18 new map 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???).mds e18 print_map 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: e18 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: btime 2024-09-16T23:52:59:633980+0000 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: legacy client fscid: 1 2024-09-16T23:53:06.158 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Filesystem 'cephfs' (1) 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: fs_name cephfs 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: epoch 18 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: modified 2024-09-16T23:52:59.633974+0000 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: tableserver 0 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: root 0 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: session_timeout 60 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: session_autoclose 300 2024-09-16T23:53:06.159 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: max_file_size 1099511627776 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: max_xattr_size 65536 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: required_client_features {} 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: last_failure 0 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: last_failure_osd_epoch 46 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: max_mds 2 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: in 0,1 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: up {0=14478,1=24269} 2024-09-16T23:53:06.160 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: failed 2024-09-16T23:53:06.161 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: damaged 2024-09-16T23:53:06.161 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: stopped 2024-09-16T23:53:06.161 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: data_pools [3] 2024-09-16T23:53:06.161 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: metadata_pool 2 2024-09-16T23:53:06.161 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: inline_data enabled 2024-09-16T23:53:06.161 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: balancer 2024-09-16T23:53:06.161 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: bal_rank_mask -1 2024-09-16T23:53:06.162 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: standby_count_wanted 1 2024-09-16T23:53:06.162 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: qdb_cluster leader: 14478 members: 14478,24269 2024-09-16T23:53:06.162 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: [mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:06.162 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: [mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:06.162 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.162 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.162 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: Standby daemons: 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: [mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout: [mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 ceph-mon[81510]: mon.smithi043@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-09-16T23:53:06.163 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 bash[81491]: a860b22006ae274b3a4a87469809c1722e0306795e61b6af0bd95b5e1648ed7d 2024-09-16T23:53:06.164 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:05 smithi043 systemd[1]: Started Ceph mon.smithi043 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:53:07.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:53:07.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: mon.smithi043 calling monitor election 2024-09-16T23:53:07.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: mon.smithi042 calling monitor election 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: mon.smithi042 is new leader, mons smithi042,smithi043 in quorum (ranks 0,1) 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: monmap epoch 3 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: last_changed 2024-09-16T23:53:05.913995+0000 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: created 2024-09-16T23:38:57.273836+0000 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: min_mon_release 19 (squid) 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: election_strategy: 1 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.smithi042 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.smithi043 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:53:07.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: mgrmap e37: smithi042.zrkumz(active, since 15s), standbys: smithi043.xveyou 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:07.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:06 smithi042 ceph-mon[111056]: pgmap v11: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T23:53:07.285 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.285 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.285 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:53:07.285 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: mon.smithi043 calling monitor election 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: mon.smithi042 calling monitor election 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: mon.smithi042 is new leader, mons smithi042,smithi043 in quorum (ranks 0,1) 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: monmap epoch 3 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: last_changed 2024-09-16T23:53:05.913995+0000 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: created 2024-09-16T23:38:57.273836+0000 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: min_mon_release 19 (squid) 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: election_strategy: 1 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.smithi042 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: 1: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.smithi043 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:53:07.286 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: mgrmap e37: smithi042.zrkumz(active, since 15s), standbys: smithi043.xveyou 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:07.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:06 smithi043 ceph-mon[81510]: pgmap v11: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T23:53:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:08 smithi043 ceph-mon[81510]: pgmap v12: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T23:53:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:08 smithi042 ceph-mon[111056]: pgmap v12: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T23:53:10.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:10.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:10.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:10.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:10.039 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:10.039 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:10.040 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:10.040 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:10 smithi042 ceph-mon[111056]: pgmap v13: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:11.092 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:10 smithi043 ceph-mon[81510]: pgmap v13: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 -- 172.21.15.42:0/1773465966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 msgr2=0x7fadd0101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1773465966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd0101260 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009910 tx=0x7fadc0009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 -- 172.21.15.42:0/1773465966 shutdown_connections 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1773465966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd0101830 0x7fadd0105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1773465966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd0101260 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 -- 172.21.15.42:0/1773465966 >> 172.21.15.42:0/1773465966 conn(0x7fadd00fc710 msgr2=0x7fadd00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 -- 172.21.15.42:0/1773465966 shutdown_connections 2024-09-16T23:53:11.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.827+0000 7fadd4a26700 1 -- 172.21.15.42:0/1773465966 wait complete. 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.828+0000 7fadd4a26700 1 Processor -- start 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.828+0000 7fadd4a26700 1 -- start start 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadd4a26700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadd4a26700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd0101830 0x7fadd019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadd4a26700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd019d220 con 0x7fadd0100e80 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadd4a26700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd01966d0 con 0x7fadd0101830 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadce59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadce59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39494/0 (socket says 172.21.15.42:39494) 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadce59c700 1 -- 172.21.15.42:0/1052018773 learned_addr learned my addr 172.21.15.42:0/1052018773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadcdd9b700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd0101830 0x7fadd019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadce59c700 1 -- 172.21.15.42:0/1052018773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd0101830 msgr2=0x7fadd019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadce59c700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd0101830 0x7fadd019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:11.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadce59c700 1 -- 172.21.15.42:0/1052018773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00095f0 con 0x7fadd0100e80 2024-09-16T23:53:11.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadce59c700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd019c650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005f50 tx=0x7fadc000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:11.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadbf7fe700 1 -- 172.21.15.42:0/1052018773 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc001d070 con 0x7fadd0100e80 2024-09-16T23:53:11.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd0196950 con 0x7fadd0100e80 2024-09-16T23:53:11.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadbf7fe700 1 -- 172.21.15.42:0/1052018773 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc000b610 con 0x7fadd0100e80 2024-09-16T23:53:11.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadbf7fe700 1 -- 172.21.15.42:0/1052018773 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc0022470 con 0x7fadd0100e80 2024-09-16T23:53:11.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.829+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd0196e40 con 0x7fadd0100e80 2024-09-16T23:53:11.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.831+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0109220 con 0x7fadd0100e80 2024-09-16T23:53:11.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.831+0000 7fadbf7fe700 1 -- 172.21.15.42:0/1052018773 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fadc0003780 con 0x7fadd0100e80 2024-09-16T23:53:11.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.832+0000 7fadbf7fe700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadb806e7c0 0x7fadb8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:11.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.832+0000 7fadbf7fe700 1 -- 172.21.15.42:0/1052018773 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fadc008fc10 con 0x7fadd0100e80 2024-09-16T23:53:11.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.832+0000 7fadcdd9b700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadb806e7c0 0x7fadb8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:11.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.832+0000 7fadcdd9b700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadb806e7c0 0x7fadb8070c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fadd0197b40 tx=0x7fadc4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:11.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:11.837+0000 7fadbf7fe700 1 -- 172.21.15.42:0/1052018773 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fadc005c770 con 0x7fadd0100e80 2024-09-16T23:53:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.074+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadd0197060 con 0x7fadb806e7c0 2024-09-16T23:53:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.075+0000 7fadbf7fe700 1 -- 172.21.15.42:0/1052018773 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fadd0197060 con 0x7fadb806e7c0 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadb806e7c0 msgr2=0x7fadb8070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadb806e7c0 0x7fadb8070c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fadd0197b40 tx=0x7fadc4009260 comp rx=0 tx=0).stop 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 msgr2=0x7fadd019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd019c650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005f50 tx=0x7fadc000f690 comp rx=0 tx=0).stop 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 shutdown_connections 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadb806e7c0 0x7fadb8070c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd0100e80 0x7fadd019c650 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 --2- 172.21.15.42:0/1052018773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd0101830 0x7fadd019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 >> 172.21.15.42:0/1052018773 conn(0x7fadd00fc710 msgr2=0x7fadd00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 shutdown_connections 2024-09-16T23:53:12.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.077+0000 7fadd4a26700 1 -- 172.21.15.42:0/1052018773 wait complete. 2024-09-16T23:53:12.086 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:53:12.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 -- 172.21.15.42:0/816946601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 msgr2=0x7f1f840ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 --2- 172.21.15.42:0/816946601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f840ff980 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009910 tx=0x7f1f74009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:12.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 -- 172.21.15.42:0/816946601 shutdown_connections 2024-09-16T23:53:12.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 --2- 172.21.15.42:0/816946601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f840fff50 0x7f1f84105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 --2- 172.21.15.42:0/816946601 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f840ff980 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 -- 172.21.15.42:0/816946601 >> 172.21.15.42:0/816946601 conn(0x7f1f84074e30 msgr2=0x7f1f84075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:12.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 -- 172.21.15.42:0/816946601 shutdown_connections 2024-09-16T23:53:12.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.187+0000 7f1f8a668700 1 -- 172.21.15.42:0/816946601 wait complete. 2024-09-16T23:53:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.188+0000 7f1f8a668700 1 Processor -- start 2024-09-16T23:53:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.188+0000 7f1f8a668700 1 -- start start 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.188+0000 7f1f8a668700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f8419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f8a668700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f840fff50 0x7f1f8419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f8a668700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419d250 con 0x7f1f840ff5a0 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f8a668700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84196700 con 0x7f1f840fff50 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f83fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f8419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f83fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f8419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39510/0 (socket says 172.21.15.42:39510) 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f83fff700 1 -- 172.21.15.42:0/4003541654 learned_addr learned my addr 172.21.15.42:0/4003541654 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f837fe700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f840fff50 0x7f1f8419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f83fff700 1 -- 172.21.15.42:0/4003541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f840fff50 msgr2=0x7f1f8419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f83fff700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f840fff50 0x7f1f8419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f83fff700 1 -- 172.21.15.42:0/4003541654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740095f0 con 0x7f1f840ff5a0 2024-09-16T23:53:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f83fff700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f8419c680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f740092e0 tx=0x7f1f7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f817fa700 1 -- 172.21.15.42:0/4003541654 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7401d070 con 0x7f1f840ff5a0 2024-09-16T23:53:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f817fa700 1 -- 172.21.15.42:0/4003541654 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f74005e00 con 0x7f1f840ff5a0 2024-09-16T23:53:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f84196980 con 0x7f1f840ff5a0 2024-09-16T23:53:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f817fa700 1 -- 172.21.15.42:0/4003541654 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f74017400 con 0x7f1f840ff5a0 2024-09-16T23:53:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.189+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f84196e70 con 0x7f1f840ff5a0 2024-09-16T23:53:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.190+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f84109270 con 0x7f1f840ff5a0 2024-09-16T23:53:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.191+0000 7f1f817fa700 1 -- 172.21.15.42:0/4003541654 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f1f7400f460 con 0x7f1f840ff5a0 2024-09-16T23:53:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.191+0000 7f1f817fa700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f6c06e7b0 0x7f1f6c070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.191+0000 7f1f817fa700 1 -- 172.21.15.42:0/4003541654 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1f7408fd00 con 0x7f1f840ff5a0 2024-09-16T23:53:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.192+0000 7f1f837fe700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f6c06e7b0 0x7f1f6c070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.192+0000 7f1f837fe700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f6c06e7b0 0x7f1f6c070c70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005950 tx=0x7f1f780058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:12.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.196+0000 7f1f817fa700 1 -- 172.21.15.42:0/4003541654 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1f7405c840 con 0x7f1f840ff5a0 2024-09-16T23:53:12.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.428+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f841970e0 con 0x7f1f6c06e7b0 2024-09-16T23:53:12.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.429+0000 7f1f817fa700 1 -- 172.21.15.42:0/4003541654 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f1f841970e0 con 0x7f1f6c06e7b0 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f6c06e7b0 msgr2=0x7f1f6c070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f6c06e7b0 0x7f1f6c070c70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005950 tx=0x7f1f780058e0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 msgr2=0x7f1f8419c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f8419c680 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f740092e0 tx=0x7f1f7400be20 comp rx=0 tx=0).stop 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 shutdown_connections 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f6c06e7b0 0x7f1f6c070c70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f840ff5a0 0x7f1f8419c680 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 --2- 172.21.15.42:0/4003541654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f840fff50 0x7f1f8419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 >> 172.21.15.42:0/4003541654 conn(0x7f1f84074e30 msgr2=0x7f1f840fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:12.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 shutdown_connections 2024-09-16T23:53:12.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.431+0000 7f1f8a668700 1 -- 172.21.15.42:0/4003541654 wait complete. 2024-09-16T23:53:12.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.538+0000 7f0d60532700 1 -- 172.21.15.42:0/3032808108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58101850 msgr2=0x7f0d581058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.538+0000 7f0d60532700 1 --2- 172.21.15.42:0/3032808108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58101850 0x7f0d581058a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d48009910 tx=0x7f0d48009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.538+0000 7f0d60532700 1 -- 172.21.15.42:0/3032808108 shutdown_connections 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.538+0000 7f0d60532700 1 --2- 172.21.15.42:0/3032808108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58101850 0x7f0d581058a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.538+0000 7f0d60532700 1 --2- 172.21.15.42:0/3032808108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d58100ea0 0x7f0d58101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 -- 172.21.15.42:0/3032808108 >> 172.21.15.42:0/3032808108 conn(0x7f0d580fc750 msgr2=0x7f0d580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 -- 172.21.15.42:0/3032808108 shutdown_connections 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 -- 172.21.15.42:0/3032808108 wait complete. 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 Processor -- start 2024-09-16T23:53:12.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 -- start start 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58100ea0 0x7f0d5819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d58101850 0x7f0d5819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5819d2d0 con 0x7f0d58100ea0 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.539+0000 7f0d60532700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d58196780 con 0x7f0d58101850 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5e2ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58100ea0 0x7f0d5819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5e2ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58100ea0 0x7f0d5819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39524/0 (socket says 172.21.15.42:39524) 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5e2ce700 1 -- 172.21.15.42:0/3245629426 learned_addr learned my addr 172.21.15.42:0/3245629426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5dacd700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d58101850 0x7f0d5819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5e2ce700 1 -- 172.21.15.42:0/3245629426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d58101850 msgr2=0x7f0d5819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5e2ce700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d58101850 0x7f0d5819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5e2ce700 1 -- 172.21.15.42:0/3245629426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480095f0 con 0x7f0d58100ea0 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5dacd700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d58101850 0x7f0d5819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d5e2ce700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58100ea0 0x7f0d5819c700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400d8d0 tx=0x7f0d5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d4f7fe700 1 -- 172.21.15.42:0/3245629426 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d54009940 con 0x7f0d58100ea0 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d4f7fe700 1 -- 172.21.15.42:0/3245629426 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d54010460 con 0x7f0d58100ea0 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d58196a60 con 0x7f0d58100ea0 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d4f7fe700 1 -- 172.21.15.42:0/3245629426 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d5400f5f0 con 0x7f0d58100ea0 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.540+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d58196fb0 con 0x7f0d58100ea0 2024-09-16T23:53:12.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.541+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d581092a0 con 0x7f0d58100ea0 2024-09-16T23:53:12.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.542+0000 7f0d4f7fe700 1 -- 172.21.15.42:0/3245629426 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f0d540105d0 con 0x7f0d58100ea0 2024-09-16T23:53:12.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.542+0000 7f0d4f7fe700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0d4406e7c0 0x7f0d44070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.542+0000 7f0d4f7fe700 1 -- 172.21.15.42:0/3245629426 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0d5408e850 con 0x7f0d58100ea0 2024-09-16T23:53:12.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.543+0000 7f0d5dacd700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0d4406e7c0 0x7f0d44070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.543+0000 7f0d5dacd700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0d4406e7c0 0x7f0d44070c80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d58197bf0 tx=0x7f0d4801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:12.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.547+0000 7f0d4f7fe700 1 -- 172.21.15.42:0/3245629426 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0d5405b3b0 con 0x7f0d58100ea0 2024-09-16T23:53:12.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.765+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d58197da0 con 0x7f0d4406e7c0 2024-09-16T23:53:12.763 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:12 smithi042 ceph-mon[111056]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:12.763 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:12 smithi042 ceph-mon[111056]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:12.763 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:12 smithi042 ceph-mon[111056]: pgmap v14: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.775+0000 7f0d4f7fe700 1 -- 172.21.15.42:0/3245629426 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0d58197da0 con 0x7f0d4406e7c0 2024-09-16T23:53:12.772 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:53:12.772 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (112s) 16s ago 12m 15.2M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (12m) 16s ago 12m 7990k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (11m) 4s ago 11m 7910k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (12m) 16s ago 12m 6665k - 18.2.1 5be31c24972a 3e1529c81158 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (11m) 4s ago 11m 6669k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (88s) 16s ago 12m 76.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (6m) 16s ago 6m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (7m) 16s ago 7m 15.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (6m) 4s ago 6m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (6m) 4s ago 6m 16.9M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (3m) 16s ago 14m 471M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (2m) 4s ago 11m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (29s) 16s ago 14m 31.3M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:53:12.773 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (7s) 4s ago 11m 26.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (2m) 16s ago 12m 16.6M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (2m) 4s ago 11m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (10m) 16s ago 10m 206M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (10m) 16s ago 10m 201M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (9m) 16s ago 9m 163M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (9m) 4s ago 9m 236M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (8m) 4s ago 8m 222M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (8m) 4s ago 8m 202M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:53:12.774 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (2m) 16s ago 12m 67.4M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:53:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0d4406e7c0 msgr2=0x7f0d44070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0d4406e7c0 0x7f0d44070c80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d58197bf0 tx=0x7f0d4801a040 comp rx=0 tx=0).stop 2024-09-16T23:53:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58100ea0 msgr2=0x7f0d5819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58100ea0 0x7f0d5819c700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400d8d0 tx=0x7f0d5400dc90 comp rx=0 tx=0).stop 2024-09-16T23:53:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 shutdown_connections 2024-09-16T23:53:12.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0d4406e7c0 0x7f0d44070c80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0d58100ea0 0x7f0d5819c700 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 --2- 172.21.15.42:0/3245629426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d58101850 0x7f0d5819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 >> 172.21.15.42:0/3245629426 conn(0x7f0d580fc750 msgr2=0x7f0d580fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 shutdown_connections 2024-09-16T23:53:12.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.777+0000 7f0d60532700 1 -- 172.21.15.42:0/3245629426 wait complete. 2024-09-16T23:53:12.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 -- 172.21.15.42:0/2120782836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac105bd0 msgr2=0x7fd5ac105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 --2- 172.21.15.42:0/2120782836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac105bd0 0x7fd5ac105fb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00b0a0 tx=0x7fd59c00b3b0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 -- 172.21.15.42:0/2120782836 shutdown_connections 2024-09-16T23:53:12.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 --2- 172.21.15.42:0/2120782836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5ac0ffc20 0x7fd5ac100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 --2- 172.21.15.42:0/2120782836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac105bd0 0x7fd5ac105fb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 -- 172.21.15.42:0/2120782836 >> 172.21.15.42:0/2120782836 conn(0x7fd5ac075670 msgr2=0x7fd5ac075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 -- 172.21.15.42:0/2120782836 shutdown_connections 2024-09-16T23:53:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.895+0000 7fd5b3122700 1 -- 172.21.15.42:0/2120782836 wait complete. 2024-09-16T23:53:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b3122700 1 Processor -- start 2024-09-16T23:53:12.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b3122700 1 -- start start 2024-09-16T23:53:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b3122700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac0ffc20 0x7fd5ac19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b3122700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5ac105bd0 0x7fd5ac19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b3122700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac19d1f0 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b3122700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac1966a0 con 0x7fd5ac105bd0 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b0ebe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac0ffc20 0x7fd5ac19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b0ebe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac0ffc20 0x7fd5ac19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49528/0 (socket says 172.21.15.42:49528) 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b0ebe700 1 -- 172.21.15.42:0/1123137830 learned_addr learned my addr 172.21.15.42:0/1123137830 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5abfff700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5ac105bd0 0x7fd5ac19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b0ebe700 1 -- 172.21.15.42:0/1123137830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5ac105bd0 msgr2=0x7fd5ac19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b0ebe700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5ac105bd0 0x7fd5ac19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.896+0000 7fd5b0ebe700 1 -- 172.21.15.42:0/1123137830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c009d00 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.897+0000 7fd5b0ebe700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac0ffc20 0x7fd5ac19c620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd59c000f80 tx=0x7fd59c00bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.897+0000 7fd5a9ffb700 1 -- 172.21.15.42:0/1123137830 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd59c00e040 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.897+0000 7fd5a9ffb700 1 -- 172.21.15.42:0/1123137830 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd59c003c10 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.897+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac196920 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.897+0000 7fd5a9ffb700 1 -- 172.21.15.42:0/1123137830 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd59c01d680 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.897+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac196e10 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.898+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac04f2e0 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.899+0000 7fd5a9ffb700 1 -- 172.21.15.42:0/1123137830 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fd59c019070 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.899+0000 7fd5a9ffb700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd59806e7c0 0x7fd598070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.899+0000 7fd5a9ffb700 1 -- 172.21.15.42:0/1123137830 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd59c08fcb0 con 0x7fd5ac0ffc20 2024-09-16T23:53:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.899+0000 7fd5abfff700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd59806e7c0 0x7fd598070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:12.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.900+0000 7fd5abfff700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd59806e7c0 0x7fd598070c80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0ffa40 tx=0x7fd594009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:12.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:12.904+0000 7fd5a9ffb700 1 -- 172.21.15.42:0/1123137830 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd59c05c940 con 0x7fd5ac0ffc20 2024-09-16T23:53:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:12 smithi043 ceph-mon[81510]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:12 smithi043 ceph-mon[81510]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:12 smithi043 ceph-mon[81510]: pgmap v14: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:13.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.229+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5ac197bb0 con 0x7fd5ac0ffc20 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.230+0000 7fd5a9ffb700 1 -- 172.21.15.42:0/1123137830 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+694 (secure 0 0 0) 0x7fd59c003d80 con 0x7fd5ac0ffc20 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:53:13.228 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:53:13.229 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:53:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd59806e7c0 msgr2=0x7fd598070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd59806e7c0 0x7fd598070c80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0ffa40 tx=0x7fd594009260 comp rx=0 tx=0).stop 2024-09-16T23:53:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac0ffc20 msgr2=0x7fd5ac19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac0ffc20 0x7fd5ac19c620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd59c000f80 tx=0x7fd59c00bc60 comp rx=0 tx=0).stop 2024-09-16T23:53:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 shutdown_connections 2024-09-16T23:53:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd59806e7c0 0x7fd598070c80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ac0ffc20 0x7fd5ac19c620 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 --2- 172.21.15.42:0/1123137830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5ac105bd0 0x7fd5ac19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 >> 172.21.15.42:0/1123137830 conn(0x7fd5ac075670 msgr2=0x7fd5ac0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.232+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 shutdown_connections 2024-09-16T23:53:13.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.233+0000 7fd5b3122700 1 -- 172.21.15.42:0/1123137830 wait complete. 2024-09-16T23:53:13.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.342+0000 7fccfccfd700 1 -- 172.21.15.42:0/952051574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 msgr2=0x7fccf8102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.342+0000 7fccfccfd700 1 --2- 172.21.15.42:0/952051574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf8102220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009940 tx=0x7fcce8009c50 comp rx=0 tx=0).stop 2024-09-16T23:53:13.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 -- 172.21.15.42:0/952051574 shutdown_connections 2024-09-16T23:53:13.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 --2- 172.21.15.42:0/952051574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf8102760 0x7fccf810ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 --2- 172.21.15.42:0/952051574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf8102220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 -- 172.21.15.42:0/952051574 >> 172.21.15.42:0/952051574 conn(0x7fccf80fb650 msgr2=0x7fccf80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:13.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 -- 172.21.15.42:0/952051574 shutdown_connections 2024-09-16T23:53:13.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 -- 172.21.15.42:0/952051574 wait complete. 2024-09-16T23:53:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 Processor -- start 2024-09-16T23:53:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.343+0000 7fccfccfd700 1 -- start start 2024-09-16T23:53:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccfccfd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf80ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccfccfd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf8102760 0x7fccf80fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccfccfd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8100520 con 0x7fccf8101e40 2024-09-16T23:53:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccfccfd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8100660 con 0x7fccf8102760 2024-09-16T23:53:13.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf80ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf80ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49560/0 (socket says 172.21.15.42:49560) 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf659c700 1 -- 172.21.15.42:0/3833100670 learned_addr learned my addr 172.21.15.42:0/3833100670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf5d9b700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf8102760 0x7fccf80fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf659c700 1 -- 172.21.15.42:0/3833100670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf8102760 msgr2=0x7fccf80fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf659c700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf8102760 0x7fccf80fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf659c700 1 -- 172.21.15.42:0/3833100670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce80095f0 con 0x7fccf8101e40 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccf659c700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf80ffaa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcce800b580 tx=0x7fcce80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fcce77fe700 1 -- 172.21.15.42:0/3833100670 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce801d070 con 0x7fccf8101e40 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fcce77fe700 1 -- 172.21.15.42:0/3833100670 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcce800f460 con 0x7fccf8101e40 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fcce77fe700 1 -- 172.21.15.42:0/3833100670 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce8021640 con 0x7fccf8101e40 2024-09-16T23:53:13.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.344+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf81008e0 con 0x7fccf8101e40 2024-09-16T23:53:13.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.345+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf81a2d90 con 0x7fccf8101e40 2024-09-16T23:53:13.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.345+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf8108350 con 0x7fccf8101e40 2024-09-16T23:53:13.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.346+0000 7fcce77fe700 1 -- 172.21.15.42:0/3833100670 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fcce800fab0 con 0x7fccf8101e40 2024-09-16T23:53:13.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.346+0000 7fcce77fe700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcce006e7c0 0x7fcce0070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:13.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.346+0000 7fcce77fe700 1 -- 172.21.15.42:0/3833100670 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcce808faf0 con 0x7fccf8101e40 2024-09-16T23:53:13.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.347+0000 7fccf5d9b700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcce006e7c0 0x7fcce0070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:13.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.347+0000 7fccf5d9b700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcce006e7c0 0x7fcce0070c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fccec005950 tx=0x7fccec0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:13.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.351+0000 7fcce77fe700 1 -- 172.21.15.42:0/3833100670 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcce8058b20 con 0x7fccf8101e40 2024-09-16T23:53:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.611+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fccf8100fd0 con 0x7fccf8101e40 2024-09-16T23:53:13.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.612+0000 7fcce77fe700 1 -- 172.21.15.42:0/3833100670 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1955 (secure 0 0 0) 0x7fcce805c1e0 con 0x7fccf8101e40 2024-09-16T23:53:13.609 INFO:teuthology.orchestra.run.smithi042.stdout:e18 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:52:59:633980+0000 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 18 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:52:59.633974+0000 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:53:13.610 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:53:13.611 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 14478,24269 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:13.612 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:53:13.613 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:13.613 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:13.613 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:13.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcce006e7c0 msgr2=0x7fcce0070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcce006e7c0 0x7fcce0070c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fccec005950 tx=0x7fccec0058e0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 msgr2=0x7fccf80ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf80ffaa0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcce800b580 tx=0x7fcce80056a0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 shutdown_connections 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcce006e7c0 0x7fcce0070c80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fccf8101e40 0x7fccf80ffaa0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 --2- 172.21.15.42:0/3833100670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccf8102760 0x7fccf80fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 >> 172.21.15.42:0/3833100670 conn(0x7fccf80fb650 msgr2=0x7fccf8105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 shutdown_connections 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.614+0000 7fccfccfd700 1 -- 172.21.15.42:0/3833100670 wait complete. 2024-09-16T23:53:13.614 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 18 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2546659330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60101850 msgr2=0x7f5e601058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2546659330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60101850 0x7f5e601058a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009910 tx=0x7f5e54009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2546659330 shutdown_connections 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2546659330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60101850 0x7f5e601058a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2546659330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e60100ea0 0x7f5e60101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2546659330 >> 172.21.15.42:0/2546659330 conn(0x7f5e600fc750 msgr2=0x7f5e600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2546659330 shutdown_connections 2024-09-16T23:53:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2546659330 wait complete. 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.723+0000 7f5e64d6d700 1 Processor -- start 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e64d6d700 1 -- start start 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e64d6d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60100ea0 0x7f5e6019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e64d6d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e60101850 0x7f5e6019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e64d6d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6019d2d0 con 0x7f5e60100ea0 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e64d6d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60196780 con 0x7f5e60101850 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60100ea0 0x7f5e6019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60100ea0 0x7f5e6019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49572/0 (socket says 172.21.15.42:49572) 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e5e59c700 1 -- 172.21.15.42:0/2264364327 learned_addr learned my addr 172.21.15.42:0/2264364327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e5dd9b700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e60101850 0x7f5e6019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e5e59c700 1 -- 172.21.15.42:0/2264364327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e60101850 msgr2=0x7f5e6019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e5e59c700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e60101850 0x7f5e6019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.724+0000 7f5e5e59c700 1 -- 172.21.15.42:0/2264364327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540095f0 con 0x7f5e60100ea0 2024-09-16T23:53:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.725+0000 7f5e5e59c700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60100ea0 0x7f5e6019c700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d8d0 tx=0x7f5e5000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.725+0000 7f5e4f7fe700 1 -- 172.21.15.42:0/2264364327 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e500098a0 con 0x7f5e60100ea0 2024-09-16T23:53:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.725+0000 7f5e4f7fe700 1 -- 172.21.15.42:0/2264364327 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e50010460 con 0x7f5e60100ea0 2024-09-16T23:53:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.725+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e60196a60 con 0x7f5e60100ea0 2024-09-16T23:53:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.725+0000 7f5e4f7fe700 1 -- 172.21.15.42:0/2264364327 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e5000f5f0 con 0x7f5e60100ea0 2024-09-16T23:53:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.725+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e60196fb0 con 0x7f5e60100ea0 2024-09-16T23:53:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.726+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e601092a0 con 0x7f5e60100ea0 2024-09-16T23:53:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.726+0000 7f5e4f7fe700 1 -- 172.21.15.42:0/2264364327 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f5e50009a00 con 0x7f5e60100ea0 2024-09-16T23:53:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.727+0000 7f5e4f7fe700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e4806e7c0 0x7f5e48070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.727+0000 7f5e4f7fe700 1 -- 172.21.15.42:0/2264364327 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5e5008e620 con 0x7f5e60100ea0 2024-09-16T23:53:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.727+0000 7f5e5dd9b700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e4806e7c0 0x7f5e48070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.727+0000 7f5e5dd9b700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e4806e7c0 0x7f5e48070c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5e60197bf0 tx=0x7f5e5401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:13.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.731+0000 7f5e4f7fe700 1 -- 172.21.15.42:0/2264364327 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5e5005b200 con 0x7f5e60100ea0 2024-09-16T23:53:13.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.963+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e60197d50 con 0x7f5e4806e7c0 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.964+0000 7f5e4f7fe700 1 -- 172.21.15.42:0/2264364327 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f5e60197d50 con 0x7f5e4806e7c0 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "4/23 daemons upgraded", 2024-09-16T23:53:13.962 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mon daemons", 2024-09-16T23:53:13.963 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:53:13.963 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:53:13.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e4806e7c0 msgr2=0x7f5e48070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e4806e7c0 0x7f5e48070c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5e60197bf0 tx=0x7f5e5401a040 comp rx=0 tx=0).stop 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60100ea0 msgr2=0x7f5e6019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60100ea0 0x7f5e6019c700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d8d0 tx=0x7f5e5000dbe0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 shutdown_connections 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e4806e7c0 0x7f5e48070c80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e60100ea0 0x7f5e6019c700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 --2- 172.21.15.42:0/2264364327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e60101850 0x7f5e6019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 >> 172.21.15.42:0/2264364327 conn(0x7f5e600fc750 msgr2=0x7f5e600fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 shutdown_connections 2024-09-16T23:53:13.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:13.967+0000 7f5e64d6d700 1 -- 172.21.15.42:0/2264364327 wait complete. 2024-09-16T23:53:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 -- 172.21.15.42:0/4051481882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 msgr2=0x7f66f41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 --2- 172.21.15.42:0/4051481882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f41047a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009910 tx=0x7f66e0009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 -- 172.21.15.42:0/4051481882 shutdown_connections 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 --2- 172.21.15.42:0/4051481882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f41047a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 --2- 172.21.15.42:0/4051481882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f40ffda0 0x7f66f4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 -- 172.21.15.42:0/4051481882 >> 172.21.15.42:0/4051481882 conn(0x7f66f40fb650 msgr2=0x7f66f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 -- 172.21.15.42:0/4051481882 shutdown_connections 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.075+0000 7f66fad46700 1 -- 172.21.15.42:0/4051481882 wait complete. 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66fad46700 1 Processor -- start 2024-09-16T23:53:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66fad46700 1 -- start start 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66fad46700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f40ffda0 0x7f66f4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66fad46700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66fad46700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41022c0 con 0x7f66f4100750 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66fad46700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4102400 con 0x7f66f40ffda0 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66f3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f4101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49600/0 (socket says 172.21.15.42:49600) 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.076+0000 7f66f3fff700 1 -- 172.21.15.42:0/3707124661 learned_addr learned my addr 172.21.15.42:0/3707124661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f8ae2700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f40ffda0 0x7f66f4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f3fff700 1 -- 172.21.15.42:0/3707124661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f40ffda0 msgr2=0x7f66f4101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f3fff700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f40ffda0 0x7f66f4101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f3fff700 1 -- 172.21.15.42:0/3707124661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e00095f0 con 0x7f66f4100750 2024-09-16T23:53:14.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f3fff700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f4101d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66e00059d0 tx=0x7f66e000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f1ffb700 1 -- 172.21.15.42:0/3707124661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e001d070 con 0x7f66f4100750 2024-09-16T23:53:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f1ffb700 1 -- 172.21.15.42:0/3707124661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66e000b610 con 0x7f66f4100750 2024-09-16T23:53:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f4102680 con 0x7f66f4100750 2024-09-16T23:53:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66f1ffb700 1 -- 172.21.15.42:0/3707124661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66e0022470 con 0x7f66f4100750 2024-09-16T23:53:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.077+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f41a2da0 con 0x7f66f4100750 2024-09-16T23:53:14.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.078+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f404f2e0 con 0x7f66f4100750 2024-09-16T23:53:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.079+0000 7f66f1ffb700 1 -- 172.21.15.42:0/3707124661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f66e000fc70 con 0x7f66f4100750 2024-09-16T23:53:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.079+0000 7f66f1ffb700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f66e406e7c0 0x7f66e4070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.079+0000 7f66f1ffb700 1 -- 172.21.15.42:0/3707124661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f66e008fbf0 con 0x7f66f4100750 2024-09-16T23:53:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.079+0000 7f66f8ae2700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f66e406e7c0 0x7f66e4070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:14.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.080+0000 7f66f8ae2700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f66e406e7c0 0x7f66e4070c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009bf0 tx=0x7f66e8009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:14.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.083+0000 7f66f1ffb700 1 -- 172.21.15.42:0/3707124661 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f66e0058ca0 con 0x7f66f4100750 2024-09-16T23:53:14.081 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:14.081 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: Detected new or changed devices on smithi043 2024-09-16T23:53:14.081 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:14.081 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:14.081 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: Reconfiguring mon.smithi042 (monmap changed)... 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: Reconfiguring daemon mon.smithi042 on smithi042 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1123137830' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:14.082 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:13 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3833100670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: Detected new or changed devices on smithi043 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: Reconfiguring mon.smithi042 (monmap changed)... 2024-09-16T23:53:14.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:53:14.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:53:14.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:14.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: Reconfiguring daemon mon.smithi042 on smithi042 2024-09-16T23:53:14.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1123137830' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:14.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:13 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3833100670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:53:14.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.402+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f66f41a3010 con 0x7f66f4100750 2024-09-16T23:53:14.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.402+0000 7f66f1ffb700 1 -- 172.21.15.42:0/3707124661 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f66e005c360 con 0x7f66f4100750 2024-09-16T23:53:14.400 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:14.400 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:14.401 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:53:14.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f66e406e7c0 msgr2=0x7f66e4070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f66e406e7c0 0x7f66e4070c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009bf0 tx=0x7f66e8009260 comp rx=0 tx=0).stop 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 msgr2=0x7f66f4101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f4101d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66e00059d0 tx=0x7f66e000f740 comp rx=0 tx=0).stop 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 shutdown_connections 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f66e406e7c0 0x7f66e4070c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66f40ffda0 0x7f66f4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 --2- 172.21.15.42:0/3707124661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f4100750 0x7f66f4101d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 >> 172.21.15.42:0/3707124661 conn(0x7f66f40fb650 msgr2=0x7f66f4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 shutdown_connections 2024-09-16T23:53:14.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:14.405+0000 7f66fad46700 1 -- 172.21.15.42:0/3707124661 wait complete. 2024-09-16T23:53:15.729 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:15.729 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:15.729 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:15.729 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: Reconfiguring mgr.smithi042.zrkumz (monmap changed)... 2024-09-16T23:53:15.730 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:53:15.730 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:53:15.730 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:15.730 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: Reconfiguring daemon mgr.smithi042.zrkumz on smithi042 2024-09-16T23:53:15.730 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3707124661' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:53:15.730 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:15 smithi042 ceph-mon[111056]: pgmap v15: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:15.804 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: Reconfiguring mgr.smithi042.zrkumz (monmap changed)... 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi042.zrkumz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: Reconfiguring daemon mgr.smithi042.zrkumz on smithi042 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3707124661' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:53:15.805 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:15 smithi043 ceph-mon[81510]: pgmap v15: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: Reconfiguring ceph-exporter.smithi042 (monmap changed)... 2024-09-16T23:53:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T23:53:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: Unable to update caps for client.ceph-exporter.smithi042 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi042"}]: dispatch 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: Reconfiguring daemon ceph-exporter.smithi042 on smithi042 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:16 smithi043 ceph-mon[81510]: pgmap v16: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: Reconfiguring ceph-exporter.smithi042 (monmap changed)... 2024-09-16T23:53:17.129 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:17.130 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:17.130 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T23:53:17.130 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: Unable to update caps for client.ceph-exporter.smithi042 2024-09-16T23:53:17.130 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi042"}]: dispatch 2024-09-16T23:53:17.130 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:17.130 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: Reconfiguring daemon ceph-exporter.smithi042 on smithi042 2024-09-16T23:53:17.130 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:16 smithi042 ceph-mon[111056]: pgmap v16: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:18 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:18 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:18 smithi042 ceph-mon[111056]: Reconfiguring crash.smithi042 (monmap changed)... 2024-09-16T23:53:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:18 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:18 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:18 smithi042 ceph-mon[111056]: Reconfiguring daemon crash.smithi042 on smithi042 2024-09-16T23:53:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:18 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:18 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:18 smithi043 ceph-mon[81510]: Reconfiguring crash.smithi042 (monmap changed)... 2024-09-16T23:53:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:18 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:18 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:18 smithi043 ceph-mon[81510]: Reconfiguring daemon crash.smithi042 on smithi042 2024-09-16T23:53:19.498 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:19 smithi042 ceph-mon[111056]: pgmap v17: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:19.498 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:19.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:19.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T23:53:19.499 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:19.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:19 smithi043 ceph-mon[81510]: pgmap v17: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:19.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:19.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:19.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T23:53:19.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:20 smithi043 ceph-mon[81510]: Reconfiguring osd.0 (monmap changed)... 2024-09-16T23:53:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:20 smithi043 ceph-mon[81510]: Reconfiguring daemon osd.0 on smithi042 2024-09-16T23:53:20.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:20 smithi042 ceph-mon[111056]: Reconfiguring osd.0 (monmap changed)... 2024-09-16T23:53:20.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:20 smithi042 ceph-mon[111056]: Reconfiguring daemon osd.0 on smithi042 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: Reconfiguring osd.1 (monmap changed)... 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: Reconfiguring daemon osd.1 on smithi042 2024-09-16T23:53:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: pgmap v18: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:21.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: fsmap cephfs:2 {0=cephfs.smithi042.anvvuy=up:active,1=cephfs.smithi043.pzcdzg=up:active} 2 up:standby 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: Reconfiguring osd.1 (monmap changed)... 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: Reconfiguring daemon osd.1 on smithi042 2024-09-16T23:53:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: pgmap v18: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:21.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:53:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:22 smithi042 ceph-mon[111056]: Reconfiguring osd.2 (monmap changed)... 2024-09-16T23:53:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T23:53:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:22 smithi042 ceph-mon[111056]: Reconfiguring daemon osd.2 on smithi042 2024-09-16T23:53:23.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:22 smithi042 ceph-mon[111056]: pgmap v19: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:22 smithi043 ceph-mon[81510]: Reconfiguring osd.2 (monmap changed)... 2024-09-16T23:53:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T23:53:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:22 smithi043 ceph-mon[81510]: Reconfiguring daemon osd.2 on smithi042 2024-09-16T23:53:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:22 smithi043 ceph-mon[81510]: pgmap v19: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:24 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:24 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:24 smithi042 ceph-mon[111056]: Reconfiguring mds.cephfs.smithi042.ctckas (monmap changed)... 2024-09-16T23:53:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:24 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:24 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:24 smithi042 ceph-mon[111056]: Reconfiguring daemon mds.cephfs.smithi042.ctckas on smithi042 2024-09-16T23:53:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:24 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:24 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:24 smithi043 ceph-mon[81510]: Reconfiguring mds.cephfs.smithi042.ctckas (monmap changed)... 2024-09-16T23:53:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:24 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:24 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:24.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:24 smithi043 ceph-mon[81510]: Reconfiguring daemon mds.cephfs.smithi042.ctckas on smithi042 2024-09-16T23:53:25.256 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:25 smithi042 ceph-mon[111056]: pgmap v20: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:25.256 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:25 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:25.256 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:25 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:25.257 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:25 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:25.257 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:25 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:25 smithi043 ceph-mon[81510]: pgmap v20: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:25 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:25 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:25 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:25 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:26.476 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: Reconfiguring mds.cephfs.smithi042.anvvuy (monmap changed)... 2024-09-16T23:53:26.476 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: Reconfiguring daemon mds.cephfs.smithi042.anvvuy on smithi042 2024-09-16T23:53:26.476 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:26.476 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:26.476 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:26.476 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:26.476 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T23:53:26.477 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi043"}]: dispatch 2024-09-16T23:53:26.477 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:26 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:26.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: Reconfiguring mds.cephfs.smithi042.anvvuy (monmap changed)... 2024-09-16T23:53:26.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: Reconfiguring daemon mds.cephfs.smithi042.anvvuy on smithi042 2024-09-16T23:53:26.525 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:26.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:26.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:26.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T23:53:26.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T23:53:26.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi043"}]: dispatch 2024-09-16T23:53:26.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:26 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:27.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:27 smithi042 ceph-mon[111056]: Reconfiguring ceph-exporter.smithi043 (monmap changed)... 2024-09-16T23:53:27.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:27 smithi042 ceph-mon[111056]: Unable to update caps for client.ceph-exporter.smithi043 2024-09-16T23:53:27.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:27 smithi042 ceph-mon[111056]: Reconfiguring daemon ceph-exporter.smithi043 on smithi043 2024-09-16T23:53:27.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:27 smithi042 ceph-mon[111056]: pgmap v21: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:27.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:27 smithi043 ceph-mon[81510]: Reconfiguring ceph-exporter.smithi043 (monmap changed)... 2024-09-16T23:53:27.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:27 smithi043 ceph-mon[81510]: Unable to update caps for client.ceph-exporter.smithi043 2024-09-16T23:53:27.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:27 smithi043 ceph-mon[81510]: Reconfiguring daemon ceph-exporter.smithi043 on smithi043 2024-09-16T23:53:27.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:27 smithi043 ceph-mon[81510]: pgmap v21: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:28 smithi042 ceph-mon[111056]: Reconfiguring crash.smithi043 (monmap changed)... 2024-09-16T23:53:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:28 smithi042 ceph-mon[111056]: Reconfiguring daemon crash.smithi043 on smithi043 2024-09-16T23:53:28.850 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:28.850 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:28.850 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:28 smithi043 ceph-mon[81510]: Reconfiguring crash.smithi043 (monmap changed)... 2024-09-16T23:53:28.850 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:28.850 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:28.850 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:28 smithi043 ceph-mon[81510]: Reconfiguring daemon crash.smithi043 on smithi043 2024-09-16T23:53:29.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:29 smithi043 ceph-mon[81510]: pgmap v22: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:29.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:29 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:29.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:29 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:29.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:29 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:53:29.603 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:29 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:53:29.603 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:29 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:29 smithi042 ceph-mon[111056]: pgmap v22: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:29 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:29 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:29 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi043.xveyou", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T23:53:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:29 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T23:53:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:29 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:30.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:30 smithi042 ceph-mon[111056]: Reconfiguring mgr.smithi043.xveyou (monmap changed)... 2024-09-16T23:53:30.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:30 smithi042 ceph-mon[111056]: Reconfiguring daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:53:30.853 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:30 smithi043 ceph-mon[81510]: Reconfiguring mgr.smithi043.xveyou (monmap changed)... 2024-09-16T23:53:30.853 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:30 smithi043 ceph-mon[81510]: Reconfiguring daemon mgr.smithi043.xveyou on smithi043 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: Reconfiguring mon.smithi043 (monmap changed)... 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: Reconfiguring daemon mon.smithi043 on smithi043 2024-09-16T23:53:31.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:31 smithi042 ceph-mon[111056]: pgmap v23: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:31.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:31.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:31.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: Reconfiguring mon.smithi043 (monmap changed)... 2024-09-16T23:53:31.872 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T23:53:31.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T23:53:31.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:31.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: Reconfiguring daemon mon.smithi043 on smithi043 2024-09-16T23:53:31.873 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:31 smithi043 ceph-mon[81510]: pgmap v23: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:32 smithi042 ceph-mon[111056]: Reconfiguring osd.3 (monmap changed)... 2024-09-16T23:53:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T23:53:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:32 smithi042 ceph-mon[111056]: Reconfiguring daemon osd.3 on smithi043 2024-09-16T23:53:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:32 smithi042 ceph-mon[111056]: pgmap v24: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:32 smithi043 ceph-mon[81510]: Reconfiguring osd.3 (monmap changed)... 2024-09-16T23:53:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T23:53:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:32 smithi043 ceph-mon[81510]: Reconfiguring daemon osd.3 on smithi043 2024-09-16T23:53:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:32 smithi043 ceph-mon[81510]: pgmap v24: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:34 smithi042 ceph-mon[111056]: Reconfiguring osd.4 (monmap changed)... 2024-09-16T23:53:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T23:53:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:34 smithi042 ceph-mon[111056]: Reconfiguring daemon osd.4 on smithi043 2024-09-16T23:53:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:34 smithi043 ceph-mon[81510]: Reconfiguring osd.4 (monmap changed)... 2024-09-16T23:53:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T23:53:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:34 smithi043 ceph-mon[81510]: Reconfiguring daemon osd.4 on smithi043 2024-09-16T23:53:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:35 smithi043 ceph-mon[81510]: pgmap v25: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T23:53:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:35 smithi042 ceph-mon[111056]: pgmap v25: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T23:53:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:36 smithi042 ceph-mon[111056]: Reconfiguring osd.5 (monmap changed)... 2024-09-16T23:53:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:36 smithi042 ceph-mon[111056]: Reconfiguring daemon osd.5 on smithi043 2024-09-16T23:53:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:53:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:36.868 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:36 smithi043 ceph-mon[81510]: Reconfiguring osd.5 (monmap changed)... 2024-09-16T23:53:36.868 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:36 smithi043 ceph-mon[81510]: Reconfiguring daemon osd.5 on smithi043 2024-09-16T23:53:36.868 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:53:36.868 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:37 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:37 smithi042 ceph-mon[111056]: Reconfiguring mds.cephfs.smithi043.pzcdzg (monmap changed)... 2024-09-16T23:53:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:37 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:37 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:37 smithi042 ceph-mon[111056]: Reconfiguring daemon mds.cephfs.smithi043.pzcdzg on smithi043 2024-09-16T23:53:37.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:37 smithi042 ceph-mon[111056]: pgmap v26: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:37.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:37 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:37.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:37 smithi043 ceph-mon[81510]: Reconfiguring mds.cephfs.smithi043.pzcdzg (monmap changed)... 2024-09-16T23:53:37.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:37 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:37.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:37 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:37.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:37 smithi043 ceph-mon[81510]: Reconfiguring daemon mds.cephfs.smithi043.pzcdzg on smithi043 2024-09-16T23:53:37.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:37 smithi043 ceph-mon[81510]: pgmap v26: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:39 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:39 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:39 smithi042 ceph-mon[111056]: Reconfiguring mds.cephfs.smithi043.riwwhn (monmap changed)... 2024-09-16T23:53:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:39 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:39 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:39 smithi042 ceph-mon[111056]: Reconfiguring daemon mds.cephfs.smithi043.riwwhn on smithi043 2024-09-16T23:53:39.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:39 smithi042 ceph-mon[111056]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:39 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:39 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:39 smithi043 ceph-mon[81510]: Reconfiguring mds.cephfs.smithi043.riwwhn (monmap changed)... 2024-09-16T23:53:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:39 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:53:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:39 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:39 smithi043 ceph-mon[81510]: Reconfiguring daemon mds.cephfs.smithi043.riwwhn on smithi043 2024-09-16T23:53:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:39 smithi043 ceph-mon[81510]: pgmap v27: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:40.797 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:40.797 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:40.797 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:40.797 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:40.798 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:40.798 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:40.798 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]: dispatch 2024-09-16T23:53:40.798 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]': finished 2024-09-16T23:53:40.798 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi043"}]: dispatch 2024-09-16T23:53:40.798 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi043"}]': finished 2024-09-16T23:53:40.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:40.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:40.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:40.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:40.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:40.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:40.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]: dispatch 2024-09-16T23:53:40.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi042"}]': finished 2024-09-16T23:53:40.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi043"}]: dispatch 2024-09-16T23:53:40.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi043"}]': finished 2024-09-16T23:53:41.593 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:41 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all mon 2024-09-16T23:53:41.593 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:41 smithi042 ceph-mon[111056]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:41.593 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:41 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:41.594 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:41 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:41.594 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:41 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:41.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:41 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all mon 2024-09-16T23:53:41.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:41 smithi043 ceph-mon[81510]: pgmap v28: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:41.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:41 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:41.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:41 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi042", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:41.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:41 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:42.740 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:42 smithi042 ceph-mon[111056]: Upgrade: Updating crash.smithi042 (1/2) 2024-09-16T23:53:42.740 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:42 smithi042 ceph-mon[111056]: Deploying daemon crash.smithi042 on smithi042 2024-09-16T23:53:42.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:42 smithi043 ceph-mon[81510]: Upgrade: Updating crash.smithi042 (1/2) 2024-09-16T23:53:42.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:42 smithi043 ceph-mon[81510]: Deploying daemon crash.smithi042 on smithi042 2024-09-16T23:53:43.802 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:43 smithi042 ceph-mon[111056]: pgmap v29: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:43.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:43 smithi043 ceph-mon[81510]: pgmap v29: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:44.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 -- 172.21.15.42:0/1915244618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640ff5a0 msgr2=0x7f42640ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 --2- 172.21.15.42:0/1915244618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640ff5a0 0x7f42640ff980 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4260009910 tx=0x7f4260009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 -- 172.21.15.42:0/1915244618 shutdown_connections 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 --2- 172.21.15.42:0/1915244618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42640fff50 0x7f4264105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 --2- 172.21.15.42:0/1915244618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640ff5a0 0x7f42640ff980 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 -- 172.21.15.42:0/1915244618 >> 172.21.15.42:0/1915244618 conn(0x7f4264074e30 msgr2=0x7f4264075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 -- 172.21.15.42:0/1915244618 shutdown_connections 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.519+0000 7f426bc3b700 1 -- 172.21.15.42:0/1915244618 wait complete. 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f426bc3b700 1 Processor -- start 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f426bc3b700 1 -- start start 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f426bc3b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42640ff5a0 0x7f426419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f426bc3b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640fff50 0x7f426419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:44.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f426bc3b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426419d250 con 0x7f42640fff50 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f426bc3b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4264196700 con 0x7f42640ff5a0 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f42691d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640fff50 0x7f426419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f42691d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640fff50 0x7f426419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37636/0 (socket says 172.21.15.42:37636) 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.520+0000 7f42691d6700 1 -- 172.21.15.42:0/2432157091 learned_addr learned my addr 172.21.15.42:0/2432157091 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f42699d7700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42640ff5a0 0x7f426419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f42691d6700 1 -- 172.21.15.42:0/2432157091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42640ff5a0 msgr2=0x7f426419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f42691d6700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42640ff5a0 0x7f426419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f42691d6700 1 -- 172.21.15.42:0/2432157091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600095f0 con 0x7f42640fff50 2024-09-16T23:53:44.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f42691d6700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640fff50 0x7f426419cbc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f425400e8f0 tx=0x7f425400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f425affd700 1 -- 172.21.15.42:0/2432157091 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4254009790 con 0x7f42640fff50 2024-09-16T23:53:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f425affd700 1 -- 172.21.15.42:0/2432157091 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f425400c810 con 0x7f42640fff50 2024-09-16T23:53:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42641969e0 con 0x7f42640fff50 2024-09-16T23:53:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f425affd700 1 -- 172.21.15.42:0/2432157091 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4254005580 con 0x7f42640fff50 2024-09-16T23:53:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.521+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4264196f30 con 0x7f42640fff50 2024-09-16T23:53:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.523+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4264109270 con 0x7f42640fff50 2024-09-16T23:53:44.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.523+0000 7f425affd700 1 -- 172.21.15.42:0/2432157091 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f4254004a00 con 0x7f42640fff50 2024-09-16T23:53:44.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.524+0000 7f425affd700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f425006e7c0 0x7f4250070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:44.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.524+0000 7f425affd700 1 -- 172.21.15.42:0/2432157091 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4254014070 con 0x7f42640fff50 2024-09-16T23:53:44.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.524+0000 7f42699d7700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f425006e7c0 0x7f4250070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:44.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.525+0000 7f42699d7700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f425006e7c0 0x7f4250070c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f42600059d0 tx=0x7f4260005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:44.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.533+0000 7f425affd700 1 -- 172.21.15.42:0/2432157091 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f425405b660 con 0x7f42640fff50 2024-09-16T23:53:44.766 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:44 smithi042 ceph-mon[111056]: pgmap v30: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:44.766 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:44 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:44.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.768+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4264197d20 con 0x7f425006e7c0 2024-09-16T23:53:44.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.770+0000 7f425affd700 1 -- 172.21.15.42:0/2432157091 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f4264197d20 con 0x7f425006e7c0 2024-09-16T23:53:44.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f425006e7c0 msgr2=0x7f4250070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:44.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f425006e7c0 0x7f4250070c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f42600059d0 tx=0x7f4260005940 comp rx=0 tx=0).stop 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640fff50 msgr2=0x7f426419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640fff50 0x7f426419cbc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f425400e8f0 tx=0x7f425400ec00 comp rx=0 tx=0).stop 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 shutdown_connections 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f425006e7c0 0x7f4250070c80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42640ff5a0 0x7f426419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 --2- 172.21.15.42:0/2432157091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42640fff50 0x7f426419cbc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 >> 172.21.15.42:0/2432157091 conn(0x7f4264074e30 msgr2=0x7f42640fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 shutdown_connections 2024-09-16T23:53:44.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.772+0000 7f426bc3b700 1 -- 172.21.15.42:0/2432157091 wait complete. 2024-09-16T23:53:44.780 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 -- 172.21.15.42:0/496186784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 msgr2=0x7fddd4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 --2- 172.21.15.42:0/496186784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd4101280 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009910 tx=0x7fddc4009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 -- 172.21.15.42:0/496186784 shutdown_connections 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 --2- 172.21.15.42:0/496186784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd4101850 0x7fddd41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 --2- 172.21.15.42:0/496186784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd4101280 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 -- 172.21.15.42:0/496186784 >> 172.21.15.42:0/496186784 conn(0x7fddd40fc750 msgr2=0x7fddd40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 -- 172.21.15.42:0/496186784 shutdown_connections 2024-09-16T23:53:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.882+0000 7fddda262700 1 -- 172.21.15.42:0/496186784 wait complete. 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddda262700 1 Processor -- start 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddda262700 1 -- start start 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddda262700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddda262700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd4101850 0x7fddd419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddda262700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd419d2c0 con 0x7fddd4100ea0 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddda262700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4196770 con 0x7fddd4101850 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddd37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddd37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37680/0 (socket says 172.21.15.42:37680) 2024-09-16T23:53:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddd37fe700 1 -- 172.21.15.42:0/461199711 learned_addr learned my addr 172.21.15.42:0/461199711 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.883+0000 7fddcaffd700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd4101850 0x7fddd419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddd37fe700 1 -- 172.21.15.42:0/461199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd4101850 msgr2=0x7fddd419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddd37fe700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd4101850 0x7fddd419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddd37fe700 1 -- 172.21.15.42:0/461199711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40095f0 con 0x7fddd4100ea0 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddd37fe700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd419c6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddc400f690 tx=0x7fddc400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddd17fa700 1 -- 172.21.15.42:0/461199711 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddc401d070 con 0x7fddd4100ea0 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddd17fa700 1 -- 172.21.15.42:0/461199711 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddc40050d0 con 0x7fddd4100ea0 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd41969f0 con 0x7fddd4100ea0 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddd17fa700 1 -- 172.21.15.42:0/461199711 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddc4022470 con 0x7fddd4100ea0 2024-09-16T23:53:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.884+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd4196ee0 con 0x7fddd4100ea0 2024-09-16T23:53:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.885+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd4109240 con 0x7fddd4100ea0 2024-09-16T23:53:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.885+0000 7fddd17fa700 1 -- 172.21.15.42:0/461199711 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fddc4003780 con 0x7fddd4100ea0 2024-09-16T23:53:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.886+0000 7fddd17fa700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fddc006e7b0 0x7fddc0070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.886+0000 7fddd17fa700 1 -- 172.21.15.42:0/461199711 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fddc408fbd0 con 0x7fddd4100ea0 2024-09-16T23:53:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.886+0000 7fddcaffd700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fddc006e7b0 0x7fddc0070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.887+0000 7fddcaffd700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fddc006e7b0 0x7fddc0070c70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddd4197be0 tx=0x7fddbc009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:44.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:44.891+0000 7fddd17fa700 1 -- 172.21.15.42:0/461199711 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fddc405c860 con 0x7fddd4100ea0 2024-09-16T23:53:45.035 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:44 smithi043 ceph-mon[81510]: pgmap v30: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:45.035 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:44 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:45.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.119+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd4197150 con 0x7fddc006e7b0 2024-09-16T23:53:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.120+0000 7fddd17fa700 1 -- 172.21.15.42:0/461199711 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fddd4197150 con 0x7fddc006e7b0 2024-09-16T23:53:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fddc006e7b0 msgr2=0x7fddc0070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fddc006e7b0 0x7fddc0070c70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddd4197be0 tx=0x7fddbc009260 comp rx=0 tx=0).stop 2024-09-16T23:53:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 msgr2=0x7fddd419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd419c6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fddc400f690 tx=0x7fddc400f6c0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 shutdown_connections 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fddc006e7b0 0x7fddc0070c70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddd4100ea0 0x7fddd419c6f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 --2- 172.21.15.42:0/461199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddd4101850 0x7fddd419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 >> 172.21.15.42:0/461199711 conn(0x7fddd40fc750 msgr2=0x7fddd40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 shutdown_connections 2024-09-16T23:53:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.122+0000 7fddda262700 1 -- 172.21.15.42:0/461199711 wait complete. 2024-09-16T23:53:45.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.230+0000 7f40ffc63700 1 -- 172.21.15.42:0/4213634658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8101e00 msgr2=0x7f40f81021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.230+0000 7f40ffc63700 1 --2- 172.21.15.42:0/4213634658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8101e00 0x7f40f81021e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009910 tx=0x7f40f4009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:45.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.230+0000 7f40ffc63700 1 -- 172.21.15.42:0/4213634658 shutdown_connections 2024-09-16T23:53:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.230+0000 7f40ffc63700 1 --2- 172.21.15.42:0/4213634658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40f8102720 0x7f40f810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.230+0000 7f40ffc63700 1 --2- 172.21.15.42:0/4213634658 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8101e00 0x7f40f81021e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.230+0000 7f40ffc63700 1 -- 172.21.15.42:0/4213634658 >> 172.21.15.42:0/4213634658 conn(0x7f40f80fb650 msgr2=0x7f40f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.230+0000 7f40ffc63700 1 -- 172.21.15.42:0/4213634658 shutdown_connections 2024-09-16T23:53:45.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40ffc63700 1 -- 172.21.15.42:0/4213634658 wait complete. 2024-09-16T23:53:45.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40ffc63700 1 Processor -- start 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40ffc63700 1 -- start start 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40ffc63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40f8101e00 0x7f40f819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40ffc63700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8102720 0x7f40f819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40ffc63700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f819d260 con 0x7f40f8102720 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40ffc63700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f8196710 con 0x7f40f8101e00 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.231+0000 7f40fd1fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8102720 0x7f40f819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40fd1fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8102720 0x7f40f819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37690/0 (socket says 172.21.15.42:37690) 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40fd1fe700 1 -- 172.21.15.42:0/650654860 learned_addr learned my addr 172.21.15.42:0/650654860 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40fd9ff700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40f8101e00 0x7f40f819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40fd1fe700 1 -- 172.21.15.42:0/650654860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40f8101e00 msgr2=0x7f40f819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40fd1fe700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40f8101e00 0x7f40f819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40fd1fe700 1 -- 172.21.15.42:0/650654860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f40095f0 con 0x7f40f8102720 2024-09-16T23:53:45.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40fd1fe700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8102720 0x7f40f819cbd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c8f0 tx=0x7f40e800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:45.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40eeffd700 1 -- 172.21.15.42:0/650654860 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e80041f0 con 0x7f40f8102720 2024-09-16T23:53:45.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40eeffd700 1 -- 172.21.15.42:0/650654860 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40e8004350 con 0x7f40f8102720 2024-09-16T23:53:45.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f81969f0 con 0x7f40f8102720 2024-09-16T23:53:45.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40eeffd700 1 -- 172.21.15.42:0/650654860 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e8003c50 con 0x7f40f8102720 2024-09-16T23:53:45.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.232+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f8196f40 con 0x7f40f8102720 2024-09-16T23:53:45.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.233+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f8108310 con 0x7f40f8102720 2024-09-16T23:53:45.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.234+0000 7f40eeffd700 1 -- 172.21.15.42:0/650654860 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f40e80076d0 con 0x7f40f8102720 2024-09-16T23:53:45.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.234+0000 7f40eeffd700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40e406e7c0 0x7f40e4070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.234+0000 7f40eeffd700 1 -- 172.21.15.42:0/650654860 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f40e808e160 con 0x7f40f8102720 2024-09-16T23:53:45.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.234+0000 7f40fd9ff700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40e406e7c0 0x7f40e4070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.235+0000 7f40fd9ff700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40e406e7c0 0x7f40e4070c80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f40f4006010 tx=0x7f40f401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:45.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.239+0000 7f40eeffd700 1 -- 172.21.15.42:0/650654860 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f40e805aca0 con 0x7f40f8102720 2024-09-16T23:53:45.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.455+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f40f8197210 con 0x7f40e406e7c0 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.465+0000 7f40eeffd700 1 -- 172.21.15.42:0/650654860 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f40f8197210 con 0x7f40e406e7c0 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (2m) 49s ago 13m 15.2M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (13m) 49s ago 13m 7990k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (12m) 36s ago 12m 7910k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 starting - - - - 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (12m) 36s ago 12m 6669k - 18.2.1 5be31c24972a 852244d65d78 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (2m) 49s ago 12m 76.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (7m) 49s ago 7m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (7m) 49s ago 7m 15.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:53:45.463 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (7m) 36s ago 7m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (7m) 36s ago 7m 16.9M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (4m) 49s ago 14m 471M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (3m) 36s ago 12m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (61s) 49s ago 14m 31.3M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (40s) 36s ago 12m 26.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (3m) 49s ago 13m 16.6M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (2m) 36s ago 12m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (11m) 49s ago 11m 206M 1217M 18.2.1 5be31c24972a 476e64dcb219 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (10m) 49s ago 10m 201M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (10m) 49s ago 10m 163M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (9m) 36s ago 9m 236M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (9m) 36s ago 9m 222M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (9m) 36s ago 9m 202M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:53:45.464 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (2m) 49s ago 12m 67.4M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:53:45.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40e406e7c0 msgr2=0x7f40e4070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40e406e7c0 0x7f40e4070c80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f40f4006010 tx=0x7f40f401a040 comp rx=0 tx=0).stop 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8102720 msgr2=0x7f40f819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8102720 0x7f40f819cbd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40e800c8f0 tx=0x7f40e800cc00 comp rx=0 tx=0).stop 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 shutdown_connections 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40e406e7c0 0x7f40e4070c80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40f8101e00 0x7f40f819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 --2- 172.21.15.42:0/650654860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40f8102720 0x7f40f819cbd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 >> 172.21.15.42:0/650654860 conn(0x7f40f80fb650 msgr2=0x7f40f8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 shutdown_connections 2024-09-16T23:53:45.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.468+0000 7f40ffc63700 1 -- 172.21.15.42:0/650654860 wait complete. 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 -- 172.21.15.42:0/1845483339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec105bd0 msgr2=0x7fd3ec105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/1845483339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec105bd0 0x7fd3ec105fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009910 tx=0x7fd3dc009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 -- 172.21.15.42:0/1845483339 shutdown_connections 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/1845483339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ec0ffc20 0x7fd3ec100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/1845483339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec105bd0 0x7fd3ec105fb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 -- 172.21.15.42:0/1845483339 >> 172.21.15.42:0/1845483339 conn(0x7fd3ec075670 msgr2=0x7fd3ec075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 -- 172.21.15.42:0/1845483339 shutdown_connections 2024-09-16T23:53:45.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.575+0000 7fd3f0db5700 1 -- 172.21.15.42:0/1845483339 wait complete. 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3f0db5700 1 Processor -- start 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3f0db5700 1 -- start start 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3f0db5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec0ffc20 0x7fd3ec19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3f0db5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ec105bd0 0x7fd3ec19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3f0db5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec19d1f0 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3f0db5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec1966a0 con 0x7fd3ec105bd0 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3ea59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec0ffc20 0x7fd3ec19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3ea59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec0ffc20 0x7fd3ec19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37714/0 (socket says 172.21.15.42:37714) 2024-09-16T23:53:45.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3ea59c700 1 -- 172.21.15.42:0/938512195 learned_addr learned my addr 172.21.15.42:0/938512195 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.576+0000 7fd3e1d9b700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ec105bd0 0x7fd3ec19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3ea59c700 1 -- 172.21.15.42:0/938512195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ec105bd0 msgr2=0x7fd3ec19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3ea59c700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ec105bd0 0x7fd3ec19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3ea59c700 1 -- 172.21.15.42:0/938512195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0095f0 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3e1d9b700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ec105bd0 0x7fd3ec19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3ea59c700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec0ffc20 0x7fd3ec19c620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006010 tx=0x7fd3dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3e3fff700 1 -- 172.21.15.42:0/938512195 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3dc01d070 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3e3fff700 1 -- 172.21.15.42:0/938512195 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3dc00f460 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec196920 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3e3fff700 1 -- 172.21.15.42:0/938512195 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3dc017630 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.577+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec196e10 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.578+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec04f2e0 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.579+0000 7fd3e3fff700 1 -- 172.21.15.42:0/938512195 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fd3dc00f5d0 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.579+0000 7fd3e3fff700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd3d806e7c0 0x7fd3d8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.579+0000 7fd3e3fff700 1 -- 172.21.15.42:0/938512195 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd3dc08fcd0 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.579+0000 7fd3e1d9b700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd3d806e7c0 0x7fd3d8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.580+0000 7fd3e1d9b700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd3d806e7c0 0x7fd3d8070c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005950 tx=0x7fd3d40058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:45.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.583+0000 7fd3e3fff700 1 -- 172.21.15.42:0/938512195 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd3dc05c830 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:45 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:45.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:45 smithi042 ceph-mon[111056]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:45.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:45 smithi042 ceph-mon[111056]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:45.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:45 smithi042 ceph-mon[111056]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:45.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.884+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd3ec197030 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.885+0000 7fd3e3fff700 1 -- 172.21.15.42:0/938512195 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+694 (secure 0 0 0) 0x7fd3dc05c3c0 con 0x7fd3ec0ffc20 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:53:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:53:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:53:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-09-16T23:53:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-16T23:53:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:53:45.884 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd3d806e7c0 msgr2=0x7fd3d8070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd3d806e7c0 0x7fd3d8070c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4005950 tx=0x7fd3d40058e0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec0ffc20 msgr2=0x7fd3ec19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec0ffc20 0x7fd3ec19c620 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc006010 tx=0x7fd3dc00be20 comp rx=0 tx=0).stop 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 shutdown_connections 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd3d806e7c0 0x7fd3d8070c80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3ec0ffc20 0x7fd3ec19c620 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 --2- 172.21.15.42:0/938512195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3ec105bd0 0x7fd3ec19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 >> 172.21.15.42:0/938512195 conn(0x7fd3ec075670 msgr2=0x7fd3ec068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 shutdown_connections 2024-09-16T23:53:45.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.887+0000 7fd3f0db5700 1 -- 172.21.15.42:0/938512195 wait complete. 2024-09-16T23:53:45.979 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:45 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:45.979 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:45 smithi043 ceph-mon[81510]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:45.979 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:45 smithi043 ceph-mon[81510]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:45.979 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:45 smithi043 ceph-mon[81510]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:45.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.995+0000 7fbeef5d0700 1 -- 172.21.15.42:0/551084709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8101850 msgr2=0x7fbee81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.995+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/551084709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8101850 0x7fbee81058a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009910 tx=0x7fbed8009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:45.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 -- 172.21.15.42:0/551084709 shutdown_connections 2024-09-16T23:53:45.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/551084709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8101850 0x7fbee81058a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/551084709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8100ea0 0x7fbee8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 -- 172.21.15.42:0/551084709 >> 172.21.15.42:0/551084709 conn(0x7fbee80fc750 msgr2=0x7fbee80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:45.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 -- 172.21.15.42:0/551084709 shutdown_connections 2024-09-16T23:53:45.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 -- 172.21.15.42:0/551084709 wait complete. 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 Processor -- start 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.996+0000 7fbeef5d0700 1 -- start start 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeef5d0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8100ea0 0x7fbee819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeef5d0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8101850 0x7fbee819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeef5d0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee819d2d0 con 0x7fbee8100ea0 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeef5d0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8196780 con 0x7fbee8101850 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeed36c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8100ea0 0x7fbee819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeed36c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8100ea0 0x7fbee819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37738/0 (socket says 172.21.15.42:37738) 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeed36c700 1 -- 172.21.15.42:0/692938258 learned_addr learned my addr 172.21.15.42:0/692938258 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeecb6b700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8101850 0x7fbee819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeed36c700 1 -- 172.21.15.42:0/692938258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8101850 msgr2=0x7fbee819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeed36c700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8101850 0x7fbee819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeed36c700 1 -- 172.21.15.42:0/692938258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed80095f0 con 0x7fbee8100ea0 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeecb6b700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8101850 0x7fbee819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbeed36c700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8100ea0 0x7fbee819c700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbee400d8d0 tx=0x7fbee400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.997+0000 7fbede7fc700 1 -- 172.21.15.42:0/692938258 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee4009940 con 0x7fbee8100ea0 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.998+0000 7fbede7fc700 1 -- 172.21.15.42:0/692938258 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbee4010460 con 0x7fbee8100ea0 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.998+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee8196a60 con 0x7fbee8100ea0 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.998+0000 7fbede7fc700 1 -- 172.21.15.42:0/692938258 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee400f5f0 con 0x7fbee8100ea0 2024-09-16T23:53:45.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.998+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee8196fb0 con 0x7fbee8100ea0 2024-09-16T23:53:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.999+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee81092a0 con 0x7fbee8100ea0 2024-09-16T23:53:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:45.999+0000 7fbede7fc700 1 -- 172.21.15.42:0/692938258 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fbee40105d0 con 0x7fbee8100ea0 2024-09-16T23:53:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.000+0000 7fbede7fc700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbed406e7c0 0x7fbed4070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.000+0000 7fbede7fc700 1 -- 172.21.15.42:0/692938258 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbee408e740 con 0x7fbee8100ea0 2024-09-16T23:53:45.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.000+0000 7fbeecb6b700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbed406e7c0 0x7fbed4070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:45.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.000+0000 7fbeecb6b700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbed406e7c0 0x7fbed4070c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbee8197bf0 tx=0x7fbed801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:46.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.004+0000 7fbede7fc700 1 -- 172.21.15.42:0/692938258 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbee405b320 con 0x7fbee8100ea0 2024-09-16T23:53:46.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.270+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbee804f2e0 con 0x7fbee8100ea0 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.271+0000 7fbede7fc700 1 -- 172.21.15.42:0/692938258 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1938 (secure 0 0 0) 0x7fbee40167b0 con 0x7fbee8100ea0 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:53:20:175453+0000 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:53:46.269 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:53:19.510146+0000 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:53:46.270 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 24269,14478 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:46.271 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:46.272 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:53:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:53:46.272 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:46.272 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.273+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbed406e7c0 msgr2=0x7fbed4070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.273+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbed406e7c0 0x7fbed4070c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbee8197bf0 tx=0x7fbed801a040 comp rx=0 tx=0).stop 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.273+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8100ea0 msgr2=0x7fbee819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.273+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8100ea0 0x7fbee819c700 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbee400d8d0 tx=0x7fbee400dc90 comp rx=0 tx=0).stop 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.273+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 shutdown_connections 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.273+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbed406e7c0 0x7fbed4070c80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.274+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbee8100ea0 0x7fbee819c700 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.274+0000 7fbeef5d0700 1 --2- 172.21.15.42:0/692938258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee8101850 0x7fbee819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.274+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 >> 172.21.15.42:0/692938258 conn(0x7fbee80fc750 msgr2=0x7fbee80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.274+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 shutdown_connections 2024-09-16T23:53:46.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.274+0000 7fbeef5d0700 1 -- 172.21.15.42:0/692938258 wait complete. 2024-09-16T23:53:46.274 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.382+0000 7fc3b759e700 1 -- 172.21.15.42:0/1647145988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 msgr2=0x7fc3b8106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.382+0000 7fc3b759e700 1 --2- 172.21.15.42:0/1647145988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8106e30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009910 tx=0x7fc3a8009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 -- 172.21.15.42:0/1647145988 shutdown_connections 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 --2- 172.21.15.42:0/1647145988 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8068580 0x7fc3b8068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 --2- 172.21.15.42:0/1647145988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8106e30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 -- 172.21.15.42:0/1647145988 >> 172.21.15.42:0/1647145988 conn(0x7fc3b8075460 msgr2=0x7fc3b8075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 -- 172.21.15.42:0/1647145988 shutdown_connections 2024-09-16T23:53:46.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 -- 172.21.15.42:0/1647145988 wait complete. 2024-09-16T23:53:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 Processor -- start 2024-09-16T23:53:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.383+0000 7fc3b759e700 1 -- start start 2024-09-16T23:53:46.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b759e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8068580 0x7fc3b80ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b759e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8100030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b759e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8100570 con 0x7fc3b8106a50 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b759e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b81006b0 con 0x7fc3b8068580 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8100030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8100030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37756/0 (socket says 172.21.15.42:37756) 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b5d9b700 1 -- 172.21.15.42:0/3550515872 learned_addr learned my addr 172.21.15.42:0/3550515872 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b659c700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8068580 0x7fc3b80ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b5d9b700 1 -- 172.21.15.42:0/3550515872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8068580 msgr2=0x7fc3b80ffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b5d9b700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8068580 0x7fc3b80ffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b5d9b700 1 -- 172.21.15.42:0/3550515872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a80095f0 con 0x7fc3b8106a50 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3b5d9b700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8100030 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00d8d0 tx=0x7fc3ac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3a77fe700 1 -- 172.21.15.42:0/3550515872 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ac0098a0 con 0x7fc3b8106a50 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3a77fe700 1 -- 172.21.15.42:0/3550515872 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3ac010460 con 0x7fc3b8106a50 2024-09-16T23:53:46.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.384+0000 7fc3a77fe700 1 -- 172.21.15.42:0/3550515872 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3ac00f5f0 con 0x7fc3b8106a50 2024-09-16T23:53:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.385+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b81a27f0 con 0x7fc3b8106a50 2024-09-16T23:53:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.385+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b81a2c00 con 0x7fc3b8106a50 2024-09-16T23:53:46.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.385+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b8108f70 con 0x7fc3b8106a50 2024-09-16T23:53:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.386+0000 7fc3a77fe700 1 -- 172.21.15.42:0/3550515872 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fc3ac009a00 con 0x7fc3b8106a50 2024-09-16T23:53:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.387+0000 7fc3a77fe700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc3a006e7c0 0x7fc3a0070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.387+0000 7fc3a77fe700 1 -- 172.21.15.42:0/3550515872 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc3ac08e720 con 0x7fc3b8106a50 2024-09-16T23:53:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.387+0000 7fc3b659c700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc3a006e7c0 0x7fc3a0070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:46.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.387+0000 7fc3b659c700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc3a006e7c0 0x7fc3a0070c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006010 tx=0x7fc3a801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:46.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.391+0000 7fc3a77fe700 1 -- 172.21.15.42:0/3550515872 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc3ac057750 con 0x7fc3b8106a50 2024-09-16T23:53:46.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.625+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3b802cce0 con 0x7fc3a006e7c0 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.626+0000 7fc3a77fe700 1 -- 172.21.15.42:0/3550515872 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fc3b802cce0 con 0x7fc3a006e7c0 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:53:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:53:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:53:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "4/23 daemons upgraded", 2024-09-16T23:53:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading crash daemons", 2024-09-16T23:53:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:53:46.625 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:53:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc3a006e7c0 msgr2=0x7fc3a0070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc3a006e7c0 0x7fc3a0070c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8006010 tx=0x7fc3a801a040 comp rx=0 tx=0).stop 2024-09-16T23:53:46.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 msgr2=0x7fc3b8100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8100030 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac00d8d0 tx=0x7fc3ac00dbe0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 shutdown_connections 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc3a006e7c0 0x7fc3a0070c80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3b8068580 0x7fc3b80ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 --2- 172.21.15.42:0/3550515872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3b8106a50 0x7fc3b8100030 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.628+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 >> 172.21.15.42:0/3550515872 conn(0x7fc3b8075460 msgr2=0x7fc3b80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.629+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 shutdown_connections 2024-09-16T23:53:46.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.629+0000 7fc3b759e700 1 -- 172.21.15.42:0/3550515872 wait complete. 2024-09-16T23:53:46.645 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/938512195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:46.645 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: Upgrade: Updating crash.smithi043 (2/2) 2024-09-16T23:53:46.645 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:46.645 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:46.645 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:46.645 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: Deploying daemon crash.smithi043 on smithi043 2024-09-16T23:53:46.646 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/692938258' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:53:46.646 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:46 smithi043 ceph-mon[81510]: pgmap v31: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:46.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 -- 172.21.15.42:0/3908557491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 msgr2=0x7f804010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 --2- 172.21.15.42:0/3908557491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804010ac90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8034009910 tx=0x7f8034009c20 comp rx=0 tx=0).stop 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 -- 172.21.15.42:0/3908557491 shutdown_connections 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 --2- 172.21.15.42:0/3908557491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804010ac90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 --2- 172.21.15.42:0/3908557491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8040101e80 0x7f8040102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 -- 172.21.15.42:0/3908557491 >> 172.21.15.42:0/3908557491 conn(0x7f80400fb610 msgr2=0x7f80400fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 -- 172.21.15.42:0/3908557491 shutdown_connections 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.740+0000 7f8046439700 1 -- 172.21.15.42:0/3908557491 wait complete. 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f8046439700 1 Processor -- start 2024-09-16T23:53:46.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f8046439700 1 -- start start 2024-09-16T23:53:46.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f8046439700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8040101e80 0x7f804019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f8046439700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f8046439700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804019d240 con 0x7f80401027a0 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f8046439700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80401966f0 con 0x7f8040101e80 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f803f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37766/0 (socket says 172.21.15.42:37766) 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.741+0000 7f803f7fe700 1 -- 172.21.15.42:0/4232230401 learned_addr learned my addr 172.21.15.42:0/4232230401 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803ffff700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8040101e80 0x7f804019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803f7fe700 1 -- 172.21.15.42:0/4232230401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8040101e80 msgr2=0x7f804019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803f7fe700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8040101e80 0x7f804019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803f7fe700 1 -- 172.21.15.42:0/4232230401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340095f0 con 0x7f80401027a0 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803f7fe700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804019cbb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8034005f50 tx=0x7f803400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:46.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803d7fa700 1 -- 172.21.15.42:0/4232230401 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f803401d070 con 0x7f80401027a0 2024-09-16T23:53:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803d7fa700 1 -- 172.21.15.42:0/4232230401 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80340052a0 con 0x7f80401027a0 2024-09-16T23:53:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8040196970 con 0x7f80401027a0 2024-09-16T23:53:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8040196e60 con 0x7f80401027a0 2024-09-16T23:53:46.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.742+0000 7f803d7fa700 1 -- 172.21.15.42:0/4232230401 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8034022470 con 0x7f80401027a0 2024-09-16T23:53:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.743+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8040108390 con 0x7f80401027a0 2024-09-16T23:53:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.744+0000 7f803d7fa700 1 -- 172.21.15.42:0/4232230401 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f8034017400 con 0x7f80401027a0 2024-09-16T23:53:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.744+0000 7f803d7fa700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f802806e7c0 0x7f8028070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:53:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.744+0000 7f803d7fa700 1 -- 172.21.15.42:0/4232230401 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f803408fc90 con 0x7f80401027a0 2024-09-16T23:53:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.744+0000 7f803ffff700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f802806e7c0 0x7f8028070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:53:46.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.745+0000 7f803ffff700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f802806e7c0 0x7f8028070c80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8030009c60 tx=0x7f8030009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:53:46.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:46.749+0000 7f803d7fa700 1 -- 172.21.15.42:0/4232230401 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f803405c9c0 con 0x7f80401027a0 2024-09-16T23:53:46.904 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/938512195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:46.905 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: Upgrade: Updating crash.smithi043 (2/2) 2024-09-16T23:53:46.905 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:46.905 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi043", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T23:53:46.905 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:46.905 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: Deploying daemon crash.smithi043 on smithi043 2024-09-16T23:53:46.905 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/692938258' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:53:46.905 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:46 smithi042 ceph-mon[111056]: pgmap v31: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:47.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.046+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f80400679c0 con 0x7f80401027a0 2024-09-16T23:53:47.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.047+0000 7f803d7fa700 1 -- 172.21.15.42:0/4232230401 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f803405c550 con 0x7f80401027a0 2024-09-16T23:53:47.045 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:47.045 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:53:47.045 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:53:47.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f802806e7c0 msgr2=0x7f8028070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f802806e7c0 0x7f8028070c80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8030009c60 tx=0x7f8030009260 comp rx=0 tx=0).stop 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 msgr2=0x7f804019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804019cbb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8034005f50 tx=0x7f803400b7d0 comp rx=0 tx=0).stop 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 shutdown_connections 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f802806e7c0 0x7f8028070c80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8040101e80 0x7f804019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 --2- 172.21.15.42:0/4232230401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80401027a0 0x7f804019cbb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 >> 172.21.15.42:0/4232230401 conn(0x7f80400fb610 msgr2=0x7f80401054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 shutdown_connections 2024-09-16T23:53:47.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:53:47.049+0000 7f8046439700 1 -- 172.21.15.42:0/4232230401 wait complete. 2024-09-16T23:53:48.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:48 smithi042 ceph-mon[111056]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:48 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4232230401' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:53:48.300 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:48 smithi043 ceph-mon[81510]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:53:48.300 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:48 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4232230401' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:53:49.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:49 smithi042 ceph-mon[111056]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:49.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:49 smithi043 ceph-mon[81510]: pgmap v32: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:51.021 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:51.022 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:51.022 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:51.022 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:50 smithi043 ceph-mon[81510]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:51.022 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:53:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:50 smithi042 ceph-mon[111056]: pgmap v33: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:53:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:53 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:53 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:53 smithi042 ceph-mon[111056]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:53 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:53.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:53 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:53.830 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:53 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:53.830 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:53 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:53.830 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:53 smithi043 ceph-mon[81510]: pgmap v34: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:53.830 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:53 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:53.830 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:53 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:54 smithi043 ceph-mon[81510]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:54 smithi042 ceph-mon[111056]: pgmap v35: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:56.008 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:56.009 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:56.009 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:56.009 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:56.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:56.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:56.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:56.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:57.406 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:57 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:57.406 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:57 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:57.406 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:57 smithi042 ceph-mon[111056]: pgmap v36: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:57.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:57 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:57.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:57 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:57.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:57 smithi043 ceph-mon[81510]: pgmap v36: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:58 smithi042 ceph-mon[111056]: pgmap v37: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:58 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:58 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:58 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:59.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:58 smithi043 ceph-mon[81510]: pgmap v37: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:53:59.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:58 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:59.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:58 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:59.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:58 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:53:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:53:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:53:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all crash 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]: dispatch 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]': finished 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi043"}]: dispatch 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi043"}]': finished 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T23:53:59.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:53:59 smithi042 ceph-mon[111056]: Upgrade: osd.0 is safe to restart 2024-09-16T23:54:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:54:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all crash 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]: dispatch 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi042"}]': finished 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi043"}]: dispatch 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi043"}]': finished 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T23:54:00.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:53:59 smithi043 ceph-mon[81510]: Upgrade: osd.0 is safe to restart 2024-09-16T23:54:01.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:01 smithi043 ceph-mon[81510]: Upgrade: Updating osd.0 2024-09-16T23:54:01.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:01 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:01.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:01 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T23:54:01.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:01 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:01.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:01 smithi043 ceph-mon[81510]: Deploying daemon osd.0 on smithi042 2024-09-16T23:54:01.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:01 smithi043 ceph-mon[81510]: pgmap v38: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:01.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:01 smithi042 ceph-mon[111056]: Upgrade: Updating osd.0 2024-09-16T23:54:01.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:01.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T23:54:01.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:01.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:01 smithi042 ceph-mon[111056]: Deploying daemon osd.0 on smithi042 2024-09-16T23:54:01.444 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:01 smithi042 ceph-mon[111056]: pgmap v38: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:02.951 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:02 smithi042 ceph-mon[111056]: pgmap v39: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:03.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:02 smithi043 ceph-mon[81510]: pgmap v39: 65 pgs: 65 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:03.273 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:03 smithi042 systemd[1]: Stopping Ceph osd.0 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:54:03.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:03 smithi042 ceph-mon[111056]: osd.0 marked itself down and dead 2024-09-16T23:54:03.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:03 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[46711]: 2024-09-16T23:54:03.400+0000 7f220827c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:54:03.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:03 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[46711]: 2024-09-16T23:54:03.400+0000 7f220827c700 -1 osd.0 49 *** Got signal Terminated *** 2024-09-16T23:54:03.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:03 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[46711]: 2024-09-16T23:54:03.400+0000 7f220827c700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T23:54:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:03 smithi043 ceph-mon[81510]: osd.0 marked itself down and dead 2024-09-16T23:54:04.467 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:04 smithi042 podman[122125]: 2024-09-16 23:54:04.189584075 +0000 UTC m=+0.881151679 container died 476e64dcb219886348b965b1239a123a936f3af071e3bce30620a73065f929de (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, ceph=True, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T23:54:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:04 smithi042 ceph-mon[111056]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:54:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:04 smithi042 ceph-mon[111056]: osdmap e50: 6 total, 5 up, 6 in 2024-09-16T23:54:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:04 smithi042 ceph-mon[111056]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:04.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:04 smithi042 podman[122125]: 2024-09-16 23:54:04.568083909 +0000 UTC m=+1.259651502 container cleanup 476e64dcb219886348b965b1239a123a936f3af071e3bce30620a73065f929de (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, GIT_BRANCH=HEAD) 2024-09-16T23:54:04.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:04 smithi042 bash[122125]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0 2024-09-16T23:54:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:04 smithi043 ceph-mon[81510]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:54:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:04 smithi043 ceph-mon[81510]: osdmap e50: 6 total, 5 up, 6 in 2024-09-16T23:54:05.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:04 smithi043 ceph-mon[81510]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:05.250 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:04 smithi042 podman[122140]: 2024-09-16 23:54:04.900349622 +0000 UTC m=+0.703490560 container remove 476e64dcb219886348b965b1239a123a936f3af071e3bce30620a73065f929de (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222) 2024-09-16T23:54:05.521 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:05 smithi042 podman[122229]: 2024-09-16 23:54:05.156731716 +0000 UTC m=+0.016514125 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:54:05.521 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:05 smithi042 podman[122229]: 2024-09-16 23:54:05.254443903 +0000 UTC m=+0.114226294 container create ac4959c88ed09d83fe1124983fefa1825fe26a72533315cc81ac29847b235d09 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:54:05.521 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:05 smithi042 podman[122229]: 2024-09-16 23:54:05.52398173 +0000 UTC m=+0.383764129 container init ac4959c88ed09d83fe1124983fefa1825fe26a72533315cc81ac29847b235d09 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T23:54:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:05 smithi042 ceph-mon[111056]: osdmap e51: 6 total, 5 up, 6 in 2024-09-16T23:54:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:05.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:05 smithi042 podman[122229]: 2024-09-16 23:54:05.527441183 +0000 UTC m=+0.387223578 container start ac4959c88ed09d83fe1124983fefa1825fe26a72533315cc81ac29847b235d09 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T23:54:05.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:05 smithi042 podman[122229]: 2024-09-16 23:54:05.615657276 +0000 UTC m=+0.475439679 container attach ac4959c88ed09d83fe1124983fefa1825fe26a72533315cc81ac29847b235d09 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T23:54:05.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:05 smithi042 podman[122229]: 2024-09-16 23:54:05.722750253 +0000 UTC m=+0.582532651 container died ac4959c88ed09d83fe1124983fefa1825fe26a72533315cc81ac29847b235d09 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T23:54:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:05 smithi043 ceph-mon[81510]: osdmap e51: 6 total, 5 up, 6 in 2024-09-16T23:54:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:06.273 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:06 smithi042 podman[122253]: 2024-09-16 23:54:06.261275434 +0000 UTC m=+0.532471180 container remove ac4959c88ed09d83fe1124983fefa1825fe26a72533315cc81ac29847b235d09 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T23:54:06.657 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:06 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-16T23:54:06.657 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:06 smithi042 systemd[1]: Stopped Ceph osd.0 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:54:06.657 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:06 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.0.service: Consumed 55.018s CPU time. 2024-09-16T23:54:06.657 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:06 smithi042 systemd[1]: Starting Ceph osd.0 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:54:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:06 smithi042 ceph-mon[111056]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:07.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:06 smithi042 podman[122350]: 2024-09-16 23:54:06.625631617 +0000 UTC m=+0.016732328 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:54:07.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:06 smithi042 podman[122350]: 2024-09-16 23:54:06.746308558 +0000 UTC m=+0.137409251 container create fd11d962740cb90cd3f625dc06b332d7683430e78020e9b1f1eccf34453a81cb (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T23:54:07.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:06 smithi043 ceph-mon[81510]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:07.418 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 podman[122350]: 2024-09-16 23:54:07.057304061 +0000 UTC m=+0.448404770 container init fd11d962740cb90cd3f625dc06b332d7683430e78020e9b1f1eccf34453a81cb (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0) 2024-09-16T23:54:07.418 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 podman[122350]: 2024-09-16 23:54:07.060910638 +0000 UTC m=+0.452011337 container start fd11d962740cb90cd3f625dc06b332d7683430e78020e9b1f1eccf34453a81cb (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:54:07.418 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 podman[122350]: 2024-09-16 23:54:07.149041161 +0000 UTC m=+0.540141866 container attach fd11d962740cb90cd3f625dc06b332d7683430e78020e9b1f1eccf34453a81cb (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T23:54:07.418 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:07.419 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 bash[122350]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:07.419 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:07.419 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 bash[122350]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:07 smithi042 ceph-mon[111056]: Health check failed: Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T23:54:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:07 smithi043 ceph-mon[81510]: Health check failed: Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T23:54:08.230 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T23:54:08.230 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 bash[122350]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T23:54:08.230 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T23:54:08.230 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:07 smithi042 bash[122350]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T23:54:08.523 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T23:54:08.523 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 bash[122350]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 bash[122350]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 bash[122350]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 bash[122350]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate[122365]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 bash[122350]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T23:54:08.524 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 podman[122350]: 2024-09-16 23:54:08.265013015 +0000 UTC m=+1.656113719 container died fd11d962740cb90cd3f625dc06b332d7683430e78020e9b1f1eccf34453a81cb (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:54:08.892 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:08 smithi042 ceph-mon[111056]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:09.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:08 smithi043 ceph-mon[81510]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:09.225 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:08 smithi042 podman[122504]: 2024-09-16 23:54:08.895151932 +0000 UTC m=+0.619002064 container remove fd11d962740cb90cd3f625dc06b332d7683430e78020e9b1f1eccf34453a81cb (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-activate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T23:54:09.225 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:09 smithi042 podman[122563]: 2024-09-16 23:54:09.104891024 +0000 UTC m=+0.107353363 container create e3e56c422ec57412a91435be900a0202932c04ea9a9351672ca10b276ea3591c (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T23:54:09.225 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:09 smithi042 podman[122563]: 2024-09-16 23:54:09.014288728 +0000 UTC m=+0.016751075 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:54:09.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:09 smithi042 podman[122563]: 2024-09-16 23:54:09.357460858 +0000 UTC m=+0.359923212 container init e3e56c422ec57412a91435be900a0202932c04ea9a9351672ca10b276ea3591c (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T23:54:09.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:09 smithi042 podman[122563]: 2024-09-16 23:54:09.361017487 +0000 UTC m=+0.363479825 container start e3e56c422ec57412a91435be900a0202932c04ea9a9351672ca10b276ea3591c (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T23:54:09.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:09 smithi042 ceph-osd[122583]: -- 172.21.15.42:0/273202873 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642f546a1e0 con 0x5642f5400000 2024-09-16T23:54:09.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:09 smithi042 bash[122563]: e3e56c422ec57412a91435be900a0202932c04ea9a9351672ca10b276ea3591c 2024-09-16T23:54:09.773 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:09 smithi042 systemd[1]: Started Ceph osd.0 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:54:10.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:10 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:10.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:10 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:10.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:10 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:10.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:10 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:10.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:10 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:10.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:10 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:11.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:10 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[122579]: 2024-09-16T23:54:10.758+0000 7fdfadcaf640 -1 Falling back to public interface 2024-09-16T23:54:11.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:11 smithi043 ceph-mon[81510]: pgmap v45: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:11 smithi042 ceph-mon[111056]: pgmap v45: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:12.720 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:12 smithi042 ceph-mon[111056]: pgmap v46: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:12 smithi043 ceph-mon[81510]: pgmap v46: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:15.908 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:15 smithi042 ceph-mon[111056]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:15.908 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:15.908 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:15.908 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:15.908 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:15 smithi043 ceph-mon[81510]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:16.173 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:15 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[122579]: 2024-09-16T23:54:15.982+0000 7fdfadcaf640 -1 osd.0 0 read_superblock omap replica is missing. 2024-09-16T23:54:16.522 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:16 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[122579]: 2024-09-16T23:54:16.261+0000 7fdfadcaf640 -1 osd.0 49 log_to_monitors true 2024-09-16T23:54:17.019 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:16 smithi042 ceph-mon[111056]: from='osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T23:54:17.019 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 16 23:54:16 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[122579]: 2024-09-16T23:54:16.679+0000 7fdfa5a49640 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:54:17.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:16 smithi043 ceph-mon[81510]: from='osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 -- 172.21.15.42:0/53429037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 msgr2=0x7f680010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 --2- 172.21.15.42:0/53429037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680010cfb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009910 tx=0x7f67e8009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 -- 172.21.15.42:0/53429037 shutdown_connections 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 --2- 172.21.15.42:0/53429037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680010cfb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 --2- 172.21.15.42:0/53429037 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000730b0 0x7f6800073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 -- 172.21.15.42:0/53429037 >> 172.21.15.42:0/53429037 conn(0x7f68000fc740 msgr2=0x7f68000feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 -- 172.21.15.42:0/53429037 shutdown_connections 2024-09-16T23:54:17.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.166+0000 7f680493e700 1 -- 172.21.15.42:0/53429037 wait complete. 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f680493e700 1 Processor -- start 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f680493e700 1 -- start start 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f680493e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000730b0 0x7f680019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f680493e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f680493e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680019f400 con 0x7f68000739d0 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f680493e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001988b0 con 0x7f68000730b0 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f67f5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f67f5d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40800/0 (socket says 172.21.15.42:40800) 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f67f5d9b700 1 -- 172.21.15.42:0/345534706 learned_addr learned my addr 172.21.15.42:0/345534706 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.167+0000 7f67fe59c700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000730b0 0x7f680019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67f5d9b700 1 -- 172.21.15.42:0/345534706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000730b0 msgr2=0x7f680019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67f5d9b700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000730b0 0x7f680019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67f5d9b700 1 -- 172.21.15.42:0/345534706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e80095f0 con 0x7f68000739d0 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67fe59c700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000730b0 0x7f680019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67f5d9b700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680019ed70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67e8006010 tx=0x7f67e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67f7fff700 1 -- 172.21.15.42:0/345534706 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e801d070 con 0x7f68000739d0 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67f7fff700 1 -- 172.21.15.42:0/345534706 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67e8004fb0 con 0x7f68000739d0 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f67f7fff700 1 -- 172.21.15.42:0/345534706 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e800f690 con 0x7f68000739d0 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6800198b30 con 0x7f68000739d0 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.168+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6800199020 con 0x7f68000739d0 2024-09-16T23:54:17.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.169+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680010a6b0 con 0x7f68000739d0 2024-09-16T23:54:17.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.170+0000 7f67f7fff700 1 -- 172.21.15.42:0/345534706 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f67e80228a0 con 0x7f68000739d0 2024-09-16T23:54:17.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.170+0000 7f67f7fff700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f67ec06e7b0 0x7f67ec070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.170+0000 7f67f7fff700 1 -- 172.21.15.42:0/345534706 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f67e80900c0 con 0x7f68000739d0 2024-09-16T23:54:17.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.170+0000 7f67fe59c700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f67ec06e7b0 0x7f67ec070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.171+0000 7f67fe59c700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f67ec06e7b0 0x7f67ec070c70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005fd0 tx=0x7f67f0005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:17.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.175+0000 7f67f7fff700 1 -- 172.21.15.42:0/345534706 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f67e805cc00 con 0x7f68000739d0 2024-09-16T23:54:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.414+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68001992f0 con 0x7f67ec06e7b0 2024-09-16T23:54:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.416+0000 7f67f7fff700 1 -- 172.21.15.42:0/345534706 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f68001992f0 con 0x7f67ec06e7b0 2024-09-16T23:54:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f67ec06e7b0 msgr2=0x7f67ec070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f67ec06e7b0 0x7f67ec070c70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f67f0005fd0 tx=0x7f67f0005dc0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 msgr2=0x7f680019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680019ed70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f67e8006010 tx=0x7f67e800be20 comp rx=0 tx=0).stop 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 shutdown_connections 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f67ec06e7b0 0x7f67ec070c70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68000730b0 0x7f680019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 --2- 172.21.15.42:0/345534706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f68000739d0 0x7f680019ed70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 >> 172.21.15.42:0/345534706 conn(0x7f68000fc740 msgr2=0x7f68001077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 shutdown_connections 2024-09-16T23:54:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.418+0000 7f680493e700 1 -- 172.21.15.42:0/345534706 wait complete. 2024-09-16T23:54:17.427 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:54:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 -- 172.21.15.42:0/4198877422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00fff50 msgr2=0x7f78c0105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 --2- 172.21.15.42:0/4198877422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00fff50 0x7f78c0105840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009910 tx=0x7f78b4009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 -- 172.21.15.42:0/4198877422 shutdown_connections 2024-09-16T23:54:17.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 --2- 172.21.15.42:0/4198877422 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00fff50 0x7f78c0105840 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 --2- 172.21.15.42:0/4198877422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00ff5a0 0x7f78c00ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 -- 172.21.15.42:0/4198877422 >> 172.21.15.42:0/4198877422 conn(0x7f78c0074e30 msgr2=0x7f78c0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 -- 172.21.15.42:0/4198877422 shutdown_connections 2024-09-16T23:54:17.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.534+0000 7f78c5a07700 1 -- 172.21.15.42:0/4198877422 wait complete. 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.535+0000 7f78c5a07700 1 Processor -- start 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.535+0000 7f78c5a07700 1 -- start start 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.535+0000 7f78c5a07700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00ff5a0 0x7f78c0199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.535+0000 7f78c5a07700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00fff50 0x7f78c0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.535+0000 7f78c5a07700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c019f190 con 0x7f78c00ff5a0 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.535+0000 7f78c5a07700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c019f300 con 0x7f78c00fff50 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.535+0000 7f78beffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00ff5a0 0x7f78c0199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78beffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00ff5a0 0x7f78c0199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40810/0 (socket says 172.21.15.42:40810) 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78beffd700 1 -- 172.21.15.42:0/2187262022 learned_addr learned my addr 172.21.15.42:0/2187262022 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:17.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78be7fc700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00fff50 0x7f78c0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78beffd700 1 -- 172.21.15.42:0/2187262022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00fff50 msgr2=0x7f78c0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78beffd700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00fff50 0x7f78c0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78beffd700 1 -- 172.21.15.42:0/2187262022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40095f0 con 0x7f78c00ff5a0 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78beffd700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00ff5a0 0x7f78c0199400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78b000d8d0 tx=0x7f78b000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78c4a05700 1 -- 172.21.15.42:0/2187262022 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78b00098a0 con 0x7f78c00ff5a0 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c019f5e0 con 0x7f78c00ff5a0 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78c4a05700 1 -- 172.21.15.42:0/2187262022 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78b0010460 con 0x7f78c00ff5a0 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78c4a05700 1 -- 172.21.15.42:0/2187262022 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78b000f5f0 con 0x7f78c00ff5a0 2024-09-16T23:54:17.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.536+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c019fb00 con 0x7f78c00ff5a0 2024-09-16T23:54:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.538+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c004f2e0 con 0x7f78c00ff5a0 2024-09-16T23:54:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.538+0000 7f78c4a05700 1 -- 172.21.15.42:0/2187262022 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f78b000f820 con 0x7f78c00ff5a0 2024-09-16T23:54:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.539+0000 7f78c4a05700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f78a806e7b0 0x7f78a8070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.539+0000 7f78c4a05700 1 -- 172.21.15.42:0/2187262022 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f78b008e780 con 0x7f78c00ff5a0 2024-09-16T23:54:17.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.539+0000 7f78be7fc700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f78a806e7b0 0x7f78a8070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.539+0000 7f78be7fc700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f78a806e7b0 0x7f78a8070c70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:17.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.543+0000 7f78c4a05700 1 -- 172.21.15.42:0/2187262022 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f78b00577b0 con 0x7f78c00ff5a0 2024-09-16T23:54:17.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.793+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78c019fdb0 con 0x7f78a806e7b0 2024-09-16T23:54:17.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.794+0000 7f78c4a05700 1 -- 172.21.15.42:0/2187262022 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f78c019fdb0 con 0x7f78a806e7b0 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f78a806e7b0 msgr2=0x7f78a8070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f78a806e7b0 0x7f78a8070c70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78b4006010 tx=0x7f78b401a040 comp rx=0 tx=0).stop 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00ff5a0 msgr2=0x7f78c0199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00ff5a0 0x7f78c0199400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78b000d8d0 tx=0x7f78b000dbe0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 shutdown_connections 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f78a806e7b0 0x7f78a8070c70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78c00ff5a0 0x7f78c0199400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 --2- 172.21.15.42:0/2187262022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78c00fff50 0x7f78c0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 >> 172.21.15.42:0/2187262022 conn(0x7f78c0074e30 msgr2=0x7f78c00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 shutdown_connections 2024-09-16T23:54:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.797+0000 7f78c5a07700 1 -- 172.21.15.42:0/2187262022 wait complete. 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 -- 172.21.15.42:0/1021080520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 msgr2=0x7fabf8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 --2- 172.21.15.42:0/1021080520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf8068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009940 tx=0x7fabe8009c50 comp rx=0 tx=0).stop 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 -- 172.21.15.42:0/1021080520 shutdown_connections 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 --2- 172.21.15.42:0/1021080520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabf8068ea0 0x7fabf810d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 --2- 172.21.15.42:0/1021080520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf8068960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 -- 172.21.15.42:0/1021080520 >> 172.21.15.42:0/1021080520 conn(0x7fabf8075460 msgr2=0x7fabf8075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 -- 172.21.15.42:0/1021080520 shutdown_connections 2024-09-16T23:54:17.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.909+0000 7fabfc951700 1 -- 172.21.15.42:0/1021080520 wait complete. 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.910+0000 7fabfc951700 1 Processor -- start 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.910+0000 7fabfc951700 1 -- start start 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.910+0000 7fabfc951700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.910+0000 7fabfc951700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabf8068ea0 0x7fabf819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.910+0000 7fabfc951700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf819f1d0 con 0x7fabf8068580 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.910+0000 7fabfc951700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8198680 con 0x7fabf8068ea0 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40820/0 (socket says 172.21.15.42:40820) 2024-09-16T23:54:17.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf6d9d700 1 -- 172.21.15.42:0/3844986637 learned_addr learned my addr 172.21.15.42:0/3844986637 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:17.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf659c700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabf8068ea0 0x7fabf819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf6d9d700 1 -- 172.21.15.42:0/3844986637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabf8068ea0 msgr2=0x7fabf819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf6d9d700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabf8068ea0 0x7fabf819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf6d9d700 1 -- 172.21.15.42:0/3844986637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80095f0 con 0x7fabf8068580 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabf6d9d700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf819e600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fabe80055f0 tx=0x7fabe80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabdffff700 1 -- 172.21.15.42:0/3844986637 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe801d070 con 0x7fabf8068580 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf8198960 con 0x7fabf8068580 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabdffff700 1 -- 172.21.15.42:0/3844986637 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabe800f460 con 0x7fabf8068580 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabdffff700 1 -- 172.21.15.42:0/3844986637 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe8021640 con 0x7fabf8068580 2024-09-16T23:54:17.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.911+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf8198eb0 con 0x7fabf8068580 2024-09-16T23:54:17.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.913+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf810a800 con 0x7fabf8068580 2024-09-16T23:54:17.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.913+0000 7fabdffff700 1 -- 172.21.15.42:0/3844986637 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7fabe800fa20 con 0x7fabf8068580 2024-09-16T23:54:17.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.914+0000 7fabdffff700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fabe006e7b0 0x7fabe0070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:17.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.914+0000 7fabdffff700 1 -- 172.21.15.42:0/3844986637 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fabe808fae0 con 0x7fabf8068580 2024-09-16T23:54:17.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.914+0000 7fabf659c700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fabe006e7b0 0x7fabe0070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:17.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.914+0000 7fabf659c700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fabe006e7b0 0x7fabe0070c70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fabec005950 tx=0x7fabec0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:17.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:17.918+0000 7fabdffff700 1 -- 172.21.15.42:0/3844986637 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fabe805c640 con 0x7fabf8068580 2024-09-16T23:54:17.916 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:17 smithi042 ceph-mon[111056]: pgmap v48: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:17.916 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:17 smithi042 ceph-mon[111056]: from='osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T23:54:17.916 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:17 smithi042 ceph-mon[111056]: osdmap e52: 6 total, 5 up, 6 in 2024-09-16T23:54:17.916 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:17 smithi042 ceph-mon[111056]: from='osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:54:17.916 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:17 smithi042 ceph-mon[111056]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:17 smithi043 ceph-mon[81510]: pgmap v48: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:17 smithi043 ceph-mon[81510]: from='osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T23:54:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:17 smithi043 ceph-mon[81510]: osdmap e52: 6 total, 5 up, 6 in 2024-09-16T23:54:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:17 smithi043 ceph-mon[81510]: from='osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:54:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:17 smithi043 ceph-mon[81510]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:18.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.143+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fabf8199be0 con 0x7fabe006e7b0 2024-09-16T23:54:18.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.153+0000 7fabdffff700 1 -- 172.21.15.42:0/3844986637 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fabf8199be0 con 0x7fabe006e7b0 2024-09-16T23:54:18.150 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (2m) 3s ago 13m 24.5M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (13m) 3s ago 13m 8094k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (12m) 25s ago 12m 7964k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (33s) 3s ago 13m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (29s) 25s ago 12m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (2m) 3s ago 13m 76.2M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (7m) 3s ago 7m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (8m) 3s ago 8m 16.4M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (8m) 25s ago 8m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (7m) 25s ago 7m 17.2M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (4m) 3s ago 15m 494M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (3m) 25s ago 12m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (94s) 3s ago 15m 46.6M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:54:18.151 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (72s) 25s ago 12m 35.5M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (3m) 3s ago 13m 16.5M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (3m) 25s ago 12m 15.4M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (9s) 3s ago 11m 11.9M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (11m) 3s ago 11m 202M 1217M 18.2.1 5be31c24972a 7213cfa6e70e 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (10m) 3s ago 10m 164M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (10m) 25s ago 10m 235M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (10m) 25s ago 10m 222M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (9m) 25s ago 9m 202M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:54:18.152 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (3m) 3s ago 13m 73.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fabe006e7b0 msgr2=0x7fabe0070c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fabe006e7b0 0x7fabe0070c70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fabec005950 tx=0x7fabec0058e0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 msgr2=0x7fabf819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf819e600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fabe80055f0 tx=0x7fabe80056d0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 shutdown_connections 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fabe006e7b0 0x7fabe0070c70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabf8068580 0x7fabf819e600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 --2- 172.21.15.42:0/3844986637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fabf8068ea0 0x7fabf819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 >> 172.21.15.42:0/3844986637 conn(0x7fabf8075460 msgr2=0x7fabf80fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:18.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 shutdown_connections 2024-09-16T23:54:18.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.155+0000 7fabfc951700 1 -- 172.21.15.42:0/3844986637 wait complete. 2024-09-16T23:54:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 -- 172.21.15.42:0/4267326363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3670105bd0 msgr2=0x7f3670105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 --2- 172.21.15.42:0/4267326363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3670105bd0 0x7f3670105fb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f366c00b0a0 tx=0x7f366c00b3b0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 -- 172.21.15.42:0/4267326363 shutdown_connections 2024-09-16T23:54:18.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 --2- 172.21.15.42:0/4267326363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36700ffc20 0x7f3670100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 --2- 172.21.15.42:0/4267326363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3670105bd0 0x7f3670105fb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 -- 172.21.15.42:0/4267326363 >> 172.21.15.42:0/4267326363 conn(0x7f3670075670 msgr2=0x7f3670075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 -- 172.21.15.42:0/4267326363 shutdown_connections 2024-09-16T23:54:18.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.263+0000 7f3677639700 1 -- 172.21.15.42:0/4267326363 wait complete. 2024-09-16T23:54:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.264+0000 7f3677639700 1 Processor -- start 2024-09-16T23:54:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.264+0000 7f3677639700 1 -- start start 2024-09-16T23:54:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.264+0000 7f3677639700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36700ffc20 0x7f3670197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:18.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f3677639700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3670105bd0 0x7f3670197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f3677639700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3670198290 con 0x7f36700ffc20 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f3677639700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36701983d0 con 0x7f3670105bd0 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36753d5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36700ffc20 0x7f3670197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36753d5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36700ffc20 0x7f3670197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40848/0 (socket says 172.21.15.42:40848) 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36753d5700 1 -- 172.21.15.42:0/903147152 learned_addr learned my addr 172.21.15.42:0/903147152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f3674bd4700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3670105bd0 0x7f3670197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36753d5700 1 -- 172.21.15.42:0/903147152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3670105bd0 msgr2=0x7f3670197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36753d5700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3670105bd0 0x7f3670197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36753d5700 1 -- 172.21.15.42:0/903147152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c009d00 con 0x7f36700ffc20 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36753d5700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36700ffc20 0x7f3670197670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f366c000f80 tx=0x7f366c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36667fc700 1 -- 172.21.15.42:0/903147152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366c00e040 con 0x7f36700ffc20 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36701a06e0 con 0x7f36700ffc20 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36667fc700 1 -- 172.21.15.42:0/903147152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f366c003c10 con 0x7f36700ffc20 2024-09-16T23:54:18.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f36667fc700 1 -- 172.21.15.42:0/903147152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f366c01d6b0 con 0x7f36700ffc20 2024-09-16T23:54:18.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.265+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36701a0ba0 con 0x7f36700ffc20 2024-09-16T23:54:18.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.266+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36701080f0 con 0x7f36700ffc20 2024-09-16T23:54:18.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.267+0000 7f36667fc700 1 -- 172.21.15.42:0/903147152 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f366c004040 con 0x7f36700ffc20 2024-09-16T23:54:18.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.267+0000 7f36667fc700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f365c06e7c0 0x7f365c070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:18.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.268+0000 7f36667fc700 1 -- 172.21.15.42:0/903147152 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f366c08ffe0 con 0x7f36700ffc20 2024-09-16T23:54:18.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.268+0000 7f3674bd4700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f365c06e7c0 0x7f365c070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:18.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.268+0000 7f3674bd4700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f365c06e7c0 0x7f365c070c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3670069210 tx=0x7f366000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:18.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.272+0000 7f36667fc700 1 -- 172.21.15.42:0/903147152 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f366c059010 con 0x7f36700ffc20 2024-09-16T23:54:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.568+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3670068cd0 con 0x7f36700ffc20 2024-09-16T23:54:18.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.569+0000 7f36667fc700 1 -- 172.21.15.42:0/903147152 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f366c05c6d0 con 0x7f36700ffc20 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:54:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:54:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-09-16T23:54:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 5 2024-09-16T23:54:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:54:18.568 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f365c06e7c0 msgr2=0x7f365c070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f365c06e7c0 0x7f365c070c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3670069210 tx=0x7f366000b410 comp rx=0 tx=0).stop 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36700ffc20 msgr2=0x7f3670197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36700ffc20 0x7f3670197670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f366c000f80 tx=0x7f366c00cd20 comp rx=0 tx=0).stop 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 shutdown_connections 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f365c06e7c0 0x7f365c070c80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36700ffc20 0x7f3670197670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 --2- 172.21.15.42:0/903147152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3670105bd0 0x7f3670197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 >> 172.21.15.42:0/903147152 conn(0x7f3670075670 msgr2=0x7f36700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:18.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 shutdown_connections 2024-09-16T23:54:18.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.571+0000 7f3677639700 1 -- 172.21.15.42:0/903147152 wait complete. 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.713+0000 7f745ffff700 1 -- 172.21.15.42:0/3211373431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460101bf0 msgr2=0x7f7460101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.713+0000 7f745ffff700 1 --2- 172.21.15.42:0/3211373431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460101bf0 0x7f7460101fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7450009910 tx=0x7f7450009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.713+0000 7f745ffff700 1 -- 172.21.15.42:0/3211373431 shutdown_connections 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.713+0000 7f745ffff700 1 --2- 172.21.15.42:0/3211373431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460102510 0x7f746010aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.714+0000 7f745ffff700 1 --2- 172.21.15.42:0/3211373431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460101bf0 0x7f7460101fd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.714+0000 7f745ffff700 1 -- 172.21.15.42:0/3211373431 >> 172.21.15.42:0/3211373431 conn(0x7f74600fb420 msgr2=0x7f74600fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.714+0000 7f745ffff700 1 -- 172.21.15.42:0/3211373431 shutdown_connections 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.714+0000 7f745ffff700 1 -- 172.21.15.42:0/3211373431 wait complete. 2024-09-16T23:54:18.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.714+0000 7f745ffff700 1 Processor -- start 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745ffff700 1 -- start start 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460101bf0 0x7f7460196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460102510 0x7f7460196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745ffff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460197250 con 0x7f7460102510 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745ffff700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746019afe0 con 0x7f7460101bf0 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460101bf0 0x7f7460196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460101bf0 0x7f7460196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:49636/0 (socket says 172.21.15.42:49636) 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745effd700 1 -- 172.21.15.42:0/1043016430 learned_addr learned my addr 172.21.15.42:0/1043016430 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745e7fc700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460102510 0x7f7460196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745effd700 1 -- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460102510 msgr2=0x7f7460196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745effd700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460102510 0x7f7460196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.715+0000 7f745effd700 1 -- 172.21.15.42:0/1043016430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74500095f0 con 0x7f7460101bf0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.716+0000 7f745effd700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460101bf0 0x7f7460196630 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7450005f50 tx=0x7f745000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.716+0000 7f7447fff700 1 -- 172.21.15.42:0/1043016430 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f745001d070 con 0x7f7460101bf0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.716+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746019b260 con 0x7f7460101bf0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.716+0000 7f7447fff700 1 -- 172.21.15.42:0/1043016430 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f745000b610 con 0x7f7460101bf0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.716+0000 7f7447fff700 1 -- 172.21.15.42:0/1043016430 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7450022470 con 0x7f7460101bf0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.716+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746019b750 con 0x7f7460101bf0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.717+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7460108100 con 0x7f7460101bf0 2024-09-16T23:54:18.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.718+0000 7f7447fff700 1 -- 172.21.15.42:0/1043016430 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f7450003780 con 0x7f7460101bf0 2024-09-16T23:54:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.719+0000 7f7447fff700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f744806e7c0 0x7f7448070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.719+0000 7f7447fff700 1 -- 172.21.15.42:0/1043016430 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f745008fe40 con 0x7f7460101bf0 2024-09-16T23:54:18.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.719+0000 7f745e7fc700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f744806e7c0 0x7f7448070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:18.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.719+0000 7f745e7fc700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f744806e7c0 0x7f7448070c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7460197c50 tx=0x7f7454009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:18.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.723+0000 7f7447fff700 1 -- 172.21.15.42:0/1043016430 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7450058f90 con 0x7f7460101bf0 2024-09-16T23:54:18.821 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:18 smithi042 ceph-mon[111056]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:54:18.821 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:18 smithi042 ceph-mon[111056]: osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998] boot 2024-09-16T23:54:18.821 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:18 smithi042 ceph-mon[111056]: osdmap e53: 6 total, 6 up, 6 in 2024-09-16T23:54:18.822 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:18 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:54:18.822 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:18 smithi042 ceph-mon[111056]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:18.822 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:18 smithi042 ceph-mon[111056]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:18.822 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:18 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/903147152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:18.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.982+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f74600623c0 con 0x7f7460101bf0 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.983+0000 7f7447fff700 1 -- 172.21.15.42:0/1043016430 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1938 (secure 0 0 0) 0x7f745005c650 con 0x7f7460101bf0 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:53:20:175453+0000 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:54:18.981 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:53:19.510146+0000 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:54:18.982 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 24269,14478 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:18.983 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:18.984 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:54:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:18.984 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:18.984 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f744806e7c0 msgr2=0x7f7448070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f744806e7c0 0x7f7448070c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7460197c50 tx=0x7f7454009260 comp rx=0 tx=0).stop 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460101bf0 msgr2=0x7f7460196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460101bf0 0x7f7460196630 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7450005f50 tx=0x7f745000f690 comp rx=0 tx=0).stop 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 shutdown_connections 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f744806e7c0 0x7f7448070c80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7460101bf0 0x7f7460196630 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 --2- 172.21.15.42:0/1043016430 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7460102510 0x7f7460196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 >> 172.21.15.42:0/1043016430 conn(0x7f74600fb420 msgr2=0x7f7460105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 shutdown_connections 2024-09-16T23:54:18.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:18.986+0000 7f745ffff700 1 -- 172.21.15.42:0/1043016430 wait complete. 2024-09-16T23:54:18.986 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-16T23:54:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 -- 172.21.15.42:0/3665755367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 msgr2=0x7f1de0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 --2- 172.21.15.42:0/3665755367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0106f60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b0a0 tx=0x7f1dd000b3b0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 -- 172.21.15.42:0/3665755367 shutdown_connections 2024-09-16T23:54:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 --2- 172.21.15.42:0/3665755367 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de0100af0 0x7f1de0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 --2- 172.21.15.42:0/3665755367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0106f60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 -- 172.21.15.42:0/3665755367 >> 172.21.15.42:0/3665755367 conn(0x7f1de00fc5e0 msgr2=0x7f1de00fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 -- 172.21.15.42:0/3665755367 shutdown_connections 2024-09-16T23:54:19.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.102+0000 7f1de75b4700 1 -- 172.21.15.42:0/3665755367 wait complete. 2024-09-16T23:54:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.103+0000 7f1de75b4700 1 Processor -- start 2024-09-16T23:54:19.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.103+0000 7f1de75b4700 1 -- start start 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.103+0000 7f1de75b4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de0100af0 0x7f1de0196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.103+0000 7f1de75b4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.103+0000 7f1de75b4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0197250 con 0x7f1de0106b80 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.103+0000 7f1de75b4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0199f80 con 0x7f1de0100af0 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de4b4f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de4b4f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40898/0 (socket says 172.21.15.42:40898) 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de4b4f700 1 -- 172.21.15.42:0/2416702288 learned_addr learned my addr 172.21.15.42:0/2416702288 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de5350700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de0100af0 0x7f1de0196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de4b4f700 1 -- 172.21.15.42:0/2416702288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de0100af0 msgr2=0x7f1de0196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de4b4f700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de0100af0 0x7f1de0196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de4b4f700 1 -- 172.21.15.42:0/2416702288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd0009d00 con 0x7f1de0106b80 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de4b4f700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0196bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400e920 tx=0x7f1dd400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:19.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1dde7fc700 1 -- 172.21.15.42:0/2416702288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd400c9d0 con 0x7f1de0106b80 2024-09-16T23:54:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de019a260 con 0x7f1de0106b80 2024-09-16T23:54:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1dde7fc700 1 -- 172.21.15.42:0/2416702288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dd4010430 con 0x7f1de0106b80 2024-09-16T23:54:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1dde7fc700 1 -- 172.21.15.42:0/2416702288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd4018630 con 0x7f1de0106b80 2024-09-16T23:54:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.104+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de019a7b0 con 0x7f1de0106b80 2024-09-16T23:54:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.106+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de004f2e0 con 0x7f1de0106b80 2024-09-16T23:54:19.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.107+0000 7f1dde7fc700 1 -- 172.21.15.42:0/2416702288 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f1dd40105a0 con 0x7f1de0106b80 2024-09-16T23:54:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.107+0000 7f1dde7fc700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dc806e7c0 0x7f1dc8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.107+0000 7f1dde7fc700 1 -- 172.21.15.42:0/2416702288 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1dd4014070 con 0x7f1de0106b80 2024-09-16T23:54:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.107+0000 7f1de5350700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dc806e7c0 0x7f1dc8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:19.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.108+0000 7f1de5350700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dc806e7c0 0x7f1dc8070c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0000f80 tx=0x7f1dd00060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:19.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.116+0000 7f1dde7fc700 1 -- 172.21.15.42:0/2416702288 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1dd40541b0 con 0x7f1de0106b80 2024-09-16T23:54:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:18 smithi043 ceph-mon[81510]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:54:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:18 smithi043 ceph-mon[81510]: osd.0 [v2:172.21.15.42:6802/2361688998,v1:172.21.15.42:6803/2361688998] boot 2024-09-16T23:54:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:18 smithi043 ceph-mon[81510]: osdmap e53: 6 total, 6 up, 6 in 2024-09-16T23:54:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:18 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T23:54:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:18 smithi043 ceph-mon[81510]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:18 smithi043 ceph-mon[81510]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:18 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/903147152' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:19.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.348+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de019ab90 con 0x7f1dc806e7c0 2024-09-16T23:54:19.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.349+0000 7f1dde7fc700 1 -- 172.21.15.42:0/2416702288 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1de019ab90 con 0x7f1dc806e7c0 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "7/23 daemons upgraded", 2024-09-16T23:54:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T23:54:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:54:19.348 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:54:19.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dc806e7c0 msgr2=0x7f1dc8070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dc806e7c0 0x7f1dc8070c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0000f80 tx=0x7f1dd00060c0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 msgr2=0x7f1de0196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0196bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400e920 tx=0x7f1dd400ec30 comp rx=0 tx=0).stop 2024-09-16T23:54:19.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 shutdown_connections 2024-09-16T23:54:19.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dc806e7c0 0x7f1dc8070c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de0100af0 0x7f1de0196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 --2- 172.21.15.42:0/2416702288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de0106b80 0x7f1de0196bc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 >> 172.21.15.42:0/2416702288 conn(0x7f1de00fc5e0 msgr2=0x7f1de00fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:19.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.351+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 shutdown_connections 2024-09-16T23:54:19.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.352+0000 7f1de75b4700 1 -- 172.21.15.42:0/2416702288 wait complete. 2024-09-16T23:54:19.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.461+0000 7f4019bc4700 1 -- 172.21.15.42:0/952581569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014068580 msgr2=0x7f40140689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.461+0000 7f4019bc4700 1 --2- 172.21.15.42:0/952581569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014068580 0x7f40140689e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009910 tx=0x7f3ffc009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.461+0000 7f4019bc4700 1 -- 172.21.15.42:0/952581569 shutdown_connections 2024-09-16T23:54:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.461+0000 7f4019bc4700 1 --2- 172.21.15.42:0/952581569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014068580 0x7f40140689e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.461+0000 7f4019bc4700 1 --2- 172.21.15.42:0/952581569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4014106cd0 0x7f40141070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.461+0000 7f4019bc4700 1 -- 172.21.15.42:0/952581569 >> 172.21.15.42:0/952581569 conn(0x7f4014075670 msgr2=0x7f4014075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:19.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.461+0000 7f4019bc4700 1 -- 172.21.15.42:0/952581569 shutdown_connections 2024-09-16T23:54:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.462+0000 7f4019bc4700 1 -- 172.21.15.42:0/952581569 wait complete. 2024-09-16T23:54:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.462+0000 7f4019bc4700 1 Processor -- start 2024-09-16T23:54:19.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.462+0000 7f4019bc4700 1 -- start start 2024-09-16T23:54:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f4019bc4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4014068580 0x7f401419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f4019bc4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014106cd0 0x7f401419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f4019bc4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401419d200 con 0x7f4014106cd0 2024-09-16T23:54:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f4019bc4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40141966b0 con 0x7f4014068580 2024-09-16T23:54:19.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f400affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014106cd0 0x7f401419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f400affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014106cd0 0x7f401419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40912/0 (socket says 172.21.15.42:40912) 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f400affd700 1 -- 172.21.15.42:0/106594070 learned_addr learned my addr 172.21.15.42:0/106594070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f40137fe700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4014068580 0x7f401419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f400affd700 1 -- 172.21.15.42:0/106594070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4014068580 msgr2=0x7f401419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f400affd700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4014068580 0x7f401419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f400affd700 1 -- 172.21.15.42:0/106594070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc0095f0 con 0x7f4014106cd0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.463+0000 7f400affd700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014106cd0 0x7f401419cb70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc006010 tx=0x7f3ffc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.464+0000 7f40117fa700 1 -- 172.21.15.42:0/106594070 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ffc01d070 con 0x7f4014106cd0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.464+0000 7f40117fa700 1 -- 172.21.15.42:0/106594070 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ffc005dc0 con 0x7f4014106cd0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.464+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4014196930 con 0x7f4014106cd0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.464+0000 7f40117fa700 1 -- 172.21.15.42:0/106594070 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ffc017400 con 0x7f4014106cd0 2024-09-16T23:54:19.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.464+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4014196e20 con 0x7f4014106cd0 2024-09-16T23:54:19.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.465+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40141091d0 con 0x7f4014106cd0 2024-09-16T23:54:19.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.466+0000 7f40117fa700 1 -- 172.21.15.42:0/106594070 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 37) v1 ==== 92653+0+0 (secure 0 0 0) 0x7f3ffc003a60 con 0x7f4014106cd0 2024-09-16T23:54:19.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.466+0000 7f40117fa700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f400006e7c0 0x7f4000070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:19.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.466+0000 7f40117fa700 1 -- 172.21.15.42:0/106594070 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3ffc08fca0 con 0x7f4014106cd0 2024-09-16T23:54:19.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.466+0000 7f40137fe700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f400006e7c0 0x7f4000070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:19.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.467+0000 7f40137fe700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f400006e7c0 0x7f4000070c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4004005950 tx=0x7f400400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:19.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.471+0000 7f40117fa700 1 -- 172.21.15.42:0/106594070 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3ffc05c7e0 con 0x7f4014106cd0 2024-09-16T23:54:19.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.768+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f401404f2e0 con 0x7f4014106cd0 2024-09-16T23:54:19.766 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: pgmap v51: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:19.766 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:19.766 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: osdmap e54: 6 total, 6 up, 6 in 2024-09-16T23:54:19.766 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1043016430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:54:19.767 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:19 smithi042 ceph-mon[111056]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:19.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.768+0000 7f40117fa700 1 -- 172.21.15.42:0/106594070 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1429 (secure 0 0 0) 0x7f3ffc05c370 con 0x7f4014106cd0 2024-09-16T23:54:19.768 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded 2024-09-16T23:54:19.768 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:54:19.768 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:54:19.768 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded 2024-09-16T23:54:19.768 INFO:teuthology.orchestra.run.smithi042.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-09-16T23:54:19.768 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-09-16T23:54:19.769 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-09-16T23:54:19.770 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-09-16T23:54:19.770 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-09-16T23:54:19.770 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-09-16T23:54:19.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.771+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f400006e7c0 msgr2=0x7f4000070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.771+0000 7f4019bc4700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f400006e7c0 0x7f4000070c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4004005950 tx=0x7f400400b410 comp rx=0 tx=0).stop 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.771+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014106cd0 msgr2=0x7f401419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.771+0000 7f4019bc4700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014106cd0 0x7f401419cb70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc006010 tx=0x7f3ffc00bd70 comp rx=0 tx=0).stop 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.772+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 shutdown_connections 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.772+0000 7f4019bc4700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f400006e7c0 0x7f4000070c80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.772+0000 7f4019bc4700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4014068580 0x7f401419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.772+0000 7f4019bc4700 1 --2- 172.21.15.42:0/106594070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4014106cd0 0x7f401419cb70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.772+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 >> 172.21.15.42:0/106594070 conn(0x7f4014075670 msgr2=0x7f40140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.772+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 shutdown_connections 2024-09-16T23:54:19.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:19.772+0000 7f4019bc4700 1 -- 172.21.15.42:0/106594070 wait complete. 2024-09-16T23:54:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: pgmap v51: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: osdmap e54: 6 total, 6 up, 6 in 2024-09-16T23:54:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:54:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1043016430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:54:20.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:19 smithi043 ceph-mon[81510]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:21.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:20 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/106594070' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:54:21.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:21.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:21.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:20 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/106594070' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:54:21.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:21.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:21 smithi043 ceph-mon[81510]: pgmap v53: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:21 smithi042 ceph-mon[111056]: pgmap v53: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:54:22.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:22 smithi042 ceph-mon[111056]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded) 2024-09-16T23:54:22.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:22 smithi042 ceph-mon[111056]: mgrmap e38: smithi042.zrkumz(active, since 92s), standbys: smithi043.xveyou 2024-09-16T23:54:23.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:22 smithi043 ceph-mon[81510]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded) 2024-09-16T23:54:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:22 smithi043 ceph-mon[81510]: mgrmap e38: smithi042.zrkumz(active, since 92s), standbys: smithi043.xveyou 2024-09-16T23:54:24.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:23 smithi042 ceph-mon[111056]: pgmap v54: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:54:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:23 smithi043 ceph-mon[81510]: pgmap v54: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-09-16T23:54:26.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:25 smithi042 ceph-mon[111056]: pgmap v55: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 774 B/s wr, 0 op/s 2024-09-16T23:54:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:25 smithi043 ceph-mon[81510]: pgmap v55: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 774 B/s wr, 0 op/s 2024-09-16T23:54:28.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:27 smithi042 ceph-mon[111056]: pgmap v56: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 687 B/s wr, 0 op/s 2024-09-16T23:54:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:27 smithi043 ceph-mon[81510]: pgmap v56: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 687 B/s wr, 0 op/s 2024-09-16T23:54:30.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:29 smithi042 ceph-mon[111056]: pgmap v57: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 614 B/s wr, 0 op/s 2024-09-16T23:54:30.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:29 smithi043 ceph-mon[81510]: pgmap v57: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 614 B/s wr, 0 op/s 2024-09-16T23:54:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:31 smithi043 ceph-mon[81510]: pgmap v58: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 515 B/s wr, 0 op/s 2024-09-16T23:54:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:31 smithi042 ceph-mon[111056]: pgmap v58: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 515 B/s wr, 0 op/s 2024-09-16T23:54:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:34.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:33 smithi042 ceph-mon[111056]: pgmap v59: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:54:34.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:33 smithi043 ceph-mon[81510]: pgmap v59: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T23:54:35.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:35.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:34 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:35.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:34 smithi042 ceph-mon[111056]: Upgrade: osd.1 is safe to restart 2024-09-16T23:54:35.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:35.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:34 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T23:54:35.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:34 smithi043 ceph-mon[81510]: Upgrade: osd.1 is safe to restart 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: pgmap v60: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: Upgrade: Updating osd.1 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: Deploying daemon osd.1 on smithi042 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:36.601 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: pgmap v60: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:36.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: Upgrade: Updating osd.1 2024-09-16T23:54:36.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:36.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T23:54:36.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:36.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: Deploying daemon osd.1 on smithi042 2024-09-16T23:54:36.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:36.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:38.533 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:38 smithi042 ceph-mon[111056]: pgmap v61: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:38.534 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:38 smithi042 systemd[1]: Stopping Ceph osd.1 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:54:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:38 smithi043 ceph-mon[81510]: pgmap v61: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:39.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:38 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[54391]: 2024-09-16T23:54:38.535+0000 7fc70338f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:54:39.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:38 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[54391]: 2024-09-16T23:54:38.535+0000 7fc70338f700 -1 osd.1 54 *** Got signal Terminated *** 2024-09-16T23:54:39.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:38 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[54391]: 2024-09-16T23:54:38.535+0000 7fc70338f700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T23:54:39.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:39 smithi043 ceph-mon[81510]: osd.1 marked itself down and dead 2024-09-16T23:54:39.728 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:39 smithi042 ceph-mon[111056]: osd.1 marked itself down and dead 2024-09-16T23:54:40.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:39 smithi042 podman[127519]: 2024-09-16 23:54:39.731117118 +0000 UTC m=+1.305391366 container died 7213cfa6e70e66fa2435cdac5a16c2bc4b5dc1d5752c2efe5c4f800590f620d1 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T23:54:40.278 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:40 smithi042 podman[127519]: 2024-09-16 23:54:40.101918041 +0000 UTC m=+1.676192297 container cleanup 7213cfa6e70e66fa2435cdac5a16c2bc4b5dc1d5752c2efe5c4f800590f620d1 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-16T23:54:40.278 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:40 smithi042 bash[127519]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1 2024-09-16T23:54:40.529 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:40 smithi043 ceph-mon[81510]: pgmap v62: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:40.529 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:40 smithi043 ceph-mon[81510]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:54:40.529 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:40 smithi043 ceph-mon[81510]: osdmap e55: 6 total, 5 up, 6 in 2024-09-16T23:54:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:40 smithi042 ceph-mon[111056]: pgmap v62: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:40 smithi042 ceph-mon[111056]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:54:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:40 smithi042 ceph-mon[111056]: osdmap e55: 6 total, 5 up, 6 in 2024-09-16T23:54:40.773 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:40 smithi042 podman[127534]: 2024-09-16 23:54:40.428780032 +0000 UTC m=+0.689720870 container remove 7213cfa6e70e66fa2435cdac5a16c2bc4b5dc1d5752c2efe5c4f800590f620d1 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS) 2024-09-16T23:54:41.129 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:40 smithi042 podman[127625]: 2024-09-16 23:54:40.685344437 +0000 UTC m=+0.016179379 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:54:41.130 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:40 smithi042 podman[127625]: 2024-09-16 23:54:40.87187889 +0000 UTC m=+0.202713856 container create d41d077f1fa5524c1138708b9f7e61b6f05ceb610554bc42a1a216874a9fd068 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-16T23:54:41.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:41 smithi042 ceph-mon[111056]: osdmap e56: 6 total, 5 up, 6 in 2024-09-16T23:54:41.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:41 smithi042 podman[127625]: 2024-09-16 23:54:41.132803613 +0000 UTC m=+0.463638550 container init d41d077f1fa5524c1138708b9f7e61b6f05ceb610554bc42a1a216874a9fd068 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T23:54:41.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:41 smithi042 podman[127625]: 2024-09-16 23:54:41.136432659 +0000 UTC m=+0.467267585 container start d41d077f1fa5524c1138708b9f7e61b6f05ceb610554bc42a1a216874a9fd068 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS) 2024-09-16T23:54:41.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:41 smithi042 podman[127625]: 2024-09-16 23:54:41.224476432 +0000 UTC m=+0.555311355 container attach d41d077f1fa5524c1138708b9f7e61b6f05ceb610554bc42a1a216874a9fd068 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T23:54:41.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:41 smithi042 podman[127625]: 2024-09-16 23:54:41.32906816 +0000 UTC m=+0.659903094 container died d41d077f1fa5524c1138708b9f7e61b6f05ceb610554bc42a1a216874a9fd068 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T23:54:41.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:41 smithi043 ceph-mon[81510]: osdmap e56: 6 total, 5 up, 6 in 2024-09-16T23:54:42.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 podman[127647]: 2024-09-16 23:54:42.022548969 +0000 UTC m=+0.682341911 container remove d41d077f1fa5524c1138708b9f7e61b6f05ceb610554bc42a1a216874a9fd068 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T23:54:42.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-16T23:54:42.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 systemd[1]: Stopped Ceph osd.1 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:54:42.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.1.service: Consumed 58.671s CPU time. 2024-09-16T23:54:42.628 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:42 smithi042 ceph-mon[111056]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:42 smithi043 ceph-mon[81510]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:54:42.629 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 systemd[1]: Starting Ceph osd.1 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:54:42.629 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 podman[127749]: 2024-09-16 23:54:42.410600787 +0000 UTC m=+0.016512550 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:54:42.629 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 podman[127749]: 2024-09-16 23:54:42.523740276 +0000 UTC m=+0.129652028 container create c72c4de3e33bcbb8716a215c38db81745f95ad429887f4e5d98a1fbb4b3b4d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T23:54:42.889 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 podman[127749]: 2024-09-16 23:54:42.751520164 +0000 UTC m=+0.357431933 container init c72c4de3e33bcbb8716a215c38db81745f95ad429887f4e5d98a1fbb4b3b4d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T23:54:42.889 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 podman[127749]: 2024-09-16 23:54:42.75514614 +0000 UTC m=+0.361057901 container start c72c4de3e33bcbb8716a215c38db81745f95ad429887f4e5d98a1fbb4b3b4d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:54:42.889 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 podman[127749]: 2024-09-16 23:54:42.859909822 +0000 UTC m=+0.465821575 container attach c72c4de3e33bcbb8716a215c38db81745f95ad429887f4e5d98a1fbb4b3b4d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T23:54:43.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:43.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 bash[127749]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:43.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:43.273 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:42 smithi042 bash[127749]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:54:43.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:43 smithi043 ceph-mon[81510]: Health check failed: Degraded data redundancy: 58/369 objects degraded (15.718%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T23:54:43.632 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-mon[111056]: Health check failed: Degraded data redundancy: 58/369 objects degraded (15.718%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T23:54:43.923 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T23:54:43.923 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 bash[127749]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T23:54:43.923 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T23:54:43.923 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 bash[127749]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T23:54:44.261 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T23:54:44.261 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 bash[127749]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T23:54:44.261 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T23:54:44.261 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 bash[127749]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T23:54:44.262 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-16T23:54:44.262 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 bash[127749]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-16T23:54:44.262 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T23:54:44.262 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 bash[127749]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T23:54:44.262 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate[127765]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-16T23:54:44.262 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 bash[127749]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-16T23:54:44.262 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:43 smithi042 podman[127915]: 2024-09-16 23:54:43.988901296 +0000 UTC m=+0.018778220 container died c72c4de3e33bcbb8716a215c38db81745f95ad429887f4e5d98a1fbb4b3b4d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T23:54:44.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:44 smithi042 ceph-mon[111056]: pgmap v66: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:44 smithi043 ceph-mon[81510]: pgmap v66: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:44.861 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:44 smithi042 podman[127915]: 2024-09-16 23:54:44.580809013 +0000 UTC m=+0.610685938 container remove c72c4de3e33bcbb8716a215c38db81745f95ad429887f4e5d98a1fbb4b3b4d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-activate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T23:54:44.862 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:44 smithi042 podman[127975]: 2024-09-16 23:54:44.701293642 +0000 UTC m=+0.017462038 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:54:44.862 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:44 smithi042 podman[127975]: 2024-09-16 23:54:44.864463492 +0000 UTC m=+0.180631913 container create c378ed398a4d3689c7f970096b32b7ea86724a4c5239b40f7a91157b338e16f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T23:54:45.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:45 smithi042 podman[127975]: 2024-09-16 23:54:45.217490544 +0000 UTC m=+0.533658930 container init c378ed398a4d3689c7f970096b32b7ea86724a4c5239b40f7a91157b338e16f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:54:45.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:45 smithi042 podman[127975]: 2024-09-16 23:54:45.221079768 +0000 UTC m=+0.537248157 container start c378ed398a4d3689c7f970096b32b7ea86724a4c5239b40f7a91157b338e16f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T23:54:45.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:45 smithi042 ceph-osd[127994]: -- 172.21.15.42:0/4294821044 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e6eceb01e0 con 0x55e6ec29e000 2024-09-16T23:54:45.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:45 smithi042 bash[127975]: c378ed398a4d3689c7f970096b32b7ea86724a4c5239b40f7a91157b338e16f3 2024-09-16T23:54:45.523 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:45 smithi042 systemd[1]: Started Ceph osd.1 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:54:46.701 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:46 smithi043 ceph-mon[81510]: pgmap v67: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:46.701 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:46.701 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:46.701 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:46 smithi042 ceph-mon[111056]: pgmap v67: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:47.272 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:46 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[127990]: 2024-09-16T23:54:46.880+0000 7f47716f7640 -1 Falling back to public interface 2024-09-16T23:54:48.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:48 smithi042 ceph-mon[111056]: pgmap v68: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:48 smithi043 ceph-mon[81510]: pgmap v68: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 -- 172.21.15.42:0/4062847648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 msgr2=0x7fe1001058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 --2- 172.21.15.42:0/4062847648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe1001058a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009910 tx=0x7fe0e8009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 -- 172.21.15.42:0/4062847648 shutdown_connections 2024-09-16T23:54:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 --2- 172.21.15.42:0/4062847648 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe1001058a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 --2- 172.21.15.42:0/4062847648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe100100ea0 0x7fe100101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 -- 172.21.15.42:0/4062847648 >> 172.21.15.42:0/4062847648 conn(0x7fe1000fc750 msgr2=0x7fe1000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:49.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 -- 172.21.15.42:0/4062847648 shutdown_connections 2024-09-16T23:54:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.889+0000 7fe107103700 1 -- 172.21.15.42:0/4062847648 wait complete. 2024-09-16T23:54:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe107103700 1 Processor -- start 2024-09-16T23:54:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe107103700 1 -- start start 2024-09-16T23:54:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe107103700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe100100ea0 0x7fe10019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe107103700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe10019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:49.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe107103700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10019d240 con 0x7fe100101850 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe107103700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100196780 con 0x7fe100100ea0 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe0fffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe10019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe0fffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe10019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51742/0 (socket says 172.21.15.42:51742) 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe0fffff700 1 -- 172.21.15.42:0/3928943693 learned_addr learned my addr 172.21.15.42:0/3928943693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe104e9f700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe100100ea0 0x7fe10019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.890+0000 7fe0fffff700 1 -- 172.21.15.42:0/3928943693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe100100ea0 msgr2=0x7fe10019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe0fffff700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe100100ea0 0x7fe10019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe0fffff700 1 -- 172.21.15.42:0/3928943693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e80095f0 con 0x7fe100101850 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe0fffff700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe10019cc40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800bda0 tx=0x7fe0e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe0fdffb700 1 -- 172.21.15.42:0/3928943693 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e801d070 con 0x7fe100101850 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe100196a00 con 0x7fe100101850 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe0fdffb700 1 -- 172.21.15.42:0/3928943693 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0e8004fb0 con 0x7fe100101850 2024-09-16T23:54:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe0fdffb700 1 -- 172.21.15.42:0/3928943693 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e800f480 con 0x7fe100101850 2024-09-16T23:54:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.891+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe100196ef0 con 0x7fe100101850 2024-09-16T23:54:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.892+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1001092a0 con 0x7fe100101850 2024-09-16T23:54:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.893+0000 7fe0fdffb700 1 -- 172.21.15.42:0/3928943693 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe0e8022870 con 0x7fe100101850 2024-09-16T23:54:49.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.893+0000 7fe0fdffb700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0ec06e680 0x7fe0ec070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:49.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.893+0000 7fe0fdffb700 1 -- 172.21.15.42:0/3928943693 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe0e808fb20 con 0x7fe100101850 2024-09-16T23:54:49.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.893+0000 7fe104e9f700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0ec06e680 0x7fe0ec070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:49.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.894+0000 7fe104e9f700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0ec06e680 0x7fe0ec070b40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000ba60 tx=0x7fe0f0005d50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:49.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:49.898+0000 7fe0fdffb700 1 -- 172.21.15.42:0/3928943693 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe0e805c950 con 0x7fe100101850 2024-09-16T23:54:50.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.133+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe100197110 con 0x7fe0ec06e680 2024-09-16T23:54:50.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.135+0000 7fe0fdffb700 1 -- 172.21.15.42:0/3928943693 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe100197110 con 0x7fe0ec06e680 2024-09-16T23:54:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.138+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0ec06e680 msgr2=0x7fe0ec070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.138+0000 7fe107103700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0ec06e680 0x7fe0ec070b40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000ba60 tx=0x7fe0f0005d50 comp rx=0 tx=0).stop 2024-09-16T23:54:50.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.138+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 msgr2=0x7fe10019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.138+0000 7fe107103700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe10019cc40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800bda0 tx=0x7fe0e800be80 comp rx=0 tx=0).stop 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.139+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 shutdown_connections 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.139+0000 7fe107103700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0ec06e680 0x7fe0ec070b40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.139+0000 7fe107103700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe100100ea0 0x7fe10019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.139+0000 7fe107103700 1 --2- 172.21.15.42:0/3928943693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe100101850 0x7fe10019cc40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.139+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 >> 172.21.15.42:0/3928943693 conn(0x7fe1000fc750 msgr2=0x7fe1000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.139+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 shutdown_connections 2024-09-16T23:54:50.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.139+0000 7fe107103700 1 -- 172.21.15.42:0/3928943693 wait complete. 2024-09-16T23:54:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:54:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 -- 172.21.15.42:0/1873459652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 msgr2=0x7f7008102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 --2- 172.21.15.42:0/1873459652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008102280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009910 tx=0x7f6ffc009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:50.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 -- 172.21.15.42:0/1873459652 shutdown_connections 2024-09-16T23:54:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 --2- 172.21.15.42:0/1873459652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70081027c0 0x7f700810acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 --2- 172.21.15.42:0/1873459652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008102280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 -- 172.21.15.42:0/1873459652 >> 172.21.15.42:0/1873459652 conn(0x7f70080fb610 msgr2=0x7f70080fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 -- 172.21.15.42:0/1873459652 shutdown_connections 2024-09-16T23:54:50.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.274+0000 7f700f252700 1 -- 172.21.15.42:0/1873459652 wait complete. 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.275+0000 7f700f252700 1 Processor -- start 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.275+0000 7f700f252700 1 -- start start 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f700f252700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f700f252700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70081027c0 0x7f7008194b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f700f252700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008195270 con 0x7f7008101ea0 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f700f252700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008199000 con 0x7f70081027c0 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f700cfee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f700cfee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008194650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51770/0 (socket says 172.21.15.42:51770) 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f700cfee700 1 -- 172.21.15.42:0/3033968357 learned_addr learned my addr 172.21.15.42:0/3033968357 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:50.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.276+0000 7f7007fff700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70081027c0 0x7f7008194b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f7007fff700 1 -- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 msgr2=0x7f7008194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f7007fff700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008194650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f7007fff700 1 -- 172.21.15.42:0/3033968357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc0095f0 con 0x7f70081027c0 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f700cfee700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008194650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f7007fff700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70081027c0 0x7f7008194b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4012be0 tx=0x7f6ff4012ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f7005ffb700 1 -- 172.21.15.42:0/3033968357 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff40106a0 con 0x7f70081027c0 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f7005ffb700 1 -- 172.21.15.42:0/3033968357 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ff4013040 con 0x7f70081027c0 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f7005ffb700 1 -- 172.21.15.42:0/3033968357 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff400bdf0 con 0x7f70081027c0 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7008199280 con 0x7f70081027c0 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.277+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70081997d0 con 0x7f70081027c0 2024-09-16T23:54:50.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.278+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70081083b0 con 0x7f70081027c0 2024-09-16T23:54:50.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.280+0000 7f7005ffb700 1 -- 172.21.15.42:0/3033968357 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6ff4010800 con 0x7f70081027c0 2024-09-16T23:54:50.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.280+0000 7f7005ffb700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ff806e680 0x7f6ff8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:50.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.280+0000 7f7005ffb700 1 -- 172.21.15.42:0/3033968357 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6ff40925e0 con 0x7f70081027c0 2024-09-16T23:54:50.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.280+0000 7f700cfee700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ff806e680 0x7f6ff8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:50.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.281+0000 7f700cfee700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ff806e680 0x7f6ff8070b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00b760 tx=0x7f6ffc005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:50.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.285+0000 7f7005ffb700 1 -- 172.21.15.42:0/3033968357 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6ff405b8b0 con 0x7f70081027c0 2024-09-16T23:54:50.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.553+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7008195a50 con 0x7f6ff806e680 2024-09-16T23:54:50.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.554+0000 7f7005ffb700 1 -- 172.21.15.42:0/3033968357 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7008195a50 con 0x7f6ff806e680 2024-09-16T23:54:50.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ff806e680 msgr2=0x7f6ff8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ff806e680 0x7f6ff8070b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00b760 tx=0x7f6ffc005a90 comp rx=0 tx=0).stop 2024-09-16T23:54:50.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70081027c0 msgr2=0x7f7008194b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70081027c0 0x7f7008194b90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4012be0 tx=0x7f6ff4012ef0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 shutdown_connections 2024-09-16T23:54:50.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ff806e680 0x7f6ff8070b40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7008101ea0 0x7f7008194650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 --2- 172.21.15.42:0/3033968357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70081027c0 0x7f7008194b90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.556+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 >> 172.21.15.42:0/3033968357 conn(0x7f70080fb610 msgr2=0x7f70081054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:50.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.557+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 shutdown_connections 2024-09-16T23:54:50.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.557+0000 7f700f252700 1 -- 172.21.15.42:0/3033968357 wait complete. 2024-09-16T23:54:50.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 -- 172.21.15.42:0/115600075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 msgr2=0x7fb688101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 --2- 172.21.15.42:0/115600075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb688101270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb678009910 tx=0x7fb678009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:50.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 -- 172.21.15.42:0/115600075 shutdown_connections 2024-09-16T23:54:50.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 --2- 172.21.15.42:0/115600075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb688101840 0x7fb688105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 --2- 172.21.15.42:0/115600075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb688101270 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 -- 172.21.15.42:0/115600075 >> 172.21.15.42:0/115600075 conn(0x7fb6880fc740 msgr2=0x7fb6880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:50.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 -- 172.21.15.42:0/115600075 shutdown_connections 2024-09-16T23:54:50.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.681+0000 7fb68e680700 1 -- 172.21.15.42:0/115600075 wait complete. 2024-09-16T23:54:50.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb68e680700 1 Processor -- start 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb68e680700 1 -- start start 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb68e680700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb68819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb68e680700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb688101840 0x7fb68819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb68e680700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68819d230 con 0x7fb688100e90 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb68e680700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6881966e0 con 0x7fb688101840 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb687fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb68819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb687fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb68819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51796/0 (socket says 172.21.15.42:51796) 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.682+0000 7fb687fff700 1 -- 172.21.15.42:0/200308294 learned_addr learned my addr 172.21.15.42:0/200308294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb67ffff700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb688101840 0x7fb68819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb687fff700 1 -- 172.21.15.42:0/200308294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb688101840 msgr2=0x7fb68819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb687fff700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb688101840 0x7fb68819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb687fff700 1 -- 172.21.15.42:0/200308294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780095f0 con 0x7fb688100e90 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb67ffff700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb688101840 0x7fb68819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:54:50.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb687fff700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb68819c660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb678006010 tx=0x7fb67800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:50.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb685ffb700 1 -- 172.21.15.42:0/200308294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67801d070 con 0x7fb688100e90 2024-09-16T23:54:50.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb685ffb700 1 -- 172.21.15.42:0/200308294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb678004fb0 con 0x7fb688100e90 2024-09-16T23:54:50.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb688196960 con 0x7fb688100e90 2024-09-16T23:54:50.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb685ffb700 1 -- 172.21.15.42:0/200308294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb678022ea0 con 0x7fb688100e90 2024-09-16T23:54:50.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.683+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb688196e50 con 0x7fb688100e90 2024-09-16T23:54:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.684+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb688109230 con 0x7fb688100e90 2024-09-16T23:54:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.685+0000 7fb685ffb700 1 -- 172.21.15.42:0/200308294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb678022470 con 0x7fb688100e90 2024-09-16T23:54:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.686+0000 7fb685ffb700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb67406e680 0x7fb674070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.686+0000 7fb685ffb700 1 -- 172.21.15.42:0/200308294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb67808fbe0 con 0x7fb688100e90 2024-09-16T23:54:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.686+0000 7fb67ffff700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb67406e680 0x7fb674070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.687+0000 7fb67ffff700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb67406e680 0x7fb674070b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb688197b50 tx=0x7fb670005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:50.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.691+0000 7fb685ffb700 1 -- 172.21.15.42:0/200308294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb67805c970 con 0x7fb688100e90 2024-09-16T23:54:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:50 smithi042 ceph-mon[111056]: pgmap v69: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:50 smithi043 ceph-mon[81510]: pgmap v69: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:50.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.925+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb688197130 con 0x7fb67406e680 2024-09-16T23:54:50.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.943+0000 7fb685ffb700 1 -- 172.21.15.42:0/200308294 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb688197130 con 0x7fb67406e680 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (3m) 0s ago 14m 24.7M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (14m) 0s ago 14m 8296k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (13m) 58s ago 13m 7964k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (66s) 0s ago 14m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (61s) 58s ago 13m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (3m) 0s ago 14m 76.6M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (8m) 0s ago 8m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (8m) 0s ago 8m 16.6M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (8m) 58s ago 8m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (8m) 58s ago 8m 17.2M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (5m) 0s ago 15m 498M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:54:50.941 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (4m) 58s ago 13m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (2m) 0s ago 15m 49.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (105s) 58s ago 13m 35.5M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (4m) 0s ago 14m 17.0M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (4m) 58s ago 13m 15.4M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (41s) 0s ago 12m 83.3M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (6s) 0s ago 11m 11.2M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (11m) 0s ago 11m 165M 1217M 18.2.1 5be31c24972a 3d6cb9cb77ab 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (11m) 58s ago 11m 235M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (10m) 58s ago 10m 222M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (10m) 58s ago 10m 202M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:54:50.942 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (3m) 0s ago 13m 73.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:54:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb67406e680 msgr2=0x7fb674070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb67406e680 0x7fb674070b40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb688197b50 tx=0x7fb670005f50 comp rx=0 tx=0).stop 2024-09-16T23:54:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 msgr2=0x7fb68819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb68819c660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb678006010 tx=0x7fb67800be20 comp rx=0 tx=0).stop 2024-09-16T23:54:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 shutdown_connections 2024-09-16T23:54:50.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb67406e680 0x7fb674070b40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb688100e90 0x7fb68819c660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 --2- 172.21.15.42:0/200308294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb688101840 0x7fb68819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.945+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 >> 172.21.15.42:0/200308294 conn(0x7fb6880fc740 msgr2=0x7fb6880fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.946+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 shutdown_connections 2024-09-16T23:54:50.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:50.946+0000 7fb68e680700 1 -- 172.21.15.42:0/200308294 wait complete. 2024-09-16T23:54:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 -- 172.21.15.42:0/4091357404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc1027c0 msgr2=0x7ff8dc10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 --2- 172.21.15.42:0/4091357404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc1027c0 0x7ff8dc10acb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009910 tx=0x7ff8c4009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 -- 172.21.15.42:0/4091357404 shutdown_connections 2024-09-16T23:54:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 --2- 172.21.15.42:0/4091357404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc1027c0 0x7ff8dc10acb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 --2- 172.21.15.42:0/4091357404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8dc101ea0 0x7ff8dc102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 -- 172.21.15.42:0/4091357404 >> 172.21.15.42:0/4091357404 conn(0x7ff8dc0fb650 msgr2=0x7ff8dc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:51.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 -- 172.21.15.42:0/4091357404 shutdown_connections 2024-09-16T23:54:51.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.063+0000 7ff8e2990700 1 -- 172.21.15.42:0/4091357404 wait complete. 2024-09-16T23:54:51.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8e2990700 1 Processor -- start 2024-09-16T23:54:51.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8e2990700 1 -- start start 2024-09-16T23:54:51.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8e2990700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc101ea0 0x7ff8dc0ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8e2990700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8dc1027c0 0x7ff8dc100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8e2990700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc100580 con 0x7ff8dc101ea0 2024-09-16T23:54:51.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8e2990700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc1006c0 con 0x7ff8dc1027c0 2024-09-16T23:54:51.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc101ea0 0x7ff8dc0ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc101ea0 0x7ff8dc0ffb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51804/0 (socket says 172.21.15.42:51804) 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8dbfff700 1 -- 172.21.15.42:0/2427461606 learned_addr learned my addr 172.21.15.42:0/2427461606 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.064+0000 7ff8db7fe700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8dc1027c0 0x7ff8dc100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8dbfff700 1 -- 172.21.15.42:0/2427461606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8dc1027c0 msgr2=0x7ff8dc100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8dbfff700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8dc1027c0 0x7ff8dc100040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8dbfff700 1 -- 172.21.15.42:0/2427461606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c40095f0 con 0x7ff8dc101ea0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8dbfff700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc101ea0 0x7ff8dc0ffb00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00d8d0 tx=0x7ff8cc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8d97fa700 1 -- 172.21.15.42:0/2427461606 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8cc0098a0 con 0x7ff8dc101ea0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc1a2a60 con 0x7ff8dc101ea0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8d97fa700 1 -- 172.21.15.42:0/2427461606 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8cc010460 con 0x7ff8dc101ea0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc1a2e70 con 0x7ff8dc101ea0 2024-09-16T23:54:51.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.065+0000 7ff8d97fa700 1 -- 172.21.15.42:0/2427461606 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8cc00f5f0 con 0x7ff8dc101ea0 2024-09-16T23:54:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.066+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc108340 con 0x7ff8dc101ea0 2024-09-16T23:54:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.067+0000 7ff8d97fa700 1 -- 172.21.15.42:0/2427461606 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff8cc009a00 con 0x7ff8dc101ea0 2024-09-16T23:54:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.067+0000 7ff8d97fa700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff8c806e680 0x7ff8c8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.067+0000 7ff8d97fa700 1 -- 172.21.15.42:0/2427461606 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff8cc08e420 con 0x7ff8dc101ea0 2024-09-16T23:54:51.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.068+0000 7ff8db7fe700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff8c806e680 0x7ff8c8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.068+0000 7ff8db7fe700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff8c806e680 0x7ff8c8070b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc1012f0 tx=0x7ff8c401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:51.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.076+0000 7ff8d97fa700 1 -- 172.21.15.42:0/2427461606 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff8cc057650 con 0x7ff8dc101ea0 2024-09-16T23:54:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.384+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff8dc02cc70 con 0x7ff8dc101ea0 2024-09-16T23:54:51.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.385+0000 7ff8d97fa700 1 -- 172.21.15.42:0/2427461606 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7ff8cc05ad10 con 0x7ff8dc101ea0 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:51.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:54:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:54:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:54:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:54:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-09-16T23:54:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 5 2024-09-16T23:54:51.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:54:51.384 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:54:51.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff8c806e680 msgr2=0x7ff8c8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff8c806e680 0x7ff8c8070b40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc1012f0 tx=0x7ff8c401a040 comp rx=0 tx=0).stop 2024-09-16T23:54:51.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc101ea0 msgr2=0x7ff8dc0ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc101ea0 0x7ff8dc0ffb00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00d8d0 tx=0x7ff8cc00dbe0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 shutdown_connections 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff8c806e680 0x7ff8c8070b40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8dc101ea0 0x7ff8dc0ffb00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 --2- 172.21.15.42:0/2427461606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8dc1027c0 0x7ff8dc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 >> 172.21.15.42:0/2427461606 conn(0x7ff8dc0fb650 msgr2=0x7ff8dc105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 shutdown_connections 2024-09-16T23:54:51.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.388+0000 7ff8e2990700 1 -- 172.21.15.42:0/2427461606 wait complete. 2024-09-16T23:54:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 -- 172.21.15.42:0/568011856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548068580 msgr2=0x7f75480689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 --2- 172.21.15.42:0/568011856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548068580 0x7f75480689e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f753c009910 tx=0x7f753c009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 -- 172.21.15.42:0/568011856 shutdown_connections 2024-09-16T23:54:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 --2- 172.21.15.42:0/568011856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548068580 0x7f75480689e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 --2- 172.21.15.42:0/568011856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7548106cd0 0x7f75481070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 -- 172.21.15.42:0/568011856 >> 172.21.15.42:0/568011856 conn(0x7f7548075670 msgr2=0x7f7548075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:51.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 -- 172.21.15.42:0/568011856 shutdown_connections 2024-09-16T23:54:51.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.507+0000 7f754e515700 1 -- 172.21.15.42:0/568011856 wait complete. 2024-09-16T23:54:51.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f754e515700 1 Processor -- start 2024-09-16T23:54:51.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f754e515700 1 -- start start 2024-09-16T23:54:51.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f754e515700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7548068580 0x7f754819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f754e515700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548106cd0 0x7f754819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f754e515700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754819d200 con 0x7f7548106cd0 2024-09-16T23:54:51.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f754e515700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75481966b0 con 0x7f7548068580 2024-09-16T23:54:51.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f75477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548106cd0 0x7f754819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f75477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548106cd0 0x7f754819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51820/0 (socket says 172.21.15.42:51820) 2024-09-16T23:54:51.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f75477fe700 1 -- 172.21.15.42:0/2473515215 learned_addr learned my addr 172.21.15.42:0/2473515215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.508+0000 7f7547fff700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7548068580 0x7f754819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f75477fe700 1 -- 172.21.15.42:0/2473515215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7548068580 msgr2=0x7f754819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f75477fe700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7548068580 0x7f754819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f75477fe700 1 -- 172.21.15.42:0/2473515215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0095f0 con 0x7f7548106cd0 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f75477fe700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548106cd0 0x7f754819cb70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f753c0092e0 tx=0x7f753c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f75457fa700 1 -- 172.21.15.42:0/2473515215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f753c01d070 con 0x7f7548106cd0 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f75457fa700 1 -- 172.21.15.42:0/2473515215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f753c005e00 con 0x7f7548106cd0 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7548196930 con 0x7f7548106cd0 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f75457fa700 1 -- 172.21.15.42:0/2473515215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f753c017400 con 0x7f7548106cd0 2024-09-16T23:54:51.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.509+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7548196e20 con 0x7f7548106cd0 2024-09-16T23:54:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.510+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75481091d0 con 0x7f7548106cd0 2024-09-16T23:54:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.511+0000 7f75457fa700 1 -- 172.21.15.42:0/2473515215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f753c00f6f0 con 0x7f7548106cd0 2024-09-16T23:54:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.511+0000 7f75457fa700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f753006e680 0x7f7530070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.512+0000 7f75457fa700 1 -- 172.21.15.42:0/2473515215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f753c08fa80 con 0x7f7548106cd0 2024-09-16T23:54:51.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.512+0000 7f7547fff700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f753006e680 0x7f7530070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.512+0000 7f7547fff700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f753006e680 0x7f7530070b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7538005950 tx=0x7f753800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:51.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.516+0000 7f75457fa700 1 -- 172.21.15.42:0/2473515215 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f753c05c800 con 0x7f7548106cd0 2024-09-16T23:54:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:51 smithi043 ceph-mon[81510]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:51 smithi043 ceph-mon[81510]: from='client.44179 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:51 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2427461606' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:51.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:51 smithi042 ceph-mon[111056]: from='client.34222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:51.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:51 smithi042 ceph-mon[111056]: from='client.44179 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:51.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:51.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:54:51.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:51.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:51.757 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:51 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2427461606' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.809+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7548197c90 con 0x7f7548106cd0 2024-09-16T23:54:51.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.810+0000 7f75457fa700 1 -- 172.21.15.42:0/2473515215 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1938 (secure 0 0 0) 0x7f753c05c390 con 0x7f7548106cd0 2024-09-16T23:54:51.807 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-09-16T23:54:51.807 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:53:20:175453+0000 2024-09-16T23:54:51.807 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:53:19.510146+0000 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:54:51.808 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:54:51.809 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 24269,14478 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:51.810 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:54:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f753006e680 msgr2=0x7f7530070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f753006e680 0x7f7530070b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7538005950 tx=0x7f753800b410 comp rx=0 tx=0).stop 2024-09-16T23:54:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548106cd0 msgr2=0x7f754819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548106cd0 0x7f754819cb70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f753c0092e0 tx=0x7f753c00bd70 comp rx=0 tx=0).stop 2024-09-16T23:54:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 shutdown_connections 2024-09-16T23:54:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f753006e680 0x7f7530070b40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7548068580 0x7f754819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 --2- 172.21.15.42:0/2473515215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7548106cd0 0x7f754819cb70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 >> 172.21.15.42:0/2473515215 conn(0x7f7548075670 msgr2=0x7f75480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 shutdown_connections 2024-09-16T23:54:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.813+0000 7f754e515700 1 -- 172.21.15.42:0/2473515215 wait complete. 2024-09-16T23:54:51.812 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-16T23:54:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 -- 172.21.15.42:0/2403644100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 msgr2=0x7f04dc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/2403644100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc1058a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009910 tx=0x7f04c8009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 -- 172.21.15.42:0/2403644100 shutdown_connections 2024-09-16T23:54:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/2403644100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc1058a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/2403644100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc100ea0 0x7f04dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 -- 172.21.15.42:0/2403644100 >> 172.21.15.42:0/2403644100 conn(0x7f04dc0fc750 msgr2=0x7f04dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:51.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 -- 172.21.15.42:0/2403644100 shutdown_connections 2024-09-16T23:54:51.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.930+0000 7f04e2d3f700 1 -- 172.21.15.42:0/2403644100 wait complete. 2024-09-16T23:54:51.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04e2d3f700 1 Processor -- start 2024-09-16T23:54:51.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04e2d3f700 1 -- start start 2024-09-16T23:54:51.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04e2d3f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc100ea0 0x7f04dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04e2d3f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04e2d3f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc073b10 con 0x7f04dc101850 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04e2d3f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc073c50 con 0x7f04dc100ea0 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51842/0 (socket says 172.21.15.42:51842) 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04dbfff700 1 -- 172.21.15.42:0/667936404 learned_addr learned my addr 172.21.15.42:0/667936404 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04e0adb700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc100ea0 0x7f04dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04dbfff700 1 -- 172.21.15.42:0/667936404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc100ea0 msgr2=0x7f04dc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04dbfff700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc100ea0 0x7f04dc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04dbfff700 1 -- 172.21.15.42:0/667936404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c80095f0 con 0x7f04dc101850 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.931+0000 7f04dbfff700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc0735d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04c80092e0 tx=0x7f04c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.932+0000 7f04d9ffb700 1 -- 172.21.15.42:0/667936404 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c801d070 con 0x7f04dc101850 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.932+0000 7f04d9ffb700 1 -- 172.21.15.42:0/667936404 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04c800f460 con 0x7f04dc101850 2024-09-16T23:54:51.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.932+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc073ed0 con 0x7f04dc101850 2024-09-16T23:54:51.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.932+0000 7f04d9ffb700 1 -- 172.21.15.42:0/667936404 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c80175d0 con 0x7f04dc101850 2024-09-16T23:54:51.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.932+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc1a2d70 con 0x7f04dc101850 2024-09-16T23:54:51.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.933+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc1092a0 con 0x7f04dc101850 2024-09-16T23:54:51.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.933+0000 7f04d9ffb700 1 -- 172.21.15.42:0/667936404 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f04c800f5d0 con 0x7f04dc101850 2024-09-16T23:54:51.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.934+0000 7f04d9ffb700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f04cc06e680 0x7f04cc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:51.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.934+0000 7f04d9ffb700 1 -- 172.21.15.42:0/667936404 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f04c808f990 con 0x7f04dc101850 2024-09-16T23:54:51.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.934+0000 7f04e0adb700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f04cc06e680 0x7f04cc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:51.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.935+0000 7f04e0adb700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f04cc06e680 0x7f04cc070b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005950 tx=0x7f04d000a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:51.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:51.938+0000 7f04d9ffb700 1 -- 172.21.15.42:0/667936404 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f04c8058c60 con 0x7f04dc101850 2024-09-16T23:54:52.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.170+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04dc0746d0 con 0x7f04cc06e680 2024-09-16T23:54:52.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.172+0000 7f04d9ffb700 1 -- 172.21.15.42:0/667936404 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f04dc0746d0 con 0x7f04cc06e680 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "8/23 daemons upgraded", 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:54:52.170 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:54:52.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f04cc06e680 msgr2=0x7f04cc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:52.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f04cc06e680 0x7f04cc070b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f04d0005950 tx=0x7f04d000a300 comp rx=0 tx=0).stop 2024-09-16T23:54:52.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 msgr2=0x7f04dc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:52.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc0735d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f04c80092e0 tx=0x7f04c800bd70 comp rx=0 tx=0).stop 2024-09-16T23:54:52.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 shutdown_connections 2024-09-16T23:54:52.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f04cc06e680 0x7f04cc070b40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc100ea0 0x7f04dc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 --2- 172.21.15.42:0/667936404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04dc101850 0x7f04dc0735d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 >> 172.21.15.42:0/667936404 conn(0x7f04dc0fc750 msgr2=0x7f04dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 shutdown_connections 2024-09-16T23:54:52.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.174+0000 7f04e2d3f700 1 -- 172.21.15.42:0/667936404 wait complete. 2024-09-16T23:54:52.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.288+0000 7f33a4833700 1 -- 172.21.15.42:0/3251461090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0106cd0 msgr2=0x7f33a01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:52.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.288+0000 7f33a4833700 1 --2- 172.21.15.42:0/3251461090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0106cd0 0x7f33a01070b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3390009910 tx=0x7f3390009c20 comp rx=0 tx=0).stop 2024-09-16T23:54:52.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.288+0000 7f33a4833700 1 -- 172.21.15.42:0/3251461090 shutdown_connections 2024-09-16T23:54:52.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.288+0000 7f33a4833700 1 --2- 172.21.15.42:0/3251461090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0068580 0x7f33a00689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.288+0000 7f33a4833700 1 --2- 172.21.15.42:0/3251461090 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0106cd0 0x7f33a01070b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.288+0000 7f33a4833700 1 -- 172.21.15.42:0/3251461090 >> 172.21.15.42:0/3251461090 conn(0x7f33a0075670 msgr2=0x7f33a0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:52.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.289+0000 7f33a4833700 1 -- 172.21.15.42:0/3251461090 shutdown_connections 2024-09-16T23:54:52.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.289+0000 7f33a4833700 1 -- 172.21.15.42:0/3251461090 wait complete. 2024-09-16T23:54:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f33a4833700 1 Processor -- start 2024-09-16T23:54:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f33a4833700 1 -- start start 2024-09-16T23:54:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f33a4833700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0068580 0x7f33a019c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f33a4833700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0106cd0 0x7f33a019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f33a4833700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a019d230 con 0x7f33a0068580 2024-09-16T23:54:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f33a4833700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01966e0 con 0x7f33a0106cd0 2024-09-16T23:54:52.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f339e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0068580 0x7f33a019c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f339e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0068580 0x7f33a019c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51852/0 (socket says 172.21.15.42:51852) 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f339e59c700 1 -- 172.21.15.42:0/3203278184 learned_addr learned my addr 172.21.15.42:0/3203278184 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.290+0000 7f339dd9b700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0106cd0 0x7f33a019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f339e59c700 1 -- 172.21.15.42:0/3203278184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0106cd0 msgr2=0x7f33a019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f339e59c700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0106cd0 0x7f33a019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f339e59c700 1 -- 172.21.15.42:0/3203278184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900095f0 con 0x7f33a0068580 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f339e59c700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0068580 0x7f33a019c660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f339000f690 tx=0x7f339000f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f338f7fe700 1 -- 172.21.15.42:0/3203278184 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339001d070 con 0x7f33a0068580 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a0196960 con 0x7f33a0068580 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f338f7fe700 1 -- 172.21.15.42:0/3203278184 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33900050d0 con 0x7f33a0068580 2024-09-16T23:54:52.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f338f7fe700 1 -- 172.21.15.42:0/3203278184 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3390022470 con 0x7f33a0068580 2024-09-16T23:54:52.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.291+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a0196e50 con 0x7f33a0068580 2024-09-16T23:54:52.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.292+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a01091f0 con 0x7f33a0068580 2024-09-16T23:54:52.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.293+0000 7f338f7fe700 1 -- 172.21.15.42:0/3203278184 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3390003780 con 0x7f33a0068580 2024-09-16T23:54:52.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.293+0000 7f338f7fe700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f338806e680 0x7f3388070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:54:52.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.293+0000 7f338f7fe700 1 -- 172.21.15.42:0/3203278184 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f339008f950 con 0x7f33a0068580 2024-09-16T23:54:52.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.293+0000 7f339dd9b700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f338806e680 0x7f3388070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:54:52.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.294+0000 7f339dd9b700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f338806e680 0x7f3388070b40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f33a0197b50 tx=0x7f3394009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:54:52.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.298+0000 7f338f7fe700 1 -- 172.21.15.42:0/3203278184 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f339005c800 con 0x7f33a0068580 2024-09-16T23:54:52.634 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:52 smithi042 ceph-mon[111056]: pgmap v70: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:52.635 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:52 smithi042 ceph-mon[111056]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:52.635 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:52 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2473515215' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:54:52.635 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:52 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[127990]: 2024-09-16T23:54:52.432+0000 7f47716f7640 -1 osd.1 0 read_superblock omap replica is missing. 2024-09-16T23:54:52.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.636+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f33a0197c20 con 0x7f33a0068580 2024-09-16T23:54:52.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.637+0000 7f338f7fe700 1 -- 172.21.15.42:0/3203278184 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1350 (secure 0 0 0) 0x7f339005c390 con 0x7f33a0068580 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 58/369 objects degraded (15.718%), 16 pgs degraded 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: osd.1 (root=default,host=smithi042) is down 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 58/369 objects degraded (15.718%), 16 pgs degraded 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-09-16T23:54:52.636 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-09-16T23:54:52.637 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f338806e680 msgr2=0x7f3388070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f338806e680 0x7f3388070b40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f33a0197b50 tx=0x7f3394009260 comp rx=0 tx=0).stop 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0068580 msgr2=0x7f33a019c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0068580 0x7f33a019c660 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f339000f690 tx=0x7f339000f6c0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 shutdown_connections 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f338806e680 0x7f3388070b40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f33a0068580 0x7f33a019c660 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 --2- 172.21.15.42:0/3203278184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33a0106cd0 0x7f33a019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 >> 172.21.15.42:0/3203278184 conn(0x7f33a0075670 msgr2=0x7f33a00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 shutdown_connections 2024-09-16T23:54:52.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:54:52.639+0000 7f33a4833700 1 -- 172.21.15.42:0/3203278184 wait complete. 2024-09-16T23:54:52.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:52 smithi043 ceph-mon[81510]: pgmap v70: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:52.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:52 smithi043 ceph-mon[81510]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:52.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:52 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2473515215' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:54:52.977 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:52 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[127990]: 2024-09-16T23:54:52.781+0000 7f47716f7640 -1 osd.1 54 log_to_monitors true 2024-09-16T23:54:53.493 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 16 23:54:53 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[127990]: 2024-09-16T23:54:53.479+0000 7f4769491640 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:54:53.493 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:53 smithi042 ceph-mon[111056]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:53.493 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:53 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3203278184' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:54:53.494 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:53 smithi042 ceph-mon[111056]: from='osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T23:54:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:53 smithi043 ceph-mon[81510]: from='client.34242 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:54:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:53 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3203278184' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:54:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:53 smithi043 ceph-mon[81510]: from='osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T23:54:54.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:54 smithi042 ceph-mon[111056]: pgmap v71: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:54.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:54 smithi042 ceph-mon[111056]: from='osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T23:54:54.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:54 smithi042 ceph-mon[111056]: osdmap e57: 6 total, 5 up, 6 in 2024-09-16T23:54:54.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:54 smithi042 ceph-mon[111056]: from='osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:54:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:54 smithi043 ceph-mon[81510]: pgmap v71: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:54 smithi043 ceph-mon[81510]: from='osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T23:54:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:54 smithi043 ceph-mon[81510]: osdmap e57: 6 total, 5 up, 6 in 2024-09-16T23:54:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:54 smithi043 ceph-mon[81510]: from='osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:54:55.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:54:55.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151] boot 2024-09-16T23:54:55.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: osdmap e58: 6 total, 6 up, 6 in 2024-09-16T23:54:55.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:54:55.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:55.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:55.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:55.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:54:55.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:55.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:55.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:55.776 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:55.776 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:55.776 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:54:55.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:54:55.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: osd.1 [v2:172.21.15.42:6810/3124187151,v1:172.21.15.42:6811/3124187151] boot 2024-09-16T23:54:55.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: osdmap e58: 6 total, 6 up, 6 in 2024-09-16T23:54:55.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:54:55.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:54:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:56 smithi042 ceph-mon[111056]: pgmap v74: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:56 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:54:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:56 smithi042 ceph-mon[111056]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-16T23:54:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:56 smithi042 ceph-mon[111056]: osdmap e59: 6 total, 6 up, 6 in 2024-09-16T23:54:56.802 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:56 smithi043 ceph-mon[81510]: pgmap v74: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:54:56.802 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:56 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:54:56.802 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:56 smithi043 ceph-mon[81510]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-16T23:54:56.802 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:56 smithi043 ceph-mon[81510]: osdmap e59: 6 total, 6 up, 6 in 2024-09-16T23:54:57.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:57 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 25/369 objects degraded (6.775%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T23:54:57.914 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:57 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 25/369 objects degraded (6.775%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T23:54:58.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:58 smithi043 ceph-mon[81510]: pgmap v76: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25/369 objects degraded (6.775%) 2024-09-16T23:54:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:58 smithi042 ceph-mon[111056]: pgmap v76: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25/369 objects degraded (6.775%) 2024-09-16T23:54:59.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:54:59 smithi043 ceph-mon[81510]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/369 objects degraded (6.775%), 7 pgs degraded) 2024-09-16T23:55:00.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:54:59 smithi042 ceph-mon[111056]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 25/369 objects degraded (6.775%), 7 pgs degraded) 2024-09-16T23:55:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:00 smithi043 ceph-mon[81510]: pgmap v77: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:01.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:00 smithi042 ceph-mon[111056]: pgmap v77: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:02.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:02 smithi043 ceph-mon[81510]: pgmap v78: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:02.969 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:02 smithi042 ceph-mon[111056]: pgmap v78: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:04.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:04 smithi043 ceph-mon[81510]: pgmap v79: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:05.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:04 smithi042 ceph-mon[111056]: pgmap v79: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:06 smithi043 ceph-mon[81510]: pgmap v80: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:07.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:06 smithi042 ceph-mon[111056]: pgmap v80: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:07.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:07.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:07.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:08 smithi043 ceph-mon[81510]: pgmap v81: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:08 smithi042 ceph-mon[111056]: pgmap v81: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:10.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:10 smithi042 ceph-mon[111056]: pgmap v82: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:10.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:10 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:55:10.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:10 smithi043 ceph-mon[81510]: pgmap v82: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:10.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:10 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: Upgrade: osd.2 is safe to restart 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: pgmap v83: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: Upgrade: Updating osd.2 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:55:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:11 smithi042 ceph-mon[111056]: Deploying daemon osd.2 on smithi042 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: Upgrade: osd.2 is safe to restart 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: pgmap v83: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: Upgrade: Updating osd.2 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:55:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:11 smithi043 ceph-mon[81510]: Deploying daemon osd.2 on smithi042 2024-09-16T23:55:14.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:13 smithi042 ceph-mon[111056]: pgmap v84: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:13 smithi043 ceph-mon[81510]: pgmap v84: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:15.160 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:14 smithi042 systemd[1]: Stopping Ceph osd.2 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:55:15.523 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:15 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[61727]: 2024-09-16T23:55:15.162+0000 7f5082fcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:55:15.523 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:15 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[61727]: 2024-09-16T23:55:15.162+0000 7f5082fcb700 -1 osd.2 59 *** Got signal Terminated *** 2024-09-16T23:55:15.523 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:15 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[61727]: 2024-09-16T23:55:15.162+0000 7f5082fcb700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T23:55:16.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:15 smithi042 ceph-mon[111056]: pgmap v85: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:16.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:15 smithi042 ceph-mon[111056]: osd.2 marked itself down and dead 2024-09-16T23:55:16.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:15 smithi043 ceph-mon[81510]: pgmap v85: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:16.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:15 smithi043 ceph-mon[81510]: osd.2 marked itself down and dead 2024-09-16T23:55:16.353 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:16 smithi042 podman[133361]: 2024-09-16 23:55:16.076676151 +0000 UTC m=+1.027668200 container died 3d6cb9cb77ab60439b3ccb197275004f16a4a8ec51f3a2fa7fef93787869445f (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-09-16T23:55:16.661 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:16 smithi042 podman[133361]: 2024-09-16 23:55:16.454442458 +0000 UTC m=+1.405434497 container cleanup 3d6cb9cb77ab60439b3ccb197275004f16a4a8ec51f3a2fa7fef93787869445f (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.1, ceph=True, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-09-16T23:55:16.661 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:16 smithi042 bash[133361]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2 2024-09-16T23:55:17.004 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:16 smithi043 ceph-mon[81510]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:55:17.004 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:16 smithi043 ceph-mon[81510]: osdmap e60: 6 total, 5 up, 6 in 2024-09-16T23:55:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:16 smithi042 ceph-mon[111056]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:55:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:16 smithi042 ceph-mon[111056]: osdmap e60: 6 total, 5 up, 6 in 2024-09-16T23:55:17.023 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:16 smithi042 podman[133380]: 2024-09-16 23:55:16.705913398 +0000 UTC m=+0.625097911 container remove 3d6cb9cb77ab60439b3ccb197275004f16a4a8ec51f3a2fa7fef93787869445f (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD) 2024-09-16T23:55:17.315 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:17 smithi042 podman[133466]: 2024-09-16 23:55:17.040088821 +0000 UTC m=+0.111576792 container create 5081184220b873c1fe75ddb520b15563d43ea3973108aa55caaf8bfbc1ae4933 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97) 2024-09-16T23:55:17.315 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:17 smithi042 podman[133466]: 2024-09-16 23:55:16.944971895 +0000 UTC m=+0.016459875 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:55:17.315 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:17 smithi042 podman[133466]: 2024-09-16 23:55:17.3182265 +0000 UTC m=+0.389714484 container init 5081184220b873c1fe75ddb520b15563d43ea3973108aa55caaf8bfbc1ae4933 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T23:55:17.662 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:17 smithi042 podman[133466]: 2024-09-16 23:55:17.321885874 +0000 UTC m=+0.393373846 container start 5081184220b873c1fe75ddb520b15563d43ea3973108aa55caaf8bfbc1ae4933 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T23:55:17.663 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:17 smithi042 podman[133466]: 2024-09-16 23:55:17.40115031 +0000 UTC m=+0.472638284 container attach 5081184220b873c1fe75ddb520b15563d43ea3973108aa55caaf8bfbc1ae4933 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-c2ddf97) 2024-09-16T23:55:17.663 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:17 smithi042 podman[133466]: 2024-09-16 23:55:17.522394817 +0000 UTC m=+0.593882799 container died 5081184220b873c1fe75ddb520b15563d43ea3973108aa55caaf8bfbc1ae4933 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-c2ddf97, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:55:18.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:17 smithi042 ceph-mon[111056]: pgmap v87: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:17 smithi042 ceph-mon[111056]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T23:55:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:17 smithi042 ceph-mon[111056]: osdmap e61: 6 total, 5 up, 6 in 2024-09-16T23:55:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:17 smithi043 ceph-mon[81510]: pgmap v87: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:17 smithi043 ceph-mon[81510]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T23:55:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:17 smithi043 ceph-mon[81510]: osdmap e61: 6 total, 5 up, 6 in 2024-09-16T23:55:18.715 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:18 smithi042 podman[133488]: 2024-09-16 23:55:18.380678142 +0000 UTC m=+0.852062718 container remove 5081184220b873c1fe75ddb520b15563d43ea3973108aa55caaf8bfbc1ae4933 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T23:55:18.715 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:18 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-16T23:55:18.715 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:18 smithi042 systemd[1]: Stopped Ceph osd.2 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:55:18.715 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:18 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.2.service: Consumed 46.623s CPU time. 2024-09-16T23:55:18.715 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:18 smithi042 systemd[1]: Starting Ceph osd.2 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:55:18.979 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:18 smithi042 ceph-mon[111056]: Health check failed: Degraded data redundancy: 13/369 objects degraded (3.523%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:18.979 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:18 smithi042 podman[133588]: 2024-09-16 23:55:18.744888323 +0000 UTC m=+0.015921346 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:55:18.979 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:18 smithi042 podman[133588]: 2024-09-16 23:55:18.873322752 +0000 UTC m=+0.144355766 container create d017e5b6086573325e1e9a4c3bfac5d79f36a9f12519d9b7e0f67e1b915ffc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T23:55:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:18 smithi043 ceph-mon[81510]: Health check failed: Degraded data redundancy: 13/369 objects degraded (3.523%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:19.255 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 podman[133588]: 2024-09-16 23:55:19.120080799 +0000 UTC m=+0.391113820 container init d017e5b6086573325e1e9a4c3bfac5d79f36a9f12519d9b7e0f67e1b915ffc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:55:19.255 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 podman[133588]: 2024-09-16 23:55:19.123621807 +0000 UTC m=+0.394654829 container start d017e5b6086573325e1e9a4c3bfac5d79f36a9f12519d9b7e0f67e1b915ffc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T23:55:19.255 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 podman[133588]: 2024-09-16 23:55:19.217647848 +0000 UTC m=+0.488680864 container attach d017e5b6086573325e1e9a4c3bfac5d79f36a9f12519d9b7e0f67e1b915ffc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97) 2024-09-16T23:55:19.523 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:19.523 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 bash[133588]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:19.523 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:19.523 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 bash[133588]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:19.993 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:19 smithi042 ceph-mon[111056]: pgmap v89: 65 pgs: 4 active+undersized, 20 peering, 3 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 13/369 objects degraded (3.523%) 2024-09-16T23:55:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:19 smithi043 ceph-mon[81510]: pgmap v89: 65 pgs: 4 active+undersized, 20 peering, 3 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 13/369 objects degraded (3.523%) 2024-09-16T23:55:20.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T23:55:20.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 bash[133588]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T23:55:20.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T23:55:20.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:19 smithi042 bash[133588]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T23:55:20.686 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T23:55:20.686 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 bash[133588]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T23:55:20.686 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 bash[133588]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 bash[133588]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 bash[133588]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate[133602]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 bash[133588]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T23:55:20.687 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:20 smithi042 podman[133588]: 2024-09-16 23:55:20.318765611 +0000 UTC m=+1.589798635 container died d017e5b6086573325e1e9a4c3bfac5d79f36a9f12519d9b7e0f67e1b915ffc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T23:55:21.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 podman[133737]: 2024-09-16 23:55:21.005403693 +0000 UTC m=+0.680220615 container remove d017e5b6086573325e1e9a4c3bfac5d79f36a9f12519d9b7e0f67e1b915ffc35 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T23:55:21.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 podman[133799]: 2024-09-16 23:55:21.121562112 +0000 UTC m=+0.016830419 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:55:21.556 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 podman[133799]: 2024-09-16 23:55:21.289882258 +0000 UTC m=+0.185150555 container create 61db938df01dc87fb13df3e944225595a7c662a190bcd9640346027d2c568ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T23:55:21.556 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 podman[133799]: 2024-09-16 23:55:21.559270985 +0000 UTC m=+0.454539294 container init 61db938df01dc87fb13df3e944225595a7c662a190bcd9640346027d2c568ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T23:55:21.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:21 smithi043 ceph-mon[81510]: pgmap v90: 65 pgs: 4 active+undersized, 20 peering, 3 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 13/369 objects degraded (3.523%) 2024-09-16T23:55:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:21 smithi042 ceph-mon[111056]: pgmap v90: 65 pgs: 4 active+undersized, 20 peering, 3 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 13/369 objects degraded (3.523%) 2024-09-16T23:55:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:22.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:22.023 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 podman[133799]: 2024-09-16 23:55:21.56290305 +0000 UTC m=+0.458171354 container start 61db938df01dc87fb13df3e944225595a7c662a190bcd9640346027d2c568ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T23:55:22.024 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 ceph-osd[133817]: -- 172.21.15.42:0/3695071565 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602ada8c1e0 con 0x5602ada22000 2024-09-16T23:55:22.024 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 bash[133799]: 61db938df01dc87fb13df3e944225595a7c662a190bcd9640346027d2c568ff8 2024-09-16T23:55:22.024 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:21 smithi042 systemd[1]: Started Ceph osd.2 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:55:22.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 -- 172.21.15.42:0/2362814891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 msgr2=0x7f35f80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 --2- 172.21.15.42:0/2362814891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f80689e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009910 tx=0x7f35e0009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 -- 172.21.15.42:0/2362814891 shutdown_connections 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 --2- 172.21.15.42:0/2362814891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f80689e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 --2- 172.21.15.42:0/2362814891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f8106cd0 0x7f35f81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 -- 172.21.15.42:0/2362814891 >> 172.21.15.42:0/2362814891 conn(0x7f35f8075670 msgr2=0x7f35f8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 -- 172.21.15.42:0/2362814891 shutdown_connections 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.759+0000 7f35fd848700 1 -- 172.21.15.42:0/2362814891 wait complete. 2024-09-16T23:55:22.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.760+0000 7f35fd848700 1 Processor -- start 2024-09-16T23:55:22.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.760+0000 7f35fd848700 1 -- start start 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fd848700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fd848700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f8106cd0 0x7f35f819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fd848700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f819d200 con 0x7f35f8068580 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fd848700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f81966b0 con 0x7f35f8106cd0 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34386/0 (socket says 172.21.15.42:34386) 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f6ffd700 1 -- 172.21.15.42:0/3676578044 learned_addr learned my addr 172.21.15.42:0/3676578044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f67fc700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f8106cd0 0x7f35f819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f6ffd700 1 -- 172.21.15.42:0/3676578044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f8106cd0 msgr2=0x7f35f819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f6ffd700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f8106cd0 0x7f35f819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f6ffd700 1 -- 172.21.15.42:0/3676578044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e00095f0 con 0x7f35f8068580 2024-09-16T23:55:22.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35f6ffd700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f819c630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35f00060b0 tx=0x7f35f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:22.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fc846700 1 -- 172.21.15.42:0/3676578044 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f000dda0 con 0x7f35f8068580 2024-09-16T23:55:22.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f8196990 con 0x7f35f8068580 2024-09-16T23:55:22.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fc846700 1 -- 172.21.15.42:0/3676578044 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35f000f040 con 0x7f35f8068580 2024-09-16T23:55:22.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.761+0000 7f35fc846700 1 -- 172.21.15.42:0/3676578044 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f0014790 con 0x7f35f8068580 2024-09-16T23:55:22.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.762+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f8196ee0 con 0x7f35f8068580 2024-09-16T23:55:22.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.762+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f81091d0 con 0x7f35f8068580 2024-09-16T23:55:22.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.763+0000 7f35fc846700 1 -- 172.21.15.42:0/3676578044 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f35f0014980 con 0x7f35f8068580 2024-09-16T23:55:22.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.763+0000 7f35fc846700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f35e406e680 0x7f35e4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:22.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.764+0000 7f35fc846700 1 -- 172.21.15.42:0/3676578044 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f35f008e050 con 0x7f35f8068580 2024-09-16T23:55:22.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.764+0000 7f35f67fc700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f35e406e680 0x7f35e4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:22.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.764+0000 7f35f67fc700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f35e406e680 0x7f35e4070b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f35f80fe200 tx=0x7f35e001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:22.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:22.768+0000 7f35fc846700 1 -- 172.21.15.42:0/3676578044 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f35f005ae80 con 0x7f35f8068580 2024-09-16T23:55:23.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.009+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35f8197160 con 0x7f35e406e680 2024-09-16T23:55:23.007 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:23.007 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:23.008 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:55:23.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.011+0000 7f35fc846700 1 -- 172.21.15.42:0/3676578044 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f35f8197160 con 0x7f35e406e680 2024-09-16T23:55:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f35e406e680 msgr2=0x7f35e4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f35e406e680 0x7f35e4070b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f35f80fe200 tx=0x7f35e001a040 comp rx=0 tx=0).stop 2024-09-16T23:55:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 msgr2=0x7f35f819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f819c630 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35f00060b0 tx=0x7f35f000d3b0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 shutdown_connections 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f35e406e680 0x7f35e4070b40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35f8068580 0x7f35f819c630 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 --2- 172.21.15.42:0/3676578044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35f8106cd0 0x7f35f819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 >> 172.21.15.42:0/3676578044 conn(0x7f35f8075670 msgr2=0x7f35f80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 shutdown_connections 2024-09-16T23:55:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.013+0000 7f35fd848700 1 -- 172.21.15.42:0/3676578044 wait complete. 2024-09-16T23:55:23.021 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:55:23.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.124+0000 7f9c33de3700 1 -- 172.21.15.42:0/820609288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c105bd0 msgr2=0x7f9c2c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.124+0000 7f9c33de3700 1 --2- 172.21.15.42:0/820609288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c105bd0 0x7f9c2c105fb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c28009910 tx=0x7f9c28009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:23.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.124+0000 7f9c33de3700 1 -- 172.21.15.42:0/820609288 shutdown_connections 2024-09-16T23:55:23.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.124+0000 7f9c33de3700 1 --2- 172.21.15.42:0/820609288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c2c0ffc20 0x7f9c2c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.124+0000 7f9c33de3700 1 --2- 172.21.15.42:0/820609288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c105bd0 0x7f9c2c105fb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.124+0000 7f9c33de3700 1 -- 172.21.15.42:0/820609288 >> 172.21.15.42:0/820609288 conn(0x7f9c2c075670 msgr2=0x7f9c2c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:23.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.124+0000 7f9c33de3700 1 -- 172.21.15.42:0/820609288 shutdown_connections 2024-09-16T23:55:23.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.125+0000 7f9c33de3700 1 -- 172.21.15.42:0/820609288 wait complete. 2024-09-16T23:55:23.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.125+0000 7f9c33de3700 1 Processor -- start 2024-09-16T23:55:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.125+0000 7f9c33de3700 1 -- start start 2024-09-16T23:55:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.125+0000 7f9c33de3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c0ffc20 0x7f9c2c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.125+0000 7f9c33de3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c2c105bd0 0x7f9c2c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.125+0000 7f9c33de3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c198290 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.125+0000 7f9c33de3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1983d0 con 0x7f9c2c105bd0 2024-09-16T23:55:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c31b7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c0ffc20 0x7f9c2c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c31b7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c0ffc20 0x7f9c2c197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34394/0 (socket says 172.21.15.42:34394) 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c31b7f700 1 -- 172.21.15.42:0/405146085 learned_addr learned my addr 172.21.15.42:0/405146085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c3137e700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c2c105bd0 0x7f9c2c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c31b7f700 1 -- 172.21.15.42:0/405146085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c2c105bd0 msgr2=0x7f9c2c197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c31b7f700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c2c105bd0 0x7f9c2c197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c31b7f700 1 -- 172.21.15.42:0/405146085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c280095f0 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c31b7f700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c0ffc20 0x7f9c2c197670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c28006010 tx=0x7f9c2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c1effd700 1 -- 172.21.15.42:0/405146085 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c2801d070 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c1effd700 1 -- 172.21.15.42:0/405146085 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c2800f460 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c1a06e0 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c1effd700 1 -- 172.21.15.42:0/405146085 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c28017630 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.126+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c1a0ba0 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.127+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c1080f0 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.128+0000 7f9c1effd700 1 -- 172.21.15.42:0/405146085 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9c2800f5d0 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:23.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:23.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:55:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.129+0000 7f9c1effd700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c1806e680 0x7f9c18070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.129+0000 7f9c1effd700 1 -- 172.21.15.42:0/405146085 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9c2808fa50 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.129+0000 7f9c3137e700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c1806e680 0x7f9c18070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.129+0000 7f9c3137e700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c1806e680 0x7f9c18070b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005950 tx=0x7f9c200058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:23.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.133+0000 7f9c1effd700 1 -- 172.21.15.42:0/405146085 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9c28058ca0 con 0x7f9c2c0ffc20 2024-09-16T23:55:23.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:23 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[133813]: 2024-09-16T23:55:23.197+0000 7f16004b5640 -1 Falling back to public interface 2024-09-16T23:55:23.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.385+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c2c0611d0 con 0x7f9c1806e680 2024-09-16T23:55:23.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.386+0000 7f9c1effd700 1 -- 172.21.15.42:0/405146085 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9c2c0611d0 con 0x7f9c1806e680 2024-09-16T23:55:23.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.388+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c1806e680 msgr2=0x7f9c18070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.388+0000 7f9c33de3700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c1806e680 0x7f9c18070b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005950 tx=0x7f9c200058e0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.388+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c0ffc20 msgr2=0x7f9c2c197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.388+0000 7f9c33de3700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c0ffc20 0x7f9c2c197670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c28006010 tx=0x7f9c2800be20 comp rx=0 tx=0).stop 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.389+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 shutdown_connections 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.389+0000 7f9c33de3700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c1806e680 0x7f9c18070b40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.389+0000 7f9c33de3700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c2c0ffc20 0x7f9c2c197670 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.389+0000 7f9c33de3700 1 --2- 172.21.15.42:0/405146085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c2c105bd0 0x7f9c2c197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.389+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 >> 172.21.15.42:0/405146085 conn(0x7f9c2c075670 msgr2=0x7f9c2c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.389+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 shutdown_connections 2024-09-16T23:55:23.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.389+0000 7f9c33de3700 1 -- 172.21.15.42:0/405146085 wait complete. 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.503+0000 7f28d833b700 1 -- 172.21.15.42:0/2232225545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 msgr2=0x7f28d01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.503+0000 7f28d833b700 1 --2- 172.21.15.42:0/2232225545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d01070b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009910 tx=0x7f28cc009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.503+0000 7f28d833b700 1 -- 172.21.15.42:0/2232225545 shutdown_connections 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.503+0000 7f28d833b700 1 --2- 172.21.15.42:0/2232225545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d0068580 0x7f28d00689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.503+0000 7f28d833b700 1 --2- 172.21.15.42:0/2232225545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d01070b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.503+0000 7f28d833b700 1 -- 172.21.15.42:0/2232225545 >> 172.21.15.42:0/2232225545 conn(0x7f28d0075670 msgr2=0x7f28d0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.503+0000 7f28d833b700 1 -- 172.21.15.42:0/2232225545 shutdown_connections 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.504+0000 7f28d833b700 1 -- 172.21.15.42:0/2232225545 wait complete. 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.504+0000 7f28d833b700 1 Processor -- start 2024-09-16T23:55:23.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.504+0000 7f28d833b700 1 -- start start 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d833b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d0068580 0x7f28d0193570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d833b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d0193ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d833b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0197700 con 0x7f28d0106cd0 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d833b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0193ff0 con 0x7f28d0068580 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d58d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d0193ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d58d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d0193ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34416/0 (socket says 172.21.15.42:34416) 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d58d6700 1 -- 172.21.15.42:0/2259203240 learned_addr learned my addr 172.21.15.42:0/2259203240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d60d7700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d0068580 0x7f28d0193570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d58d6700 1 -- 172.21.15.42:0/2259203240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d0068580 msgr2=0x7f28d0193570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d58d6700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d0068580 0x7f28d0193570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d58d6700 1 -- 172.21.15.42:0/2259203240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0095f0 con 0x7f28d0106cd0 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d60d7700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d0068580 0x7f28d0193570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.505+0000 7f28d58d6700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d0193ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d900 tx=0x7f28c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.506+0000 7f28c77fe700 1 -- 172.21.15.42:0/2259203240 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c00041f0 con 0x7f28d0106cd0 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.506+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d01942d0 con 0x7f28d0106cd0 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.506+0000 7f28c77fe700 1 -- 172.21.15.42:0/2259203240 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28c0004cf0 con 0x7f28d0106cd0 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.506+0000 7f28c77fe700 1 -- 172.21.15.42:0/2259203240 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c000b750 con 0x7f28d0106cd0 2024-09-16T23:55:23.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.506+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d01a72a0 con 0x7f28d0106cd0 2024-09-16T23:55:23.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.507+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d004f2e0 con 0x7f28d0106cd0 2024-09-16T23:55:23.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.508+0000 7f28c77fe700 1 -- 172.21.15.42:0/2259203240 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f28c0004350 con 0x7f28d0106cd0 2024-09-16T23:55:23.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.508+0000 7f28c77fe700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f28bc06e680 0x7f28bc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.508+0000 7f28c77fe700 1 -- 172.21.15.42:0/2259203240 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f28c008e720 con 0x7f28d0106cd0 2024-09-16T23:55:23.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.508+0000 7f28d60d7700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f28bc06e680 0x7f28bc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.509+0000 7f28d60d7700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f28bc06e680 0x7f28bc070b40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005b40 tx=0x7f28cc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:23.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.513+0000 7f28c77fe700 1 -- 172.21.15.42:0/2259203240 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f28c0057970 con 0x7f28d0106cd0 2024-09-16T23:55:23.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.742+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f28d00ff190 con 0x7f28bc06e680 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.752+0000 7f28c77fe700 1 -- 172.21.15.42:0/2259203240 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f28d00ff190 con 0x7f28bc06e680 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (4m) 33s ago 14m 24.7M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (15m) 33s ago 15m 8296k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (13m) 91s ago 13m 7964k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (99s) 33s ago 15m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (94s) 91s ago 13m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (3m) 33s ago 14m 76.6M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (9m) 33s ago 9m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:55:23.750 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (9m) 33s ago 9m 16.6M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (9m) 91s ago 9m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (9m) 91s ago 9m 17.2M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (5m) 33s ago 16m 498M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (4m) 91s ago 13m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (2m) 33s ago 16m 49.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (2m) 91s ago 13m 35.5M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (4m) 33s ago 14m 17.0M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (4m) 91s ago 13m 15.4M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (74s) 33s ago 12m 83.3M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (39s) 33s ago 12m 11.2M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 starting - - - 1217M 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (11m) 91s ago 11m 235M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (11m) 91s ago 11m 222M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:55:23.751 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (10m) 91s ago 10m 202M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:55:23.752 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (4m) 33s ago 14m 73.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f28bc06e680 msgr2=0x7f28bc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f28bc06e680 0x7f28bc070b40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005b40 tx=0x7f28cc01a040 comp rx=0 tx=0).stop 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 msgr2=0x7f28d0193ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d0193ab0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f28c000d900 tx=0x7f28c000dcc0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 shutdown_connections 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f28bc06e680 0x7f28bc070b40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d0068580 0x7f28d0193570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 --2- 172.21.15.42:0/2259203240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28d0106cd0 0x7f28d0193ab0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 >> 172.21.15.42:0/2259203240 conn(0x7f28d0075670 msgr2=0x7f28d00fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 shutdown_connections 2024-09-16T23:55:23.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.754+0000 7f28d833b700 1 -- 172.21.15.42:0/2259203240 wait complete. 2024-09-16T23:55:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/1870860240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 msgr2=0x7f9e480ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/1870860240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e480ff980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009910 tx=0x7f9e3c009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/1870860240 shutdown_connections 2024-09-16T23:55:23.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/1870860240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e480fff50 0x7f9e48105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/1870860240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e480ff980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/1870860240 >> 172.21.15.42:0/1870860240 conn(0x7f9e48074e30 msgr2=0x7f9e48075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/1870860240 shutdown_connections 2024-09-16T23:55:23.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.871+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/1870860240 wait complete. 2024-09-16T23:55:23.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:23 smithi042 ceph-mon[111056]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:23.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:23 smithi042 ceph-mon[111056]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-09-16T23:55:23.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:23 smithi042 ceph-mon[111056]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:23.870 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:23 smithi042 ceph-mon[111056]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4ebf1700 1 Processor -- start 2024-09-16T23:55:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4ebf1700 1 -- start start 2024-09-16T23:55:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4ebf1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e4819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4ebf1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e480fff50 0x7f9e4819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4ebf1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4819d250 con 0x7f9e480ff5a0 2024-09-16T23:55:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4ebf1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e48196700 con 0x7f9e480fff50 2024-09-16T23:55:23.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4c98d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e4819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4c98d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e4819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34442/0 (socket says 172.21.15.42:34442) 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e4c98d700 1 -- 172.21.15.42:0/2256503188 learned_addr learned my addr 172.21.15.42:0/2256503188 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.872+0000 7f9e47fff700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e480fff50 0x7f9e4819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e4c98d700 1 -- 172.21.15.42:0/2256503188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e480fff50 msgr2=0x7f9e4819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e4c98d700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e480fff50 0x7f9e4819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e4c98d700 1 -- 172.21.15.42:0/2256503188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0095f0 con 0x7f9e480ff5a0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e4c98d700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e4819c680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c006010 tx=0x7f9e3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e45ffb700 1 -- 172.21.15.42:0/2256503188 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e3c01d070 con 0x7f9e480ff5a0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e48196980 con 0x7f9e480ff5a0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e45ffb700 1 -- 172.21.15.42:0/2256503188 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e3c00f460 con 0x7f9e480ff5a0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e45ffb700 1 -- 172.21.15.42:0/2256503188 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e3c017630 con 0x7f9e480ff5a0 2024-09-16T23:55:23.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.873+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e48196e70 con 0x7f9e480ff5a0 2024-09-16T23:55:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.875+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e48109270 con 0x7f9e480ff5a0 2024-09-16T23:55:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.875+0000 7f9e45ffb700 1 -- 172.21.15.42:0/2256503188 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9e3c00f5d0 con 0x7f9e480ff5a0 2024-09-16T23:55:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.876+0000 7f9e45ffb700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e3806e680 0x7f9e38070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.876+0000 7f9e45ffb700 1 -- 172.21.15.42:0/2256503188 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9e3c08fa70 con 0x7f9e480ff5a0 2024-09-16T23:55:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.876+0000 7f9e47fff700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e3806e680 0x7f9e38070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:23.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.877+0000 7f9e47fff700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e3806e680 0x7f9e38070b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005950 tx=0x7f9e340058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:23.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:23.881+0000 7f9e45ffb700 1 -- 172.21.15.42:0/2256503188 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9e3c05c7f0 con 0x7f9e480ff5a0 2024-09-16T23:55:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:23 smithi043 ceph-mon[81510]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:23 smithi043 ceph-mon[81510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 5 pgs peering) 2024-09-16T23:55:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:23 smithi043 ceph-mon[81510]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:23 smithi043 ceph-mon[81510]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:24.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.193+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9e48197c40 con 0x7f9e480ff5a0 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.193+0000 7f9e45ffb700 1 -- 172.21.15.42:0/2256503188 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f9e3c05c380 con 0x7f9e480ff5a0 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:55:24.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:55:24.192 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e3806e680 msgr2=0x7f9e38070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e3806e680 0x7f9e38070b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9e34005950 tx=0x7f9e340058e0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 msgr2=0x7f9e4819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e4819c680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c006010 tx=0x7f9e3c00be20 comp rx=0 tx=0).stop 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 shutdown_connections 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e3806e680 0x7f9e38070b40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e480ff5a0 0x7f9e4819c680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 --2- 172.21.15.42:0/2256503188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e480fff50 0x7f9e4819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 >> 172.21.15.42:0/2256503188 conn(0x7f9e48074e30 msgr2=0x7f9e480fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 shutdown_connections 2024-09-16T23:55:24.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.196+0000 7f9e4ebf1700 1 -- 172.21.15.42:0/2256503188 wait complete. 2024-09-16T23:55:24.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 -- 172.21.15.42:0/2972932451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9564105bd0 msgr2=0x7f9564105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 --2- 172.21.15.42:0/2972932451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9564105bd0 0x7f9564105fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9554009910 tx=0x7f9554009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:24.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 -- 172.21.15.42:0/2972932451 shutdown_connections 2024-09-16T23:55:24.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 --2- 172.21.15.42:0/2972932451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95640ffc20 0x7f9564100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 --2- 172.21.15.42:0/2972932451 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9564105bd0 0x7f9564105fb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 -- 172.21.15.42:0/2972932451 >> 172.21.15.42:0/2972932451 conn(0x7f9564075670 msgr2=0x7f9564075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 -- 172.21.15.42:0/2972932451 shutdown_connections 2024-09-16T23:55:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.314+0000 7f9569b72700 1 -- 172.21.15.42:0/2972932451 wait complete. 2024-09-16T23:55:24.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f9569b72700 1 Processor -- start 2024-09-16T23:55:24.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f9569b72700 1 -- start start 2024-09-16T23:55:24.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f9569b72700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95640ffc20 0x7f956419e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:24.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f9569b72700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9564105bd0 0x7f956419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:24.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f9569b72700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95641989c0 con 0x7f95640ffc20 2024-09-16T23:55:24.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f9569b72700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564198b30 con 0x7f9564105bd0 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f95637fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95640ffc20 0x7f956419e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f95637fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95640ffc20 0x7f956419e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34458/0 (socket says 172.21.15.42:34458) 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f95637fe700 1 -- 172.21.15.42:0/4130578119 learned_addr learned my addr 172.21.15.42:0/4130578119 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.315+0000 7f9562ffd700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9564105bd0 0x7f956419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f95637fe700 1 -- 172.21.15.42:0/4130578119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9564105bd0 msgr2=0x7f956419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f95637fe700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9564105bd0 0x7f956419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f95637fe700 1 -- 172.21.15.42:0/4130578119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95540095f0 con 0x7f95640ffc20 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f95637fe700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95640ffc20 0x7f956419e9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9554006010 tx=0x7f955400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f9560ff9700 1 -- 172.21.15.42:0/4130578119 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955401d070 con 0x7f95640ffc20 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f9560ff9700 1 -- 172.21.15.42:0/4130578119 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f955400f460 con 0x7f95640ffc20 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9564198db0 con 0x7f95640ffc20 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f9560ff9700 1 -- 172.21.15.42:0/4130578119 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9554017630 con 0x7f95640ffc20 2024-09-16T23:55:24.315 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.316+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95641992a0 con 0x7f95640ffc20 2024-09-16T23:55:24.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.317+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f956404f2e0 con 0x7f95640ffc20 2024-09-16T23:55:24.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.318+0000 7f9560ff9700 1 -- 172.21.15.42:0/4130578119 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f955400f5d0 con 0x7f95640ffc20 2024-09-16T23:55:24.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.318+0000 7f9560ff9700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f954c06e680 0x7f954c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:24.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.319+0000 7f9562ffd700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f954c06e680 0x7f954c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:24.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.319+0000 7f9560ff9700 1 -- 172.21.15.42:0/4130578119 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f955408fb00 con 0x7f95640ffc20 2024-09-16T23:55:24.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.319+0000 7f9562ffd700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f954c06e680 0x7f954c070b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f956419a1f0 tx=0x7f9558009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:24.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.323+0000 7f9560ff9700 1 -- 172.21.15.42:0/4130578119 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9554058d50 con 0x7f95640ffc20 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.580+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9564199580 con 0x7f95640ffc20 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.581+0000 7f9560ff9700 1 -- 172.21.15.42:0/4130578119 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1938 (secure 0 0 0) 0x7f955405c410 con 0x7f95640ffc20 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:53:20:175453+0000 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:55:24.579 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:53:19.510146+0000 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:55:24.580 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 24269,14478 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:24.581 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f954c06e680 msgr2=0x7f954c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f954c06e680 0x7f954c070b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f956419a1f0 tx=0x7f9558009190 comp rx=0 tx=0).stop 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95640ffc20 msgr2=0x7f956419e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95640ffc20 0x7f956419e9d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9554006010 tx=0x7f955400bd70 comp rx=0 tx=0).stop 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 shutdown_connections 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f954c06e680 0x7f954c070b40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95640ffc20 0x7f956419e9d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 --2- 172.21.15.42:0/4130578119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9564105bd0 0x7f956419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 >> 172.21.15.42:0/4130578119 conn(0x7f9564075670 msgr2=0x7f9564068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 shutdown_connections 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.583+0000 7f9569b72700 1 -- 172.21.15.42:0/4130578119 wait complete. 2024-09-16T23:55:24.583 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-16T23:55:24.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.711+0000 7f99e0973700 1 -- 172.21.15.42:0/1070608637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 msgr2=0x7f99dc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.711+0000 7f99e0973700 1 --2- 172.21.15.42:0/1070608637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc1058a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009910 tx=0x7f99c4009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:24.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.711+0000 7f99e0973700 1 -- 172.21.15.42:0/1070608637 shutdown_connections 2024-09-16T23:55:24.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.711+0000 7f99e0973700 1 --2- 172.21.15.42:0/1070608637 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc1058a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.711+0000 7f99e0973700 1 --2- 172.21.15.42:0/1070608637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc100ea0 0x7f99dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.711+0000 7f99e0973700 1 -- 172.21.15.42:0/1070608637 >> 172.21.15.42:0/1070608637 conn(0x7f99dc0fc750 msgr2=0x7f99dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:24.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.711+0000 7f99e0973700 1 -- 172.21.15.42:0/1070608637 shutdown_connections 2024-09-16T23:55:24.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.712+0000 7f99e0973700 1 -- 172.21.15.42:0/1070608637 wait complete. 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.712+0000 7f99e0973700 1 Processor -- start 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99e0973700 1 -- start start 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99e0973700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc100ea0 0x7f99dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99e0973700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99e0973700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc073b10 con 0x7f99dc101850 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99e0973700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc073c50 con 0x7f99dc100ea0 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99d1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99d1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34478/0 (socket says 172.21.15.42:34478) 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99d1d9b700 1 -- 172.21.15.42:0/1655904324 learned_addr learned my addr 172.21.15.42:0/1655904324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:24.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.713+0000 7f99da59c700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc100ea0 0x7f99dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:24.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99d1d9b700 1 -- 172.21.15.42:0/1655904324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc100ea0 msgr2=0x7f99dc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99d1d9b700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc100ea0 0x7f99dc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99d1d9b700 1 -- 172.21.15.42:0/1655904324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c40095f0 con 0x7f99dc101850 2024-09-16T23:55:24.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99d1d9b700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc0735d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99c40059d0 tx=0x7f99c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:24.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99d3fff700 1 -- 172.21.15.42:0/1655904324 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c401d070 con 0x7f99dc101850 2024-09-16T23:55:24.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc073ed0 con 0x7f99dc101850 2024-09-16T23:55:24.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc1a2e30 con 0x7f99dc101850 2024-09-16T23:55:24.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.714+0000 7f99d3fff700 1 -- 172.21.15.42:0/1655904324 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99c400b610 con 0x7f99dc101850 2024-09-16T23:55:24.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.715+0000 7f99d3fff700 1 -- 172.21.15.42:0/1655904324 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c4022470 con 0x7f99dc101850 2024-09-16T23:55:24.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.715+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc1092a0 con 0x7f99dc101850 2024-09-16T23:55:24.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.716+0000 7f99d3fff700 1 -- 172.21.15.42:0/1655904324 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f99c4003710 con 0x7f99dc101850 2024-09-16T23:55:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.717+0000 7f99d3fff700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f99c806e680 0x7f99c8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.717+0000 7f99d3fff700 1 -- 172.21.15.42:0/1655904324 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f99c408fa80 con 0x7f99dc101850 2024-09-16T23:55:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.718+0000 7f99da59c700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f99c806e680 0x7f99c8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.718+0000 7f99da59c700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f99c806e680 0x7f99c8070b40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00f520 tx=0x7f99cc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.722+0000 7f99d3fff700 1 -- 172.21.15.42:0/1655904324 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f99c4058c30 con 0x7f99dc101850 2024-09-16T23:55:24.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.956+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99dc0746d0 con 0x7f99c806e680 2024-09-16T23:55:24.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.957+0000 7f99d3fff700 1 -- 172.21.15.42:0/1655904324 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f99dc0746d0 con 0x7f99c806e680 2024-09-16T23:55:24.955 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:55:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:55:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:55:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:55:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:55:24.955 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:55:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:55:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:55:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:55:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "8/23 daemons upgraded", 2024-09-16T23:55:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T23:55:24.956 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:55:24.956 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f99c806e680 msgr2=0x7f99c8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f99c806e680 0x7f99c8070b40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00f520 tx=0x7f99cc00b040 comp rx=0 tx=0).stop 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 msgr2=0x7f99dc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc0735d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f99c40059d0 tx=0x7f99c400f690 comp rx=0 tx=0).stop 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 shutdown_connections 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f99c806e680 0x7f99c8070b40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99dc100ea0 0x7f99dc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 --2- 172.21.15.42:0/1655904324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99dc101850 0x7f99dc0735d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:24.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 >> 172.21.15.42:0/1655904324 conn(0x7f99dc0fc750 msgr2=0x7f99dc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:24.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 shutdown_connections 2024-09-16T23:55:24.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:24.960+0000 7f99e0973700 1 -- 172.21.15.42:0/1655904324 wait complete. 2024-09-16T23:55:25.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:24 smithi042 ceph-mon[111056]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:25.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:24 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 45/369 objects degraded (12.195%), 12 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:25.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:24 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2256503188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:25.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:24 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4130578119' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 -- 172.21.15.42:0/3199532105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 msgr2=0x7f47c8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 --2- 172.21.15.42:0/3199532105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c8101280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47b8009910 tx=0x7f47b8009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 -- 172.21.15.42:0/3199532105 shutdown_connections 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 --2- 172.21.15.42:0/3199532105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c8101850 0x7f47c81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 --2- 172.21.15.42:0/3199532105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c8101280 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 -- 172.21.15.42:0/3199532105 >> 172.21.15.42:0/3199532105 conn(0x7f47c80fc750 msgr2=0x7f47c80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 -- 172.21.15.42:0/3199532105 shutdown_connections 2024-09-16T23:55:25.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.070+0000 7f47cd212700 1 -- 172.21.15.42:0/3199532105 wait complete. 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47cd212700 1 Processor -- start 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47cd212700 1 -- start start 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47cd212700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47cd212700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c8101850 0x7f47c819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47cd212700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c819d2c0 con 0x7f47c8100ea0 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47cd212700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8196770 con 0x7f47c8101850 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47c6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47c6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34494/0 (socket says 172.21.15.42:34494) 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47c6d9d700 1 -- 172.21.15.42:0/2109242829 learned_addr learned my addr 172.21.15.42:0/2109242829 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:25.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47c659c700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c8101850 0x7f47c819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47c6d9d700 1 -- 172.21.15.42:0/2109242829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c8101850 msgr2=0x7f47c819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47c6d9d700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c8101850 0x7f47c819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.071+0000 7f47c6d9d700 1 -- 172.21.15.42:0/2109242829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b80095f0 con 0x7f47c8100ea0 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.072+0000 7f47c6d9d700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c819c6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006010 tx=0x7f47b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.072+0000 7f47affff700 1 -- 172.21.15.42:0/2109242829 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b801d070 con 0x7f47c8100ea0 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.072+0000 7f47affff700 1 -- 172.21.15.42:0/2109242829 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47b800f460 con 0x7f47c8100ea0 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.072+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c81969f0 con 0x7f47c8100ea0 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.072+0000 7f47affff700 1 -- 172.21.15.42:0/2109242829 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47b8017630 con 0x7f47c8100ea0 2024-09-16T23:55:25.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.072+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c8196ee0 con 0x7f47c8100ea0 2024-09-16T23:55:25.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.073+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c8109240 con 0x7f47c8100ea0 2024-09-16T23:55:25.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.073+0000 7f47affff700 1 -- 172.21.15.42:0/2109242829 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f47b800f5d0 con 0x7f47c8100ea0 2024-09-16T23:55:25.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.074+0000 7f47affff700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b006e680 0x7f47b0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:25.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.074+0000 7f47affff700 1 -- 172.21.15.42:0/2109242829 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f47b808f9d0 con 0x7f47c8100ea0 2024-09-16T23:55:25.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.074+0000 7f47c659c700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b006e680 0x7f47b0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:25.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.075+0000 7f47c659c700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b006e680 0x7f47b0070b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005950 tx=0x7f47bc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.079+0000 7f47affff700 1 -- 172.21.15.42:0/2109242829 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f47b805c7d0 con 0x7f47c8100ea0 2024-09-16T23:55:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:24 smithi043 ceph-mon[81510]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:24 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 45/369 objects degraded (12.195%), 12 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:24 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2256503188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:24 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4130578119' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:55:25.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.379+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f47c804f2e0 con 0x7f47c8100ea0 2024-09-16T23:55:25.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.379+0000 7f47affff700 1 -- 172.21.15.42:0/2109242829 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1131 (secure 0 0 0) 0x7f47b805c360 con 0x7f47c8100ea0 2024-09-16T23:55:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 45/369 objects degraded (12.195%), 12 pgs degraded 2024-09-16T23:55:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:55:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:55:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T23:55:25.377 INFO:teuthology.orchestra.run.smithi042.stdout: osd.2 (root=default,host=smithi042) is down 2024-09-16T23:55:25.377 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 45/369 objects degraded (12.195%), 12 pgs degraded 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.17 is active+undersized+degraded, acting [5,1] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-09-16T23:55:25.378 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-09-16T23:55:25.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b006e680 msgr2=0x7f47b0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b006e680 0x7f47b0070b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005950 tx=0x7f47bc0058e0 comp rx=0 tx=0).stop 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 msgr2=0x7f47c819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c819c6f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47b8006010 tx=0x7f47b800be20 comp rx=0 tx=0).stop 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 shutdown_connections 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b006e680 0x7f47b0070b40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8100ea0 0x7f47c819c6f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 --2- 172.21.15.42:0/2109242829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c8101850 0x7f47c819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 >> 172.21.15.42:0/2109242829 conn(0x7f47c80fc750 msgr2=0x7f47c80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 shutdown_connections 2024-09-16T23:55:25.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:25.383+0000 7f47cd212700 1 -- 172.21.15.42:0/2109242829 wait complete. 2024-09-16T23:55:25.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:25 smithi042 ceph-mon[111056]: pgmap v92: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:25.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:25 smithi042 ceph-mon[111056]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:25.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:25 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2109242829' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:55:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:25 smithi043 ceph-mon[81510]: pgmap v92: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:25 smithi043 ceph-mon[81510]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:25 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2109242829' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:55:28.282 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:28 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[133813]: 2024-09-16T23:55:28.181+0000 7f16004b5640 -1 osd.2 0 read_superblock omap replica is missing. 2024-09-16T23:55:28.282 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:28 smithi042 ceph-mon[111056]: pgmap v93: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:28.282 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.282 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.282 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.282 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:28 smithi043 ceph-mon[81510]: pgmap v93: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:28.773 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:28 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[133813]: 2024-09-16T23:55:28.431+0000 7f16004b5640 -1 osd.2 59 log_to_monitors true 2024-09-16T23:55:29.323 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:29 smithi042 ceph-mon[111056]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T23:55:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:29 smithi043 ceph-mon[81510]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T23:55:30.023 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 16 23:55:29 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[133813]: 2024-09-16T23:55:29.743+0000 7f15f4247640 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:55:30.289 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:30 smithi042 ceph-mon[111056]: pgmap v94: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:30.289 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:30 smithi042 ceph-mon[111056]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T23:55:30.289 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:30 smithi042 ceph-mon[111056]: osdmap e62: 6 total, 5 up, 6 in 2024-09-16T23:55:30.289 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:30 smithi042 ceph-mon[111056]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:55:30.290 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:30 smithi042 ceph-mon[111056]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' 2024-09-16T23:55:30.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:30 smithi043 ceph-mon[81510]: pgmap v94: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:30.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:30 smithi043 ceph-mon[81510]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T23:55:30.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:30 smithi043 ceph-mon[81510]: osdmap e62: 6 total, 5 up, 6 in 2024-09-16T23:55:30.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:30 smithi043 ceph-mon[81510]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-09-16T23:55:30.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:30 smithi043 ceph-mon[81510]: from='osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771]' entity='osd.2' 2024-09-16T23:55:31.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:55:31.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771] boot 2024-09-16T23:55:31.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: osdmap e63: 6 total, 6 up, 6 in 2024-09-16T23:55:31.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:55:31.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:31.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:31.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:55:31.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:55:31.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: osd.2 [v2:172.21.15.42:6818/2970204771,v1:172.21.15.42:6819/2970204771] boot 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: osdmap e63: 6 total, 6 up, 6 in 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:55:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: pgmap v96: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T23:55:32.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:32 smithi043 ceph-mon[81510]: osdmap e64: 6 total, 6 up, 6 in 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: pgmap v96: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/369 objects degraded (12.195%) 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T23:55:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:32 smithi042 ceph-mon[111056]: osdmap e64: 6 total, 6 up, 6 in 2024-09-16T23:55:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:33 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 18/369 objects degraded (4.878%), 5 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:33.526 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:33 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 18/369 objects degraded (4.878%), 5 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:34 smithi043 ceph-mon[81510]: pgmap v99: 65 pgs: 15 peering, 7 active+undersized, 5 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18/369 objects degraded (4.878%) 2024-09-16T23:55:34.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:34 smithi042 ceph-mon[111056]: pgmap v99: 65 pgs: 15 peering, 7 active+undersized, 5 active+undersized+degraded, 38 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18/369 objects degraded (4.878%) 2024-09-16T23:55:35.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:35 smithi042 ceph-mon[111056]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/369 objects degraded (4.878%), 5 pgs degraded) 2024-09-16T23:55:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:35 smithi043 ceph-mon[81510]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/369 objects degraded (4.878%), 5 pgs degraded) 2024-09-16T23:55:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:36 smithi042 ceph-mon[111056]: pgmap v100: 65 pgs: 15 peering, 50 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:36 smithi043 ceph-mon[81510]: pgmap v100: 65 pgs: 15 peering, 50 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:38 smithi042 ceph-mon[111056]: pgmap v101: 65 pgs: 5 peering, 60 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:38 smithi043 ceph-mon[81510]: pgmap v101: 65 pgs: 5 peering, 60 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:40.430 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:40 smithi043 ceph-mon[81510]: pgmap v102: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:40.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:40 smithi042 ceph-mon[111056]: pgmap v102: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:42 smithi042 ceph-mon[111056]: pgmap v103: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:42 smithi043 ceph-mon[81510]: pgmap v103: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:44.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:44 smithi042 ceph-mon[111056]: pgmap v104: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:44 smithi043 ceph-mon[81510]: pgmap v104: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:46.505 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:46 smithi043 ceph-mon[81510]: pgmap v105: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:46.505 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:46.505 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:46 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:46.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:46 smithi042 ceph-mon[111056]: pgmap v105: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:46.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:46.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:46 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:47.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:47 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:47.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:47 smithi042 ceph-mon[111056]: Upgrade: osd.3 is safe to restart 2024-09-16T23:55:47.543 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:47 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T23:55:47.544 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:47 smithi043 ceph-mon[81510]: Upgrade: osd.3 is safe to restart 2024-09-16T23:55:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:48 smithi042 ceph-mon[111056]: pgmap v106: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:48 smithi042 ceph-mon[111056]: Upgrade: Updating osd.3 2024-09-16T23:55:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:48 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:48 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T23:55:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:48 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:55:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:48 smithi042 ceph-mon[111056]: Deploying daemon osd.3 on smithi043 2024-09-16T23:55:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:48 smithi043 ceph-mon[81510]: pgmap v106: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:48 smithi043 ceph-mon[81510]: Upgrade: Updating osd.3 2024-09-16T23:55:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:48 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:48 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T23:55:48.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:48 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:55:48.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:48 smithi043 ceph-mon[81510]: Deploying daemon osd.3 on smithi043 2024-09-16T23:55:50.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:50 smithi043 ceph-mon[81510]: pgmap v107: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:50.998 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:50 smithi043 systemd[1]: Stopping Ceph osd.3 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:55:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:50 smithi042 ceph-mon[111056]: pgmap v107: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:51.378 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:51 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[41071]: 2024-09-16T23:55:51.019+0000 7f0d5daef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:55:51.378 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:51 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[41071]: 2024-09-16T23:55:51.019+0000 7f0d5daef700 -1 osd.3 64 *** Got signal Terminated *** 2024-09-16T23:55:51.378 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:51 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[41071]: 2024-09-16T23:55:51.019+0000 7f0d5daef700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T23:55:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:51 smithi043 ceph-mon[81510]: osd.3 marked itself down and dead 2024-09-16T23:55:52.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:55:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:51 smithi042 ceph-mon[111056]: osd.3 marked itself down and dead 2024-09-16T23:55:52.353 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:52 smithi043 podman[90544]: 2024-09-16 23:55:52.098335142 +0000 UTC m=+1.196610892 container died f5dba7ca4af5f0e863a7557255b9d85ec33b17a248f4081e6812a86c761b1af8 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T23:55:52.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:52 smithi043 ceph-mon[81510]: pgmap v108: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:52.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:52 smithi043 ceph-mon[81510]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:55:52.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:52 smithi043 ceph-mon[81510]: osdmap e65: 6 total, 5 up, 6 in 2024-09-16T23:55:52.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:52 smithi042 ceph-mon[111056]: pgmap v108: 65 pgs: 65 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:52.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:52 smithi042 ceph-mon[111056]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:55:52.924 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:52 smithi042 ceph-mon[111056]: osdmap e65: 6 total, 5 up, 6 in 2024-09-16T23:55:53.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:52 smithi043 podman[90544]: 2024-09-16 23:55:52.636843811 +0000 UTC m=+1.735119550 container cleanup f5dba7ca4af5f0e863a7557255b9d85ec33b17a248f4081e6812a86c761b1af8 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, ceph=True, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T23:55:53.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:52 smithi043 bash[90544]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3 2024-09-16T23:55:53.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:52 smithi043 podman[90570]: 2024-09-16 23:55:52.838442612 +0000 UTC m=+0.737905182 container remove f5dba7ca4af5f0e863a7557255b9d85ec33b17a248f4081e6812a86c761b1af8 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, RELEASE=HEAD) 2024-09-16T23:55:53.513 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:53 smithi043 podman[90661]: 2024-09-16 23:55:53.158891958 +0000 UTC m=+0.016261255 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:55:53.514 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:53 smithi043 podman[90661]: 2024-09-16 23:55:53.289437477 +0000 UTC m=+0.146806807 container create 9a32a1becbca80943eb826b08c8d872abfc1c32c157bf4f4a15e5ac838a1be55 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-16T23:55:53.514 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:53 smithi043 podman[90661]: 2024-09-16 23:55:53.51690764 +0000 UTC m=+0.374276937 container init 9a32a1becbca80943eb826b08c8d872abfc1c32c157bf4f4a15e5ac838a1be55 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T23:55:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:53 smithi043 ceph-mon[81510]: osdmap e66: 6 total, 5 up, 6 in 2024-09-16T23:55:53.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:53 smithi043 podman[90661]: 2024-09-16 23:55:53.520356379 +0000 UTC m=+0.377725672 container start 9a32a1becbca80943eb826b08c8d872abfc1c32c157bf4f4a15e5ac838a1be55 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T23:55:53.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:53 smithi043 podman[90661]: 2024-09-16 23:55:53.600015826 +0000 UTC m=+0.457385137 container attach 9a32a1becbca80943eb826b08c8d872abfc1c32c157bf4f4a15e5ac838a1be55 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:55:53.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:53 smithi043 podman[90661]: 2024-09-16 23:55:53.712754495 +0000 UTC m=+0.570123796 container died 9a32a1becbca80943eb826b08c8d872abfc1c32c157bf4f4a15e5ac838a1be55 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T23:55:54.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:53 smithi042 ceph-mon[111056]: osdmap e66: 6 total, 5 up, 6 in 2024-09-16T23:55:54.876 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:54 smithi043 ceph-mon[81510]: pgmap v111: 65 pgs: 25 peering, 3 stale+active+clean, 37 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:54.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:54 smithi043 ceph-mon[81510]: Health check failed: Reduced data availability: 13 pgs peering (PG_AVAILABILITY) 2024-09-16T23:55:54.877 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:54 smithi043 podman[90685]: 2024-09-16 23:55:54.420548976 +0000 UTC m=+0.701561445 container remove 9a32a1becbca80943eb826b08c8d872abfc1c32c157bf4f4a15e5ac838a1be55 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T23:55:54.877 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:54 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-16T23:55:54.877 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:54 smithi043 systemd[1]: Stopped Ceph osd.3 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:55:54.877 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:54 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.3.service: Consumed 1min 6.926s CPU time. 2024-09-16T23:55:54.877 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:54 smithi043 systemd[1]: Starting Ceph osd.3 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:55:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:54 smithi042 ceph-mon[111056]: pgmap v111: 65 pgs: 25 peering, 3 stale+active+clean, 37 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:55:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:54 smithi042 ceph-mon[111056]: Health check failed: Reduced data availability: 13 pgs peering (PG_AVAILABILITY) 2024-09-16T23:55:55.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:54 smithi043 podman[90783]: 2024-09-16 23:55:54.782568653 +0000 UTC m=+0.017003392 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:55:55.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:54 smithi043 podman[90783]: 2024-09-16 23:55:54.905557729 +0000 UTC m=+0.139992510 container create 30c747cf81dc52e128f01e3cb56438fcdf9edcb5e980f16cf03777f55725a385 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate, RELEASE=squid-c2ddf97, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T23:55:55.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 -- 172.21.15.42:0/854958598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 msgr2=0x7f3b94073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/854958598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b94073490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009910 tx=0x7f3b88009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 -- 172.21.15.42:0/854958598 shutdown_connections 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/854958598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b940739d0 0x7f3b9410cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/854958598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b94073490 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 -- 172.21.15.42:0/854958598 >> 172.21.15.42:0/854958598 conn(0x7f3b940fc750 msgr2=0x7f3b940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 -- 172.21.15.42:0/854958598 shutdown_connections 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.508+0000 7f3b9b08a700 1 -- 172.21.15.42:0/854958598 wait complete. 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b9b08a700 1 Processor -- start 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b9b08a700 1 -- start start 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b9b08a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b9419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:55.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b9b08a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b940739d0 0x7f3b9419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b9b08a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9419f410 con 0x7f3b940730b0 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b9b08a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b941988c0 con 0x7f3b940739d0 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b98e26700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b9419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b98e26700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b9419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38708/0 (socket says 172.21.15.42:38708) 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b98e26700 1 -- 172.21.15.42:0/273111142 learned_addr learned my addr 172.21.15.42:0/273111142 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.509+0000 7f3b93fff700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b940739d0 0x7f3b9419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b98e26700 1 -- 172.21.15.42:0/273111142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b940739d0 msgr2=0x7f3b9419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:55.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b98e26700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b940739d0 0x7f3b9419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b98e26700 1 -- 172.21.15.42:0/273111142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b880095f0 con 0x7f3b940730b0 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b93fff700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b940739d0 0x7f3b9419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b98e26700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b9419e840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005f50 tx=0x7f3b8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b91ffb700 1 -- 172.21.15.42:0/273111142 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8801d070 con 0x7f3b940730b0 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b91ffb700 1 -- 172.21.15.42:0/273111142 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b8800b610 con 0x7f3b940730b0 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b94198b40 con 0x7f3b940730b0 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b91ffb700 1 -- 172.21.15.42:0/273111142 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b88022470 con 0x7f3b940730b0 2024-09-16T23:55:55.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.510+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b94199030 con 0x7f3b940730b0 2024-09-16T23:55:55.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.511+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9410a6c0 con 0x7f3b940730b0 2024-09-16T23:55:55.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.512+0000 7f3b91ffb700 1 -- 172.21.15.42:0/273111142 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3b88003780 con 0x7f3b940730b0 2024-09-16T23:55:55.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.512+0000 7f3b91ffb700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b8406e680 0x7f3b84070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:55.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.513+0000 7f3b91ffb700 1 -- 172.21.15.42:0/273111142 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3b8808f920 con 0x7f3b940730b0 2024-09-16T23:55:55.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.513+0000 7f3b93fff700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b8406e680 0x7f3b84070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:55.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.513+0000 7f3b93fff700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b8406e680 0x7f3b84070b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b940fde90 tx=0x7f3b80009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:55.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.517+0000 7f3b91ffb700 1 -- 172.21.15.42:0/273111142 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3b8805c680 con 0x7f3b940730b0 2024-09-16T23:55:55.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:55 smithi043 podman[90783]: 2024-09-16 23:55:55.149524859 +0000 UTC m=+0.383959598 container init 30c747cf81dc52e128f01e3cb56438fcdf9edcb5e980f16cf03777f55725a385 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97) 2024-09-16T23:55:55.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:55 smithi043 podman[90783]: 2024-09-16 23:55:55.153274848 +0000 UTC m=+0.387709597 container start 30c747cf81dc52e128f01e3cb56438fcdf9edcb5e980f16cf03777f55725a385 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS) 2024-09-16T23:55:55.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:55 smithi043 podman[90783]: 2024-09-16 23:55:55.241138682 +0000 UTC m=+0.475573422 container attach 30c747cf81dc52e128f01e3cb56438fcdf9edcb5e980f16cf03777f55725a385 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T23:55:55.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:55 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:55.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:55 smithi043 bash[90783]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:55.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:55 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:55.629 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:55 smithi043 bash[90783]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:55:55.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.750+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b94199e40 con 0x7f3b8406e680 2024-09-16T23:55:55.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.751+0000 7f3b91ffb700 1 -- 172.21.15.42:0/273111142 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3b94199e40 con 0x7f3b8406e680 2024-09-16T23:55:55.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.753+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b8406e680 msgr2=0x7f3b84070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:55.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.753+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b8406e680 0x7f3b84070b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3b940fde90 tx=0x7f3b80009260 comp rx=0 tx=0).stop 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.753+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 msgr2=0x7f3b9419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.753+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b9419e840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005f50 tx=0x7f3b8800f690 comp rx=0 tx=0).stop 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.754+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 shutdown_connections 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.754+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b8406e680 0x7f3b84070b40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.754+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b940730b0 0x7f3b9419e840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.754+0000 7f3b9b08a700 1 --2- 172.21.15.42:0/273111142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b940739d0 0x7f3b9419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.754+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 >> 172.21.15.42:0/273111142 conn(0x7f3b940fc750 msgr2=0x7f3b94107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.754+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 shutdown_connections 2024-09-16T23:55:55.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.754+0000 7f3b9b08a700 1 -- 172.21.15.42:0/273111142 wait complete. 2024-09-16T23:55:55.764 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:55:55.764 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:55 smithi042 ceph-mon[111056]: pgmap v112: 65 pgs: 6 active+undersized, 25 peering, 7 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23/369 objects degraded (6.233%) 2024-09-16T23:55:55.764 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:55 smithi042 ceph-mon[111056]: Health check failed: Degraded data redundancy: 23/369 objects degraded (6.233%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:55.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 -- 172.21.15.42:0/853253056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0100ea0 msgr2=0x7f36a0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:55.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/853253056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0100ea0 0x7f36a0101280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f369c009910 tx=0x7f369c009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:55.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 -- 172.21.15.42:0/853253056 shutdown_connections 2024-09-16T23:55:55.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/853253056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a0101850 0x7f36a01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/853253056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0100ea0 0x7f36a0101280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 -- 172.21.15.42:0/853253056 >> 172.21.15.42:0/853253056 conn(0x7f36a00fc750 msgr2=0x7f36a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:55.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 -- 172.21.15.42:0/853253056 shutdown_connections 2024-09-16T23:55:55.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.865+0000 7f36a7ff4700 1 -- 172.21.15.42:0/853253056 wait complete. 2024-09-16T23:55:55.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a7ff4700 1 Processor -- start 2024-09-16T23:55:55.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a7ff4700 1 -- start start 2024-09-16T23:55:55.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a7ff4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a0100ea0 0x7f36a019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a7ff4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0101850 0x7f36a019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a7ff4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a019d2c0 con 0x7f36a0101850 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a7ff4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0196770 con 0x7f36a0100ea0 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a558f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0101850 0x7f36a019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a558f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0101850 0x7f36a019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38718/0 (socket says 172.21.15.42:38718) 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.866+0000 7f36a558f700 1 -- 172.21.15.42:0/3729585338 learned_addr learned my addr 172.21.15.42:0/3729585338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a5d90700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a0100ea0 0x7f36a019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a558f700 1 -- 172.21.15.42:0/3729585338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a0100ea0 msgr2=0x7f36a019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a558f700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a0100ea0 0x7f36a019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a558f700 1 -- 172.21.15.42:0/3729585338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0095f0 con 0x7f36a0101850 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a5d90700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a0100ea0 0x7f36a019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:55:55.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a558f700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0101850 0x7f36a019cc30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f369400e8f0 tx=0x7f369400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:55.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f3692ffd700 1 -- 172.21.15.42:0/3729585338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3694009790 con 0x7f36a0101850 2024-09-16T23:55:55.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f3692ffd700 1 -- 172.21.15.42:0/3729585338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3694004cf0 con 0x7f36a0101850 2024-09-16T23:55:55.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a0196a50 con 0x7f36a0101850 2024-09-16T23:55:55.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f3692ffd700 1 -- 172.21.15.42:0/3729585338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3694005690 con 0x7f36a0101850 2024-09-16T23:55:55.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.867+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a0196fa0 con 0x7f36a0101850 2024-09-16T23:55:55.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.868+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a0109240 con 0x7f36a0101850 2024-09-16T23:55:55.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.869+0000 7f3692ffd700 1 -- 172.21.15.42:0/3729585338 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3694004750 con 0x7f36a0101850 2024-09-16T23:55:55.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.869+0000 7f3692ffd700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f368c06e680 0x7f368c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:55.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.869+0000 7f3692ffd700 1 -- 172.21.15.42:0/3729585338 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3694014070 con 0x7f36a0101850 2024-09-16T23:55:55.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.869+0000 7f36a5d90700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f368c06e680 0x7f368c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:55.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.870+0000 7f36a5d90700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f368c06e680 0x7f368c070b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f369c000c00 tx=0x7f369c009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:55.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:55.873+0000 7f3692ffd700 1 -- 172.21.15.42:0/3729585338 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f369405b750 con 0x7f36a0101850 2024-09-16T23:55:56.024 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:55 smithi043 ceph-mon[81510]: pgmap v112: 65 pgs: 6 active+undersized, 25 peering, 7 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 23/369 objects degraded (6.233%) 2024-09-16T23:55:56.024 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:55 smithi043 ceph-mon[81510]: Health check failed: Degraded data redundancy: 23/369 objects degraded (6.233%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T23:55:56.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.105+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36a0197cf0 con 0x7f368c06e680 2024-09-16T23:55:56.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.106+0000 7f3692ffd700 1 -- 172.21.15.42:0/3729585338 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f36a0197cf0 con 0x7f368c06e680 2024-09-16T23:55:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f368c06e680 msgr2=0x7f368c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f368c06e680 0x7f368c070b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f369c000c00 tx=0x7f369c009f70 comp rx=0 tx=0).stop 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0101850 msgr2=0x7f36a019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0101850 0x7f36a019cc30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f369400e8f0 tx=0x7f369400ecb0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 shutdown_connections 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f368c06e680 0x7f368c070b40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36a0100ea0 0x7f36a019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 --2- 172.21.15.42:0/3729585338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36a0101850 0x7f36a019cc30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 >> 172.21.15.42:0/3729585338 conn(0x7f36a00fc750 msgr2=0x7f36a00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 shutdown_connections 2024-09-16T23:55:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.108+0000 7f36a7ff4700 1 -- 172.21.15.42:0/3729585338 wait complete. 2024-09-16T23:55:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1624449548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 msgr2=0x7fe1b010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1624449548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b010ac10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009910 tx=0x7fe1a4009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1624449548 shutdown_connections 2024-09-16T23:55:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1624449548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b010ac10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1624449548 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b0101e00 0x7fe1b01021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1624449548 >> 172.21.15.42:0/1624449548 conn(0x7fe1b00fb650 msgr2=0x7fe1b00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:56.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1624449548 shutdown_connections 2024-09-16T23:55:56.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.216+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1624449548 wait complete. 2024-09-16T23:55:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1b4efc700 1 Processor -- start 2024-09-16T23:55:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1b4efc700 1 -- start start 2024-09-16T23:55:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1b4efc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b0101e00 0x7fe1b019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1b4efc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1b4efc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b019d260 con 0x7fe1b0102720 2024-09-16T23:55:56.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1b4efc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b0196710 con 0x7fe1b0101e00 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1add9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1add9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38744/0 (socket says 172.21.15.42:38744) 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1add9b700 1 -- 172.21.15.42:0/1169660597 learned_addr learned my addr 172.21.15.42:0/1169660597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.217+0000 7fe1ae59c700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b0101e00 0x7fe1b019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe1add9b700 1 -- 172.21.15.42:0/1169660597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b0101e00 msgr2=0x7fe1b019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe1add9b700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b0101e00 0x7fe1b019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe1add9b700 1 -- 172.21.15.42:0/1169660597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a40095f0 con 0x7fe1b0102720 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe1add9b700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b019cbd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40059d0 tx=0x7fe1a400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe19f7fe700 1 -- 172.21.15.42:0/1169660597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a401d070 con 0x7fe1b0102720 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b0196990 con 0x7fe1b0102720 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe19f7fe700 1 -- 172.21.15.42:0/1169660597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1a400b610 con 0x7fe1b0102720 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe19f7fe700 1 -- 172.21.15.42:0/1169660597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1a4022470 con 0x7fe1b0102720 2024-09-16T23:55:56.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.218+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b0196e80 con 0x7fe1b0102720 2024-09-16T23:55:56.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.219+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b0108310 con 0x7fe1b0102720 2024-09-16T23:55:56.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.220+0000 7fe19f7fe700 1 -- 172.21.15.42:0/1169660597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe1a40050d0 con 0x7fe1b0102720 2024-09-16T23:55:56.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.220+0000 7fe19f7fe700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe19806e680 0x7fe198070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.220+0000 7fe19f7fe700 1 -- 172.21.15.42:0/1169660597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe1a408fa90 con 0x7fe1b0102720 2024-09-16T23:55:56.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.220+0000 7fe1ae59c700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe19806e680 0x7fe198070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.221+0000 7fe1ae59c700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe19806e680 0x7fe198070b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009c60 tx=0x7fe1a0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:56.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.225+0000 7fe19f7fe700 1 -- 172.21.15.42:0/1169660597 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe1a405c810 con 0x7fe1b0102720 2024-09-16T23:55:56.309 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T23:55:56.309 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 bash[90783]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T23:55:56.309 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T23:55:56.310 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 bash[90783]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T23:55:56.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.443+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1b0197160 con 0x7fe19806e680 2024-09-16T23:55:56.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.452+0000 7fe19f7fe700 1 -- 172.21.15.42:0/1169660597 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe1b0197160 con 0x7fe19806e680 2024-09-16T23:55:56.451 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:55:56.451 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (4m) 29s ago 15m 24.5M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:55:56.451 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (15m) 29s ago 15m 8564k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:55:56.451 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (14m) 2m ago 14m 7964k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (2m) 29s ago 15m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (2m) 2m ago 14m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (4m) 29s ago 15m 80.7M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (9m) 29s ago 9m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (9m) 29s ago 9m 16.9M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (9m) 2m ago 9m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (9m) 2m ago 9m 17.2M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (6m) 29s ago 16m 500M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (5m) 2m ago 14m 438M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (3m) 29s ago 16m 50.9M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (2m) 2m ago 14m 35.5M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (5m) 29s ago 15m 16.2M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (5m) 2m ago 14m 15.4M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:55:56.452 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (107s) 29s ago 13m 83.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (71s) 29s ago 12m 82.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (35s) 29s ago 12m 11.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (12m) 2m ago 12m 235M 2241M 18.2.1 5be31c24972a f5dba7ca4af5 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (11m) 2m ago 11m 222M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (11m) 2m ago 11m 202M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (4m) 29s ago 14m 76.3M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe19806e680 msgr2=0x7fe198070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe19806e680 0x7fe198070b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009c60 tx=0x7fe1a0009260 comp rx=0 tx=0).stop 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 msgr2=0x7fe1b019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b019cbd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40059d0 tx=0x7fe1a400f690 comp rx=0 tx=0).stop 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 shutdown_connections 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe19806e680 0x7fe198070b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1b0101e00 0x7fe1b019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 --2- 172.21.15.42:0/1169660597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe1b0102720 0x7fe1b019cbd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 >> 172.21.15.42:0/1169660597 conn(0x7fe1b00fb650 msgr2=0x7fe1b0105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 shutdown_connections 2024-09-16T23:55:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.455+0000 7fe1b4efc700 1 -- 172.21.15.42:0/1169660597 wait complete. 2024-09-16T23:55:56.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.561+0000 7fb115f0b700 1 -- 172.21.15.42:0/2887960933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110101e00 msgr2=0x7fb1101021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.561+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2887960933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110101e00 0x7fb1101021e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb100009910 tx=0x7fb100009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 -- 172.21.15.42:0/2887960933 shutdown_connections 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2887960933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110102720 0x7fb11010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2887960933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110101e00 0x7fb1101021e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 -- 172.21.15.42:0/2887960933 >> 172.21.15.42:0/2887960933 conn(0x7fb1100fb650 msgr2=0x7fb1100fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 -- 172.21.15.42:0/2887960933 shutdown_connections 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 -- 172.21.15.42:0/2887960933 wait complete. 2024-09-16T23:55:56.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 Processor -- start 2024-09-16T23:55:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.562+0000 7fb115f0b700 1 -- start start 2024-09-16T23:55:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb115f0b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110101e00 0x7fb1100ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb115f0b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110102720 0x7fb1100fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb115f0b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1101004e0 con 0x7fb110102720 2024-09-16T23:55:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb115f0b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb110100620 con 0x7fb110101e00 2024-09-16T23:55:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110102720 0x7fb1100fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110102720 0x7fb1100fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38764/0 (socket says 172.21.15.42:38764) 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110101e00 0x7fb1100ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10effd700 1 -- 172.21.15.42:0/2622702867 learned_addr learned my addr 172.21.15.42:0/2622702867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10effd700 1 -- 172.21.15.42:0/2622702867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110101e00 msgr2=0x7fb1100ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10effd700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110101e00 0x7fb1100ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10effd700 1 -- 172.21.15.42:0/2622702867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000095f0 con 0x7fb110102720 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10f7fe700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110101e00 0x7fb1100ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10effd700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110102720 0x7fb1100fffa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb10400d8d0 tx=0x7fb10400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10cff9700 1 -- 172.21.15.42:0/2622702867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1040041f0 con 0x7fb110102720 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1101a2a90 con 0x7fb110102720 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.563+0000 7fb10cff9700 1 -- 172.21.15.42:0/2622702867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb104004350 con 0x7fb110102720 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.564+0000 7fb10cff9700 1 -- 172.21.15.42:0/2622702867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb104003c80 con 0x7fb110102720 2024-09-16T23:55:56.564 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.564+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1101a2ea0 con 0x7fb110102720 2024-09-16T23:55:56.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.565+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb110108310 con 0x7fb110102720 2024-09-16T23:55:56.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.565+0000 7fb10cff9700 1 -- 172.21.15.42:0/2622702867 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb10400f690 con 0x7fb110102720 2024-09-16T23:55:56.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.566+0000 7fb10cff9700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0f806e680 0x7fb0f8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.566+0000 7fb10f7fe700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0f806e680 0x7fb0f8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.566+0000 7fb10cff9700 1 -- 172.21.15.42:0/2622702867 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb10408e180 con 0x7fb110102720 2024-09-16T23:55:56.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.566+0000 7fb10f7fe700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0f806e680 0x7fb0f8070b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb10000b5c0 tx=0x7fb10001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:56.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.570+0000 7fb10cff9700 1 -- 172.21.15.42:0/2622702867 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb1040573d0 con 0x7fb110102720 2024-09-16T23:55:56.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T23:55:56.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 bash[90783]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T23:55:56.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T23:55:56.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 bash[90783]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T23:55:56.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-16T23:55:56.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 bash[90783]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-16T23:55:56.629 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T23:55:56.629 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 bash[90783]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T23:55:56.629 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate[90799]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-16T23:55:56.629 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 bash[90783]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-16T23:55:56.629 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 podman[90783]: 2024-09-16 23:55:56.340964763 +0000 UTC m=+1.575399504 container died 30c747cf81dc52e128f01e3cb56438fcdf9edcb5e980f16cf03777f55725a385 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-16T23:55:56.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:56 smithi042 ceph-mon[111056]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:56 smithi042 ceph-mon[111056]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:56 smithi042 ceph-mon[111056]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:56.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.870+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb11004f2e0 con 0x7fb110102720 2024-09-16T23:55:56.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.870+0000 7fb10cff9700 1 -- 172.21.15.42:0/2622702867 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7fb10405aa90 con 0x7fb110102720 2024-09-16T23:55:56.869 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:55:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:55:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:55:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:55:56.869 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 3 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 7 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:55:56.870 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:55:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0f806e680 msgr2=0x7fb0f8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0f806e680 0x7fb0f8070b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb10000b5c0 tx=0x7fb10001a040 comp rx=0 tx=0).stop 2024-09-16T23:55:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110102720 msgr2=0x7fb1100fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110102720 0x7fb1100fffa0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb10400d8d0 tx=0x7fb10400dbe0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 shutdown_connections 2024-09-16T23:55:56.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0f806e680 0x7fb0f8070b40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb110101e00 0x7fb1100ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 --2- 172.21.15.42:0/2622702867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb110102720 0x7fb1100fffa0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 >> 172.21.15.42:0/2622702867 conn(0x7fb1100fb650 msgr2=0x7fb110105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:56.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 shutdown_connections 2024-09-16T23:55:56.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.872+0000 7fb115f0b700 1 -- 172.21.15.42:0/2622702867 wait complete. 2024-09-16T23:55:56.933 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-mon[81510]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:56.933 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-mon[81510]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:56.934 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:56 smithi043 ceph-mon[81510]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:56.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 -- 172.21.15.42:0/1034433941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8101de0 msgr2=0x7f5fe81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/1034433941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8101de0 0x7f5fe81021c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009910 tx=0x7f5fdc009c20 comp rx=0 tx=0).stop 2024-09-16T23:55:56.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 -- 172.21.15.42:0/1034433941 shutdown_connections 2024-09-16T23:55:56.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/1034433941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe8102700 0x7f5fe810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/1034433941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8101de0 0x7f5fe81021c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 -- 172.21.15.42:0/1034433941 >> 172.21.15.42:0/1034433941 conn(0x7f5fe80fb650 msgr2=0x7f5fe80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:56.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 -- 172.21.15.42:0/1034433941 shutdown_connections 2024-09-16T23:55:56.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 -- 172.21.15.42:0/1034433941 wait complete. 2024-09-16T23:55:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.981+0000 7f5fefb0a700 1 Processor -- start 2024-09-16T23:55:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fefb0a700 1 -- start start 2024-09-16T23:55:56.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fefb0a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe8101de0 0x7f5fe819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fefb0a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8102700 0x7f5fe819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fefb0a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe819d270 con 0x7f5fe8102700 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fefb0a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8196720 con 0x7f5fe8101de0 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed0a5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8102700 0x7f5fe819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed0a5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8102700 0x7f5fe819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38782/0 (socket says 172.21.15.42:38782) 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed0a5700 1 -- 172.21.15.42:0/2076883943 learned_addr learned my addr 172.21.15.42:0/2076883943 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed8a6700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe8101de0 0x7f5fe819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed0a5700 1 -- 172.21.15.42:0/2076883943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe8101de0 msgr2=0x7f5fe819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed0a5700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe8101de0 0x7f5fe819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed0a5700 1 -- 172.21.15.42:0/2076883943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0095f0 con 0x7f5fe8102700 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.982+0000 7f5fed0a5700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8102700 0x7f5fe819cbe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e940 tx=0x7f5fe400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.983+0000 7f5fdaffd700 1 -- 172.21.15.42:0/2076883943 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fe40096e0 con 0x7f5fe8102700 2024-09-16T23:55:56.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.983+0000 7f5fdaffd700 1 -- 172.21.15.42:0/2076883943 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fe400c7c0 con 0x7f5fe8102700 2024-09-16T23:55:56.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.983+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe8196a00 con 0x7f5fe8102700 2024-09-16T23:55:56.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.983+0000 7f5fdaffd700 1 -- 172.21.15.42:0/2076883943 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fe4005580 con 0x7f5fe8102700 2024-09-16T23:55:56.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.983+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe8196f50 con 0x7f5fe8102700 2024-09-16T23:55:56.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.984+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe81082f0 con 0x7f5fe8102700 2024-09-16T23:55:56.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.984+0000 7f5fdaffd700 1 -- 172.21.15.42:0/2076883943 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5fe400c930 con 0x7f5fe8102700 2024-09-16T23:55:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.985+0000 7f5fdaffd700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5fd406e680 0x7f5fd4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.985+0000 7f5fdaffd700 1 -- 172.21.15.42:0/2076883943 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5fe4014070 con 0x7f5fe8102700 2024-09-16T23:55:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.985+0000 7f5fed8a6700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5fd406e680 0x7f5fd4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:56.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.985+0000 7f5fed8a6700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5fd406e680 0x7f5fd4070b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0059d0 tx=0x7f5fdc005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:56.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:56.989+0000 7f5fdaffd700 1 -- 172.21.15.42:0/2076883943 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5fe405b510 con 0x7f5fe8102700 2024-09-16T23:55:57.201 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:56 smithi043 podman[90940]: 2024-09-16 23:55:56.936770063 +0000 UTC m=+0.583589395 container remove 30c747cf81dc52e128f01e3cb56438fcdf9edcb5e980f16cf03777f55725a385 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-c2ddf97) 2024-09-16T23:55:57.202 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:57 smithi043 podman[91001]: 2024-09-16 23:55:57.048524132 +0000 UTC m=+0.016452727 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:55:57.202 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:57 smithi043 podman[91001]: 2024-09-16 23:55:57.204795131 +0000 UTC m=+0.172723759 container create 47a2dcb81243df0477c22e3d5a394cc996ff47e734eddcab40a6c4b75a5d4421 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-c2ddf97, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T23:55:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.241+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5fe804f2e0 con 0x7f5fe8102700 2024-09-16T23:55:57.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.242+0000 7f5fdaffd700 1 -- 172.21.15.42:0/2076883943 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1938 (secure 0 0 0) 0x7f5fe405b0a0 con 0x7f5fe8102700 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:53:20:175453+0000 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:55:57.241 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:53:19.510146+0000 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:55:57.242 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 24269,14478 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:57.243 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:55:57.244 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:55:57.244 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:57.244 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:55:57.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5fd406e680 msgr2=0x7f5fd4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5fd406e680 0x7f5fd4070b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0059d0 tx=0x7f5fdc005940 comp rx=0 tx=0).stop 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8102700 msgr2=0x7f5fe819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8102700 0x7f5fe819cbe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e940 tx=0x7f5fe400ec50 comp rx=0 tx=0).stop 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 shutdown_connections 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5fd406e680 0x7f5fd4070b40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fe8101de0 0x7f5fe819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 --2- 172.21.15.42:0/2076883943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fe8102700 0x7f5fe819cbe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 >> 172.21.15.42:0/2076883943 conn(0x7f5fe80fb650 msgr2=0x7f5fe8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 shutdown_connections 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.244+0000 7f5fefb0a700 1 -- 172.21.15.42:0/2076883943 wait complete. 2024-09-16T23:55:57.245 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-16T23:55:57.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 -- 172.21.15.42:0/2368282885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608101de0 msgr2=0x7fb6081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/2368282885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608101de0 0x7fb6081021c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800b0a0 tx=0x7fb5f800b3b0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 -- 172.21.15.42:0/2368282885 shutdown_connections 2024-09-16T23:55:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/2368282885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb608102700 0x7fb60810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/2368282885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608101de0 0x7fb6081021c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 -- 172.21.15.42:0/2368282885 >> 172.21.15.42:0/2368282885 conn(0x7fb6080fb610 msgr2=0x7fb6080fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 -- 172.21.15.42:0/2368282885 shutdown_connections 2024-09-16T23:55:57.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 -- 172.21.15.42:0/2368282885 wait complete. 2024-09-16T23:55:57.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.356+0000 7fb60d5f8700 1 Processor -- start 2024-09-16T23:55:57.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb60d5f8700 1 -- start start 2024-09-16T23:55:57.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb60d5f8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb608101de0 0x7fb60819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb60d5f8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608102700 0x7fb60819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb60d5f8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60819d3a0 con 0x7fb608102700 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb60d5f8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6081968a0 con 0x7fb608101de0 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb6067fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608102700 0x7fb60819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb6067fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608102700 0x7fb60819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38800/0 (socket says 172.21.15.42:38800) 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb6067fc700 1 -- 172.21.15.42:0/696998565 learned_addr learned my addr 172.21.15.42:0/696998565 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb606ffd700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb608101de0 0x7fb60819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb6067fc700 1 -- 172.21.15.42:0/696998565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb608101de0 msgr2=0x7fb60819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb6067fc700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb608101de0 0x7fb60819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.357+0000 7fb6067fc700 1 -- 172.21.15.42:0/696998565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009d00 con 0x7fb608102700 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.358+0000 7fb6067fc700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608102700 0x7fb60819cd10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00b400 tx=0x7fb5fc00b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.358+0000 7fb5effff700 1 -- 172.21.15.42:0/696998565 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5fc011800 con 0x7fb608102700 2024-09-16T23:55:57.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.358+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb608196b80 con 0x7fb608102700 2024-09-16T23:55:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.358+0000 7fb5effff700 1 -- 172.21.15.42:0/696998565 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5fc00be80 con 0x7fb608102700 2024-09-16T23:55:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.358+0000 7fb5effff700 1 -- 172.21.15.42:0/696998565 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5fc010e70 con 0x7fb608102700 2024-09-16T23:55:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.358+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6081970d0 con 0x7fb608102700 2024-09-16T23:55:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.359+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6081082f0 con 0x7fb608102700 2024-09-16T23:55:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.359+0000 7fb5effff700 1 -- 172.21.15.42:0/696998565 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb5fc01a400 con 0x7fb608102700 2024-09-16T23:55:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.360+0000 7fb5effff700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb5f006e680 0x7fb5f0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:57.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.360+0000 7fb5effff700 1 -- 172.21.15.42:0/696998565 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb5fc08dff0 con 0x7fb608102700 2024-09-16T23:55:57.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.360+0000 7fb606ffd700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb5f006e680 0x7fb5f0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:57.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.360+0000 7fb606ffd700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb5f006e680 0x7fb5f0070b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000f80 tx=0x7fb5f800b7b0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:57.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.364+0000 7fb5effff700 1 -- 172.21.15.42:0/696998565 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb5fc05ad70 con 0x7fb608102700 2024-09-16T23:55:57.512 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:57 smithi043 podman[91001]: 2024-09-16 23:55:57.41525199 +0000 UTC m=+0.383180582 container init 47a2dcb81243df0477c22e3d5a394cc996ff47e734eddcab40a6c4b75a5d4421 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T23:55:57.512 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:57 smithi043 podman[91001]: 2024-09-16 23:55:57.418686601 +0000 UTC m=+0.386615184 container start 47a2dcb81243df0477c22e3d5a394cc996ff47e734eddcab40a6c4b75a5d4421 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T23:55:57.512 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-osd[91021]: -- 172.21.15.43:0/2378159961 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a162861e0 con 0x556a1542fc00 2024-09-16T23:55:57.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.596+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb608197e70 con 0x7fb5f006e680 2024-09-16T23:55:57.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.597+0000 7fb5effff700 1 -- 172.21.15.42:0/696998565 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb608197e70 con 0x7fb5f006e680 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "9/23 daemons upgraded", 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T23:55:57.596 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:55:57.597 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb5f006e680 msgr2=0x7fb5f0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb5f006e680 0x7fb5f0070b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8000f80 tx=0x7fb5f800b7b0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608102700 msgr2=0x7fb60819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608102700 0x7fb60819cd10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00b400 tx=0x7fb5fc00b710 comp rx=0 tx=0).stop 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 shutdown_connections 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb5f006e680 0x7fb5f0070b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb608101de0 0x7fb60819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 --2- 172.21.15.42:0/696998565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb608102700 0x7fb60819cd10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 >> 172.21.15.42:0/696998565 conn(0x7fb6080fb610 msgr2=0x7fb608105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 shutdown_connections 2024-09-16T23:55:57.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.599+0000 7fb60d5f8700 1 -- 172.21.15.42:0/696998565 wait complete. 2024-09-16T23:55:57.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 -- 172.21.15.42:0/3054879538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 msgr2=0x7ff280101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 --2- 172.21.15.42:0/3054879538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff280101280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b0a0 tx=0x7ff27000b3b0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 -- 172.21.15.42:0/3054879538 shutdown_connections 2024-09-16T23:55:57.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 --2- 172.21.15.42:0/3054879538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff280101850 0x7ff2801058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 --2- 172.21.15.42:0/3054879538 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff280101280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 -- 172.21.15.42:0/3054879538 >> 172.21.15.42:0/3054879538 conn(0x7ff2800fc750 msgr2=0x7ff2800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:57.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 -- 172.21.15.42:0/3054879538 shutdown_connections 2024-09-16T23:55:57.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.708+0000 7ff284c8e700 1 -- 172.21.15.42:0/3054879538 wait complete. 2024-09-16T23:55:57.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff284c8e700 1 Processor -- start 2024-09-16T23:55:57.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff284c8e700 1 -- start start 2024-09-16T23:55:57.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff284c8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff28019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:57.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff284c8e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff280101850 0x7ff28019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:57.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff284c8e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28019d2e0 con 0x7ff280100ea0 2024-09-16T23:55:57.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff284c8e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280196790 con 0x7ff280101850 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff27e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff28019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff27e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff28019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38824/0 (socket says 172.21.15.42:38824) 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff27e59c700 1 -- 172.21.15.42:0/94215087 learned_addr learned my addr 172.21.15.42:0/94215087 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.709+0000 7ff27dd9b700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff280101850 0x7ff28019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff27e59c700 1 -- 172.21.15.42:0/94215087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff280101850 msgr2=0x7ff28019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff27e59c700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff280101850 0x7ff28019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff27e59c700 1 -- 172.21.15.42:0/94215087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff270009d00 con 0x7ff280100ea0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff27e59c700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff28019c710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff270000f80 tx=0x7ff27000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff26f7fe700 1 -- 172.21.15.42:0/94215087 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff27000e040 con 0x7ff280100ea0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff26f7fe700 1 -- 172.21.15.42:0/94215087 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff270003c10 con 0x7ff280100ea0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff280196a10 con 0x7ff280100ea0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff26f7fe700 1 -- 172.21.15.42:0/94215087 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff27001d680 con 0x7ff280100ea0 2024-09-16T23:55:57.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.710+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff280196f00 con 0x7ff280100ea0 2024-09-16T23:55:57.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.711+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff280109240 con 0x7ff280100ea0 2024-09-16T23:55:57.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.712+0000 7ff26f7fe700 1 -- 172.21.15.42:0/94215087 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff270019070 con 0x7ff280100ea0 2024-09-16T23:55:57.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.712+0000 7ff26f7fe700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff26806e680 0x7ff268070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:55:57.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.712+0000 7ff26f7fe700 1 -- 172.21.15.42:0/94215087 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff27008fad0 con 0x7ff280100ea0 2024-09-16T23:55:57.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.712+0000 7ff27dd9b700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff26806e680 0x7ff268070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:55:57.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.713+0000 7ff27dd9b700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff26806e680 0x7ff268070b40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff280197c00 tx=0x7ff274009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:55:57.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:57.717+0000 7ff26f7fe700 1 -- 172.21.15.42:0/94215087 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff27005c8e0 con 0x7ff280100ea0 2024-09-16T23:55:57.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:57 smithi043 bash[91001]: 47a2dcb81243df0477c22e3d5a394cc996ff47e734eddcab40a6c4b75a5d4421 2024-09-16T23:55:57.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:57 smithi043 systemd[1]: Started Ceph osd.3 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:55:57.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-mon[81510]: pgmap v113: 65 pgs: 15 active+undersized, 7 peering, 16 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 65/369 objects degraded (17.615%) 2024-09-16T23:55:57.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2622702867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:57.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2076883943' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:55:57.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:57.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:57.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:55:57.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:57 smithi043 ceph-mon[81510]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:58.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.015+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff280197cf0 con 0x7ff280100ea0 2024-09-16T23:55:58.014 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:57 smithi042 ceph-mon[111056]: pgmap v113: 65 pgs: 15 active+undersized, 7 peering, 16 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 65/369 objects degraded (17.615%) 2024-09-16T23:55:58.014 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:57 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2622702867' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:55:58.014 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:57 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2076883943' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:55:58.014 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:57 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:58.014 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:57 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:55:58.015 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:57 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:55:58.015 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:57 smithi042 ceph-mon[111056]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:55:58.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.015+0000 7ff26f7fe700 1 -- 172.21.15.42:0/94215087 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1690 (secure 0 0 0) 0x7ff270003d80 con 0x7ff280100ea0 2024-09-16T23:55:58.015 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 3 pgs peering; Degraded data redundancy: 65/369 objects degraded (17.615%), 16 pgs degraded 2024-09-16T23:55:58.015 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:55:58.015 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:55:58.015 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T23:55:58.015 INFO:teuthology.orchestra.run.smithi042.stdout: osd.3 (root=default,host=smithi043) is down 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 3 pgs peering 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 1.0 is stuck peering for 3m, current state peering, last acting [0,1] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.5 is stuck peering for 6m, current state peering, last acting [0,4] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 3.15 is stuck peering for 5m, current state peering, last acting [0,4] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 65/369 objects degraded (17.615%), 16 pgs degraded 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-09-16T23:55:58.016 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.11 is active+undersized+degraded, acting [4,1] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.12 is active+undersized+degraded, acting [1,0] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-09-16T23:55:58.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff26806e680 msgr2=0x7ff268070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff26806e680 0x7ff268070b40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff280197c00 tx=0x7ff274009260 comp rx=0 tx=0).stop 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 msgr2=0x7ff28019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff28019c710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff270000f80 tx=0x7ff27000bc60 comp rx=0 tx=0).stop 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 shutdown_connections 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff26806e680 0x7ff268070b40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280100ea0 0x7ff28019c710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 --2- 172.21.15.42:0/94215087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff280101850 0x7ff28019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 >> 172.21.15.42:0/94215087 conn(0x7ff2800fc750 msgr2=0x7ff2800fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 shutdown_connections 2024-09-16T23:55:58.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:55:58.018+0000 7ff284c8e700 1 -- 172.21.15.42:0/94215087 wait complete. 2024-09-16T23:55:59.010 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:58 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/94215087' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:55:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:58 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/94215087' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:55:59.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:55:59 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[91017]: 2024-09-16T23:55:59.320+0000 7f17607a8640 -1 Falling back to public interface 2024-09-16T23:56:00.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:59 smithi042 ceph-mon[111056]: pgmap v114: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:00.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:55:59 smithi042 ceph-mon[111056]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T23:56:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:59 smithi043 ceph-mon[81510]: pgmap v114: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:55:59 smithi043 ceph-mon[81510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T23:56:01.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:01 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:01.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:01 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:01.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:01 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:01.602 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:01 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:01.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:01.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:01.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:01.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:01 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:02.364 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:02 smithi043 ceph-mon[81510]: pgmap v115: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:02 smithi042 ceph-mon[111056]: pgmap v115: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-mon[81510]: pgmap v116: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:56:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:04 smithi042 ceph-mon[111056]: pgmap v116: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:04 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 73/369 objects degraded (19.783%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:56:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:05.313 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:56:04 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[91017]: 2024-09-16T23:56:04.979+0000 7f17607a8640 -1 osd.3 0 read_superblock omap replica is missing. 2024-09-16T23:56:05.313 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[91017]: 2024-09-16T23:56:05.315+0000 7f17607a8640 -1 osd.3 64 log_to_monitors true 2024-09-16T23:56:05.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:56:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:05.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:05 smithi043 ceph-mon[81510]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-16T23:56:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:56:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:05 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:05 smithi042 ceph-mon[111056]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-16T23:56:06.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 16 23:56:06 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[91017]: 2024-09-16T23:56:06.343+0000 7f1758542640 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:56:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:06 smithi043 ceph-mon[81510]: pgmap v117: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:06 smithi043 ceph-mon[81510]: from='osd.3 [v2:172.21.15.43:6800/1754132159,v1:172.21.15.43:6801/1754132159]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:56:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:06 smithi043 ceph-mon[81510]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:56:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:06.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:06 smithi042 ceph-mon[111056]: pgmap v117: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:06 smithi042 ceph-mon[111056]: from='osd.3 [v2:172.21.15.43:6800/1754132159,v1:172.21.15.43:6801/1754132159]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:56:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:06 smithi042 ceph-mon[111056]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T23:56:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:07.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:07 smithi043 ceph-mon[81510]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T23:56:07.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:07 smithi043 ceph-mon[81510]: from='osd.3 [v2:172.21.15.43:6800/1754132159,v1:172.21.15.43:6801/1754132159]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:07.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:07 smithi043 ceph-mon[81510]: osdmap e67: 6 total, 5 up, 6 in 2024-09-16T23:56:07.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:07 smithi043 ceph-mon[81510]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:07 smithi042 ceph-mon[111056]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T23:56:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:07 smithi042 ceph-mon[111056]: from='osd.3 [v2:172.21.15.43:6800/1754132159,v1:172.21.15.43:6801/1754132159]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:07 smithi042 ceph-mon[111056]: osdmap e67: 6 total, 5 up, 6 in 2024-09-16T23:56:07.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:07 smithi042 ceph-mon[111056]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:08 smithi043 ceph-mon[81510]: pgmap v119: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:08 smithi043 ceph-mon[81510]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:56:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:08 smithi043 ceph-mon[81510]: osd.3 [v2:172.21.15.43:6800/1754132159,v1:172.21.15.43:6801/1754132159] boot 2024-09-16T23:56:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:08 smithi043 ceph-mon[81510]: osdmap e68: 6 total, 6 up, 6 in 2024-09-16T23:56:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:56:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:08 smithi043 ceph-mon[81510]: osdmap e69: 6 total, 6 up, 6 in 2024-09-16T23:56:08.733 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:08 smithi042 ceph-mon[111056]: pgmap v119: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/369 objects degraded (19.783%) 2024-09-16T23:56:08.733 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:08 smithi042 ceph-mon[111056]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:56:08.733 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:08 smithi042 ceph-mon[111056]: osd.3 [v2:172.21.15.43:6800/1754132159,v1:172.21.15.43:6801/1754132159] boot 2024-09-16T23:56:08.734 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:08 smithi042 ceph-mon[111056]: osdmap e68: 6 total, 6 up, 6 in 2024-09-16T23:56:08.734 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T23:56:08.734 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:08 smithi042 ceph-mon[111056]: osdmap e69: 6 total, 6 up, 6 in 2024-09-16T23:56:09.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:09 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 61/369 objects degraded (16.531%), 15 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:09.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:09 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 61/369 objects degraded (16.531%), 15 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:10.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:10 smithi042 ceph-mon[111056]: pgmap v122: 65 pgs: 16 active+undersized, 15 active+undersized+degraded, 34 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 61/369 objects degraded (16.531%) 2024-09-16T23:56:10.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:10 smithi043 ceph-mon[81510]: pgmap v122: 65 pgs: 16 active+undersized, 15 active+undersized+degraded, 34 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 61/369 objects degraded (16.531%) 2024-09-16T23:56:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:12 smithi042 ceph-mon[111056]: pgmap v123: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:56:12.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:12 smithi043 ceph-mon[81510]: pgmap v123: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 58/369 objects degraded (15.718%) 2024-09-16T23:56:13.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:13 smithi042 ceph-mon[111056]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 58/369 objects degraded (15.718%), 14 pgs degraded) 2024-09-16T23:56:13.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:13 smithi043 ceph-mon[81510]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 58/369 objects degraded (15.718%), 14 pgs degraded) 2024-09-16T23:56:14.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:14 smithi042 ceph-mon[111056]: pgmap v124: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:14 smithi043 ceph-mon[81510]: pgmap v124: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:16.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:16 smithi042 ceph-mon[111056]: pgmap v125: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:16 smithi043 ceph-mon[81510]: pgmap v125: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:18.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:18 smithi042 ceph-mon[111056]: pgmap v126: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:18.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:18 smithi043 ceph-mon[81510]: pgmap v126: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:19.770 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:19 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:19.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:19 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:20.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:20 smithi042 ceph-mon[111056]: pgmap v127: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:20.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:20 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:20.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:20 smithi042 ceph-mon[111056]: Upgrade: osd.4 is safe to restart 2024-09-16T23:56:20.776 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:20 smithi043 ceph-mon[81510]: pgmap v127: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:20.776 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:20 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T23:56:20.776 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:20 smithi043 ceph-mon[81510]: Upgrade: osd.4 is safe to restart 2024-09-16T23:56:21.836 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: pgmap v128: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:21.836 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:21.837 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:21.837 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:21.837 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: Upgrade: Updating osd.4 2024-09-16T23:56:21.837 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:21.837 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T23:56:21.837 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:21.837 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:21 smithi043 ceph-mon[81510]: Deploying daemon osd.4 on smithi043 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: pgmap v128: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: Upgrade: Updating osd.4 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:22.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:21 smithi042 ceph-mon[111056]: Deploying daemon osd.4 on smithi043 2024-09-16T23:56:23.903 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:23 smithi042 ceph-mon[111056]: pgmap v129: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:23.982 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:23 smithi043 systemd[1]: Stopping Ceph osd.4 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:56:23.982 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:23 smithi043 ceph-mon[81510]: pgmap v129: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:24.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:23 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[47483]: 2024-09-16T23:56:23.984+0000 7fea0eba1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:56:24.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:23 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[47483]: 2024-09-16T23:56:23.984+0000 7fea0eba1700 -1 osd.4 69 *** Got signal Terminated *** 2024-09-16T23:56:24.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:23 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[47483]: 2024-09-16T23:56:23.984+0000 7fea0eba1700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T23:56:25.000 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:24 smithi043 ceph-mon[81510]: osd.4 marked itself down and dead 2024-09-16T23:56:25.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:24 smithi042 ceph-mon[111056]: osd.4 marked itself down and dead 2024-09-16T23:56:25.309 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:25 smithi043 podman[95735]: 2024-09-16 23:56:25.04206063 +0000 UTC m=+1.162251729 container died ea12742b844c4926caf4917272d0ca07e0e49c20c2bf7a6a982bb11f9fd51d6c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T23:56:25.627 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:25 smithi043 podman[95735]: 2024-09-16 23:56:25.418563733 +0000 UTC m=+1.538754822 container cleanup ea12742b844c4926caf4917272d0ca07e0e49c20c2bf7a6a982bb11f9fd51d6c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-09-16T23:56:25.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:25 smithi043 bash[95735]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4 2024-09-16T23:56:26.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:25 smithi042 ceph-mon[111056]: pgmap v130: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:26.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:25 smithi042 ceph-mon[111056]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:56:26.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:25 smithi042 ceph-mon[111056]: osdmap e70: 6 total, 5 up, 6 in 2024-09-16T23:56:26.033 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:25 smithi043 ceph-mon[81510]: pgmap v130: 65 pgs: 65 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:26.033 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:25 smithi043 ceph-mon[81510]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:56:26.033 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:25 smithi043 ceph-mon[81510]: osdmap e70: 6 total, 5 up, 6 in 2024-09-16T23:56:26.034 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:25 smithi043 podman[95761]: 2024-09-16 23:56:25.670088634 +0000 UTC m=+0.628571585 container remove ea12742b844c4926caf4917272d0ca07e0e49c20c2bf7a6a982bb11f9fd51d6c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T23:56:26.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:26 smithi043 podman[95852]: 2024-09-16 23:56:25.939434608 +0000 UTC m=+0.016172856 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:56:26.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:26 smithi043 podman[95852]: 2024-09-16 23:56:26.054190678 +0000 UTC m=+0.130928907 container create 768bea108784bcb036cef3021a4e9318c7676f9a506f283c683b6ba8bfce99db (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T23:56:26.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:26 smithi043 podman[95852]: 2024-09-16 23:56:26.281733648 +0000 UTC m=+0.358471889 container init 768bea108784bcb036cef3021a4e9318c7676f9a506f283c683b6ba8bfce99db (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97) 2024-09-16T23:56:26.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:26 smithi043 podman[95852]: 2024-09-16 23:56:26.285238215 +0000 UTC m=+0.361976450 container start 768bea108784bcb036cef3021a4e9318c7676f9a506f283c683b6ba8bfce99db (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T23:56:26.640 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:26 smithi043 podman[95852]: 2024-09-16 23:56:26.381691631 +0000 UTC m=+0.458429879 container attach 768bea108784bcb036cef3021a4e9318c7676f9a506f283c683b6ba8bfce99db (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T23:56:26.640 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:26 smithi043 podman[95852]: 2024-09-16 23:56:26.477869028 +0000 UTC m=+0.554607278 container died 768bea108784bcb036cef3021a4e9318c7676f9a506f283c683b6ba8bfce99db (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T23:56:26.933 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:26 smithi043 ceph-mon[81510]: osdmap e71: 6 total, 5 up, 6 in 2024-09-16T23:56:27.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:26 smithi042 ceph-mon[111056]: osdmap e71: 6 total, 5 up, 6 in 2024-09-16T23:56:27.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 podman[95876]: 2024-09-16 23:56:27.227505729 +0000 UTC m=+0.743572667 container remove 768bea108784bcb036cef3021a4e9318c7676f9a506f283c683b6ba8bfce99db (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T23:56:27.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.4.service: Deactivated successfully. 2024-09-16T23:56:27.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 systemd[1]: Stopped Ceph osd.4 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:56:27.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.4.service: Consumed 57.778s CPU time. 2024-09-16T23:56:27.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 systemd[1]: Starting Ceph osd.4 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:56:27.961 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:27 smithi043 ceph-mon[81510]: pgmap v133: 65 pgs: 8 active+undersized, 4 stale+active+clean, 8 active+undersized+degraded, 45 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 33/369 objects degraded (8.943%) 2024-09-16T23:56:27.961 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:27 smithi043 ceph-mon[81510]: Health check failed: Degraded data redundancy: 33/369 objects degraded (8.943%), 8 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:27.962 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 podman[95977]: 2024-09-16 23:56:27.580812703 +0000 UTC m=+0.016549742 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:56:27.962 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 podman[95977]: 2024-09-16 23:56:27.687103923 +0000 UTC m=+0.122840947 container create 575ac6aa9ab4e2ddef75caac082034fe4f618bd9667ed09a0794f8f1f6f8c132 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-16T23:56:27.962 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 podman[95977]: 2024-09-16 23:56:27.881374929 +0000 UTC m=+0.317111968 container init 575ac6aa9ab4e2ddef75caac082034fe4f618bd9667ed09a0794f8f1f6f8c132 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T23:56:27.962 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 podman[95977]: 2024-09-16 23:56:27.884850663 +0000 UTC m=+0.320587701 container start 575ac6aa9ab4e2ddef75caac082034fe4f618bd9667ed09a0794f8f1f6f8c132 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T23:56:27.962 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:27 smithi043 podman[95977]: 2024-09-16 23:56:27.964589603 +0000 UTC m=+0.400326630 container attach 575ac6aa9ab4e2ddef75caac082034fe4f618bd9667ed09a0794f8f1f6f8c132 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T23:56:28.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:27 smithi042 ceph-mon[111056]: pgmap v133: 65 pgs: 8 active+undersized, 4 stale+active+clean, 8 active+undersized+degraded, 45 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 33/369 objects degraded (8.943%) 2024-09-16T23:56:28.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:27 smithi042 ceph-mon[111056]: Health check failed: Degraded data redundancy: 33/369 objects degraded (8.943%), 8 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:28.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 -- 172.21.15.42:0/2960977897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724101e60 msgr2=0x7fa724102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/2960977897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724101e60 0x7fa724102240 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa714009910 tx=0x7fa714009c20 comp rx=0 tx=0).stop 2024-09-16T23:56:28.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 -- 172.21.15.42:0/2960977897 shutdown_connections 2024-09-16T23:56:28.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/2960977897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724102780 0x7fa72410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/2960977897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724101e60 0x7fa724102240 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 -- 172.21.15.42:0/2960977897 >> 172.21.15.42:0/2960977897 conn(0x7fa7240fb650 msgr2=0x7fa7240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:28.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 -- 172.21.15.42:0/2960977897 shutdown_connections 2024-09-16T23:56:28.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.145+0000 7fa72a5bc700 1 -- 172.21.15.42:0/2960977897 wait complete. 2024-09-16T23:56:28.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa72a5bc700 1 Processor -- start 2024-09-16T23:56:28.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa72a5bc700 1 -- start start 2024-09-16T23:56:28.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa72a5bc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101e60 0x7fa72419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa72a5bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724102780 0x7fa72419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa72a5bc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72419d220 con 0x7fa724102780 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa72a5bc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa724196720 con 0x7fa724101e60 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724102780 0x7fa72419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa7237fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724102780 0x7fa72419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35350/0 (socket says 172.21.15.42:35350) 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa7237fe700 1 -- 172.21.15.42:0/1306187491 learned_addr learned my addr 172.21.15.42:0/1306187491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.146+0000 7fa723fff700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101e60 0x7fa72419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa7237fe700 1 -- 172.21.15.42:0/1306187491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101e60 msgr2=0x7fa72419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa7237fe700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101e60 0x7fa72419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa7237fe700 1 -- 172.21.15.42:0/1306187491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7140095f0 con 0x7fa724102780 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa723fff700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101e60 0x7fa72419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa7237fe700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724102780 0x7fa72419cb90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa71800e8f0 tx=0x7fa71800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:28.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa7217fa700 1 -- 172.21.15.42:0/1306187491 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa718009790 con 0x7fa724102780 2024-09-16T23:56:28.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa7217fa700 1 -- 172.21.15.42:0/1306187491 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa71800c810 con 0x7fa724102780 2024-09-16T23:56:28.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa724196a00 con 0x7fa724102780 2024-09-16T23:56:28.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa7217fa700 1 -- 172.21.15.42:0/1306187491 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa718005580 con 0x7fa724102780 2024-09-16T23:56:28.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.147+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa724196f50 con 0x7fa724102780 2024-09-16T23:56:28.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.148+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa724108310 con 0x7fa724102780 2024-09-16T23:56:28.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.149+0000 7fa7217fa700 1 -- 172.21.15.42:0/1306187491 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa71800c980 con 0x7fa724102780 2024-09-16T23:56:28.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.149+0000 7fa7217fa700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa70c06e680 0x7fa70c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.150+0000 7fa723fff700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa70c06e680 0x7fa70c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.150+0000 7fa7217fa700 1 -- 172.21.15.42:0/1306187491 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa718014070 con 0x7fa724102780 2024-09-16T23:56:28.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.150+0000 7fa723fff700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa70c06e680 0x7fa70c070b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa714005b40 tx=0x7fa714005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:28.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.154+0000 7fa7217fa700 1 -- 172.21.15.42:0/1306187491 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa71805b570 con 0x7fa724102780 2024-09-16T23:56:28.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:56:28.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 bash[95977]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:56:28.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:56:28.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 bash[95977]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:56:28.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.383+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa724197220 con 0x7fa70c06e680 2024-09-16T23:56:28.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.384+0000 7fa7217fa700 1 -- 172.21.15.42:0/1306187491 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa724197220 con 0x7fa70c06e680 2024-09-16T23:56:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa70c06e680 msgr2=0x7fa70c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa70c06e680 0x7fa70c070b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa714005b40 tx=0x7fa714005a90 comp rx=0 tx=0).stop 2024-09-16T23:56:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724102780 msgr2=0x7fa72419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724102780 0x7fa72419cb90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa71800e8f0 tx=0x7fa71800ec00 comp rx=0 tx=0).stop 2024-09-16T23:56:28.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 shutdown_connections 2024-09-16T23:56:28.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa70c06e680 0x7fa70c070b40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa724101e60 0x7fa72419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 --2- 172.21.15.42:0/1306187491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa724102780 0x7fa72419cb90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.386+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 >> 172.21.15.42:0/1306187491 conn(0x7fa7240fb650 msgr2=0x7fa724105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:28.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.387+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 shutdown_connections 2024-09-16T23:56:28.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.387+0000 7fa72a5bc700 1 -- 172.21.15.42:0/1306187491 wait complete. 2024-09-16T23:56:28.395 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.496+0000 7fc882b58700 1 -- 172.21.15.42:0/2872845315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c0ffda0 msgr2=0x7fc87c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.496+0000 7fc882b58700 1 --2- 172.21.15.42:0/2872845315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c0ffda0 0x7fc87c100180 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009910 tx=0x7fc86c009c20 comp rx=0 tx=0).stop 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 -- 172.21.15.42:0/2872845315 shutdown_connections 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 --2- 172.21.15.42:0/2872845315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87c100750 0x7fc87c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 --2- 172.21.15.42:0/2872845315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c0ffda0 0x7fc87c100180 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 -- 172.21.15.42:0/2872845315 >> 172.21.15.42:0/2872845315 conn(0x7fc87c0fb650 msgr2=0x7fc87c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 -- 172.21.15.42:0/2872845315 shutdown_connections 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 -- 172.21.15.42:0/2872845315 wait complete. 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 Processor -- start 2024-09-16T23:56:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 -- start start 2024-09-16T23:56:28.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87c0ffda0 0x7fc87c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c100750 0x7fc87c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c19d3b0 con 0x7fc87c100750 2024-09-16T23:56:28.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.497+0000 7fc882b58700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c196860 con 0x7fc87c0ffda0 2024-09-16T23:56:28.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc87bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c100750 0x7fc87c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc87bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c100750 0x7fc87c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35362/0 (socket says 172.21.15.42:35362) 2024-09-16T23:56:28.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc87bfff700 1 -- 172.21.15.42:0/3270589452 learned_addr learned my addr 172.21.15.42:0/3270589452 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc8808f4700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87c0ffda0 0x7fc87c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc87bfff700 1 -- 172.21.15.42:0/3270589452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87c0ffda0 msgr2=0x7fc87c19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc87bfff700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87c0ffda0 0x7fc87c19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc87bfff700 1 -- 172.21.15.42:0/3270589452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c0095f0 con 0x7fc87c100750 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc8808f4700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87c0ffda0 0x7fc87c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc87bfff700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c100750 0x7fc87c19cd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc86400b480 tx=0x7fc86400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc879ffb700 1 -- 172.21.15.42:0/3270589452 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8640118c0 con 0x7fc87c100750 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc879ffb700 1 -- 172.21.15.42:0/3270589452 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc864010380 con 0x7fc87c100750 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c196b40 con 0x7fc87c100750 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.498+0000 7fc879ffb700 1 -- 172.21.15.42:0/3270589452 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc86400f4f0 con 0x7fc87c100750 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.499+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c197090 con 0x7fc87c100750 2024-09-16T23:56:28.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.499+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c04f2e0 con 0x7fc87c100750 2024-09-16T23:56:28.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.500+0000 7fc879ffb700 1 -- 172.21.15.42:0/3270589452 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc8640104f0 con 0x7fc87c100750 2024-09-16T23:56:28.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.500+0000 7fc879ffb700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86806e680 0x7fc868070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.500+0000 7fc879ffb700 1 -- 172.21.15.42:0/3270589452 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc86408e210 con 0x7fc87c100750 2024-09-16T23:56:28.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.500+0000 7fc8808f4700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86806e680 0x7fc868070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.501+0000 7fc8808f4700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86806e680 0x7fc868070b40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc86c006010 tx=0x7fc86c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:28.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.505+0000 7fc879ffb700 1 -- 172.21.15.42:0/3270589452 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc86405af90 con 0x7fc87c100750 2024-09-16T23:56:28.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.735+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc87c197380 con 0x7fc86806e680 2024-09-16T23:56:28.734 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:28 smithi042 ceph-mon[111056]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:28.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.737+0000 7fc879ffb700 1 -- 172.21.15.42:0/3270589452 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc87c197380 con 0x7fc86806e680 2024-09-16T23:56:28.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86806e680 msgr2=0x7fc868070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86806e680 0x7fc868070b40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc86c006010 tx=0x7fc86c00b540 comp rx=0 tx=0).stop 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c100750 msgr2=0x7fc87c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c100750 0x7fc87c19cd20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc86400b480 tx=0x7fc86400b840 comp rx=0 tx=0).stop 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 shutdown_connections 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86806e680 0x7fc868070b40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc87c0ffda0 0x7fc87c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 --2- 172.21.15.42:0/3270589452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc87c100750 0x7fc87c19cd20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 >> 172.21.15.42:0/3270589452 conn(0x7fc87c0fb650 msgr2=0x7fc87c103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 shutdown_connections 2024-09-16T23:56:28.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.739+0000 7fc882b58700 1 -- 172.21.15.42:0/3270589452 wait complete. 2024-09-16T23:56:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.847+0000 7fd7d344e700 1 -- 172.21.15.42:0/3830370213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0fff50 msgr2=0x7fd7cc105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.847+0000 7fd7d344e700 1 --2- 172.21.15.42:0/3830370213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0fff50 0x7fd7cc105840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009910 tx=0x7fd7bc009c20 comp rx=0 tx=0).stop 2024-09-16T23:56:28.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.847+0000 7fd7d344e700 1 -- 172.21.15.42:0/3830370213 shutdown_connections 2024-09-16T23:56:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.847+0000 7fd7d344e700 1 --2- 172.21.15.42:0/3830370213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0fff50 0x7fd7cc105840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.847+0000 7fd7d344e700 1 --2- 172.21.15.42:0/3830370213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7cc0ff5a0 0x7fd7cc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.847+0000 7fd7d344e700 1 -- 172.21.15.42:0/3830370213 >> 172.21.15.42:0/3830370213 conn(0x7fd7cc074e30 msgr2=0x7fd7cc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 -- 172.21.15.42:0/3830370213 shutdown_connections 2024-09-16T23:56:28.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 -- 172.21.15.42:0/3830370213 wait complete. 2024-09-16T23:56:28.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 Processor -- start 2024-09-16T23:56:28.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 -- start start 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0ff5a0 0x7fd7cc100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7cc0fff50 0x7fd7cc100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc100fa0 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.848+0000 7fd7d344e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc1010e0 con 0x7fd7cc0fff50 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d11ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0ff5a0 0x7fd7cc100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d11ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0ff5a0 0x7fd7cc100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35378/0 (socket says 172.21.15.42:35378) 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d11ea700 1 -- 172.21.15.42:0/1306667582 learned_addr learned my addr 172.21.15.42:0/1306667582 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d09e9700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7cc0fff50 0x7fd7cc100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d11ea700 1 -- 172.21.15.42:0/1306667582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7cc0fff50 msgr2=0x7fd7cc100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d11ea700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7cc0fff50 0x7fd7cc100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d11ea700 1 -- 172.21.15.42:0/1306667582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc0095f0 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d11ea700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0ff5a0 0x7fd7cc100520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c930 tx=0x7fd7c800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7c27fc700 1 -- 172.21.15.42:0/1306667582 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c8007ad0 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7c27fc700 1 -- 172.21.15.42:0/1306667582 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7c8007c30 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc1a2a40 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7c27fc700 1 -- 172.21.15.42:0/1306667582 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7c8018610 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.849+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc1a2da0 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.851+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc109200 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.851+0000 7fd7c27fc700 1 -- 172.21.15.42:0/1306667582 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd7c800f450 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.852+0000 7fd7c27fc700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7b806e680 0x7fd7b8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.852+0000 7fd7c27fc700 1 -- 172.21.15.42:0/1306667582 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd7c808e6f0 con 0x7fd7cc0ff5a0 2024-09-16T23:56:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.852+0000 7fd7d09e9700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7b806e680 0x7fd7b8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:28.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.852+0000 7fd7d09e9700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7b806e680 0x7fd7b8070b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc101d10 tx=0x7fd7bc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:28.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:28.856+0000 7fd7c27fc700 1 -- 172.21.15.42:0/1306667582 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd7c80579c0 con 0x7fd7cc0ff5a0 2024-09-16T23:56:29.043 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:28 smithi043 ceph-mon[81510]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:29.044 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T23:56:29.044 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 bash[95977]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T23:56:29.044 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T23:56:29.044 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:28 smithi043 bash[95977]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T23:56:29.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.072+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7cc0611d0 con 0x7fd7b806e680 2024-09-16T23:56:29.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.082+0000 7fd7c27fc700 1 -- 172.21.15.42:0/1306667582 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd7cc0611d0 con 0x7fd7b806e680 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (5m) 62s ago 15m 24.5M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (16m) 62s ago 16m 8564k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (15m) 28s ago 15m 8190k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (2m) 62s ago 16m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (2m) 28s ago 15m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (4m) 62s ago 15m 80.7M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (10m) 62s ago 10m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (10m) 62s ago 10m 16.9M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (10m) 28s ago 10m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (10m) 28s ago 10m 18.1M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:56:29.080 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (6m) 62s ago 17m 500M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (5m) 28s ago 14m 440M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (3m) 62s ago 17m 50.9M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (3m) 28s ago 14m 40.5M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (5m) 62s ago 16m 16.2M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (5m) 28s ago 14m 15.5M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (2m) 62s ago 14m 83.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (104s) 62s ago 13m 82.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (67s) 62s ago 13m 11.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (32s) 28s ago 12m 11.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (12m) 28s ago 12m 223M 2241M 18.2.1 5be31c24972a ea12742b844c 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (11m) 28s ago 11m 205M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (5m) 62s ago 15m 76.3M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7b806e680 msgr2=0x7fd7b8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7b806e680 0x7fd7b8070b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc101d10 tx=0x7fd7bc01a040 comp rx=0 tx=0).stop 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0ff5a0 msgr2=0x7fd7cc100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0ff5a0 0x7fd7cc100520 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800c930 tx=0x7fd7c800ccf0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 shutdown_connections 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7b806e680 0x7fd7b8070b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7cc0ff5a0 0x7fd7cc100520 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 --2- 172.21.15.42:0/1306667582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7cc0fff50 0x7fd7cc100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 >> 172.21.15.42:0/1306667582 conn(0x7fd7cc074e30 msgr2=0x7fd7cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 shutdown_connections 2024-09-16T23:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.084+0000 7fd7d344e700 1 -- 172.21.15.42:0/1306667582 wait complete. 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 -- 172.21.15.42:0/2261385777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 msgr2=0x7f9d841047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2261385777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d841047a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009910 tx=0x7f9d6c009c20 comp rx=0 tx=0).stop 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 -- 172.21.15.42:0/2261385777 shutdown_connections 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2261385777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d841047a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2261385777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d840ffda0 0x7f9d84100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 -- 172.21.15.42:0/2261385777 >> 172.21.15.42:0/2261385777 conn(0x7f9d840fb650 msgr2=0x7f9d840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 -- 172.21.15.42:0/2261385777 shutdown_connections 2024-09-16T23:56:29.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.191+0000 7f9d8af59700 1 -- 172.21.15.42:0/2261385777 wait complete. 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d8af59700 1 Processor -- start 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d8af59700 1 -- start start 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d8af59700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d840ffda0 0x7f9d84101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d8af59700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d84101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d8af59700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d841022c0 con 0x7f9d84100750 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d8af59700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d84102400 con 0x7f9d840ffda0 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d84101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d84101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35400/0 (socket says 172.21.15.42:35400) 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d83fff700 1 -- 172.21.15.42:0/2798752577 learned_addr learned my addr 172.21.15.42:0/2798752577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:56:29.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d88cf5700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d840ffda0 0x7f9d84101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d83fff700 1 -- 172.21.15.42:0/2798752577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d840ffda0 msgr2=0x7f9d84101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d83fff700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d840ffda0 0x7f9d84101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.192+0000 7f9d83fff700 1 -- 172.21.15.42:0/2798752577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0095f0 con 0x7f9d84100750 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.193+0000 7f9d83fff700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d84101d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00bd70 tx=0x7f9d6c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.193+0000 7f9d81ffb700 1 -- 172.21.15.42:0/2798752577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d6c01d070 con 0x7f9d84100750 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.193+0000 7f9d81ffb700 1 -- 172.21.15.42:0/2798752577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d6c005dc0 con 0x7f9d84100750 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.193+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d84102680 con 0x7f9d84100750 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.193+0000 7f9d81ffb700 1 -- 172.21.15.42:0/2798752577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d6c017400 con 0x7f9d84100750 2024-09-16T23:56:29.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.193+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d841a2da0 con 0x7f9d84100750 2024-09-16T23:56:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.194+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d841091b0 con 0x7f9d84100750 2024-09-16T23:56:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.194+0000 7f9d81ffb700 1 -- 172.21.15.42:0/2798752577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9d6c00f460 con 0x7f9d84100750 2024-09-16T23:56:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.195+0000 7f9d81ffb700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9d7006e680 0x7f9d70070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.195+0000 7f9d81ffb700 1 -- 172.21.15.42:0/2798752577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9d6c08fa70 con 0x7f9d84100750 2024-09-16T23:56:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.195+0000 7f9d88cf5700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9d7006e680 0x7f9d70070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.195+0000 7f9d88cf5700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9d7006e680 0x7f9d70070b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005950 tx=0x7f9d7400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:29.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.199+0000 7f9d81ffb700 1 -- 172.21.15.42:0/2798752577 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9d6c058cc0 con 0x7f9d84100750 2024-09-16T23:56:29.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T23:56:29.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 bash[95977]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T23:56:29.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T23:56:29.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 bash[95977]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T23:56:29.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-16T23:56:29.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 bash[95977]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-16T23:56:29.379 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T23:56:29.379 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 bash[95977]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T23:56:29.379 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate[95992]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-16T23:56:29.379 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 bash[95977]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-16T23:56:29.379 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 podman[95977]: 2024-09-16 23:56:29.076696308 +0000 UTC m=+1.512433343 container died 575ac6aa9ab4e2ddef75caac082034fe4f618bd9667ed09a0794f8f1f6f8c132 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T23:56:29.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.498+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d8404f2e0 con 0x7f9d84100750 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.498+0000 7f9d81ffb700 1 -- 172.21.15.42:0/2798752577 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f9d6c05c380 con 0x7f9d84100750 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:56:29.496 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 8 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:56:29.497 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:56:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9d7006e680 msgr2=0x7f9d70070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9d7006e680 0x7f9d70070b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005950 tx=0x7f9d7400b410 comp rx=0 tx=0).stop 2024-09-16T23:56:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 msgr2=0x7f9d84101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d84101d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00bd70 tx=0x7f9d6c00bda0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 shutdown_connections 2024-09-16T23:56:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9d7006e680 0x7f9d70070b40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d840ffda0 0x7f9d84101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 --2- 172.21.15.42:0/2798752577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d84100750 0x7f9d84101d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 >> 172.21.15.42:0/2798752577 conn(0x7f9d840fb650 msgr2=0x7f9d84069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.500+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 shutdown_connections 2024-09-16T23:56:29.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.501+0000 7f9d8af59700 1 -- 172.21.15.42:0/2798752577 wait complete. 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 -- 172.21.15.42:0/1485743413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 msgr2=0x7f7108100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 --2- 172.21.15.42:0/1485743413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f7108100180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009910 tx=0x7f70f8009c20 comp rx=0 tx=0).stop 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 -- 172.21.15.42:0/1485743413 shutdown_connections 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 --2- 172.21.15.42:0/1485743413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7108100750 0x7f71081047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 --2- 172.21.15.42:0/1485743413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f7108100180 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 -- 172.21.15.42:0/1485743413 >> 172.21.15.42:0/1485743413 conn(0x7f71080fb650 msgr2=0x7f71080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 -- 172.21.15.42:0/1485743413 shutdown_connections 2024-09-16T23:56:29.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.609+0000 7f710ccca700 1 -- 172.21.15.42:0/1485743413 wait complete. 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.610+0000 7f710ccca700 1 Processor -- start 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.610+0000 7f710ccca700 1 -- start start 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.610+0000 7f710ccca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f71081976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.610+0000 7f710ccca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7108100750 0x7f7108197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.610+0000 7f710ccca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108198250 con 0x7f71080ffda0 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.610+0000 7f710ccca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108198390 con 0x7f7108100750 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.610+0000 7f710659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f71081976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f710659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f71081976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35410/0 (socket says 172.21.15.42:35410) 2024-09-16T23:56:29.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f710659c700 1 -- 172.21.15.42:0/503336261 learned_addr learned my addr 172.21.15.42:0/503336261 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f7105d9b700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7108100750 0x7f7108197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f710659c700 1 -- 172.21.15.42:0/503336261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7108100750 msgr2=0x7f7108197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f710659c700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7108100750 0x7f7108197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f710659c700 1 -- 172.21.15.42:0/503336261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f80095f0 con 0x7f71080ffda0 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f710659c700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f71081976f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70f8006010 tx=0x7f70f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f70ff7fe700 1 -- 172.21.15.42:0/503336261 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f801d070 con 0x7f71080ffda0 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f70ff7fe700 1 -- 172.21.15.42:0/503336261 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70f8005dc0 con 0x7f71080ffda0 2024-09-16T23:56:29.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.611+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71081a0760 con 0x7f71080ffda0 2024-09-16T23:56:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.612+0000 7f70ff7fe700 1 -- 172.21.15.42:0/503336261 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f8017400 con 0x7f71080ffda0 2024-09-16T23:56:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.612+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71081a0b70 con 0x7f71080ffda0 2024-09-16T23:56:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.613+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7108108140 con 0x7f71080ffda0 2024-09-16T23:56:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.613+0000 7f70ff7fe700 1 -- 172.21.15.42:0/503336261 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f70f8003a60 con 0x7f71080ffda0 2024-09-16T23:56:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.613+0000 7f70ff7fe700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f70f406e680 0x7f70f4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.613+0000 7f70ff7fe700 1 -- 172.21.15.42:0/503336261 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f70f808fa30 con 0x7f71080ffda0 2024-09-16T23:56:29.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.613+0000 7f7105d9b700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f70f406e680 0x7f70f4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.614+0000 7f7105d9b700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f70f406e680 0x7f70f4070b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71080696c0 tx=0x7f70f000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:29.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.618+0000 7f70ff7fe700 1 -- 172.21.15.42:0/503336261 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f70f8058c80 con 0x7f71080ffda0 2024-09-16T23:56:29.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:29 smithi042 ceph-mon[111056]: pgmap v134: 65 pgs: 16 active+undersized, 1 stale+active+clean, 16 active+undersized+degraded, 32 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:29 smithi042 ceph-mon[111056]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:29 smithi042 ceph-mon[111056]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:29.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:29 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2798752577' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:29.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-mon[81510]: pgmap v134: 65 pgs: 16 active+undersized, 1 stale+active+clean, 16 active+undersized+degraded, 32 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:29.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-mon[81510]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:29.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-mon[81510]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:29.807 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:29 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2798752577' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:29.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.872+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7108107b20 con 0x7f71080ffda0 2024-09-16T23:56:29.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.872+0000 7f70ff7fe700 1 -- 172.21.15.42:0/503336261 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1938 (secure 0 0 0) 0x7f70f805c340 con 0x7f71080ffda0 2024-09-16T23:56:29.870 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-09-16T23:56:29.870 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:53:20:175453+0000 2024-09-16T23:56:29.870 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:53:19.510146+0000 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:56:29.871 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:56:29.872 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 24269,14478 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:56:29.873 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:56:29.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.874+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f70f406e680 msgr2=0x7f70f4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.874+0000 7f710ccca700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f70f406e680 0x7f70f4070b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71080696c0 tx=0x7f70f000b410 comp rx=0 tx=0).stop 2024-09-16T23:56:29.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.874+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 msgr2=0x7f71081976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.874+0000 7f710ccca700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f71081976f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f70f8006010 tx=0x7f70f800bd70 comp rx=0 tx=0).stop 2024-09-16T23:56:29.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.875+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 shutdown_connections 2024-09-16T23:56:29.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.875+0000 7f710ccca700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f70f406e680 0x7f70f4070b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.875+0000 7f710ccca700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f71080ffda0 0x7f71081976f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.875+0000 7f710ccca700 1 --2- 172.21.15.42:0/503336261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7108100750 0x7f7108197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.875+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 >> 172.21.15.42:0/503336261 conn(0x7f71080fb650 msgr2=0x7f71080fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:29.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.875+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 shutdown_connections 2024-09-16T23:56:29.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.875+0000 7f710ccca700 1 -- 172.21.15.42:0/503336261 wait complete. 2024-09-16T23:56:29.875 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-16T23:56:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 -- 172.21.15.42:0/1866063352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc84100750 msgr2=0x7fcc841047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 --2- 172.21.15.42:0/1866063352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc84100750 0x7fcc841047a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009910 tx=0x7fcc6c009c20 comp rx=0 tx=0).stop 2024-09-16T23:56:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 -- 172.21.15.42:0/1866063352 shutdown_connections 2024-09-16T23:56:29.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 --2- 172.21.15.42:0/1866063352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc84100750 0x7fcc841047a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 --2- 172.21.15.42:0/1866063352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc840ffda0 0x7fcc84100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 -- 172.21.15.42:0/1866063352 >> 172.21.15.42:0/1866063352 conn(0x7fcc840fb650 msgr2=0x7fcc840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 -- 172.21.15.42:0/1866063352 shutdown_connections 2024-09-16T23:56:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.984+0000 7fcc8a227700 1 -- 172.21.15.42:0/1866063352 wait complete. 2024-09-16T23:56:29.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc8a227700 1 Processor -- start 2024-09-16T23:56:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc8a227700 1 -- start start 2024-09-16T23:56:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc8a227700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc840ffda0 0x7fcc8419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc8a227700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc84100750 0x7fcc8419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc8a227700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8419d360 con 0x7fcc840ffda0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc8a227700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc84196810 con 0x7fcc84100750 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc840ffda0 0x7fcc8419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc840ffda0 0x7fcc8419c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35420/0 (socket says 172.21.15.42:35420) 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.985+0000 7fcc837fe700 1 -- 172.21.15.42:0/3749139653 learned_addr learned my addr 172.21.15.42:0/3749139653 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc82ffd700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc84100750 0x7fcc8419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc837fe700 1 -- 172.21.15.42:0/3749139653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc84100750 msgr2=0x7fcc8419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc837fe700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc84100750 0x7fcc8419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc837fe700 1 -- 172.21.15.42:0/3749139653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0095f0 con 0x7fcc840ffda0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc837fe700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc840ffda0 0x7fcc8419c790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400b4b0 tx=0x7fcc7400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc80ff9700 1 -- 172.21.15.42:0/3749139653 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc74011800 con 0x7fcc840ffda0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc80ff9700 1 -- 172.21.15.42:0/3749139653 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc74011e20 con 0x7fcc840ffda0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc84196af0 con 0x7fcc840ffda0 2024-09-16T23:56:29.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc80ff9700 1 -- 172.21.15.42:0/3749139653 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7400f2e0 con 0x7fcc840ffda0 2024-09-16T23:56:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.986+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc84197040 con 0x7fcc840ffda0 2024-09-16T23:56:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.987+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8404f2e0 con 0x7fcc840ffda0 2024-09-16T23:56:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.988+0000 7fcc80ff9700 1 -- 172.21.15.42:0/3749139653 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fcc74010380 con 0x7fcc840ffda0 2024-09-16T23:56:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.988+0000 7fcc80ff9700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc7006e680 0x7fcc70070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.988+0000 7fcc80ff9700 1 -- 172.21.15.42:0/3749139653 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcc7408e0d0 con 0x7fcc840ffda0 2024-09-16T23:56:29.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.988+0000 7fcc82ffd700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc7006e680 0x7fcc70070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:29.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.989+0000 7fcc82ffd700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc7006e680 0x7fcc70070b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c006010 tx=0x7fcc6c005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:29.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:29.992+0000 7fcc80ff9700 1 -- 172.21.15.42:0/3749139653 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcc7405af80 con 0x7fcc840ffda0 2024-09-16T23:56:30.128 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:29 smithi043 podman[96133]: 2024-09-16 23:56:29.809951382 +0000 UTC m=+0.726626680 container remove 575ac6aa9ab4e2ddef75caac082034fe4f618bd9667ed09a0794f8f1f6f8c132 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T23:56:30.128 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:30 smithi043 podman[96191]: 2024-09-16 23:56:29.928511986 +0000 UTC m=+0.016774060 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:56:30.128 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:30 smithi043 podman[96191]: 2024-09-16 23:56:30.036081442 +0000 UTC m=+0.124343506 container create bd894ca62c9c93ce2774f04122a7f23083267c186ae000aa58b08064f0ec0be1 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T23:56:30.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.225+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc84197380 con 0x7fcc7006e680 2024-09-16T23:56:30.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.227+0000 7fcc80ff9700 1 -- 172.21.15.42:0/3749139653 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fcc84197380 con 0x7fcc7006e680 2024-09-16T23:56:30.224 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:56:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:56:30.224 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "10/23 daemons upgraded", 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:56:30.225 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:56:30.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc7006e680 msgr2=0x7fcc70070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:30.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc7006e680 0x7fcc70070b40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c006010 tx=0x7fcc6c005960 comp rx=0 tx=0).stop 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc840ffda0 msgr2=0x7fcc8419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc840ffda0 0x7fcc8419c790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400b4b0 tx=0x7fcc7400b870 comp rx=0 tx=0).stop 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 shutdown_connections 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc7006e680 0x7fcc70070b40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc840ffda0 0x7fcc8419c790 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 --2- 172.21.15.42:0/3749139653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc84100750 0x7fcc8419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 >> 172.21.15.42:0/3749139653 conn(0x7fcc840fb650 msgr2=0x7fcc84104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 shutdown_connections 2024-09-16T23:56:30.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.229+0000 7fcc8a227700 1 -- 172.21.15.42:0/3749139653 wait complete. 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.344+0000 7fe8d6e07700 1 -- 172.21.15.42:0/4271905834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0101850 msgr2=0x7fe8d01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.344+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/4271905834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0101850 0x7fe8d01058a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009910 tx=0x7fe8bc009c20 comp rx=0 tx=0).stop 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 -- 172.21.15.42:0/4271905834 shutdown_connections 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/4271905834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0101850 0x7fe8d01058a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/4271905834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d0100ea0 0x7fe8d0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 -- 172.21.15.42:0/4271905834 >> 172.21.15.42:0/4271905834 conn(0x7fe8d00fc750 msgr2=0x7fe8d00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 -- 172.21.15.42:0/4271905834 shutdown_connections 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 -- 172.21.15.42:0/4271905834 wait complete. 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 Processor -- start 2024-09-16T23:56:30.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 -- start start 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0100ea0 0x7fe8d0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.345+0000 7fe8d6e07700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d0101850 0x7fe8d00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d6e07700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d0073b10 con 0x7fe8d0100ea0 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d6e07700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d0073c50 con 0x7fe8d0101850 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d4ba3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0100ea0 0x7fe8d0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d4ba3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0100ea0 0x7fe8d0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35434/0 (socket says 172.21.15.42:35434) 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d4ba3700 1 -- 172.21.15.42:0/3811254312 learned_addr learned my addr 172.21.15.42:0/3811254312 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8cffff700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d0101850 0x7fe8d00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d4ba3700 1 -- 172.21.15.42:0/3811254312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d0101850 msgr2=0x7fe8d00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:30.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d4ba3700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d0101850 0x7fe8d00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d4ba3700 1 -- 172.21.15.42:0/3811254312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc0095f0 con 0x7fe8d0100ea0 2024-09-16T23:56:30.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d4ba3700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0100ea0 0x7fe8d0073090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c930 tx=0x7fe8c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:30.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8cdffb700 1 -- 172.21.15.42:0/3811254312 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c4007ad0 con 0x7fe8d0100ea0 2024-09-16T23:56:30.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8cdffb700 1 -- 172.21.15.42:0/3811254312 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe8c400ce80 con 0x7fe8d0100ea0 2024-09-16T23:56:30.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8cdffb700 1 -- 172.21.15.42:0/3811254312 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8c4018630 con 0x7fe8d0100ea0 2024-09-16T23:56:30.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.346+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d01a2b00 con 0x7fe8d0100ea0 2024-09-16T23:56:30.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.347+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d01a2e60 con 0x7fe8d0100ea0 2024-09-16T23:56:30.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.348+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d01092a0 con 0x7fe8d0100ea0 2024-09-16T23:56:30.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.348+0000 7fe8cdffb700 1 -- 172.21.15.42:0/3811254312 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe8c4007c30 con 0x7fe8d0100ea0 2024-09-16T23:56:30.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.348+0000 7fe8cdffb700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe8c006e680 0x7fe8c0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:56:30.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.348+0000 7fe8cdffb700 1 -- 172.21.15.42:0/3811254312 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe8c408e4f0 con 0x7fe8d0100ea0 2024-09-16T23:56:30.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.348+0000 7fe8cffff700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe8c006e680 0x7fe8c0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:56:30.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.349+0000 7fe8cffff700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe8c006e680 0x7fe8c0070b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0074880 tx=0x7fe8bc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:56:30.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.353+0000 7fe8cdffb700 1 -- 172.21.15.42:0/3811254312 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe8c40577c0 con 0x7fe8d0100ea0 2024-09-16T23:56:30.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:30 smithi043 podman[96191]: 2024-09-16 23:56:30.280056805 +0000 UTC m=+0.368318879 container init bd894ca62c9c93ce2774f04122a7f23083267c186ae000aa58b08064f0ec0be1 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T23:56:30.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:30 smithi043 podman[96191]: 2024-09-16 23:56:30.283316383 +0000 UTC m=+0.371578448 container start bd894ca62c9c93ce2774f04122a7f23083267c186ae000aa58b08064f0ec0be1 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T23:56:30.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:30 smithi043 ceph-osd[96210]: -- 172.21.15.43:0/1833611484 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa3f2b81e0 con 0x55aa3e461c00 2024-09-16T23:56:30.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:30 smithi043 bash[96191]: bd894ca62c9c93ce2774f04122a7f23083267c186ae000aa58b08064f0ec0be1 2024-09-16T23:56:30.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:30 smithi043 systemd[1]: Started Ceph osd.4 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.656+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8d004f2e0 con 0x7fe8d0100ea0 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.656+0000 7fe8cdffb700 1 -- 172.21.15.42:0/3811254312 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1352 (secure 0 0 0) 0x7fe8c405ae80 con 0x7fe8d0100ea0 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 64/369 objects degraded (17.344%), 16 pgs degraded 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: osd.4 (root=default,host=smithi043) is down 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 64/369 objects degraded (17.344%), 16 pgs degraded 2024-09-16T23:56:30.654 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-09-16T23:56:30.655 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-09-16T23:56:30.656 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-09-16T23:56:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe8c006e680 msgr2=0x7fe8c0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe8c006e680 0x7fe8c0070b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0074880 tx=0x7fe8bc01a040 comp rx=0 tx=0).stop 2024-09-16T23:56:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0100ea0 msgr2=0x7fe8d0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0100ea0 0x7fe8d0073090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8c400c930 tx=0x7fe8c400cc40 comp rx=0 tx=0).stop 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 shutdown_connections 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe8c006e680 0x7fe8c0070b40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8d0100ea0 0x7fe8d0073090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 --2- 172.21.15.42:0/3811254312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d0101850 0x7fe8d00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 >> 172.21.15.42:0/3811254312 conn(0x7fe8d00fc750 msgr2=0x7fe8d00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 shutdown_connections 2024-09-16T23:56:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:56:30.659+0000 7fe8d6e07700 1 -- 172.21.15.42:0/3811254312 wait complete. 2024-09-16T23:56:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:30 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/503336261' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:56:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:30 smithi042 ceph-mon[111056]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:30 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:30 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:30 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:56:31.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:30 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/503336261' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:56:31.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:30 smithi043 ceph-mon[81510]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:56:31.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:30 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:31.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:30 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:31.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:30 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:56:31.969 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:31 smithi043 ceph-mon[81510]: pgmap v135: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:31.969 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:31 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3811254312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:56:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:31 smithi042 ceph-mon[111056]: pgmap v135: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:31 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3811254312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:56:32.377 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:32 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-16T23:56:32.173+0000 7f401722d640 -1 Falling back to public interface 2024-09-16T23:56:34.750 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:34 smithi043 ceph-mon[81510]: pgmap v136: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:34.750 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.751 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.751 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.751 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.751 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:34 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 64/369 objects degraded (17.344%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:34 smithi042 ceph-mon[111056]: pgmap v136: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:34 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 64/369 objects degraded (17.344%), 16 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:36 smithi042 ceph-mon[111056]: pgmap v137: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:36.783 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:36 smithi043 ceph-mon[81510]: pgmap v137: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:36.783 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:36.783 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:37.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:37 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-16T23:56:37.774+0000 7f401722d640 -1 osd.4 0 read_superblock omap replica is missing. 2024-09-16T23:56:38.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-16T23:56:38.122+0000 7f401722d640 -1 osd.4 69 log_to_monitors true 2024-09-16T23:56:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: pgmap v138: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='osd.4 [v2:172.21.15.43:6808/3371245465,v1:172.21.15.43:6809/3371245465]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:56:38.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:38 smithi042 ceph-mon[111056]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: pgmap v138: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:56:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:38.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:38.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:56:38.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:38.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='osd.4 [v2:172.21.15.43:6808/3371245465,v1:172.21.15.43:6809/3371245465]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:56:38.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:38 smithi043 ceph-mon[81510]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T23:56:39.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:39 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:39.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:39 smithi042 ceph-mon[111056]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T23:56:39.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:39 smithi042 ceph-mon[111056]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T23:56:39.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:39 smithi042 ceph-mon[111056]: osdmap e72: 6 total, 5 up, 6 in 2024-09-16T23:56:39.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:39 smithi042 ceph-mon[111056]: from='osd.4 [v2:172.21.15.43:6808/3371245465,v1:172.21.15.43:6809/3371245465]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:39.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:39 smithi042 ceph-mon[111056]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:39.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:39 smithi042 ceph-mon[111056]: from='osd.4 ' entity='osd.4' 2024-09-16T23:56:39.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-16T23:56:39.398+0000 7f400afbf640 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:56:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-mon[81510]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T23:56:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-mon[81510]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T23:56:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-mon[81510]: osdmap e72: 6 total, 5 up, 6 in 2024-09-16T23:56:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-mon[81510]: from='osd.4 [v2:172.21.15.43:6808/3371245465,v1:172.21.15.43:6809/3371245465]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-mon[81510]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:56:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:39 smithi043 ceph-mon[81510]: from='osd.4 ' entity='osd.4' 2024-09-16T23:56:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:40 smithi042 ceph-mon[111056]: pgmap v139: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:40 smithi042 ceph-mon[111056]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:56:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:40 smithi042 ceph-mon[111056]: osd.4 [v2:172.21.15.43:6808/3371245465,v1:172.21.15.43:6809/3371245465] boot 2024-09-16T23:56:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:40 smithi042 ceph-mon[111056]: osdmap e73: 6 total, 6 up, 6 in 2024-09-16T23:56:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:40 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:56:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:40 smithi043 ceph-mon[81510]: pgmap v139: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:40 smithi043 ceph-mon[81510]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:56:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:40 smithi043 ceph-mon[81510]: osd.4 [v2:172.21.15.43:6808/3371245465,v1:172.21.15.43:6809/3371245465] boot 2024-09-16T23:56:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:40 smithi043 ceph-mon[81510]: osdmap e73: 6 total, 6 up, 6 in 2024-09-16T23:56:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:40 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T23:56:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:42 smithi042 ceph-mon[111056]: pgmap v142: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:42 smithi042 ceph-mon[111056]: osdmap e74: 6 total, 6 up, 6 in 2024-09-16T23:56:42.843 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:42 smithi043 ceph-mon[81510]: pgmap v142: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 64/369 objects degraded (17.344%) 2024-09-16T23:56:42.843 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:42 smithi043 ceph-mon[81510]: osdmap e74: 6 total, 6 up, 6 in 2024-09-16T23:56:43.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:43 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 22/369 objects degraded (5.962%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:43.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:43 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 22/369 objects degraded (5.962%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T23:56:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:44 smithi042 ceph-mon[111056]: pgmap v144: 65 pgs: 10 peering, 7 active+undersized, 7 active+undersized+degraded, 41 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/369 objects degraded (5.962%) 2024-09-16T23:56:44.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:44 smithi043 ceph-mon[81510]: pgmap v144: 65 pgs: 10 peering, 7 active+undersized, 7 active+undersized+degraded, 41 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/369 objects degraded (5.962%) 2024-09-16T23:56:45.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:45 smithi042 ceph-mon[111056]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/369 objects degraded (5.962%), 7 pgs degraded) 2024-09-16T23:56:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:45 smithi043 ceph-mon[81510]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/369 objects degraded (5.962%), 7 pgs degraded) 2024-09-16T23:56:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:46 smithi042 ceph-mon[111056]: pgmap v145: 65 pgs: 10 peering, 55 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:46 smithi043 ceph-mon[81510]: pgmap v145: 65 pgs: 10 peering, 55 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:48 smithi042 ceph-mon[111056]: pgmap v146: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:48 smithi043 ceph-mon[81510]: pgmap v146: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:50 smithi042 ceph-mon[111056]: pgmap v147: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:50.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:50 smithi043 ceph-mon[81510]: pgmap v147: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:51 smithi043 ceph-mon[81510]: pgmap v148: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:56:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:51 smithi042 ceph-mon[111056]: pgmap v148: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:53 smithi043 ceph-mon[81510]: pgmap v149: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:53 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:53 smithi043 ceph-mon[81510]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:53.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:53 smithi043 ceph-mon[81510]: Upgrade: osd.5 is safe to restart 2024-09-16T23:56:54.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:53 smithi042 ceph-mon[111056]: pgmap v149: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:54.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:53 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:54.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:53 smithi042 ceph-mon[111056]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T23:56:54.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:53 smithi042 ceph-mon[111056]: Upgrade: osd.5 is safe to restart 2024-09-16T23:56:55.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:55 smithi042 ceph-mon[111056]: Upgrade: Updating osd.5 2024-09-16T23:56:55.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:55.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T23:56:55.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:55.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:55 smithi042 ceph-mon[111056]: Deploying daemon osd.5 on smithi043 2024-09-16T23:56:55.836 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:55 smithi043 ceph-mon[81510]: Upgrade: Updating osd.5 2024-09-16T23:56:55.836 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:55.836 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T23:56:55.836 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:56:55.836 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:55 smithi043 ceph-mon[81510]: Deploying daemon osd.5 on smithi043 2024-09-16T23:56:56.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:56 smithi042 ceph-mon[111056]: pgmap v150: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:56 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:56.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:56 smithi043 ceph-mon[81510]: pgmap v150: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:56.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:56 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:56:57.857 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:57 smithi043 systemd[1]: Stopping Ceph osd.5 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:56:58.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:57 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[54365]: 2024-09-16T23:56:57.858+0000 7fee0a003700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T23:56:58.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:57 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[54365]: 2024-09-16T23:56:57.858+0000 7fee0a003700 -1 osd.5 74 *** Got signal Terminated *** 2024-09-16T23:56:58.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:57 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[54365]: 2024-09-16T23:56:57.858+0000 7fee0a003700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T23:56:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:58 smithi042 ceph-mon[111056]: pgmap v151: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:58 smithi042 ceph-mon[111056]: osd.5 marked itself down and dead 2024-09-16T23:56:58.862 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:58 smithi043 ceph-mon[81510]: pgmap v151: 65 pgs: 65 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:56:58.862 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:58 smithi043 ceph-mon[81510]: osd.5 marked itself down and dead 2024-09-16T23:56:59.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:58 smithi043 podman[101067]: 2024-09-16 23:56:58.865639066 +0000 UTC m=+1.104410632 container died efd271d8ddb009b829dadc8e6cc98cd087fd7c3a6207de2a806b0ea9eb70a38a (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, RELEASE=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-09-16T23:56:59.424 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:59 smithi043 podman[101067]: 2024-09-16 23:56:59.242215131 +0000 UTC m=+1.480986697 container cleanup efd271d8ddb009b829dadc8e6cc98cd087fd7c3a6207de2a806b0ea9eb70a38a (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=HEAD) 2024-09-16T23:56:59.424 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:59 smithi043 bash[101067]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5 2024-09-16T23:56:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:59 smithi042 ceph-mon[111056]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:56:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:59 smithi042 ceph-mon[111056]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-16T23:56:59.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:56:59 smithi042 ceph-mon[111056]: osdmap e75: 6 total, 5 up, 6 in 2024-09-16T23:56:59.861 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:59 smithi043 ceph-mon[81510]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T23:56:59.861 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:59 smithi043 ceph-mon[81510]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-16T23:56:59.861 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:56:59 smithi043 ceph-mon[81510]: osdmap e75: 6 total, 5 up, 6 in 2024-09-16T23:56:59.862 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:59 smithi043 podman[101092]: 2024-09-16 23:56:59.427051568 +0000 UTC m=+0.560895240 container remove efd271d8ddb009b829dadc8e6cc98cd087fd7c3a6207de2a806b0ea9eb70a38a (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-09-16T23:57:00.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:59 smithi043 podman[101182]: 2024-09-16 23:56:59.767372751 +0000 UTC m=+0.016493732 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:57:00.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:56:59 smithi043 podman[101182]: 2024-09-16 23:56:59.869635865 +0000 UTC m=+0.118756882 container create 713ea3e6c70d78503cf31591710f97f0a33aa5be60997e32759ec9b5a7c45ae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T23:57:00.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 podman[101182]: 2024-09-16 23:57:00.097669799 +0000 UTC m=+0.346790772 container init 713ea3e6c70d78503cf31591710f97f0a33aa5be60997e32759ec9b5a7c45ae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:57:00.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 podman[101182]: 2024-09-16 23:57:00.101223185 +0000 UTC m=+0.350344153 container start 713ea3e6c70d78503cf31591710f97f0a33aa5be60997e32759ec9b5a7c45ae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-16T23:57:00.448 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 podman[101182]: 2024-09-16 23:57:00.197483585 +0000 UTC m=+0.446604569 container attach 713ea3e6c70d78503cf31591710f97f0a33aa5be60997e32759ec9b5a7c45ae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T23:57:00.448 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 podman[101182]: 2024-09-16 23:57:00.29586547 +0000 UTC m=+0.544986451 container died 713ea3e6c70d78503cf31591710f97f0a33aa5be60997e32759ec9b5a7c45ae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T23:57:00.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:00 smithi042 ceph-mon[111056]: pgmap v153: 65 pgs: 3 peering, 12 stale+active+clean, 50 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:00.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:00 smithi042 ceph-mon[111056]: osdmap e76: 6 total, 5 up, 6 in 2024-09-16T23:57:00.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:00 smithi042 ceph-mon[111056]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T23:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 -- 172.21.15.42:0/2645369202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 msgr2=0x7f77b81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 --2- 172.21.15.42:0/2645369202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b81070b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009910 tx=0x7f77a8009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 -- 172.21.15.42:0/2645369202 shutdown_connections 2024-09-16T23:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 --2- 172.21.15.42:0/2645369202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8068580 0x7f77b80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 --2- 172.21.15.42:0/2645369202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b81070b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 -- 172.21.15.42:0/2645369202 >> 172.21.15.42:0/2645369202 conn(0x7f77b8075670 msgr2=0x7f77b8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 -- 172.21.15.42:0/2645369202 shutdown_connections 2024-09-16T23:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.784+0000 7f77bdcad700 1 -- 172.21.15.42:0/2645369202 wait complete. 2024-09-16T23:57:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.785+0000 7f77bdcad700 1 Processor -- start 2024-09-16T23:57:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.785+0000 7f77bdcad700 1 -- start start 2024-09-16T23:57:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.785+0000 7f77bdcad700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8068580 0x7f77b80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.785+0000 7f77bdcad700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.785+0000 7f77bdcad700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b8100770 con 0x7f77b8106cd0 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.785+0000 7f77bdcad700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b81008b0 con 0x7f77b8068580 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77aeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77aeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b8100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57776/0 (socket says 172.21.15.42:57776) 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77aeffd700 1 -- 172.21.15.42:0/241254299 learned_addr learned my addr 172.21.15.42:0/241254299 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77b77fe700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8068580 0x7f77b80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77aeffd700 1 -- 172.21.15.42:0/241254299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8068580 msgr2=0x7f77b80ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77aeffd700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8068580 0x7f77b80ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77aeffd700 1 -- 172.21.15.42:0/241254299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a80095f0 con 0x7f77b8106cd0 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77aeffd700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b8100230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f77a000e920 tx=0x7f77a000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77b57fa700 1 -- 172.21.15.42:0/241254299 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a0009790 con 0x7f77b8106cd0 2024-09-16T23:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77b57fa700 1 -- 172.21.15.42:0/241254299 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77a000cba0 con 0x7f77b8106cd0 2024-09-16T23:57:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77b57fa700 1 -- 172.21.15.42:0/241254299 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a0018650 con 0x7f77b8106cd0 2024-09-16T23:57:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b81a2a60 con 0x7f77b8106cd0 2024-09-16T23:57:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.786+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b81a2e70 con 0x7f77b8106cd0 2024-09-16T23:57:00.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.787+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b81091d0 con 0x7f77b8106cd0 2024-09-16T23:57:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.788+0000 7f77b57fa700 1 -- 172.21.15.42:0/241254299 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f77a000cd10 con 0x7f77b8106cd0 2024-09-16T23:57:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.789+0000 7f77b57fa700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f77a406e680 0x7f77a4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.789+0000 7f77b57fa700 1 -- 172.21.15.42:0/241254299 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f77a0014070 con 0x7f77b8106cd0 2024-09-16T23:57:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.789+0000 7f77b77fe700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f77a406e680 0x7f77a4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:00.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.789+0000 7f77b77fe700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f77a406e680 0x7f77a4070b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b760 tx=0x7f77a8009f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:00.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:00.793+0000 7f77b57fa700 1 -- 172.21.15.42:0/241254299 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f77a00541b0 con 0x7f77b8106cd0 2024-09-16T23:57:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:00 smithi043 ceph-mon[81510]: pgmap v153: 65 pgs: 3 peering, 12 stale+active+clean, 50 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:00 smithi043 ceph-mon[81510]: osdmap e76: 6 total, 5 up, 6 in 2024-09-16T23:57:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:00 smithi043 ceph-mon[81510]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T23:57:01.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.023+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77b80611d0 con 0x7f77a406e680 2024-09-16T23:57:01.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.025+0000 7f77b57fa700 1 -- 172.21.15.42:0/241254299 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f77b80611d0 con 0x7f77a406e680 2024-09-16T23:57:01.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f77a406e680 msgr2=0x7f77a4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f77a406e680 0x7f77a4070b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b760 tx=0x7f77a8009f90 comp rx=0 tx=0).stop 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 msgr2=0x7f77b8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b8100230 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f77a000e920 tx=0x7f77a000ece0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 shutdown_connections 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f77a406e680 0x7f77a4070b40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77b8068580 0x7f77b80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 --2- 172.21.15.42:0/241254299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77b8106cd0 0x7f77b8100230 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 >> 172.21.15.42:0/241254299 conn(0x7f77b8075670 msgr2=0x7f77b80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 shutdown_connections 2024-09-16T23:57:01.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.027+0000 7f77bdcad700 1 -- 172.21.15.42:0/241254299 wait complete. 2024-09-16T23:57:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 -- 172.21.15.42:0/2011166617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 msgr2=0x7fd9281047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2011166617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd9281047a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009910 tx=0x7fd91c009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 -- 172.21.15.42:0/2011166617 shutdown_connections 2024-09-16T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2011166617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd9281047a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2011166617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9280ffda0 0x7fd928100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 -- 172.21.15.42:0/2011166617 >> 172.21.15.42:0/2011166617 conn(0x7fd9280fb650 msgr2=0x7fd9280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:01.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 -- 172.21.15.42:0/2011166617 shutdown_connections 2024-09-16T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 -- 172.21.15.42:0/2011166617 wait complete. 2024-09-16T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 Processor -- start 2024-09-16T23:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.137+0000 7fd92e76c700 1 -- start start 2024-09-16T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd92e76c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9280ffda0 0x7fd92819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd92e76c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd92819bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd92e76c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92819c1f0 con 0x7fd928100750 2024-09-16T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd92e76c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9281956a0 con 0x7fd9280ffda0 2024-09-16T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd9277fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd92819bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd9277fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd92819bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57794/0 (socket says 172.21.15.42:57794) 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd927fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9280ffda0 0x7fd92819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd9277fe700 1 -- 172.21.15.42:0/2346229096 learned_addr learned my addr 172.21.15.42:0/2346229096 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd927fff700 1 -- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 msgr2=0x7fd92819bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd927fff700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd92819bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd927fff700 1 -- 172.21.15.42:0/2346229096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0095f0 con 0x7fd9280ffda0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd9277fe700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd92819bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.138+0000 7fd927fff700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9280ffda0 0x7fd92819b620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd91800e970 tx=0x7fd91800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.139+0000 7fd9257fa700 1 -- 172.21.15.42:0/2346229096 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91800c9d0 con 0x7fd9280ffda0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.139+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd928195980 con 0x7fd9280ffda0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.139+0000 7fd9257fa700 1 -- 172.21.15.42:0/2346229096 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd918010430 con 0x7fd9280ffda0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.139+0000 7fd9257fa700 1 -- 172.21.15.42:0/2346229096 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd918018630 con 0x7fd9280ffda0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.139+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd928195ed0 con 0x7fd9280ffda0 2024-09-16T23:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.140+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9281081a0 con 0x7fd9280ffda0 2024-09-16T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.141+0000 7fd9257fa700 1 -- 172.21.15.42:0/2346229096 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd91803cba0 con 0x7fd9280ffda0 2024-09-16T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.141+0000 7fd9257fa700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd91006e680 0x7fd910070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.141+0000 7fd9257fa700 1 -- 172.21.15.42:0/2346229096 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd918014070 con 0x7fd9280ffda0 2024-09-16T23:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.141+0000 7fd9277fe700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd91006e680 0x7fd910070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.142+0000 7fd9277fe700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd91006e680 0x7fd910070b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd9280fbec0 tx=0x7fd91c005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:01.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.145+0000 7fd9257fa700 1 -- 172.21.15.42:0/2346229096 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd91805b610 con 0x7fd9280ffda0 2024-09-16T23:57:01.160 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 podman[101206]: 2024-09-16 23:57:00.909608951 +0000 UTC m=+0.607889102 container remove 713ea3e6c70d78503cf31591710f97f0a33aa5be60997e32759ec9b5a7c45ae5 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T23:57:01.160 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-16T23:57:01.160 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 systemd[1]: Stopped Ceph osd.5 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:57:01.160 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:00 smithi043 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.5.service: Consumed 57.696s CPU time. 2024-09-16T23:57:01.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.379+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9281960f0 con 0x7fd91006e680 2024-09-16T23:57:01.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.380+0000 7fd9257fa700 1 -- 172.21.15.42:0/2346229096 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd9281960f0 con 0x7fd91006e680 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd91006e680 msgr2=0x7fd910070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd91006e680 0x7fd910070b40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd9280fbec0 tx=0x7fd91c005a90 comp rx=0 tx=0).stop 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9280ffda0 msgr2=0x7fd92819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9280ffda0 0x7fd92819b620 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd91800e970 tx=0x7fd91800ec80 comp rx=0 tx=0).stop 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 shutdown_connections 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd91006e680 0x7fd910070b40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9280ffda0 0x7fd92819b620 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 --2- 172.21.15.42:0/2346229096 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd928100750 0x7fd92819bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 >> 172.21.15.42:0/2346229096 conn(0x7fd9280fb650 msgr2=0x7fd9280fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:01.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 shutdown_connections 2024-09-16T23:57:01.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.382+0000 7fd92e76c700 1 -- 172.21.15.42:0/2346229096 wait complete. 2024-09-16T23:57:01.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.490+0000 7f8096831700 1 -- 172.21.15.42:0/1931143421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80900ff970 msgr2=0x7f80900ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.490+0000 7f8096831700 1 --2- 172.21.15.42:0/1931143421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80900ff970 0x7f80900ffd50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8080009910 tx=0x7f8080009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:01.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.490+0000 7f8096831700 1 -- 172.21.15.42:0/1931143421 shutdown_connections 2024-09-16T23:57:01.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.490+0000 7f8096831700 1 --2- 172.21.15.42:0/1931143421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8090100320 0x7f809010cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.490+0000 7f8096831700 1 --2- 172.21.15.42:0/1931143421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80900ff970 0x7f80900ffd50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.490+0000 7f8096831700 1 -- 172.21.15.42:0/1931143421 >> 172.21.15.42:0/1931143421 conn(0x7f80900fb650 msgr2=0x7f80900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:01.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.491+0000 7f8096831700 1 -- 172.21.15.42:0/1931143421 shutdown_connections 2024-09-16T23:57:01.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.491+0000 7f8096831700 1 -- 172.21.15.42:0/1931143421 wait complete. 2024-09-16T23:57:01.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.491+0000 7f8096831700 1 Processor -- start 2024-09-16T23:57:01.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.491+0000 7f8096831700 1 -- start start 2024-09-16T23:57:01.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f8096831700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80900ff970 0x7f809019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f8096831700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8090100320 0x7f809019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f8096831700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809019f440 con 0x7f8090100320 2024-09-16T23:57:01.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f8096831700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80901988f0 con 0x7f80900ff970 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8090100320 0x7f809019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8090100320 0x7f809019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57810/0 (socket says 172.21.15.42:57810) 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808f7fe700 1 -- 172.21.15.42:0/1553672199 learned_addr learned my addr 172.21.15.42:0/1553672199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808ffff700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80900ff970 0x7f809019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808f7fe700 1 -- 172.21.15.42:0/1553672199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80900ff970 msgr2=0x7f809019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808f7fe700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80900ff970 0x7f809019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808f7fe700 1 -- 172.21.15.42:0/1553672199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800095f0 con 0x7f8090100320 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808f7fe700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8090100320 0x7f809019edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f808400e920 tx=0x7f808400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808d7fa700 1 -- 172.21.15.42:0/1553672199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f808400c9d0 con 0x7f8090100320 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808d7fa700 1 -- 172.21.15.42:0/1553672199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8084010430 con 0x7f8090100320 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8090198bd0 con 0x7f8090100320 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f808d7fa700 1 -- 172.21.15.42:0/1553672199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8084018630 con 0x7f8090100320 2024-09-16T23:57:01.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.492+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8090199120 con 0x7f8090100320 2024-09-16T23:57:01.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.493+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809010a750 con 0x7f8090100320 2024-09-16T23:57:01.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.494+0000 7f808d7fa700 1 -- 172.21.15.42:0/1553672199 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f80840105a0 con 0x7f8090100320 2024-09-16T23:57:01.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.494+0000 7f808d7fa700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f807806e680 0x7f8078070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.495+0000 7f808d7fa700 1 -- 172.21.15.42:0/1553672199 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8084014070 con 0x7f8090100320 2024-09-16T23:57:01.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.495+0000 7f808ffff700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f807806e680 0x7f8078070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.495+0000 7f808ffff700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f807806e680 0x7f8078070b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8080009fd0 tx=0x7f8080005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:01.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.499+0000 7f808d7fa700 1 -- 172.21.15.42:0/1553672199 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f80840541b0 con 0x7f8090100320 2024-09-16T23:57:01.507 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 systemd[1]: Starting Ceph osd.5 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-16T23:57:01.507 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 podman[101304]: 2024-09-16 23:57:01.278779585 +0000 UTC m=+0.016640500 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:57:01.507 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 podman[101304]: 2024-09-16 23:57:01.402937545 +0000 UTC m=+0.140798462 container create 085623a5be15d3abcb1a3e327961c745c14f149fc65443925e97438cfc43916a (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T23:57:01.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.716+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8090199340 con 0x7f807806e680 2024-09-16T23:57:01.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.726+0000 7f808d7fa700 1 -- 172.21.15.42:0/1553672199 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8090199340 con 0x7f807806e680 2024-09-16T23:57:01.724 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:57:01.724 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (5m) 94s ago 16m 24.5M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:57:01.724 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (16m) 94s ago 16m 8564k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (15m) 28s ago 15m 8363k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (3m) 94s ago 16m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (3m) 28s ago 15m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (5m) 94s ago 16m 80.7M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (10m) 94s ago 10m 131M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (10m) 94s ago 10m 16.9M - 18.2.1 5be31c24972a 04bd30e094b0 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (10m) 28s ago 10m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (10m) 28s ago 10m 18.3M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (7m) 94s ago 17m 500M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (6m) 28s ago 15m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (4m) 94s ago 18m 50.9M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:57:01.725 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (3m) 28s ago 15m 40.6M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (6m) 94s ago 16m 16.2M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (6m) 28s ago 15m 15.6M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (2m) 94s ago 14m 83.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (2m) 94s ago 14m 82.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (100s) 94s ago 13m 11.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (64s) 28s ago 13m 86.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (31s) 28s ago 12m 11.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (12m) 28s ago 12m 206M 2241M 18.2.1 5be31c24972a efd271d8ddb0 2024-09-16T23:57:01.726 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (5m) 94s ago 16m 76.3M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f807806e680 msgr2=0x7f8078070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f807806e680 0x7f8078070b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8080009fd0 tx=0x7f8080005a90 comp rx=0 tx=0).stop 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8090100320 msgr2=0x7f809019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8090100320 0x7f809019edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f808400e920 tx=0x7f808400ec30 comp rx=0 tx=0).stop 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 shutdown_connections 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f807806e680 0x7f8078070b40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80900ff970 0x7f809019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 --2- 172.21.15.42:0/1553672199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8090100320 0x7f809019edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 >> 172.21.15.42:0/1553672199 conn(0x7f80900fb650 msgr2=0x7f80900fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 shutdown_connections 2024-09-16T23:57:01.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.729+0000 7f8096831700 1 -- 172.21.15.42:0/1553672199 wait complete. 2024-09-16T23:57:01.757 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 podman[101304]: 2024-09-16 23:57:01.62343116 +0000 UTC m=+0.361292077 container init 085623a5be15d3abcb1a3e327961c745c14f149fc65443925e97438cfc43916a (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T23:57:01.757 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 podman[101304]: 2024-09-16 23:57:01.62690481 +0000 UTC m=+0.364765725 container start 085623a5be15d3abcb1a3e327961c745c14f149fc65443925e97438cfc43916a (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T23:57:01.758 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 podman[101304]: 2024-09-16 23:57:01.725167155 +0000 UTC m=+0.463028079 container attach 085623a5be15d3abcb1a3e327961c745c14f149fc65443925e97438cfc43916a (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 -- 172.21.15.42:0/976924517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184101e00 msgr2=0x7f21841021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 --2- 172.21.15.42:0/976924517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184101e00 0x7f21841021e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2174009910 tx=0x7f2174009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 -- 172.21.15.42:0/976924517 shutdown_connections 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 --2- 172.21.15.42:0/976924517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184102720 0x7f218410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 --2- 172.21.15.42:0/976924517 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184101e00 0x7f21841021e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 -- 172.21.15.42:0/976924517 >> 172.21.15.42:0/976924517 conn(0x7f21840fb650 msgr2=0x7f21840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 -- 172.21.15.42:0/976924517 shutdown_connections 2024-09-16T23:57:01.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.835+0000 7f2188f72700 1 -- 172.21.15.42:0/976924517 wait complete. 2024-09-16T23:57:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2188f72700 1 Processor -- start 2024-09-16T23:57:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2188f72700 1 -- start start 2024-09-16T23:57:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2188f72700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184101e00 0x7f218419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2188f72700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184102720 0x7f218419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2188f72700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218419d260 con 0x7f2184102720 2024-09-16T23:57:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2188f72700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2184196710 con 0x7f2184101e00 2024-09-16T23:57:01.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2181d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184102720 0x7f218419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2181d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184102720 0x7f218419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57830/0 (socket says 172.21.15.42:57830) 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f2181d9b700 1 -- 172.21.15.42:0/1988017128 learned_addr learned my addr 172.21.15.42:0/1988017128 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.836+0000 7f218259c700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184101e00 0x7f218419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f2181d9b700 1 -- 172.21.15.42:0/1988017128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184101e00 msgr2=0x7f218419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f2181d9b700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184101e00 0x7f218419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f2181d9b700 1 -- 172.21.15.42:0/1988017128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740095f0 con 0x7f2184102720 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f218259c700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184101e00 0x7f218419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f2181d9b700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184102720 0x7f218419cbd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f217800e8c0 tx=0x7f217800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f21737fe700 1 -- 172.21.15.42:0/1988017128 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21780041f0 con 0x7f2184102720 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f21737fe700 1 -- 172.21.15.42:0/1988017128 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f217800cca0 con 0x7f2184102720 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21841969f0 con 0x7f2184102720 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f21737fe700 1 -- 172.21.15.42:0/1988017128 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2178003de0 con 0x7f2184102720 2024-09-16T23:57:01.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.837+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2184196f40 con 0x7f2184102720 2024-09-16T23:57:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.838+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2184108310 con 0x7f2184102720 2024-09-16T23:57:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.839+0000 7f21737fe700 1 -- 172.21.15.42:0/1988017128 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f217800c7e0 con 0x7f2184102720 2024-09-16T23:57:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.839+0000 7f21737fe700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f216c06e680 0x7f216c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:01.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.839+0000 7f218259c700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f216c06e680 0x7f216c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.839+0000 7f21737fe700 1 -- 172.21.15.42:0/1988017128 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2178014070 con 0x7f2184102720 2024-09-16T23:57:01.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.840+0000 7f218259c700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f216c06e680 0x7f216c070b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f217400b760 tx=0x7f2174005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:01.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:01.844+0000 7f21737fe700 1 -- 172.21.15.42:0/1988017128 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f217805b730 con 0x7f2184102720 2024-09-16T23:57:02.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:57:02.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 bash[101304]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:57:02.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:57:02.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:01 smithi043 bash[101304]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T23:57:02.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.144+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f218404f2e0 con 0x7f2184102720 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.144+0000 7f21737fe700 1 -- 172.21.15.42:0/1988017128 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+708 (secure 0 0 0) 0x7f2178004550 con 0x7f2184102720 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:57:02.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 5 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 9 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:57:02.143 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f216c06e680 msgr2=0x7f216c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f216c06e680 0x7f216c070b40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f217400b760 tx=0x7f2174005940 comp rx=0 tx=0).stop 2024-09-16T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184102720 msgr2=0x7f218419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184102720 0x7f218419cbd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f217800e8c0 tx=0x7f217800ebd0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 shutdown_connections 2024-09-16T23:57:02.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f216c06e680 0x7f216c070b40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2184101e00 0x7f218419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 --2- 172.21.15.42:0/1988017128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2184102720 0x7f218419cbd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.146+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 >> 172.21.15.42:0/1988017128 conn(0x7f21840fb650 msgr2=0x7f2184105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.147+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 shutdown_connections 2024-09-16T23:57:02.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.147+0000 7f2188f72700 1 -- 172.21.15.42:0/1988017128 wait complete. 2024-09-16T23:57:02.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.254+0000 7f51c7ddb700 1 -- 172.21.15.42:0/509938885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c00ffb40 msgr2=0x7f51c00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.254+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/509938885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c00ffb40 0x7f51c00fffc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009910 tx=0x7f51b0009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 -- 172.21.15.42:0/509938885 shutdown_connections 2024-09-16T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/509938885 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c00ffb40 0x7f51c00fffc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/509938885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c0105bf0 0x7f51c0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 -- 172.21.15.42:0/509938885 >> 172.21.15.42:0/509938885 conn(0x7f51c00fb650 msgr2=0x7f51c00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 -- 172.21.15.42:0/509938885 shutdown_connections 2024-09-16T23:57:02.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 -- 172.21.15.42:0/509938885 wait complete. 2024-09-16T23:57:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 Processor -- start 2024-09-16T23:57:02.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.255+0000 7f51c7ddb700 1 -- start start 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c7ddb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c00ffb40 0x7f51c01017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c7ddb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c0105bf0 0x7f51c0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c7ddb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0102230 con 0x7f51c0105bf0 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c7ddb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0102370 con 0x7f51c00ffb40 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5376700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c0105bf0 0x7f51c0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5376700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c0105bf0 0x7f51c0101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57846/0 (socket says 172.21.15.42:57846) 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5376700 1 -- 172.21.15.42:0/4175025560 learned_addr learned my addr 172.21.15.42:0/4175025560 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5b77700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c00ffb40 0x7f51c01017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5376700 1 -- 172.21.15.42:0/4175025560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c00ffb40 msgr2=0x7f51c01017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5376700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c00ffb40 0x7f51c01017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5376700 1 -- 172.21.15.42:0/4175025560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00095f0 con 0x7f51c0105bf0 2024-09-16T23:57:02.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5b77700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c00ffb40 0x7f51c01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:57:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c5376700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c0105bf0 0x7f51c0101cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51b0006010 tx=0x7f51b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51b6ffd700 1 -- 172.21.15.42:0/4175025560 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b001d070 con 0x7f51c0105bf0 2024-09-16T23:57:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51b6ffd700 1 -- 172.21.15.42:0/4175025560 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51b0004fb0 con 0x7f51c0105bf0 2024-09-16T23:57:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51b6ffd700 1 -- 172.21.15.42:0/4175025560 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51b0022ea0 con 0x7f51c0105bf0 2024-09-16T23:57:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.256+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c01025f0 con 0x7f51c0105bf0 2024-09-16T23:57:02.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.257+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c01a2e70 con 0x7f51c0105bf0 2024-09-16T23:57:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.257+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c004f2e0 con 0x7f51c0105bf0 2024-09-16T23:57:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.258+0000 7f51b6ffd700 1 -- 172.21.15.42:0/4175025560 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f51b0022470 con 0x7f51c0105bf0 2024-09-16T23:57:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.258+0000 7f51b6ffd700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f51ac06e680 0x7f51ac070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.259+0000 7f51b6ffd700 1 -- 172.21.15.42:0/4175025560 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f51b008fa30 con 0x7f51c0105bf0 2024-09-16T23:57:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.259+0000 7f51c5b77700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f51ac06e680 0x7f51ac070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.259+0000 7f51c5b77700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f51ac06e680 0x7f51ac070b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0099d0 tx=0x7f51bc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:02.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.263+0000 7f51b6ffd700 1 -- 172.21.15.42:0/4175025560 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f51b0058db0 con 0x7f51c0105bf0 2024-09-16T23:57:02.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.517+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f51c0102ce0 con 0x7f51c0105bf0 2024-09-16T23:57:02.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.518+0000 7f51b6ffd700 1 -- 172.21.15.42:0/4175025560 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1938 (secure 0 0 0) 0x7f51b0027070 con 0x7f51c0105bf0 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:e19 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:53:20:175453+0000 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 19 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T23:57:02.516 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:53:19.510146+0000 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 46 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:57:02.517 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=14478,1=24269} 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 14478 members: 24269,14478 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{0:14478} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.42:6828/1702969563,v1:172.21.15.42:6829/1702969563] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{1:24269} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.43:6824/3963651257,v1:172.21.15.43:6825/3963651257] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:57:02.518 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:02.519 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:02.519 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:57:02.519 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:02.519 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:14490} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/3253599157,v1:172.21.15.42:6827/3253599157] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:57:02.519 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f51ac06e680 msgr2=0x7f51ac070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f51ac06e680 0x7f51ac070b40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51bc0099d0 tx=0x7f51bc008040 comp rx=0 tx=0).stop 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c0105bf0 msgr2=0x7f51c0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c0105bf0 0x7f51c0101cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51b0006010 tx=0x7f51b000be20 comp rx=0 tx=0).stop 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 shutdown_connections 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f51ac06e680 0x7f51ac070b40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51c00ffb40 0x7f51c01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 --2- 172.21.15.42:0/4175025560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f51c0105bf0 0x7f51c0101cf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 >> 172.21.15.42:0/4175025560 conn(0x7f51c00fb650 msgr2=0x7f51c0068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 shutdown_connections 2024-09-16T23:57:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.520+0000 7f51c7ddb700 1 -- 172.21.15.42:0/4175025560 wait complete. 2024-09-16T23:57:02.521 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-16T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 -- 172.21.15.42:0/1723081587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1660100750 msgr2=0x7f16601047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 --2- 172.21.15.42:0/1723081587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1660100750 0x7f16601047a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f164c009910 tx=0x7f164c009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 -- 172.21.15.42:0/1723081587 shutdown_connections 2024-09-16T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 --2- 172.21.15.42:0/1723081587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1660100750 0x7f16601047a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 --2- 172.21.15.42:0/1723081587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600ffda0 0x7f1660100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 -- 172.21.15.42:0/1723081587 >> 172.21.15.42:0/1723081587 conn(0x7f16600fb650 msgr2=0x7f16600fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 -- 172.21.15.42:0/1723081587 shutdown_connections 2024-09-16T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 -- 172.21.15.42:0/1723081587 wait complete. 2024-09-16T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.629+0000 7f1666a9b700 1 Processor -- start 2024-09-16T23:57:02.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1666a9b700 1 -- start start 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1666a9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16600ffda0 0x7f1660101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1666a9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1660100750 0x7f1660101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1666a9b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16601022c0 con 0x7f16600ffda0 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1666a9b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1660102400 con 0x7f1660100750 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1664837700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16600ffda0 0x7f1660101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1664837700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16600ffda0 0x7f1660101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38928/0 (socket says 172.21.15.42:38928) 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1664837700 1 -- 172.21.15.42:0/3106653389 learned_addr learned my addr 172.21.15.42:0/3106653389 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f165ffff700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1660100750 0x7f1660101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1664837700 1 -- 172.21.15.42:0/3106653389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1660100750 msgr2=0x7f1660101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1664837700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1660100750 0x7f1660101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1664837700 1 -- 172.21.15.42:0/3106653389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c0095f0 con 0x7f16600ffda0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f165ffff700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1660100750 0x7f1660101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.630+0000 7f1664837700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16600ffda0 0x7f1660101840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f165400e920 tx=0x7f165400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.631+0000 7f165dffb700 1 -- 172.21.15.42:0/3106653389 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165400c9d0 con 0x7f16600ffda0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.631+0000 7f165dffb700 1 -- 172.21.15.42:0/3106653389 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1654010430 con 0x7f16600ffda0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.631+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16601a2a70 con 0x7f16600ffda0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.631+0000 7f165dffb700 1 -- 172.21.15.42:0/3106653389 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1654018630 con 0x7f16600ffda0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.631+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16601a2e20 con 0x7f16600ffda0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.632+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16601091b0 con 0x7f16600ffda0 2024-09-16T23:57:02.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.632+0000 7f165dffb700 1 -- 172.21.15.42:0/3106653389 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f16540105a0 con 0x7f16600ffda0 2024-09-16T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.633+0000 7f165dffb700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f165006e680 0x7f1650070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.633+0000 7f165dffb700 1 -- 172.21.15.42:0/3106653389 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1654014070 con 0x7f16600ffda0 2024-09-16T23:57:02.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.633+0000 7f165ffff700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f165006e680 0x7f1650070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.633+0000 7f165ffff700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f165006e680 0x7f1650070b40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f164c0059d0 tx=0x7f164c005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:02.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.637+0000 7f165dffb700 1 -- 172.21.15.42:0/3106653389 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f16540541b0 con 0x7f16600ffda0 2024-09-16T23:57:02.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:02 smithi042 ceph-mon[111056]: pgmap v155: 65 pgs: 15 peering, 5 stale+active+clean, 45 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:02 smithi042 ceph-mon[111056]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:02 smithi042 ceph-mon[111056]: from='client.44279 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:02 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1988017128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:02.791 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-mon[81510]: pgmap v155: 65 pgs: 15 peering, 5 stale+active+clean, 45 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:02.791 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-mon[81510]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:02.791 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-mon[81510]: from='client.44279 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:02.791 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1988017128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:02.792 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T23:57:02.792 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 bash[101304]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T23:57:02.792 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T23:57:02.792 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 bash[101304]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T23:57:02.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.865+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16601026b0 con 0x7f165006e680 2024-09-16T23:57:02.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.867+0000 7f165dffb700 1 -- 172.21.15.42:0/3106653389 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f16601026b0 con 0x7f165006e680 2024-09-16T23:57:02.864 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:57:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:57:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:57:02.864 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "11/23 daemons upgraded", 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:57:02.865 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f165006e680 msgr2=0x7f1650070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f165006e680 0x7f1650070b40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f164c0059d0 tx=0x7f164c005940 comp rx=0 tx=0).stop 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16600ffda0 msgr2=0x7f1660101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16600ffda0 0x7f1660101840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f165400e920 tx=0x7f165400ec30 comp rx=0 tx=0).stop 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 shutdown_connections 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f165006e680 0x7f1650070b40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f16600ffda0 0x7f1660101840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 --2- 172.21.15.42:0/3106653389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1660100750 0x7f1660101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 >> 172.21.15.42:0/3106653389 conn(0x7f16600fb650 msgr2=0x7f1660069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 shutdown_connections 2024-09-16T23:57:02.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.869+0000 7f1666a9b700 1 -- 172.21.15.42:0/3106653389 wait complete. 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 -- 172.21.15.42:0/933557906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b4105bd0 msgr2=0x7fb1b4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 --2- 172.21.15.42:0/933557906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b4105bd0 0x7fb1b4105fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b0a0 tx=0x7fb1a400b3b0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 -- 172.21.15.42:0/933557906 shutdown_connections 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 --2- 172.21.15.42:0/933557906 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b40ffc20 0x7fb1b4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 --2- 172.21.15.42:0/933557906 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b4105bd0 0x7fb1b4105fb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 -- 172.21.15.42:0/933557906 >> 172.21.15.42:0/933557906 conn(0x7fb1b4075670 msgr2=0x7fb1b4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 -- 172.21.15.42:0/933557906 shutdown_connections 2024-09-16T23:57:02.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.977+0000 7fb1b9394700 1 -- 172.21.15.42:0/933557906 wait complete. 2024-09-16T23:57:02.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b9394700 1 Processor -- start 2024-09-16T23:57:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b9394700 1 -- start start 2024-09-16T23:57:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b9394700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b40ffc20 0x7fb1b419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b9394700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4105bd0 0x7fb1b419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b9394700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b419d1d0 con 0x7fb1b40ffc20 2024-09-16T23:57:02.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b9394700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4196710 con 0x7fb1b4105bd0 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b40ffc20 0x7fb1b419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b40ffc20 0x7fb1b419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38946/0 (socket says 172.21.15.42:38946) 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b2ffd700 1 -- 172.21.15.42:0/3902447251 learned_addr learned my addr 172.21.15.42:0/3902447251 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b27fc700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4105bd0 0x7fb1b419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b2ffd700 1 -- 172.21.15.42:0/3902447251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4105bd0 msgr2=0x7fb1b419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b2ffd700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4105bd0 0x7fb1b419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.978+0000 7fb1b2ffd700 1 -- 172.21.15.42:0/3902447251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009d00 con 0x7fb1b40ffc20 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.979+0000 7fb1b2ffd700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b40ffc20 0x7fb1b419c690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b070 tx=0x7fb1a400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.979+0000 7fb19bfff700 1 -- 172.21.15.42:0/3902447251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1a400e040 con 0x7fb1b40ffc20 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.979+0000 7fb19bfff700 1 -- 172.21.15.42:0/3902447251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1a4003e60 con 0x7fb1b40ffc20 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.979+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b4196990 con 0x7fb1b40ffc20 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.979+0000 7fb19bfff700 1 -- 172.21.15.42:0/3902447251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1a401d910 con 0x7fb1b40ffc20 2024-09-16T23:57:02.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.979+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b4196e80 con 0x7fb1b40ffc20 2024-09-16T23:57:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.980+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b404f2e0 con 0x7fb1b40ffc20 2024-09-16T23:57:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.980+0000 7fb19bfff700 1 -- 172.21.15.42:0/3902447251 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb1a4004820 con 0x7fb1b40ffc20 2024-09-16T23:57:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.981+0000 7fb19bfff700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb19c06e680 0x7fb19c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.981+0000 7fb19bfff700 1 -- 172.21.15.42:0/3902447251 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb1a408fe20 con 0x7fb1b40ffc20 2024-09-16T23:57:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.981+0000 7fb1b27fc700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb19c06e680 0x7fb19c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.981+0000 7fb1b27fc700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb19c06e680 0x7fb19c070b40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8005ea0 tx=0x7fb1a8005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:02.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:02.985+0000 7fb19bfff700 1 -- 172.21.15.42:0/3902447251 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb1a405cc20 con 0x7fb1b40ffc20 2024-09-16T23:57:03.064 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 bash[101304]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 bash[101304]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 bash[101304]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 bash[101304]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate[101319]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 bash[101304]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-16T23:57:03.065 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:02 smithi043 podman[101304]: 2024-09-16 23:57:02.824401177 +0000 UTC m=+1.562262093 container died 085623a5be15d3abcb1a3e327961c745c14f149fc65443925e97438cfc43916a (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T23:57:03.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.280+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb1b4197c20 con 0x7fb1b40ffc20 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.280+0000 7fb19bfff700 1 -- 172.21.15.42:0/3902447251 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1253 (secure 0 0 0) 0x7fb1a4017500 con 0x7fb1b40ffc20 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Reduced data availability: 8 pgs peering 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout: osd.5 (root=default,host=smithi043) is down 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-09-16T23:57:03.278 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 8 pgs peering 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.2 is stuck peering for 7m, current state peering, last acting [1,0] 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.3 is stuck peering for 6m, current state peering, last acting [2,1] 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.17 is stuck peering for 2m, current state peering, last acting [2,1] 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 2.18 is stuck peering for 8m, current state peering, last acting [4,3] 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 3.5 is stuck peering for 6m, current state peering, last acting [2,3] 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 3.e is stuck peering for 6m, current state peering, last acting [4,1] 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 3.1c is stuck peering for 6m, current state peering, last acting [4,1] 2024-09-16T23:57:03.279 INFO:teuthology.orchestra.run.smithi042.stdout: pg 3.1d is stuck peering for 6m, current state peering, last acting [4,1] 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb19c06e680 msgr2=0x7fb19c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb19c06e680 0x7fb19c070b40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8005ea0 tx=0x7fb1a8005e30 comp rx=0 tx=0).stop 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b40ffc20 msgr2=0x7fb1b419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b40ffc20 0x7fb1b419c690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b070 tx=0x7fb1a400cdd0 comp rx=0 tx=0).stop 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 shutdown_connections 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb19c06e680 0x7fb19c070b40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb1b40ffc20 0x7fb1b419c690 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:03.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 --2- 172.21.15.42:0/3902447251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b4105bd0 0x7fb1b419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:03.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.282+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 >> 172.21.15.42:0/3902447251 conn(0x7fb1b4075670 msgr2=0x7fb1b4103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:03.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.283+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 shutdown_connections 2024-09-16T23:57:03.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:03.283+0000 7fb1b9394700 1 -- 172.21.15.42:0/3902447251 wait complete. 2024-09-16T23:57:03.733 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:03 smithi043 ceph-mon[81510]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:03.733 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:03 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4175025560' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:57:03.733 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:03 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3902447251' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:57:03.734 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:03 smithi043 podman[101457]: 2024-09-16 23:57:03.493532283 +0000 UTC m=+0.662793362 container remove 085623a5be15d3abcb1a3e327961c745c14f149fc65443925e97438cfc43916a (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T23:57:03.734 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:03 smithi043 podman[101518]: 2024-09-16 23:57:03.61021234 +0000 UTC m=+0.016757388 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-16T23:57:03.734 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:03 smithi043 podman[101518]: 2024-09-16 23:57:03.736612341 +0000 UTC m=+0.143157387 container create dc14c8f3f72c5ef14a0af27c8deec68367633e606ccba4681c3ba2ae47cbcc71 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T23:57:03.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:03 smithi042 ceph-mon[111056]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:03.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:03 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4175025560' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:57:03.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:03 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3902447251' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:57:04.053 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:03 smithi043 podman[101518]: 2024-09-16 23:57:03.951764112 +0000 UTC m=+0.358309166 container init dc14c8f3f72c5ef14a0af27c8deec68367633e606ccba4681c3ba2ae47cbcc71 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T23:57:04.053 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:03 smithi043 podman[101518]: 2024-09-16 23:57:03.955173586 +0000 UTC m=+0.361718631 container start dc14c8f3f72c5ef14a0af27c8deec68367633e606ccba4681c3ba2ae47cbcc71 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T23:57:04.053 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:03 smithi043 ceph-osd[101537]: -- 172.21.15.43:0/2687806484 <== mon.1 v2:172.21.15.43:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5614c38001e0 con 0x5614c3796000 2024-09-16T23:57:04.053 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:04 smithi043 bash[101518]: dc14c8f3f72c5ef14a0af27c8deec68367633e606ccba4681c3ba2ae47cbcc71 2024-09-16T23:57:04.377 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:04 smithi043 systemd[1]: Started Ceph osd.5 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-16T23:57:04.610 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:04 smithi042 ceph-mon[111056]: pgmap v156: 65 pgs: 7 active+undersized, 15 peering, 6 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/369 objects degraded (8.672%) 2024-09-16T23:57:04.610 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:04 smithi042 ceph-mon[111056]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:04.610 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:04 smithi042 ceph-mon[111056]: Health check failed: Degraded data redundancy: 32/369 objects degraded (8.672%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T23:57:04.610 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:04.610 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:04.610 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:04 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:04.734 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:04 smithi043 ceph-mon[81510]: pgmap v156: 65 pgs: 7 active+undersized, 15 peering, 6 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 32/369 objects degraded (8.672%) 2024-09-16T23:57:04.734 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:04 smithi043 ceph-mon[81510]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:04.734 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:04 smithi043 ceph-mon[81510]: Health check failed: Degraded data redundancy: 32/369 objects degraded (8.672%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T23:57:04.734 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:04.734 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:04.734 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:04 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:05.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:05 smithi042 ceph-mon[111056]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 8 pgs peering) 2024-09-16T23:57:05.834 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:05 smithi043 ceph-mon[81510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 8 pgs peering) 2024-09-16T23:57:06.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:05 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-16T23:57:05.835+0000 7f14bb1c7640 -1 Falling back to public interface 2024-09-16T23:57:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:06 smithi042 ceph-mon[111056]: pgmap v157: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:06.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:06 smithi043 ceph-mon[81510]: pgmap v157: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:07 smithi042 ceph-mon[111056]: pgmap v158: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:07 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:07 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:07 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:07 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:08.280 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:07 smithi043 ceph-mon[81510]: pgmap v158: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:08.280 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:07 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:08.280 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:07 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:08.281 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:07 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:08.281 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:07 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:10.210 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:09 smithi043 ceph-mon[81510]: pgmap v159: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:10.211 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:09 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 56/369 objects degraded (15.176%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T23:57:10.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:09 smithi042 ceph-mon[111056]: pgmap v159: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:10.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:09 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 56/369 objects degraded (15.176%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T23:57:11.217 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:10 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-16T23:57:10.884+0000 7f14bb1c7640 -1 osd.5 0 read_superblock omap replica is missing. 2024-09-16T23:57:11.217 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:11 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-16T23:57:11.219+0000 7f14bb1c7640 -1 osd.5 74 log_to_monitors true 2024-09-16T23:57:12.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: pgmap v160: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:12.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all osd 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-16T23:57:12.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: Upgrade: Setting require_osd_release to 19 squid 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='osd.5 [v2:172.21.15.43:6816/3016136738,v1:172.21.15.43:6817/3016136738]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-mon[111056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:57:12.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:12 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-16T23:57:12.182+0000 7fa27606e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T23:57:12.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: pgmap v160: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:12.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all osd 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-09-16T23:57:12.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: Upgrade: Setting require_osd_release to 19 squid 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='osd.5 [v2:172.21.15.43:6816/3016136738,v1:172.21.15.43:6817/3016136738]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:57:12.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-mon[81510]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: osdmap e77: 6 total, 5 up, 6 in 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: from='osd.5 [v2:172.21.15.43:6816/3016136738,v1:172.21.15.43:6817/3016136738]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T23:57:13.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T23:57:13.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T23:57:13.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T23:57:13.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: osdmap e78: 6 total, 5 up, 6 in 2024-09-16T23:57:13.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-16T23:57:13.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:13 smithi042 ceph-mon[111056]: from='osd.5 ' entity='osd.5' 2024-09-16T23:57:13.377 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 16 23:57:12 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-16T23:57:12.928+0000 7f14aef59640 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: osdmap e77: 6 total, 5 up, 6 in 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: from='osd.5 [v2:172.21.15.43:6816/3016136738,v1:172.21.15.43:6817/3016136738]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T23:57:13.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T23:57:13.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T23:57:13.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: osdmap e78: 6 total, 5 up, 6 in 2024-09-16T23:57:13.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-16T23:57:13.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:13 smithi043 ceph-mon[81510]: from='osd.5 ' entity='osd.5' 2024-09-16T23:57:14.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: pgmap v163: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:14.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6828/4052502543,v1:172.21.15.42:6829/4052502543] up:boot 2024-09-16T23:57:14.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.43:6824/3469372237,v1:172.21.15.43:6825/3469372237] up:boot 2024-09-16T23:57:14.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:14.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:57:14.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:57:14.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:14.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:57:14.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:14.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:57:14.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: osd.5 [v2:172.21.15.43:6816/3016136738,v1:172.21.15.43:6817/3016136738] boot 2024-09-16T23:57:14.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: osdmap e79: 6 total, 6 up, 6 in 2024-09-16T23:57:14.443 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:14 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:57:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: pgmap v163: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 56/369 objects degraded (15.176%) 2024-09-16T23:57:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6828/4052502543,v1:172.21.15.42:6829/4052502543] up:boot 2024-09-16T23:57:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.43:6824/3469372237,v1:172.21.15.43:6825/3469372237] up:boot 2024-09-16T23:57:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:57:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:57:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:14.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.ctckas", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:57:14.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:14.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T23:57:14.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: osd.5 [v2:172.21.15.43:6816/3016136738,v1:172.21.15.43:6817/3016136738] boot 2024-09-16T23:57:14.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: osdmap e79: 6 total, 6 up, 6 in 2024-09-16T23:57:14.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:14 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T23:57:15.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:15 smithi042 ceph-mon[111056]: Upgrade: Updating mds.cephfs.smithi042.ctckas 2024-09-16T23:57:15.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:15 smithi042 ceph-mon[111056]: Deploying daemon mds.cephfs.smithi042.ctckas on smithi042 2024-09-16T23:57:15.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:15 smithi042 ceph-mon[111056]: osdmap e80: 6 total, 6 up, 6 in 2024-09-16T23:57:15.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:15 smithi043 ceph-mon[81510]: Upgrade: Updating mds.cephfs.smithi042.ctckas 2024-09-16T23:57:15.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:15 smithi043 ceph-mon[81510]: Deploying daemon mds.cephfs.smithi042.ctckas on smithi042 2024-09-16T23:57:15.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:15 smithi043 ceph-mon[81510]: osdmap e80: 6 total, 6 up, 6 in 2024-09-16T23:57:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:16 smithi042 ceph-mon[111056]: pgmap v165: 65 pgs: 2 peering, 13 active+undersized, 13 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 54/369 objects degraded (14.634%) 2024-09-16T23:57:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:16 smithi042 ceph-mon[111056]: Health check update: Degraded data redundancy: 54/369 objects degraded (14.634%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T23:57:16.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:16 smithi043 ceph-mon[81510]: pgmap v165: 65 pgs: 2 peering, 13 active+undersized, 13 active+undersized+degraded, 37 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 54/369 objects degraded (14.634%) 2024-09-16T23:57:16.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:16 smithi043 ceph-mon[81510]: Health check update: Degraded data redundancy: 54/369 objects degraded (14.634%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T23:57:17.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:17 smithi042 ceph-mon[111056]: osdmap e81: 6 total, 6 up, 6 in 2024-09-16T23:57:17.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:17 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:57:17.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:17 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:57:17.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:17 smithi043 ceph-mon[81510]: osdmap e81: 6 total, 6 up, 6 in 2024-09-16T23:57:17.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:17 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:57:17.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:17 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:57:18.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:18 smithi042 ceph-mon[111056]: pgmap v168: 65 pgs: 2 peering, 11 active+undersized, 8 active+undersized+degraded, 44 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26/369 objects degraded (7.046%) 2024-09-16T23:57:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:18 smithi043 ceph-mon[81510]: pgmap v168: 65 pgs: 2 peering, 11 active+undersized, 8 active+undersized+degraded, 44 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26/369 objects degraded (7.046%) 2024-09-16T23:57:20.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:20 smithi042 ceph-mon[111056]: pgmap v169: 65 pgs: 2 peering, 1 active+undersized, 2 active+undersized+degraded, 60 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5/369 objects degraded (1.355%) 2024-09-16T23:57:20.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:20 smithi043 ceph-mon[81510]: pgmap v169: 65 pgs: 2 peering, 1 active+undersized, 2 active+undersized+degraded, 60 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5/369 objects degraded (1.355%) 2024-09-16T23:57:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:21 smithi042 ceph-mon[111056]: pgmap v170: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:21 smithi043 ceph-mon[81510]: pgmap v170: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:22.980 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:22 smithi042 ceph-mon[111056]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/369 objects degraded (1.355%), 2 pgs degraded) 2024-09-16T23:57:22.981 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:22 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:boot 2024-09-16T23:57:22.981 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:22 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:22.981 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:22 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:57:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:22 smithi043 ceph-mon[81510]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 5/369 objects degraded (1.355%), 2 pgs degraded) 2024-09-16T23:57:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:22 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:boot 2024-09-16T23:57:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:22 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:22 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.ctckas"}]: dispatch 2024-09-16T23:57:23.841 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:23 smithi042 ceph-mon[111056]: pgmap v171: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:23 smithi043 ceph-mon[81510]: pgmap v171: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:25.952 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:25 smithi042 ceph-mon[111056]: pgmap v172: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:25.952 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:25 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:25 smithi043 ceph-mon[81510]: pgmap v172: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:25 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.732 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:27 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.732 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:27 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.732 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:27 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.732 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:27 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:27 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:27 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:27 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:27.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:27 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:28.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:28 smithi043 ceph-mon[81510]: pgmap v173: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:28.727 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:28 smithi042 ceph-mon[111056]: pgmap v173: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:30.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:30 smithi042 ceph-mon[111056]: pgmap v174: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:30.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:30 smithi043 ceph-mon[81510]: pgmap v174: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:57:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:31.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:31.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:31.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:31.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:31 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:57:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:31.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:31.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:31.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:31.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:31 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:32.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:32 smithi042 ceph-mon[111056]: pgmap v175: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:32.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:32.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:57:32.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:32.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:32 smithi043 ceph-mon[81510]: pgmap v175: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:32.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:32.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi042.anvvuy", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:57:32.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 -- 172.21.15.42:0/2821522574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e80ffda0 msgr2=0x7f03e8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/2821522574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e80ffda0 0x7f03e8100180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009910 tx=0x7f03d8009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 -- 172.21.15.42:0/2821522574 shutdown_connections 2024-09-16T23:57:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/2821522574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03e8100750 0x7f03e81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/2821522574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e80ffda0 0x7f03e8100180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 -- 172.21.15.42:0/2821522574 >> 172.21.15.42:0/2821522574 conn(0x7f03e80fb650 msgr2=0x7f03e80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 -- 172.21.15.42:0/2821522574 shutdown_connections 2024-09-16T23:57:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 -- 172.21.15.42:0/2821522574 wait complete. 2024-09-16T23:57:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.397+0000 7f03ed2b8700 1 Processor -- start 2024-09-16T23:57:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03ed2b8700 1 -- start start 2024-09-16T23:57:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03ed2b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03e80ffda0 0x7f03e8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03ed2b8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e8100750 0x7f03e8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03ed2b8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e8102350 con 0x7f03e8100750 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03ed2b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e81024c0 con 0x7f03e80ffda0 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03e67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e8100750 0x7f03e8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03e67fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e8100750 0x7f03e8101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40010/0 (socket says 172.21.15.42:40010) 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03e67fc700 1 -- 172.21.15.42:0/3676712234 learned_addr learned my addr 172.21.15.42:0/3676712234 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03e6ffd700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03e80ffda0 0x7f03e8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03e67fc700 1 -- 172.21.15.42:0/3676712234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03e80ffda0 msgr2=0x7f03e8101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03e67fc700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03e80ffda0 0x7f03e8101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.398+0000 7f03e67fc700 1 -- 172.21.15.42:0/3676712234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d80095f0 con 0x7f03e8100750 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.399+0000 7f03e67fc700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e8100750 0x7f03e8101d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03d000d8d0 tx=0x7f03d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.399+0000 7f03dffff700 1 -- 172.21.15.42:0/3676712234 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d0004a00 con 0x7f03e8100750 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.399+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e81a2ac0 con 0x7f03e8100750 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.399+0000 7f03dffff700 1 -- 172.21.15.42:0/3676712234 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03d00054e0 con 0x7f03e8100750 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.399+0000 7f03dffff700 1 -- 172.21.15.42:0/3676712234 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03d000fc20 con 0x7f03e8100750 2024-09-16T23:57:33.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.399+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e81a2f30 con 0x7f03e8100750 2024-09-16T23:57:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.400+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e804f2e0 con 0x7f03e8100750 2024-09-16T23:57:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.400+0000 7f03dffff700 1 -- 172.21.15.42:0/3676712234 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f03d0005020 con 0x7f03e8100750 2024-09-16T23:57:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.401+0000 7f03dffff700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f03d406e680 0x7f03d4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.401+0000 7f03dffff700 1 -- 172.21.15.42:0/3676712234 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f03d008df10 con 0x7f03e8100750 2024-09-16T23:57:33.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.401+0000 7f03e6ffd700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f03d406e680 0x7f03d4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:33.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.401+0000 7f03e6ffd700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f03d406e680 0x7f03d4070b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f03d8006010 tx=0x7f03d801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:33.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.405+0000 7f03dffff700 1 -- 172.21.15.42:0/3676712234 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f03d00570b0 con 0x7f03e8100750 2024-09-16T23:57:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:33 smithi042 ceph-mon[111056]: Upgrade: Updating mds.cephfs.smithi042.anvvuy 2024-09-16T23:57:33.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:33 smithi042 ceph-mon[111056]: Deploying daemon mds.cephfs.smithi042.anvvuy on smithi042 2024-09-16T23:57:33.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:33 smithi043 ceph-mon[81510]: Upgrade: Updating mds.cephfs.smithi042.anvvuy 2024-09-16T23:57:33.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:33 smithi043 ceph-mon[81510]: Deploying daemon mds.cephfs.smithi042.anvvuy on smithi042 2024-09-16T23:57:33.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.642+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03e80fc4c0 con 0x7f03d406e680 2024-09-16T23:57:33.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.643+0000 7f03dffff700 1 -- 172.21.15.42:0/3676712234 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f03e80fc4c0 con 0x7f03d406e680 2024-09-16T23:57:33.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f03d406e680 msgr2=0x7f03d4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f03d406e680 0x7f03d4070b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f03d8006010 tx=0x7f03d801a040 comp rx=0 tx=0).stop 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e8100750 msgr2=0x7f03e8101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e8100750 0x7f03e8101d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f03d000d8d0 tx=0x7f03d000dbe0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 shutdown_connections 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f03d406e680 0x7f03d4070b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03e80ffda0 0x7f03e8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 --2- 172.21.15.42:0/3676712234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e8100750 0x7f03e8101d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 >> 172.21.15.42:0/3676712234 conn(0x7f03e80fb650 msgr2=0x7f03e8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 shutdown_connections 2024-09-16T23:57:33.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.645+0000 7f03ed2b8700 1 -- 172.21.15.42:0/3676712234 wait complete. 2024-09-16T23:57:33.653 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:57:33.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.755+0000 7fdd95557700 1 -- 172.21.15.42:0/4191342782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 msgr2=0x7fdd90101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.755+0000 7fdd95557700 1 --2- 172.21.15.42:0/4191342782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90101270 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009910 tx=0x7fdd80009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.755+0000 7fdd95557700 1 -- 172.21.15.42:0/4191342782 shutdown_connections 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.755+0000 7fdd95557700 1 --2- 172.21.15.42:0/4191342782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90101840 0x7fdd90105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.755+0000 7fdd95557700 1 --2- 172.21.15.42:0/4191342782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90101270 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.755+0000 7fdd95557700 1 -- 172.21.15.42:0/4191342782 >> 172.21.15.42:0/4191342782 conn(0x7fdd900fc740 msgr2=0x7fdd900feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.756+0000 7fdd95557700 1 -- 172.21.15.42:0/4191342782 shutdown_connections 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.756+0000 7fdd95557700 1 -- 172.21.15.42:0/4191342782 wait complete. 2024-09-16T23:57:33.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.756+0000 7fdd95557700 1 Processor -- start 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.756+0000 7fdd95557700 1 -- start start 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd95557700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd95557700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90101840 0x7fdd900735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd95557700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90073b10 con 0x7fdd90100e90 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd95557700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90073c50 con 0x7fdd90101840 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40020/0 (socket says 172.21.15.42:40020) 2024-09-16T23:57:33.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8effd700 1 -- 172.21.15.42:0/1754434877 learned_addr learned my addr 172.21.15.42:0/1754434877 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8e7fc700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90101840 0x7fdd900735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8effd700 1 -- 172.21.15.42:0/1754434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90101840 msgr2=0x7fdd900735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8effd700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90101840 0x7fdd900735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8effd700 1 -- 172.21.15.42:0/1754434877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800095f0 con 0x7fdd90100e90 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.757+0000 7fdd8effd700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90073090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd8000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.758+0000 7fdd77fff700 1 -- 172.21.15.42:0/1754434877 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd8001d070 con 0x7fdd90100e90 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.758+0000 7fdd77fff700 1 -- 172.21.15.42:0/1754434877 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd8000f460 con 0x7fdd90100e90 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.758+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd90073ed0 con 0x7fdd90100e90 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.758+0000 7fdd77fff700 1 -- 172.21.15.42:0/1754434877 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd80017610 con 0x7fdd90100e90 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.758+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd901a2e80 con 0x7fdd90100e90 2024-09-16T23:57:33.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.759+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd90109230 con 0x7fdd90100e90 2024-09-16T23:57:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.759+0000 7fdd77fff700 1 -- 172.21.15.42:0/1754434877 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fdd80003a60 con 0x7fdd90100e90 2024-09-16T23:57:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.760+0000 7fdd77fff700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd7806e680 0x7fdd78070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.760+0000 7fdd77fff700 1 -- 172.21.15.42:0/1754434877 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdd8008fb20 con 0x7fdd90100e90 2024-09-16T23:57:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.760+0000 7fdd8e7fc700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd7806e680 0x7fdd78070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:33.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.760+0000 7fdd8e7fc700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd7806e680 0x7fdd78070b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd90074880 tx=0x7fdd84009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:33.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:33.764+0000 7fdd77fff700 1 -- 172.21.15.42:0/1754434877 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdd80058d40 con 0x7fdd90100e90 2024-09-16T23:57:34.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.019+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd900611d0 con 0x7fdd7806e680 2024-09-16T23:57:34.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.021+0000 7fdd77fff700 1 -- 172.21.15.42:0/1754434877 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdd900611d0 con 0x7fdd7806e680 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd7806e680 msgr2=0x7fdd78070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd7806e680 0x7fdd78070b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd90074880 tx=0x7fdd84009190 comp rx=0 tx=0).stop 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 msgr2=0x7fdd90073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90073090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd8000bd70 comp rx=0 tx=0).stop 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 shutdown_connections 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd7806e680 0x7fdd78070b40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd90100e90 0x7fdd90073090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 --2- 172.21.15.42:0/1754434877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd90101840 0x7fdd900735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 >> 172.21.15.42:0/1754434877 conn(0x7fdd900fc740 msgr2=0x7fdd900fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 shutdown_connections 2024-09-16T23:57:34.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.023+0000 7fdd95557700 1 -- 172.21.15.42:0/1754434877 wait complete. 2024-09-16T23:57:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 -- 172.21.15.42:0/3821885847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 msgr2=0x7f3da0105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 --2- 172.21.15.42:0/3821885847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da0105890 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009910 tx=0x7f3d8c009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 -- 172.21.15.42:0/3821885847 shutdown_connections 2024-09-16T23:57:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 --2- 172.21.15.42:0/3821885847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da0105890 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 --2- 172.21.15.42:0/3821885847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3da0100e90 0x7f3da0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 -- 172.21.15.42:0/3821885847 >> 172.21.15.42:0/3821885847 conn(0x7f3da00fc740 msgr2=0x7f3da00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:34.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 -- 172.21.15.42:0/3821885847 shutdown_connections 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 -- 172.21.15.42:0/3821885847 wait complete. 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.153+0000 7f3da6d5b700 1 Processor -- start 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3da6d5b700 1 -- start start 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3da6d5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3da0100e90 0x7f3da0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3da6d5b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3da6d5b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da0073b10 con 0x7f3da0101840 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3da6d5b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da0073c50 con 0x7f3da0100e90 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3d9ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3d9ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40046/0 (socket says 172.21.15.42:40046) 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3d9ffff700 1 -- 172.21.15.42:0/3180407465 learned_addr learned my addr 172.21.15.42:0/3180407465 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3da4af7700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3da0100e90 0x7f3da0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3d9ffff700 1 -- 172.21.15.42:0/3180407465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3da0100e90 msgr2=0x7f3da0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3d9ffff700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3da0100e90 0x7f3da0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.154+0000 7f3d9ffff700 1 -- 172.21.15.42:0/3180407465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0095f0 con 0x7f3da0101840 2024-09-16T23:57:34.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.155+0000 7f3d9ffff700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da00735d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c006010 tx=0x7f3d8c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:34.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.155+0000 7f3d9dffb700 1 -- 172.21.15.42:0/3180407465 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d8c01d070 con 0x7f3da0101840 2024-09-16T23:57:34.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.155+0000 7f3d9dffb700 1 -- 172.21.15.42:0/3180407465 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d8c0050d0 con 0x7f3da0101840 2024-09-16T23:57:34.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.155+0000 7f3d9dffb700 1 -- 172.21.15.42:0/3180407465 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d8c017400 con 0x7f3da0101840 2024-09-16T23:57:34.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.155+0000 7f3da6d5b700 1 -- 172.21.15.42:0/3180407465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da0073ed0 con 0x7f3da0101840 2024-09-16T23:57:34.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.155+0000 7f3da6d5b700 1 -- 172.21.15.42:0/3180407465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da01a2da0 con 0x7f3da0101840 2024-09-16T23:57:34.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.156+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d800052f0 con 0x7f3da0101840 2024-09-16T23:57:34.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.157+0000 7f3d9dffb700 1 -- 172.21.15.42:0/3180407465 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3d8c00faf0 con 0x7f3da0101840 2024-09-16T23:57:34.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.157+0000 7f3d9dffb700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d9006e680 0x7f3d90070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.158+0000 7f3d9dffb700 1 -- 172.21.15.42:0/3180407465 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d8c08f8a0 con 0x7f3da0101840 2024-09-16T23:57:34.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.158+0000 7f3da4af7700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d9006e680 0x7f3d90070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.158+0000 7f3da4af7700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d9006e680 0x7f3d90070b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009c60 tx=0x7f3d94009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:34.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.162+0000 7f3d9dffb700 1 -- 172.21.15.42:0/3180407465 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3d8c058ac0 con 0x7f3da0101840 2024-09-16T23:57:34.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.384+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d80000bc0 con 0x7f3d9006e680 2024-09-16T23:57:34.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.393+0000 7f3d9dffb700 1 -- 172.21.15.42:0/3180407465 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f3d80000bc0 con 0x7f3d9006e680 2024-09-16T23:57:34.392 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:57:34.392 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (6m) 7s ago 16m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:57:34.392 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (17m) 7s ago 17m 8916k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:57:34.392 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (16m) 27s ago 16m 8644k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:57:34.392 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (3m) 7s ago 17m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:57:34.392 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (3m) 27s ago 16m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:57:34.392 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (5m) 7s ago 16m 80.3M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (11m) 7s ago 11m 11.5M - 18.2.1 5be31c24972a c86c76c278e5 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (13s) 7s ago 11m 14.7M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (11m) 27s ago 11m 121M - 18.2.1 5be31c24972a f6db4c801c38 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (11m) 27s ago 11m 18.6M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (8m) 7s ago 18m 504M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (7m) 27s ago 15m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (4m) 7s ago 18m 54.8M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (4m) 27s ago 15m 42.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:57:34.393 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (6m) 7s ago 17m 16.1M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (6m) 27s ago 16m 15.4M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (3m) 7s ago 15m 88.4M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (2m) 7s ago 14m 85.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (2m) 7s ago 14m 80.6M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (97s) 27s ago 13m 88.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (64s) 27s ago 13m 91.4M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (30s) 27s ago 12m 14.2M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-16T23:57:34.394 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (6m) 7s ago 16m 76.7M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:57:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d9006e680 msgr2=0x7f3d90070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d9006e680 0x7f3d90070b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009c60 tx=0x7f3d94009260 comp rx=0 tx=0).stop 2024-09-16T23:57:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 msgr2=0x7f3da00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da00735d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c006010 tx=0x7f3d8c00f690 comp rx=0 tx=0).stop 2024-09-16T23:57:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 shutdown_connections 2024-09-16T23:57:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d9006e680 0x7f3d90070b40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3da0100e90 0x7f3da0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 --2- 172.21.15.42:0/3180407465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3da0101840 0x7f3da00735d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 >> 172.21.15.42:0/3180407465 conn(0x7f3da00fc740 msgr2=0x7f3da00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 shutdown_connections 2024-09-16T23:57:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.396+0000 7f3d8b7fe700 1 -- 172.21.15.42:0/3180407465 wait complete. 2024-09-16T23:57:34.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 -- 172.21.15.42:0/2811671731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c40ffc20 msgr2=0x7fa5c4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/2811671731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c40ffc20 0x7fa5c4100080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009910 tx=0x7fa5b8009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:34.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 -- 172.21.15.42:0/2811671731 shutdown_connections 2024-09-16T23:57:34.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/2811671731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c40ffc20 0x7fa5c4100080 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/2811671731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c4105bd0 0x7fa5c4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 -- 172.21.15.42:0/2811671731 >> 172.21.15.42:0/2811671731 conn(0x7fa5c4075670 msgr2=0x7fa5c4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:34.504 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:34 smithi042 ceph-mon[111056]: pgmap v176: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 -- 172.21.15.42:0/2811671731 shutdown_connections 2024-09-16T23:57:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.505+0000 7fa5c9c44700 1 -- 172.21.15.42:0/2811671731 wait complete. 2024-09-16T23:57:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.506+0000 7fa5c9c44700 1 Processor -- start 2024-09-16T23:57:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.506+0000 7fa5c9c44700 1 -- start start 2024-09-16T23:57:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.506+0000 7fa5c9c44700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c40ffc20 0x7fa5c419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.506+0000 7fa5c9c44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4105bd0 0x7fa5c419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.506+0000 7fa5c9c44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c419d350 con 0x7fa5c4105bd0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.506+0000 7fa5c9c44700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c4196800 con 0x7fa5c40ffc20 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4105bd0 0x7fa5c419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4105bd0 0x7fa5c419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40070/0 (socket says 172.21.15.42:40070) 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c2ffd700 1 -- 172.21.15.42:0/1926858162 learned_addr learned my addr 172.21.15.42:0/1926858162 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c37fe700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c40ffc20 0x7fa5c419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c2ffd700 1 -- 172.21.15.42:0/1926858162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c40ffc20 msgr2=0x7fa5c419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c2ffd700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c40ffc20 0x7fa5c419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c2ffd700 1 -- 172.21.15.42:0/1926858162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b80095f0 con 0x7fa5c4105bd0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c2ffd700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4105bd0 0x7fa5c419ccc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c00 tx=0x7fa5b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c0ff9700 1 -- 172.21.15.42:0/1926858162 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b801d070 con 0x7fa5c4105bd0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c4196a80 con 0x7fa5c4105bd0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c0ff9700 1 -- 172.21.15.42:0/1926858162 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5b800f460 con 0x7fa5c4105bd0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c0ff9700 1 -- 172.21.15.42:0/1926858162 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b8017630 con 0x7fa5c4105bd0 2024-09-16T23:57:34.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.507+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c4196f70 con 0x7fa5c4105bd0 2024-09-16T23:57:34.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.508+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c404f2e0 con 0x7fa5c4105bd0 2024-09-16T23:57:34.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.509+0000 7fa5c0ff9700 1 -- 172.21.15.42:0/1926858162 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa5b800f5d0 con 0x7fa5c4105bd0 2024-09-16T23:57:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.510+0000 7fa5c0ff9700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5ac06e680 0x7fa5ac070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.510+0000 7fa5c0ff9700 1 -- 172.21.15.42:0/1926858162 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa5b808fe80 con 0x7fa5c4105bd0 2024-09-16T23:57:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.510+0000 7fa5c37fe700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5ac06e680 0x7fa5ac070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.511+0000 7fa5c37fe700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5ac06e680 0x7fa5ac070b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4006fd0 tx=0x7fa5b4008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:34.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.514+0000 7fa5c0ff9700 1 -- 172.21.15.42:0/1926858162 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa5b805cbd0 con 0x7fa5c4105bd0 2024-09-16T23:57:34.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.820+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa5c4197d10 con 0x7fa5c4105bd0 2024-09-16T23:57:34.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.821+0000 7fa5c0ff9700 1 -- 172.21.15.42:0/1926858162 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fa5b80163b0 con 0x7fa5c4105bd0 2024-09-16T23:57:34.818 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:57:34.818 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 1 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:57:34.819 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-09-16T23:57:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 11 2024-09-16T23:57:34.820 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:57:34.820 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:57:34.821 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5ac06e680 msgr2=0x7fa5ac070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5ac06e680 0x7fa5ac070b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4006fd0 tx=0x7fa5b4008040 comp rx=0 tx=0).stop 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4105bd0 msgr2=0x7fa5c419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4105bd0 0x7fa5c419ccc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8000c00 tx=0x7fa5b800be20 comp rx=0 tx=0).stop 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 shutdown_connections 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5ac06e680 0x7fa5ac070b40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c40ffc20 0x7fa5c419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 --2- 172.21.15.42:0/1926858162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5c4105bd0 0x7fa5c419ccc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.823+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 >> 172.21.15.42:0/1926858162 conn(0x7fa5c4075670 msgr2=0x7fa5c40686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.824+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 shutdown_connections 2024-09-16T23:57:34.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.824+0000 7fa5c9c44700 1 -- 172.21.15.42:0/1926858162 wait complete. 2024-09-16T23:57:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:34 smithi043 ceph-mon[81510]: pgmap v176: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 -- 172.21.15.42:0/2267254444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 msgr2=0x7f56481070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 --2- 172.21.15.42:0/2267254444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f56481070b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f563800b0a0 tx=0x7f563800b3b0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 -- 172.21.15.42:0/2267254444 shutdown_connections 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 --2- 172.21.15.42:0/2267254444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5648068580 0x7f56480689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 --2- 172.21.15.42:0/2267254444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f56481070b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 -- 172.21.15.42:0/2267254444 >> 172.21.15.42:0/2267254444 conn(0x7f5648075670 msgr2=0x7f5648075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 -- 172.21.15.42:0/2267254444 shutdown_connections 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.932+0000 7f564ea72700 1 -- 172.21.15.42:0/2267254444 wait complete. 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f564ea72700 1 Processor -- start 2024-09-16T23:57:34.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f564ea72700 1 -- start start 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f564ea72700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5648068580 0x7f564819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f564ea72700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f564819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f564ea72700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564819d390 con 0x7f5648106cd0 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f564ea72700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5648196840 con 0x7f5648068580 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f5647fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f564819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f5647fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f564819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40088/0 (socket says 172.21.15.42:40088) 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f5647fff700 1 -- 172.21.15.42:0/837357473 learned_addr learned my addr 172.21.15.42:0/837357473 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f564c80e700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5648068580 0x7f564819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f5647fff700 1 -- 172.21.15.42:0/837357473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5648068580 msgr2=0x7f564819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:34.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f5647fff700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5648068580 0x7f564819c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:34.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.933+0000 7f5647fff700 1 -- 172.21.15.42:0/837357473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5638009d00 con 0x7f5648106cd0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.934+0000 7f5647fff700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f564819cd00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f563000b4b0 tx=0x7f563000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.934+0000 7f5645ffb700 1 -- 172.21.15.42:0/837357473 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5630011800 con 0x7f5648106cd0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.934+0000 7f5645ffb700 1 -- 172.21.15.42:0/837357473 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5630011e20 con 0x7f5648106cd0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.934+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5648196b20 con 0x7f5648106cd0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.934+0000 7f5645ffb700 1 -- 172.21.15.42:0/837357473 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f563000f2e0 con 0x7f5648106cd0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.934+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5648197070 con 0x7f5648106cd0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.935+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56481091d0 con 0x7f5648106cd0 2024-09-16T23:57:34.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.935+0000 7f5645ffb700 1 -- 172.21.15.42:0/837357473 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5630010380 con 0x7f5648106cd0 2024-09-16T23:57:34.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.936+0000 7f5645ffb700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f563406e680 0x7f5634070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:34.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.936+0000 7f5645ffb700 1 -- 172.21.15.42:0/837357473 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f563008e180 con 0x7f5648106cd0 2024-09-16T23:57:34.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.936+0000 7f564c80e700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f563406e680 0x7f5634070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:34.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.936+0000 7f564c80e700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f563406e680 0x7f5634070b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5638009480 tx=0x7f563800b7f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:34.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:34.940+0000 7f5645ffb700 1 -- 172.21.15.42:0/837357473 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f563005af80 con 0x7f5648106cd0 2024-09-16T23:57:35.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.193+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f564804f2e0 con 0x7f5648106cd0 2024-09-16T23:57:35.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.194+0000 7f5645ffb700 1 -- 172.21.15.42:0/837357473 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1731 (secure 0 0 0) 0x7f563005ab10 con 0x7f5648106cd0 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:e24 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:57:34:468457+0000 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-16T23:57:35.192 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-16T23:57:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:35.194 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:24279} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.43:6826/2825700082,v1:172.21.15.43:6827/2825700082] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:57:35.195 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44307} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/3469372237,v1:172.21.15.43:6825/3469372237] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f563406e680 msgr2=0x7f5634070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f563406e680 0x7f5634070b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5638009480 tx=0x7f563800b7f0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 msgr2=0x7f564819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f564819cd00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f563000b4b0 tx=0x7f563000b870 comp rx=0 tx=0).stop 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 shutdown_connections 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f563406e680 0x7f5634070b40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5648068580 0x7f564819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 --2- 172.21.15.42:0/837357473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5648106cd0 0x7f564819cd00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 >> 172.21.15.42:0/837357473 conn(0x7f5648075670 msgr2=0x7f56480fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:35.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 shutdown_connections 2024-09-16T23:57:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.197+0000 7f564ea72700 1 -- 172.21.15.42:0/837357473 wait complete. 2024-09-16T23:57:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 24 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.306+0000 7f621157d700 1 -- 172.21.15.42:0/2043831693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 msgr2=0x7f620c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.306+0000 7f621157d700 1 --2- 172.21.15.42:0/2043831693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c101280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009910 tx=0x7f61fc009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.307+0000 7f621157d700 1 -- 172.21.15.42:0/2043831693 shutdown_connections 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.307+0000 7f621157d700 1 --2- 172.21.15.42:0/2043831693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620c101850 0x7f620c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.307+0000 7f621157d700 1 --2- 172.21.15.42:0/2043831693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c101280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.307+0000 7f621157d700 1 -- 172.21.15.42:0/2043831693 >> 172.21.15.42:0/2043831693 conn(0x7f620c0fc750 msgr2=0x7f620c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.307+0000 7f621157d700 1 -- 172.21.15.42:0/2043831693 shutdown_connections 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.307+0000 7f621157d700 1 -- 172.21.15.42:0/2043831693 wait complete. 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.307+0000 7f621157d700 1 Processor -- start 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f621157d700 1 -- start start 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f621157d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:35.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f621157d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620c101850 0x7f620c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f621157d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c19d230 con 0x7f620c100ea0 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f621157d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c196770 con 0x7f620c101850 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40118/0 (socket says 172.21.15.42:40118) 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620affd700 1 -- 172.21.15.42:0/838537816 learned_addr learned my addr 172.21.15.42:0/838537816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620a7fc700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620c101850 0x7f620c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620affd700 1 -- 172.21.15.42:0/838537816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620c101850 msgr2=0x7f620c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620affd700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620c101850 0x7f620c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620affd700 1 -- 172.21.15.42:0/838537816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0095f0 con 0x7f620c100ea0 2024-09-16T23:57:35.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.308+0000 7f620affd700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c19c6f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00b5c0 tx=0x7f61fc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:35.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.309+0000 7f61f3fff700 1 -- 172.21.15.42:0/838537816 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc01d070 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.309+0000 7f61f3fff700 1 -- 172.21.15.42:0/838537816 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61fc00f460 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.309+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c1969f0 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.309+0000 7f61f3fff700 1 -- 172.21.15.42:0/838537816 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc017630 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.309+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c196ee0 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.310+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c109240 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.310+0000 7f61f3fff700 1 -- 172.21.15.42:0/838537816 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f61fc00f5d0 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.311+0000 7f61f3fff700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61f406e680 0x7f61f4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.311+0000 7f61f3fff700 1 -- 172.21.15.42:0/838537816 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f61fc08fbb0 con 0x7f620c100ea0 2024-09-16T23:57:35.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.311+0000 7f620a7fc700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61f406e680 0x7f61f4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:35.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.311+0000 7f620a7fc700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61f406e680 0x7f61f4070b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f620c197be0 tx=0x7f620000a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:35.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.315+0000 7f61f3fff700 1 -- 172.21.15.42:0/838537816 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f61fc096080 con 0x7f620c100ea0 2024-09-16T23:57:35.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.549+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f620c197d90 con 0x7f61f406e680 2024-09-16T23:57:35.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.550+0000 7f61f3fff700 1 -- 172.21.15.42:0/838537816 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f620c197d90 con 0x7f61f406e680 2024-09-16T23:57:35.548 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:57:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:57:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:57:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:57:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:57:35.548 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr", 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "osd" 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "13/23 daemons upgraded", 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:57:35.549 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:57:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.552+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61f406e680 msgr2=0x7f61f4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.552+0000 7f621157d700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61f406e680 0x7f61f4070b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f620c197be0 tx=0x7f620000a380 comp rx=0 tx=0).stop 2024-09-16T23:57:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.552+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 msgr2=0x7f620c19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.552+0000 7f621157d700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c19c6f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00b5c0 tx=0x7f61fc00bd50 comp rx=0 tx=0).stop 2024-09-16T23:57:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.553+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 shutdown_connections 2024-09-16T23:57:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.553+0000 7f621157d700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61f406e680 0x7f61f4070b40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.553+0000 7f621157d700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f620c100ea0 0x7f620c19c6f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.553+0000 7f621157d700 1 --2- 172.21.15.42:0/838537816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620c101850 0x7f620c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.553+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 >> 172.21.15.42:0/838537816 conn(0x7f620c0fc750 msgr2=0x7f620c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.553+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 shutdown_connections 2024-09-16T23:57:35.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.553+0000 7f621157d700 1 -- 172.21.15.42:0/838537816 wait complete. 2024-09-16T23:57:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 -- 172.21.15.42:0/2617054308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 msgr2=0x7fa220100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 --2- 172.21.15.42:0/2617054308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa220100180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009910 tx=0x7fa21c009c20 comp rx=0 tx=0).stop 2024-09-16T23:57:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 -- 172.21.15.42:0/2617054308 shutdown_connections 2024-09-16T23:57:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 --2- 172.21.15.42:0/2617054308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa220100750 0x7fa2201047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 --2- 172.21.15.42:0/2617054308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa220100180 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 -- 172.21.15.42:0/2617054308 >> 172.21.15.42:0/2617054308 conn(0x7fa2200fb650 msgr2=0x7fa2200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 -- 172.21.15.42:0/2617054308 shutdown_connections 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 -- 172.21.15.42:0/2617054308 wait complete. 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 Processor -- start 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.663+0000 7fa2282ce700 1 -- start start 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa2282ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa22019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa2282ce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa220100750 0x7fa22019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa2282ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22019d3b0 con 0x7fa2200ffda0 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa2282ce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa220196860 con 0x7fa220100750 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa22606a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa22019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa22606a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa22019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40136/0 (socket says 172.21.15.42:40136) 2024-09-16T23:57:35.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa22606a700 1 -- 172.21.15.42:0/663577761 learned_addr learned my addr 172.21.15.42:0/663577761 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:57:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa22606a700 1 -- 172.21.15.42:0/663577761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa220100750 msgr2=0x7fa22019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa22606a700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa220100750 0x7fa22019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa22606a700 1 -- 172.21.15.42:0/663577761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0095f0 con 0x7fa2200ffda0 2024-09-16T23:57:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa22606a700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa22019c7e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006010 tx=0x7fa21c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa2177fe700 1 -- 172.21.15.42:0/663577761 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa21c01d070 con 0x7fa2200ffda0 2024-09-16T23:57:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa2177fe700 1 -- 172.21.15.42:0/663577761 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa21c004fb0 con 0x7fa2200ffda0 2024-09-16T23:57:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.664+0000 7fa2177fe700 1 -- 172.21.15.42:0/663577761 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa21c00f690 con 0x7fa2200ffda0 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.665+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa220196ae0 con 0x7fa2200ffda0 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.665+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa220196f80 con 0x7fa2200ffda0 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.665+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22004f2e0 con 0x7fa2200ffda0 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.666+0000 7fa2177fe700 1 -- 172.21.15.42:0/663577761 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa21c0228a0 con 0x7fa2200ffda0 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.666+0000 7fa2177fe700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa20c06e680 0x7fa20c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.667+0000 7fa2177fe700 1 -- 172.21.15.42:0/663577761 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa21c08fed0 con 0x7fa2200ffda0 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.667+0000 7fa225869700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa20c06e680 0x7fa20c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:57:35.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.667+0000 7fa225869700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa20c06e680 0x7fa20c070b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa220106760 tx=0x7fa210006d20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:57:35.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.671+0000 7fa2177fe700 1 -- 172.21.15.42:0/663577761 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa21c05cba0 con 0x7fa2200ffda0 2024-09-16T23:57:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:35 smithi042 ceph-mon[111056]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:35 smithi042 ceph-mon[111056]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:35.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:35 smithi042 ceph-mon[111056]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:35.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:35 smithi042 ceph-mon[111056]: osdmap e82: 6 total, 6 up, 6 in 2024-09-16T23:57:35.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:35 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:57:35.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:35 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1926858162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:35.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:35 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/837357473' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:57:35.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:35 smithi043 ceph-mon[81510]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:35.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:35 smithi043 ceph-mon[81510]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:35.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:35 smithi043 ceph-mon[81510]: from='client.34400 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:35.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:35 smithi043 ceph-mon[81510]: osdmap e82: 6 total, 6 up, 6 in 2024-09-16T23:57:35.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:35 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:57:35.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:35 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1926858162' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:35.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:35 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/837357473' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:57:35.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.969+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa2201971f0 con 0x7fa2200ffda0 2024-09-16T23:57:35.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.970+0000 7fa2177fe700 1 -- 172.21.15.42:0/663577761 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa21c027090 con 0x7fa2200ffda0 2024-09-16T23:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T23:57:35.968 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T23:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-16T23:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T23:57:35.969 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T23:57:35.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa20c06e680 msgr2=0x7fa20c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa20c06e680 0x7fa20c070b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa220106760 tx=0x7fa210006d20 comp rx=0 tx=0).stop 2024-09-16T23:57:35.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 msgr2=0x7fa22019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:57:35.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa22019c7e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006010 tx=0x7fa21c00be20 comp rx=0 tx=0).stop 2024-09-16T23:57:35.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 shutdown_connections 2024-09-16T23:57:35.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa20c06e680 0x7fa20c070b40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2200ffda0 0x7fa22019c7e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 --2- 172.21.15.42:0/663577761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa220100750 0x7fa22019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:57:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 >> 172.21.15.42:0/663577761 conn(0x7fa2200fb650 msgr2=0x7fa220103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:57:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 shutdown_connections 2024-09-16T23:57:35.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:57:35.972+0000 7fa2282ce700 1 -- 172.21.15.42:0/663577761 wait complete. 2024-09-16T23:57:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:36 smithi042 ceph-mon[111056]: pgmap v178: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:36 smithi042 ceph-mon[111056]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:36 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/663577761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:57:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:36 smithi043 ceph-mon[81510]: pgmap v178: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:36 smithi043 ceph-mon[81510]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:57:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:36 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/663577761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:57:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:38 smithi042 ceph-mon[111056]: pgmap v179: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:38 smithi043 ceph-mon[81510]: pgmap v179: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:39.731 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:39 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:boot 2024-09-16T23:57:39.731 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:39 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:39.732 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:39 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:57:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:39 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:boot 2024-09-16T23:57:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:39 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:39.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:39 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi042.anvvuy"}]: dispatch 2024-09-16T23:57:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:40 smithi042 ceph-mon[111056]: pgmap v180: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:40.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:40 smithi043 ceph-mon[81510]: pgmap v180: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:42.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:42 smithi042 ceph-mon[111056]: pgmap v181: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:42.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:42 smithi043 ceph-mon[81510]: pgmap v181: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:44.239 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:43 smithi042 ceph-mon[111056]: pgmap v182: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:44.239 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:43 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:44.239 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:43 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:44.239 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:43 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:44.239 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:43 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:44.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:43 smithi043 ceph-mon[81510]: pgmap v182: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:44.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:43 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:44.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:43 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:44.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:43 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:44.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:43 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:46.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:46 smithi042 ceph-mon[111056]: pgmap v183: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:46.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:46 smithi043 ceph-mon[81510]: pgmap v183: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: pgmap v184: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:57:48.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:48.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:48.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:48.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:48.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:48.129 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:47 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: pgmap v184: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:48.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:48.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:48.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:47 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:57:49.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:49 smithi042 ceph-mon[111056]: Upgrade: Updating mds.cephfs.smithi043.pzcdzg 2024-09-16T23:57:49.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:49 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:49.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:49 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:57:49.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:49 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:49.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:49 smithi042 ceph-mon[111056]: Deploying daemon mds.cephfs.smithi043.pzcdzg on smithi043 2024-09-16T23:57:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:49 smithi043 ceph-mon[81510]: Upgrade: Updating mds.cephfs.smithi043.pzcdzg 2024-09-16T23:57:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:49 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:49 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.pzcdzg", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:57:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:49 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:57:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:49 smithi043 ceph-mon[81510]: Deploying daemon mds.cephfs.smithi043.pzcdzg on smithi043 2024-09-16T23:57:50.606 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:50 smithi043 ceph-mon[81510]: pgmap v185: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:50 smithi042 ceph-mon[111056]: pgmap v185: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:57:52.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:52 smithi042 ceph-mon[111056]: pgmap v186: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:52.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:52 smithi042 ceph-mon[111056]: osdmap e83: 6 total, 6 up, 6 in 2024-09-16T23:57:52.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:52 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:57:52.823 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:52 smithi043 ceph-mon[81510]: pgmap v186: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:52.823 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:52 smithi043 ceph-mon[81510]: osdmap e83: 6 total, 6 up, 6 in 2024-09-16T23:57:52.823 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:52 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:57:54.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:54 smithi042 ceph-mon[111056]: pgmap v188: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:54.576 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:54.576 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:54.576 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:54.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:54 smithi043 ceph-mon[81510]: pgmap v188: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:54 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:54 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:54 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:57:55.680 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:55 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] up:boot 2024-09-16T23:57:55.680 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:55 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:55.680 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:55 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:57:55.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:55 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] up:boot 2024-09-16T23:57:55.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:55 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:57:55.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:55 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-16T23:57:56.694 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:56 smithi043 ceph-mon[81510]: pgmap v189: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:56 smithi042 ceph-mon[111056]: pgmap v189: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:58.623 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:58 smithi043 ceph-mon[81510]: pgmap v190: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:58.623 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:58 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:58.623 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:58 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:58.623 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:58 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:58.623 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:57:58 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:58 smithi042 ceph-mon[111056]: pgmap v190: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:57:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:58 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:58 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:58 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:57:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:57:58 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:00.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:00 smithi043 ceph-mon[81510]: pgmap v191: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:00.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:00 smithi042 ceph-mon[111056]: pgmap v191: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: pgmap v192: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:02.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:02 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: pgmap v192: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:58:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T23:58:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:58:02.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:02.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:02.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:02.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:02 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:04.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:03 smithi043 ceph-mon[81510]: pgmap v193: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:03 smithi043 ceph-mon[81510]: Upgrade: Updating mds.cephfs.smithi043.riwwhn 2024-09-16T23:58:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:03 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:03 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:58:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:03 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:58:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:03 smithi043 ceph-mon[81510]: Deploying daemon mds.cephfs.smithi043.riwwhn on smithi043 2024-09-16T23:58:04.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:03 smithi042 ceph-mon[111056]: pgmap v193: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:04.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:03 smithi042 ceph-mon[111056]: Upgrade: Updating mds.cephfs.smithi043.riwwhn 2024-09-16T23:58:04.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:03 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:04.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:03 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi043.riwwhn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T23:58:04.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:03 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T23:58:04.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:03 smithi042 ceph-mon[111056]: Deploying daemon mds.cephfs.smithi043.riwwhn on smithi043 2024-09-16T23:58:06.086 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:05 smithi042 ceph-mon[111056]: pgmap v194: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:06.086 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:05 smithi042 ceph-mon[111056]: osdmap e84: 6 total, 6 up, 6 in 2024-09-16T23:58:06.086 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:05 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:58:06.086 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 -- 172.21.15.42:0/1647509844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c100ea0 msgr2=0x7fe08c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 --2- 172.21.15.42:0/1647509844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c100ea0 0x7fe08c101280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009910 tx=0x7fe07c009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 -- 172.21.15.42:0/1647509844 shutdown_connections 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 --2- 172.21.15.42:0/1647509844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe08c101850 0x7fe08c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 --2- 172.21.15.42:0/1647509844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c100ea0 0x7fe08c101280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 -- 172.21.15.42:0/1647509844 >> 172.21.15.42:0/1647509844 conn(0x7fe08c0fc750 msgr2=0x7fe08c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 -- 172.21.15.42:0/1647509844 shutdown_connections 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.087+0000 7fe0922c6700 1 -- 172.21.15.42:0/1647509844 wait complete. 2024-09-16T23:58:06.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe0922c6700 1 Processor -- start 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe0922c6700 1 -- start start 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe0922c6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe08c100ea0 0x7fe08c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe0922c6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c101850 0x7fe08c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe0922c6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c073b10 con 0x7fe08c101850 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe0922c6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c073c50 con 0x7fe08c100ea0 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c101850 0x7fe08c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe08b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c101850 0x7fe08c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56340/0 (socket says 172.21.15.42:56340) 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe08b7fe700 1 -- 172.21.15.42:0/3733384330 learned_addr learned my addr 172.21.15.42:0/3733384330 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.088+0000 7fe08bfff700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe08c100ea0 0x7fe08c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe08b7fe700 1 -- 172.21.15.42:0/3733384330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe08c100ea0 msgr2=0x7fe08c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe08b7fe700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe08c100ea0 0x7fe08c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe08b7fe700 1 -- 172.21.15.42:0/3733384330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c0095f0 con 0x7fe08c101850 2024-09-16T23:58:06.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe08b7fe700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c101850 0x7fe08c0735d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe08000b480 tx=0x7fe08000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:06.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe0897fa700 1 -- 172.21.15.42:0/3733384330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe080011800 con 0x7fe08c101850 2024-09-16T23:58:06.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c1a2b60 con 0x7fe08c101850 2024-09-16T23:58:06.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe0897fa700 1 -- 172.21.15.42:0/3733384330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe080011e20 con 0x7fe08c101850 2024-09-16T23:58:06.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe0897fa700 1 -- 172.21.15.42:0/3733384330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe080010ea0 con 0x7fe08c101850 2024-09-16T23:58:06.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.089+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c1a2f70 con 0x7fe08c101850 2024-09-16T23:58:06.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.090+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c109240 con 0x7fe08c101850 2024-09-16T23:58:06.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.091+0000 7fe0897fa700 1 -- 172.21.15.42:0/3733384330 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe080010380 con 0x7fe08c101850 2024-09-16T23:58:06.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.091+0000 7fe0897fa700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe07406e680 0x7fe074070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.091+0000 7fe0897fa700 1 -- 172.21.15.42:0/3733384330 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe08008e170 con 0x7fe08c101850 2024-09-16T23:58:06.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.091+0000 7fe08bfff700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe07406e680 0x7fe074070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.092+0000 7fe08bfff700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe07406e680 0x7fe074070b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe07c006010 tx=0x7fe07c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:06.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.096+0000 7fe0897fa700 1 -- 172.21.15.42:0/3733384330 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe080057310 con 0x7fe08c101850 2024-09-16T23:58:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:05 smithi043 ceph-mon[81510]: pgmap v194: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:05 smithi043 ceph-mon[81510]: osdmap e84: 6 total, 6 up, 6 in 2024-09-16T23:58:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:05 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-16T23:58:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:06.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.326+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe08c0611d0 con 0x7fe07406e680 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.327+0000 7fe0897fa700 1 -- 172.21.15.42:0/3733384330 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe08c0611d0 con 0x7fe07406e680 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe07406e680 msgr2=0x7fe074070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe07406e680 0x7fe074070b40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe07c006010 tx=0x7fe07c01a040 comp rx=0 tx=0).stop 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c101850 msgr2=0x7fe08c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c101850 0x7fe08c0735d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe08000b480 tx=0x7fe08000b790 comp rx=0 tx=0).stop 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 shutdown_connections 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe07406e680 0x7fe074070b40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe08c100ea0 0x7fe08c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 --2- 172.21.15.42:0/3733384330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe08c101850 0x7fe08c0735d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 >> 172.21.15.42:0/3733384330 conn(0x7fe08c0fc750 msgr2=0x7fe08c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 shutdown_connections 2024-09-16T23:58:06.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.330+0000 7fe0922c6700 1 -- 172.21.15.42:0/3733384330 wait complete. 2024-09-16T23:58:06.339 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 -- 172.21.15.42:0/2732522819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 msgr2=0x7fe840105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 --2- 172.21.15.42:0/2732522819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840105840 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe834009910 tx=0x7fe834009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 -- 172.21.15.42:0/2732522819 shutdown_connections 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 --2- 172.21.15.42:0/2732522819 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840105840 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 --2- 172.21.15.42:0/2732522819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8400ff5a0 0x7fe8400ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 -- 172.21.15.42:0/2732522819 >> 172.21.15.42:0/2732522819 conn(0x7fe840074e30 msgr2=0x7fe840075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 -- 172.21.15.42:0/2732522819 shutdown_connections 2024-09-16T23:58:06.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.439+0000 7fe845909700 1 -- 172.21.15.42:0/2732522819 wait complete. 2024-09-16T23:58:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe845909700 1 Processor -- start 2024-09-16T23:58:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe845909700 1 -- start start 2024-09-16T23:58:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe845909700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8400ff5a0 0x7fe840100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe845909700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe845909700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe840100fa0 con 0x7fe8400fff50 2024-09-16T23:58:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe845909700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8401010e0 con 0x7fe8400ff5a0 2024-09-16T23:58:06.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe83e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56358/0 (socket says 172.21.15.42:56358) 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe83e7fc700 1 -- 172.21.15.42:0/3793611914 learned_addr learned my addr 172.21.15.42:0/3793611914 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.440+0000 7fe83effd700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8400ff5a0 0x7fe840100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe83e7fc700 1 -- 172.21.15.42:0/3793611914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8400ff5a0 msgr2=0x7fe840100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe83e7fc700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8400ff5a0 0x7fe840100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe83e7fc700 1 -- 172.21.15.42:0/3793611914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340095f0 con 0x7fe8400fff50 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe83effd700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8400ff5a0 0x7fe840100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe83e7fc700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840100a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe834009fd0 tx=0x7fe83400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe844907700 1 -- 172.21.15.42:0/3793611914 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83401d070 con 0x7fe8400fff50 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe844907700 1 -- 172.21.15.42:0/3793611914 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe834022470 con 0x7fe8400fff50 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe840101360 con 0x7fe8400fff50 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe844907700 1 -- 172.21.15.42:0/3793611914 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe834017650 con 0x7fe8400fff50 2024-09-16T23:58:06.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.441+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8401a2d10 con 0x7fe8400fff50 2024-09-16T23:58:06.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.442+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840109200 con 0x7fe8400fff50 2024-09-16T23:58:06.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.443+0000 7fe844907700 1 -- 172.21.15.42:0/3793611914 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe8340225e0 con 0x7fe8400fff50 2024-09-16T23:58:06.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.443+0000 7fe844907700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe82806e680 0x7fe828070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.443+0000 7fe844907700 1 -- 172.21.15.42:0/3793611914 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe83408fbc0 con 0x7fe8400fff50 2024-09-16T23:58:06.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.443+0000 7fe83effd700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe82806e680 0x7fe828070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.444+0000 7fe83effd700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe82806e680 0x7fe828070b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe830011490 tx=0x7fe830009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:06.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.448+0000 7fe844907700 1 -- 172.21.15.42:0/3793611914 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe834058d60 con 0x7fe8400fff50 2024-09-16T23:58:06.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.678+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8400611d0 con 0x7fe82806e680 2024-09-16T23:58:06.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.679+0000 7fe844907700 1 -- 172.21.15.42:0/3793611914 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe8400611d0 con 0x7fe82806e680 2024-09-16T23:58:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe82806e680 msgr2=0x7fe828070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe82806e680 0x7fe828070b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe830011490 tx=0x7fe830009260 comp rx=0 tx=0).stop 2024-09-16T23:58:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 msgr2=0x7fe840100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840100a60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe834009fd0 tx=0x7fe83400f690 comp rx=0 tx=0).stop 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 shutdown_connections 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe82806e680 0x7fe828070b40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8400ff5a0 0x7fe840100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 --2- 172.21.15.42:0/3793611914 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe8400fff50 0x7fe840100a60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 >> 172.21.15.42:0/3793611914 conn(0x7fe840074e30 msgr2=0x7fe8400fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 shutdown_connections 2024-09-16T23:58:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.682+0000 7fe845909700 1 -- 172.21.15.42:0/3793611914 wait complete. 2024-09-16T23:58:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.790+0000 7fee10de9700 1 -- 172.21.15.42:0/3688414676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c0ffda0 msgr2=0x7fee0c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.790+0000 7fee10de9700 1 --2- 172.21.15.42:0/3688414676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c0ffda0 0x7fee0c100180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009910 tx=0x7fedfc009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:06.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.790+0000 7fee10de9700 1 -- 172.21.15.42:0/3688414676 shutdown_connections 2024-09-16T23:58:06.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.790+0000 7fee10de9700 1 --2- 172.21.15.42:0/3688414676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee0c100750 0x7fee0c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.790+0000 7fee10de9700 1 --2- 172.21.15.42:0/3688414676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c0ffda0 0x7fee0c100180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.790+0000 7fee10de9700 1 -- 172.21.15.42:0/3688414676 >> 172.21.15.42:0/3688414676 conn(0x7fee0c0fb650 msgr2=0x7fee0c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:06.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.791+0000 7fee10de9700 1 -- 172.21.15.42:0/3688414676 shutdown_connections 2024-09-16T23:58:06.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.791+0000 7fee10de9700 1 -- 172.21.15.42:0/3688414676 wait complete. 2024-09-16T23:58:06.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.791+0000 7fee10de9700 1 Processor -- start 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.791+0000 7fee10de9700 1 -- start start 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee10de9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee0c0ffda0 0x7fee0c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee10de9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c100750 0x7fee0c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee10de9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c1022c0 con 0x7fee0c100750 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee10de9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c102400 con 0x7fee0c0ffda0 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee03fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c100750 0x7fee0c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee0a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee0c0ffda0 0x7fee0c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee03fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c100750 0x7fee0c101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56364/0 (socket says 172.21.15.42:56364) 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee03fff700 1 -- 172.21.15.42:0/2296507381 learned_addr learned my addr 172.21.15.42:0/2296507381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee03fff700 1 -- 172.21.15.42:0/2296507381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee0c0ffda0 msgr2=0x7fee0c101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee03fff700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee0c0ffda0 0x7fee0c101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:06.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee03fff700 1 -- 172.21.15.42:0/2296507381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc0095f0 con 0x7fee0c100750 2024-09-16T23:58:06.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee0a59c700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee0c0ffda0 0x7fee0c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:58:06.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee03fff700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c100750 0x7fee0c101d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e8f0 tx=0x7fedf400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:06.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee037fe700 1 -- 172.21.15.42:0/2296507381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedf4009790 con 0x7fee0c100750 2024-09-16T23:58:06.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.792+0000 7fee037fe700 1 -- 172.21.15.42:0/2296507381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedf400c810 con 0x7fee0c100750 2024-09-16T23:58:06.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.793+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0c1a2ae0 con 0x7fee0c100750 2024-09-16T23:58:06.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.793+0000 7fee037fe700 1 -- 172.21.15.42:0/2296507381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedf4005580 con 0x7fee0c100750 2024-09-16T23:58:06.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.793+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c1a2ef0 con 0x7fee0c100750 2024-09-16T23:58:06.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.794+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c04f2e0 con 0x7fee0c100750 2024-09-16T23:58:06.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.794+0000 7fee037fe700 1 -- 172.21.15.42:0/2296507381 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fedf400c980 con 0x7fee0c100750 2024-09-16T23:58:06.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.795+0000 7fee037fe700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fedf806e680 0x7fedf8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:06.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.795+0000 7fee037fe700 1 -- 172.21.15.42:0/2296507381 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fedf4014070 con 0x7fee0c100750 2024-09-16T23:58:06.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.795+0000 7fee0a59c700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fedf806e680 0x7fedf8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:06.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.795+0000 7fee0a59c700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fedf806e680 0x7fedf8070b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005b40 tx=0x7fedfc005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:06.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:06.799+0000 7fee037fe700 1 -- 172.21.15.42:0/2296507381 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fedf4057a40 con 0x7fee0c100750 2024-09-16T23:58:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:06 smithi043 ceph-mon[81510]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:07.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.018+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fee0c0fc440 con 0x7fedf806e680 2024-09-16T23:58:07.017 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:06 smithi042 ceph-mon[111056]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:07.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.028+0000 7fee037fe700 1 -- 172.21.15.42:0/2296507381 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fee0c0fc440 con 0x7fedf806e680 2024-09-16T23:58:07.027 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:58:07.027 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (6m) 24s ago 17m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:58:07.027 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (17m) 24s ago 17m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (16m) 9s ago 16m 8749k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (4m) 24s ago 17m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (4m) 9s ago 16m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (6m) 24s ago 17m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (29s) 24s ago 11m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (46s) 24s ago 11m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (13s) 9s ago 11m 12.9M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (11m) 9s ago 11m 18.9M - 18.2.1 5be31c24972a 6920f4678224 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (8m) 24s ago 18m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (7m) 9s ago 16m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (5m) 24s ago 19m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (5m) 9s ago 16m 42.9M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:58:07.028 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (7m) 24s ago 17m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (7m) 9s ago 16m 15.6M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (3m) 24s ago 15m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (3m) 24s ago 15m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (2m) 24s ago 14m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (2m) 9s ago 14m 89.8M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (97s) 9s ago 13m 92.5M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (63s) 9s ago 13m 86.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (7m) 24s ago 17m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:58:07.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fedf806e680 msgr2=0x7fedf8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fedf806e680 0x7fedf8070b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fedfc005b40 tx=0x7fedfc005a90 comp rx=0 tx=0).stop 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c100750 msgr2=0x7fee0c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c100750 0x7fee0c101d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedf400e8f0 tx=0x7fedf400ec00 comp rx=0 tx=0).stop 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 shutdown_connections 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fedf806e680 0x7fedf8070b40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fee0c0ffda0 0x7fee0c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 --2- 172.21.15.42:0/2296507381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee0c100750 0x7fee0c101d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 >> 172.21.15.42:0/2296507381 conn(0x7fee0c0fb650 msgr2=0x7fee0c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 shutdown_connections 2024-09-16T23:58:07.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.031+0000 7fee10de9700 1 -- 172.21.15.42:0/2296507381 wait complete. 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 -- 172.21.15.42:0/2293472644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80101850 msgr2=0x7f4f801058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 --2- 172.21.15.42:0/2293472644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80101850 0x7f4f801058a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009910 tx=0x7f4f70009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 -- 172.21.15.42:0/2293472644 shutdown_connections 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 --2- 172.21.15.42:0/2293472644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80101850 0x7f4f801058a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 --2- 172.21.15.42:0/2293472644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f80100ea0 0x7f4f80101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 -- 172.21.15.42:0/2293472644 >> 172.21.15.42:0/2293472644 conn(0x7f4f800fc750 msgr2=0x7f4f800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 -- 172.21.15.42:0/2293472644 shutdown_connections 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.138+0000 7f4f8837b700 1 -- 172.21.15.42:0/2293472644 wait complete. 2024-09-16T23:58:07.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f8837b700 1 Processor -- start 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f8837b700 1 -- start start 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f8837b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80100ea0 0x7f4f80073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f8837b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f80101850 0x7f4f800735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f8837b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80073b10 con 0x7f4f80100ea0 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f8837b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80073c50 con 0x7f4f80101850 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f86117700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80100ea0 0x7f4f80073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f86117700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80100ea0 0x7f4f80073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56388/0 (socket says 172.21.15.42:56388) 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f86117700 1 -- 172.21.15.42:0/3396208472 learned_addr learned my addr 172.21.15.42:0/3396208472 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f85916700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f80101850 0x7f4f800735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f86117700 1 -- 172.21.15.42:0/3396208472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f80101850 msgr2=0x7f4f800735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f86117700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f80101850 0x7f4f800735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.139+0000 7f4f86117700 1 -- 172.21.15.42:0/3396208472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f700095f0 con 0x7f4f80100ea0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.140+0000 7f4f86117700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80100ea0 0x7f4f80073090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0060b0 tx=0x7f4f7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.140+0000 7f4f777fe700 1 -- 172.21.15.42:0/3396208472 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f7c00dda0 con 0x7f4f80100ea0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.140+0000 7f4f777fe700 1 -- 172.21.15.42:0/3396208472 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f7c00f040 con 0x7f4f80100ea0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.140+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f801a2b00 con 0x7f4f80100ea0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.140+0000 7f4f777fe700 1 -- 172.21.15.42:0/3396208472 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f7c0146f0 con 0x7f4f80100ea0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.140+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f801a2f10 con 0x7f4f80100ea0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.141+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f801092a0 con 0x7f4f80100ea0 2024-09-16T23:58:07.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.141+0000 7f4f777fe700 1 -- 172.21.15.42:0/3396208472 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f4f7c015400 con 0x7f4f80100ea0 2024-09-16T23:58:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.142+0000 7f4f777fe700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4f6c06e680 0x7f4f6c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.142+0000 7f4f777fe700 1 -- 172.21.15.42:0/3396208472 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4f7c08dfa0 con 0x7f4f80100ea0 2024-09-16T23:58:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.142+0000 7f4f85916700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4f6c06e680 0x7f4f6c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.142+0000 7f4f85916700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4f6c06e680 0x7f4f6c070b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f80074880 tx=0x7f4f7001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:07.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.146+0000 7f4f777fe700 1 -- 172.21.15.42:0/3396208472 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4f7c057270 con 0x7f4f80100ea0 2024-09-16T23:58:07.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.442+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4f8004f2e0 con 0x7f4f80100ea0 2024-09-16T23:58:07.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.442+0000 7f4f777fe700 1 -- 172.21.15.42:0/3396208472 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f4f7c05a930 con 0x7f4f80100ea0 2024-09-16T23:58:07.441 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:58:07.441 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 3 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:58:07.442 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 13 2024-09-16T23:58:07.443 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:58:07.443 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:58:07.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.444+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4f6c06e680 msgr2=0x7f4f6c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.444+0000 7f4f8837b700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4f6c06e680 0x7f4f6c070b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4f80074880 tx=0x7f4f7001a040 comp rx=0 tx=0).stop 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.444+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80100ea0 msgr2=0x7f4f80073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.444+0000 7f4f8837b700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80100ea0 0x7f4f80073090 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0060b0 tx=0x7f4f7c00d3b0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.445+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 shutdown_connections 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.445+0000 7f4f8837b700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4f6c06e680 0x7f4f6c070b40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.445+0000 7f4f8837b700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4f80100ea0 0x7f4f80073090 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.445+0000 7f4f8837b700 1 --2- 172.21.15.42:0/3396208472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f80101850 0x7f4f800735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.445+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 >> 172.21.15.42:0/3396208472 conn(0x7f4f800fc750 msgr2=0x7f4f800fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.445+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 shutdown_connections 2024-09-16T23:58:07.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.445+0000 7f4f8837b700 1 -- 172.21.15.42:0/3396208472 wait complete. 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 -- 172.21.15.42:0/776096661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068100e90 msgr2=0x7ff068101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/776096661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068100e90 0x7ff068101270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff05800b0a0 tx=0x7ff05800b3b0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 -- 172.21.15.42:0/776096661 shutdown_connections 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/776096661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068101840 0x7ff068105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/776096661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068100e90 0x7ff068101270 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 -- 172.21.15.42:0/776096661 >> 172.21.15.42:0/776096661 conn(0x7ff0680fc740 msgr2=0x7ff0680feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 -- 172.21.15.42:0/776096661 shutdown_connections 2024-09-16T23:58:07.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.553+0000 7ff06f3fc700 1 -- 172.21.15.42:0/776096661 wait complete. 2024-09-16T23:58:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06f3fc700 1 Processor -- start 2024-09-16T23:58:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06f3fc700 1 -- start start 2024-09-16T23:58:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06f3fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068100e90 0x7ff06819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06f3fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068101840 0x7ff06819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06f3fc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06819d230 con 0x7ff068101840 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06f3fc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0681966e0 con 0x7ff068100e90 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06c997700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068101840 0x7ff06819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06d198700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068100e90 0x7ff06819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06d198700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068100e90 0x7ff06819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:44838/0 (socket says 172.21.15.42:44838) 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06c997700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068101840 0x7ff06819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56408/0 (socket says 172.21.15.42:56408) 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06d198700 1 -- 172.21.15.42:0/3689605497 learned_addr learned my addr 172.21.15.42:0/3689605497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06d198700 1 -- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068101840 msgr2=0x7ff06819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06d198700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068101840 0x7ff06819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06d198700 1 -- 172.21.15.42:0/3689605497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058009d00 con 0x7ff068100e90 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.554+0000 7ff06c997700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068101840 0x7ff06819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.555+0000 7ff06d198700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068100e90 0x7ff06819c660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff058000f80 tx=0x7ff05800cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:07.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.555+0000 7ff0667fc700 1 -- 172.21.15.42:0/3689605497 <== mon.1 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05800e040 con 0x7ff068100e90 2024-09-16T23:58:07.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.555+0000 7ff0667fc700 1 -- 172.21.15.42:0/3689605497 <== mon.1 v2:172.21.15.43:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff058003c10 con 0x7ff068100e90 2024-09-16T23:58:07.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.555+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff068196960 con 0x7ff068100e90 2024-09-16T23:58:07.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.555+0000 7ff0667fc700 1 -- 172.21.15.42:0/3689605497 <== mon.1 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05801d6b0 con 0x7ff068100e90 2024-09-16T23:58:07.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.555+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff068196e50 con 0x7ff068100e90 2024-09-16T23:58:07.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.556+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff068109230 con 0x7ff068100e90 2024-09-16T23:58:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.556+0000 7ff0667fc700 1 -- 172.21.15.42:0/3689605497 <== mon.1 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff058004040 con 0x7ff068100e90 2024-09-16T23:58:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.557+0000 7ff0667fc700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff05006e680 0x7ff050070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.557+0000 7ff0667fc700 1 -- 172.21.15.42:0/3689605497 <== mon.1 v2:172.21.15.43:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff05808fe10 con 0x7ff068100e90 2024-09-16T23:58:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.557+0000 7ff06c997700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff05006e680 0x7ff050070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.557+0000 7ff06c997700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff05006e680 0x7ff050070b40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00ffd0 tx=0x7ff05c00f820 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:07.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.562+0000 7ff0667fc700 1 -- 172.21.15.42:0/3689605497 <== mon.1 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff05805cac0 con 0x7ff068100e90 2024-09-16T23:58:07.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.817+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff068108c10 con 0x7ff068100e90 2024-09-16T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.818+0000 7ff0667fc700 1 -- 172.21.15.42:0/3689605497 <== mon.1 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1733 (secure 0 0 0) 0x7ff0580916e0 con 0x7ff068100e90 2024-09-16T23:58:07.817 INFO:teuthology.orchestra.run.smithi042.stdout:e28 2024-09-16T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:04:819536+0000 2024-09-16T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:58:07.818 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-16T23:58:07.819 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:58:07.820 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-16T23:58:07.821 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-16T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-16T23:58:07.822 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:07.823 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:58:07.824 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff05006e680 msgr2=0x7ff050070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff05006e680 0x7ff050070b40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00ffd0 tx=0x7ff05c00f820 comp rx=0 tx=0).stop 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068100e90 msgr2=0x7ff06819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068100e90 0x7ff06819c660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff058000f80 tx=0x7ff05800cd20 comp rx=0 tx=0).stop 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 shutdown_connections 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff05006e680 0x7ff050070b40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068100e90 0x7ff06819c660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 --2- 172.21.15.42:0/3689605497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff068101840 0x7ff06819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 >> 172.21.15.42:0/3689605497 conn(0x7ff0680fc740 msgr2=0x7ff0680fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 shutdown_connections 2024-09-16T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.820+0000 7ff06f3fc700 1 -- 172.21.15.42:0/3689605497 wait complete. 2024-09-16T23:58:07.826 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-09-16T23:58:07.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 -- 172.21.15.42:0/2242659060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c105350 msgr2=0x7f928c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 --2- 172.21.15.42:0/2242659060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c105350 0x7f928c105730 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9280009910 tx=0x7f9280009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:07.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 -- 172.21.15.42:0/2242659060 shutdown_connections 2024-09-16T23:58:07.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 --2- 172.21.15.42:0/2242659060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f928c0ff340 0x7f928c0ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 --2- 172.21.15.42:0/2242659060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c105350 0x7f928c105730 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 -- 172.21.15.42:0/2242659060 >> 172.21.15.42:0/2242659060 conn(0x7f928c074e30 msgr2=0x7f928c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:07.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 -- 172.21.15.42:0/2242659060 shutdown_connections 2024-09-16T23:58:07.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 -- 172.21.15.42:0/2242659060 wait complete. 2024-09-16T23:58:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 Processor -- start 2024-09-16T23:58:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.931+0000 7f9292cac700 1 -- start start 2024-09-16T23:58:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9292cac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c0ff340 0x7f928c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9292cac700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f928c105350 0x7f928c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9292cac700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c19d240 con 0x7f928c0ff340 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9292cac700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c1966f0 con 0x7f928c105350 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9290a48700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c0ff340 0x7f928c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9290a48700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c0ff340 0x7f928c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56426/0 (socket says 172.21.15.42:56426) 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9290a48700 1 -- 172.21.15.42:0/2204254203 learned_addr learned my addr 172.21.15.42:0/2204254203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f928bfff700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f928c105350 0x7f928c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9290a48700 1 -- 172.21.15.42:0/2204254203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f928c105350 msgr2=0x7f928c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9290a48700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f928c105350 0x7f928c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9290a48700 1 -- 172.21.15.42:0/2204254203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92800095f0 con 0x7f928c0ff340 2024-09-16T23:58:07.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9290a48700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c0ff340 0x7f928c19c670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9280006010 tx=0x7f928000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9289ffb700 1 -- 172.21.15.42:0/2204254203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f928001d070 con 0x7f928c0ff340 2024-09-16T23:58:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9289ffb700 1 -- 172.21.15.42:0/2204254203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9280005dc0 con 0x7f928c0ff340 2024-09-16T23:58:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9289ffb700 1 -- 172.21.15.42:0/2204254203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9280017400 con 0x7f928c0ff340 2024-09-16T23:58:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.932+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c196970 con 0x7f928c0ff340 2024-09-16T23:58:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.933+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c196e60 con 0x7f928c0ff340 2024-09-16T23:58:07.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.934+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c109880 con 0x7f928c0ff340 2024-09-16T23:58:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.934+0000 7f9289ffb700 1 -- 172.21.15.42:0/2204254203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9280003a60 con 0x7f928c0ff340 2024-09-16T23:58:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.935+0000 7f9289ffb700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f927c06e680 0x7f927c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.935+0000 7f9289ffb700 1 -- 172.21.15.42:0/2204254203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f928008fae0 con 0x7f928c0ff340 2024-09-16T23:58:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.935+0000 7f928bfff700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f927c06e680 0x7f927c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:07.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.935+0000 7f928bfff700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f927c06e680 0x7f927c070b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f928c197b60 tx=0x7f927800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:07.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:07.939+0000 7f9289ffb700 1 -- 172.21.15.42:0/2204254203 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f928005c7b0 con 0x7f928c0ff340 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: pgmap v196: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3396208472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:58:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:07 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3689605497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:58:08.168 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.169+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f928c197ca0 con 0x7f927c06e680 2024-09-16T23:58:08.168 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: pgmap v196: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:08.168 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:08.168 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:08.168 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3396208472' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:08.169 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:08.169 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:08.169 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T23:58:08.169 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:07 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3689605497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:58:08.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.170+0000 7f9289ffb700 1 -- 172.21.15.42:0/2204254203 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f928c197ca0 con 0x7f927c06e680 2024-09-16T23:58:08.169 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr", 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "osd" 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "15/23 daemons upgraded", 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T23:58:08.170 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:58:08.171 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:58:08.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.173+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f927c06e680 msgr2=0x7f927c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.173+0000 7f9292cac700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f927c06e680 0x7f927c070b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f928c197b60 tx=0x7f927800b410 comp rx=0 tx=0).stop 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.173+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c0ff340 msgr2=0x7f928c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.173+0000 7f9292cac700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c0ff340 0x7f928c19c670 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9280006010 tx=0x7f928000bd70 comp rx=0 tx=0).stop 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.174+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 shutdown_connections 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.174+0000 7f9292cac700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f927c06e680 0x7f927c070b40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.174+0000 7f9292cac700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f928c0ff340 0x7f928c19c670 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.174+0000 7f9292cac700 1 --2- 172.21.15.42:0/2204254203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f928c105350 0x7f928c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.174+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 >> 172.21.15.42:0/2204254203 conn(0x7f928c074e30 msgr2=0x7f928c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.174+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 shutdown_connections 2024-09-16T23:58:08.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.174+0000 7f9292cac700 1 -- 172.21.15.42:0/2204254203 wait complete. 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 -- 172.21.15.42:0/67303967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b880ffc20 msgr2=0x7f5b88100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 --2- 172.21.15.42:0/67303967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b880ffc20 0x7f5b88100080 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009910 tx=0x7f5b78009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 -- 172.21.15.42:0/67303967 shutdown_connections 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 --2- 172.21.15.42:0/67303967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b880ffc20 0x7f5b88100080 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 --2- 172.21.15.42:0/67303967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b88105bd0 0x7f5b88105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 -- 172.21.15.42:0/67303967 >> 172.21.15.42:0/67303967 conn(0x7f5b88075670 msgr2=0x7f5b88075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 -- 172.21.15.42:0/67303967 shutdown_connections 2024-09-16T23:58:08.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.283+0000 7f5b8facd700 1 -- 172.21.15.42:0/67303967 wait complete. 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8facd700 1 Processor -- start 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8facd700 1 -- start start 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8facd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b880ffc20 0x7f5b8819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8facd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b88105bd0 0x7f5b8819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8facd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8819d2c0 con 0x7f5b88105bd0 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8facd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b88196740 con 0x7f5b880ffc20 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8d068700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b88105bd0 0x7f5b8819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8d068700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b88105bd0 0x7f5b8819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56450/0 (socket says 172.21.15.42:56450) 2024-09-16T23:58:08.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8d068700 1 -- 172.21.15.42:0/196633000 learned_addr learned my addr 172.21.15.42:0/196633000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.284+0000 7f5b8d869700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b880ffc20 0x7f5b8819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b8d068700 1 -- 172.21.15.42:0/196633000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b880ffc20 msgr2=0x7f5b8819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b8d068700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b880ffc20 0x7f5b8819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b8d068700 1 -- 172.21.15.42:0/196633000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b780095f0 con 0x7f5b88105bd0 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b8d068700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b88105bd0 0x7f5b8819cbe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b78006010 tx=0x7f5b7800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b7effd700 1 -- 172.21.15.42:0/196633000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7801d070 con 0x7f5b88105bd0 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b881969c0 con 0x7f5b88105bd0 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b7effd700 1 -- 172.21.15.42:0/196633000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b78005dc0 con 0x7f5b88105bd0 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b7effd700 1 -- 172.21.15.42:0/196633000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b78017400 con 0x7f5b88105bd0 2024-09-16T23:58:08.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.285+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b88196eb0 con 0x7f5b88105bd0 2024-09-16T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.286+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8804f2e0 con 0x7f5b88105bd0 2024-09-16T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.287+0000 7f5b7effd700 1 -- 172.21.15.42:0/196633000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5b7800f460 con 0x7f5b88105bd0 2024-09-16T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.287+0000 7f5b7effd700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b7406e680 0x7f5b74070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.287+0000 7f5b7effd700 1 -- 172.21.15.42:0/196633000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5b7808fa60 con 0x7f5b88105bd0 2024-09-16T23:58:08.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.287+0000 7f5b8d869700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b7406e680 0x7f5b74070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:08.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.288+0000 7f5b8d869700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b7406e680 0x7f5b74070b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005950 tx=0x7f5b8400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:08.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.291+0000 7f5b7effd700 1 -- 172.21.15.42:0/196633000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5b7805c7f0 con 0x7f5b88105bd0 2024-09-16T23:58:08.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.593+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b88197170 con 0x7f5b88105bd0 2024-09-16T23:58:08.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.594+0000 7f5b7effd700 1 -- 172.21.15.42:0/196633000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5b7805c380 con 0x7f5b88105bd0 2024-09-16T23:58:08.593 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T23:58:08.593 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T23:58:08.593 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-16T23:58:08.593 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:58:08.593 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:58:08.593 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T23:58:08.593 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T23:58:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b7406e680 msgr2=0x7f5b74070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b7406e680 0x7f5b74070b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5b84005950 tx=0x7f5b8400b410 comp rx=0 tx=0).stop 2024-09-16T23:58:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b88105bd0 msgr2=0x7f5b8819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:08.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b88105bd0 0x7f5b8819cbe0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b78006010 tx=0x7f5b7800bd70 comp rx=0 tx=0).stop 2024-09-16T23:58:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 shutdown_connections 2024-09-16T23:58:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b7406e680 0x7f5b74070b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b880ffc20 0x7f5b8819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 --2- 172.21.15.42:0/196633000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b88105bd0 0x7f5b8819cbe0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 >> 172.21.15.42:0/196633000 conn(0x7f5b88075670 msgr2=0x7f5b88103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 shutdown_connections 2024-09-16T23:58:08.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:08.596+0000 7f5b8facd700 1 -- 172.21.15.42:0/196633000 wait complete. 2024-09-16T23:58:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:08 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] up:boot 2024-09-16T23:58:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:08 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:58:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:58:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:08 smithi042 ceph-mon[111056]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:08 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/196633000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:58:09.084 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:08 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] up:boot 2024-09-16T23:58:09.084 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:08 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-16T23:58:09.084 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi043.riwwhn"}]: dispatch 2024-09-16T23:58:09.084 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:08 smithi043 ceph-mon[81510]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:09.084 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:08 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/196633000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:58:10.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:09 smithi043 ceph-mon[81510]: pgmap v197: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:10.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:09 smithi042 ceph-mon[111056]: pgmap v197: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:11.811 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:11.812 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:11.812 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:11.812 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-16T23:58:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:12 smithi043 ceph-mon[81510]: pgmap v198: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:12 smithi042 ceph-mon[111056]: pgmap v198: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:14.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:14 smithi042 ceph-mon[111056]: pgmap v199: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:14 smithi043 ceph-mon[81510]: pgmap v199: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:16.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:16 smithi042 ceph-mon[111056]: pgmap v200: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:16 smithi043 ceph-mon[81510]: pgmap v200: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:18.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:18 smithi042 ceph-mon[111056]: pgmap v201: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:18.835 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:18 smithi043 ceph-mon[81510]: pgmap v201: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:20.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:20 smithi042 ceph-mon[111056]: pgmap v202: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:20.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:20 smithi043 ceph-mon[81510]: pgmap v202: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:22.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:22 smithi042 ceph-mon[111056]: pgmap v203: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:22.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:22 smithi043 ceph-mon[81510]: pgmap v203: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:24.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:24 smithi042 ceph-mon[111056]: pgmap v204: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:24.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:24 smithi043 ceph-mon[81510]: pgmap v204: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:26.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:26 smithi042 ceph-mon[111056]: pgmap v205: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:26.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:26 smithi043 ceph-mon[81510]: pgmap v205: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:28 smithi042 ceph-mon[111056]: pgmap v206: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:28.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:28 smithi043 ceph-mon[81510]: pgmap v206: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:30.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:30 smithi042 ceph-mon[111056]: pgmap v207: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:30.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:30 smithi043 ceph-mon[81510]: pgmap v207: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:32.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:32 smithi042 ceph-mon[111056]: pgmap v208: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:32.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:32 smithi043 ceph-mon[81510]: pgmap v208: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:34 smithi042 ceph-mon[111056]: pgmap v209: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:34 smithi043 ceph-mon[81510]: pgmap v209: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:36 smithi042 ceph-mon[111056]: pgmap v210: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:36 smithi043 ceph-mon[81510]: pgmap v210: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:38.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 -- 172.21.15.42:0/2387730701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860106cd0 msgr2=0x7f68601070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:38.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 --2- 172.21.15.42:0/2387730701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860106cd0 0x7f68601070b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f685000b0a0 tx=0x7f685000b3b0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 -- 172.21.15.42:0/2387730701 shutdown_connections 2024-09-16T23:58:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 --2- 172.21.15.42:0/2387730701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6860068580 0x7f68600689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 --2- 172.21.15.42:0/2387730701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860106cd0 0x7f68601070b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 -- 172.21.15.42:0/2387730701 >> 172.21.15.42:0/2387730701 conn(0x7f6860075670 msgr2=0x7f6860075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 -- 172.21.15.42:0/2387730701 shutdown_connections 2024-09-16T23:58:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.721+0000 7f68660fd700 1 -- 172.21.15.42:0/2387730701 wait complete. 2024-09-16T23:58:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.722+0000 7f68660fd700 1 Processor -- start 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.722+0000 7f68660fd700 1 -- start start 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.722+0000 7f68660fd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860068580 0x7f686019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.722+0000 7f68660fd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6860106cd0 0x7f686019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.722+0000 7f68660fd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686019d260 con 0x7f6860068580 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.722+0000 7f68660fd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6860196710 con 0x7f6860106cd0 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860068580 0x7f686019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860068580 0x7f686019c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52480/0 (socket says 172.21.15.42:52480) 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685f7fe700 1 -- 172.21.15.42:0/2813198674 learned_addr learned my addr 172.21.15.42:0/2813198674 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685effd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6860106cd0 0x7f686019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:38.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685f7fe700 1 -- 172.21.15.42:0/2813198674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6860106cd0 msgr2=0x7f686019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685f7fe700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6860106cd0 0x7f686019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685f7fe700 1 -- 172.21.15.42:0/2813198674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6850009d00 con 0x7f6860068580 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685f7fe700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860068580 0x7f686019c690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f685000b070 tx=0x7f685000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685cff9700 1 -- 172.21.15.42:0/2813198674 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f685000e040 con 0x7f6860068580 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685cff9700 1 -- 172.21.15.42:0/2813198674 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6850003de0 con 0x7f6860068580 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6860196990 con 0x7f6860068580 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.723+0000 7f685cff9700 1 -- 172.21.15.42:0/2813198674 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f685001d8e0 con 0x7f6860068580 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.724+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6860196e80 con 0x7f6860068580 2024-09-16T23:58:38.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.724+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68601091f0 con 0x7f6860068580 2024-09-16T23:58:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.725+0000 7f685cff9700 1 -- 172.21.15.42:0/2813198674 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6850004040 con 0x7f6860068580 2024-09-16T23:58:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.726+0000 7f685cff9700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f684806e680 0x7f6848070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.726+0000 7f685cff9700 1 -- 172.21.15.42:0/2813198674 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f685008ff50 con 0x7f6860068580 2024-09-16T23:58:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.726+0000 7f685effd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f684806e680 0x7f6848070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.726+0000 7f685effd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f684806e680 0x7f6848070b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6854005fd0 tx=0x7f6854005f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:38.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.730+0000 7f685cff9700 1 -- 172.21.15.42:0/2813198674 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f685005cb80 con 0x7f6860068580 2024-09-16T23:58:38.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:38 smithi042 ceph-mon[111056]: pgmap v211: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:38 smithi043 ceph-mon[81510]: pgmap v211: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:38.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.961+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68601970a0 con 0x7f684806e680 2024-09-16T23:58:38.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.962+0000 7f685cff9700 1 -- 172.21.15.42:0/2813198674 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f68601970a0 con 0x7f684806e680 2024-09-16T23:58:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f684806e680 msgr2=0x7f6848070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f684806e680 0x7f6848070b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6854005fd0 tx=0x7f6854005f60 comp rx=0 tx=0).stop 2024-09-16T23:58:38.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860068580 msgr2=0x7f686019c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860068580 0x7f686019c690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f685000b070 tx=0x7f685000cdd0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 shutdown_connections 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f684806e680 0x7f6848070b40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6860068580 0x7f686019c690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 --2- 172.21.15.42:0/2813198674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6860106cd0 0x7f686019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 >> 172.21.15.42:0/2813198674 conn(0x7f6860075670 msgr2=0x7f68600fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 shutdown_connections 2024-09-16T23:58:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:38.964+0000 7f68660fd700 1 -- 172.21.15.42:0/2813198674 wait complete. 2024-09-16T23:58:38.973 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:58:39.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 -- 172.21.15.42:0/1779011177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 msgr2=0x7fc03c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 --2- 172.21.15.42:0/1779011177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c101280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009910 tx=0x7fc02c009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:39.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 -- 172.21.15.42:0/1779011177 shutdown_connections 2024-09-16T23:58:39.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 --2- 172.21.15.42:0/1779011177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c101850 0x7fc03c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 --2- 172.21.15.42:0/1779011177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c101280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 -- 172.21.15.42:0/1779011177 >> 172.21.15.42:0/1779011177 conn(0x7fc03c0fc750 msgr2=0x7fc03c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:39.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 -- 172.21.15.42:0/1779011177 shutdown_connections 2024-09-16T23:58:39.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.073+0000 7fc040f85700 1 -- 172.21.15.42:0/1779011177 wait complete. 2024-09-16T23:58:39.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.074+0000 7fc040f85700 1 Processor -- start 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.074+0000 7fc040f85700 1 -- start start 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.074+0000 7fc040f85700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.074+0000 7fc040f85700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c101850 0x7fc03c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.074+0000 7fc040f85700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c19d2c0 con 0x7fc03c100ea0 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.074+0000 7fc040f85700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c196770 con 0x7fc03c101850 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc03a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52494/0 (socket says 172.21.15.42:52494) 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc03a59c700 1 -- 172.21.15.42:0/111708570 learned_addr learned my addr 172.21.15.42:0/111708570 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc031d9b700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c101850 0x7fc03c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc03a59c700 1 -- 172.21.15.42:0/111708570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c101850 msgr2=0x7fc03c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc03a59c700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c101850 0x7fc03c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc03a59c700 1 -- 172.21.15.42:0/111708570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c0095f0 con 0x7fc03c100ea0 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc03a59c700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c19c6f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc02c006010 tx=0x7fc02c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc033fff700 1 -- 172.21.15.42:0/111708570 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02c01d070 con 0x7fc03c100ea0 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc033fff700 1 -- 172.21.15.42:0/111708570 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc02c005dc0 con 0x7fc03c100ea0 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c1969f0 con 0x7fc03c100ea0 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc033fff700 1 -- 172.21.15.42:0/111708570 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02c017400 con 0x7fc03c100ea0 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.075+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c196ee0 con 0x7fc03c100ea0 2024-09-16T23:58:39.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.076+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c109240 con 0x7fc03c100ea0 2024-09-16T23:58:39.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.077+0000 7fc033fff700 1 -- 172.21.15.42:0/111708570 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc02c003a60 con 0x7fc03c100ea0 2024-09-16T23:58:39.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.078+0000 7fc033fff700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc02806e680 0x7fc028070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.078+0000 7fc031d9b700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc02806e680 0x7fc028070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.078+0000 7fc033fff700 1 -- 172.21.15.42:0/111708570 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc02c08fab0 con 0x7fc03c100ea0 2024-09-16T23:58:39.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.078+0000 7fc031d9b700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc02806e680 0x7fc028070b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc03c197be0 tx=0x7fc02400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:39.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.082+0000 7fc033fff700 1 -- 172.21.15.42:0/111708570 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc02c05c780 con 0x7fc03c100ea0 2024-09-16T23:58:39.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.324+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc03c197d90 con 0x7fc02806e680 2024-09-16T23:58:39.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.325+0000 7fc033fff700 1 -- 172.21.15.42:0/111708570 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc03c197d90 con 0x7fc02806e680 2024-09-16T23:58:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc02806e680 msgr2=0x7fc028070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc02806e680 0x7fc028070b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc03c197be0 tx=0x7fc02400b410 comp rx=0 tx=0).stop 2024-09-16T23:58:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 msgr2=0x7fc03c19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c19c6f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc02c006010 tx=0x7fc02c00bd70 comp rx=0 tx=0).stop 2024-09-16T23:58:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 shutdown_connections 2024-09-16T23:58:39.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc02806e680 0x7fc028070b40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc03c100ea0 0x7fc03c19c6f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 --2- 172.21.15.42:0/111708570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03c101850 0x7fc03c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.327+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 >> 172.21.15.42:0/111708570 conn(0x7fc03c0fc750 msgr2=0x7fc03c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.328+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 shutdown_connections 2024-09-16T23:58:39.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.328+0000 7fc040f85700 1 -- 172.21.15.42:0/111708570 wait complete. 2024-09-16T23:58:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 -- 172.21.15.42:0/600629097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734068580 msgr2=0x7f37340689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 --2- 172.21.15.42:0/600629097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734068580 0x7f37340689e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3724009910 tx=0x7f3724009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 -- 172.21.15.42:0/600629097 shutdown_connections 2024-09-16T23:58:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 --2- 172.21.15.42:0/600629097 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734068580 0x7f37340689e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 --2- 172.21.15.42:0/600629097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3734106cd0 0x7f37341070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 -- 172.21.15.42:0/600629097 >> 172.21.15.42:0/600629097 conn(0x7f3734075670 msgr2=0x7f3734075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 -- 172.21.15.42:0/600629097 shutdown_connections 2024-09-16T23:58:39.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.436+0000 7f373bca8700 1 -- 172.21.15.42:0/600629097 wait complete. 2024-09-16T23:58:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f373bca8700 1 Processor -- start 2024-09-16T23:58:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f373bca8700 1 -- start start 2024-09-16T23:58:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f373bca8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3734068580 0x7f373419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f373bca8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734106cd0 0x7f373419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f373bca8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373419d220 con 0x7f3734106cd0 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f373bca8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37341966d0 con 0x7f3734068580 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f3739243700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734106cd0 0x7f373419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f3739243700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734106cd0 0x7f373419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52520/0 (socket says 172.21.15.42:52520) 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.437+0000 7f3739243700 1 -- 172.21.15.42:0/1841716780 learned_addr learned my addr 172.21.15.42:0/1841716780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f3739a44700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3734068580 0x7f373419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f3739243700 1 -- 172.21.15.42:0/1841716780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3734068580 msgr2=0x7f373419c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f3739243700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3734068580 0x7f373419c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f3739243700 1 -- 172.21.15.42:0/1841716780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37240095f0 con 0x7f3734106cd0 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f3739a44700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3734068580 0x7f373419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f3739243700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734106cd0 0x7f373419cb90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3724006010 tx=0x7f372400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f372affd700 1 -- 172.21.15.42:0/1841716780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f372401d070 con 0x7f3734106cd0 2024-09-16T23:58:39.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f372affd700 1 -- 172.21.15.42:0/1841716780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3724004fb0 con 0x7f3734106cd0 2024-09-16T23:58:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3734196950 con 0x7f3734106cd0 2024-09-16T23:58:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f372affd700 1 -- 172.21.15.42:0/1841716780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3724022ea0 con 0x7f3734106cd0 2024-09-16T23:58:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.438+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3734196e40 con 0x7f3734106cd0 2024-09-16T23:58:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.439+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37341091d0 con 0x7f3734106cd0 2024-09-16T23:58:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.440+0000 7f372affd700 1 -- 172.21.15.42:0/1841716780 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3724022470 con 0x7f3734106cd0 2024-09-16T23:58:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.440+0000 7f372affd700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f372006e680 0x7f3720070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.440+0000 7f372affd700 1 -- 172.21.15.42:0/1841716780 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f372408faf0 con 0x7f3734106cd0 2024-09-16T23:58:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.440+0000 7f3739a44700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f372006e680 0x7f3720070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.441+0000 7f3739a44700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f372006e680 0x7f3720070b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3730005fd0 tx=0x7f3730005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.445+0000 7f372affd700 1 -- 172.21.15.42:0/1841716780 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f372405ca10 con 0x7f3734106cd0 2024-09-16T23:58:39.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.665+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3734197170 con 0x7f372006e680 2024-09-16T23:58:39.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.675+0000 7f372affd700 1 -- 172.21.15.42:0/1841716780 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f3734197170 con 0x7f372006e680 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (7m) 56s ago 18m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (18m) 56s ago 18m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (17m) 29s ago 17m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (4m) 56s ago 18m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (4m) 29s ago 17m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (6m) 56s ago 17m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (62s) 56s ago 12m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (78s) 56s ago 12m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (45s) 29s ago 12m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-16T23:58:39.673 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (32s) 29s ago 12m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (9m) 56s ago 19m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (8m) 29s ago 17m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (5m) 56s ago 19m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (5m) 29s ago 16m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (7m) 56s ago 18m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (7m) 29s ago 17m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (4m) 56s ago 16m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (3m) 56s ago 15m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (3m) 56s ago 15m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (2m) 29s ago 14m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (2m) 29s ago 14m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (96s) 29s ago 13m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-16T23:58:39.674 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (7m) 56s ago 17m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:58:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f372006e680 msgr2=0x7f3720070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f372006e680 0x7f3720070b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3730005fd0 tx=0x7f3730005dc0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734106cd0 msgr2=0x7f373419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734106cd0 0x7f373419cb90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3724006010 tx=0x7f372400be20 comp rx=0 tx=0).stop 2024-09-16T23:58:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 shutdown_connections 2024-09-16T23:58:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f372006e680 0x7f3720070b40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3734068580 0x7f373419c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 --2- 172.21.15.42:0/1841716780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3734106cd0 0x7f373419cb90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.677+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 >> 172.21.15.42:0/1841716780 conn(0x7f3734075670 msgr2=0x7f37340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.678+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 shutdown_connections 2024-09-16T23:58:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.678+0000 7f373bca8700 1 -- 172.21.15.42:0/1841716780 wait complete. 2024-09-16T23:58:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 -- 172.21.15.42:0/2336511492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 msgr2=0x7f06d8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/2336511492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d8100180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f06c800b0a0 tx=0x7f06c800b3b0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 -- 172.21.15.42:0/2336511492 shutdown_connections 2024-09-16T23:58:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/2336511492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d8100750 0x7f06d81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/2336511492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d8100180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 -- 172.21.15.42:0/2336511492 >> 172.21.15.42:0/2336511492 conn(0x7f06d80fb650 msgr2=0x7f06d80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 -- 172.21.15.42:0/2336511492 shutdown_connections 2024-09-16T23:58:39.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.784+0000 7f06dd1b2700 1 -- 172.21.15.42:0/2336511492 wait complete. 2024-09-16T23:58:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.785+0000 7f06dd1b2700 1 Processor -- start 2024-09-16T23:58:39.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.785+0000 7f06dd1b2700 1 -- start start 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06dd1b2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06dd1b2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d8100750 0x7f06d819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06dd1b2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d819f4d0 con 0x7f06d80ffda0 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06dd1b2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8198980 con 0x7f06d8100750 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52540/0 (socket says 172.21.15.42:52540) 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d6d9d700 1 -- 172.21.15.42:0/362934578 learned_addr learned my addr 172.21.15.42:0/362934578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d659c700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d8100750 0x7f06d819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d6d9d700 1 -- 172.21.15.42:0/362934578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d8100750 msgr2=0x7f06d819ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d6d9d700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d8100750 0x7f06d819ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d6d9d700 1 -- 172.21.15.42:0/362934578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c8009d00 con 0x7f06d80ffda0 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06d6d9d700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d819e900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f06c8000f80 tx=0x7f06c800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:39.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.786+0000 7f06cffff700 1 -- 172.21.15.42:0/362934578 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c800e040 con 0x7f06d80ffda0 2024-09-16T23:58:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.787+0000 7f06cffff700 1 -- 172.21.15.42:0/362934578 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06c8003c10 con 0x7f06d80ffda0 2024-09-16T23:58:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.787+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d8198c00 con 0x7f06d80ffda0 2024-09-16T23:58:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.787+0000 7f06cffff700 1 -- 172.21.15.42:0/362934578 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06c801d680 con 0x7f06d80ffda0 2024-09-16T23:58:39.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.787+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d81990f0 con 0x7f06d80ffda0 2024-09-16T23:58:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.787+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d804f2e0 con 0x7f06d80ffda0 2024-09-16T23:58:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.788+0000 7f06cffff700 1 -- 172.21.15.42:0/362934578 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f06c8019070 con 0x7f06d80ffda0 2024-09-16T23:58:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.789+0000 7f06cffff700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f06c406e680 0x7f06c4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.789+0000 7f06cffff700 1 -- 172.21.15.42:0/362934578 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f06c808fbc0 con 0x7f06d80ffda0 2024-09-16T23:58:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.789+0000 7f06d659c700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f06c406e680 0x7f06c4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:39.787 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.789+0000 7f06d659c700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f06c406e680 0x7f06c4070b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06d80fbf10 tx=0x7f06c0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:39.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:39.793+0000 7f06cffff700 1 -- 172.21.15.42:0/362934578 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f06c805c890 con 0x7f06d80ffda0 2024-09-16T23:58:40.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.089+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f06d8199e90 con 0x7f06d80ffda0 2024-09-16T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.089+0000 7f06cffff700 1 -- 172.21.15.42:0/362934578 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f06c8003d80 con 0x7f06d80ffda0 2024-09-16T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:58:40.087 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:58:40.088 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.091+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f06c406e680 msgr2=0x7f06c4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f06c406e680 0x7f06c4070b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f06d80fbf10 tx=0x7f06c0009260 comp rx=0 tx=0).stop 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 msgr2=0x7f06d819e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d819e900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f06c8000f80 tx=0x7f06c800bc60 comp rx=0 tx=0).stop 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 shutdown_connections 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f06c406e680 0x7f06c4070b40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06d80ffda0 0x7f06d819e900 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 --2- 172.21.15.42:0/362934578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06d8100750 0x7f06d819ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 >> 172.21.15.42:0/362934578 conn(0x7f06d80fb650 msgr2=0x7f06d8069700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 shutdown_connections 2024-09-16T23:58:40.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.092+0000 7f06dd1b2700 1 -- 172.21.15.42:0/362934578 wait complete. 2024-09-16T23:58:40.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 -- 172.21.15.42:0/1984482224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb0105bf0 msgr2=0x7f1cb0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/1984482224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb0105bf0 0x7f1cb0105fd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00b0a0 tx=0x7f1cac00b3b0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 -- 172.21.15.42:0/1984482224 shutdown_connections 2024-09-16T23:58:40.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/1984482224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cb00ffb40 0x7f1cb00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/1984482224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb0105bf0 0x7f1cb0105fd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 -- 172.21.15.42:0/1984482224 >> 172.21.15.42:0/1984482224 conn(0x7f1cb00fb610 msgr2=0x7f1cb00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:40.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 -- 172.21.15.42:0/1984482224 shutdown_connections 2024-09-16T23:58:40.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.200+0000 7f1cb72b1700 1 -- 172.21.15.42:0/1984482224 wait complete. 2024-09-16T23:58:40.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb72b1700 1 Processor -- start 2024-09-16T23:58:40.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb72b1700 1 -- start start 2024-09-16T23:58:40.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb72b1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb00ffb40 0x7f1cb019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb72b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cb0105bf0 0x7f1cb019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb72b1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb019f390 con 0x7f1cb00ffb40 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb72b1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0198840 con 0x7f1cb0105bf0 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb504d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb00ffb40 0x7f1cb019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb504d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb00ffb40 0x7f1cb019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52548/0 (socket says 172.21.15.42:52548) 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb504d700 1 -- 172.21.15.42:0/3602106595 learned_addr learned my addr 172.21.15.42:0/3602106595 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb484c700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cb0105bf0 0x7f1cb019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb504d700 1 -- 172.21.15.42:0/3602106595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cb0105bf0 msgr2=0x7f1cb019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb504d700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cb0105bf0 0x7f1cb019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.201+0000 7f1cb504d700 1 -- 172.21.15.42:0/3602106595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac009d00 con 0x7f1cb00ffb40 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.202+0000 7f1cb504d700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb00ffb40 0x7f1cb019e7c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1cac000f80 tx=0x7f1cac00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:40.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.202+0000 7f1ca67fc700 1 -- 172.21.15.42:0/3602106595 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac00e040 con 0x7f1cb00ffb40 2024-09-16T23:58:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.202+0000 7f1ca67fc700 1 -- 172.21.15.42:0/3602106595 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cac003de0 con 0x7f1cb00ffb40 2024-09-16T23:58:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.202+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb0198ac0 con 0x7f1cb00ffb40 2024-09-16T23:58:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.202+0000 7f1ca67fc700 1 -- 172.21.15.42:0/3602106595 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac01d790 con 0x7f1cb00ffb40 2024-09-16T23:58:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.202+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb0198fb0 con 0x7f1cb00ffb40 2024-09-16T23:58:40.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.203+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb004f2e0 con 0x7f1cb00ffb40 2024-09-16T23:58:40.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.204+0000 7f1ca67fc700 1 -- 172.21.15.42:0/3602106595 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1cac004040 con 0x7f1cb00ffb40 2024-09-16T23:58:40.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.204+0000 7f1ca67fc700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c9c06e680 0x7f1c9c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.204+0000 7f1ca67fc700 1 -- 172.21.15.42:0/3602106595 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1cac08fdc0 con 0x7f1cb00ffb40 2024-09-16T23:58:40.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.204+0000 7f1cb484c700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c9c06e680 0x7f1c9c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.205+0000 7f1cb484c700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c9c06e680 0x7f1c9c070b40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005fd0 tx=0x7f1ca0005f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:40.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.208+0000 7f1ca67fc700 1 -- 172.21.15.42:0/3602106595 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1cac05cb10 con 0x7f1cb00ffb40 2024-09-16T23:58:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.473+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1cb0199220 con 0x7f1cb00ffb40 2024-09-16T23:58:40.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.473+0000 7f1ca67fc700 1 -- 172.21.15.42:0/3602106595 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f1cac017790 con 0x7f1cb00ffb40 2024-09-16T23:58:40.471 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-16T23:58:40.471 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-16T23:58:40.471 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:58:40.472 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-16T23:58:40.473 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:58:40.474 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:58:40.475 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:58:40.475 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:58:40.475 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:58:40.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.475+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c9c06e680 msgr2=0x7f1c9c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.475+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c9c06e680 0x7f1c9c070b40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0005fd0 tx=0x7f1ca0005f60 comp rx=0 tx=0).stop 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.475+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb00ffb40 msgr2=0x7f1cb019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.475+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb00ffb40 0x7f1cb019e7c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1cac000f80 tx=0x7f1cac00cdd0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.476+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 shutdown_connections 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.476+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c9c06e680 0x7f1c9c070b40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.476+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1cb00ffb40 0x7f1cb019e7c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.476+0000 7f1cb72b1700 1 --2- 172.21.15.42:0/3602106595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1cb0105bf0 0x7f1cb019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.476+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 >> 172.21.15.42:0/3602106595 conn(0x7f1cb00fb610 msgr2=0x7f1cb0068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.476+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 shutdown_connections 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.476+0000 7f1cb72b1700 1 -- 172.21.15.42:0/3602106595 wait complete. 2024-09-16T23:58:40.476 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-16T23:58:40.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 -- 172.21.15.42:0/1983846027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490100ea0 msgr2=0x7f1490101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 --2- 172.21.15.42:0/1983846027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490100ea0 0x7f1490101280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1480009910 tx=0x7f1480009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 -- 172.21.15.42:0/1983846027 shutdown_connections 2024-09-16T23:58:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 --2- 172.21.15.42:0/1983846027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1490101850 0x7f14901058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 --2- 172.21.15.42:0/1983846027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490100ea0 0x7f1490101280 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 -- 172.21.15.42:0/1983846027 >> 172.21.15.42:0/1983846027 conn(0x7f14900fc750 msgr2=0x7f14900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 -- 172.21.15.42:0/1983846027 shutdown_connections 2024-09-16T23:58:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.584+0000 7f1494d13700 1 -- 172.21.15.42:0/1983846027 wait complete. 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.585+0000 7f1494d13700 1 Processor -- start 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.585+0000 7f1494d13700 1 -- start start 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.585+0000 7f1494d13700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1490100ea0 0x7f1490073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.585+0000 7f1494d13700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490101850 0x7f14900735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.585+0000 7f1494d13700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490073b10 con 0x7f1490101850 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.585+0000 7f1494d13700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1490073c50 con 0x7f1490100ea0 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490101850 0x7f14900735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490101850 0x7f14900735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52564/0 (socket says 172.21.15.42:52564) 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148dd9b700 1 -- 172.21.15.42:0/3422664594 learned_addr learned my addr 172.21.15.42:0/3422664594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148e59c700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1490100ea0 0x7f1490073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148dd9b700 1 -- 172.21.15.42:0/3422664594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1490100ea0 msgr2=0x7f1490073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148dd9b700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1490100ea0 0x7f1490073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148dd9b700 1 -- 172.21.15.42:0/3422664594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800095f0 con 0x7f1490101850 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f148dd9b700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490101850 0x7f14900735d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f148400c8f0 tx=0x7f148400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f147f7fe700 1 -- 172.21.15.42:0/3422664594 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1484007a50 con 0x7f1490101850 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f147f7fe700 1 -- 172.21.15.42:0/3422664594 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1484007bb0 con 0x7f1490101850 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14901a2af0 con 0x7f1490101850 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f147f7fe700 1 -- 172.21.15.42:0/3422664594 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1484018630 con 0x7f1490101850 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.586+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14901a2f00 con 0x7f1490101850 2024-09-16T23:58:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.587+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1490109240 con 0x7f1490101850 2024-09-16T23:58:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.588+0000 7f147f7fe700 1 -- 172.21.15.42:0/3422664594 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1484007d20 con 0x7f1490101850 2024-09-16T23:58:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.588+0000 7f147f7fe700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f147806e680 0x7f1478070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.588+0000 7f147f7fe700 1 -- 172.21.15.42:0/3422664594 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f148408e540 con 0x7f1490101850 2024-09-16T23:58:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.588+0000 7f148e59c700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f147806e680 0x7f1478070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.589+0000 7f148e59c700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f147806e680 0x7f1478070b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480005fd0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:40.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.593+0000 7f147f7fe700 1 -- 172.21.15.42:0/3422664594 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1484057760 con 0x7f1490101850 2024-09-16T23:58:40.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:40 smithi042 ceph-mon[111056]: pgmap v212: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:40 smithi042 ceph-mon[111056]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:40 smithi042 ceph-mon[111056]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:40 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/362934578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:40 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3602106595' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:58:40.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.825+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14900611d0 con 0x7f147806e680 2024-09-16T23:58:40.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.826+0000 7f147f7fe700 1 -- 172.21.15.42:0/3422664594 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f14900611d0 con 0x7f147806e680 2024-09-16T23:58:40.823 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-16T23:58:40.824 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T23:58:40.825 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:58:40.825 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f147806e680 msgr2=0x7f1478070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f147806e680 0x7f1478070b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480005fd0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490101850 msgr2=0x7f14900735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490101850 0x7f14900735d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f148400c8f0 tx=0x7f148400ccb0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 shutdown_connections 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f147806e680 0x7f1478070b40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1490100ea0 0x7f1490073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 --2- 172.21.15.42:0/3422664594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1490101850 0x7f14900735d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 >> 172.21.15.42:0/3422664594 conn(0x7f14900fc750 msgr2=0x7f14900fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:40.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 shutdown_connections 2024-09-16T23:58:40.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.828+0000 7f1494d13700 1 -- 172.21.15.42:0/3422664594 wait complete. 2024-09-16T23:58:40.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:40 smithi043 ceph-mon[81510]: pgmap v212: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:40.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:40 smithi043 ceph-mon[81510]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:40.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:40 smithi043 ceph-mon[81510]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:40.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:40 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/362934578' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:58:40.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:40 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3602106595' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:58:40.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 -- 172.21.15.42:0/3421307302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 msgr2=0x7f1a500689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 --2- 172.21.15.42:0/3421307302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500689e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009910 tx=0x7f1a40009c20 comp rx=0 tx=0).stop 2024-09-16T23:58:40.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 -- 172.21.15.42:0/3421307302 shutdown_connections 2024-09-16T23:58:40.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 --2- 172.21.15.42:0/3421307302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500689e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 --2- 172.21.15.42:0/3421307302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a50106cd0 0x7f1a501070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 -- 172.21.15.42:0/3421307302 >> 172.21.15.42:0/3421307302 conn(0x7f1a50075670 msgr2=0x7f1a50075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:40.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 -- 172.21.15.42:0/3421307302 shutdown_connections 2024-09-16T23:58:40.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.936+0000 7f1a57820700 1 -- 172.21.15.42:0/3421307302 wait complete. 2024-09-16T23:58:40.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a57820700 1 Processor -- start 2024-09-16T23:58:40.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a57820700 1 -- start start 2024-09-16T23:58:40.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a57820700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a57820700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a50106cd0 0x7f1a50100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a57820700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a50100770 con 0x7f1a50068580 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a57820700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a501008b0 con 0x7f1a50106cd0 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a555bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a555bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52588/0 (socket says 172.21.15.42:52588) 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a555bc700 1 -- 172.21.15.42:0/4223440185 learned_addr learned my addr 172.21.15.42:0/4223440185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.937+0000 7f1a54dbb700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a50106cd0 0x7f1a50100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a555bc700 1 -- 172.21.15.42:0/4223440185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a50106cd0 msgr2=0x7f1a50100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:40.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a555bc700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a50106cd0 0x7f1a50100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a555bc700 1 -- 172.21.15.42:0/4223440185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400095f0 con 0x7f1a50068580 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a555bc700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500ffcf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00e920 tx=0x7f1a4c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a467fc700 1 -- 172.21.15.42:0/4223440185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a4c00c9d0 con 0x7f1a50068580 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a467fc700 1 -- 172.21.15.42:0/4223440185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a4c010430 con 0x7f1a50068580 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a501a2a30 con 0x7f1a50068580 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a467fc700 1 -- 172.21.15.42:0/4223440185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a4c018630 con 0x7f1a50068580 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.938+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a501a2e40 con 0x7f1a50068580 2024-09-16T23:58:40.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.939+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a501091d0 con 0x7f1a50068580 2024-09-16T23:58:40.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.940+0000 7f1a467fc700 1 -- 172.21.15.42:0/4223440185 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1a4c0105a0 con 0x7f1a50068580 2024-09-16T23:58:40.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.940+0000 7f1a467fc700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1a3c06e680 0x7f1a3c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:58:40.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.940+0000 7f1a467fc700 1 -- 172.21.15.42:0/4223440185 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a4c014070 con 0x7f1a50068580 2024-09-16T23:58:40.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.940+0000 7f1a54dbb700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1a3c06e680 0x7f1a3c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:58:40.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.941+0000 7f1a54dbb700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1a3c06e680 0x7f1a3c070b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a501014e0 tx=0x7f1a40005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:58:40.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:40.944+0000 7f1a467fc700 1 -- 172.21.15.42:0/4223440185 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1a4c0541b0 con 0x7f1a50068580 2024-09-16T23:58:41.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.236+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1a50100a70 con 0x7f1a50068580 2024-09-16T23:58:41.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.237+0000 7f1a467fc700 1 -- 172.21.15.42:0/4223440185 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f1a50100a70 con 0x7f1a50068580 2024-09-16T23:58:41.235 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T23:58:41.235 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T23:58:41.235 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-16T23:58:41.235 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:58:41.235 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:58:41.235 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T23:58:41.235 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1a3c06e680 msgr2=0x7f1a3c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1a3c06e680 0x7f1a3c070b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1a501014e0 tx=0x7f1a40005a90 comp rx=0 tx=0).stop 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 msgr2=0x7f1a500ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500ffcf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c00e920 tx=0x7f1a4c00ec30 comp rx=0 tx=0).stop 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 shutdown_connections 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1a3c06e680 0x7f1a3c070b40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a50068580 0x7f1a500ffcf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:41.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 --2- 172.21.15.42:0/4223440185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a50106cd0 0x7f1a50100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:58:41.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 >> 172.21.15.42:0/4223440185 conn(0x7f1a50075670 msgr2=0x7f1a500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:58:41.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 shutdown_connections 2024-09-16T23:58:41.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:58:41.239+0000 7f1a57820700 1 -- 172.21.15.42:0/4223440185 wait complete. 2024-09-16T23:58:41.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:41 smithi042 ceph-mon[111056]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:41.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:41 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4223440185' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:58:41.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:41 smithi043 ceph-mon[81510]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:41.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:41 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4223440185' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:58:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:42 smithi042 ceph-mon[111056]: pgmap v213: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:42 smithi042 ceph-mon[111056]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:42.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:42 smithi043 ceph-mon[81510]: pgmap v213: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:42.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:42 smithi043 ceph-mon[81510]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:58:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:44 smithi042 ceph-mon[111056]: pgmap v214: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:44.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:44 smithi043 ceph-mon[81510]: pgmap v214: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:46 smithi042 ceph-mon[111056]: pgmap v215: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:46.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:46 smithi043 ceph-mon[81510]: pgmap v215: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:48 smithi042 ceph-mon[111056]: pgmap v216: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:48 smithi043 ceph-mon[81510]: pgmap v216: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:50 smithi042 ceph-mon[111056]: pgmap v217: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:50 smithi043 ceph-mon[81510]: pgmap v217: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:58:52.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:52 smithi042 ceph-mon[111056]: pgmap v218: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:52.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:52 smithi043 ceph-mon[81510]: pgmap v218: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:54.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:54 smithi042 ceph-mon[111056]: pgmap v219: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:54 smithi043 ceph-mon[81510]: pgmap v219: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:56 smithi042 ceph-mon[111056]: pgmap v220: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:56.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:56 smithi043 ceph-mon[81510]: pgmap v220: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:58:58 smithi042 ceph-mon[111056]: pgmap v221: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:58:58.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:58:58 smithi043 ceph-mon[81510]: pgmap v221: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:00 smithi043 ceph-mon[81510]: pgmap v222: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:00 smithi042 ceph-mon[111056]: pgmap v222: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:02.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:02 smithi043 ceph-mon[81510]: pgmap v223: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:03.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:02 smithi042 ceph-mon[111056]: pgmap v223: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:04 smithi043 ceph-mon[81510]: pgmap v224: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:04 smithi042 ceph-mon[111056]: pgmap v224: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:06 smithi043 ceph-mon[81510]: pgmap v225: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:07.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:06 smithi042 ceph-mon[111056]: pgmap v225: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:07.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:08 smithi043 ceph-mon[81510]: pgmap v226: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:08 smithi042 ceph-mon[111056]: pgmap v226: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:10.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:10 smithi043 ceph-mon[81510]: pgmap v227: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:10 smithi042 ceph-mon[111056]: pgmap v227: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 -- 172.21.15.42:0/1935085657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498102720 msgr2=0x7fa49810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 --2- 172.21.15.42:0/1935085657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498102720 0x7fa49810ac10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa488009910 tx=0x7fa488009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 -- 172.21.15.42:0/1935085657 shutdown_connections 2024-09-16T23:59:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 --2- 172.21.15.42:0/1935085657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498102720 0x7fa49810ac10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 --2- 172.21.15.42:0/1935085657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa498101e00 0x7fa4981021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 -- 172.21.15.42:0/1935085657 >> 172.21.15.42:0/1935085657 conn(0x7fa4980fb650 msgr2=0x7fa4980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 -- 172.21.15.42:0/1935085657 shutdown_connections 2024-09-16T23:59:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 -- 172.21.15.42:0/1935085657 wait complete. 2024-09-16T23:59:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.363+0000 7fa49fdda700 1 Processor -- start 2024-09-16T23:59:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49fdda700 1 -- start start 2024-09-16T23:59:11.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49fdda700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498101e00 0x7fa49819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:11.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49fdda700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa498102720 0x7fa49819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:11.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49fdda700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49819d260 con 0x7fa498101e00 2024-09-16T23:59:11.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49fdda700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa498196710 con 0x7fa498102720 2024-09-16T23:59:11.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49db76700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498101e00 0x7fa49819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49db76700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498101e00 0x7fa49819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44538/0 (socket says 172.21.15.42:44538) 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49db76700 1 -- 172.21.15.42:0/2144032057 learned_addr learned my addr 172.21.15.42:0/2144032057 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49d375700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa498102720 0x7fa49819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49db76700 1 -- 172.21.15.42:0/2144032057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa498102720 msgr2=0x7fa49819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49db76700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa498102720 0x7fa49819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.364+0000 7fa49db76700 1 -- 172.21.15.42:0/2144032057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880095f0 con 0x7fa498101e00 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.365+0000 7fa49db76700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498101e00 0x7fa49819c690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e920 tx=0x7fa49400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.365+0000 7fa48effd700 1 -- 172.21.15.42:0/2144032057 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa49400c9d0 con 0x7fa498101e00 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.365+0000 7fa48effd700 1 -- 172.21.15.42:0/2144032057 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa494010430 con 0x7fa498101e00 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.365+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4981969f0 con 0x7fa498101e00 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.365+0000 7fa48effd700 1 -- 172.21.15.42:0/2144032057 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa494018630 con 0x7fa498101e00 2024-09-16T23:59:11.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.365+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa498196f10 con 0x7fa498101e00 2024-09-16T23:59:11.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.366+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa498108310 con 0x7fa498101e00 2024-09-16T23:59:11.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.366+0000 7fa48effd700 1 -- 172.21.15.42:0/2144032057 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa4940105a0 con 0x7fa498101e00 2024-09-16T23:59:11.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.367+0000 7fa48effd700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa48406e680 0x7fa484070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:11.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.367+0000 7fa48effd700 1 -- 172.21.15.42:0/2144032057 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa494014070 con 0x7fa498101e00 2024-09-16T23:59:11.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.367+0000 7fa49d375700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa48406e680 0x7fa484070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:11.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.368+0000 7fa49d375700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa48406e680 0x7fa484070b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa488005b40 tx=0x7fa488005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:11.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.372+0000 7fa48effd700 1 -- 172.21.15.42:0/2144032057 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa4940541b0 con 0x7fa498101e00 2024-09-16T23:59:11.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.605+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa498197130 con 0x7fa48406e680 2024-09-16T23:59:11.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.606+0000 7fa48effd700 1 -- 172.21.15.42:0/2144032057 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa498197130 con 0x7fa48406e680 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa48406e680 msgr2=0x7fa484070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa48406e680 0x7fa484070b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa488005b40 tx=0x7fa488005a90 comp rx=0 tx=0).stop 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498101e00 msgr2=0x7fa49819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498101e00 0x7fa49819c690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa49400e920 tx=0x7fa49400ec30 comp rx=0 tx=0).stop 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 shutdown_connections 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa48406e680 0x7fa484070b40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa498101e00 0x7fa49819c690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 --2- 172.21.15.42:0/2144032057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa498102720 0x7fa49819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 >> 172.21.15.42:0/2144032057 conn(0x7fa4980fb650 msgr2=0x7fa498105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 shutdown_connections 2024-09-16T23:59:11.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.609+0000 7fa49fdda700 1 -- 172.21.15.42:0/2144032057 wait complete. 2024-09-16T23:59:11.617 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:59:11.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 -- 172.21.15.42:0/2651310578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31640ffda0 msgr2=0x7f3164100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 --2- 172.21.15.42:0/2651310578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31640ffda0 0x7f3164100180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3154009910 tx=0x7f3154009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 -- 172.21.15.42:0/2651310578 shutdown_connections 2024-09-16T23:59:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 --2- 172.21.15.42:0/2651310578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3164100750 0x7f31641047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 --2- 172.21.15.42:0/2651310578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f31640ffda0 0x7f3164100180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 -- 172.21.15.42:0/2651310578 >> 172.21.15.42:0/2651310578 conn(0x7f31640fb650 msgr2=0x7f31640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 -- 172.21.15.42:0/2651310578 shutdown_connections 2024-09-16T23:59:11.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.718+0000 7f31693b8700 1 -- 172.21.15.42:0/2651310578 wait complete. 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.719+0000 7f31693b8700 1 Processor -- start 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.719+0000 7f31693b8700 1 -- start start 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.719+0000 7f31693b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31640ffda0 0x7f31641976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.719+0000 7f31693b8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3164100750 0x7f3164197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.719+0000 7f31693b8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3164198250 con 0x7f3164100750 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.719+0000 7f31693b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31641983c0 con 0x7f31640ffda0 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f315bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3164100750 0x7f3164197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:11.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f315bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3164100750 0x7f3164197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44562/0 (socket says 172.21.15.42:44562) 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f315bfff700 1 -- 172.21.15.42:0/942969996 learned_addr learned my addr 172.21.15.42:0/942969996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f3162ffd700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31640ffda0 0x7f31641976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f315bfff700 1 -- 172.21.15.42:0/942969996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31640ffda0 msgr2=0x7f31641976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f315bfff700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31640ffda0 0x7f31641976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f315bfff700 1 -- 172.21.15.42:0/942969996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31540095f0 con 0x7f3164100750 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f315bfff700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3164100750 0x7f3164197c30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f314c00e970 tx=0x7f314c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f3160ff9700 1 -- 172.21.15.42:0/942969996 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f314c00c9d0 con 0x7f3164100750 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f3160ff9700 1 -- 172.21.15.42:0/942969996 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f314c004830 con 0x7f3164100750 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f3160ff9700 1 -- 172.21.15.42:0/942969996 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f314c005660 con 0x7f3164100750 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31641a0760 con 0x7f3164100750 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.720+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31641a0c00 con 0x7f3164100750 2024-09-16T23:59:11.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.721+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164108140 con 0x7f3164100750 2024-09-16T23:59:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.722+0000 7f3160ff9700 1 -- 172.21.15.42:0/942969996 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f314c00cb30 con 0x7f3164100750 2024-09-16T23:59:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.723+0000 7f3160ff9700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f315006e680 0x7f3150070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.723+0000 7f3160ff9700 1 -- 172.21.15.42:0/942969996 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f314c014070 con 0x7f3164100750 2024-09-16T23:59:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.723+0000 7f3162ffd700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f315006e680 0x7f3150070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.723+0000 7f3162ffd700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f315006e680 0x7f3150070b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31540059d0 tx=0x7f3154005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:11.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.727+0000 7f3160ff9700 1 -- 172.21.15.42:0/942969996 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f314c057b00 con 0x7f3164100750 2024-09-16T23:59:11.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.954+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f316402cce0 con 0x7f315006e680 2024-09-16T23:59:11.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.955+0000 7f3160ff9700 1 -- 172.21.15.42:0/942969996 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f316402cce0 con 0x7f315006e680 2024-09-16T23:59:11.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f315006e680 msgr2=0x7f3150070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f315006e680 0x7f3150070b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f31540059d0 tx=0x7f3154005940 comp rx=0 tx=0).stop 2024-09-16T23:59:11.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3164100750 msgr2=0x7f3164197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:11.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3164100750 0x7f3164197c30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f314c00e970 tx=0x7f314c00ec80 comp rx=0 tx=0).stop 2024-09-16T23:59:11.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 shutdown_connections 2024-09-16T23:59:11.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f315006e680 0x7f3150070b40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31640ffda0 0x7f31641976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 --2- 172.21.15.42:0/942969996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3164100750 0x7f3164197c30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:11.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.957+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 >> 172.21.15.42:0/942969996 conn(0x7f31640fb650 msgr2=0x7f31640fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:11.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.958+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 shutdown_connections 2024-09-16T23:59:11.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:11.958+0000 7f31693b8700 1 -- 172.21.15.42:0/942969996 wait complete. 2024-09-16T23:59:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/1135090293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8102720 msgr2=0x7fd8b810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/1135090293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8102720 0x7fd8b810ac10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009910 tx=0x7fd8ac009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:12.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/1135090293 shutdown_connections 2024-09-16T23:59:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/1135090293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8102720 0x7fd8b810ac10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/1135090293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b8101e00 0x7fd8b81021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/1135090293 >> 172.21.15.42:0/1135090293 conn(0x7fd8b80fb650 msgr2=0x7fd8b80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/1135090293 shutdown_connections 2024-09-16T23:59:12.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/1135090293 wait complete. 2024-09-16T23:59:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.066+0000 7fd8bcd6f700 1 Processor -- start 2024-09-16T23:59:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8bcd6f700 1 -- start start 2024-09-16T23:59:12.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8bcd6f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8101e00 0x7fd8b80ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8bcd6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b8102720 0x7fd8b80fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8bcd6f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81004e0 con 0x7fd8b8101e00 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8bcd6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8100620 con 0x7fd8b8102720 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8101e00 0x7fd8b80ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8101e00 0x7fd8b80ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44580/0 (socket says 172.21.15.42:44580) 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b659c700 1 -- 172.21.15.42:0/62465442 learned_addr learned my addr 172.21.15.42:0/62465442 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b5d9b700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b8102720 0x7fd8b80fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b659c700 1 -- 172.21.15.42:0/62465442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b8102720 msgr2=0x7fd8b80fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b659c700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b8102720 0x7fd8b80fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b659c700 1 -- 172.21.15.42:0/62465442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0095f0 con 0x7fd8b8101e00 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.067+0000 7fd8b659c700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8101e00 0x7fd8b80ffa60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800e920 tx=0x7fd8a800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.068+0000 7fd8a77fe700 1 -- 172.21.15.42:0/62465442 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a800c9d0 con 0x7fd8b8101e00 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.068+0000 7fd8a77fe700 1 -- 172.21.15.42:0/62465442 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8a8010430 con 0x7fd8b8101e00 2024-09-16T23:59:12.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.068+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b81a2a90 con 0x7fd8b8101e00 2024-09-16T23:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.068+0000 7fd8a77fe700 1 -- 172.21.15.42:0/62465442 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a8018630 con 0x7fd8b8101e00 2024-09-16T23:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.068+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b81a2e40 con 0x7fd8b8101e00 2024-09-16T23:59:12.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.069+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b8108310 con 0x7fd8b8101e00 2024-09-16T23:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.070+0000 7fd8a77fe700 1 -- 172.21.15.42:0/62465442 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd8a80105a0 con 0x7fd8b8101e00 2024-09-16T23:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.070+0000 7fd8a77fe700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd8a006e680 0x7fd8a0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.070+0000 7fd8a77fe700 1 -- 172.21.15.42:0/62465442 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd8a8014070 con 0x7fd8b8101e00 2024-09-16T23:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.070+0000 7fd8b5d9b700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd8a006e680 0x7fd8a0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.071+0000 7fd8b5d9b700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd8a006e680 0x7fd8a0070b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac005b40 tx=0x7fd8ac005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:12.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.074+0000 7fd8a77fe700 1 -- 172.21.15.42:0/62465442 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd8a80541b0 con 0x7fd8b8101e00 2024-09-16T23:59:12.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.296+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b80611d0 con 0x7fd8a006e680 2024-09-16T23:59:12.303 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.305+0000 7fd8a77fe700 1 -- 172.21.15.42:0/62465442 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd8b80611d0 con 0x7fd8a006e680 2024-09-16T23:59:12.303 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:59:12.303 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (7m) 89s ago 18m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:59:12.303 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (18m) 89s ago 18m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:59:12.303 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (17m) 61s ago 17m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:59:12.303 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (5m) 89s ago 18m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (5m) 61s ago 17m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (7m) 89s ago 18m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (94s) 89s ago 12m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (111s) 89s ago 13m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (78s) 61s ago 12m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (65s) 61s ago 12m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (9m) 89s ago 20m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (8m) 61s ago 17m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (6m) 89s ago 20m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (6m) 61s ago 17m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (8m) 89s ago 18m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (8m) 61s ago 17m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (5m) 89s ago 16m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:59:12.304 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (4m) 89s ago 16m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:59:12.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (3m) 89s ago 15m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:59:12.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (3m) 61s ago 15m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-16T23:59:12.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (2m) 61s ago 14m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-16T23:59:12.305 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (2m) 61s ago 14m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-16T23:59:12.305 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (8m) 89s ago 18m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd8a006e680 msgr2=0x7fd8a0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd8a006e680 0x7fd8a0070b40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac005b40 tx=0x7fd8ac005a90 comp rx=0 tx=0).stop 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8101e00 msgr2=0x7fd8b80ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8101e00 0x7fd8b80ffa60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800e920 tx=0x7fd8a800ec30 comp rx=0 tx=0).stop 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 shutdown_connections 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd8a006e680 0x7fd8a0070b40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8b8101e00 0x7fd8b80ffa60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 --2- 172.21.15.42:0/62465442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8b8102720 0x7fd8b80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 >> 172.21.15.42:0/62465442 conn(0x7fd8b80fb650 msgr2=0x7fd8b8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:12.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 shutdown_connections 2024-09-16T23:59:12.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.308+0000 7fd8bcd6f700 1 -- 172.21.15.42:0/62465442 wait complete. 2024-09-16T23:59:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 -- 172.21.15.42:0/2912868382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 msgr2=0x7f1830101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 --2- 172.21.15.42:0/2912868382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f1830101280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f182c009910 tx=0x7f182c009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 -- 172.21.15.42:0/2912868382 shutdown_connections 2024-09-16T23:59:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 --2- 172.21.15.42:0/2912868382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1830101850 0x7f18301058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 --2- 172.21.15.42:0/2912868382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f1830101280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 -- 172.21.15.42:0/2912868382 >> 172.21.15.42:0/2912868382 conn(0x7f18300fc750 msgr2=0x7f18300feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:12.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 -- 172.21.15.42:0/2912868382 shutdown_connections 2024-09-16T23:59:12.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.416+0000 7f183793c700 1 -- 172.21.15.42:0/2912868382 wait complete. 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.417+0000 7f183793c700 1 Processor -- start 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.417+0000 7f183793c700 1 -- start start 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.417+0000 7f183793c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f183019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.417+0000 7f183793c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1830101850 0x7f183019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.417+0000 7f183793c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f183019d2e0 con 0x7f1830100ea0 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.417+0000 7f183793c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830196790 con 0x7f1830101850 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.417+0000 7f18356d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f183019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18356d8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f183019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44598/0 (socket says 172.21.15.42:44598) 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18356d8700 1 -- 172.21.15.42:0/7726375 learned_addr learned my addr 172.21.15.42:0/7726375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f1834ed7700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1830101850 0x7f183019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18356d8700 1 -- 172.21.15.42:0/7726375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1830101850 msgr2=0x7f183019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18356d8700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1830101850 0x7f183019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18356d8700 1 -- 172.21.15.42:0/7726375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0095f0 con 0x7f1830100ea0 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18356d8700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f183019c710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f182c006010 tx=0x7f182c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18267fc700 1 -- 172.21.15.42:0/7726375 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f182c01d070 con 0x7f1830100ea0 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18267fc700 1 -- 172.21.15.42:0/7726375 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f182c004fb0 con 0x7f1830100ea0 2024-09-16T23:59:12.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830196a10 con 0x7f1830100ea0 2024-09-16T23:59:12.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f18267fc700 1 -- 172.21.15.42:0/7726375 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f182c022ea0 con 0x7f1830100ea0 2024-09-16T23:59:12.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.418+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1830196f00 con 0x7f1830100ea0 2024-09-16T23:59:12.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.419+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830109240 con 0x7f1830100ea0 2024-09-16T23:59:12.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.420+0000 7f18267fc700 1 -- 172.21.15.42:0/7726375 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f182c022470 con 0x7f1830100ea0 2024-09-16T23:59:12.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.421+0000 7f18267fc700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f181c06e680 0x7f181c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.421+0000 7f18267fc700 1 -- 172.21.15.42:0/7726375 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f182c08fbf0 con 0x7f1830100ea0 2024-09-16T23:59:12.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.421+0000 7f1834ed7700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f181c06e680 0x7f181c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.421+0000 7f1834ed7700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f181c06e680 0x7f181c070b40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1820005fd0 tx=0x7f182000a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:12.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.425+0000 7f18267fc700 1 -- 172.21.15.42:0/7726375 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f182c05c970 con 0x7f1830100ea0 2024-09-16T23:59:12.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.722+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1830197d40 con 0x7f1830100ea0 2024-09-16T23:59:12.720 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:12 smithi042 ceph-mon[111056]: pgmap v228: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:12.720 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:12 smithi042 ceph-mon[111056]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:12.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.723+0000 7f18267fc700 1 -- 172.21.15.42:0/7726375 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f182c027290 con 0x7f1830100ea0 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:59:12.724 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:59:12.725 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f181c06e680 msgr2=0x7f181c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f181c06e680 0x7f181c070b40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1820005fd0 tx=0x7f182000a560 comp rx=0 tx=0).stop 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 msgr2=0x7f183019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f183019c710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f182c006010 tx=0x7f182c00be20 comp rx=0 tx=0).stop 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 shutdown_connections 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f181c06e680 0x7f181c070b40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1830100ea0 0x7f183019c710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 --2- 172.21.15.42:0/7726375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1830101850 0x7f183019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.725+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 >> 172.21.15.42:0/7726375 conn(0x7f18300fc750 msgr2=0x7f18300fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:12.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.726+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 shutdown_connections 2024-09-16T23:59:12.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.726+0000 7f183793c700 1 -- 172.21.15.42:0/7726375 wait complete. 2024-09-16T23:59:12.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.834+0000 7fd279119700 1 -- 172.21.15.42:0/4197433918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 msgr2=0x7fd274100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.834+0000 7fd279119700 1 --2- 172.21.15.42:0/4197433918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274100180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd264009910 tx=0x7fd264009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:12.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.835+0000 7fd279119700 1 -- 172.21.15.42:0/4197433918 shutdown_connections 2024-09-16T23:59:12.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.835+0000 7fd279119700 1 --2- 172.21.15.42:0/4197433918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd274100750 0x7fd2741047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.835+0000 7fd279119700 1 --2- 172.21.15.42:0/4197433918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274100180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.835+0000 7fd279119700 1 -- 172.21.15.42:0/4197433918 >> 172.21.15.42:0/4197433918 conn(0x7fd2740fb650 msgr2=0x7fd2740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:12.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.835+0000 7fd279119700 1 -- 172.21.15.42:0/4197433918 shutdown_connections 2024-09-16T23:59:12.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.835+0000 7fd279119700 1 -- 172.21.15.42:0/4197433918 wait complete. 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.835+0000 7fd279119700 1 Processor -- start 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd279119700 1 -- start start 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd279119700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd279119700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd274100750 0x7fd274101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd279119700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd274102350 con 0x7fd2740ffda0 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd279119700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2741024c0 con 0x7fd274100750 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd272d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd272d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33644/0 (socket says 172.21.15.42:33644) 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd272d9d700 1 -- 172.21.15.42:0/1532258415 learned_addr learned my addr 172.21.15.42:0/1532258415 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd27259c700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd274100750 0x7fd274101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd272d9d700 1 -- 172.21.15.42:0/1532258415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd274100750 msgr2=0x7fd274101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:12.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd272d9d700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd274100750 0x7fd274101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd272d9d700 1 -- 172.21.15.42:0/1532258415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640095f0 con 0x7fd2740ffda0 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd27259c700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd274100750 0x7fd274101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.836+0000 7fd272d9d700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274101840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd264006010 tx=0x7fd26400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.837+0000 7fd25bfff700 1 -- 172.21.15.42:0/1532258415 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26401d070 con 0x7fd2740ffda0 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.837+0000 7fd25bfff700 1 -- 172.21.15.42:0/1532258415 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd26400f460 con 0x7fd2740ffda0 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.837+0000 7fd25bfff700 1 -- 172.21.15.42:0/1532258415 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd264017630 con 0x7fd2740ffda0 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.837+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2741a2aa0 con 0x7fd2740ffda0 2024-09-16T23:59:12.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.837+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2741a2e50 con 0x7fd2740ffda0 2024-09-16T23:59:12.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.838+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27404f2e0 con 0x7fd2740ffda0 2024-09-16T23:59:12.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.838+0000 7fd25bfff700 1 -- 172.21.15.42:0/1532258415 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd26400f5d0 con 0x7fd2740ffda0 2024-09-16T23:59:12.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.839+0000 7fd25bfff700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd25c06e680 0x7fd25c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:12.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.839+0000 7fd25bfff700 1 -- 172.21.15.42:0/1532258415 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd26408fab0 con 0x7fd2740ffda0 2024-09-16T23:59:12.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.839+0000 7fd27259c700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd25c06e680 0x7fd25c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:12.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.839+0000 7fd27259c700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd25c06e680 0x7fd25c070b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd268005950 tx=0x7fd2680058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:12.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:12.843+0000 7fd25bfff700 1 -- 172.21.15.42:0/1532258415 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd264058cd0 con 0x7fd2740ffda0 2024-09-16T23:59:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:12 smithi043 ceph-mon[81510]: pgmap v228: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:12 smithi043 ceph-mon[81510]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.102+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd2741a3040 con 0x7fd2740ffda0 2024-09-16T23:59:13.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.103+0000 7fd25bfff700 1 -- 172.21.15.42:0/1532258415 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fd26405c390 con 0x7fd2740ffda0 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-16T23:59:13.101 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:59:13.102 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:59:13.103 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:13.104 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd25c06e680 msgr2=0x7fd25c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd25c06e680 0x7fd25c070b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd268005950 tx=0x7fd2680058e0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 msgr2=0x7fd274101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274101840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd264006010 tx=0x7fd26400be20 comp rx=0 tx=0).stop 2024-09-16T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 shutdown_connections 2024-09-16T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd25c06e680 0x7fd25c070b40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2740ffda0 0x7fd274101840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 --2- 172.21.15.42:0/1532258415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd274100750 0x7fd274101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 >> 172.21.15.42:0/1532258415 conn(0x7fd2740fb650 msgr2=0x7fd274103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 shutdown_connections 2024-09-16T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.106+0000 7fd279119700 1 -- 172.21.15.42:0/1532258415 wait complete. 2024-09-16T23:59:13.106 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-16T23:59:13.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 -- 172.21.15.42:0/3940251328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 msgr2=0x7f21c4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 --2- 172.21.15.42:0/3940251328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c4101270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009910 tx=0x7f21c0009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:13.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 -- 172.21.15.42:0/3940251328 shutdown_connections 2024-09-16T23:59:13.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 --2- 172.21.15.42:0/3940251328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c4101840 0x7f21c4105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 --2- 172.21.15.42:0/3940251328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c4101270 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 -- 172.21.15.42:0/3940251328 >> 172.21.15.42:0/3940251328 conn(0x7f21c40fc740 msgr2=0x7f21c40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:13.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 -- 172.21.15.42:0/3940251328 shutdown_connections 2024-09-16T23:59:13.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 -- 172.21.15.42:0/3940251328 wait complete. 2024-09-16T23:59:13.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.215+0000 7f21cb710700 1 Processor -- start 2024-09-16T23:59:13.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21cb710700 1 -- start start 2024-09-16T23:59:13.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21cb710700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:13.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21cb710700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c4101840 0x7f21c419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21cb710700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c419d250 con 0x7f21c4100e90 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21cb710700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4196700 con 0x7f21c4101840 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c94ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c94ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c419c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33670/0 (socket says 172.21.15.42:33670) 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c94ac700 1 -- 172.21.15.42:0/382262778 learned_addr learned my addr 172.21.15.42:0/382262778 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c8cab700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c4101840 0x7f21c419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c94ac700 1 -- 172.21.15.42:0/382262778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c4101840 msgr2=0x7f21c419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c94ac700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c4101840 0x7f21c419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c94ac700 1 -- 172.21.15.42:0/382262778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c00095f0 con 0x7f21c4100e90 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.216+0000 7f21c94ac700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c419c680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006010 tx=0x7f21c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:13.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.217+0000 7f21b67fc700 1 -- 172.21.15.42:0/382262778 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c001d070 con 0x7f21c4100e90 2024-09-16T23:59:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.217+0000 7f21b67fc700 1 -- 172.21.15.42:0/382262778 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21c0005dc0 con 0x7f21c4100e90 2024-09-16T23:59:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.217+0000 7f21b67fc700 1 -- 172.21.15.42:0/382262778 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c0017400 con 0x7f21c4100e90 2024-09-16T23:59:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.217+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c4196980 con 0x7f21c4100e90 2024-09-16T23:59:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.217+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c4196e70 con 0x7f21c4100e90 2024-09-16T23:59:13.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.218+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c4109230 con 0x7f21c4100e90 2024-09-16T23:59:13.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.218+0000 7f21b67fc700 1 -- 172.21.15.42:0/382262778 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f21c0003a60 con 0x7f21c4100e90 2024-09-16T23:59:13.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.219+0000 7f21b67fc700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21b006e680 0x7f21b0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:13.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.219+0000 7f21b67fc700 1 -- 172.21.15.42:0/382262778 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f21c008fae0 con 0x7f21c4100e90 2024-09-16T23:59:13.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.219+0000 7f21c8cab700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21b006e680 0x7f21b0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:13.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.219+0000 7f21c8cab700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21b006e680 0x7f21b0070b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21c4197b70 tx=0x7f21b800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:13.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.223+0000 7f21b67fc700 1 -- 172.21.15.42:0/382262778 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f21c005c7b0 con 0x7f21c4100e90 2024-09-16T23:59:13.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.453+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21c4197090 con 0x7f21b006e680 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.454+0000 7f21b67fc700 1 -- 172.21.15.42:0/382262778 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f21c4197090 con 0x7f21b006e680 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:59:13.452 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:59:13.453 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-16T23:59:13.453 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:59:13.453 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:59:13.453 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-16T23:59:13.453 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T23:59:13.453 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:59:13.453 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21b006e680 msgr2=0x7f21b0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21b006e680 0x7f21b0070b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21c4197b70 tx=0x7f21b800b410 comp rx=0 tx=0).stop 2024-09-16T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 msgr2=0x7f21c419c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c419c680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006010 tx=0x7f21c000bd70 comp rx=0 tx=0).stop 2024-09-16T23:59:13.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 shutdown_connections 2024-09-16T23:59:13.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21b006e680 0x7f21b0070b40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21c4100e90 0x7f21c419c680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.456+0000 7f21cb710700 1 --2- 172.21.15.42:0/382262778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21c4101840 0x7f21c419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.457+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 >> 172.21.15.42:0/382262778 conn(0x7f21c40fc740 msgr2=0x7f21c40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:13.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.457+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 shutdown_connections 2024-09-16T23:59:13.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.457+0000 7f21cb710700 1 -- 172.21.15.42:0/382262778 wait complete. 2024-09-16T23:59:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 -- 172.21.15.42:0/1695674978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918101850 msgr2=0x7fa9181058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 --2- 172.21.15.42:0/1695674978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918101850 0x7fa9181058a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009910 tx=0x7fa90c009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 -- 172.21.15.42:0/1695674978 shutdown_connections 2024-09-16T23:59:13.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 --2- 172.21.15.42:0/1695674978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918101850 0x7fa9181058a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 --2- 172.21.15.42:0/1695674978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918100ea0 0x7fa918101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 -- 172.21.15.42:0/1695674978 >> 172.21.15.42:0/1695674978 conn(0x7fa9180fc750 msgr2=0x7fa9180feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 -- 172.21.15.42:0/1695674978 shutdown_connections 2024-09-16T23:59:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.568+0000 7fa91df60700 1 -- 172.21.15.42:0/1695674978 wait complete. 2024-09-16T23:59:13.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa91df60700 1 Processor -- start 2024-09-16T23:59:13.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa91df60700 1 -- start start 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa91df60700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918100ea0 0x7fa91819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa91df60700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101850 0x7fa91819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa91df60700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91819d360 con 0x7fa918100ea0 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa91df60700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918196810 con 0x7fa918101850 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa9177fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918100ea0 0x7fa91819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa9177fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918100ea0 0x7fa91819c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33688/0 (socket says 172.21.15.42:33688) 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa9177fe700 1 -- 172.21.15.42:0/3770264131 learned_addr learned my addr 172.21.15.42:0/3770264131 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.569+0000 7fa916ffd700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101850 0x7fa91819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa9177fe700 1 -- 172.21.15.42:0/3770264131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101850 msgr2=0x7fa91819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa9177fe700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101850 0x7fa91819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa9177fe700 1 -- 172.21.15.42:0/3770264131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c0095f0 con 0x7fa918100ea0 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa916ffd700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101850 0x7fa91819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa9177fe700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918100ea0 0x7fa91819c790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b4b0 tx=0x7fa90800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa914ff9700 1 -- 172.21.15.42:0/3770264131 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa908011800 con 0x7fa918100ea0 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa914ff9700 1 -- 172.21.15.42:0/3770264131 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa908011e20 con 0x7fa918100ea0 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa918196af0 con 0x7fa918100ea0 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa918197040 con 0x7fa918100ea0 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.570+0000 7fa914ff9700 1 -- 172.21.15.42:0/3770264131 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa90800f2e0 con 0x7fa918100ea0 2024-09-16T23:59:13.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.571+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9181091d0 con 0x7fa918100ea0 2024-09-16T23:59:13.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.572+0000 7fa914ff9700 1 -- 172.21.15.42:0/3770264131 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa908010380 con 0x7fa918100ea0 2024-09-16T23:59:13.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.572+0000 7fa914ff9700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa90006e680 0x7fa900070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:13.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.572+0000 7fa914ff9700 1 -- 172.21.15.42:0/3770264131 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa90808e230 con 0x7fa918100ea0 2024-09-16T23:59:13.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.572+0000 7fa916ffd700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa90006e680 0x7fa900070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:13.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.573+0000 7fa916ffd700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa90006e680 0x7fa900070b40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa9180fd160 tx=0x7fa90c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:13.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.576+0000 7fa914ff9700 1 -- 172.21.15.42:0/3770264131 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa90805af00 con 0x7fa918100ea0 2024-09-16T23:59:13.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:13 smithi042 ceph-mon[111056]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:13.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:13 smithi042 ceph-mon[111056]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:13.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:13 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/7726375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:59:13.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:13 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1532258415' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:59:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.876+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa918197d20 con 0x7fa918100ea0 2024-09-16T23:59:13.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.877+0000 7fa914ff9700 1 -- 172.21.15.42:0/3770264131 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa90805aa90 con 0x7fa918100ea0 2024-09-16T23:59:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T23:59:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T23:59:13.875 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-16T23:59:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:59:13.875 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:59:13.875 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T23:59:13.875 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa90006e680 msgr2=0x7fa900070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa90006e680 0x7fa900070b40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa9180fd160 tx=0x7fa90c01a040 comp rx=0 tx=0).stop 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918100ea0 msgr2=0x7fa91819c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918100ea0 0x7fa91819c790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b4b0 tx=0x7fa90800b870 comp rx=0 tx=0).stop 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 shutdown_connections 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa90006e680 0x7fa900070b40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa918100ea0 0x7fa91819c790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 --2- 172.21.15.42:0/3770264131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa918101850 0x7fa91819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 >> 172.21.15.42:0/3770264131 conn(0x7fa9180fc750 msgr2=0x7fa9180fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 shutdown_connections 2024-09-16T23:59:13.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:13.879+0000 7fa91df60700 1 -- 172.21.15.42:0/3770264131 wait complete. 2024-09-16T23:59:13.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:13 smithi043 ceph-mon[81510]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:13.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:13 smithi043 ceph-mon[81510]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:13.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:13 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/7726375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:59:13.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:13 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1532258415' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:59:14.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:14 smithi042 ceph-mon[111056]: pgmap v229: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:14.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:14 smithi042 ceph-mon[111056]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:14.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:14 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3770264131' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:59:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:14 smithi043 ceph-mon[81510]: pgmap v229: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:14 smithi043 ceph-mon[81510]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:14 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3770264131' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:59:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:16 smithi043 ceph-mon[81510]: pgmap v230: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:16 smithi042 ceph-mon[111056]: pgmap v230: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:18 smithi043 ceph-mon[81510]: pgmap v231: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:18 smithi042 ceph-mon[111056]: pgmap v231: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:20.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:20 smithi043 ceph-mon[81510]: pgmap v232: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:21.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:20 smithi042 ceph-mon[111056]: pgmap v232: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:22.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:22 smithi043 ceph-mon[81510]: pgmap v233: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:23.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:22 smithi042 ceph-mon[111056]: pgmap v233: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:24.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:24 smithi042 ceph-mon[111056]: pgmap v234: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:24.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:24 smithi043 ceph-mon[81510]: pgmap v234: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:26.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:26 smithi043 ceph-mon[81510]: pgmap v235: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:27.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:26 smithi042 ceph-mon[111056]: pgmap v235: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:28.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:28 smithi043 ceph-mon[81510]: pgmap v236: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:29.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:28 smithi042 ceph-mon[111056]: pgmap v236: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:30.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:30 smithi043 ceph-mon[81510]: pgmap v237: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:30.893 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:30 smithi042 ceph-mon[111056]: pgmap v237: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:32.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:32 smithi043 ceph-mon[81510]: pgmap v238: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:33.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:32 smithi042 ceph-mon[111056]: pgmap v238: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:34.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:34 smithi043 ceph-mon[81510]: pgmap v239: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:35.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:34 smithi042 ceph-mon[111056]: pgmap v239: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:36.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:36 smithi043 ceph-mon[81510]: pgmap v240: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:36.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:36 smithi042 ceph-mon[111056]: pgmap v240: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:38 smithi043 ceph-mon[81510]: pgmap v241: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:38 smithi042 ceph-mon[111056]: pgmap v241: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:40.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:40 smithi043 ceph-mon[81510]: pgmap v242: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:41.000 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:40 smithi042 ceph-mon[111056]: pgmap v242: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:42.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:42 smithi043 ceph-mon[81510]: pgmap v243: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:43.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:42 smithi042 ceph-mon[111056]: pgmap v243: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 -- 172.21.15.42:0/319430766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00440ffb40 msgr2=0x7f00440fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 --2- 172.21.15.42:0/319430766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00440ffb40 0x7f00440fffc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0038009910 tx=0x7f0038009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 -- 172.21.15.42:0/319430766 shutdown_connections 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 --2- 172.21.15.42:0/319430766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00440ffb40 0x7f00440fffc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 --2- 172.21.15.42:0/319430766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0044105bf0 0x7f0044105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 -- 172.21.15.42:0/319430766 >> 172.21.15.42:0/319430766 conn(0x7f00440fb650 msgr2=0x7f00440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 -- 172.21.15.42:0/319430766 shutdown_connections 2024-09-16T23:59:44.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.003+0000 7f004b0ce700 1 -- 172.21.15.42:0/319430766 wait complete. 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f004b0ce700 1 Processor -- start 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f004b0ce700 1 -- start start 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f004b0ce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00440ffb40 0x7f00441017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f004b0ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0044105bf0 0x7f0044101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f004b0ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044102230 con 0x7f0044105bf0 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f004b0ce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044102370 con 0x7f00440ffb40 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f0043fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0044105bf0 0x7f0044101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f0043fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0044105bf0 0x7f0044101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55236/0 (socket says 172.21.15.42:55236) 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f0043fff700 1 -- 172.21.15.42:0/2376901356 learned_addr learned my addr 172.21.15.42:0/2376901356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.004+0000 7f0048e6a700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00440ffb40 0x7f00441017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f0043fff700 1 -- 172.21.15.42:0/2376901356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00440ffb40 msgr2=0x7f00441017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f0043fff700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00440ffb40 0x7f00441017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f0043fff700 1 -- 172.21.15.42:0/2376901356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380095f0 con 0x7f0044105bf0 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f0043fff700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0044105bf0 0x7f0044101cf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0038006010 tx=0x7f003800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f0041ffb700 1 -- 172.21.15.42:0/2376901356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f003801d070 con 0x7f0044105bf0 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00441025f0 con 0x7f0044105bf0 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f0041ffb700 1 -- 172.21.15.42:0/2376901356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f003800f460 con 0x7f0044105bf0 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f0041ffb700 1 -- 172.21.15.42:0/2376901356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0038017610 con 0x7f0044105bf0 2024-09-16T23:59:44.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.005+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00441a2ce0 con 0x7f0044105bf0 2024-09-16T23:59:44.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.006+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004404f2e0 con 0x7f0044105bf0 2024-09-16T23:59:44.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.007+0000 7f0041ffb700 1 -- 172.21.15.42:0/2376901356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f003800f5d0 con 0x7f0044105bf0 2024-09-16T23:59:44.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.007+0000 7f0041ffb700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f002c06e680 0x7f002c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.007+0000 7f0041ffb700 1 -- 172.21.15.42:0/2376901356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f003808fb90 con 0x7f0044105bf0 2024-09-16T23:59:44.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.008+0000 7f0048e6a700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f002c06e680 0x7f002c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.008+0000 7f0048e6a700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f002c06e680 0x7f002c070b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0034005950 tx=0x7f003400a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:44.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.012+0000 7f0041ffb700 1 -- 172.21.15.42:0/2376901356 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0038058d10 con 0x7f0044105bf0 2024-09-16T23:59:44.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.243+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00440695c0 con 0x7f002c06e680 2024-09-16T23:59:44.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.244+0000 7f0041ffb700 1 -- 172.21.15.42:0/2376901356 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f00440695c0 con 0x7f002c06e680 2024-09-16T23:59:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f002c06e680 msgr2=0x7f002c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f002c06e680 0x7f002c070b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0034005950 tx=0x7f003400a300 comp rx=0 tx=0).stop 2024-09-16T23:59:44.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0044105bf0 msgr2=0x7f0044101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0044105bf0 0x7f0044101cf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0038006010 tx=0x7f003800bd70 comp rx=0 tx=0).stop 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 shutdown_connections 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f002c06e680 0x7f002c070b40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00440ffb40 0x7f00441017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 --2- 172.21.15.42:0/2376901356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0044105bf0 0x7f0044101cf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 >> 172.21.15.42:0/2376901356 conn(0x7f00440fb650 msgr2=0x7f0044068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 shutdown_connections 2024-09-16T23:59:44.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.247+0000 7f004b0ce700 1 -- 172.21.15.42:0/2376901356 wait complete. 2024-09-16T23:59:44.255 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-16T23:59:44.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.357+0000 7f4583397700 1 -- 172.21.15.42:0/3967446481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c100020 msgr2=0x7f457c103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.357+0000 7f4583397700 1 --2- 172.21.15.42:0/3967446481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c100020 0x7f457c103f10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f456c009910 tx=0x7f456c009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:44.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.357+0000 7f4583397700 1 -- 172.21.15.42:0/3967446481 shutdown_connections 2024-09-16T23:59:44.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.357+0000 7f4583397700 1 --2- 172.21.15.42:0/3967446481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c100020 0x7f457c103f10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.357+0000 7f4583397700 1 --2- 172.21.15.42:0/3967446481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457c0ff670 0x7f457c0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.357+0000 7f4583397700 1 -- 172.21.15.42:0/3967446481 >> 172.21.15.42:0/3967446481 conn(0x7f457c074e30 msgr2=0x7f457c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:44.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.358+0000 7f4583397700 1 -- 172.21.15.42:0/3967446481 shutdown_connections 2024-09-16T23:59:44.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.358+0000 7f4583397700 1 -- 172.21.15.42:0/3967446481 wait complete. 2024-09-16T23:59:44.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.358+0000 7f4583397700 1 Processor -- start 2024-09-16T23:59:44.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.358+0000 7f4583397700 1 -- start start 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4583397700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c0ff670 0x7f457c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4583397700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457c100020 0x7f457c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4583397700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c19d200 con 0x7f457c0ff670 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4583397700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c1966b0 con 0x7f457c100020 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4581133700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c0ff670 0x7f457c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4581133700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c0ff670 0x7f457c19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55256/0 (socket says 172.21.15.42:55256) 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4581133700 1 -- 172.21.15.42:0/3293448915 learned_addr learned my addr 172.21.15.42:0/3293448915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:44.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4580932700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457c100020 0x7f457c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4581133700 1 -- 172.21.15.42:0/3293448915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457c100020 msgr2=0x7f457c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4581133700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457c100020 0x7f457c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4581133700 1 -- 172.21.15.42:0/3293448915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0095f0 con 0x7f457c0ff670 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4580932700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457c100020 0x7f457c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4581133700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c0ff670 0x7f457c19c630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f457800dd40 tx=0x7f457800a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f45727fc700 1 -- 172.21.15.42:0/3293448915 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457800ad10 con 0x7f457c0ff670 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f45727fc700 1 -- 172.21.15.42:0/3293448915 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f457800ae70 con 0x7f457c0ff670 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c196990 con 0x7f457c0ff670 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f45727fc700 1 -- 172.21.15.42:0/3293448915 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4578016960 con 0x7f457c0ff670 2024-09-16T23:59:44.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.359+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457c196ee0 con 0x7f457c0ff670 2024-09-16T23:59:44.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.360+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457c04f2e0 con 0x7f457c0ff670 2024-09-16T23:59:44.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.361+0000 7f45727fc700 1 -- 172.21.15.42:0/3293448915 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f457800e450 con 0x7f457c0ff670 2024-09-16T23:59:44.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.361+0000 7f45727fc700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f456806e680 0x7f4568070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.362+0000 7f45727fc700 1 -- 172.21.15.42:0/3293448915 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f457808e7e0 con 0x7f457c0ff670 2024-09-16T23:59:44.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.362+0000 7f4580932700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f456806e680 0x7f4568070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.362+0000 7f4580932700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f456806e680 0x7f4568070b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f457c105f90 tx=0x7f456c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:44.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.366+0000 7f45727fc700 1 -- 172.21.15.42:0/3293448915 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f457805b5e0 con 0x7f457c0ff670 2024-09-16T23:59:44.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.598+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f457c197250 con 0x7f456806e680 2024-09-16T23:59:44.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.599+0000 7f45727fc700 1 -- 172.21.15.42:0/3293448915 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f457c197250 con 0x7f456806e680 2024-09-16T23:59:44.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f456806e680 msgr2=0x7f4568070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f456806e680 0x7f4568070b40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f457c105f90 tx=0x7f456c01a040 comp rx=0 tx=0).stop 2024-09-16T23:59:44.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c0ff670 msgr2=0x7f457c19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c0ff670 0x7f457c19c630 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f457800dd40 tx=0x7f457800a530 comp rx=0 tx=0).stop 2024-09-16T23:59:44.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 shutdown_connections 2024-09-16T23:59:44.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f456806e680 0x7f4568070b40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f457c0ff670 0x7f457c19c630 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 --2- 172.21.15.42:0/3293448915 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f457c100020 0x7f457c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 >> 172.21.15.42:0/3293448915 conn(0x7f457c074e30 msgr2=0x7f457c0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:44.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 shutdown_connections 2024-09-16T23:59:44.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.601+0000 7f4583397700 1 -- 172.21.15.42:0/3293448915 wait complete. 2024-09-16T23:59:44.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.710+0000 7f9ca6a95700 1 -- 172.21.15.42:0/226965161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 msgr2=0x7f9ca0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.710+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/226965161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca0100180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009910 tx=0x7f9c94009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:44.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.710+0000 7f9ca6a95700 1 -- 172.21.15.42:0/226965161 shutdown_connections 2024-09-16T23:59:44.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.710+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/226965161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca0100750 0x7f9ca01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.710+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/226965161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca0100180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.710+0000 7f9ca6a95700 1 -- 172.21.15.42:0/226965161 >> 172.21.15.42:0/226965161 conn(0x7f9ca00fb650 msgr2=0x7f9ca00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:44.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 -- 172.21.15.42:0/226965161 shutdown_connections 2024-09-16T23:59:44.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 -- 172.21.15.42:0/226965161 wait complete. 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 Processor -- start 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 -- start start 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca0100750 0x7f9ca019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca019d120 con 0x7f9ca00ffda0 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.711+0000 7f9ca6a95700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca01966b0 con 0x7f9ca0100750 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca4831700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca4831700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55274/0 (socket says 172.21.15.42:55274) 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca4831700 1 -- 172.21.15.42:0/2299569855 learned_addr learned my addr 172.21.15.42:0/2299569855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9c9ffff700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca0100750 0x7f9ca019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca4831700 1 -- 172.21.15.42:0/2299569855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca0100750 msgr2=0x7f9ca019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca4831700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca0100750 0x7f9ca019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca4831700 1 -- 172.21.15.42:0/2299569855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940095f0 con 0x7f9ca00ffda0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca4831700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca019c5c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c94006010 tx=0x7f9c9400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9c9dffb700 1 -- 172.21.15.42:0/2299569855 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c9401d070 con 0x7f9ca00ffda0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9c9dffb700 1 -- 172.21.15.42:0/2299569855 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c94005dc0 con 0x7f9ca00ffda0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca0196930 con 0x7f9ca00ffda0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.712+0000 7f9c9dffb700 1 -- 172.21.15.42:0/2299569855 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c94017400 con 0x7f9ca00ffda0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.713+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca0196e20 con 0x7f9ca00ffda0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.713+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca004f2e0 con 0x7f9ca00ffda0 2024-09-16T23:59:44.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.714+0000 7f9c9dffb700 1 -- 172.21.15.42:0/2299569855 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9c94003a60 con 0x7f9ca00ffda0 2024-09-16T23:59:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.715+0000 7f9c9dffb700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c9006e6d0 0x7f9c90070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.715+0000 7f9c9dffb700 1 -- 172.21.15.42:0/2299569855 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c9408fad0 con 0x7f9ca00ffda0 2024-09-16T23:59:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.715+0000 7f9c9ffff700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c9006e6d0 0x7f9c90070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:44.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.715+0000 7f9c9ffff700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c9006e6d0 0x7f9c90070b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0197e10 tx=0x7f9c8c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:44.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.719+0000 7f9c9dffb700 1 -- 172.21.15.42:0/2299569855 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9c9405c780 con 0x7f9ca00ffda0 2024-09-16T23:59:44.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:44 smithi043 ceph-mon[81510]: pgmap v244: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:44.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.939+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ca01971b0 con 0x7f9c9006e6d0 2024-09-16T23:59:44.937 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:44 smithi042 ceph-mon[111056]: pgmap v244: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:44.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.951+0000 7f9c9dffb700 1 -- 172.21.15.42:0/2299569855 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9ca01971b0 con 0x7f9c9006e6d0 2024-09-16T23:59:44.948 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T23:59:44.948 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (8m) 2m ago 19m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-16T23:59:44.949 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (19m) 2m ago 19m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-16T23:59:44.949 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (18m) 94s ago 18m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-16T23:59:44.949 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (6m) 2m ago 19m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-16T23:59:44.949 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (5m) 94s ago 18m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-16T23:59:44.949 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (8m) 2m ago 18m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-16T23:59:44.949 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (2m) 2m ago 13m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-16T23:59:44.949 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (2m) 2m ago 13m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (111s) 94s ago 13m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (97s) 94s ago 13m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (10m) 2m ago 20m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (9m) 94s ago 18m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (7m) 2m ago 20m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (6m) 94s ago 18m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (9m) 2m ago 19m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (8m) 94s ago 18m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-16T23:59:44.950 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (5m) 2m ago 17m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-16T23:59:44.951 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (5m) 2m ago 16m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-16T23:59:44.951 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (4m) 2m ago 16m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-16T23:59:44.951 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (3m) 94s ago 15m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-16T23:59:44.951 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (3m) 94s ago 15m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-16T23:59:44.951 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (2m) 94s ago 15m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-16T23:59:44.951 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (8m) 2m ago 18m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-16T23:59:44.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c9006e6d0 msgr2=0x7f9c90070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c9006e6d0 0x7f9c90070b90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0197e10 tx=0x7f9c8c00b410 comp rx=0 tx=0).stop 2024-09-16T23:59:44.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 msgr2=0x7f9ca019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:44.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca019c5c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9c94006010 tx=0x7f9c9400bd70 comp rx=0 tx=0).stop 2024-09-16T23:59:44.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 shutdown_connections 2024-09-16T23:59:44.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c9006e6d0 0x7f9c90070b90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ca00ffda0 0x7f9ca019c5c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 --2- 172.21.15.42:0/2299569855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ca0100750 0x7f9ca019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:44.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 >> 172.21.15.42:0/2299569855 conn(0x7f9ca00fb650 msgr2=0x7f9ca0103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:44.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 shutdown_connections 2024-09-16T23:59:44.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:44.953+0000 7f9ca6a95700 1 -- 172.21.15.42:0/2299569855 wait complete. 2024-09-16T23:59:45.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 -- 172.21.15.42:0/2224910818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 msgr2=0x7f85081047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 --2- 172.21.15.42:0/2224910818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f85081047a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009910 tx=0x7f84f8009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:45.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 -- 172.21.15.42:0/2224910818 shutdown_connections 2024-09-16T23:59:45.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 --2- 172.21.15.42:0/2224910818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f85081047a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 --2- 172.21.15.42:0/2224910818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85080ffda0 0x7f8508100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 -- 172.21.15.42:0/2224910818 >> 172.21.15.42:0/2224910818 conn(0x7f85080fb650 msgr2=0x7f85080fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 -- 172.21.15.42:0/2224910818 shutdown_connections 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.061+0000 7f850f654700 1 -- 172.21.15.42:0/2224910818 wait complete. 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850f654700 1 Processor -- start 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850f654700 1 -- start start 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850f654700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85080ffda0 0x7f8508196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850f654700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f8508196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850f654700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8508197520 con 0x7f8508100750 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850f654700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850819b330 con 0x7f85080ffda0 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850cbef700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f8508196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850cbef700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f8508196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55298/0 (socket says 172.21.15.42:55298) 2024-09-16T23:59:45.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850cbef700 1 -- 172.21.15.42:0/2578843702 learned_addr learned my addr 172.21.15.42:0/2578843702 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850d3f0700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85080ffda0 0x7f8508196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850cbef700 1 -- 172.21.15.42:0/2578843702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85080ffda0 msgr2=0x7f8508196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850cbef700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85080ffda0 0x7f8508196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.062+0000 7f850cbef700 1 -- 172.21.15.42:0/2578843702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80095f0 con 0x7f8508100750 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.063+0000 7f850cbef700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f8508196e90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f84f80092e0 tx=0x7f84f800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.063+0000 7f84fe7fc700 1 -- 172.21.15.42:0/2578843702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f801d070 con 0x7f8508100750 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.063+0000 7f84fe7fc700 1 -- 172.21.15.42:0/2578843702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84f800f460 con 0x7f8508100750 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.063+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850819b5b0 con 0x7f8508100750 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.063+0000 7f84fe7fc700 1 -- 172.21.15.42:0/2578843702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f80175d0 con 0x7f8508100750 2024-09-16T23:59:45.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.063+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850819baa0 con 0x7f8508100750 2024-09-16T23:59:45.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.064+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850804f2e0 con 0x7f8508100750 2024-09-16T23:59:45.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.065+0000 7f84fe7fc700 1 -- 172.21.15.42:0/2578843702 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f84f800f5d0 con 0x7f8508100750 2024-09-16T23:59:45.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.065+0000 7f84fe7fc700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f84f406e680 0x7f84f4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.065+0000 7f84fe7fc700 1 -- 172.21.15.42:0/2578843702 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f84f808fb70 con 0x7f8508100750 2024-09-16T23:59:45.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.065+0000 7f850d3f0700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f84f406e680 0x7f84f4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.066+0000 7f850d3f0700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f84f406e680 0x7f84f4070b40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8504005950 tx=0x7f850400a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:45.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.070+0000 7f84fe7fc700 1 -- 172.21.15.42:0/2578843702 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f84f805c840 con 0x7f8508100750 2024-09-16T23:59:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.374+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f850819bcc0 con 0x7f8508100750 2024-09-16T23:59:45.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.374+0000 7f84fe7fc700 1 -- 172.21.15.42:0/2578843702 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f84f805c3d0 con 0x7f8508100750 2024-09-16T23:59:45.372 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:59:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-16T23:59:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:59:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-16T23:59:45.373 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:59:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.376+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f84f406e680 msgr2=0x7f84f4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.376+0000 7f850f654700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f84f406e680 0x7f84f4070b40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8504005950 tx=0x7f850400a300 comp rx=0 tx=0).stop 2024-09-16T23:59:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.376+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 msgr2=0x7f8508196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.376+0000 7f850f654700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f8508196e90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f84f80092e0 tx=0x7f84f800bd70 comp rx=0 tx=0).stop 2024-09-16T23:59:45.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.377+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 shutdown_connections 2024-09-16T23:59:45.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.377+0000 7f850f654700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f84f406e680 0x7f84f4070b40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.377+0000 7f850f654700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85080ffda0 0x7f8508196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.377+0000 7f850f654700 1 --2- 172.21.15.42:0/2578843702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8508100750 0x7f8508196e90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.377+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 >> 172.21.15.42:0/2578843702 conn(0x7f85080fb650 msgr2=0x7f8508103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:45.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.377+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 shutdown_connections 2024-09-16T23:59:45.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.377+0000 7f850f654700 1 -- 172.21.15.42:0/2578843702 wait complete. 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.486+0000 7f4c25c39700 1 -- 172.21.15.42:0/3925260933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 msgr2=0x7f4c20101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 --2- 172.21.15.42:0/3925260933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20101280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009910 tx=0x7f4c10009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 -- 172.21.15.42:0/3925260933 shutdown_connections 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 --2- 172.21.15.42:0/3925260933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c20101850 0x7f4c201058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 --2- 172.21.15.42:0/3925260933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20101280 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 -- 172.21.15.42:0/3925260933 >> 172.21.15.42:0/3925260933 conn(0x7f4c200fc750 msgr2=0x7f4c200feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 -- 172.21.15.42:0/3925260933 shutdown_connections 2024-09-16T23:59:45.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 -- 172.21.15.42:0/3925260933 wait complete. 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.487+0000 7f4c25c39700 1 Processor -- start 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c25c39700 1 -- start start 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c25c39700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c25c39700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c20101850 0x7f4c200735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c25c39700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20073b10 con 0x7f4c20100ea0 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c25c39700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c20073c50 con 0x7f4c20101850 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55314/0 (socket says 172.21.15.42:55314) 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1f7fe700 1 -- 172.21.15.42:0/1432271093 learned_addr learned my addr 172.21.15.42:0/1432271093 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1effd700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c20101850 0x7f4c200735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1f7fe700 1 -- 172.21.15.42:0/1432271093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c20101850 msgr2=0x7f4c200735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1f7fe700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c20101850 0x7f4c200735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1f7fe700 1 -- 172.21.15.42:0/1432271093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c100095f0 con 0x7f4c20100ea0 2024-09-16T23:59:45.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.488+0000 7f4c1f7fe700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20073090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000b5c0 tx=0x7f4c1000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:45.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.489+0000 7f4c1cff9700 1 -- 172.21.15.42:0/1432271093 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c1001d070 con 0x7f4c20100ea0 2024-09-16T23:59:45.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.489+0000 7f4c1cff9700 1 -- 172.21.15.42:0/1432271093 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c1000f460 con 0x7f4c20100ea0 2024-09-16T23:59:45.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.489+0000 7f4c1cff9700 1 -- 172.21.15.42:0/1432271093 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c10017630 con 0x7f4c20100ea0 2024-09-16T23:59:45.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.489+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c20073ed0 con 0x7f4c20100ea0 2024-09-16T23:59:45.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.489+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c201a2dd0 con 0x7f4c20100ea0 2024-09-16T23:59:45.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.490+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c20109240 con 0x7f4c20100ea0 2024-09-16T23:59:45.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.490+0000 7f4c1cff9700 1 -- 172.21.15.42:0/1432271093 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f4c1000f5d0 con 0x7f4c20100ea0 2024-09-16T23:59:45.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.491+0000 7f4c1cff9700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4c0c06e680 0x7f4c0c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.491+0000 7f4c1cff9700 1 -- 172.21.15.42:0/1432271093 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4c1008faf0 con 0x7f4c20100ea0 2024-09-16T23:59:45.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.491+0000 7f4c1effd700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4c0c06e680 0x7f4c0c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.491+0000 7f4c1effd700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4c0c06e680 0x7f4c0c070b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c20074880 tx=0x7f4c0800a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:45.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.495+0000 7f4c1cff9700 1 -- 172.21.15.42:0/1432271093 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4c10058d10 con 0x7f4c20100ea0 2024-09-16T23:59:45.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.746+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4c20074680 con 0x7f4c20100ea0 2024-09-16T23:59:45.744 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:45 smithi042 ceph-mon[111056]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:45.745 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:45 smithi042 ceph-mon[111056]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:45.745 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:45 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2578843702' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:59:45.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.747+0000 7f4c1cff9700 1 -- 172.21.15.42:0/1432271093 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f4c1005c3d0 con 0x7f4c20100ea0 2024-09-16T23:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-16T23:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-16T23:59:45.745 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-16T23:59:45.746 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-16T23:59:45.747 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:45.748 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-16T23:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-16T23:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:45.749 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4c0c06e680 msgr2=0x7f4c0c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4c0c06e680 0x7f4c0c070b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c20074880 tx=0x7f4c0800a380 comp rx=0 tx=0).stop 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 msgr2=0x7f4c20073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20073090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000b5c0 tx=0x7f4c1000bd50 comp rx=0 tx=0).stop 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 shutdown_connections 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4c0c06e680 0x7f4c0c070b40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4c20100ea0 0x7f4c20073090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 --2- 172.21.15.42:0/1432271093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c20101850 0x7f4c200735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 >> 172.21.15.42:0/1432271093 conn(0x7f4c200fc750 msgr2=0x7f4c200fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:45.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 shutdown_connections 2024-09-16T23:59:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.749+0000 7f4c25c39700 1 -- 172.21.15.42:0/1432271093 wait complete. 2024-09-16T23:59:45.751 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-16T23:59:45.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 -- 172.21.15.42:0/586260324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 msgr2=0x7fbfd00ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 --2- 172.21.15.42:0/586260324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd00ff980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009910 tx=0x7fbfc0009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:45.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 -- 172.21.15.42:0/586260324 shutdown_connections 2024-09-16T23:59:45.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 --2- 172.21.15.42:0/586260324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd00fff50 0x7fbfd0105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 --2- 172.21.15.42:0/586260324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd00ff980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 -- 172.21.15.42:0/586260324 >> 172.21.15.42:0/586260324 conn(0x7fbfd0074e30 msgr2=0x7fbfd0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 -- 172.21.15.42:0/586260324 shutdown_connections 2024-09-16T23:59:45.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.858+0000 7fbfd591c700 1 -- 172.21.15.42:0/586260324 wait complete. 2024-09-16T23:59:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.859+0000 7fbfd591c700 1 Processor -- start 2024-09-16T23:59:45.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.859+0000 7fbfd591c700 1 -- start start 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.859+0000 7fbfd591c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd019c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.859+0000 7fbfd591c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd00fff50 0x7fbfd019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.859+0000 7fbfd591c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd019d250 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.859+0000 7fbfd591c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd0196700 con 0x7fbfd00fff50 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfceffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd019c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfceffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd019c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55334/0 (socket says 172.21.15.42:55334) 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfceffd700 1 -- 172.21.15.42:0/1062948415 learned_addr learned my addr 172.21.15.42:0/1062948415 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfce7fc700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd00fff50 0x7fbfd019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfceffd700 1 -- 172.21.15.42:0/1062948415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd00fff50 msgr2=0x7fbfd019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfceffd700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd00fff50 0x7fbfd019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfceffd700 1 -- 172.21.15.42:0/1062948415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00095f0 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfce7fc700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd00fff50 0x7fbfd019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T23:59:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfceffd700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd019c680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006010 tx=0x7fbfc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfd491a700 1 -- 172.21.15.42:0/1062948415 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc001d070 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfd491a700 1 -- 172.21.15.42:0/1062948415 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfc000f460 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd0196980 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfd491a700 1 -- 172.21.15.42:0/1062948415 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc0017630 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.860+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd0196e70 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.861+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd0109270 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.862+0000 7fbfd491a700 1 -- 172.21.15.42:0/1062948415 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fbfc000f5d0 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.862+0000 7fbfd491a700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbfb806e680 0x7fbfb8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.863+0000 7fbfd491a700 1 -- 172.21.15.42:0/1062948415 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbfc008fad0 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.863+0000 7fbfce7fc700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbfb806e680 0x7fbfb8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:45.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.863+0000 7fbfce7fc700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbfb806e680 0x7fbfb8070b40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005fd0 tx=0x7fbfc4005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:45.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:45.867+0000 7fbfd491a700 1 -- 172.21.15.42:0/1062948415 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbfc005c820 con 0x7fbfd00ff5a0 2024-09-16T23:59:45.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:45 smithi043 ceph-mon[81510]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:45 smithi043 ceph-mon[81510]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:45 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2578843702' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T23:59:46.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.102+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfd0197d20 con 0x7fbfb806e680 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.103+0000 7fbfd491a700 1 -- 172.21.15.42:0/1062948415 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbfd0197d20 con 0x7fbfb806e680 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-16T23:59:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-16T23:59:46.102 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-16T23:59:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbfb806e680 msgr2=0x7fbfb8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:46.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbfb806e680 0x7fbfb8070b40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005fd0 tx=0x7fbfc4005ee0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 msgr2=0x7fbfd019c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd019c680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0006010 tx=0x7fbfc000be20 comp rx=0 tx=0).stop 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 shutdown_connections 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbfb806e680 0x7fbfb8070b40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbfd00ff5a0 0x7fbfd019c680 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 --2- 172.21.15.42:0/1062948415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfd00fff50 0x7fbfd019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 >> 172.21.15.42:0/1062948415 conn(0x7fbfd0074e30 msgr2=0x7fbfd00fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 shutdown_connections 2024-09-16T23:59:46.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.106+0000 7fbfd591c700 1 -- 172.21.15.42:0/1062948415 wait complete. 2024-09-16T23:59:46.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 -- 172.21.15.42:0/485274917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068580 msgr2=0x7fa32c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:46.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 --2- 172.21.15.42:0/485274917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068580 0x7fa32c068960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa328009910 tx=0x7fa328009c20 comp rx=0 tx=0).stop 2024-09-16T23:59:46.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 -- 172.21.15.42:0/485274917 shutdown_connections 2024-09-16T23:59:46.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 --2- 172.21.15.42:0/485274917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c068ea0 0x7fa32c10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 --2- 172.21.15.42:0/485274917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068580 0x7fa32c068960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 -- 172.21.15.42:0/485274917 >> 172.21.15.42:0/485274917 conn(0x7fa32c075670 msgr2=0x7fa32c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 -- 172.21.15.42:0/485274917 shutdown_connections 2024-09-16T23:59:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.214+0000 7fa333fb4700 1 -- 172.21.15.42:0/485274917 wait complete. 2024-09-16T23:59:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa333fb4700 1 Processor -- start 2024-09-16T23:59:46.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa333fb4700 1 -- start start 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa333fb4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c068580 0x7fa32c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa333fb4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068ea0 0x7fa32c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa333fb4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c19f430 con 0x7fa32c068ea0 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa333fb4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c1988e0 con 0x7fa32c068580 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa33154f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068ea0 0x7fa32c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa33154f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068ea0 0x7fa32c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55354/0 (socket says 172.21.15.42:55354) 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa33154f700 1 -- 172.21.15.42:0/3224967181 learned_addr learned my addr 172.21.15.42:0/3224967181 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.215+0000 7fa331d50700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c068580 0x7fa32c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa33154f700 1 -- 172.21.15.42:0/3224967181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c068580 msgr2=0x7fa32c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa33154f700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c068580 0x7fa32c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa33154f700 1 -- 172.21.15.42:0/3224967181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3280095f0 con 0x7fa32c068ea0 2024-09-16T23:59:46.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa33154f700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068ea0 0x7fa32c19eda0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00d8d0 tx=0x7fa31c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa322ffd700 1 -- 172.21.15.42:0/3224967181 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31c0098a0 con 0x7fa32c068ea0 2024-09-16T23:59:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa322ffd700 1 -- 172.21.15.42:0/3224967181 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa31c010460 con 0x7fa32c068ea0 2024-09-16T23:59:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c198bc0 con 0x7fa32c068ea0 2024-09-16T23:59:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa322ffd700 1 -- 172.21.15.42:0/3224967181 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31c00f5f0 con 0x7fa32c068ea0 2024-09-16T23:59:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.216+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c199110 con 0x7fa32c068ea0 2024-09-16T23:59:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.217+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c10aa00 con 0x7fa32c068ea0 2024-09-16T23:59:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.217+0000 7fa322ffd700 1 -- 172.21.15.42:0/3224967181 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa31c009a00 con 0x7fa32c068ea0 2024-09-16T23:59:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.218+0000 7fa322ffd700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa31806e680 0x7fa318070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T23:59:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.218+0000 7fa322ffd700 1 -- 172.21.15.42:0/3224967181 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa31c08e450 con 0x7fa32c068ea0 2024-09-16T23:59:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.218+0000 7fa331d50700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa31806e680 0x7fa318070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T23:59:46.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.219+0000 7fa331d50700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa31806e680 0x7fa318070b40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b5c0 tx=0x7fa32801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T23:59:46.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.222+0000 7fa322ffd700 1 -- 172.21.15.42:0/3224967181 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa31c05b1a0 con 0x7fa32c068ea0 2024-09-16T23:59:46.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.519+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa32c04f2e0 con 0x7fa32c068ea0 2024-09-16T23:59:46.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.520+0000 7fa322ffd700 1 -- 172.21.15.42:0/3224967181 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa31c05ad30 con 0x7fa32c068ea0 2024-09-16T23:59:46.521 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-16T23:59:46.521 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T23:59:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-16T23:59:46.521 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-16T23:59:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-16T23:59:46.521 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-16T23:59:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa31806e680 msgr2=0x7fa318070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa31806e680 0x7fa318070b40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa32800b5c0 tx=0x7fa32801a040 comp rx=0 tx=0).stop 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068ea0 msgr2=0x7fa32c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068ea0 0x7fa32c19eda0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00d8d0 tx=0x7fa31c00dbe0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 shutdown_connections 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa31806e680 0x7fa318070b40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa32c068580 0x7fa32c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 --2- 172.21.15.42:0/3224967181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa32c068ea0 0x7fa32c19eda0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T23:59:46.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 >> 172.21.15.42:0/3224967181 conn(0x7fa32c075670 msgr2=0x7fa32c0fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T23:59:46.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 shutdown_connections 2024-09-16T23:59:46.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-16T23:59:46.522+0000 7fa333fb4700 1 -- 172.21.15.42:0/3224967181 wait complete. 2024-09-16T23:59:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:46 smithi042 ceph-mon[111056]: pgmap v245: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:46 smithi042 ceph-mon[111056]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:46 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1432271093' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:59:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:46 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3224967181' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:59:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:46 smithi043 ceph-mon[81510]: pgmap v245: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:46 smithi043 ceph-mon[81510]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:46 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1432271093' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T23:59:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:46 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3224967181' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T23:59:47.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:47 smithi043 ceph-mon[81510]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:48.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:47 smithi042 ceph-mon[111056]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T23:59:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:48 smithi043 ceph-mon[81510]: pgmap v246: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:49.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:48 smithi042 ceph-mon[111056]: pgmap v246: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:50 smithi043 ceph-mon[81510]: pgmap v247: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:50 smithi042 ceph-mon[111056]: pgmap v247: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:52.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T23:59:52.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:52 smithi043 ceph-mon[81510]: pgmap v248: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:53.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:52 smithi042 ceph-mon[111056]: pgmap v248: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:54 smithi043 ceph-mon[81510]: pgmap v249: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:55.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:54 smithi042 ceph-mon[111056]: pgmap v249: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:56.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:56 smithi043 ceph-mon[81510]: pgmap v250: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:57.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:56 smithi042 ceph-mon[111056]: pgmap v250: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:58.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 16 23:59:58 smithi043 ceph-mon[81510]: pgmap v251: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T23:59:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 16 23:59:58 smithi042 ceph-mon[111056]: pgmap v251: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:00:00.000+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:00:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:00:00.000+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:00:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:00:00.000+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-17T00:00:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:00:00.000+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:00:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:00:00.000+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:00:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:00:00.000+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:00:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:00:00.000+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: pgmap v252: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: fs cephfs is degraded 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:00:01.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:00 smithi042 ceph-mon[111056]: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: pgmap v252: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: fs cephfs is degraded 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:00:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:00 smithi043 ceph-mon[81510]: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:00:03.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:02 smithi042 ceph-mon[111056]: pgmap v253: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:03.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:02 smithi043 ceph-mon[81510]: pgmap v253: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:04.936 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:04 smithi043 ceph-mon[81510]: pgmap v254: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:05.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:04 smithi042 ceph-mon[111056]: pgmap v254: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:06.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:06 smithi042 ceph-mon[111056]: pgmap v255: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:07.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:06 smithi043 ceph-mon[81510]: pgmap v255: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:09.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:08 smithi042 ceph-mon[111056]: pgmap v256: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:09.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:08 smithi043 ceph-mon[81510]: pgmap v256: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:11.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:10 smithi042 ceph-mon[111056]: pgmap v257: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:11.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:10 smithi043 ceph-mon[81510]: pgmap v257: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:12.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:11 smithi042 ceph-mon[111056]: pgmap v258: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:11 smithi043 ceph-mon[81510]: pgmap v258: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:14.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:13 smithi042 ceph-mon[111056]: pgmap v259: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:14.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:13 smithi043 ceph-mon[81510]: pgmap v259: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:16.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:15 smithi042 ceph-mon[111056]: pgmap v260: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:15 smithi043 ceph-mon[81510]: pgmap v260: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.640+0000 7f744a3fb700 1 -- 172.21.15.42:0/3046186470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 msgr2=0x7f7444102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.640+0000 7f744a3fb700 1 --2- 172.21.15.42:0/3046186470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f7444102280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7434009910 tx=0x7f7434009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.640+0000 7f744a3fb700 1 -- 172.21.15.42:0/3046186470 shutdown_connections 2024-09-17T00:00:16.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.640+0000 7f744a3fb700 1 --2- 172.21.15.42:0/3046186470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74441027c0 0x7f744410acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:16.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.640+0000 7f744a3fb700 1 --2- 172.21.15.42:0/3046186470 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f7444102280 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:16.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.640+0000 7f744a3fb700 1 -- 172.21.15.42:0/3046186470 >> 172.21.15.42:0/3046186470 conn(0x7f74440fb650 msgr2=0x7f74440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:16.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.641+0000 7f744a3fb700 1 -- 172.21.15.42:0/3046186470 shutdown_connections 2024-09-17T00:00:16.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.641+0000 7f744a3fb700 1 -- 172.21.15.42:0/3046186470 wait complete. 2024-09-17T00:00:16.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.641+0000 7f744a3fb700 1 Processor -- start 2024-09-17T00:00:16.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.641+0000 7f744a3fb700 1 -- start start 2024-09-17T00:00:16.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f744a3fb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f744419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f744a3fb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74441027c0 0x7f744419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f744a3fb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744419d370 con 0x7f7444101ea0 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f744a3fb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7444196820 con 0x7f74441027c0 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f7443fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f744419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f7443fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f744419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43514/0 (socket says 172.21.15.42:43514) 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f7443fff700 1 -- 172.21.15.42:0/183949898 learned_addr learned my addr 172.21.15.42:0/183949898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f74437fe700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74441027c0 0x7f744419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f7443fff700 1 -- 172.21.15.42:0/183949898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74441027c0 msgr2=0x7f744419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f7443fff700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74441027c0 0x7f744419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f7443fff700 1 -- 172.21.15.42:0/183949898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74340095f0 con 0x7f7444101ea0 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f7443fff700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f744419c7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7434006010 tx=0x7f743400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.642+0000 7f74417fa700 1 -- 172.21.15.42:0/183949898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743401d070 con 0x7f7444101ea0 2024-09-17T00:00:16.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.643+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7444196aa0 con 0x7f7444101ea0 2024-09-17T00:00:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.643+0000 7f74417fa700 1 -- 172.21.15.42:0/183949898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7434004fb0 con 0x7f7444101ea0 2024-09-17T00:00:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.643+0000 7f74417fa700 1 -- 172.21.15.42:0/183949898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7434022ea0 con 0x7f7444101ea0 2024-09-17T00:00:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.643+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7444196f90 con 0x7f7444101ea0 2024-09-17T00:00:16.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.644+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7444108380 con 0x7f7444101ea0 2024-09-17T00:00:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.644+0000 7f74417fa700 1 -- 172.21.15.42:0/183949898 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f7434022470 con 0x7f7444101ea0 2024-09-17T00:00:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.645+0000 7f74417fa700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f742c06e680 0x7f742c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.645+0000 7f74417fa700 1 -- 172.21.15.42:0/183949898 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f743408fbf0 con 0x7f7444101ea0 2024-09-17T00:00:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.645+0000 7f74437fe700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f742c06e680 0x7f742c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:16.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.645+0000 7f74437fe700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f742c06e680 0x7f742c070b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7438006fd0 tx=0x7f7438008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:16.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.649+0000 7f74417fa700 1 -- 172.21.15.42:0/183949898 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f743405c970 con 0x7f7444101ea0 2024-09-17T00:00:16.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.880+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74441971b0 con 0x7f742c06e680 2024-09-17T00:00:16.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.881+0000 7f74417fa700 1 -- 172.21.15.42:0/183949898 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f74441971b0 con 0x7f742c06e680 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f742c06e680 msgr2=0x7f742c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f742c06e680 0x7f742c070b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7438006fd0 tx=0x7f7438008040 comp rx=0 tx=0).stop 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 msgr2=0x7f744419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f744419c7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7434006010 tx=0x7f743400be20 comp rx=0 tx=0).stop 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 shutdown_connections 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f742c06e680 0x7f742c070b40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7444101ea0 0x7f744419c7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 --2- 172.21.15.42:0/183949898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74441027c0 0x7f744419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 >> 172.21.15.42:0/183949898 conn(0x7f74440fb650 msgr2=0x7f74441054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:16.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 shutdown_connections 2024-09-17T00:00:16.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.883+0000 7f744a3fb700 1 -- 172.21.15.42:0/183949898 wait complete. 2024-09-17T00:00:16.893 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:00:17.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 -- 172.21.15.42:0/3879949676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 msgr2=0x7f63680689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 --2- 172.21.15.42:0/3879949676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680689e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6358009910 tx=0x7f6358009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:17.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 -- 172.21.15.42:0/3879949676 shutdown_connections 2024-09-17T00:00:17.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 --2- 172.21.15.42:0/3879949676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680689e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 --2- 172.21.15.42:0/3879949676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106cd0 0x7f63681070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 -- 172.21.15.42:0/3879949676 >> 172.21.15.42:0/3879949676 conn(0x7f6368075670 msgr2=0x7f6368075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 -- 172.21.15.42:0/3879949676 shutdown_connections 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.993+0000 7f6370134700 1 -- 172.21.15.42:0/3879949676 wait complete. 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f6370134700 1 Processor -- start 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f6370134700 1 -- start start 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f6370134700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f6370134700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106cd0 0x7f6368100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f6370134700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368100770 con 0x7f6368068580 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f6370134700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63681008b0 con 0x7f6368106cd0 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f636ded0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f636ded0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43544/0 (socket says 172.21.15.42:43544) 2024-09-17T00:00:17.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f636ded0700 1 -- 172.21.15.42:0/3699762651 learned_addr learned my addr 172.21.15.42:0/3699762651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.994+0000 7f636d6cf700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106cd0 0x7f6368100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f636ded0700 1 -- 172.21.15.42:0/3699762651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106cd0 msgr2=0x7f6368100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f636ded0700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106cd0 0x7f6368100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f636ded0700 1 -- 172.21.15.42:0/3699762651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63580095f0 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f636ded0700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680ffcf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f636400d8d0 tx=0x7f636400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f635effd700 1 -- 172.21.15.42:0/3699762651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63640098a0 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f635effd700 1 -- 172.21.15.42:0/3699762651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6364010460 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63681a2a30 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f635effd700 1 -- 172.21.15.42:0/3699762651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f636400f5f0 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.995+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63681a2de0 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.996+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63681091d0 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.997+0000 7f635effd700 1 -- 172.21.15.42:0/3699762651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6364009a00 con 0x7f6368068580 2024-09-17T00:00:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.997+0000 7f635effd700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f635406e680 0x7f6354070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.997+0000 7f635effd700 1 -- 172.21.15.42:0/3699762651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f636408e450 con 0x7f6368068580 2024-09-17T00:00:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.997+0000 7f636d6cf700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f635406e680 0x7f6354070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:16.998+0000 7f636d6cf700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f635406e680 0x7f6354070b40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f635800b5c0 tx=0x7f635801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.001+0000 7f635effd700 1 -- 172.21.15.42:0/3699762651 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6364057670 con 0x7f6368068580 2024-09-17T00:00:17.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.230+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f636802cce0 con 0x7f635406e680 2024-09-17T00:00:17.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.232+0000 7f635effd700 1 -- 172.21.15.42:0/3699762651 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f636802cce0 con 0x7f635406e680 2024-09-17T00:00:17.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f635406e680 msgr2=0x7f6354070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f635406e680 0x7f6354070b40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f635800b5c0 tx=0x7f635801a040 comp rx=0 tx=0).stop 2024-09-17T00:00:17.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 msgr2=0x7f63680ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680ffcf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f636400d8d0 tx=0x7f636400dbe0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 shutdown_connections 2024-09-17T00:00:17.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f635406e680 0x7f6354070b40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6368068580 0x7f63680ffcf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 --2- 172.21.15.42:0/3699762651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6368106cd0 0x7f6368100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 >> 172.21.15.42:0/3699762651 conn(0x7f6368075670 msgr2=0x7f63680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:17.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 shutdown_connections 2024-09-17T00:00:17.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.234+0000 7f6370134700 1 -- 172.21.15.42:0/3699762651 wait complete. 2024-09-17T00:00:17.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.342+0000 7f6fd0a16700 1 -- 172.21.15.42:0/1942988049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 msgr2=0x7f6fcc10acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.342+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/1942988049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc10acb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009910 tx=0x7f6fc0009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:17.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.343+0000 7f6fd0a16700 1 -- 172.21.15.42:0/1942988049 shutdown_connections 2024-09-17T00:00:17.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.343+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/1942988049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc10acb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.343+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/1942988049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fcc101ea0 0x7f6fcc102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.343+0000 7f6fd0a16700 1 -- 172.21.15.42:0/1942988049 >> 172.21.15.42:0/1942988049 conn(0x7f6fcc0fb650 msgr2=0x7f6fcc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:17.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.343+0000 7f6fd0a16700 1 -- 172.21.15.42:0/1942988049 shutdown_connections 2024-09-17T00:00:17.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.343+0000 7f6fd0a16700 1 -- 172.21.15.42:0/1942988049 wait complete. 2024-09-17T00:00:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.343+0000 7f6fd0a16700 1 Processor -- start 2024-09-17T00:00:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fd0a16700 1 -- start start 2024-09-17T00:00:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fd0a16700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fcc101ea0 0x7f6fcc19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fd0a16700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fd0a16700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc19d230 con 0x7f6fcc1027c0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fd0a16700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1966e0 con 0x7f6fcc101ea0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fc9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fc9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43556/0 (socket says 172.21.15.42:43556) 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fc9d9b700 1 -- 172.21.15.42:0/541406562 learned_addr learned my addr 172.21.15.42:0/541406562 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fca59c700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fcc101ea0 0x7f6fcc19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fc9d9b700 1 -- 172.21.15.42:0/541406562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fcc101ea0 msgr2=0x7f6fcc19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fc9d9b700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fcc101ea0 0x7f6fcc19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fc9d9b700 1 -- 172.21.15.42:0/541406562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00095f0 con 0x7f6fcc1027c0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.344+0000 7f6fc9d9b700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc19cba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00092e0 tx=0x7f6fc000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.345+0000 7f6fbb7fe700 1 -- 172.21.15.42:0/541406562 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc001d070 con 0x7f6fcc1027c0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.345+0000 7f6fbb7fe700 1 -- 172.21.15.42:0/541406562 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fc0005dc0 con 0x7f6fcc1027c0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.345+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc196960 con 0x7f6fcc1027c0 2024-09-17T00:00:17.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.345+0000 7f6fbb7fe700 1 -- 172.21.15.42:0/541406562 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc0017400 con 0x7f6fcc1027c0 2024-09-17T00:00:17.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.345+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc196e50 con 0x7f6fcc1027c0 2024-09-17T00:00:17.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.346+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc108340 con 0x7f6fcc1027c0 2024-09-17T00:00:17.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.346+0000 7f6fbb7fe700 1 -- 172.21.15.42:0/541406562 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6fc000f6f0 con 0x7f6fcc1027c0 2024-09-17T00:00:17.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.347+0000 7f6fbb7fe700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6fb406e680 0x7f6fb4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.347+0000 7f6fbb7fe700 1 -- 172.21.15.42:0/541406562 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6fc008f980 con 0x7f6fcc1027c0 2024-09-17T00:00:17.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.347+0000 7f6fca59c700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6fb406e680 0x7f6fb4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.347+0000 7f6fca59c700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6fb406e680 0x7f6fb4070b40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005950 tx=0x7f6fbc00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:17.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.351+0000 7f6fbb7fe700 1 -- 172.21.15.42:0/541406562 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6fc005c6d0 con 0x7f6fcc1027c0 2024-09-17T00:00:17.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.572+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fcc1970c0 con 0x7f6fb406e680 2024-09-17T00:00:17.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.581+0000 7f6fbb7fe700 1 -- 172.21.15.42:0/541406562 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f6fcc1970c0 con 0x7f6fb406e680 2024-09-17T00:00:17.580 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:00:17.581 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (8m) 2m ago 19m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:00:17.581 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (19m) 2m ago 19m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:00:17.581 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (18m) 2m ago 18m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:00:17.581 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (6m) 2m ago 19m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:00:17.581 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (6m) 2m ago 18m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:00:17.582 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (8m) 2m ago 19m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:00:17.582 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (2m) 2m ago 13m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:00:17.582 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (2m) 2m ago 14m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:00:17.582 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (2m) 2m ago 14m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:00:17.582 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (2m) 2m ago 13m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:00:17.582 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (10m) 2m ago 21m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:00:17.583 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (9m) 2m ago 18m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:00:17.583 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (7m) 2m ago 21m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:00:17.583 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (7m) 2m ago 18m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:00:17.583 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (9m) 2m ago 19m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:00:17.583 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (9m) 2m ago 18m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:00:17.583 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 2m ago 17m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:00:17.583 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (5m) 2m ago 17m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:00:17.584 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (4m) 2m ago 16m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:00:17.584 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (4m) 2m ago 16m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:00:17.584 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (3m) 2m ago 16m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:00:17.584 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (3m) 2m ago 15m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:00:17.584 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (9m) 2m ago 19m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:00:17.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6fb406e680 msgr2=0x7f6fb4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6fb406e680 0x7f6fb4070b40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc005950 tx=0x7f6fbc00b410 comp rx=0 tx=0).stop 2024-09-17T00:00:17.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 msgr2=0x7f6fcc19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc19cba0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00092e0 tx=0x7f6fc000bd70 comp rx=0 tx=0).stop 2024-09-17T00:00:17.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 shutdown_connections 2024-09-17T00:00:17.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6fb406e680 0x7f6fb4070b40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fcc101ea0 0x7f6fcc19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 --2- 172.21.15.42:0/541406562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6fcc1027c0 0x7f6fcc19cba0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 >> 172.21.15.42:0/541406562 conn(0x7f6fcc0fb650 msgr2=0x7f6fcc105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:17.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 shutdown_connections 2024-09-17T00:00:17.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.584+0000 7f6fd0a16700 1 -- 172.21.15.42:0/541406562 wait complete. 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1635642536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 msgr2=0x7f19e4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1635642536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e4101280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f19d400b0a0 tx=0x7f19d400b3b0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1635642536 shutdown_connections 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1635642536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e4101850 0x7f19e41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1635642536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e4101280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1635642536 >> 172.21.15.42:0/1635642536 conn(0x7f19e40fc750 msgr2=0x7f19e40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1635642536 shutdown_connections 2024-09-17T00:00:17.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.692+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1635642536 wait complete. 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19eb7b8700 1 Processor -- start 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19eb7b8700 1 -- start start 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19eb7b8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19eb7b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e4101850 0x7f19e419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19eb7b8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e419d390 con 0x7f19e4100ea0 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19eb7b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4196840 con 0x7f19e4101850 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19e9554700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19e9554700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43570/0 (socket says 172.21.15.42:43570) 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19e9554700 1 -- 172.21.15.42:0/1882375305 learned_addr learned my addr 172.21.15.42:0/1882375305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19e8d53700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e4101850 0x7f19e419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19e9554700 1 -- 172.21.15.42:0/1882375305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e4101850 msgr2=0x7f19e419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19e9554700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e4101850 0x7f19e419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.693+0000 7f19e9554700 1 -- 172.21.15.42:0/1882375305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4009d00 con 0x7f19e4100ea0 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.694+0000 7f19e9554700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e419c7c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19d400b070 tx=0x7f19d400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.694+0000 7f19e27fc700 1 -- 172.21.15.42:0/1882375305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d400e040 con 0x7f19e4100ea0 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.694+0000 7f19e27fc700 1 -- 172.21.15.42:0/1882375305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19d4003e60 con 0x7f19e4100ea0 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.694+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e4196ac0 con 0x7f19e4100ea0 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.694+0000 7f19e27fc700 1 -- 172.21.15.42:0/1882375305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d401d910 con 0x7f19e4100ea0 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.694+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e4196fb0 con 0x7f19e4100ea0 2024-09-17T00:00:17.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.695+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e4109240 con 0x7f19e4100ea0 2024-09-17T00:00:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.696+0000 7f19e27fc700 1 -- 172.21.15.42:0/1882375305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f19d4004820 con 0x7f19e4100ea0 2024-09-17T00:00:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.696+0000 7f19e27fc700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f19cc06e680 0x7f19cc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.696+0000 7f19e27fc700 1 -- 172.21.15.42:0/1882375305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f19d408ffc0 con 0x7f19e4100ea0 2024-09-17T00:00:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.696+0000 7f19e8d53700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f19cc06e680 0x7f19cc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:17.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.697+0000 7f19e8d53700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f19cc06e680 0x7f19cc070b40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f19d8006fd0 tx=0x7f19d8008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:17.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.701+0000 7f19e27fc700 1 -- 172.21.15.42:0/1882375305 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f19d405cc90 con 0x7f19e4100ea0 2024-09-17T00:00:17.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.998+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19e404f2e0 con 0x7f19e4100ea0 2024-09-17T00:00:17.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:17 smithi042 ceph-mon[111056]: pgmap v261: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:17.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:17 smithi042 ceph-mon[111056]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:17.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:17 smithi042 ceph-mon[111056]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:17.998 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:17 smithi042 ceph-mon[111056]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:17.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:17.999+0000 7f19e27fc700 1 -- 172.21.15.42:0/1882375305 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f19d405c820 con 0x7f19e4100ea0 2024-09-17T00:00:18.000 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:00:18.000 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:00:18.000 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:00:18.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:18.000 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:00:18.001 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:00:18.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:18.001 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:00:18.001 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:00:18.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:18.002 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:00:18.002 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:00:18.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:18.002 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:00:18.002 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:00:18.003 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:00:18.003 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:00:18.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f19cc06e680 msgr2=0x7f19cc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f19cc06e680 0x7f19cc070b40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f19d8006fd0 tx=0x7f19d8008040 comp rx=0 tx=0).stop 2024-09-17T00:00:18.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 msgr2=0x7f19e419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e419c7c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f19d400b070 tx=0x7f19d400cdd0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 shutdown_connections 2024-09-17T00:00:18.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f19cc06e680 0x7f19cc070b40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19e4100ea0 0x7f19e419c7c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 --2- 172.21.15.42:0/1882375305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19e4101850 0x7f19e419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 >> 172.21.15.42:0/1882375305 conn(0x7f19e40fc750 msgr2=0x7f19e40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:18.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 shutdown_connections 2024-09-17T00:00:18.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.001+0000 7f19eb7b8700 1 -- 172.21.15.42:0/1882375305 wait complete. 2024-09-17T00:00:18.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 -- 172.21.15.42:0/845819078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef0105a20 msgr2=0x7efef0105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 --2- 172.21.15.42:0/845819078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef0105a20 0x7efef0105e00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efee000b0a0 tx=0x7efee000b3b0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 -- 172.21.15.42:0/845819078 shutdown_connections 2024-09-17T00:00:18.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 --2- 172.21.15.42:0/845819078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef00ffb60 0x7efef00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 --2- 172.21.15.42:0/845819078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef0105a20 0x7efef0105e00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 -- 172.21.15.42:0/845819078 >> 172.21.15.42:0/845819078 conn(0x7efef00754a0 msgr2=0x7efef00758b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:18.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 -- 172.21.15.42:0/845819078 shutdown_connections 2024-09-17T00:00:18.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.109+0000 7efef50ab700 1 -- 172.21.15.42:0/845819078 wait complete. 2024-09-17T00:00:18.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.110+0000 7efef50ab700 1 Processor -- start 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.110+0000 7efef50ab700 1 -- start start 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.110+0000 7efef50ab700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef00ffb60 0x7efef0197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.110+0000 7efef50ab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef0105a20 0x7efef0197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.110+0000 7efef50ab700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef01980c0 con 0x7efef00ffb60 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.110+0000 7efef50ab700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0198200 con 0x7efef0105a20 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeeed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef00ffb60 0x7efef0197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeeed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef00ffb60 0x7efef0197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43580/0 (socket says 172.21.15.42:43580) 2024-09-17T00:00:18.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeeed9d700 1 -- 172.21.15.42:0/799768931 learned_addr learned my addr 172.21.15.42:0/799768931 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeee59c700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef0105a20 0x7efef0197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeeed9d700 1 -- 172.21.15.42:0/799768931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef0105a20 msgr2=0x7efef0197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeeed9d700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef0105a20 0x7efef0197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeeed9d700 1 -- 172.21.15.42:0/799768931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee0009d00 con 0x7efef00ffb60 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efeeed9d700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef00ffb60 0x7efef0197560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efee000b070 tx=0x7efee000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efed7fff700 1 -- 172.21.15.42:0/799768931 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efee000e040 con 0x7efef00ffb60 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef01a05d0 con 0x7efef00ffb60 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efed7fff700 1 -- 172.21.15.42:0/799768931 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efee0003e60 con 0x7efef00ffb60 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efed7fff700 1 -- 172.21.15.42:0/799768931 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efee001d910 con 0x7efef00ffb60 2024-09-17T00:00:18.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.111+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef01a09e0 con 0x7efef00ffb60 2024-09-17T00:00:18.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.112+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0107fd0 con 0x7efef00ffb60 2024-09-17T00:00:18.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.113+0000 7efed7fff700 1 -- 172.21.15.42:0/799768931 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7efee0004820 con 0x7efef00ffb60 2024-09-17T00:00:18.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.113+0000 7efed7fff700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efed806e680 0x7efed8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.114+0000 7efed7fff700 1 -- 172.21.15.42:0/799768931 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efee008ffb0 con 0x7efef00ffb60 2024-09-17T00:00:18.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.114+0000 7efeee59c700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efed806e680 0x7efed8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.114+0000 7efeee59c700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efed806e680 0x7efed8070b40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efef0068e30 tx=0x7efee4005ed0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:18.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.118+0000 7efed7fff700 1 -- 172.21.15.42:0/799768931 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efee0059150 con 0x7efef00ffb60 2024-09-17T00:00:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:17 smithi043 ceph-mon[81510]: pgmap v261: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:17 smithi043 ceph-mon[81510]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:17 smithi043 ceph-mon[81510]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:17 smithi043 ceph-mon[81510]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:18.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.371+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efef004f2e0 con 0x7efef00ffb60 2024-09-17T00:00:18.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.371+0000 7efed7fff700 1 -- 172.21.15.42:0/799768931 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7efee005c810 con 0x7efef00ffb60 2024-09-17T00:00:18.370 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:00:18.371 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:00:18.371 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:00:18.371 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:00:18.371 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:00:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:18.371 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:00:18.372 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:00:18.372 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:00:18.372 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:00:18.372 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:00:18.372 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:00:18.372 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:00:18.372 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:00:18.373 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:00:18.374 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:00:18.374 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:00:18.374 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:00:18.374 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:00:18.374 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:00:18.374 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:00:18.374 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:18.376 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:00:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:18.376 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:18.376 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:18.376 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:18.376 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:18.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efed806e680 msgr2=0x7efed8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efed806e680 0x7efed8070b40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efef0068e30 tx=0x7efee4005ed0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef00ffb60 msgr2=0x7efef0197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef00ffb60 0x7efef0197560 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efee000b070 tx=0x7efee000cdd0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 shutdown_connections 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efed806e680 0x7efed8070b40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efef00ffb60 0x7efef0197560 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 --2- 172.21.15.42:0/799768931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef0105a20 0x7efef0197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.373+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 >> 172.21.15.42:0/799768931 conn(0x7efef00754a0 msgr2=0x7efef00feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:18.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.374+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 shutdown_connections 2024-09-17T00:00:18.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.374+0000 7efef50ab700 1 -- 172.21.15.42:0/799768931 wait complete. 2024-09-17T00:00:18.379 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:00:18.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.482+0000 7fb1046d2700 1 -- 172.21.15.42:0/2618472024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0fff50 msgr2=0x7fb0fc105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.482+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2618472024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0fff50 0x7fb0fc105840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009910 tx=0x7fb0ec009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:18.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 -- 172.21.15.42:0/2618472024 shutdown_connections 2024-09-17T00:00:18.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2618472024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0fff50 0x7fb0fc105840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2618472024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0fc0ff5a0 0x7fb0fc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 -- 172.21.15.42:0/2618472024 >> 172.21.15.42:0/2618472024 conn(0x7fb0fc074e30 msgr2=0x7fb0fc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:18.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 -- 172.21.15.42:0/2618472024 shutdown_connections 2024-09-17T00:00:18.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 -- 172.21.15.42:0/2618472024 wait complete. 2024-09-17T00:00:18.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 Processor -- start 2024-09-17T00:00:18.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.483+0000 7fb1046d2700 1 -- start start 2024-09-17T00:00:18.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb1046d2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0ff5a0 0x7fb0fc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb1046d2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0fc0fff50 0x7fb0fc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb1046d2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc19d210 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb1046d2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1966c0 con 0x7fb0fc0fff50 2024-09-17T00:00:18.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb10246e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0ff5a0 0x7fb0fc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb10246e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0ff5a0 0x7fb0fc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43594/0 (socket says 172.21.15.42:43594) 2024-09-17T00:00:18.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb10246e700 1 -- 172.21.15.42:0/2332993622 learned_addr learned my addr 172.21.15.42:0/2332993622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:18.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb101c6d700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0fc0fff50 0x7fb0fc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb10246e700 1 -- 172.21.15.42:0/2332993622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0fc0fff50 msgr2=0x7fb0fc19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb10246e700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0fc0fff50 0x7fb0fc19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb10246e700 1 -- 172.21.15.42:0/2332993622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0095f0 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb10246e700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0ff5a0 0x7fb0fc19c640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c8a0 tx=0x7fb0f800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:18.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb0f37fe700 1 -- 172.21.15.42:0/2332993622 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f80078e0 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc1969a0 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.484+0000 7fb0f37fe700 1 -- 172.21.15.42:0/2332993622 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0f800f450 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.485+0000 7fb0f37fe700 1 -- 172.21.15.42:0/2332993622 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f800e5e0 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.485+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc196ef0 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.485+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc109200 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.486+0000 7fb0f37fe700 1 -- 172.21.15.42:0/2332993622 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb0f800f5c0 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.487+0000 7fb0f37fe700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0e806e680 0x7fb0e8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.487+0000 7fb0f37fe700 1 -- 172.21.15.42:0/2332993622 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb0f808e690 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.487+0000 7fb101c6d700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0e806e680 0x7fb0e8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.487+0000 7fb101c6d700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0e806e680 0x7fb0e8070b40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc197b30 tx=0x7fb0ec01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:18.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.491+0000 7fb0f37fe700 1 -- 172.21.15.42:0/2332993622 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb0f805b360 con 0x7fb0fc0ff5a0 2024-09-17T00:00:18.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.721+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0fc197170 con 0x7fb0e806e680 2024-09-17T00:00:18.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.722+0000 7fb0f37fe700 1 -- 172.21.15.42:0/2332993622 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb0fc197170 con 0x7fb0e806e680 2024-09-17T00:00:18.721 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:00:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:00:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:00:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:00:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:00:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:00:18.722 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0e806e680 msgr2=0x7fb0e8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0e806e680 0x7fb0e8070b40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc197b30 tx=0x7fb0ec01a040 comp rx=0 tx=0).stop 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0ff5a0 msgr2=0x7fb0fc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0ff5a0 0x7fb0fc19c640 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c8a0 tx=0x7fb0f800cbb0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 shutdown_connections 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0e806e680 0x7fb0e8070b40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0fc0ff5a0 0x7fb0fc19c640 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 --2- 172.21.15.42:0/2332993622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0fc0fff50 0x7fb0fc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 >> 172.21.15.42:0/2332993622 conn(0x7fb0fc074e30 msgr2=0x7fb0fc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:18.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 shutdown_connections 2024-09-17T00:00:18.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.725+0000 7fb1046d2700 1 -- 172.21.15.42:0/2332993622 wait complete. 2024-09-17T00:00:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.833+0000 7febe86e6700 1 -- 172.21.15.42:0/355638594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0102780 msgr2=0x7febe010ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.833+0000 7febe86e6700 1 --2- 172.21.15.42:0/355638594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0102780 0x7febe010ac70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febd0009910 tx=0x7febd0009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 -- 172.21.15.42:0/355638594 shutdown_connections 2024-09-17T00:00:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 --2- 172.21.15.42:0/355638594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0102780 0x7febe010ac70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 --2- 172.21.15.42:0/355638594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febe0101e60 0x7febe0102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 -- 172.21.15.42:0/355638594 >> 172.21.15.42:0/355638594 conn(0x7febe00fb650 msgr2=0x7febe00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 -- 172.21.15.42:0/355638594 shutdown_connections 2024-09-17T00:00:18.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 -- 172.21.15.42:0/355638594 wait complete. 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 Processor -- start 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.834+0000 7febe86e6700 1 -- start start 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe86e6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0101e60 0x7febe019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe86e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febe0102780 0x7febe019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe86e6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe019d210 con 0x7febe0101e60 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe86e6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe01966c0 con 0x7febe0102780 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe6482700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0101e60 0x7febe019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe6482700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0101e60 0x7febe019c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43616/0 (socket says 172.21.15.42:43616) 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe6482700 1 -- 172.21.15.42:0/1869871311 learned_addr learned my addr 172.21.15.42:0/1869871311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe5c81700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febe0102780 0x7febe019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe6482700 1 -- 172.21.15.42:0/1869871311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febe0102780 msgr2=0x7febe019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe6482700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febe0102780 0x7febe019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe6482700 1 -- 172.21.15.42:0/1869871311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00095f0 con 0x7febe0101e60 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe6482700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0101e60 0x7febe019c640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c8a0 tx=0x7febdc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febd77fe700 1 -- 172.21.15.42:0/1869871311 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febdc0078e0 con 0x7febe0101e60 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febd77fe700 1 -- 172.21.15.42:0/1869871311 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febdc00f450 con 0x7febe0101e60 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe01969a0 con 0x7febe0101e60 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.835+0000 7febd77fe700 1 -- 172.21.15.42:0/1869871311 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febdc00e5e0 con 0x7febe0101e60 2024-09-17T00:00:18.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.836+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe0196ef0 con 0x7febe0101e60 2024-09-17T00:00:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.836+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe0108300 con 0x7febe0101e60 2024-09-17T00:00:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.837+0000 7febd77fe700 1 -- 172.21.15.42:0/1869871311 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7febdc00f5c0 con 0x7febe0101e60 2024-09-17T00:00:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.837+0000 7febd77fe700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7febcc06e680 0x7febcc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.838+0000 7febd77fe700 1 -- 172.21.15.42:0/1869871311 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7febdc08e660 con 0x7febe0101e60 2024-09-17T00:00:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.838+0000 7febe5c81700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7febcc06e680 0x7febcc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:18.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.838+0000 7febe5c81700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7febcc06e680 0x7febcc070b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febe0197b30 tx=0x7febd001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:18.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:18.842+0000 7febd77fe700 1 -- 172.21.15.42:0/1869871311 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7febdc05b330 con 0x7febe0101e60 2024-09-17T00:00:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:18 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1882375305' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:00:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:18 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/799768931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:00:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:18 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1882375305' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:00:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:18 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/799768931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:00:19.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.137+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7febe004f2e0 con 0x7febe0101e60 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.137+0000 7febd77fe700 1 -- 172.21.15.42:0/1869871311 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7febdc05aec0 con 0x7febe0101e60 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:00:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:00:19.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7febcc06e680 msgr2=0x7febcc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:19.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7febcc06e680 0x7febcc070b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febe0197b30 tx=0x7febd001a040 comp rx=0 tx=0).stop 2024-09-17T00:00:19.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0101e60 msgr2=0x7febe019c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:19.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0101e60 0x7febe019c640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c8a0 tx=0x7febdc00cbb0 comp rx=0 tx=0).stop 2024-09-17T00:00:19.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 shutdown_connections 2024-09-17T00:00:19.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7febcc06e680 0x7febcc070b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:19.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febe0101e60 0x7febe019c640 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:19.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 --2- 172.21.15.42:0/1869871311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febe0102780 0x7febe019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:19.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.139+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 >> 172.21.15.42:0/1869871311 conn(0x7febe00fb650 msgr2=0x7febe0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:19.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.140+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 shutdown_connections 2024-09-17T00:00:19.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:19.140+0000 7febe86e6700 1 -- 172.21.15.42:0/1869871311 wait complete. 2024-09-17T00:00:20.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:19 smithi042 ceph-mon[111056]: pgmap v262: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:20.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:19 smithi042 ceph-mon[111056]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:20.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:19 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1869871311' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:00:20.094 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:19 smithi043 ceph-mon[81510]: pgmap v262: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:20.094 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:19 smithi043 ceph-mon[81510]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:20.096 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:19 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1869871311' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:00:21.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:21.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:21 smithi042 ceph-mon[111056]: pgmap v263: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:21 smithi043 ceph-mon[81510]: pgmap v263: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:23 smithi042 ceph-mon[111056]: pgmap v264: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:23 smithi043 ceph-mon[81510]: pgmap v264: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:26.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:25 smithi042 ceph-mon[111056]: pgmap v265: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:26.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:25 smithi043 ceph-mon[81510]: pgmap v265: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:28.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:27 smithi042 ceph-mon[111056]: pgmap v266: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:28.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:27 smithi043 ceph-mon[81510]: pgmap v266: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:30.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:29 smithi042 ceph-mon[111056]: pgmap v267: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:30.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:29 smithi043 ceph-mon[81510]: pgmap v267: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:32.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:31 smithi042 ceph-mon[111056]: pgmap v268: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:32.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:31 smithi043 ceph-mon[81510]: pgmap v268: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:34.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:33 smithi042 ceph-mon[111056]: pgmap v269: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:34.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:33 smithi043 ceph-mon[81510]: pgmap v269: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:36.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:35 smithi042 ceph-mon[111056]: pgmap v270: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:36.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:35 smithi043 ceph-mon[81510]: pgmap v270: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:38.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:37 smithi043 ceph-mon[81510]: pgmap v271: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:38.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:37 smithi042 ceph-mon[111056]: pgmap v271: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:40.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:39 smithi043 ceph-mon[81510]: pgmap v272: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:40.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:39 smithi042 ceph-mon[111056]: pgmap v272: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:42.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:41 smithi042 ceph-mon[111056]: pgmap v273: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:42.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:41 smithi043 ceph-mon[81510]: pgmap v273: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:44.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:43 smithi043 ceph-mon[81510]: pgmap v274: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:44.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:43 smithi042 ceph-mon[111056]: pgmap v274: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:46.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:45 smithi043 ceph-mon[81510]: pgmap v275: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:46.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:45 smithi042 ceph-mon[111056]: pgmap v275: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:48.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:47 smithi043 ceph-mon[81510]: pgmap v276: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:48.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:47 smithi042 ceph-mon[111056]: pgmap v276: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:49.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.264+0000 7f132d9a8700 1 -- 172.21.15.42:0/2768385530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 msgr2=0x7f1328101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.264+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2768385530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f1328101260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1318009910 tx=0x7f1318009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.264+0000 7f132d9a8700 1 -- 172.21.15.42:0/2768385530 shutdown_connections 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.264+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2768385530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328101830 0x7f1328105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.264+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2768385530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f1328101260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.264+0000 7f132d9a8700 1 -- 172.21.15.42:0/2768385530 >> 172.21.15.42:0/2768385530 conn(0x7f13280fc710 msgr2=0x7f13280feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.265+0000 7f132d9a8700 1 -- 172.21.15.42:0/2768385530 shutdown_connections 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.265+0000 7f132d9a8700 1 -- 172.21.15.42:0/2768385530 wait complete. 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.265+0000 7f132d9a8700 1 Processor -- start 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.265+0000 7f132d9a8700 1 -- start start 2024-09-17T00:00:49.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f132d9a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f132819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f132d9a8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328101830 0x7f132819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f132d9a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132819d220 con 0x7f1328100e80 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f132d9a8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13281966d0 con 0x7f1328101830 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f132819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f1326ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f132819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56980/0 (socket says 172.21.15.42:56980) 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f1326ffd700 1 -- 172.21.15.42:0/2863232520 learned_addr learned my addr 172.21.15.42:0/2863232520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f13267fc700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328101830 0x7f132819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f1326ffd700 1 -- 172.21.15.42:0/2863232520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328101830 msgr2=0x7f132819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f1326ffd700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328101830 0x7f132819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f1326ffd700 1 -- 172.21.15.42:0/2863232520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180095f0 con 0x7f1328100e80 2024-09-17T00:00:49.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f1326ffd700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f132819c650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1318006010 tx=0x7f131800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.266+0000 7f132c9a6700 1 -- 172.21.15.42:0/2863232520 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f131801d070 con 0x7f1328100e80 2024-09-17T00:00:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.267+0000 7f132c9a6700 1 -- 172.21.15.42:0/2863232520 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f131800f460 con 0x7f1328100e80 2024-09-17T00:00:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.267+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1328196950 con 0x7f1328100e80 2024-09-17T00:00:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.267+0000 7f132c9a6700 1 -- 172.21.15.42:0/2863232520 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1318017610 con 0x7f1328100e80 2024-09-17T00:00:49.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.267+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1328196e40 con 0x7f1328100e80 2024-09-17T00:00:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.268+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1328109220 con 0x7f1328100e80 2024-09-17T00:00:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.268+0000 7f132c9a6700 1 -- 172.21.15.42:0/2863232520 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f131800f5d0 con 0x7f1328100e80 2024-09-17T00:00:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.269+0000 7f132c9a6700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f131006e680 0x7f1310070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.269+0000 7f132c9a6700 1 -- 172.21.15.42:0/2863232520 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f131808fc10 con 0x7f1328100e80 2024-09-17T00:00:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.269+0000 7f13267fc700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f131006e680 0x7f1310070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.269+0000 7f13267fc700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f131006e680 0x7f1310070b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1328197b40 tx=0x7f131c00a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:49.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.273+0000 7f132c9a6700 1 -- 172.21.15.42:0/2863232520 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f131805c8e0 con 0x7f1328100e80 2024-09-17T00:00:49.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.506+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1328197cc0 con 0x7f131006e680 2024-09-17T00:00:49.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.507+0000 7f132c9a6700 1 -- 172.21.15.42:0/2863232520 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1328197cc0 con 0x7f131006e680 2024-09-17T00:00:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f131006e680 msgr2=0x7f1310070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f131006e680 0x7f1310070b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1328197b40 tx=0x7f131c00a300 comp rx=0 tx=0).stop 2024-09-17T00:00:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 msgr2=0x7f132819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f132819c650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1318006010 tx=0x7f131800bd70 comp rx=0 tx=0).stop 2024-09-17T00:00:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 shutdown_connections 2024-09-17T00:00:49.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f131006e680 0x7f1310070b40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1328100e80 0x7f132819c650 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 --2- 172.21.15.42:0/2863232520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1328101830 0x7f132819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.509+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 >> 172.21.15.42:0/2863232520 conn(0x7f13280fc710 msgr2=0x7f13280fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.510+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 shutdown_connections 2024-09-17T00:00:49.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.510+0000 7f132d9a8700 1 -- 172.21.15.42:0/2863232520 wait complete. 2024-09-17T00:00:49.518 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:00:49.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 -- 172.21.15.42:0/1647406586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 msgr2=0x7faaec10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/1647406586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec10ac10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faae0009910 tx=0x7faae0009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:49.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 -- 172.21.15.42:0/1647406586 shutdown_connections 2024-09-17T00:00:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/1647406586 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec10ac10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/1647406586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaec101e00 0x7faaec1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 -- 172.21.15.42:0/1647406586 >> 172.21.15.42:0/1647406586 conn(0x7faaec0fb650 msgr2=0x7faaec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 -- 172.21.15.42:0/1647406586 shutdown_connections 2024-09-17T00:00:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.619+0000 7faaf1f9d700 1 -- 172.21.15.42:0/1647406586 wait complete. 2024-09-17T00:00:49.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.620+0000 7faaf1f9d700 1 Processor -- start 2024-09-17T00:00:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.620+0000 7faaf1f9d700 1 -- start start 2024-09-17T00:00:49.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.620+0000 7faaf1f9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaec101e00 0x7faaec0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.620+0000 7faaf1f9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.620+0000 7faaf1f9d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec1004e0 con 0x7faaec102720 2024-09-17T00:00:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.620+0000 7faaf1f9d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec100620 con 0x7faaec101e00 2024-09-17T00:00:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec0fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56998/0 (socket says 172.21.15.42:56998) 2024-09-17T00:00:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeaffd700 1 -- 172.21.15.42:0/4008643062 learned_addr learned my addr 172.21.15.42:0/4008643062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:49.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeb7fe700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaec101e00 0x7faaec0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeaffd700 1 -- 172.21.15.42:0/4008643062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaec101e00 msgr2=0x7faaec0ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeaffd700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaec101e00 0x7faaec0ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeaffd700 1 -- 172.21.15.42:0/4008643062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae00095f0 con 0x7faaec102720 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaeaffd700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec0fffa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faae00059d0 tx=0x7faae000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faae8ff9700 1 -- 172.21.15.42:0/4008643062 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae001d070 con 0x7faaec102720 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faae8ff9700 1 -- 172.21.15.42:0/4008643062 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faae000b610 con 0x7faaec102720 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faae8ff9700 1 -- 172.21.15.42:0/4008643062 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faae0022470 con 0x7faaec102720 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec1008a0 con 0x7faaec102720 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.621+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec1a2dc0 con 0x7faaec102720 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.622+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec108310 con 0x7faaec102720 2024-09-17T00:00:49.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.623+0000 7faae8ff9700 1 -- 172.21.15.42:0/4008643062 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7faae00050d0 con 0x7faaec102720 2024-09-17T00:00:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.623+0000 7faae8ff9700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faad406e680 0x7faad4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.624+0000 7faae8ff9700 1 -- 172.21.15.42:0/4008643062 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faae008fab0 con 0x7faaec102720 2024-09-17T00:00:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.624+0000 7faaeb7fe700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faad406e680 0x7faad4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.624+0000 7faaeb7fe700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faad406e680 0x7faad4070b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faadc009bf0 tx=0x7faadc009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:49.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.628+0000 7faae8ff9700 1 -- 172.21.15.42:0/4008643062 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faae0058cd0 con 0x7faaec102720 2024-09-17T00:00:49.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.860+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faaec0611d0 con 0x7faad406e680 2024-09-17T00:00:49.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.862+0000 7faae8ff9700 1 -- 172.21.15.42:0/4008643062 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faaec0611d0 con 0x7faad406e680 2024-09-17T00:00:49.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faad406e680 msgr2=0x7faad4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faad406e680 0x7faad4070b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faadc009bf0 tx=0x7faadc009260 comp rx=0 tx=0).stop 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 msgr2=0x7faaec0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec0fffa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faae00059d0 tx=0x7faae000f740 comp rx=0 tx=0).stop 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 shutdown_connections 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faad406e680 0x7faad4070b40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faaec101e00 0x7faaec0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 --2- 172.21.15.42:0/4008643062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faaec102720 0x7faaec0fffa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 >> 172.21.15.42:0/4008643062 conn(0x7faaec0fb650 msgr2=0x7faaec105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:49.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 shutdown_connections 2024-09-17T00:00:49.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.864+0000 7faaf1f9d700 1 -- 172.21.15.42:0/4008643062 wait complete. 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3807760747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8101850 msgr2=0x7f23d81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3807760747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8101850 0x7f23d81058a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009910 tx=0x7f23cc009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3807760747 shutdown_connections 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3807760747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8101850 0x7f23d81058a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3807760747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d8100ea0 0x7f23d8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3807760747 >> 172.21.15.42:0/3807760747 conn(0x7f23d80fc750 msgr2=0x7f23d80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3807760747 shutdown_connections 2024-09-17T00:00:49.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.972+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3807760747 wait complete. 2024-09-17T00:00:49.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23dd9a3700 1 Processor -- start 2024-09-17T00:00:49.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23dd9a3700 1 -- start start 2024-09-17T00:00:49.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23dd9a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8100ea0 0x7f23d819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23dd9a3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d8101850 0x7f23d819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23dd9a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d819f520 con 0x7f23d8100ea0 2024-09-17T00:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23dd9a3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81989d0 con 0x7f23d8101850 2024-09-17T00:00:49.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8100ea0 0x7f23d819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8100ea0 0x7f23d819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57024/0 (socket says 172.21.15.42:57024) 2024-09-17T00:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23d6ffd700 1 -- 172.21.15.42:0/3890637866 learned_addr learned my addr 172.21.15.42:0/3890637866 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.973+0000 7f23d67fc700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d8101850 0x7f23d819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23d6ffd700 1 -- 172.21.15.42:0/3890637866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d8101850 msgr2=0x7f23d819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23d6ffd700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d8101850 0x7f23d819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:49.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23d6ffd700 1 -- 172.21.15.42:0/3890637866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0095f0 con 0x7f23d8100ea0 2024-09-17T00:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23d6ffd700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8100ea0 0x7f23d819e950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b4b0 tx=0x7f23c800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23dc9a1700 1 -- 172.21.15.42:0/3890637866 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c8011800 con 0x7f23d8100ea0 2024-09-17T00:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23dc9a1700 1 -- 172.21.15.42:0/3890637866 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23c8011e20 con 0x7f23d8100ea0 2024-09-17T00:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d8198cb0 con 0x7f23d8100ea0 2024-09-17T00:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23dc9a1700 1 -- 172.21.15.42:0/3890637866 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23c800f2e0 con 0x7f23d8100ea0 2024-09-17T00:00:49.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.974+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d81991d0 con 0x7f23d8100ea0 2024-09-17T00:00:49.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.975+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d8105020 con 0x7f23d8100ea0 2024-09-17T00:00:49.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.976+0000 7f23dc9a1700 1 -- 172.21.15.42:0/3890637866 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f23c8010380 con 0x7f23d8100ea0 2024-09-17T00:00:49.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.976+0000 7f23dc9a1700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f23c006e680 0x7f23c0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.976+0000 7f23d67fc700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f23c006e680 0x7f23c0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.976+0000 7f23dc9a1700 1 -- 172.21.15.42:0/3890637866 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f23c808e1b0 con 0x7f23d8100ea0 2024-09-17T00:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.977+0000 7f23d67fc700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f23c006e680 0x7f23c0070b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23cc006010 tx=0x7f23cc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:49.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:49.981+0000 7f23dc9a1700 1 -- 172.21.15.42:0/3890637866 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f23c805ae80 con 0x7f23d8100ea0 2024-09-17T00:00:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:49 smithi043 ceph-mon[81510]: pgmap v277: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:50.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:49 smithi043 ceph-mon[81510]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:50.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.199+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f23d8199450 con 0x7f23c006e680 2024-09-17T00:00:50.197 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:49 smithi042 ceph-mon[111056]: pgmap v277: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:50.197 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:49 smithi042 ceph-mon[111056]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:50.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.209+0000 7f23dc9a1700 1 -- 172.21.15.42:0/3890637866 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f23d8199450 con 0x7f23c006e680 2024-09-17T00:00:50.207 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:00:50.207 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (9m) 3m ago 20m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:00:50.207 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (20m) 3m ago 20m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:00:50.207 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (19m) 2m ago 19m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (7m) 3m ago 20m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (7m) 2m ago 19m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (9m) 3m ago 20m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (3m) 3m ago 14m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (3m) 3m ago 14m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (2m) 2m ago 14m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (2m) 2m ago 14m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (11m) 3m ago 21m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (10m) 2m ago 19m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (8m) 3m ago 21m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (7m) 2m ago 19m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (10m) 3m ago 20m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:00:50.208 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (10m) 2m ago 19m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:00:50.209 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 3m ago 18m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:00:50.212 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (6m) 3m ago 17m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:00:50.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (5m) 3m ago 17m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:00:50.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (4m) 2m ago 17m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:00:50.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (4m) 2m ago 16m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:00:50.213 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (3m) 2m ago 16m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:00:50.214 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (9m) 3m ago 19m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:00:50.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f23c006e680 msgr2=0x7f23c0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f23c006e680 0x7f23c0070b40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f23cc006010 tx=0x7f23cc01a040 comp rx=0 tx=0).stop 2024-09-17T00:00:50.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8100ea0 msgr2=0x7f23d819e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8100ea0 0x7f23d819e950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f23c800b4b0 tx=0x7f23c800b7c0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 shutdown_connections 2024-09-17T00:00:50.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f23c006e680 0x7f23c0070b40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f23d8100ea0 0x7f23d819e950 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 --2- 172.21.15.42:0/3890637866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d8101850 0x7f23d819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 >> 172.21.15.42:0/3890637866 conn(0x7f23d80fc750 msgr2=0x7f23d80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 shutdown_connections 2024-09-17T00:00:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.212+0000 7f23dd9a3700 1 -- 172.21.15.42:0/3890637866 wait complete. 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 -- 172.21.15.42:0/1226753884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 msgr2=0x7f883c100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 --2- 172.21.15.42:0/1226753884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c100fd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8830009910 tx=0x7f8830009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 -- 172.21.15.42:0/1226753884 shutdown_connections 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 --2- 172.21.15.42:0/1226753884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c100fd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 --2- 172.21.15.42:0/1226753884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f883c106c00 0x7f883c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 -- 172.21.15.42:0/1226753884 >> 172.21.15.42:0/1226753884 conn(0x7f883c0fc640 msgr2=0x7f883c0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 -- 172.21.15.42:0/1226753884 shutdown_connections 2024-09-17T00:00:50.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.319+0000 7f8841009700 1 -- 172.21.15.42:0/1226753884 wait complete. 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f8841009700 1 Processor -- start 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f8841009700 1 -- start start 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f8841009700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f8841009700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f883c106c00 0x7f883c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f8841009700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c197290 con 0x7f883c100b50 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f8841009700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c19b020 con 0x7f883c106c00 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f883ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f883ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57044/0 (socket says 172.21.15.42:57044) 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f883ad9d700 1 -- 172.21.15.42:0/3804954789 learned_addr learned my addr 172.21.15.42:0/3804954789 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:50.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.320+0000 7f883a59c700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f883c106c00 0x7f883c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f883ad9d700 1 -- 172.21.15.42:0/3804954789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f883c106c00 msgr2=0x7f883c196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f883ad9d700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f883c106c00 0x7f883c196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f883ad9d700 1 -- 172.21.15.42:0/3804954789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300095f0 con 0x7f883c100b50 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f883ad9d700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c196670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c8a0 tx=0x7f882c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f8823fff700 1 -- 172.21.15.42:0/3804954789 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c0078e0 con 0x7f883c100b50 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c19b300 con 0x7f883c100b50 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c19b8d0 con 0x7f883c100b50 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f8823fff700 1 -- 172.21.15.42:0/3804954789 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f882c00f450 con 0x7f883c100b50 2024-09-17T00:00:50.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.321+0000 7f8823fff700 1 -- 172.21.15.42:0/3804954789 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c00e5e0 con 0x7f883c100b50 2024-09-17T00:00:50.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.322+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8828005320 con 0x7f883c100b50 2024-09-17T00:00:50.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.323+0000 7f8823fff700 1 -- 172.21.15.42:0/3804954789 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f882c00f5c0 con 0x7f883c100b50 2024-09-17T00:00:50.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.323+0000 7f8823fff700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f882406e680 0x7f8824070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:50.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.323+0000 7f8823fff700 1 -- 172.21.15.42:0/3804954789 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f882c08e800 con 0x7f883c100b50 2024-09-17T00:00:50.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.323+0000 7f883a59c700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f882406e680 0x7f8824070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:50.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.324+0000 7f883a59c700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f882406e680 0x7f8824070b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f883c197c90 tx=0x7f883001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:50.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.327+0000 7f8823fff700 1 -- 172.21.15.42:0/3804954789 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f882c057a00 con 0x7f883c100b50 2024-09-17T00:00:50.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.622+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8828005cc0 con 0x7f883c100b50 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.622+0000 7f8823fff700 1 -- 172.21.15.42:0/3804954789 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f882c05b0c0 con 0x7f883c100b50 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:00:50.620 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:00:50.621 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f882406e680 msgr2=0x7f8824070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f882406e680 0x7f8824070b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f883c197c90 tx=0x7f883001a040 comp rx=0 tx=0).stop 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 msgr2=0x7f883c196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c196670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c8a0 tx=0x7f882c00cbb0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 shutdown_connections 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f882406e680 0x7f8824070b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f883c100b50 0x7f883c196670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 --2- 172.21.15.42:0/3804954789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f883c106c00 0x7f883c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 >> 172.21.15.42:0/3804954789 conn(0x7f883c0fc640 msgr2=0x7f883c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:50.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 shutdown_connections 2024-09-17T00:00:50.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.624+0000 7f8841009700 1 -- 172.21.15.42:0/3804954789 wait complete. 2024-09-17T00:00:50.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1120839233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40ff5a0 msgr2=0x7fb9b40ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1120839233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40ff5a0 0x7fb9b40ff980 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b0a0 tx=0x7fb9a400b3b0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1120839233 shutdown_connections 2024-09-17T00:00:50.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1120839233 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b40fff50 0x7fb9b4105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1120839233 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40ff5a0 0x7fb9b40ff980 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1120839233 >> 172.21.15.42:0/1120839233 conn(0x7fb9b4074e30 msgr2=0x7fb9b4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:50.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1120839233 shutdown_connections 2024-09-17T00:00:50.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.734+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1120839233 wait complete. 2024-09-17T00:00:50.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9bafeb700 1 Processor -- start 2024-09-17T00:00:50.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9bafeb700 1 -- start start 2024-09-17T00:00:50.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9bafeb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b40ff5a0 0x7fb9b419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:50.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9bafeb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40fff50 0x7fb9b419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:50.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9bafeb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b419d240 con 0x7fb9b40fff50 2024-09-17T00:00:50.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9bafeb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b4196780 con 0x7fb9b40ff5a0 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9b3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40fff50 0x7fb9b419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9b3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40fff50 0x7fb9b419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57072/0 (socket says 172.21.15.42:57072) 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9b3fff700 1 -- 172.21.15.42:0/1005323801 learned_addr learned my addr 172.21.15.42:0/1005323801 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9b8d87700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b40ff5a0 0x7fb9b419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9b3fff700 1 -- 172.21.15.42:0/1005323801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b40ff5a0 msgr2=0x7fb9b419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9b3fff700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b40ff5a0 0x7fb9b419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.735+0000 7fb9b3fff700 1 -- 172.21.15.42:0/1005323801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4009d00 con 0x7fb9b40fff50 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.736+0000 7fb9b3fff700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40fff50 0x7fb9b419cc40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c960 tx=0x7fb9a800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.736+0000 7fb9b1ffb700 1 -- 172.21.15.42:0/1005323801 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a8007a70 con 0x7fb9b40fff50 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.736+0000 7fb9b1ffb700 1 -- 172.21.15.42:0/1005323801 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9a8007bd0 con 0x7fb9b40fff50 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.736+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b4196a60 con 0x7fb9b40fff50 2024-09-17T00:00:50.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.736+0000 7fb9b1ffb700 1 -- 172.21.15.42:0/1005323801 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a8018630 con 0x7fb9b40fff50 2024-09-17T00:00:50.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.736+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b4196fb0 con 0x7fb9b40fff50 2024-09-17T00:00:50.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.737+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b4109270 con 0x7fb9b40fff50 2024-09-17T00:00:50.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.737+0000 7fb9b1ffb700 1 -- 172.21.15.42:0/1005323801 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb9a800f450 con 0x7fb9b40fff50 2024-09-17T00:00:50.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.738+0000 7fb9b1ffb700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb99c06e680 0x7fb99c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:50.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.738+0000 7fb9b1ffb700 1 -- 172.21.15.42:0/1005323801 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb9a808e540 con 0x7fb9b40fff50 2024-09-17T00:00:50.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.738+0000 7fb9b8d87700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb99c06e680 0x7fb99c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:50.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.738+0000 7fb9b8d87700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb99c06e680 0x7fb99c070b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4000f80 tx=0x7fb9a40060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:50.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:50.742+0000 7fb9b1ffb700 1 -- 172.21.15.42:0/1005323801 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb9a805b340 con 0x7fb9b40fff50 2024-09-17T00:00:51.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.002+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb9b404f2e0 con 0x7fb9b40fff50 2024-09-17T00:00:51.000 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:50 smithi042 ceph-mon[111056]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:51.000 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:50 smithi042 ceph-mon[111056]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:51.000 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:50 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3804954789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:00:51.000 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.002+0000 7fb9b1ffb700 1 -- 172.21.15.42:0/1005323801 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fb9a805aed0 con 0x7fb9b40fff50 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:51.001 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:00:51.002 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:00:51.003 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:51.004 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:00:51.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb99c06e680 msgr2=0x7fb99c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb99c06e680 0x7fb99c070b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4000f80 tx=0x7fb9a40060c0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40fff50 msgr2=0x7fb9b419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40fff50 0x7fb9b419cc40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c960 tx=0x7fb9a800cc70 comp rx=0 tx=0).stop 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 shutdown_connections 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb99c06e680 0x7fb99c070b40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b40ff5a0 0x7fb9b419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 --2- 172.21.15.42:0/1005323801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9b40fff50 0x7fb9b419cc40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 >> 172.21.15.42:0/1005323801 conn(0x7fb9b4074e30 msgr2=0x7fb9b40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 shutdown_connections 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.006+0000 7fb9bafeb700 1 -- 172.21.15.42:0/1005323801 wait complete. 2024-09-17T00:00:51.006 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 -- 172.21.15.42:0/3811871468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 msgr2=0x7f97541022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 --2- 172.21.15.42:0/3811871468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f97541022a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9748009940 tx=0x7f9748009c50 comp rx=0 tx=0).stop 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 -- 172.21.15.42:0/3811871468 shutdown_connections 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 --2- 172.21.15.42:0/3811871468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97541027e0 0x7f975410acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 --2- 172.21.15.42:0/3811871468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f97541022a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 -- 172.21.15.42:0/3811871468 >> 172.21.15.42:0/3811871468 conn(0x7f97540fb650 msgr2=0x7f97540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 -- 172.21.15.42:0/3811871468 shutdown_connections 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 -- 172.21.15.42:0/3811871468 wait complete. 2024-09-17T00:00:51.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 Processor -- start 2024-09-17T00:00:51.116 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.117+0000 7f975b3c7700 1 -- start start 2024-09-17T00:00:51.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f975b3c7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f975419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:51.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f975b3c7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97541027e0 0x7f975419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:51.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f975b3c7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975419d240 con 0x7f9754101ec0 2024-09-17T00:00:51.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f975b3c7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97541966f0 con 0x7f97541027e0 2024-09-17T00:00:51.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9759163700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f975419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9759163700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f975419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57098/0 (socket says 172.21.15.42:57098) 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9759163700 1 -- 172.21.15.42:0/2551513529 learned_addr learned my addr 172.21.15.42:0/2551513529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9758962700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97541027e0 0x7f975419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9759163700 1 -- 172.21.15.42:0/2551513529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97541027e0 msgr2=0x7f975419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9759163700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97541027e0 0x7f975419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9759163700 1 -- 172.21.15.42:0/2551513529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480095f0 con 0x7f9754101ec0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9758962700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97541027e0 0x7f975419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f9759163700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f975419c670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97480055f0 tx=0x7f97480056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f97467fc700 1 -- 172.21.15.42:0/2551513529 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974801d070 con 0x7f9754101ec0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.118+0000 7f97467fc700 1 -- 172.21.15.42:0/2551513529 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f974800f460 con 0x7f9754101ec0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.119+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9754196970 con 0x7f9754101ec0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.119+0000 7f97467fc700 1 -- 172.21.15.42:0/2551513529 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9748021620 con 0x7f9754101ec0 2024-09-17T00:00:51.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.119+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9754196e60 con 0x7f9754101ec0 2024-09-17T00:00:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.119+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9754108370 con 0x7f9754101ec0 2024-09-17T00:00:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.120+0000 7f97467fc700 1 -- 172.21.15.42:0/2551513529 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f974800fa60 con 0x7f9754101ec0 2024-09-17T00:00:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.121+0000 7f97467fc700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f974006e680 0x7f9740070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.121+0000 7f97467fc700 1 -- 172.21.15.42:0/2551513529 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f974808f9d0 con 0x7f9754101ec0 2024-09-17T00:00:51.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.121+0000 7f9758962700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f974006e680 0x7f9740070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:51.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.121+0000 7f9758962700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f974006e680 0x7f9740070b40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005dc0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:51.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.125+0000 7f97467fc700 1 -- 172.21.15.42:0/2551513529 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f974805c6a0 con 0x7f9754101ec0 2024-09-17T00:00:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:50 smithi043 ceph-mon[81510]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:50 smithi043 ceph-mon[81510]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:50 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3804954789' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:00:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:00:51.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.355+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9754197d10 con 0x7f974006e680 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.356+0000 7f97467fc700 1 -- 172.21.15.42:0/2551513529 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9754197d10 con 0x7f974006e680 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:00:51.354 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:00:51.355 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:00:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.358+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f974006e680 msgr2=0x7f9740070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.358+0000 7f975b3c7700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f974006e680 0x7f9740070b40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005dc0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.358+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 msgr2=0x7f975419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.358+0000 7f975b3c7700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f975419c670 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97480055f0 tx=0x7f97480056d0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.359+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 shutdown_connections 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.359+0000 7f975b3c7700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f974006e680 0x7f9740070b40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.359+0000 7f975b3c7700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9754101ec0 0x7f975419c670 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.359+0000 7f975b3c7700 1 --2- 172.21.15.42:0/2551513529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97541027e0 0x7f975419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.359+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 >> 172.21.15.42:0/2551513529 conn(0x7f97540fb650 msgr2=0x7f97541054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.359+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 shutdown_connections 2024-09-17T00:00:51.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.359+0000 7f975b3c7700 1 -- 172.21.15.42:0/2551513529 wait complete. 2024-09-17T00:00:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 -- 172.21.15.42:0/1504514543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 msgr2=0x7fbe001058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 --2- 172.21.15.42:0/1504514543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe001058a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009910 tx=0x7fbde8009c20 comp rx=0 tx=0).stop 2024-09-17T00:00:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 -- 172.21.15.42:0/1504514543 shutdown_connections 2024-09-17T00:00:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 --2- 172.21.15.42:0/1504514543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe001058a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 --2- 172.21.15.42:0/1504514543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00100ea0 0x7fbe00101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 -- 172.21.15.42:0/1504514543 >> 172.21.15.42:0/1504514543 conn(0x7fbe000fc750 msgr2=0x7fbe000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 -- 172.21.15.42:0/1504514543 shutdown_connections 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.466+0000 7fbe05364700 1 -- 172.21.15.42:0/1504514543 wait complete. 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbe05364700 1 Processor -- start 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbe05364700 1 -- start start 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbe05364700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00100ea0 0x7fbe00073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbe05364700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbe05364700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00073b10 con 0x7fbe00101850 2024-09-17T00:00:51.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbe05364700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00073c50 con 0x7fbe00100ea0 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbdf7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe000735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57108/0 (socket says 172.21.15.42:57108) 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbdf7fff700 1 -- 172.21.15.42:0/1565587999 learned_addr learned my addr 172.21.15.42:0/1565587999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.467+0000 7fbdfeffd700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00100ea0 0x7fbe00073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdf7fff700 1 -- 172.21.15.42:0/1565587999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00100ea0 msgr2=0x7fbe00073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdf7fff700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00100ea0 0x7fbe00073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdf7fff700 1 -- 172.21.15.42:0/1565587999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80095f0 con 0x7fbe00101850 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdfeffd700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00100ea0 0x7fbe00073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdf7fff700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe000735d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbde800bda0 tx=0x7fbde800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdfcff9700 1 -- 172.21.15.42:0/1565587999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde801d070 con 0x7fbe00101850 2024-09-17T00:00:51.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdfcff9700 1 -- 172.21.15.42:0/1565587999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbde800f460 con 0x7fbe00101850 2024-09-17T00:00:51.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbdfcff9700 1 -- 172.21.15.42:0/1565587999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde8017630 con 0x7fbe00101850 2024-09-17T00:00:51.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe00073ed0 con 0x7fbe00101850 2024-09-17T00:00:51.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.468+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe001a2de0 con 0x7fbe00101850 2024-09-17T00:00:51.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.469+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe001092a0 con 0x7fbe00101850 2024-09-17T00:00:51.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.470+0000 7fbdfcff9700 1 -- 172.21.15.42:0/1565587999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fbde800fa40 con 0x7fbe00101850 2024-09-17T00:00:51.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.470+0000 7fbdfcff9700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbdec06e680 0x7fbdec070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:00:51.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.470+0000 7fbdfcff9700 1 -- 172.21.15.42:0/1565587999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbde808fba0 con 0x7fbe00101850 2024-09-17T00:00:51.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.470+0000 7fbdfeffd700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbdec06e680 0x7fbdec070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:00:51.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.471+0000 7fbdfeffd700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbdec06e680 0x7fbdec070b40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000bc10 tx=0x7fbdf000b5f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:00:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.474+0000 7fbdfcff9700 1 -- 172.21.15.42:0/1565587999 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbde8058ca0 con 0x7fbe00101850 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.768+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbe00074660 con 0x7fbe00101850 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.768+0000 7fbdfcff9700 1 -- 172.21.15.42:0/1565587999 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fbde805c360 con 0x7fbe00101850 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:00:51.766 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:00:51.767 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbdec06e680 msgr2=0x7fbdec070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbdec06e680 0x7fbdec070b40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000bc10 tx=0x7fbdf000b5f0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 msgr2=0x7fbe000735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe000735d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbde800bda0 tx=0x7fbde800be80 comp rx=0 tx=0).stop 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 shutdown_connections 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbdec06e680 0x7fbdec070b40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe00100ea0 0x7fbe00073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 --2- 172.21.15.42:0/1565587999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe00101850 0x7fbe000735d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:00:51.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 >> 172.21.15.42:0/1565587999 conn(0x7fbe000fc750 msgr2=0x7fbe000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:00:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 shutdown_connections 2024-09-17T00:00:51.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:00:51.771+0000 7fbe05364700 1 -- 172.21.15.42:0/1565587999 wait complete. 2024-09-17T00:00:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:51 smithi042 ceph-mon[111056]: pgmap v278: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:51 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1005323801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:00:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:51 smithi042 ceph-mon[111056]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:52.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:51 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1565587999' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:00:52.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:51 smithi043 ceph-mon[81510]: pgmap v278: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:52.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:51 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1005323801' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:00:52.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:51 smithi043 ceph-mon[81510]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:00:52.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:51 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1565587999' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:00:54.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:53 smithi043 ceph-mon[81510]: pgmap v279: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:54.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:53 smithi042 ceph-mon[111056]: pgmap v279: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:56.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:55 smithi043 ceph-mon[81510]: pgmap v280: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:56.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:55 smithi042 ceph-mon[111056]: pgmap v280: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:57 smithi043 ceph-mon[81510]: pgmap v281: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:00:58.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:57 smithi042 ceph-mon[111056]: pgmap v281: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:00:59 smithi043 ceph-mon[81510]: pgmap v282: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:00.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:00:59 smithi042 ceph-mon[111056]: pgmap v282: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:02.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:01 smithi043 ceph-mon[81510]: pgmap v283: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:01 smithi042 ceph-mon[111056]: pgmap v283: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:03 smithi043 ceph-mon[81510]: pgmap v284: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:04.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:03 smithi042 ceph-mon[111056]: pgmap v284: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:06.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:06 smithi042 ceph-mon[111056]: pgmap v285: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:06.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:06 smithi043 ceph-mon[81510]: pgmap v285: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:08.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:08 smithi042 ceph-mon[111056]: pgmap v286: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:08 smithi043 ceph-mon[81510]: pgmap v286: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:10.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:10 smithi042 ceph-mon[111056]: pgmap v287: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:10.613 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:10 smithi043 ceph-mon[81510]: pgmap v287: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:12.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:12 smithi042 ceph-mon[111056]: pgmap v288: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:12 smithi043 ceph-mon[81510]: pgmap v288: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:14.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:14 smithi042 ceph-mon[111056]: pgmap v289: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:14 smithi043 ceph-mon[81510]: pgmap v289: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:16 smithi042 ceph-mon[111056]: pgmap v290: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:16.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:16 smithi043 ceph-mon[81510]: pgmap v290: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:18.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:18 smithi042 ceph-mon[111056]: pgmap v291: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:18.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:18 smithi043 ceph-mon[81510]: pgmap v291: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:20.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:20 smithi042 ceph-mon[111056]: pgmap v292: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:20.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:20 smithi043 ceph-mon[81510]: pgmap v292: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 -- 172.21.15.42:0/4287261290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41280ffda0 msgr2=0x7f4128100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 --2- 172.21.15.42:0/4287261290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41280ffda0 0x7f4128100180 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4118009910 tx=0x7f4118009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 -- 172.21.15.42:0/4287261290 shutdown_connections 2024-09-17T00:01:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 --2- 172.21.15.42:0/4287261290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4128100750 0x7f41281047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 --2- 172.21.15.42:0/4287261290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41280ffda0 0x7f4128100180 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 -- 172.21.15.42:0/4287261290 >> 172.21.15.42:0/4287261290 conn(0x7f41280fb650 msgr2=0x7f41280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:21.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 -- 172.21.15.42:0/4287261290 shutdown_connections 2024-09-17T00:01:21.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.896+0000 7f412cf28700 1 -- 172.21.15.42:0/4287261290 wait complete. 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f412cf28700 1 Processor -- start 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f412cf28700 1 -- start start 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f412cf28700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41280ffda0 0x7f412819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f412cf28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4128100750 0x7f412819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f412cf28700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412819f460 con 0x7f4128100750 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f412cf28700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128198910 con 0x7f41280ffda0 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f4125d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4128100750 0x7f412819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:21.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f4125d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4128100750 0x7f412819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58386/0 (socket says 172.21.15.42:58386) 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f4125d9b700 1 -- 172.21.15.42:0/162149052 learned_addr learned my addr 172.21.15.42:0/162149052 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f412659c700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41280ffda0 0x7f412819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f4125d9b700 1 -- 172.21.15.42:0/162149052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41280ffda0 msgr2=0x7f412819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f4125d9b700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41280ffda0 0x7f412819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.897+0000 7f4125d9b700 1 -- 172.21.15.42:0/162149052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41180095f0 con 0x7f4128100750 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.898+0000 7f4125d9b700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4128100750 0x7f412819edd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e970 tx=0x7f411c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.898+0000 7f41177fe700 1 -- 172.21.15.42:0/162149052 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411c00c9d0 con 0x7f4128100750 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.898+0000 7f41177fe700 1 -- 172.21.15.42:0/162149052 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f411c004830 con 0x7f4128100750 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.898+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4128198bf0 con 0x7f4128100750 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.898+0000 7f41177fe700 1 -- 172.21.15.42:0/162149052 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411c005660 con 0x7f4128100750 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.898+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4128199140 con 0x7f4128100750 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.899+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412804f2e0 con 0x7f4128100750 2024-09-17T00:01:21.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.899+0000 7f41177fe700 1 -- 172.21.15.42:0/162149052 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f411c00cb30 con 0x7f4128100750 2024-09-17T00:01:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.900+0000 7f41177fe700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f411006e680 0x7f4110070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.900+0000 7f41177fe700 1 -- 172.21.15.42:0/162149052 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f411c014070 con 0x7f4128100750 2024-09-17T00:01:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.900+0000 7f412659c700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f411006e680 0x7f4110070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:21.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.901+0000 7f412659c700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f411006e680 0x7f4110070b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f41180059d0 tx=0x7f4118005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:21.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:21.905+0000 7f41177fe700 1 -- 172.21.15.42:0/162149052 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f411c05b5f0 con 0x7f4128100750 2024-09-17T00:01:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.135+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41281994c0 con 0x7f411006e680 2024-09-17T00:01:22.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.136+0000 7f41177fe700 1 -- 172.21.15.42:0/162149052 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f41281994c0 con 0x7f411006e680 2024-09-17T00:01:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f411006e680 msgr2=0x7f4110070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f411006e680 0x7f4110070b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f41180059d0 tx=0x7f4118005940 comp rx=0 tx=0).stop 2024-09-17T00:01:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4128100750 msgr2=0x7f412819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4128100750 0x7f412819edd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f411c00e970 tx=0x7f411c00ec80 comp rx=0 tx=0).stop 2024-09-17T00:01:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 shutdown_connections 2024-09-17T00:01:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f411006e680 0x7f4110070b40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41280ffda0 0x7f412819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 --2- 172.21.15.42:0/162149052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4128100750 0x7f412819edd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 >> 172.21.15.42:0/162149052 conn(0x7f41280fb650 msgr2=0x7f41280696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.138+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 shutdown_connections 2024-09-17T00:01:22.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.139+0000 7f412cf28700 1 -- 172.21.15.42:0/162149052 wait complete. 2024-09-17T00:01:22.147 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:01:22.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 -- 172.21.15.42:0/177239278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 msgr2=0x7f280c100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 --2- 172.21.15.42:0/177239278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c100ea0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009910 tx=0x7f27f4009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:22.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 -- 172.21.15.42:0/177239278 shutdown_connections 2024-09-17T00:01:22.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 --2- 172.21.15.42:0/177239278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c100ea0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 --2- 172.21.15.42:0/177239278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f280c106a20 0x7f280c106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 -- 172.21.15.42:0/177239278 >> 172.21.15.42:0/177239278 conn(0x7f280c0fc510 msgr2=0x7f280c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:22.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 -- 172.21.15.42:0/177239278 shutdown_connections 2024-09-17T00:01:22.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.249+0000 7f2810801700 1 -- 172.21.15.42:0/177239278 wait complete. 2024-09-17T00:01:22.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f2810801700 1 Processor -- start 2024-09-17T00:01:22.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f2810801700 1 -- start start 2024-09-17T00:01:22.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f2810801700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f2810801700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f280c106a20 0x7f280c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f2810801700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c073b10 con 0x7f280c100a20 2024-09-17T00:01:22.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f2810801700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c073c50 con 0x7f280c106a20 2024-09-17T00:01:22.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f280ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f280ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58412/0 (socket says 172.21.15.42:58412) 2024-09-17T00:01:22.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.250+0000 7f280ad9d700 1 -- 172.21.15.42:0/2058056354 learned_addr learned my addr 172.21.15.42:0/2058056354 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f27fa59c700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f280c106a20 0x7f280c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f280ad9d700 1 -- 172.21.15.42:0/2058056354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f280c106a20 msgr2=0x7f280c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f280ad9d700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f280c106a20 0x7f280c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f280ad9d700 1 -- 172.21.15.42:0/2058056354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f40095f0 con 0x7f280c100a20 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f27fa59c700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f280c106a20 0x7f280c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f280ad9d700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c073090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00ada0 tx=0x7f27fc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f2808d99700 1 -- 172.21.15.42:0/2058056354 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27fc00f070 con 0x7f280c100a20 2024-09-17T00:01:22.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f2808d99700 1 -- 172.21.15.42:0/2058056354 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27fc007bf0 con 0x7f280c100a20 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f2808d99700 1 -- 172.21.15.42:0/2058056354 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27fc005690 con 0x7f280c100a20 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c1a27d0 con 0x7f280c100a20 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.251+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c1a2be0 con 0x7f280c100a20 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.252+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c108fb0 con 0x7f280c100a20 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.253+0000 7f2808d99700 1 -- 172.21.15.42:0/2058056354 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f27fc005850 con 0x7f280c100a20 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.253+0000 7f2808d99700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f27ec06e680 0x7f27ec070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.253+0000 7f2808d99700 1 -- 172.21.15.42:0/2058056354 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f27fc08dff0 con 0x7f280c100a20 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.253+0000 7f27fa59c700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f27ec06e680 0x7f27ec070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.254+0000 7f27fa59c700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f27ec06e680 0x7f27ec070b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27f400b5c0 tx=0x7f27f4005480 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:22.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.258+0000 7f2808d99700 1 -- 172.21.15.42:0/2058056354 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f27fc057210 con 0x7f280c100a20 2024-09-17T00:01:22.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.488+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f280c02cce0 con 0x7f27ec06e680 2024-09-17T00:01:22.486 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:22 smithi042 ceph-mon[111056]: pgmap v293: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:22.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.490+0000 7f2808d99700 1 -- 172.21.15.42:0/2058056354 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f280c02cce0 con 0x7f27ec06e680 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f27ec06e680 msgr2=0x7f27ec070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f27ec06e680 0x7f27ec070b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f27f400b5c0 tx=0x7f27f4005480 comp rx=0 tx=0).stop 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 msgr2=0x7f280c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c073090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00ada0 tx=0x7f27fc00c330 comp rx=0 tx=0).stop 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 shutdown_connections 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f27ec06e680 0x7f27ec070b40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f280c100a20 0x7f280c073090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 --2- 172.21.15.42:0/2058056354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f280c106a20 0x7f280c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 >> 172.21.15.42:0/2058056354 conn(0x7f280c0fc510 msgr2=0x7f280c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:22.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 shutdown_connections 2024-09-17T00:01:22.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.492+0000 7f2810801700 1 -- 172.21.15.42:0/2058056354 wait complete. 2024-09-17T00:01:22.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 -- 172.21.15.42:0/3698666085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 msgr2=0x7fb4101058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 --2- 172.21.15.42:0/3698666085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb4101058a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb400009910 tx=0x7fb400009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 -- 172.21.15.42:0/3698666085 shutdown_connections 2024-09-17T00:01:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 --2- 172.21.15.42:0/3698666085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb4101058a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 --2- 172.21.15.42:0/3698666085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb410100ea0 0x7fb410101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 -- 172.21.15.42:0/3698666085 >> 172.21.15.42:0/3698666085 conn(0x7fb4100fc750 msgr2=0x7fb4100feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 -- 172.21.15.42:0/3698666085 shutdown_connections 2024-09-17T00:01:22.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.601+0000 7fb41671e700 1 -- 172.21.15.42:0/3698666085 wait complete. 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb41671e700 1 Processor -- start 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb41671e700 1 -- start start 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb41671e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb410100ea0 0x7fb41019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb41671e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb41019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb41671e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019f470 con 0x7fb410101850 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb41671e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410198920 con 0x7fb410100ea0 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb40f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb41019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb40f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb41019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49644/0 (socket says 172.21.15.42:49644) 2024-09-17T00:01:22.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb40f7fe700 1 -- 172.21.15.42:0/3396140564 learned_addr learned my addr 172.21.15.42:0/3396140564 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.602+0000 7fb40ffff700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb410100ea0 0x7fb41019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40f7fe700 1 -- 172.21.15.42:0/3396140564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb410100ea0 msgr2=0x7fb41019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40f7fe700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb410100ea0 0x7fb41019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40f7fe700 1 -- 172.21.15.42:0/3396140564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4000095f0 con 0x7fb410101850 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40ffff700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb410100ea0 0x7fb41019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40f7fe700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb41019ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4000092e0 tx=0x7fb40000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40d7fa700 1 -- 172.21.15.42:0/3396140564 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb40001d070 con 0x7fb410101850 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb410198ba0 con 0x7fb410101850 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40d7fa700 1 -- 172.21.15.42:0/3396140564 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb400005dc0 con 0x7fb410101850 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb40d7fa700 1 -- 172.21.15.42:0/3396140564 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb400017400 con 0x7fb410101850 2024-09-17T00:01:22.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.603+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb410199090 con 0x7fb410101850 2024-09-17T00:01:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.604+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410105020 con 0x7fb410101850 2024-09-17T00:01:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.604+0000 7fb40d7fa700 1 -- 172.21.15.42:0/3396140564 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb40000f460 con 0x7fb410101850 2024-09-17T00:01:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.605+0000 7fb40d7fa700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb3fc06e680 0x7fb3fc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.605+0000 7fb40d7fa700 1 -- 172.21.15.42:0/3396140564 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb40008fc10 con 0x7fb410101850 2024-09-17T00:01:22.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.605+0000 7fb40ffff700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb3fc06e680 0x7fb3fc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.606+0000 7fb40ffff700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb3fc06e680 0x7fb3fc070b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8005950 tx=0x7fb3f80058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:22.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.609+0000 7fb40d7fa700 1 -- 172.21.15.42:0/3396140564 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb40005c840 con 0x7fb410101850 2024-09-17T00:01:22.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:22 smithi043 ceph-mon[81510]: pgmap v293: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:22.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.832+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb410199ea0 con 0x7fb3fc06e680 2024-09-17T00:01:22.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.841+0000 7fb40d7fa700 1 -- 172.21.15.42:0/3396140564 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fb410199ea0 con 0x7fb3fc06e680 2024-09-17T00:01:22.840 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:01:22.840 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (10m) 3m ago 20m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:01:22.840 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (21m) 3m ago 21m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:01:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (19m) 3m ago 19m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:01:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (7m) 3m ago 20m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:01:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (7m) 3m ago 19m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:01:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (9m) 3m ago 20m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:01:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (3m) 3m ago 15m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:01:22.841 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (4m) 3m ago 15m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:01:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (3m) 3m ago 15m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:01:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (3m) 3m ago 15m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:01:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (11m) 3m ago 22m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:01:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (10m) 3m ago 19m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:01:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (8m) 3m ago 22m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:01:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (8m) 3m ago 19m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:01:22.842 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (10m) 3m ago 20m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:01:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (10m) 3m ago 19m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:01:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 3m ago 18m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:01:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (6m) 3m ago 18m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:01:22.843 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (6m) 3m ago 17m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:01:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (5m) 3m ago 17m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:01:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (4m) 3m ago 17m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:01:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (4m) 3m ago 16m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:01:22.844 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (10m) 3m ago 20m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:01:22.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.843+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb3fc06e680 msgr2=0x7fb3fc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.843+0000 7fb41671e700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb3fc06e680 0x7fb3fc070b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8005950 tx=0x7fb3f80058e0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.843+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 msgr2=0x7fb41019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.843+0000 7fb41671e700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb41019ede0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4000092e0 tx=0x7fb40000be20 comp rx=0 tx=0).stop 2024-09-17T00:01:22.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.844+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 shutdown_connections 2024-09-17T00:01:22.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.844+0000 7fb41671e700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb3fc06e680 0x7fb3fc070b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.844+0000 7fb41671e700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb410100ea0 0x7fb41019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.844+0000 7fb41671e700 1 --2- 172.21.15.42:0/3396140564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb410101850 0x7fb41019ede0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.844+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 >> 172.21.15.42:0/3396140564 conn(0x7fb4100fc750 msgr2=0x7fb4100feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:22.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.844+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 shutdown_connections 2024-09-17T00:01:22.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.844+0000 7fb41671e700 1 -- 172.21.15.42:0/3396140564 wait complete. 2024-09-17T00:01:22.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 -- 172.21.15.42:0/911065854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 msgr2=0x7f069c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 --2- 172.21.15.42:0/911065854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c0fffc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0690009910 tx=0x7f0690009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:22.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 -- 172.21.15.42:0/911065854 shutdown_connections 2024-09-17T00:01:22.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 --2- 172.21.15.42:0/911065854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c0fffc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 --2- 172.21.15.42:0/911065854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f069c105bf0 0x7f069c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 -- 172.21.15.42:0/911065854 >> 172.21.15.42:0/911065854 conn(0x7f069c0fb650 msgr2=0x7f069c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:22.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 -- 172.21.15.42:0/911065854 shutdown_connections 2024-09-17T00:01:22.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.951+0000 7f06a20bf700 1 -- 172.21.15.42:0/911065854 wait complete. 2024-09-17T00:01:22.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f06a20bf700 1 Processor -- start 2024-09-17T00:01:22.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f06a20bf700 1 -- start start 2024-09-17T00:01:22.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f06a20bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f06a20bf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f069c105bf0 0x7f069c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f06a20bf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c19d0f0 con 0x7f069c0ffb40 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f06a20bf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c196630 con 0x7f069c105bf0 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f069b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f069b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49654/0 (socket says 172.21.15.42:49654) 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f069b7fe700 1 -- 172.21.15.42:0/3380329514 learned_addr learned my addr 172.21.15.42:0/3380329514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.952+0000 7f069affd700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f069c105bf0 0x7f069c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f069b7fe700 1 -- 172.21.15.42:0/3380329514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f069c105bf0 msgr2=0x7f069c19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f069b7fe700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f069c105bf0 0x7f069c19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f069b7fe700 1 -- 172.21.15.42:0/3380329514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900095f0 con 0x7f069c0ffb40 2024-09-17T00:01:22.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f069b7fe700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c19c5b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c930 tx=0x7f068c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:22.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f0698ff9700 1 -- 172.21.15.42:0/3380329514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068c007ad0 con 0x7f069c0ffb40 2024-09-17T00:01:22.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c196910 con 0x7f069c0ffb40 2024-09-17T00:01:22.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f0698ff9700 1 -- 172.21.15.42:0/3380329514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f068c00ce80 con 0x7f069c0ffb40 2024-09-17T00:01:22.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f0698ff9700 1 -- 172.21.15.42:0/3380329514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068c018630 con 0x7f069c0ffb40 2024-09-17T00:01:22.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.953+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c196e60 con 0x7f069c0ffb40 2024-09-17T00:01:22.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.954+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c04f2e0 con 0x7f069c0ffb40 2024-09-17T00:01:22.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.955+0000 7f0698ff9700 1 -- 172.21.15.42:0/3380329514 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f068c007c30 con 0x7f069c0ffb40 2024-09-17T00:01:22.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.955+0000 7f0698ff9700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f068406e680 0x7f0684070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:22.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.955+0000 7f0698ff9700 1 -- 172.21.15.42:0/3380329514 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f068c08e6c0 con 0x7f069c0ffb40 2024-09-17T00:01:22.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.955+0000 7f069affd700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f068406e680 0x7f0684070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:22.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.956+0000 7f069affd700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f068406e680 0x7f0684070b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f069c0feb70 tx=0x7f069001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:22.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:22.960+0000 7f0698ff9700 1 -- 172.21.15.42:0/3380329514 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f068c05b390 con 0x7f069c0ffb40 2024-09-17T00:01:23.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.254+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f069c197140 con 0x7f069c0ffb40 2024-09-17T00:01:23.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.254+0000 7f0698ff9700 1 -- 172.21.15.42:0/3380329514 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f068c05af20 con 0x7f069c0ffb40 2024-09-17T00:01:23.252 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:01:23.252 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:01:23.252 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:01:23.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:01:23.253 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:01:23.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f068406e680 msgr2=0x7f0684070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f068406e680 0x7f0684070b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f069c0feb70 tx=0x7f069001a040 comp rx=0 tx=0).stop 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 msgr2=0x7f069c19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c19c5b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c930 tx=0x7f068c00cc40 comp rx=0 tx=0).stop 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 shutdown_connections 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f068406e680 0x7f0684070b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f069c0ffb40 0x7f069c19c5b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 --2- 172.21.15.42:0/3380329514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f069c105bf0 0x7f069c19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 >> 172.21.15.42:0/3380329514 conn(0x7f069c0fb650 msgr2=0x7f069c068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 shutdown_connections 2024-09-17T00:01:23.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.257+0000 7f06a20bf700 1 -- 172.21.15.42:0/3380329514 wait complete. 2024-09-17T00:01:23.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 -- 172.21.15.42:0/1500788773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11180ff990 msgr2=0x7f11180ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 --2- 172.21.15.42:0/1500788773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11180ff990 0x7f11180ffd70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1114009910 tx=0x7f1114009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:23.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 -- 172.21.15.42:0/1500788773 shutdown_connections 2024-09-17T00:01:23.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 --2- 172.21.15.42:0/1500788773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1118100340 0x7f111810d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 --2- 172.21.15.42:0/1500788773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11180ff990 0x7f11180ffd70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 -- 172.21.15.42:0/1500788773 >> 172.21.15.42:0/1500788773 conn(0x7f11180fb650 msgr2=0x7f11180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:23.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 -- 172.21.15.42:0/1500788773 shutdown_connections 2024-09-17T00:01:23.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.365+0000 7f111fc66700 1 -- 172.21.15.42:0/1500788773 wait complete. 2024-09-17T00:01:23.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.366+0000 7f111fc66700 1 Processor -- start 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.366+0000 7f111fc66700 1 -- start start 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.366+0000 7f111fc66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11180ff990 0x7f111819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.366+0000 7f111fc66700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1118100340 0x7f111819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.366+0000 7f111fc66700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111819f430 con 0x7f1118100340 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.366+0000 7f111fc66700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11181988e0 con 0x7f11180ff990 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111d201700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1118100340 0x7f111819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111d201700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1118100340 0x7f111819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49676/0 (socket says 172.21.15.42:49676) 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111d201700 1 -- 172.21.15.42:0/2367681510 learned_addr learned my addr 172.21.15.42:0/2367681510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111da02700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11180ff990 0x7f111819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111d201700 1 -- 172.21.15.42:0/2367681510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11180ff990 msgr2=0x7f111819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111d201700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11180ff990 0x7f111819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111d201700 1 -- 172.21.15.42:0/2367681510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11140095f0 con 0x7f1118100340 2024-09-17T00:01:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111d201700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1118100340 0x7f111819eda0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f110800d8d0 tx=0x7f110800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f110effd700 1 -- 172.21.15.42:0/2367681510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11080098a0 con 0x7f1118100340 2024-09-17T00:01:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f110effd700 1 -- 172.21.15.42:0/2367681510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1108010460 con 0x7f1118100340 2024-09-17T00:01:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1118198bc0 con 0x7f1118100340 2024-09-17T00:01:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f110effd700 1 -- 172.21.15.42:0/2367681510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110800f5f0 con 0x7f1118100340 2024-09-17T00:01:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.367+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1118199110 con 0x7f1118100340 2024-09-17T00:01:23.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.368+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111810a770 con 0x7f1118100340 2024-09-17T00:01:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.369+0000 7f110effd700 1 -- 172.21.15.42:0/2367681510 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1108009a00 con 0x7f1118100340 2024-09-17T00:01:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.369+0000 7f110effd700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f110406e680 0x7f1104070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.369+0000 7f110effd700 1 -- 172.21.15.42:0/2367681510 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f110808e4f0 con 0x7f1118100340 2024-09-17T00:01:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.370+0000 7f111da02700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f110406e680 0x7f1104070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:23.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.370+0000 7f111da02700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f110406e680 0x7f1104070b40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1114006010 tx=0x7f111400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:23.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.374+0000 7f110effd700 1 -- 172.21.15.42:0/2367681510 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f110805b1a0 con 0x7f1118100340 2024-09-17T00:01:23.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:23 smithi042 ceph-mon[111056]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:23.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:23 smithi042 ceph-mon[111056]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:23.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.628+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f111804f2e0 con 0x7f1118100340 2024-09-17T00:01:23.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.628+0000 7f110effd700 1 -- 172.21.15.42:0/2367681510 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f110805ad30 con 0x7f1118100340 2024-09-17T00:01:23.626 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:01:23.626 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:01:23.626 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:01:23.626 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:01:23.626 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:01:23.627 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:01:23.628 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:23.629 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:23.630 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:23.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:23 smithi043 ceph-mon[81510]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:23.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:23 smithi043 ceph-mon[81510]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.630+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f110406e680 msgr2=0x7f1104070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.630+0000 7f111fc66700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f110406e680 0x7f1104070b40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1114006010 tx=0x7f111400b540 comp rx=0 tx=0).stop 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.630+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1118100340 msgr2=0x7f111819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.630+0000 7f111fc66700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1118100340 0x7f111819eda0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f110800d8d0 tx=0x7f110800dbe0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.631+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 shutdown_connections 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.631+0000 7f111fc66700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f110406e680 0x7f1104070b40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.631+0000 7f111fc66700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11180ff990 0x7f111819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.631+0000 7f111fc66700 1 --2- 172.21.15.42:0/2367681510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1118100340 0x7f111819eda0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.631+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 >> 172.21.15.42:0/2367681510 conn(0x7f11180fb650 msgr2=0x7f11180fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:23.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.631+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 shutdown_connections 2024-09-17T00:01:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.631+0000 7f111fc66700 1 -- 172.21.15.42:0/2367681510 wait complete. 2024-09-17T00:01:23.632 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:01:23.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 -- 172.21.15.42:0/2702490500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8824105bf0 msgr2=0x7f8824105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 --2- 172.21.15.42:0/2702490500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8824105bf0 0x7f8824105fd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8818009910 tx=0x7f8818009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 -- 172.21.15.42:0/2702490500 shutdown_connections 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 --2- 172.21.15.42:0/2702490500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88240ffb40 0x7f88240fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 --2- 172.21.15.42:0/2702490500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8824105bf0 0x7f8824105fd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 -- 172.21.15.42:0/2702490500 >> 172.21.15.42:0/2702490500 conn(0x7f88240fb650 msgr2=0x7f88240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 -- 172.21.15.42:0/2702490500 shutdown_connections 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.740+0000 7f882af83700 1 -- 172.21.15.42:0/2702490500 wait complete. 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f882af83700 1 Processor -- start 2024-09-17T00:01:23.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f882af83700 1 -- start start 2024-09-17T00:01:23.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f882af83700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88240ffb40 0x7f882419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:23.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f882af83700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8824105bf0 0x7f882419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:23.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f882af83700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882419d170 con 0x7f88240ffb40 2024-09-17T00:01:23.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f882af83700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8824196620 con 0x7f8824105bf0 2024-09-17T00:01:23.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f8828d1f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88240ffb40 0x7f882419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:23.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f8828d1f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88240ffb40 0x7f882419c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49706/0 (socket says 172.21.15.42:49706) 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.741+0000 7f8828d1f700 1 -- 172.21.15.42:0/4110826122 learned_addr learned my addr 172.21.15.42:0/4110826122 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8823fff700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8824105bf0 0x7f882419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8828d1f700 1 -- 172.21.15.42:0/4110826122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8824105bf0 msgr2=0x7f882419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8828d1f700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8824105bf0 0x7f882419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8828d1f700 1 -- 172.21.15.42:0/4110826122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180095f0 con 0x7f88240ffb40 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8828d1f700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88240ffb40 0x7f882419c5a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f88180092e0 tx=0x7f881800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8821ffb700 1 -- 172.21.15.42:0/4110826122 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f881801d070 con 0x7f88240ffb40 2024-09-17T00:01:23.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8821ffb700 1 -- 172.21.15.42:0/4110826122 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8818005e00 con 0x7f88240ffb40 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88241968a0 con 0x7f88240ffb40 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f8821ffb700 1 -- 172.21.15.42:0/4110826122 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8818017400 con 0x7f88240ffb40 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.742+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8824196d90 con 0x7f88240ffb40 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.743+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882404f2e0 con 0x7f88240ffb40 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.744+0000 7f8821ffb700 1 -- 172.21.15.42:0/4110826122 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f881800f460 con 0x7f88240ffb40 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.744+0000 7f8821ffb700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f881406e680 0x7f8814070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.744+0000 7f8821ffb700 1 -- 172.21.15.42:0/4110826122 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f881808fa90 con 0x7f88240ffb40 2024-09-17T00:01:23.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.744+0000 7f8823fff700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f881406e680 0x7f8814070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:23.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.745+0000 7f8823fff700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f881406e680 0x7f8814070b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8810005950 tx=0x7f88100058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:23.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.749+0000 7f8821ffb700 1 -- 172.21.15.42:0/4110826122 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f881805c7e0 con 0x7f88240ffb40 2024-09-17T00:01:23.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.977+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88241971e0 con 0x7f881406e680 2024-09-17T00:01:23.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.979+0000 7f8821ffb700 1 -- 172.21.15.42:0/4110826122 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f88241971e0 con 0x7f881406e680 2024-09-17T00:01:23.976 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:01:23.976 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:01:23.977 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:01:23.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f881406e680 msgr2=0x7f8814070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f881406e680 0x7f8814070b40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8810005950 tx=0x7f88100058e0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88240ffb40 msgr2=0x7f882419c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:23.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88240ffb40 0x7f882419c5a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f88180092e0 tx=0x7f881800be20 comp rx=0 tx=0).stop 2024-09-17T00:01:23.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 shutdown_connections 2024-09-17T00:01:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f881406e680 0x7f8814070b40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88240ffb40 0x7f882419c5a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 --2- 172.21.15.42:0/4110826122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8824105bf0 0x7f882419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.981+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 >> 172.21.15.42:0/4110826122 conn(0x7f88240fb650 msgr2=0x7f8824068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.982+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 shutdown_connections 2024-09-17T00:01:23.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:23.982+0000 7f882af83700 1 -- 172.21.15.42:0/4110826122 wait complete. 2024-09-17T00:01:24.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 -- 172.21.15.42:0/2438162442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 msgr2=0x7fab3c105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 --2- 172.21.15.42:0/2438162442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c105e00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009910 tx=0x7fab2c009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 -- 172.21.15.42:0/2438162442 shutdown_connections 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 --2- 172.21.15.42:0/2438162442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c0ffb60 0x7fab3c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 --2- 172.21.15.42:0/2438162442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c105e00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 -- 172.21.15.42:0/2438162442 >> 172.21.15.42:0/2438162442 conn(0x7fab3c0754a0 msgr2=0x7fab3c0758b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 -- 172.21.15.42:0/2438162442 shutdown_connections 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.090+0000 7fab43008700 1 -- 172.21.15.42:0/2438162442 wait complete. 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab43008700 1 Processor -- start 2024-09-17T00:01:24.090 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab43008700 1 -- start start 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab43008700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c0ffb60 0x7fab3c1015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab43008700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab43008700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c102100 con 0x7fab3c105a20 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab43008700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c102270 con 0x7fab3c0ffb60 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab3bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab3bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49718/0 (socket says 172.21.15.42:49718) 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab3bfff700 1 -- 172.21.15.42:0/2233443926 learned_addr learned my addr 172.21.15.42:0/2233443926 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab40da4700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c0ffb60 0x7fab3c1015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab3bfff700 1 -- 172.21.15.42:0/2233443926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c0ffb60 msgr2=0x7fab3c1015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab3bfff700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c0ffb60 0x7fab3c1015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.091+0000 7fab3bfff700 1 -- 172.21.15.42:0/2233443926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c0095f0 con 0x7fab3c105a20 2024-09-17T00:01:24.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.092+0000 7fab3bfff700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c101b30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab3000d900 tx=0x7fab3000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.092+0000 7fab39ffb700 1 -- 172.21.15.42:0/2233443926 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab30004a00 con 0x7fab3c105a20 2024-09-17T00:01:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.092+0000 7fab39ffb700 1 -- 172.21.15.42:0/2233443926 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab300054e0 con 0x7fab3c105a20 2024-09-17T00:01:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.092+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3c1a28c0 con 0x7fab3c105a20 2024-09-17T00:01:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.092+0000 7fab39ffb700 1 -- 172.21.15.42:0/2233443926 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab30009d00 con 0x7fab3c105a20 2024-09-17T00:01:24.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.092+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3c1a2d30 con 0x7fab3c105a20 2024-09-17T00:01:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.093+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3c108fe0 con 0x7fab3c105a20 2024-09-17T00:01:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.093+0000 7fab39ffb700 1 -- 172.21.15.42:0/2233443926 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fab30005020 con 0x7fab3c105a20 2024-09-17T00:01:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.094+0000 7fab39ffb700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab2406e680 0x7fab24070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.094+0000 7fab39ffb700 1 -- 172.21.15.42:0/2233443926 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fab3008e520 con 0x7fab3c105a20 2024-09-17T00:01:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.094+0000 7fab40da4700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab2406e680 0x7fab24070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:24.093 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.094+0000 7fab40da4700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab2406e680 0x7fab24070b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b5c0 tx=0x7fab2c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:24.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.098+0000 7fab39ffb700 1 -- 172.21.15.42:0/2233443926 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fab30057740 con 0x7fab3c105a20 2024-09-17T00:01:24.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.401+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab3c02cc70 con 0x7fab3c105a20 2024-09-17T00:01:24.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.401+0000 7fab39ffb700 1 -- 172.21.15.42:0/2233443926 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fab300171c0 con 0x7fab3c105a20 2024-09-17T00:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:01:24.399 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.403+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab2406e680 msgr2=0x7fab24070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.403+0000 7fab43008700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab2406e680 0x7fab24070b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00b5c0 tx=0x7fab2c01a040 comp rx=0 tx=0).stop 2024-09-17T00:01:24.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 msgr2=0x7fab3c101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c101b30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fab3000d900 tx=0x7fab3000dc10 comp rx=0 tx=0).stop 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 shutdown_connections 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab2406e680 0x7fab24070b40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab3c0ffb60 0x7fab3c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 --2- 172.21.15.42:0/2233443926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab3c105a20 0x7fab3c101b30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 >> 172.21.15.42:0/2233443926 conn(0x7fab3c0754a0 msgr2=0x7fab3c068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 shutdown_connections 2024-09-17T00:01:24.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:24.404+0000 7fab43008700 1 -- 172.21.15.42:0/2233443926 wait complete. 2024-09-17T00:01:24.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:24 smithi042 ceph-mon[111056]: pgmap v294: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:24 smithi042 ceph-mon[111056]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:24 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3380329514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:01:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:24 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2367681510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:01:24.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:24 smithi043 ceph-mon[81510]: pgmap v294: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:24 smithi043 ceph-mon[81510]: from='client.34606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:24 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3380329514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:01:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:24 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2367681510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:01:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:25 smithi042 ceph-mon[111056]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:25 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2233443926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:01:25.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:25 smithi043 ceph-mon[81510]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:25 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2233443926' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:01:26.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:26 smithi042 ceph-mon[111056]: pgmap v295: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:26.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:26 smithi043 ceph-mon[81510]: pgmap v295: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:28.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:28 smithi042 ceph-mon[111056]: pgmap v296: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:28.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:28 smithi043 ceph-mon[81510]: pgmap v296: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:30.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:30 smithi042 ceph-mon[111056]: pgmap v297: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:30.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:30 smithi043 ceph-mon[81510]: pgmap v297: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:32.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:32 smithi042 ceph-mon[111056]: pgmap v298: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:32 smithi043 ceph-mon[81510]: pgmap v298: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:34.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:34 smithi042 ceph-mon[111056]: pgmap v299: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:34 smithi043 ceph-mon[81510]: pgmap v299: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:36 smithi042 ceph-mon[111056]: pgmap v300: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:36 smithi043 ceph-mon[81510]: pgmap v300: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:38 smithi042 ceph-mon[111056]: pgmap v301: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:38 smithi043 ceph-mon[81510]: pgmap v301: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:40.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:40 smithi042 ceph-mon[111056]: pgmap v302: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:40.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:40 smithi043 ceph-mon[81510]: pgmap v302: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:42 smithi042 ceph-mon[111056]: pgmap v303: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:42 smithi043 ceph-mon[81510]: pgmap v303: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:44.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:44 smithi042 ceph-mon[111056]: pgmap v304: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:44 smithi043 ceph-mon[81510]: pgmap v304: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:46.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:46 smithi042 ceph-mon[111056]: pgmap v305: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:46.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:46 smithi043 ceph-mon[81510]: pgmap v305: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:48.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:48 smithi042 ceph-mon[111056]: pgmap v306: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:48.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:48 smithi043 ceph-mon[81510]: pgmap v306: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:50.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:50 smithi043 ceph-mon[81510]: pgmap v307: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:50 smithi042 ceph-mon[111056]: pgmap v307: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:51.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:01:52.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:52 smithi043 ceph-mon[81510]: pgmap v308: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:52 smithi042 ceph-mon[111056]: pgmap v308: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:54.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 -- 172.21.15.42:0/3770566678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c4100750 msgr2=0x7f40c41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:54.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/3770566678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c4100750 0x7f40c41047a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009910 tx=0x7f40ac009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:54.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 -- 172.21.15.42:0/3770566678 shutdown_connections 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/3770566678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c4100750 0x7f40c41047a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/3770566678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c40ffda0 0x7f40c4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 -- 172.21.15.42:0/3770566678 >> 172.21.15.42:0/3770566678 conn(0x7f40c40fb650 msgr2=0x7f40c40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 -- 172.21.15.42:0/3770566678 shutdown_connections 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.529+0000 7f40c9a4f700 1 -- 172.21.15.42:0/3770566678 wait complete. 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c9a4f700 1 Processor -- start 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c9a4f700 1 -- start start 2024-09-17T00:01:54.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c9a4f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c40ffda0 0x7f40c419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c9a4f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c4100750 0x7f40c419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c9a4f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c419d240 con 0x7f40c40ffda0 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c9a4f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c41966f0 con 0x7f40c4100750 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c40ffda0 0x7f40c419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c40ffda0 0x7f40c419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60430/0 (socket says 172.21.15.42:60430) 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c2ffd700 1 -- 172.21.15.42:0/2264337203 learned_addr learned my addr 172.21.15.42:0/2264337203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40bbfff700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c4100750 0x7f40c419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c2ffd700 1 -- 172.21.15.42:0/2264337203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c4100750 msgr2=0x7f40c419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:54.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c2ffd700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c4100750 0x7f40c419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.530+0000 7f40c2ffd700 1 -- 172.21.15.42:0/2264337203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0095f0 con 0x7f40c40ffda0 2024-09-17T00:01:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.531+0000 7f40c2ffd700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c40ffda0 0x7f40c419c670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40b400d8d0 tx=0x7f40b400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.531+0000 7f40c8a4d700 1 -- 172.21.15.42:0/2264337203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b40098a0 con 0x7f40c40ffda0 2024-09-17T00:01:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.531+0000 7f40c8a4d700 1 -- 172.21.15.42:0/2264337203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40b4010460 con 0x7f40c40ffda0 2024-09-17T00:01:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.531+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c41969d0 con 0x7f40c40ffda0 2024-09-17T00:01:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.531+0000 7f40c8a4d700 1 -- 172.21.15.42:0/2264337203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b400f5f0 con 0x7f40c40ffda0 2024-09-17T00:01:54.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.531+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c4196f20 con 0x7f40c40ffda0 2024-09-17T00:01:54.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.532+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c41091b0 con 0x7f40c40ffda0 2024-09-17T00:01:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.533+0000 7f40c8a4d700 1 -- 172.21.15.42:0/2264337203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f40b4009a00 con 0x7f40c40ffda0 2024-09-17T00:01:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.533+0000 7f40c8a4d700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40b006e680 0x7f40b0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.533+0000 7f40c8a4d700 1 -- 172.21.15.42:0/2264337203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f40b408e530 con 0x7f40c40ffda0 2024-09-17T00:01:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.533+0000 7f40bbfff700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40b006e680 0x7f40b0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:54.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.534+0000 7f40bbfff700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40b006e680 0x7f40b0070b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f40c4197b60 tx=0x7f40ac01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:54.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.538+0000 7f40c8a4d700 1 -- 172.21.15.42:0/2264337203 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f40b405b200 con 0x7f40c40ffda0 2024-09-17T00:01:54.535 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:54 smithi042 ceph-mon[111056]: pgmap v309: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:54 smithi043 ceph-mon[81510]: pgmap v309: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:54.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.769+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40c4197190 con 0x7f40b006e680 2024-09-17T00:01:54.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.770+0000 7f40c8a4d700 1 -- 172.21.15.42:0/2264337203 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f40c4197190 con 0x7f40b006e680 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40b006e680 msgr2=0x7f40b0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40b006e680 0x7f40b0070b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f40c4197b60 tx=0x7f40ac01a040 comp rx=0 tx=0).stop 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c40ffda0 msgr2=0x7f40c419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c40ffda0 0x7f40c419c670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f40b400d8d0 tx=0x7f40b400dbe0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 shutdown_connections 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f40b006e680 0x7f40b0070b40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40c40ffda0 0x7f40c419c670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 --2- 172.21.15.42:0/2264337203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40c4100750 0x7f40c419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 >> 172.21.15.42:0/2264337203 conn(0x7f40c40fb650 msgr2=0x7f40c4069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:54.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 shutdown_connections 2024-09-17T00:01:54.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.772+0000 7f40c9a4f700 1 -- 172.21.15.42:0/2264337203 wait complete. 2024-09-17T00:01:54.781 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:01:54.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 -- 172.21.15.42:0/4189517100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 msgr2=0x7fae701058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:54.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 --2- 172.21.15.42:0/4189517100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae701058a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fae64009910 tx=0x7fae64009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:54.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 -- 172.21.15.42:0/4189517100 shutdown_connections 2024-09-17T00:01:54.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 --2- 172.21.15.42:0/4189517100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae701058a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 --2- 172.21.15.42:0/4189517100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae70100ea0 0x7fae70101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 -- 172.21.15.42:0/4189517100 >> 172.21.15.42:0/4189517100 conn(0x7fae700fc750 msgr2=0x7fae700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:54.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 -- 172.21.15.42:0/4189517100 shutdown_connections 2024-09-17T00:01:54.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.882+0000 7fae7591d700 1 -- 172.21.15.42:0/4189517100 wait complete. 2024-09-17T00:01:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.883+0000 7fae7591d700 1 Processor -- start 2024-09-17T00:01:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.883+0000 7fae7591d700 1 -- start start 2024-09-17T00:01:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.883+0000 7fae7591d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae70100ea0 0x7fae7019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.883+0000 7fae7591d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae7019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.883+0000 7fae7591d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7019d2d0 con 0x7fae70101850 2024-09-17T00:01:54.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.883+0000 7fae7591d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae70196780 con 0x7fae70100ea0 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae7019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae7019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60440/0 (socket says 172.21.15.42:60440) 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6e7fc700 1 -- 172.21.15.42:0/1001874432 learned_addr learned my addr 172.21.15.42:0/1001874432 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6effd700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae70100ea0 0x7fae7019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6e7fc700 1 -- 172.21.15.42:0/1001874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae70100ea0 msgr2=0x7fae7019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6e7fc700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae70100ea0 0x7fae7019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6e7fc700 1 -- 172.21.15.42:0/1001874432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae640095f0 con 0x7fae70101850 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae6e7fc700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae7019cc40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae640092e0 tx=0x7fae6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae7491b700 1 -- 172.21.15.42:0/1001874432 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6401d070 con 0x7fae70101850 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae7491b700 1 -- 172.21.15.42:0/1001874432 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae64005dc0 con 0x7fae70101850 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae7491b700 1 -- 172.21.15.42:0/1001874432 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae64017400 con 0x7fae70101850 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae70196a00 con 0x7fae70101850 2024-09-17T00:01:54.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.884+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae70196ef0 con 0x7fae70101850 2024-09-17T00:01:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.885+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae701092a0 con 0x7fae70101850 2024-09-17T00:01:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.886+0000 7fae7491b700 1 -- 172.21.15.42:0/1001874432 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fae6400f460 con 0x7fae70101850 2024-09-17T00:01:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.886+0000 7fae7491b700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fae5806e680 0x7fae58070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:54.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.887+0000 7fae7491b700 1 -- 172.21.15.42:0/1001874432 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fae6408faa0 con 0x7fae70101850 2024-09-17T00:01:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.887+0000 7fae6effd700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fae5806e680 0x7fae58070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:54.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.887+0000 7fae6effd700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fae5806e680 0x7fae58070b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae60005fd0 tx=0x7fae60005f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:54.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:54.891+0000 7fae7491b700 1 -- 172.21.15.42:0/1001874432 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fae6405c7f0 con 0x7fae70101850 2024-09-17T00:01:55.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.123+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae70197220 con 0x7fae5806e680 2024-09-17T00:01:55.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.125+0000 7fae7491b700 1 -- 172.21.15.42:0/1001874432 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fae70197220 con 0x7fae5806e680 2024-09-17T00:01:55.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fae5806e680 msgr2=0x7fae58070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fae5806e680 0x7fae58070b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fae60005fd0 tx=0x7fae60005f60 comp rx=0 tx=0).stop 2024-09-17T00:01:55.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 msgr2=0x7fae7019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae7019cc40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fae640092e0 tx=0x7fae6400be20 comp rx=0 tx=0).stop 2024-09-17T00:01:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 shutdown_connections 2024-09-17T00:01:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fae5806e680 0x7fae58070b40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae70100ea0 0x7fae7019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 --2- 172.21.15.42:0/1001874432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fae70101850 0x7fae7019cc40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 >> 172.21.15.42:0/1001874432 conn(0x7fae700fc750 msgr2=0x7fae700fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 shutdown_connections 2024-09-17T00:01:55.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.127+0000 7fae7591d700 1 -- 172.21.15.42:0/1001874432 wait complete. 2024-09-17T00:01:55.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 -- 172.21.15.42:0/1671293880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 msgr2=0x7ff57c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 --2- 172.21.15.42:0/1671293880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c101260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff570009910 tx=0x7ff570009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:55.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 -- 172.21.15.42:0/1671293880 shutdown_connections 2024-09-17T00:01:55.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 --2- 172.21.15.42:0/1671293880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c101830 0x7ff57c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 --2- 172.21.15.42:0/1671293880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c101260 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 -- 172.21.15.42:0/1671293880 >> 172.21.15.42:0/1671293880 conn(0x7ff57c0fc710 msgr2=0x7ff57c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:55.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 -- 172.21.15.42:0/1671293880 shutdown_connections 2024-09-17T00:01:55.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.234+0000 7ff582b37700 1 -- 172.21.15.42:0/1671293880 wait complete. 2024-09-17T00:01:55.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff582b37700 1 Processor -- start 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff582b37700 1 -- start start 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff582b37700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff582b37700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c101830 0x7ff57c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff582b37700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c073b10 con 0x7ff57c100e80 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff582b37700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c073c50 con 0x7ff57c101830 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff5808d3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff5808d3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60454/0 (socket says 172.21.15.42:60454) 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.235+0000 7ff5808d3700 1 -- 172.21.15.42:0/266146203 learned_addr learned my addr 172.21.15.42:0/266146203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff57bfff700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c101830 0x7ff57c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff5808d3700 1 -- 172.21.15.42:0/266146203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c101830 msgr2=0x7ff57c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff5808d3700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c101830 0x7ff57c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff5808d3700 1 -- 172.21.15.42:0/266146203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5700095f0 con 0x7ff57c100e80 2024-09-17T00:01:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff5808d3700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c073090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff570006010 tx=0x7ff57000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff579ffb700 1 -- 172.21.15.42:0/266146203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff57001d070 con 0x7ff57c100e80 2024-09-17T00:01:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff579ffb700 1 -- 172.21.15.42:0/266146203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff570005dc0 con 0x7ff57c100e80 2024-09-17T00:01:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c073ed0 con 0x7ff57c100e80 2024-09-17T00:01:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff579ffb700 1 -- 172.21.15.42:0/266146203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff570017400 con 0x7ff57c100e80 2024-09-17T00:01:55.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.236+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c1a2d80 con 0x7ff57c100e80 2024-09-17T00:01:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.237+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c109220 con 0x7ff57c100e80 2024-09-17T00:01:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.238+0000 7ff579ffb700 1 -- 172.21.15.42:0/266146203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff570003a60 con 0x7ff57c100e80 2024-09-17T00:01:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.239+0000 7ff579ffb700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff56c06e680 0x7ff56c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.239+0000 7ff579ffb700 1 -- 172.21.15.42:0/266146203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff57008fad0 con 0x7ff57c100e80 2024-09-17T00:01:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.239+0000 7ff57bfff700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff56c06e680 0x7ff56c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.239+0000 7ff57bfff700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff56c06e680 0x7ff56c070b40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074880 tx=0x7ff56800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:55.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.243+0000 7ff579ffb700 1 -- 172.21.15.42:0/266146203 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff570058c70 con 0x7ff57c100e80 2024-09-17T00:01:55.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.463+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff57c0611d0 con 0x7ff56c06e680 2024-09-17T00:01:55.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.473+0000 7ff579ffb700 1 -- 172.21.15.42:0/266146203 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7ff57c0611d0 con 0x7ff56c06e680 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (10m) 4m ago 21m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (21m) 4m ago 21m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (20m) 3m ago 20m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (8m) 4m ago 21m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (8m) 3m ago 20m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (10m) 4m ago 21m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (4m) 4m ago 15m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (4m) 4m ago 15m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (4m) 3m ago 15m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (3m) 3m ago 15m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:01:55.471 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (12m) 4m ago 22m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (11m) 3m ago 20m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (9m) 4m ago 22m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (8m) 3m ago 20m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (11m) 4m ago 21m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (11m) 3m ago 20m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 4m ago 19m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (7m) 4m ago 18m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (6m) 4m ago 18m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (5m) 3m ago 18m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (5m) 3m ago 17m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (4m) 3m ago 17m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:01:55.472 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (10m) 4m ago 20m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:01:55.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff56c06e680 msgr2=0x7ff56c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff56c06e680 0x7ff56c070b40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff57c074880 tx=0x7ff56800b410 comp rx=0 tx=0).stop 2024-09-17T00:01:55.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 msgr2=0x7ff57c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c073090 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff570006010 tx=0x7ff57000bd70 comp rx=0 tx=0).stop 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 shutdown_connections 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff56c06e680 0x7ff56c070b40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff57c100e80 0x7ff57c073090 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 --2- 172.21.15.42:0/266146203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff57c101830 0x7ff57c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 >> 172.21.15.42:0/266146203 conn(0x7ff57c0fc710 msgr2=0x7ff57c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 shutdown_connections 2024-09-17T00:01:55.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.475+0000 7ff582b37700 1 -- 172.21.15.42:0/266146203 wait complete. 2024-09-17T00:01:55.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 -- 172.21.15.42:0/3119998766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 msgr2=0x7fd8481058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 --2- 172.21.15.42:0/3119998766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd8481058a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009910 tx=0x7fd83c009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 -- 172.21.15.42:0/3119998766 shutdown_connections 2024-09-17T00:01:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 --2- 172.21.15.42:0/3119998766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd8481058a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 --2- 172.21.15.42:0/3119998766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd848100ea0 0x7fd848101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 -- 172.21.15.42:0/3119998766 >> 172.21.15.42:0/3119998766 conn(0x7fd8480fc750 msgr2=0x7fd8480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 -- 172.21.15.42:0/3119998766 shutdown_connections 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.582+0000 7fd84cab8700 1 -- 172.21.15.42:0/3119998766 wait complete. 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.583+0000 7fd84cab8700 1 Processor -- start 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.583+0000 7fd84cab8700 1 -- start start 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.583+0000 7fd84cab8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd848100ea0 0x7fd84819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.583+0000 7fd84cab8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd84819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.583+0000 7fd84cab8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84819d250 con 0x7fd848101850 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.583+0000 7fd84cab8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd848196700 con 0x7fd848100ea0 2024-09-17T00:01:55.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd845d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd84819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd845d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd84819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60468/0 (socket says 172.21.15.42:60468) 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd845d9b700 1 -- 172.21.15.42:0/1068279169 learned_addr learned my addr 172.21.15.42:0/1068279169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd845d9b700 1 -- 172.21.15.42:0/1068279169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd848100ea0 msgr2=0x7fd84819c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd845d9b700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd848100ea0 0x7fd84819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd845d9b700 1 -- 172.21.15.42:0/1068279169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0095f0 con 0x7fd848101850 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd845d9b700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd84819cbc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00bd70 tx=0x7fd83c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd8377fe700 1 -- 172.21.15.42:0/1068279169 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd83c01d070 con 0x7fd848101850 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd8377fe700 1 -- 172.21.15.42:0/1068279169 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd83c005dc0 con 0x7fd848101850 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd848196980 con 0x7fd848101850 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd8377fe700 1 -- 172.21.15.42:0/1068279169 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd83c017400 con 0x7fd848101850 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.584+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd848196e70 con 0x7fd848101850 2024-09-17T00:01:55.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.585+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8481091b0 con 0x7fd848101850 2024-09-17T00:01:55.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.586+0000 7fd8377fe700 1 -- 172.21.15.42:0/1068279169 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd83c00f7f0 con 0x7fd848101850 2024-09-17T00:01:55.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.586+0000 7fd8377fe700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd83006e680 0x7fd830070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.586+0000 7fd8377fe700 1 -- 172.21.15.42:0/1068279169 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd83c08fad0 con 0x7fd848101850 2024-09-17T00:01:55.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.586+0000 7fd84659c700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd83006e680 0x7fd830070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.587+0000 7fd84659c700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd83006e680 0x7fd830070b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd838005950 tx=0x7fd8380058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:55.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.591+0000 7fd8377fe700 1 -- 172.21.15.42:0/1068279169 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd83c05c820 con 0x7fd848101850 2024-09-17T00:01:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.885+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd84804f2e0 con 0x7fd848101850 2024-09-17T00:01:55.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.886+0000 7fd8377fe700 1 -- 172.21.15.42:0/1068279169 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd83c05c3b0 con 0x7fd848101850 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:01:55.884 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:01:55.885 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd83006e680 msgr2=0x7fd830070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd83006e680 0x7fd830070b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd838005950 tx=0x7fd8380058e0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 msgr2=0x7fd84819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd84819cbc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00bd70 tx=0x7fd83c00be50 comp rx=0 tx=0).stop 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 shutdown_connections 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd83006e680 0x7fd830070b40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd848100ea0 0x7fd84819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 --2- 172.21.15.42:0/1068279169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd848101850 0x7fd84819cbc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 >> 172.21.15.42:0/1068279169 conn(0x7fd8480fc750 msgr2=0x7fd848105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 shutdown_connections 2024-09-17T00:01:55.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.888+0000 7fd84cab8700 1 -- 172.21.15.42:0/1068279169 wait complete. 2024-09-17T00:01:55.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.996+0000 7fdb8e19b700 1 -- 172.21.15.42:0/3369989067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 msgr2=0x7fdb88101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.996+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/3369989067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb88101280 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009910 tx=0x7fdb78009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:55.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.996+0000 7fdb8e19b700 1 -- 172.21.15.42:0/3369989067 shutdown_connections 2024-09-17T00:01:55.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.996+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/3369989067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101850 0x7fdb881058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.996+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/3369989067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb88101280 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.996+0000 7fdb8e19b700 1 -- 172.21.15.42:0/3369989067 >> 172.21.15.42:0/3369989067 conn(0x7fdb880fc750 msgr2=0x7fdb880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:55.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.997+0000 7fdb8e19b700 1 -- 172.21.15.42:0/3369989067 shutdown_connections 2024-09-17T00:01:55.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.997+0000 7fdb8e19b700 1 -- 172.21.15.42:0/3369989067 wait complete. 2024-09-17T00:01:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.997+0000 7fdb8e19b700 1 Processor -- start 2024-09-17T00:01:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.997+0000 7fdb8e19b700 1 -- start start 2024-09-17T00:01:55.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb8e19b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb8819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb8e19b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101850 0x7fdb8819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb8e19b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8819d2c0 con 0x7fdb88100ea0 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb8e19b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88196770 con 0x7fdb88101850 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb8819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb8819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60492/0 (socket says 172.21.15.42:60492) 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb877fe700 1 -- 172.21.15.42:0/2396271152 learned_addr learned my addr 172.21.15.42:0/2396271152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb86ffd700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101850 0x7fdb8819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb877fe700 1 -- 172.21.15.42:0/2396271152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101850 msgr2=0x7fdb8819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb877fe700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101850 0x7fdb8819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb877fe700 1 -- 172.21.15.42:0/2396271152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb780095f0 con 0x7fdb88100ea0 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb86ffd700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101850 0x7fdb8819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb877fe700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb8819c6f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdb780092e0 tx=0x7fdb7800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:55.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb84ff9700 1 -- 172.21.15.42:0/2396271152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7801d070 con 0x7fdb88100ea0 2024-09-17T00:01:55.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb84ff9700 1 -- 172.21.15.42:0/2396271152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb78005e00 con 0x7fdb88100ea0 2024-09-17T00:01:55.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb881969f0 con 0x7fdb88100ea0 2024-09-17T00:01:55.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb84ff9700 1 -- 172.21.15.42:0/2396271152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb78017400 con 0x7fdb88100ea0 2024-09-17T00:01:55.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.998+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb88196ee0 con 0x7fdb88100ea0 2024-09-17T00:01:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:55.999+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb88109240 con 0x7fdb88100ea0 2024-09-17T00:01:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.000+0000 7fdb84ff9700 1 -- 172.21.15.42:0/2396271152 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fdb7800f460 con 0x7fdb88100ea0 2024-09-17T00:01:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.000+0000 7fdb84ff9700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdb7006e680 0x7fdb70070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.001+0000 7fdb84ff9700 1 -- 172.21.15.42:0/2396271152 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdb7808fb30 con 0x7fdb88100ea0 2024-09-17T00:01:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.001+0000 7fdb86ffd700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdb7006e680 0x7fdb70070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:55.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.001+0000 7fdb86ffd700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdb7006e680 0x7fdb70070b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c005950 tx=0x7fdb7c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:56.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.005+0000 7fdb84ff9700 1 -- 172.21.15.42:0/2396271152 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdb7805c7e0 con 0x7fdb88100ea0 2024-09-17T00:01:56.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.258+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb88197d80 con 0x7fdb88100ea0 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.258+0000 7fdb84ff9700 1 -- 172.21.15.42:0/2396271152 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fdb88197d80 con 0x7fdb88100ea0 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:01:56.256 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:01:56.257 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:01:56.258 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:56.259 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:01:56.259 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:01:56.259 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:56.259 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:56.259 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:56.259 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdb7006e680 msgr2=0x7fdb70070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdb7006e680 0x7fdb70070b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c005950 tx=0x7fdb7c0058e0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 msgr2=0x7fdb8819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb8819c6f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdb780092e0 tx=0x7fdb7800be20 comp rx=0 tx=0).stop 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 shutdown_connections 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdb7006e680 0x7fdb70070b40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb88100ea0 0x7fdb8819c6f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 --2- 172.21.15.42:0/2396271152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb88101850 0x7fdb8819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.260+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 >> 172.21.15.42:0/2396271152 conn(0x7fdb880fc750 msgr2=0x7fdb880fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.261+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 shutdown_connections 2024-09-17T00:01:56.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.261+0000 7fdb8e19b700 1 -- 172.21.15.42:0/2396271152 wait complete. 2024-09-17T00:01:56.261 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 -- 172.21.15.42:0/1283691295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c100750 msgr2=0x7f195c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 --2- 172.21.15.42:0/1283691295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c100750 0x7f195c1047a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1944009910 tx=0x7f1944009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 -- 172.21.15.42:0/1283691295 shutdown_connections 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 --2- 172.21.15.42:0/1283691295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c100750 0x7f195c1047a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 --2- 172.21.15.42:0/1283691295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195c0ffda0 0x7f195c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 -- 172.21.15.42:0/1283691295 >> 172.21.15.42:0/1283691295 conn(0x7f195c0fb650 msgr2=0x7f195c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 -- 172.21.15.42:0/1283691295 shutdown_connections 2024-09-17T00:01:56.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.370+0000 7f1960a5c700 1 -- 172.21.15.42:0/1283691295 wait complete. 2024-09-17T00:01:56.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f1960a5c700 1 Processor -- start 2024-09-17T00:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f1960a5c700 1 -- start start 2024-09-17T00:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f1960a5c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c0ffda0 0x7f195c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f1960a5c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195c100750 0x7f195c197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f1960a5c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c198300 con 0x7f195c0ffda0 2024-09-17T00:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f1960a5c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c198470 con 0x7f195c100750 2024-09-17T00:01:56.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f195a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c0ffda0 0x7f195c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f195a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c0ffda0 0x7f195c1977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60516/0 (socket says 172.21.15.42:60516) 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f195a59c700 1 -- 172.21.15.42:0/527799739 learned_addr learned my addr 172.21.15.42:0/527799739 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f1951d9b700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195c100750 0x7f195c197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f195a59c700 1 -- 172.21.15.42:0/527799739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195c100750 msgr2=0x7f195c197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.371+0000 7f195a59c700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195c100750 0x7f195c197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.372+0000 7f195a59c700 1 -- 172.21.15.42:0/527799739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19440095f0 con 0x7f195c0ffda0 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.372+0000 7f195a59c700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c0ffda0 0x7f195c1977a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f194c00d8d0 tx=0x7f194c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.372+0000 7f1953fff700 1 -- 172.21.15.42:0/527799739 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f194c009940 con 0x7f195c0ffda0 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.372+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c1a0810 con 0x7f195c0ffda0 2024-09-17T00:01:56.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.372+0000 7f1953fff700 1 -- 172.21.15.42:0/527799739 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f194c010460 con 0x7f195c0ffda0 2024-09-17T00:01:56.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.372+0000 7f1953fff700 1 -- 172.21.15.42:0/527799739 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f194c00f5f0 con 0x7f195c0ffda0 2024-09-17T00:01:56.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.372+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c1a0cd0 con 0x7f195c0ffda0 2024-09-17T00:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.373+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c1081a0 con 0x7f195c0ffda0 2024-09-17T00:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.374+0000 7f1953fff700 1 -- 172.21.15.42:0/527799739 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f194c0105d0 con 0x7f195c0ffda0 2024-09-17T00:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.374+0000 7f1953fff700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f194806e680 0x7f1948070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.374+0000 7f1953fff700 1 -- 172.21.15.42:0/527799739 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f194c08e680 con 0x7f195c0ffda0 2024-09-17T00:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.374+0000 7f1951d9b700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f194806e680 0x7f1948070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:56.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.375+0000 7f1951d9b700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f194806e680 0x7f1948070b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1944006010 tx=0x7f194401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:56.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.378+0000 7f1953fff700 1 -- 172.21.15.42:0/527799739 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f194c057820 con 0x7f195c0ffda0 2024-09-17T00:01:56.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:56 smithi043 ceph-mon[81510]: pgmap v310: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:56.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:56 smithi043 ceph-mon[81510]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:56.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:56 smithi043 ceph-mon[81510]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:56.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:56 smithi043 ceph-mon[81510]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:56.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:56 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1068279169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:01:56.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:56 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2396271152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:01:56.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:56 smithi042 ceph-mon[111056]: pgmap v310: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:56 smithi042 ceph-mon[111056]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:56 smithi042 ceph-mon[111056]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:56 smithi042 ceph-mon[111056]: from='client.34634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:56 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1068279169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:01:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:56 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2396271152' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:01:56.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.608+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f195c069530 con 0x7f194806e680 2024-09-17T00:01:56.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.609+0000 7f1953fff700 1 -- 172.21.15.42:0/527799739 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f195c069530 con 0x7f194806e680 2024-09-17T00:01:56.607 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:01:56.607 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:01:56.607 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:01:56.608 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:01:56.609 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:01:56.609 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:01:56.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f194806e680 msgr2=0x7f1948070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f194806e680 0x7f1948070b40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1944006010 tx=0x7f194401a040 comp rx=0 tx=0).stop 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c0ffda0 msgr2=0x7f195c1977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c0ffda0 0x7f195c1977a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f194c00d8d0 tx=0x7f194c00dc90 comp rx=0 tx=0).stop 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 shutdown_connections 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f194806e680 0x7f1948070b40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f195c0ffda0 0x7f195c1977a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 --2- 172.21.15.42:0/527799739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f195c100750 0x7f195c197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.611+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 >> 172.21.15.42:0/527799739 conn(0x7f195c0fb650 msgr2=0x7f195c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.612+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 shutdown_connections 2024-09-17T00:01:56.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.612+0000 7f1960a5c700 1 -- 172.21.15.42:0/527799739 wait complete. 2024-09-17T00:01:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 -- 172.21.15.42:0/2813448166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 msgr2=0x7f5ac810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 --2- 172.21.15.42:0/2813448166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac810ac70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009910 tx=0x7f5ab0009c20 comp rx=0 tx=0).stop 2024-09-17T00:01:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 -- 172.21.15.42:0/2813448166 shutdown_connections 2024-09-17T00:01:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 --2- 172.21.15.42:0/2813448166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac810ac70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 --2- 172.21.15.42:0/2813448166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac8101e60 0x7f5ac8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 -- 172.21.15.42:0/2813448166 >> 172.21.15.42:0/2813448166 conn(0x7f5ac80fb650 msgr2=0x7f5ac80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:56.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 -- 172.21.15.42:0/2813448166 shutdown_connections 2024-09-17T00:01:56.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.719+0000 7f5acd135700 1 -- 172.21.15.42:0/2813448166 wait complete. 2024-09-17T00:01:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.720+0000 7f5acd135700 1 Processor -- start 2024-09-17T00:01:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.720+0000 7f5acd135700 1 -- start start 2024-09-17T00:01:56.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.720+0000 7f5acd135700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac8101e60 0x7f5ac819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.720+0000 7f5acd135700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.720+0000 7f5acd135700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac819d270 con 0x7f5ac8102780 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.720+0000 7f5acd135700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8196720 con 0x7f5ac8101e60 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.721+0000 7f5abe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.721+0000 7f5abe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60526/0 (socket says 172.21.15.42:60526) 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.721+0000 7f5abe59c700 1 -- 172.21.15.42:0/3517141820 learned_addr learned my addr 172.21.15.42:0/3517141820 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.721+0000 7f5ac6d9d700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac8101e60 0x7f5ac819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.721+0000 7f5abe59c700 1 -- 172.21.15.42:0/3517141820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac8101e60 msgr2=0x7f5ac819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.721+0000 7f5abe59c700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac8101e60 0x7f5ac819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.721+0000 7f5abe59c700 1 -- 172.21.15.42:0/3517141820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab00095f0 con 0x7f5ac8102780 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.722+0000 7f5abe59c700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac819cbe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000bda0 tx=0x7f5ab000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.722+0000 7f5ac4d99700 1 -- 172.21.15.42:0/3517141820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab001d070 con 0x7f5ac8102780 2024-09-17T00:01:56.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.722+0000 7f5ac4d99700 1 -- 172.21.15.42:0/3517141820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ab000f460 con 0x7f5ac8102780 2024-09-17T00:01:56.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.722+0000 7f5ac4d99700 1 -- 172.21.15.42:0/3517141820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ab0017630 con 0x7f5ac8102780 2024-09-17T00:01:56.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.722+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac81969a0 con 0x7f5ac8102780 2024-09-17T00:01:56.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.722+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac8196e90 con 0x7f5ac8102780 2024-09-17T00:01:56.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.723+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac8108300 con 0x7f5ac8102780 2024-09-17T00:01:56.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.723+0000 7f5ac4d99700 1 -- 172.21.15.42:0/3517141820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5ab0003a60 con 0x7f5ac8102780 2024-09-17T00:01:56.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.724+0000 7f5ac4d99700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ab406e680 0x7f5ab4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:01:56.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.724+0000 7f5ac4d99700 1 -- 172.21.15.42:0/3517141820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5ab008fa50 con 0x7f5ac8102780 2024-09-17T00:01:56.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.724+0000 7f5ac6d9d700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ab406e680 0x7f5ab4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:01:56.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.724+0000 7f5ac6d9d700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ab406e680 0x7f5ab4070b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005950 tx=0x7f5ab800a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:01:56.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:56.728+0000 7f5ac4d99700 1 -- 172.21.15.42:0/3517141820 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5ab005c7a0 con 0x7f5ac8102780 2024-09-17T00:01:57.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.026+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5ac804f2e0 con 0x7f5ac8102780 2024-09-17T00:01:57.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.026+0000 7f5ac4d99700 1 -- 172.21.15.42:0/3517141820 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5ab005c330 con 0x7f5ac8102780 2024-09-17T00:01:57.024 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:01:57.024 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:01:57.024 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:01:57.024 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:01:57.025 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:01:57.025 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:01:57.025 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:01:57.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.028+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ab406e680 msgr2=0x7f5ab4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:57.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.028+0000 7f5acd135700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ab406e680 0x7f5ab4070b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005950 tx=0x7f5ab800a300 comp rx=0 tx=0).stop 2024-09-17T00:01:57.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.028+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 msgr2=0x7f5ac819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:01:57.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.028+0000 7f5acd135700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac819cbe0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000bda0 tx=0x7f5ab000bdd0 comp rx=0 tx=0).stop 2024-09-17T00:01:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.029+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 shutdown_connections 2024-09-17T00:01:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.029+0000 7f5acd135700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ab406e680 0x7f5ab4070b40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.029+0000 7f5acd135700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ac8101e60 0x7f5ac819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.029+0000 7f5acd135700 1 --2- 172.21.15.42:0/3517141820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ac8102780 0x7f5ac819cbe0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:01:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.029+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 >> 172.21.15.42:0/3517141820 conn(0x7f5ac80fb650 msgr2=0x7f5ac8105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:01:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.029+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 shutdown_connections 2024-09-17T00:01:57.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:01:57.029+0000 7f5acd135700 1 -- 172.21.15.42:0/3517141820 wait complete. 2024-09-17T00:01:57.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:57 smithi043 ceph-mon[81510]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:57.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:57 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3517141820' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:01:57.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:57 smithi042 ceph-mon[111056]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:01:57.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:57 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3517141820' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:01:58.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:01:58 smithi043 ceph-mon[81510]: pgmap v311: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:01:58.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:01:58 smithi042 ceph-mon[111056]: pgmap v311: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:00.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:00 smithi043 ceph-mon[81510]: pgmap v312: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:00.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:00 smithi042 ceph-mon[111056]: pgmap v312: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:02 smithi043 ceph-mon[81510]: pgmap v313: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:02 smithi042 ceph-mon[111056]: pgmap v313: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:04.630 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:04 smithi043 ceph-mon[81510]: pgmap v314: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:04 smithi042 ceph-mon[111056]: pgmap v314: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:06 smithi043 ceph-mon[81510]: pgmap v315: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:06.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:06 smithi042 ceph-mon[111056]: pgmap v315: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:08 smithi043 ceph-mon[81510]: pgmap v316: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:08 smithi042 ceph-mon[111056]: pgmap v316: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:10.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:10 smithi043 ceph-mon[81510]: pgmap v317: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:10.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:10 smithi042 ceph-mon[111056]: pgmap v317: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:12 smithi043 ceph-mon[81510]: pgmap v318: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:12 smithi042 ceph-mon[111056]: pgmap v318: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:14.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:14 smithi043 ceph-mon[81510]: pgmap v319: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:14.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:14 smithi042 ceph-mon[111056]: pgmap v319: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:16.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:16 smithi043 ceph-mon[81510]: pgmap v320: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:16.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:16 smithi042 ceph-mon[111056]: pgmap v320: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:18 smithi043 ceph-mon[81510]: pgmap v321: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:18.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:18 smithi042 ceph-mon[111056]: pgmap v321: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:20.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:20 smithi043 ceph-mon[81510]: pgmap v322: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:20.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:20 smithi042 ceph-mon[111056]: pgmap v322: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:21.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:22.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:22 smithi043 ceph-mon[81510]: pgmap v323: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:22.672 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:22 smithi042 ceph-mon[111056]: pgmap v323: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:24 smithi043 ceph-mon[81510]: pgmap v324: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:24.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:24 smithi042 ceph-mon[111056]: pgmap v324: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:26.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:26 smithi043 ceph-mon[81510]: pgmap v325: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:26 smithi042 ceph-mon[111056]: pgmap v325: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.147+0000 7fb72092b700 1 -- 172.21.15.42:0/1508745342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c0ffda0 msgr2=0x7fb71c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 --2- 172.21.15.42:0/1508745342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c0ffda0 0x7fb71c100180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009910 tx=0x7fb70c009c20 comp rx=0 tx=0).stop 2024-09-17T00:02:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 -- 172.21.15.42:0/1508745342 shutdown_connections 2024-09-17T00:02:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 --2- 172.21.15.42:0/1508745342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb71c100750 0x7fb71c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 --2- 172.21.15.42:0/1508745342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c0ffda0 0x7fb71c100180 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 -- 172.21.15.42:0/1508745342 >> 172.21.15.42:0/1508745342 conn(0x7fb71c0fb650 msgr2=0x7fb71c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:27.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 -- 172.21.15.42:0/1508745342 shutdown_connections 2024-09-17T00:02:27.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 -- 172.21.15.42:0/1508745342 wait complete. 2024-09-17T00:02:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.148+0000 7fb72092b700 1 Processor -- start 2024-09-17T00:02:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb72092b700 1 -- start start 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb72092b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb71c0ffda0 0x7fb71c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb72092b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c100750 0x7fb71c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb72092b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c1022c0 con 0x7fb71c100750 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb72092b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c102400 con 0x7fb71c0ffda0 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb719d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c100750 0x7fb71c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb719d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c100750 0x7fb71c101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50346/0 (socket says 172.21.15.42:50346) 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb719d9b700 1 -- 172.21.15.42:0/234800402 learned_addr learned my addr 172.21.15.42:0/234800402 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb71a59c700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb71c0ffda0 0x7fb71c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb719d9b700 1 -- 172.21.15.42:0/234800402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb71c0ffda0 msgr2=0x7fb71c101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb719d9b700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb71c0ffda0 0x7fb71c101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb719d9b700 1 -- 172.21.15.42:0/234800402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0095f0 con 0x7fb71c100750 2024-09-17T00:02:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.149+0000 7fb719d9b700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c100750 0x7fb71c101d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e8f0 tx=0x7fb71000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.150+0000 7fb70b7fe700 1 -- 172.21.15.42:0/234800402 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb710009790 con 0x7fb71c100750 2024-09-17T00:02:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.150+0000 7fb70b7fe700 1 -- 172.21.15.42:0/234800402 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb71000c810 con 0x7fb71c100750 2024-09-17T00:02:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.150+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71c1a2af0 con 0x7fb71c100750 2024-09-17T00:02:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.150+0000 7fb70b7fe700 1 -- 172.21.15.42:0/234800402 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb710005580 con 0x7fb71c100750 2024-09-17T00:02:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.150+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71c1a2f00 con 0x7fb71c100750 2024-09-17T00:02:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.151+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71c04f2e0 con 0x7fb71c100750 2024-09-17T00:02:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.151+0000 7fb70b7fe700 1 -- 172.21.15.42:0/234800402 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb71000c980 con 0x7fb71c100750 2024-09-17T00:02:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.152+0000 7fb70b7fe700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb70406e680 0x7fb704070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.152+0000 7fb70b7fe700 1 -- 172.21.15.42:0/234800402 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb710014070 con 0x7fb71c100750 2024-09-17T00:02:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.152+0000 7fb71a59c700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb70406e680 0x7fb704070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.152+0000 7fb71a59c700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb70406e680 0x7fb704070b40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00b760 tx=0x7fb70c005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:27.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.156+0000 7fb70b7fe700 1 -- 172.21.15.42:0/234800402 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb710057a60 con 0x7fb71c100750 2024-09-17T00:02:27.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.384+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb71c0fc480 con 0x7fb70406e680 2024-09-17T00:02:27.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.385+0000 7fb70b7fe700 1 -- 172.21.15.42:0/234800402 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb71c0fc480 con 0x7fb70406e680 2024-09-17T00:02:27.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb70406e680 msgr2=0x7fb704070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb70406e680 0x7fb704070b40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00b760 tx=0x7fb70c005a90 comp rx=0 tx=0).stop 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c100750 msgr2=0x7fb71c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c100750 0x7fb71c101d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb71000e8f0 tx=0x7fb71000ec00 comp rx=0 tx=0).stop 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 shutdown_connections 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb70406e680 0x7fb704070b40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb71c0ffda0 0x7fb71c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 --2- 172.21.15.42:0/234800402 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb71c100750 0x7fb71c101d80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 >> 172.21.15.42:0/234800402 conn(0x7fb71c0fb650 msgr2=0x7fb71c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 shutdown_connections 2024-09-17T00:02:27.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.388+0000 7fb72092b700 1 -- 172.21.15.42:0/234800402 wait complete. 2024-09-17T00:02:27.396 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:02:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 -- 172.21.15.42:0/2587749412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 msgr2=0x7f13c4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 --2- 172.21.15.42:0/2587749412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4101280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f13b8009910 tx=0x7f13b8009c20 comp rx=0 tx=0).stop 2024-09-17T00:02:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 -- 172.21.15.42:0/2587749412 shutdown_connections 2024-09-17T00:02:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 --2- 172.21.15.42:0/2587749412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c4101850 0x7f13c41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 --2- 172.21.15.42:0/2587749412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4101280 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 -- 172.21.15.42:0/2587749412 >> 172.21.15.42:0/2587749412 conn(0x7f13c40fc750 msgr2=0x7f13c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:27.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 -- 172.21.15.42:0/2587749412 shutdown_connections 2024-09-17T00:02:27.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 -- 172.21.15.42:0/2587749412 wait complete. 2024-09-17T00:02:27.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 Processor -- start 2024-09-17T00:02:27.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.497+0000 7f13cbfda700 1 -- start start 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13cbfda700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13cbfda700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c4101850 0x7f13c40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13cbfda700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4073b10 con 0x7f13c4100ea0 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13cbfda700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4073c50 con 0x7f13c4101850 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9d76700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9d76700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50356/0 (socket says 172.21.15.42:50356) 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9d76700 1 -- 172.21.15.42:0/1890189243 learned_addr learned my addr 172.21.15.42:0/1890189243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9575700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c4101850 0x7f13c40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9d76700 1 -- 172.21.15.42:0/1890189243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c4101850 msgr2=0x7f13c40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9d76700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c4101850 0x7f13c40735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9d76700 1 -- 172.21.15.42:0/1890189243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b80095f0 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9575700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c4101850 0x7f13c40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13c9d76700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4073090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f13b8006010 tx=0x7f13b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13b6ffd700 1 -- 172.21.15.42:0/1890189243 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b801d070 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c4073ed0 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13b6ffd700 1 -- 172.21.15.42:0/1890189243 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13b8004fb0 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.498+0000 7f13b6ffd700 1 -- 172.21.15.42:0/1890189243 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b8022ea0 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.499+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c41a2e90 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.500+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c4109240 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.500+0000 7f13b6ffd700 1 -- 172.21.15.42:0/1890189243 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f13b8022470 con 0x7f13c4100ea0 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.500+0000 7f13b6ffd700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f13b006e680 0x7f13b0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.501+0000 7f13b6ffd700 1 -- 172.21.15.42:0/1890189243 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f13b808fbd0 con 0x7f13c4100ea0 2024-09-17T00:02:27.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.501+0000 7f13c9575700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f13b006e680 0x7f13b0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.501+0000 7f13c9575700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f13b006e680 0x7f13b0070b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13c4074880 tx=0x7f13c0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:27.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.505+0000 7f13b6ffd700 1 -- 172.21.15.42:0/1890189243 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f13b8058e20 con 0x7f13c4100ea0 2024-09-17T00:02:27.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.736+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13c40746d0 con 0x7f13b006e680 2024-09-17T00:02:27.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.738+0000 7f13b6ffd700 1 -- 172.21.15.42:0/1890189243 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f13c40746d0 con 0x7f13b006e680 2024-09-17T00:02:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f13b006e680 msgr2=0x7f13b0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f13b006e680 0x7f13b0070b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f13c4074880 tx=0x7f13c0008040 comp rx=0 tx=0).stop 2024-09-17T00:02:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 msgr2=0x7f13c4073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4073090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f13b8006010 tx=0x7f13b800be20 comp rx=0 tx=0).stop 2024-09-17T00:02:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 shutdown_connections 2024-09-17T00:02:27.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f13b006e680 0x7f13b0070b40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f13c4100ea0 0x7f13c4073090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 --2- 172.21.15.42:0/1890189243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13c4101850 0x7f13c40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 >> 172.21.15.42:0/1890189243 conn(0x7f13c40fc750 msgr2=0x7f13c40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:27.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 shutdown_connections 2024-09-17T00:02:27.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.740+0000 7f13cbfda700 1 -- 172.21.15.42:0/1890189243 wait complete. 2024-09-17T00:02:27.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 -- 172.21.15.42:0/722955133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 msgr2=0x7f8d7c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 --2- 172.21.15.42:0/722955133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c102220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009940 tx=0x7f8d64009c50 comp rx=0 tx=0).stop 2024-09-17T00:02:27.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 -- 172.21.15.42:0/722955133 shutdown_connections 2024-09-17T00:02:27.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 --2- 172.21.15.42:0/722955133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d7c102760 0x7f8d7c10ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 --2- 172.21.15.42:0/722955133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c102220 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 -- 172.21.15.42:0/722955133 >> 172.21.15.42:0/722955133 conn(0x7f8d7c0fb610 msgr2=0x7f8d7c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:27.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 -- 172.21.15.42:0/722955133 shutdown_connections 2024-09-17T00:02:27.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.856+0000 7f8d8315a700 1 -- 172.21.15.42:0/722955133 wait complete. 2024-09-17T00:02:27.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d8315a700 1 Processor -- start 2024-09-17T00:02:27.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d8315a700 1 -- start start 2024-09-17T00:02:27.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d8315a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c0ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d8315a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d7c102760 0x7f8d7c0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d8315a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c100520 con 0x7f8d7c101e40 2024-09-17T00:02:27.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d8315a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c100660 con 0x7f8d7c102760 2024-09-17T00:02:27.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d80ef6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c0ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d80ef6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c0ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50378/0 (socket says 172.21.15.42:50378) 2024-09-17T00:02:27.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d80ef6700 1 -- 172.21.15.42:0/3011996568 learned_addr learned my addr 172.21.15.42:0/3011996568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:27.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d7bfff700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d7c102760 0x7f8d7c0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d80ef6700 1 -- 172.21.15.42:0/3011996568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d7c102760 msgr2=0x7f8d7c0fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:27.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d80ef6700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d7c102760 0x7f8d7c0fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:27.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d80ef6700 1 -- 172.21.15.42:0/3011996568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d640095f0 con 0x7f8d7c101e40 2024-09-17T00:02:27.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.857+0000 7f8d80ef6700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c0ffaa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8d640055f0 tx=0x7f8d640056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:27.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.858+0000 7f8d79ffb700 1 -- 172.21.15.42:0/3011996568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d6401d070 con 0x7f8d7c101e40 2024-09-17T00:02:27.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.858+0000 7f8d79ffb700 1 -- 172.21.15.42:0/3011996568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d6400f460 con 0x7f8d7c101e40 2024-09-17T00:02:27.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.858+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7c1008e0 con 0x7f8d7c101e40 2024-09-17T00:02:27.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.858+0000 7f8d79ffb700 1 -- 172.21.15.42:0/3011996568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d64021620 con 0x7f8d7c101e40 2024-09-17T00:02:27.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.858+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7c1a2da0 con 0x7f8d7c101e40 2024-09-17T00:02:27.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.859+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7c108350 con 0x7f8d7c101e40 2024-09-17T00:02:27.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.859+0000 7f8d79ffb700 1 -- 172.21.15.42:0/3011996568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8d6400fa60 con 0x7f8d7c101e40 2024-09-17T00:02:27.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.860+0000 7f8d79ffb700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6806e680 0x7f8d68070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:27.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.860+0000 7f8d79ffb700 1 -- 172.21.15.42:0/3011996568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d6408f9a0 con 0x7f8d7c101e40 2024-09-17T00:02:27.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.860+0000 7f8d7bfff700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6806e680 0x7f8d68070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:27.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.861+0000 7f8d7bfff700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6806e680 0x7f8d68070b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c101290 tx=0x7f8d6c007b80 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:27.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:27.864+0000 7f8d79ffb700 1 -- 172.21.15.42:0/3011996568 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8d64058b40 con 0x7f8d7c101e40 2024-09-17T00:02:28.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.085+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d7c0611d0 con 0x7f8d6806e680 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.094+0000 7f8d79ffb700 1 -- 172.21.15.42:0/3011996568 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f8d7c0611d0 con 0x7f8d6806e680 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (11m) 4m ago 21m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (22m) 4m ago 22m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (21m) 4m ago 21m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (8m) 4m ago 22m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (8m) 4m ago 20m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:02:28.092 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (10m) 4m ago 21m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (4m) 4m ago 16m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (5m) 4m ago 16m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (4m) 4m ago 16m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (4m) 4m ago 16m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (12m) 4m ago 23m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (11m) 4m ago 20m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (9m) 4m ago 23m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (9m) 4m ago 20m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (11m) 4m ago 22m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (11m) 4m ago 20m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (8m) 4m ago 20m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (7m) 4m ago 19m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:02:28.093 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (7m) 4m ago 19m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:02:28.094 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (6m) 4m ago 18m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:02:28.094 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (5m) 4m ago 18m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:02:28.094 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (5m) 4m ago 17m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:02:28.094 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (11m) 4m ago 21m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:02:28.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6806e680 msgr2=0x7f8d68070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6806e680 0x7f8d68070b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c101290 tx=0x7f8d6c007b80 comp rx=0 tx=0).stop 2024-09-17T00:02:28.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 msgr2=0x7f8d7c0ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c0ffaa0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8d640055f0 tx=0x7f8d640056d0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 shutdown_connections 2024-09-17T00:02:28.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6806e680 0x7f8d68070b40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d7c101e40 0x7f8d7c0ffaa0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 --2- 172.21.15.42:0/3011996568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d7c102760 0x7f8d7c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 >> 172.21.15.42:0/3011996568 conn(0x7f8d7c0fb610 msgr2=0x7f8d7c105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 shutdown_connections 2024-09-17T00:02:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.097+0000 7f8d8315a700 1 -- 172.21.15.42:0/3011996568 wait complete. 2024-09-17T00:02:28.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 -- 172.21.15.42:0/2441398202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb4101e80 msgr2=0x7f6cb4102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 --2- 172.21.15.42:0/2441398202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb4101e80 0x7f6cb4102260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009910 tx=0x7f6cb0009c20 comp rx=0 tx=0).stop 2024-09-17T00:02:28.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 -- 172.21.15.42:0/2441398202 shutdown_connections 2024-09-17T00:02:28.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 --2- 172.21.15.42:0/2441398202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb41027a0 0x7f6cb410ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 --2- 172.21.15.42:0/2441398202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb4101e80 0x7f6cb4102260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 -- 172.21.15.42:0/2441398202 >> 172.21.15.42:0/2441398202 conn(0x7f6cb40fb650 msgr2=0x7f6cb40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 -- 172.21.15.42:0/2441398202 shutdown_connections 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.203+0000 7f6cbb299700 1 -- 172.21.15.42:0/2441398202 wait complete. 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.204+0000 7f6cbb299700 1 Processor -- start 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.204+0000 7f6cbb299700 1 -- start start 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.204+0000 7f6cbb299700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb4101e80 0x7f6cb419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.204+0000 7f6cbb299700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb41027a0 0x7f6cb419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.204+0000 7f6cbb299700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb419d380 con 0x7f6cb41027a0 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.204+0000 7f6cbb299700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb4196830 con 0x7f6cb4101e80 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.204+0000 7f6cb8834700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb41027a0 0x7f6cb419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cb8834700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb41027a0 0x7f6cb419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50398/0 (socket says 172.21.15.42:50398) 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cb8834700 1 -- 172.21.15.42:0/3638921745 learned_addr learned my addr 172.21.15.42:0/3638921745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:28.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cb9035700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb4101e80 0x7f6cb419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:28.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cb8834700 1 -- 172.21.15.42:0/3638921745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb4101e80 msgr2=0x7f6cb419c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cb8834700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb4101e80 0x7f6cb419c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cb8834700 1 -- 172.21.15.42:0/3638921745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00095f0 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cb8834700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb41027a0 0x7f6cb419ccf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b480 tx=0x7f6ca400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6caa7fc700 1 -- 172.21.15.42:0/3638921745 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ca4011800 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6caa7fc700 1 -- 172.21.15.42:0/3638921745 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ca4011e20 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb4196b10 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.205+0000 7f6caa7fc700 1 -- 172.21.15.42:0/3638921745 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ca4010ea0 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.206+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb4197060 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.206+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb4108360 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.207+0000 7f6caa7fc700 1 -- 172.21.15.42:0/3638921745 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6ca4010380 con 0x7f6cb41027a0 2024-09-17T00:02:28.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.207+0000 7f6caa7fc700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ca006e680 0x7f6ca0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:28.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.207+0000 7f6caa7fc700 1 -- 172.21.15.42:0/3638921745 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6ca408e0d0 con 0x7f6cb41027a0 2024-09-17T00:02:28.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.207+0000 7f6cb9035700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ca006e680 0x7f6ca0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:28.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.208+0000 7f6cb9035700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ca006e680 0x7f6ca0070b40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0006010 tx=0x7f6cb001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:28.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.212+0000 7f6caa7fc700 1 -- 172.21.15.42:0/3638921745 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6ca405ae20 con 0x7f6cb41027a0 2024-09-17T00:02:28.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.513+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6cb41972e0 con 0x7f6cb41027a0 2024-09-17T00:02:28.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.513+0000 7f6caa7fc700 1 -- 172.21.15.42:0/3638921745 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f6ca405a9b0 con 0x7f6cb41027a0 2024-09-17T00:02:28.511 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:02:28.511 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:02:28.511 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:02:28.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:02:28.511 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:02:28.512 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:02:28.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ca006e680 msgr2=0x7f6ca0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ca006e680 0x7f6ca0070b40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0006010 tx=0x7f6cb001a040 comp rx=0 tx=0).stop 2024-09-17T00:02:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb41027a0 msgr2=0x7f6cb419ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb41027a0 0x7f6cb419ccf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b480 tx=0x7f6ca400b790 comp rx=0 tx=0).stop 2024-09-17T00:02:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 shutdown_connections 2024-09-17T00:02:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ca006e680 0x7f6ca0070b40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb4101e80 0x7f6cb419c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 --2- 172.21.15.42:0/3638921745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6cb41027a0 0x7f6cb419ccf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 >> 172.21.15.42:0/3638921745 conn(0x7f6cb40fb650 msgr2=0x7f6cb41054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 shutdown_connections 2024-09-17T00:02:28.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.516+0000 7f6cbb299700 1 -- 172.21.15.42:0/3638921745 wait complete. 2024-09-17T00:02:28.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 -- 172.21.15.42:0/151706820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240101850 msgr2=0x7fd2401058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 --2- 172.21.15.42:0/151706820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240101850 0x7fd2401058a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd230009910 tx=0x7fd230009c20 comp rx=0 tx=0).stop 2024-09-17T00:02:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 -- 172.21.15.42:0/151706820 shutdown_connections 2024-09-17T00:02:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 --2- 172.21.15.42:0/151706820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240101850 0x7fd2401058a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 --2- 172.21.15.42:0/151706820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd240100ea0 0x7fd240101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 -- 172.21.15.42:0/151706820 >> 172.21.15.42:0/151706820 conn(0x7fd2400fc750 msgr2=0x7fd2400feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:28.624 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:28 smithi042 ceph-mon[111056]: pgmap v326: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:28.624 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:28 smithi042 ceph-mon[111056]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 -- 172.21.15.42:0/151706820 shutdown_connections 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 -- 172.21.15.42:0/151706820 wait complete. 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.625+0000 7fd247c5b700 1 Processor -- start 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd247c5b700 1 -- start start 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd247c5b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100ea0 0x7fd24019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd247c5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd240101850 0x7fd24019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd247c5b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019d2d0 con 0x7fd240100ea0 2024-09-17T00:02:28.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd247c5b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd240196780 con 0x7fd240101850 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd2459f7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100ea0 0x7fd24019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd2459f7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100ea0 0x7fd24019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50420/0 (socket says 172.21.15.42:50420) 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd2459f7700 1 -- 172.21.15.42:0/1525182033 learned_addr learned my addr 172.21.15.42:0/1525182033 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd2451f6700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd240101850 0x7fd24019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd2459f7700 1 -- 172.21.15.42:0/1525182033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd240101850 msgr2=0x7fd24019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd2459f7700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd240101850 0x7fd24019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.626+0000 7fd2459f7700 1 -- 172.21.15.42:0/1525182033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2300095f0 con 0x7fd240100ea0 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.627+0000 7fd2459f7700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100ea0 0x7fd24019c700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00d8d0 tx=0x7fd23c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:28.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.627+0000 7fd236ffd700 1 -- 172.21.15.42:0/1525182033 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23c0098a0 con 0x7fd240100ea0 2024-09-17T00:02:28.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.627+0000 7fd236ffd700 1 -- 172.21.15.42:0/1525182033 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd23c010460 con 0x7fd240100ea0 2024-09-17T00:02:28.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.627+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd240196a60 con 0x7fd240100ea0 2024-09-17T00:02:28.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.627+0000 7fd236ffd700 1 -- 172.21.15.42:0/1525182033 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23c00f5f0 con 0x7fd240100ea0 2024-09-17T00:02:28.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.627+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd240196fb0 con 0x7fd240100ea0 2024-09-17T00:02:28.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.628+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2401092a0 con 0x7fd240100ea0 2024-09-17T00:02:28.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.629+0000 7fd236ffd700 1 -- 172.21.15.42:0/1525182033 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd23c009a00 con 0x7fd240100ea0 2024-09-17T00:02:28.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:28 smithi043 ceph-mon[81510]: pgmap v326: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:28.629 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:28 smithi043 ceph-mon[81510]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:28.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.629+0000 7fd236ffd700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd22c06e680 0x7fd22c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:28.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.629+0000 7fd236ffd700 1 -- 172.21.15.42:0/1525182033 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd23c08e480 con 0x7fd240100ea0 2024-09-17T00:02:28.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.629+0000 7fd2451f6700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd22c06e680 0x7fd22c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:28.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.630+0000 7fd2451f6700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd22c06e680 0x7fd22c070b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd240197bf0 tx=0x7fd23001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:28.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.634+0000 7fd236ffd700 1 -- 172.21.15.42:0/1525182033 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd23c05b1d0 con 0x7fd240100ea0 2024-09-17T00:02:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.889+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd24004f2e0 con 0x7fd240100ea0 2024-09-17T00:02:28.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.889+0000 7fd236ffd700 1 -- 172.21.15.42:0/1525182033 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fd23c05ad60 con 0x7fd240100ea0 2024-09-17T00:02:28.887 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:02:28.887 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:02:28.887 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:02:28.887 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:02:28.887 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:02:28.887 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:02:28.888 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:02:28.889 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:02:28.890 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:02:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd22c06e680 msgr2=0x7fd22c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd22c06e680 0x7fd22c070b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd240197bf0 tx=0x7fd23001a040 comp rx=0 tx=0).stop 2024-09-17T00:02:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100ea0 msgr2=0x7fd24019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100ea0 0x7fd24019c700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00d8d0 tx=0x7fd23c00dbe0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 shutdown_connections 2024-09-17T00:02:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd22c06e680 0x7fd22c070b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd240100ea0 0x7fd24019c700 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 --2- 172.21.15.42:0/1525182033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd240101850 0x7fd24019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 >> 172.21.15.42:0/1525182033 conn(0x7fd2400fc750 msgr2=0x7fd2400fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 shutdown_connections 2024-09-17T00:02:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:28.891+0000 7fd247c5b700 1 -- 172.21.15.42:0/1525182033 wait complete. 2024-09-17T00:02:28.892 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.000+0000 7f643c680700 1 -- 172.21.15.42:0/1791169334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434100ea0 msgr2=0x7f6434101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.000+0000 7f643c680700 1 --2- 172.21.15.42:0/1791169334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434100ea0 0x7f6434101280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6430009910 tx=0x7f6430009c20 comp rx=0 tx=0).stop 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 -- 172.21.15.42:0/1791169334 shutdown_connections 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 --2- 172.21.15.42:0/1791169334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6434101850 0x7f64341058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 --2- 172.21.15.42:0/1791169334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434100ea0 0x7f6434101280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 -- 172.21.15.42:0/1791169334 >> 172.21.15.42:0/1791169334 conn(0x7f64340fc750 msgr2=0x7f64340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 -- 172.21.15.42:0/1791169334 shutdown_connections 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 -- 172.21.15.42:0/1791169334 wait complete. 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 Processor -- start 2024-09-17T00:02:28.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.001+0000 7f643c680700 1 -- start start 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f643c680700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6434100ea0 0x7f643419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f643c680700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434101850 0x7f643419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f643c680700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643419d2c0 con 0x7f6434101850 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f643c680700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434196770 con 0x7f6434100ea0 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f6439c1b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434101850 0x7f643419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f6439c1b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434101850 0x7f643419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50430/0 (socket says 172.21.15.42:50430) 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f6439c1b700 1 -- 172.21.15.42:0/358371775 learned_addr learned my addr 172.21.15.42:0/358371775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f643a41c700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6434100ea0 0x7f643419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f6439c1b700 1 -- 172.21.15.42:0/358371775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6434100ea0 msgr2=0x7f643419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:29.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f6439c1b700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6434100ea0 0x7f643419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f6439c1b700 1 -- 172.21.15.42:0/358371775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64300095f0 con 0x7f6434101850 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f6439c1b700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434101850 0x7f643419cc30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f642400c8f0 tx=0x7f642400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f642b7fe700 1 -- 172.21.15.42:0/358371775 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64240078a0 con 0x7f6434101850 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f642b7fe700 1 -- 172.21.15.42:0/358371775 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f642400f450 con 0x7f6434101850 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434196a50 con 0x7f6434101850 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.002+0000 7f642b7fe700 1 -- 172.21.15.42:0/358371775 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6424018610 con 0x7f6434101850 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.003+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6434196fa0 con 0x7f6434101850 2024-09-17T00:02:29.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.003+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6434109240 con 0x7f6434101850 2024-09-17T00:02:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.004+0000 7f642b7fe700 1 -- 172.21.15.42:0/358371775 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f642400f5c0 con 0x7f6434101850 2024-09-17T00:02:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.004+0000 7f642b7fe700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f642006e680 0x7f6420070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.005+0000 7f642b7fe700 1 -- 172.21.15.42:0/358371775 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f642408e5c0 con 0x7f6434101850 2024-09-17T00:02:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.005+0000 7f643a41c700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f642006e680 0x7f6420070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:29.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.005+0000 7f643a41c700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f642006e680 0x7f6420070b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6430005f50 tx=0x7f643001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:29.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.009+0000 7f642b7fe700 1 -- 172.21.15.42:0/358371775 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f642405b290 con 0x7f6434101850 2024-09-17T00:02:29.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.240+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6434197cf0 con 0x7f642006e680 2024-09-17T00:02:29.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.242+0000 7f642b7fe700 1 -- 172.21.15.42:0/358371775 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6434197cf0 con 0x7f642006e680 2024-09-17T00:02:29.239 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:02:29.240 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:02:29.241 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:02:29.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f642006e680 msgr2=0x7f6420070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f642006e680 0x7f6420070b40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6430005f50 tx=0x7f643001a040 comp rx=0 tx=0).stop 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434101850 msgr2=0x7f643419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434101850 0x7f643419cc30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f642400c8f0 tx=0x7f642400cc00 comp rx=0 tx=0).stop 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 shutdown_connections 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f642006e680 0x7f6420070b40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6434100ea0 0x7f643419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 --2- 172.21.15.42:0/358371775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6434101850 0x7f643419cc30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 >> 172.21.15.42:0/358371775 conn(0x7f64340fc750 msgr2=0x7f64340fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 shutdown_connections 2024-09-17T00:02:29.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.244+0000 7f643c680700 1 -- 172.21.15.42:0/358371775 wait complete. 2024-09-17T00:02:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.352+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/2318817432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac0ffc20 msgr2=0x7f4aac100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.352+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/2318817432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac0ffc20 0x7f4aac100080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009910 tx=0x7f4a98009c20 comp rx=0 tx=0).stop 2024-09-17T00:02:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.352+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/2318817432 shutdown_connections 2024-09-17T00:02:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.352+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/2318817432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac0ffc20 0x7f4aac100080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.352+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/2318817432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac105bd0 0x7f4aac105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.352+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/2318817432 >> 172.21.15.42:0/2318817432 conn(0x7f4aac075670 msgr2=0x7f4aac075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:29.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.353+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/2318817432 shutdown_connections 2024-09-17T00:02:29.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.353+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/2318817432 wait complete. 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.353+0000 7f4ab2e0e700 1 Processor -- start 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.353+0000 7f4ab2e0e700 1 -- start start 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4ab2e0e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac0ffc20 0x7f4aac197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4ab2e0e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac105bd0 0x7f4aac197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4ab2e0e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac198360 con 0x7f4aac105bd0 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4ab2e0e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1984d0 con 0x7f4aac0ffc20 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4aabfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac105bd0 0x7f4aac197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4aabfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac105bd0 0x7f4aac197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50438/0 (socket says 172.21.15.42:50438) 2024-09-17T00:02:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4aabfff700 1 -- 172.21.15.42:0/3330754838 learned_addr learned my addr 172.21.15.42:0/3330754838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:29.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4ab0baa700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac0ffc20 0x7f4aac197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:29.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4aabfff700 1 -- 172.21.15.42:0/3330754838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac0ffc20 msgr2=0x7f4aac197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:29.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4aabfff700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac0ffc20 0x7f4aac197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4aabfff700 1 -- 172.21.15.42:0/3330754838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a980095f0 con 0x7f4aac105bd0 2024-09-17T00:02:29.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.354+0000 7f4aabfff700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac105bd0 0x7f4aac197c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005f50 tx=0x7f4a9800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.355+0000 7f4aa9ffb700 1 -- 172.21.15.42:0/3330754838 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a9801d070 con 0x7f4aac105bd0 2024-09-17T00:02:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.355+0000 7f4aa9ffb700 1 -- 172.21.15.42:0/3330754838 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a9800b610 con 0x7f4aac105bd0 2024-09-17T00:02:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.355+0000 7f4aa9ffb700 1 -- 172.21.15.42:0/3330754838 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a98022470 con 0x7f4aac105bd0 2024-09-17T00:02:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.355+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac1a07b0 con 0x7f4aac105bd0 2024-09-17T00:02:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.355+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac1a0c50 con 0x7f4aac105bd0 2024-09-17T00:02:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.356+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac1080d0 con 0x7f4aac105bd0 2024-09-17T00:02:29.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.357+0000 7f4aa9ffb700 1 -- 172.21.15.42:0/3330754838 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f4a98003780 con 0x7f4aac105bd0 2024-09-17T00:02:29.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.358+0000 7f4aa9ffb700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4a9c06e680 0x7f4a9c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:29.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.358+0000 7f4aa9ffb700 1 -- 172.21.15.42:0/3330754838 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4a9808f980 con 0x7f4aac105bd0 2024-09-17T00:02:29.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.358+0000 7f4ab0baa700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4a9c06e680 0x7f4a9c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:29.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.358+0000 7f4ab0baa700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4a9c06e680 0x7f4a9c070b40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009c60 tx=0x7f4aa0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:29.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.362+0000 7f4aa9ffb700 1 -- 172.21.15.42:0/3330754838 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4a98058ba0 con 0x7f4aac105bd0 2024-09-17T00:02:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:29 smithi042 ceph-mon[111056]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:29 smithi042 ceph-mon[111056]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:29 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3638921745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:02:29.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:29 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1525182033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:02:29.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:29 smithi043 ceph-mon[81510]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:29.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:29 smithi043 ceph-mon[81510]: from='client.34662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:29.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:29 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3638921745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:02:29.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:29 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1525182033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:02:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.660+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4aac04f2e0 con 0x7f4aac105bd0 2024-09-17T00:02:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.660+0000 7f4aa9ffb700 1 -- 172.21.15.42:0/3330754838 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f4a9805c260 con 0x7f4aac105bd0 2024-09-17T00:02:29.658 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:02:29.658 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:02:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:02:29.659 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:02:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:02:29.659 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:02:29.659 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:02:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4a9c06e680 msgr2=0x7f4a9c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4a9c06e680 0x7f4a9c070b40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009c60 tx=0x7f4aa0009260 comp rx=0 tx=0).stop 2024-09-17T00:02:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac105bd0 msgr2=0x7f4aac197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac105bd0 0x7f4aac197c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005f50 tx=0x7f4a9800f690 comp rx=0 tx=0).stop 2024-09-17T00:02:29.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 shutdown_connections 2024-09-17T00:02:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4a9c06e680 0x7f4a9c070b40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4aac0ffc20 0x7f4aac197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 --2- 172.21.15.42:0/3330754838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4aac105bd0 0x7f4aac197c80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.662+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 >> 172.21.15.42:0/3330754838 conn(0x7f4aac075670 msgr2=0x7f4aac0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.663+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 shutdown_connections 2024-09-17T00:02:29.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:29.663+0000 7f4ab2e0e700 1 -- 172.21.15.42:0/3330754838 wait complete. 2024-09-17T00:02:30.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:30 smithi043 ceph-mon[81510]: pgmap v327: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:30.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:30 smithi043 ceph-mon[81510]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:30.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:30 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3330754838' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:02:30.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:30 smithi042 ceph-mon[111056]: pgmap v327: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:30.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:30 smithi042 ceph-mon[111056]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:02:30.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:30 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3330754838' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:02:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:32 smithi043 ceph-mon[81510]: pgmap v328: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:32 smithi042 ceph-mon[111056]: pgmap v328: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:34 smithi043 ceph-mon[81510]: pgmap v329: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:34 smithi042 ceph-mon[111056]: pgmap v329: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:36 smithi043 ceph-mon[81510]: pgmap v330: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:36.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:36 smithi042 ceph-mon[111056]: pgmap v330: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:38 smithi043 ceph-mon[81510]: pgmap v331: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:38 smithi042 ceph-mon[111056]: pgmap v331: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:40.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:40 smithi043 ceph-mon[81510]: pgmap v332: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:40 smithi042 ceph-mon[111056]: pgmap v332: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:42 smithi043 ceph-mon[81510]: pgmap v333: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:42.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:42 smithi042 ceph-mon[111056]: pgmap v333: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:44 smithi043 ceph-mon[81510]: pgmap v334: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:44 smithi042 ceph-mon[111056]: pgmap v334: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:46.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:46 smithi043 ceph-mon[81510]: pgmap v335: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:46 smithi042 ceph-mon[111056]: pgmap v335: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:48.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:48 smithi042 ceph-mon[111056]: pgmap v336: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:48 smithi043 ceph-mon[81510]: pgmap v336: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:50 smithi042 ceph-mon[111056]: pgmap v337: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:50.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:50 smithi043 ceph-mon[81510]: pgmap v337: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:51.651 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:02:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:52 smithi042 ceph-mon[111056]: pgmap v338: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:52.881 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:52 smithi043 ceph-mon[81510]: pgmap v338: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:53.272 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149569]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:53.273 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149569]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:53.273 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149569]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:53.273 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149569]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:54.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149572]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:02:54.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149572]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:54.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149572]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:54.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:02:53 smithi042 sudo[149572]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:54.317 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149575]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:54.318 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149575]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:54.318 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149575]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:54.318 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149575]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:54.594 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149578]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:02:54.594 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149578]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:54.594 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149578]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:54.594 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149578]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:54.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:54 smithi043 ceph-mon[81510]: pgmap v339: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:54 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:02:54.947 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:54 smithi042 ceph-mon[111056]: pgmap v339: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:54.947 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:54 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:02:55.272 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:55.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:55.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149581]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:55.273 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:54 smithi042 sudo[149581]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:55.773 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:55 smithi042 sudo[149584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:02:55.773 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:55 smithi042 sudo[149584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:55.773 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:55 smithi042 sudo[149584]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:55.773 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:02:55 smithi042 sudo[149584]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:56.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:55 smithi043 sudo[114059]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:56.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:55 smithi043 sudo[114059]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:56.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:55 smithi043 sudo[114059]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:56.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:55 smithi043 sudo[114059]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:56.596 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114062]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:02:56.596 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114062]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:56.596 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114062]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:56.597 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114062]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:56.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:56 smithi043 ceph-mon[81510]: pgmap v340: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:56.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:56 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:02:56.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:56.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:56.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114065]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:56.878 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:56 smithi043 sudo[114065]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:57.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:56 smithi042 ceph-mon[111056]: pgmap v340: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:02:57.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:56 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:02:57.627 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114068]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:02:57.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114068]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:57.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114068]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:57.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114068]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:58.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114071]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:58.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114071]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:58.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114071]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:58.128 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:57 smithi043 sudo[114071]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:58.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:58 smithi043 sudo[114074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:02:58.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:58 smithi043 sudo[114074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:58.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:58 smithi043 sudo[114074]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:58.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:02:58 smithi043 sudo[114074]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:59.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:58 smithi042 sudo[149587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:58 smithi042 sudo[149587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:58 smithi042 sudo[149587]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:58 smithi042 sudo[149587]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:59.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 ceph-mon[81510]: pgmap v341: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-17T00:02:59.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 ceph-mon[81510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T00:02:59.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:59 smithi042 sudo[149590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:02:59.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:59 smithi042 sudo[149590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:59.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:59 smithi042 sudo[149590]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:59.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:59 smithi042 sudo[149590]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:02:59.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:59 smithi042 ceph-mon[111056]: pgmap v341: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-17T00:02:59.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:02:59 smithi042 ceph-mon[111056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T00:02:59.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.776+0000 7f12e30d7700 1 -- 172.21.15.42:0/3030223989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc101e60 msgr2=0x7f12dc102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.776+0000 7f12e30d7700 1 --2- 172.21.15.42:0/3030223989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc101e60 0x7f12dc102240 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009910 tx=0x7f12d0009c20 comp rx=0 tx=0).stop 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.776+0000 7f12e30d7700 1 -- 172.21.15.42:0/3030223989 shutdown_connections 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.776+0000 7f12e30d7700 1 --2- 172.21.15.42:0/3030223989 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12dc102780 0x7f12dc10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.776+0000 7f12e30d7700 1 --2- 172.21.15.42:0/3030223989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc101e60 0x7f12dc102240 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.776+0000 7f12e30d7700 1 -- 172.21.15.42:0/3030223989 >> 172.21.15.42:0/3030223989 conn(0x7f12dc0fb650 msgr2=0x7f12dc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 -- 172.21.15.42:0/3030223989 shutdown_connections 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 -- 172.21.15.42:0/3030223989 wait complete. 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 Processor -- start 2024-09-17T00:02:59.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 -- start start 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12dc101e60 0x7f12dc0ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc102780 0x7f12dc100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc100540 con 0x7f12dc102780 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.777+0000 7f12e30d7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc100680 con 0x7f12dc101e60 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc102780 0x7f12dc100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc102780 0x7f12dc100000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42992/0 (socket says 172.21.15.42:42992) 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12dbfff700 1 -- 172.21.15.42:0/2571362545 learned_addr learned my addr 172.21.15.42:0/2571362545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12e0e73700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12dc101e60 0x7f12dc0ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:59.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12dbfff700 1 -- 172.21.15.42:0/2571362545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12dc101e60 msgr2=0x7f12dc0ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12dbfff700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12dc101e60 0x7f12dc0ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12dbfff700 1 -- 172.21.15.42:0/2571362545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00095f0 con 0x7f12dc102780 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12dbfff700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc102780 0x7f12dc100000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f12c800d900 tx=0x7f12c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12d9ffb700 1 -- 172.21.15.42:0/2571362545 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12c8009900 con 0x7f12dc102780 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12d9ffb700 1 -- 172.21.15.42:0/2571362545 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12c8010460 con 0x7f12dc102780 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc1a2aa0 con 0x7f12dc102780 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.778+0000 7f12d9ffb700 1 -- 172.21.15.42:0/2571362545 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12c800f5f0 con 0x7f12dc102780 2024-09-17T00:02:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.779+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc1a2e50 con 0x7f12dc102780 2024-09-17T00:02:59.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.780+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc108310 con 0x7f12dc102780 2024-09-17T00:02:59.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.780+0000 7f12d9ffb700 1 -- 172.21.15.42:0/2571362545 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f12c80105d0 con 0x7f12dc102780 2024-09-17T00:02:59.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.781+0000 7f12d9ffb700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f12cc06e680 0x7f12cc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:02:59.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.781+0000 7f12d9ffb700 1 -- 172.21.15.42:0/2571362545 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f12c808e090 con 0x7f12dc102780 2024-09-17T00:02:59.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.781+0000 7f12e0e73700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f12cc06e680 0x7f12cc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:02:59.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.781+0000 7f12e0e73700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f12cc06e680 0x7f12cc070b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f12d0006010 tx=0x7f12d001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:02:59.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:02:59.785+0000 7f12d9ffb700 1 -- 172.21.15.42:0/2571362545 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f12c80572b0 con 0x7f12dc102780 2024-09-17T00:02:59.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114077]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-17T00:02:59.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114077]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:02:59.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114077]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:02:59.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114077]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:03:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.021+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12dc02cce0 con 0x7f12cc06e680 2024-09-17T00:03:00.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.022+0000 7f12d9ffb700 1 -- 172.21.15.42:0/2571362545 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f12dc02cce0 con 0x7f12cc06e680 2024-09-17T00:03:00.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f12cc06e680 msgr2=0x7f12cc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f12cc06e680 0x7f12cc070b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f12d0006010 tx=0x7f12d001a040 comp rx=0 tx=0).stop 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc102780 msgr2=0x7f12dc100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc102780 0x7f12dc100000 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f12c800d900 tx=0x7f12c800dc10 comp rx=0 tx=0).stop 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 shutdown_connections 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f12cc06e680 0x7f12cc070b40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12dc101e60 0x7f12dc0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 --2- 172.21.15.42:0/2571362545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12dc102780 0x7f12dc100000 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 >> 172.21.15.42:0/2571362545 conn(0x7f12dc0fb650 msgr2=0x7f12dc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 shutdown_connections 2024-09-17T00:03:00.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.024+0000 7f12e30d7700 1 -- 172.21.15.42:0/2571362545 wait complete. 2024-09-17T00:03:00.034 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:03:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 -- 172.21.15.42:0/3922454679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c100ea0 msgr2=0x7f1f4c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/3922454679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c100ea0 0x7f1f4c101280 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009910 tx=0x7f1f3c009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:00.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 -- 172.21.15.42:0/3922454679 shutdown_connections 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/3922454679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c101850 0x7f1f4c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/3922454679 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c100ea0 0x7f1f4c101280 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 -- 172.21.15.42:0/3922454679 >> 172.21.15.42:0/3922454679 conn(0x7f1f4c0fc750 msgr2=0x7f1f4c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 -- 172.21.15.42:0/3922454679 shutdown_connections 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.135+0000 7f1f50bfa700 1 -- 172.21.15.42:0/3922454679 wait complete. 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f50bfa700 1 Processor -- start 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f50bfa700 1 -- start start 2024-09-17T00:03:00.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f50bfa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c100ea0 0x7f1f4c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f50bfa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c101850 0x7f1f4c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f50bfa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c073b10 con 0x7f1f4c101850 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f50bfa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c073c50 con 0x7f1f4c100ea0 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f49d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c101850 0x7f1f4c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f49d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c101850 0x7f1f4c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43020/0 (socket says 172.21.15.42:43020) 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f4a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c100ea0 0x7f1f4c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f49d9b700 1 -- 172.21.15.42:0/1462417506 learned_addr learned my addr 172.21.15.42:0/1462417506 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f49d9b700 1 -- 172.21.15.42:0/1462417506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c100ea0 msgr2=0x7f1f4c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f49d9b700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c100ea0 0x7f1f4c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f49d9b700 1 -- 172.21.15.42:0/1462417506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c0095f0 con 0x7f1f4c101850 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.136+0000 7f1f4a59c700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c100ea0 0x7f1f4c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.137+0000 7f1f49d9b700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c101850 0x7f1f4c0735d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c960 tx=0x7f1f4000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.137+0000 7f1f3b7fe700 1 -- 172.21.15.42:0/1462417506 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f40007a70 con 0x7f1f4c101850 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.137+0000 7f1f3b7fe700 1 -- 172.21.15.42:0/1462417506 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f40007bd0 con 0x7f1f4c101850 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.137+0000 7f1f3b7fe700 1 -- 172.21.15.42:0/1462417506 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f40018630 con 0x7f1f4c101850 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.137+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c1a2af0 con 0x7f1f4c101850 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.137+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c1a2e50 con 0x7f1f4c101850 2024-09-17T00:03:00.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.138+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c109240 con 0x7f1f4c101850 2024-09-17T00:03:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.139+0000 7f1f3b7fe700 1 -- 172.21.15.42:0/1462417506 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1f4000f450 con 0x7f1f4c101850 2024-09-17T00:03:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.139+0000 7f1f3b7fe700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f3406e680 0x7f1f34070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.139+0000 7f1f3b7fe700 1 -- 172.21.15.42:0/1462417506 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1f4008e5e0 con 0x7f1f4c101850 2024-09-17T00:03:00.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.139+0000 7f1f4a59c700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f3406e680 0x7f1f34070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.140+0000 7f1f4a59c700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f3406e680 0x7f1f34070b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006010 tx=0x7f1f3c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:00.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.144+0000 7f1f3b7fe700 1 -- 172.21.15.42:0/1462417506 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1f40057810 con 0x7f1f4c101850 2024-09-17T00:03:00.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.379+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f4c0611d0 con 0x7f1f3406e680 2024-09-17T00:03:00.377 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: pgmap v342: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-17T00:03:00.377 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T00:03:00.377 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-17T00:03:00.377 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-17T00:03:00.378 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T00:03:00.378 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-17T00:03:00.378 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-17T00:03:00.378 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:00 smithi042 ceph-mon[111056]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T00:03:00.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-17T00:03:00.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-17T00:03:00.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114080]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:02:59 smithi043 sudo[114080]: pam_unix(sudo:session): session closed for user root 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: pgmap v342: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi042"}]: dispatch 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "mon metadata", "id": "smithi043"}]: dispatch 2024-09-17T00:03:00.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:00 smithi043 ceph-mon[81510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-17T00:03:00.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.380+0000 7f1f3b7fe700 1 -- 172.21.15.42:0/1462417506 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1f4c0611d0 con 0x7f1f3406e680 2024-09-17T00:03:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f3406e680 msgr2=0x7f1f34070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f3406e680 0x7f1f34070b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c006010 tx=0x7f1f3c01a040 comp rx=0 tx=0).stop 2024-09-17T00:03:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c101850 msgr2=0x7f1f4c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c101850 0x7f1f4c0735d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1f4000c960 tx=0x7f1f4000cc70 comp rx=0 tx=0).stop 2024-09-17T00:03:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 shutdown_connections 2024-09-17T00:03:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f3406e680 0x7f1f34070b40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c100ea0 0x7f1f4c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 --2- 172.21.15.42:0/1462417506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f4c101850 0x7f1f4c0735d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 >> 172.21.15.42:0/1462417506 conn(0x7f1f4c0fc750 msgr2=0x7f1f4c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 shutdown_connections 2024-09-17T00:03:00.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.382+0000 7f1f50bfa700 1 -- 172.21.15.42:0/1462417506 wait complete. 2024-09-17T00:03:00.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 -- 172.21.15.42:0/4199522209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c0ffda0 msgr2=0x7f9c1c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 --2- 172.21.15.42:0/4199522209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c0ffda0 0x7f9c1c100180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009910 tx=0x7f9c0c009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 -- 172.21.15.42:0/4199522209 shutdown_connections 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 --2- 172.21.15.42:0/4199522209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c100750 0x7f9c1c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 --2- 172.21.15.42:0/4199522209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c0ffda0 0x7f9c1c100180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 -- 172.21.15.42:0/4199522209 >> 172.21.15.42:0/4199522209 conn(0x7f9c1c0fb650 msgr2=0x7f9c1c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 -- 172.21.15.42:0/4199522209 shutdown_connections 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.490+0000 7f9c217a1700 1 -- 172.21.15.42:0/4199522209 wait complete. 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c217a1700 1 Processor -- start 2024-09-17T00:03:00.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c217a1700 1 -- start start 2024-09-17T00:03:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c217a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c0ffda0 0x7f9c1c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c217a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c100750 0x7f9c1c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c217a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c19d200 con 0x7f9c1c100750 2024-09-17T00:03:00.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c217a1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c1966b0 con 0x7f9c1c0ffda0 2024-09-17T00:03:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c1a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c100750 0x7f9c1c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c1a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c100750 0x7f9c1c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43026/0 (socket says 172.21.15.42:43026) 2024-09-17T00:03:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c1a7fc700 1 -- 172.21.15.42:0/1614478412 learned_addr learned my addr 172.21.15.42:0/1614478412 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.491+0000 7f9c1affd700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c0ffda0 0x7f9c1c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c1a7fc700 1 -- 172.21.15.42:0/1614478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c0ffda0 msgr2=0x7f9c1c19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c1a7fc700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c0ffda0 0x7f9c1c19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c1a7fc700 1 -- 172.21.15.42:0/1614478412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0095f0 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c1a7fc700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c100750 0x7f9c1c19cb70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000d900 tx=0x7f9c1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c03fff700 1 -- 172.21.15.42:0/1614478412 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c10009900 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c03fff700 1 -- 172.21.15.42:0/1614478412 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c10010460 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c196990 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c03fff700 1 -- 172.21.15.42:0/1614478412 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1000f5f0 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.492+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c196ee0 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.493+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1c04f2e0 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.494+0000 7f9c03fff700 1 -- 172.21.15.42:0/1614478412 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9c100105d0 con 0x7f9c1c100750 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.494+0000 7f9c03fff700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c0406e680 0x7f9c04070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.494+0000 7f9c03fff700 1 -- 172.21.15.42:0/1614478412 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c1008e120 con 0x7f9c1c100750 2024-09-17T00:03:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.494+0000 7f9c1affd700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c0406e680 0x7f9c04070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.495+0000 7f9c1affd700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c0406e680 0x7f9c04070b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c006010 tx=0x7f9c0c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:00.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.499+0000 7f9c03fff700 1 -- 172.21.15.42:0/1614478412 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9c1005ae70 con 0x7f9c1c100750 2024-09-17T00:03:00.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.715+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c1c197260 con 0x7f9c0406e680 2024-09-17T00:03:00.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.725+0000 7f9c03fff700 1 -- 172.21.15.42:0/1614478412 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f9c1c197260 con 0x7f9c0406e680 2024-09-17T00:03:00.723 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:03:00.723 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (11m) 5m ago 22m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:03:00.723 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (22m) 5m ago 22m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:03:00.723 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (21m) 4m ago 21m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:03:00.723 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (9m) 5m ago 22m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:03:00.723 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (9m) 4m ago 21m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (11m) 5m ago 22m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (5m) 5m ago 16m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (5m) 5m ago 16m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (5m) 4m ago 16m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (4m) 4m ago 16m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (13m) 5m ago 23m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (12m) 4m ago 21m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (10m) 5m ago 23m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (9m) 4m ago 21m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (12m) 5m ago 22m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (12m) 4m ago 21m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (8m) 5m ago 20m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (8m) 5m ago 20m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:03:00.724 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (7m) 5m ago 19m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:03:00.725 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (7m) 4m ago 19m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:03:00.725 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (6m) 4m ago 18m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:03:00.725 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (5m) 4m ago 18m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:03:00.725 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (11m) 5m ago 21m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.727+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c0406e680 msgr2=0x7f9c04070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.727+0000 7f9c217a1700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c0406e680 0x7f9c04070b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c006010 tx=0x7f9c0c01a040 comp rx=0 tx=0).stop 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.727+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c100750 msgr2=0x7f9c1c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.727+0000 7f9c217a1700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c100750 0x7f9c1c19cb70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000d900 tx=0x7f9c1000dc10 comp rx=0 tx=0).stop 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.727+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 shutdown_connections 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.727+0000 7f9c217a1700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9c0406e680 0x7f9c04070b40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.727+0000 7f9c217a1700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c1c0ffda0 0x7f9c1c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.728+0000 7f9c217a1700 1 --2- 172.21.15.42:0/1614478412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9c1c100750 0x7f9c1c19cb70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.728+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 >> 172.21.15.42:0/1614478412 conn(0x7f9c1c0fb650 msgr2=0x7f9c1c103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.728+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 shutdown_connections 2024-09-17T00:03:00.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.728+0000 7f9c217a1700 1 -- 172.21.15.42:0/1614478412 wait complete. 2024-09-17T00:03:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2044838963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 msgr2=0x7fbd04101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2044838963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd04101280 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbd0000b0a0 tx=0x7fbd0000b3b0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2044838963 shutdown_connections 2024-09-17T00:03:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2044838963 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd041058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2044838963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd04101280 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2044838963 >> 172.21.15.42:0/2044838963 conn(0x7fbd040fc750 msgr2=0x7fbd040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2044838963 shutdown_connections 2024-09-17T00:03:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2044838963 wait complete. 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 Processor -- start 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.835+0000 7fbd0bfb1700 1 -- start start 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd0bfb1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd0bfb1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd0bfb1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0419d2c0 con 0x7fbd04100ea0 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd0bfb1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd04196770 con 0x7fbd04101850 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd09d4d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd09d4d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43052/0 (socket says 172.21.15.42:43052) 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd09d4d700 1 -- 172.21.15.42:0/2068804116 learned_addr learned my addr 172.21.15.42:0/2068804116 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd0954c700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd09d4d700 1 -- 172.21.15.42:0/2068804116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 msgr2=0x7fbd0419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:00.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd09d4d700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd09d4d700 1 -- 172.21.15.42:0/2068804116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009d00 con 0x7fbd04100ea0 2024-09-17T00:03:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbd09d4d700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c6f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000f80 tx=0x7fbd0000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.836+0000 7fbcfaffd700 1 -- 172.21.15.42:0/2068804116 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0000e040 con 0x7fbd04100ea0 2024-09-17T00:03:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.837+0000 7fbcfaffd700 1 -- 172.21.15.42:0/2068804116 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd00003c10 con 0x7fbd04100ea0 2024-09-17T00:03:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.837+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd041969f0 con 0x7fbd04100ea0 2024-09-17T00:03:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.837+0000 7fbcfaffd700 1 -- 172.21.15.42:0/2068804116 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd0001d680 con 0x7fbd04100ea0 2024-09-17T00:03:00.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.837+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd04196ee0 con 0x7fbd04100ea0 2024-09-17T00:03:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.838+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd04109240 con 0x7fbd04100ea0 2024-09-17T00:03:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.838+0000 7fbcfaffd700 1 -- 172.21.15.42:0/2068804116 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fbd00019070 con 0x7fbd04100ea0 2024-09-17T00:03:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.839+0000 7fbcfaffd700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbcf006e680 0x7fbcf0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.839+0000 7fbcfaffd700 1 -- 172.21.15.42:0/2068804116 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbd0008fae0 con 0x7fbd04100ea0 2024-09-17T00:03:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.839+0000 7fbd0954c700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbcf006e680 0x7fbcf0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:00.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.839+0000 7fbd0954c700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbcf006e680 0x7fbcf0070b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd04197be0 tx=0x7fbcf4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:00.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:00.843+0000 7fbcfaffd700 1 -- 172.21.15.42:0/2068804116 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbd0005c8e0 con 0x7fbd04100ea0 2024-09-17T00:03:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.140+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd04197d20 con 0x7fbd04100ea0 2024-09-17T00:03:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.140+0000 7fbcfaffd700 1 -- 172.21.15.42:0/2068804116 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fbd00003d80 con 0x7fbd04100ea0 2024-09-17T00:03:01.138 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:03:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:03:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:03:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:03:01.138 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:03:01.139 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbcf006e680 msgr2=0x7fbcf0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbcf006e680 0x7fbcf0070b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd04197be0 tx=0x7fbcf4009260 comp rx=0 tx=0).stop 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 msgr2=0x7fbd0419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c6f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbd00000f80 tx=0x7fbd0000bc60 comp rx=0 tx=0).stop 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 shutdown_connections 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbcf006e680 0x7fbcf0070b40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd04100ea0 0x7fbd0419c6f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 --2- 172.21.15.42:0/2068804116 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd04101850 0x7fbd0419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 >> 172.21.15.42:0/2068804116 conn(0x7fbd040fc750 msgr2=0x7fbd040fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:01.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 shutdown_connections 2024-09-17T00:03:01.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.142+0000 7fbd0bfb1700 1 -- 172.21.15.42:0/2068804116 wait complete. 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 -- 172.21.15.42:0/2629889835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4101e00 msgr2=0x7fceb41021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 --2- 172.21.15.42:0/2629889835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4101e00 0x7fceb41021e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcea400b0a0 tx=0x7fcea400b3b0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 -- 172.21.15.42:0/2629889835 shutdown_connections 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 --2- 172.21.15.42:0/2629889835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb4102720 0x7fceb410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 --2- 172.21.15.42:0/2629889835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4101e00 0x7fceb41021e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 -- 172.21.15.42:0/2629889835 >> 172.21.15.42:0/2629889835 conn(0x7fceb40fb650 msgr2=0x7fceb40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 -- 172.21.15.42:0/2629889835 shutdown_connections 2024-09-17T00:03:01.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.250+0000 7fcebb397700 1 -- 172.21.15.42:0/2629889835 wait complete. 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fcebb397700 1 Processor -- start 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fcebb397700 1 -- start start 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fcebb397700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb4101e00 0x7fceb419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fcebb397700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4102720 0x7fceb419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fcebb397700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb419d260 con 0x7fceb4102720 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fcebb397700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb4196710 con 0x7fceb4101e00 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fceb8932700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4102720 0x7fceb419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fceb8932700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4102720 0x7fceb419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43074/0 (socket says 172.21.15.42:43074) 2024-09-17T00:03:01.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fceb8932700 1 -- 172.21.15.42:0/3819381147 learned_addr learned my addr 172.21.15.42:0/3819381147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.251+0000 7fceb9133700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb4101e00 0x7fceb419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fceb8932700 1 -- 172.21.15.42:0/3819381147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb4101e00 msgr2=0x7fceb419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fceb8932700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb4101e00 0x7fceb419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fceb8932700 1 -- 172.21.15.42:0/3819381147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea4009d00 con 0x7fceb4102720 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fceb8932700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4102720 0x7fceb419cbd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcea800d8d0 tx=0x7fcea800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fceb27fc700 1 -- 172.21.15.42:0/3819381147 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcea80041f0 con 0x7fceb4102720 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fceb27fc700 1 -- 172.21.15.42:0/3819381147 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcea8004350 con 0x7fceb4102720 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb41969f0 con 0x7fceb4102720 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fceb27fc700 1 -- 172.21.15.42:0/3819381147 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcea8003c80 con 0x7fceb4102720 2024-09-17T00:03:01.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.252+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb4196f40 con 0x7fceb4102720 2024-09-17T00:03:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.253+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb4108310 con 0x7fceb4102720 2024-09-17T00:03:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.254+0000 7fceb27fc700 1 -- 172.21.15.42:0/3819381147 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fcea800f690 con 0x7fceb4102720 2024-09-17T00:03:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.254+0000 7fceb27fc700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce9c06e680 0x7fce9c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.254+0000 7fceb27fc700 1 -- 172.21.15.42:0/3819381147 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcea808e210 con 0x7fceb4102720 2024-09-17T00:03:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.254+0000 7fceb9133700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce9c06e680 0x7fce9c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.255+0000 7fceb9133700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce9c06e680 0x7fce9c070b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcea4000f80 tx=0x7fcea40060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:01.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.259+0000 7fceb27fc700 1 -- 172.21.15.42:0/3819381147 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcea805aee0 con 0x7fceb4102720 2024-09-17T00:03:01.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:01 smithi043 ceph-mon[81510]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:01 smithi043 ceph-mon[81510]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:01.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.510+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fceb404f2e0 con 0x7fceb4102720 2024-09-17T00:03:01.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:01 smithi042 ceph-mon[111056]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:01.508 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:01 smithi042 ceph-mon[111056]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:01.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.510+0000 7fceb27fc700 1 -- 172.21.15.42:0/3819381147 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fcea805aa70 con 0x7fceb4102720 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:03:01.509 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:03:01.510 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:01.511 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:01.512 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce9c06e680 msgr2=0x7fce9c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce9c06e680 0x7fce9c070b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcea4000f80 tx=0x7fcea40060c0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4102720 msgr2=0x7fceb419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4102720 0x7fceb419cbd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcea800d8d0 tx=0x7fcea800dbe0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 shutdown_connections 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce9c06e680 0x7fce9c070b40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fceb4101e00 0x7fceb419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 --2- 172.21.15.42:0/3819381147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fceb4102720 0x7fceb419cbd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 >> 172.21.15.42:0/3819381147 conn(0x7fceb40fb650 msgr2=0x7fceb4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 shutdown_connections 2024-09-17T00:03:01.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.513+0000 7fcebb397700 1 -- 172.21.15.42:0/3819381147 wait complete. 2024-09-17T00:03:01.514 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:03:01.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 -- 172.21.15.42:0/1535759999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 msgr2=0x7f079c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 --2- 172.21.15.42:0/1535759999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c101270 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f078c009910 tx=0x7f078c009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 -- 172.21.15.42:0/1535759999 shutdown_connections 2024-09-17T00:03:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 --2- 172.21.15.42:0/1535759999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f079c101840 0x7f079c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 --2- 172.21.15.42:0/1535759999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c101270 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 -- 172.21.15.42:0/1535759999 >> 172.21.15.42:0/1535759999 conn(0x7f079c0fc740 msgr2=0x7f079c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 -- 172.21.15.42:0/1535759999 shutdown_connections 2024-09-17T00:03:01.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.622+0000 7f07a116d700 1 -- 172.21.15.42:0/1535759999 wait complete. 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f07a116d700 1 Processor -- start 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f07a116d700 1 -- start start 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f07a116d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f07a116d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f079c101840 0x7f079c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f07a116d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c19d250 con 0x7f079c100e90 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f07a116d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079c196700 con 0x7f079c101840 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f079ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f079ad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43080/0 (socket says 172.21.15.42:43080) 2024-09-17T00:03:01.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.623+0000 7f079ad9d700 1 -- 172.21.15.42:0/1126482040 learned_addr learned my addr 172.21.15.42:0/1126482040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f079259c700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f079c101840 0x7f079c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f079ad9d700 1 -- 172.21.15.42:0/1126482040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f079c101840 msgr2=0x7f079c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f079ad9d700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f079c101840 0x7f079c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f079ad9d700 1 -- 172.21.15.42:0/1126482040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0095f0 con 0x7f079c100e90 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f079259c700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f079c101840 0x7f079c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f079ad9d700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c19c680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f0798d99700 1 -- 172.21.15.42:0/1126482040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078c01d070 con 0x7f079c100e90 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079c196980 con 0x7f079c100e90 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f0798d99700 1 -- 172.21.15.42:0/1126482040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f078c00f460 con 0x7f079c100e90 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f0798d99700 1 -- 172.21.15.42:0/1126482040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078c017630 con 0x7f079c100e90 2024-09-17T00:03:01.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.624+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079c196e70 con 0x7f079c100e90 2024-09-17T00:03:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.625+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079c109230 con 0x7f079c100e90 2024-09-17T00:03:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.626+0000 7f0798d99700 1 -- 172.21.15.42:0/1126482040 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f078c00f5d0 con 0x7f079c100e90 2024-09-17T00:03:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.626+0000 7f0798d99700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f078806e680 0x7f0788070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.626+0000 7f0798d99700 1 -- 172.21.15.42:0/1126482040 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f078c08fab0 con 0x7f079c100e90 2024-09-17T00:03:01.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.626+0000 7f079259c700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f078806e680 0x7f0788070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.627+0000 7f079259c700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f078806e680 0x7f0788070b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0784005fd0 tx=0x7f0784005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:01.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.631+0000 7f0798d99700 1 -- 172.21.15.42:0/1126482040 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f078c05c800 con 0x7f079c100e90 2024-09-17T00:03:01.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.864+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f079c197000 con 0x7f078806e680 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.865+0000 7f0798d99700 1 -- 172.21.15.42:0/1126482040 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f079c197000 con 0x7f078806e680 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:03:01.863 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:03:01.864 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:03:01.864 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:03:01.864 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:03:01.864 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:03:01.864 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:03:01.864 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:03:01.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f078806e680 msgr2=0x7f0788070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f078806e680 0x7f0788070b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0784005fd0 tx=0x7f0784005ee0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 msgr2=0x7f079c19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c19c680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078c006010 tx=0x7f078c00be20 comp rx=0 tx=0).stop 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 shutdown_connections 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f078806e680 0x7f0788070b40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f079c100e90 0x7f079c19c680 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 --2- 172.21.15.42:0/1126482040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f079c101840 0x7f079c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 >> 172.21.15.42:0/1126482040 conn(0x7f079c0fc740 msgr2=0x7f079c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 shutdown_connections 2024-09-17T00:03:01.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.868+0000 7f07a116d700 1 -- 172.21.15.42:0/1126482040 wait complete. 2024-09-17T00:03:01.973 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1400682089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c100ea0 msgr2=0x7f5b3c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1400682089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c100ea0 0x7f5b3c101280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009910 tx=0x7f5b2c009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1400682089 shutdown_connections 2024-09-17T00:03:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1400682089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b3c101850 0x7f5b3c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1400682089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c100ea0 0x7f5b3c101280 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1400682089 >> 172.21.15.42:0/1400682089 conn(0x7f5b3c0fc750 msgr2=0x7f5b3c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1400682089 shutdown_connections 2024-09-17T00:03:01.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1400682089 wait complete. 2024-09-17T00:03:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 Processor -- start 2024-09-17T00:03:01.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.976+0000 7f5b42b9c700 1 -- start start 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b42b9c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b3c100ea0 0x7f5b3c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b42b9c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c101850 0x7f5b3c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b42b9c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c19d2c0 con 0x7f5b3c101850 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b42b9c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c196770 con 0x7f5b3c100ea0 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b3bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c101850 0x7f5b3c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b3bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c101850 0x7f5b3c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43108/0 (socket says 172.21.15.42:43108) 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b3bfff700 1 -- 172.21.15.42:0/1852032755 learned_addr learned my addr 172.21.15.42:0/1852032755 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b40938700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b3c100ea0 0x7f5b3c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b3bfff700 1 -- 172.21.15.42:0/1852032755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b3c100ea0 msgr2=0x7f5b3c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b3bfff700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b3c100ea0 0x7f5b3c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b3bfff700 1 -- 172.21.15.42:0/1852032755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0095f0 con 0x7f5b3c101850 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b3bfff700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c101850 0x7f5b3c19cc30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d900 tx=0x7f5b3000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:01.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b39ffb700 1 -- 172.21.15.42:0/1852032755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b30009900 con 0x7f5b3c101850 2024-09-17T00:03:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.977+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c196a50 con 0x7f5b3c101850 2024-09-17T00:03:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.978+0000 7f5b39ffb700 1 -- 172.21.15.42:0/1852032755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b30010460 con 0x7f5b3c101850 2024-09-17T00:03:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.978+0000 7f5b39ffb700 1 -- 172.21.15.42:0/1852032755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3000f5f0 con 0x7f5b3c101850 2024-09-17T00:03:01.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.978+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c196fa0 con 0x7f5b3c101850 2024-09-17T00:03:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.979+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c109240 con 0x7f5b3c101850 2024-09-17T00:03:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.979+0000 7f5b39ffb700 1 -- 172.21.15.42:0/1852032755 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5b300105d0 con 0x7f5b3c101850 2024-09-17T00:03:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.980+0000 7f5b39ffb700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b2406e680 0x7f5b24070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.980+0000 7f5b39ffb700 1 -- 172.21.15.42:0/1852032755 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5b3008e240 con 0x7f5b3c101850 2024-09-17T00:03:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.980+0000 7f5b40938700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b2406e680 0x7f5b24070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:01.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.980+0000 7f5b40938700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b2406e680 0x7f5b24070b40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c006010 tx=0x7f5b2c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:01.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:01.984+0000 7f5b39ffb700 1 -- 172.21.15.42:0/1852032755 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5b3005af10 con 0x7f5b3c101850 2024-09-17T00:03:02.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:02 smithi042 ceph-mon[111056]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:02 smithi042 ceph-mon[111056]: pgmap v343: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-17T00:03:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:02 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2068804116' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:03:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:02 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3819381147' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:03:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.279+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b3c04f2e0 con 0x7f5b3c101850 2024-09-17T00:03:02.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.280+0000 7f5b39ffb700 1 -- 172.21.15.42:0/1852032755 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5b3005aaa0 con 0x7f5b3c101850 2024-09-17T00:03:02.277 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:03:02.277 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:03:02.278 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:03:02.278 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:03:02.278 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:03:02.278 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:03:02.278 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:03:02.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b2406e680 msgr2=0x7f5b24070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:02.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b2406e680 0x7f5b24070b40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c006010 tx=0x7f5b2c01a040 comp rx=0 tx=0).stop 2024-09-17T00:03:02.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c101850 msgr2=0x7f5b3c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c101850 0x7f5b3c19cc30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000d900 tx=0x7f5b3000dc10 comp rx=0 tx=0).stop 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 shutdown_connections 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5b2406e680 0x7f5b24070b40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b3c100ea0 0x7f5b3c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 --2- 172.21.15.42:0/1852032755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5b3c101850 0x7f5b3c19cc30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 >> 172.21.15.42:0/1852032755 conn(0x7f5b3c0fc750 msgr2=0x7f5b3c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 shutdown_connections 2024-09-17T00:03:02.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:02.282+0000 7f5b42b9c700 1 -- 172.21.15.42:0/1852032755 wait complete. 2024-09-17T00:03:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:02 smithi043 ceph-mon[81510]: from='client.34692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:02 smithi043 ceph-mon[81510]: pgmap v343: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-17T00:03:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:02 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2068804116' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:03:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:02 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3819381147' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:03:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:03 smithi043 ceph-mon[81510]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:03 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1852032755' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:03:03.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:03 smithi042 ceph-mon[111056]: from='client.34704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:03.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:03 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1852032755' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:03:04.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:04 smithi043 ceph-mon[81510]: pgmap v344: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 87 KiB/s wr, 7 op/s 2024-09-17T00:03:04.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:04 smithi042 ceph-mon[111056]: pgmap v344: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 87 KiB/s wr, 7 op/s 2024-09-17T00:03:06.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:06 smithi042 ceph-mon[111056]: pgmap v345: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 87 KiB/s wr, 7 op/s 2024-09-17T00:03:06.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:06.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:06 smithi043 ceph-mon[81510]: pgmap v345: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 87 KiB/s wr, 7 op/s 2024-09-17T00:03:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:08.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:08 smithi042 ceph-mon[111056]: pgmap v346: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 87 KiB/s wr, 7 op/s 2024-09-17T00:03:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:08 smithi043 ceph-mon[81510]: pgmap v346: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 87 KiB/s wr, 7 op/s 2024-09-17T00:03:10.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:10 smithi042 ceph-mon[111056]: pgmap v347: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-17T00:03:10.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:10 smithi043 ceph-mon[81510]: pgmap v347: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-17T00:03:12.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:12 smithi042 ceph-mon[111056]: pgmap v348: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-17T00:03:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:12 smithi043 ceph-mon[81510]: pgmap v348: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-17T00:03:14.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:14 smithi042 ceph-mon[111056]: pgmap v349: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-17T00:03:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:14 smithi043 ceph-mon[81510]: pgmap v349: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-17T00:03:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:16 smithi042 ceph-mon[111056]: pgmap v350: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:16.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:16 smithi043 ceph-mon[81510]: pgmap v350: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:18.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:18 smithi042 ceph-mon[111056]: pgmap v351: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:18 smithi043 ceph-mon[81510]: pgmap v351: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:20.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:20 smithi042 ceph-mon[111056]: pgmap v352: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:20.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:20 smithi043 ceph-mon[81510]: pgmap v352: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:21.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:22.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:22 smithi042 ceph-mon[111056]: pgmap v353: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:22.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:22 smithi043 ceph-mon[81510]: pgmap v353: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:24.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:24 smithi042 ceph-mon[111056]: pgmap v354: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:24 smithi043 ceph-mon[81510]: pgmap v354: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:26.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:26 smithi042 ceph-mon[111056]: pgmap v355: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:26.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:26 smithi043 ceph-mon[81510]: pgmap v355: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:28.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:28 smithi042 ceph-mon[111056]: pgmap v356: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:28.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:28 smithi043 ceph-mon[81510]: pgmap v356: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:30.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:30 smithi042 ceph-mon[111056]: pgmap v357: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:30.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:30 smithi043 ceph-mon[81510]: pgmap v357: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:32.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.405+0000 7f625bce3700 1 -- 172.21.15.42:0/2579945606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 msgr2=0x7f62541021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.405+0000 7f625bce3700 1 --2- 172.21.15.42:0/2579945606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f62541021e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6250009910 tx=0x7f6250009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.405+0000 7f625bce3700 1 -- 172.21.15.42:0/2579945606 shutdown_connections 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.405+0000 7f625bce3700 1 --2- 172.21.15.42:0/2579945606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102720 0x7f625410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.405+0000 7f625bce3700 1 --2- 172.21.15.42:0/2579945606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f62541021e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.405+0000 7f625bce3700 1 -- 172.21.15.42:0/2579945606 >> 172.21.15.42:0/2579945606 conn(0x7f62540fb650 msgr2=0x7f62540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 -- 172.21.15.42:0/2579945606 shutdown_connections 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 -- 172.21.15.42:0/2579945606 wait complete. 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 Processor -- start 2024-09-17T00:03:32.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 -- start start 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f625419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102720 0x7f625419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625419d260 con 0x7f6254101e00 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.406+0000 7f625bce3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254196710 con 0x7f6254102720 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f6259a7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f625419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f6259a7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f625419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46572/0 (socket says 172.21.15.42:46572) 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f6259a7f700 1 -- 172.21.15.42:0/3340693663 learned_addr learned my addr 172.21.15.42:0/3340693663 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:32.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f6259a7f700 1 -- 172.21.15.42:0/3340693663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102720 msgr2=0x7f625419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f6259a7f700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102720 0x7f625419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f6259a7f700 1 -- 172.21.15.42:0/3340693663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62500095f0 con 0x7f6254101e00 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f6259a7f700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f625419c690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62500092e0 tx=0x7f625000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f624affd700 1 -- 172.21.15.42:0/3340693663 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625001d070 con 0x7f6254101e00 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f624affd700 1 -- 172.21.15.42:0/3340693663 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6250005e00 con 0x7f6254101e00 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6254196990 con 0x7f6254101e00 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f624affd700 1 -- 172.21.15.42:0/3340693663 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6250017400 con 0x7f6254101e00 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.407+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6254196e80 con 0x7f6254101e00 2024-09-17T00:03:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.408+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254108310 con 0x7f6254101e00 2024-09-17T00:03:32.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.409+0000 7f624affd700 1 -- 172.21.15.42:0/3340693663 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f625000f460 con 0x7f6254101e00 2024-09-17T00:03:32.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.409+0000 7f624affd700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f624006e680 0x7f6240070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:32.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.410+0000 7f624affd700 1 -- 172.21.15.42:0/3340693663 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f625008fb90 con 0x7f6254101e00 2024-09-17T00:03:32.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.410+0000 7f625927e700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f624006e680 0x7f6240070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:32.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.410+0000 7f625927e700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f624006e680 0x7f6240070b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6244005fd0 tx=0x7f6244005f60 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:32.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.414+0000 7f624affd700 1 -- 172.21.15.42:0/3340693663 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f625005c860 con 0x7f6254101e00 2024-09-17T00:03:32.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:32 smithi042 ceph-mon[111056]: pgmap v358: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:32 smithi043 ceph-mon[81510]: pgmap v358: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:32.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.644+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6254197160 con 0x7f624006e680 2024-09-17T00:03:32.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.646+0000 7f624affd700 1 -- 172.21.15.42:0/3340693663 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6254197160 con 0x7f624006e680 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f624006e680 msgr2=0x7f6240070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f624006e680 0x7f6240070b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6244005fd0 tx=0x7f6244005f60 comp rx=0 tx=0).stop 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 msgr2=0x7f625419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f625419c690 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f62500092e0 tx=0x7f625000be20 comp rx=0 tx=0).stop 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 shutdown_connections 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f624006e680 0x7f6240070b40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6254101e00 0x7f625419c690 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 --2- 172.21.15.42:0/3340693663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6254102720 0x7f625419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 >> 172.21.15.42:0/3340693663 conn(0x7f62540fb650 msgr2=0x7f6254105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:32.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 shutdown_connections 2024-09-17T00:03:32.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.648+0000 7f625bce3700 1 -- 172.21.15.42:0/3340693663 wait complete. 2024-09-17T00:03:32.656 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:03:32.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.757+0000 7fe9ae22a700 1 -- 172.21.15.42:0/3943834781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8101850 msgr2=0x7fe9a81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.757+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/3943834781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8101850 0x7fe9a81058a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009870 tx=0x7fe99c009b80 comp rx=0 tx=0).stop 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 -- 172.21.15.42:0/3943834781 shutdown_connections 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/3943834781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8101850 0x7fe9a81058a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/3943834781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a8100ea0 0x7fe9a8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 -- 172.21.15.42:0/3943834781 >> 172.21.15.42:0/3943834781 conn(0x7fe9a80fc750 msgr2=0x7fe9a80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 -- 172.21.15.42:0/3943834781 shutdown_connections 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 -- 172.21.15.42:0/3943834781 wait complete. 2024-09-17T00:03:32.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 Processor -- start 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.758+0000 7fe9ae22a700 1 -- start start 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9ae22a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8100ea0 0x7fe9a819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9ae22a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a8101850 0x7fe9a819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9ae22a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a819d2d0 con 0x7fe9a8100ea0 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9ae22a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8196780 con 0x7fe9a8101850 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8100ea0 0x7fe9a819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8100ea0 0x7fe9a819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60086/0 (socket says 172.21.15.42:60086) 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a77fe700 1 -- 172.21.15.42:0/2870414114 learned_addr learned my addr 172.21.15.42:0/2870414114 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a6ffd700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a8101850 0x7fe9a819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:32.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a77fe700 1 -- 172.21.15.42:0/2870414114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a8101850 msgr2=0x7fe9a819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a77fe700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a8101850 0x7fe9a819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a77fe700 1 -- 172.21.15.42:0/2870414114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c009520 con 0x7fe9a8100ea0 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a77fe700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8100ea0 0x7fe9a819c700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe99800e940 tx=0x7fe99800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a4ff9700 1 -- 172.21.15.42:0/2870414114 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe99800eee0 con 0x7fe9a8100ea0 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.759+0000 7fe9a4ff9700 1 -- 172.21.15.42:0/2870414114 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe998004cf0 con 0x7fe9a8100ea0 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.760+0000 7fe9a4ff9700 1 -- 172.21.15.42:0/2870414114 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe998005650 con 0x7fe9a8100ea0 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.760+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a8196a60 con 0x7fe9a8100ea0 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.760+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a8196f30 con 0x7fe9a8100ea0 2024-09-17T00:03:32.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.761+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a81092a0 con 0x7fe9a8100ea0 2024-09-17T00:03:32.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.761+0000 7fe9a4ff9700 1 -- 172.21.15.42:0/2870414114 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe998004750 con 0x7fe9a8100ea0 2024-09-17T00:03:32.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.762+0000 7fe9a4ff9700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe99006e680 0x7fe990070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:32.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.762+0000 7fe9a4ff9700 1 -- 172.21.15.42:0/2870414114 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe998014070 con 0x7fe9a8100ea0 2024-09-17T00:03:32.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.762+0000 7fe9a6ffd700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe99006e680 0x7fe990070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:32.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.762+0000 7fe9a6ffd700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe99006e680 0x7fe990070b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009f50 tx=0x7fe99c019040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:32.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.766+0000 7fe9a4ff9700 1 -- 172.21.15.42:0/2870414114 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe99805b6e0 con 0x7fe9a8100ea0 2024-09-17T00:03:32.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.996+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9a8197da0 con 0x7fe99006e680 2024-09-17T00:03:32.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:32.998+0000 7fe9a4ff9700 1 -- 172.21.15.42:0/2870414114 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe9a8197da0 con 0x7fe99006e680 2024-09-17T00:03:32.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe99006e680 msgr2=0x7fe990070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe99006e680 0x7fe990070b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009f50 tx=0x7fe99c019040 comp rx=0 tx=0).stop 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8100ea0 msgr2=0x7fe9a819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8100ea0 0x7fe9a819c700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe99800e940 tx=0x7fe99800ed00 comp rx=0 tx=0).stop 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 shutdown_connections 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe99006e680 0x7fe990070b40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a8100ea0 0x7fe9a819c700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 --2- 172.21.15.42:0/2870414114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9a8101850 0x7fe9a819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 >> 172.21.15.42:0/2870414114 conn(0x7fe9a80fc750 msgr2=0x7fe9a80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 shutdown_connections 2024-09-17T00:03:32.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.001+0000 7fe9ae22a700 1 -- 172.21.15.42:0/2870414114 wait complete. 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 -- 172.21.15.42:0/4212810280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c101850 msgr2=0x7ff48c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 --2- 172.21.15.42:0/4212810280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c101850 0x7ff48c1058a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff480009910 tx=0x7ff480009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 -- 172.21.15.42:0/4212810280 shutdown_connections 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 --2- 172.21.15.42:0/4212810280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c101850 0x7ff48c1058a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 --2- 172.21.15.42:0/4212810280 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c100ea0 0x7ff48c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 -- 172.21.15.42:0/4212810280 >> 172.21.15.42:0/4212810280 conn(0x7ff48c0fc750 msgr2=0x7ff48c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 -- 172.21.15.42:0/4212810280 shutdown_connections 2024-09-17T00:03:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.109+0000 7ff493e12700 1 -- 172.21.15.42:0/4212810280 wait complete. 2024-09-17T00:03:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff493e12700 1 Processor -- start 2024-09-17T00:03:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff493e12700 1 -- start start 2024-09-17T00:03:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff493e12700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c100ea0 0x7ff48c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff493e12700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c101850 0x7ff48c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff493e12700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c19f410 con 0x7ff48c100ea0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff493e12700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c198950 con 0x7ff48c101850 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff491bae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c100ea0 0x7ff48c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff491bae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c100ea0 0x7ff48c19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60100/0 (socket says 172.21.15.42:60100) 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.110+0000 7ff491bae700 1 -- 172.21.15.42:0/4173692152 learned_addr learned my addr 172.21.15.42:0/4173692152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff4913ad700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c101850 0x7ff48c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff491bae700 1 -- 172.21.15.42:0/4173692152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c101850 msgr2=0x7ff48c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff491bae700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c101850 0x7ff48c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff491bae700 1 -- 172.21.15.42:0/4173692152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800095f0 con 0x7ff48c100ea0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff4913ad700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c101850 0x7ff48c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff491bae700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c100ea0 0x7ff48c19e8d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff48800c930 tx=0x7ff48800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff47effd700 1 -- 172.21.15.42:0/4173692152 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff488007ad0 con 0x7ff48c100ea0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff47effd700 1 -- 172.21.15.42:0/4173692152 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff488007c30 con 0x7ff48c100ea0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c198c30 con 0x7ff48c100ea0 2024-09-17T00:03:33.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff47effd700 1 -- 172.21.15.42:0/4173692152 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff488018610 con 0x7ff48c100ea0 2024-09-17T00:03:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.111+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c199180 con 0x7ff48c100ea0 2024-09-17T00:03:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.112+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c105020 con 0x7ff48c100ea0 2024-09-17T00:03:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.113+0000 7ff47effd700 1 -- 172.21.15.42:0/4173692152 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff48800f450 con 0x7ff48c100ea0 2024-09-17T00:03:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.113+0000 7ff47effd700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff47806e680 0x7ff478070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.113+0000 7ff47effd700 1 -- 172.21.15.42:0/4173692152 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff48808e780 con 0x7ff48c100ea0 2024-09-17T00:03:33.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.113+0000 7ff4913ad700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff47806e680 0x7ff478070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.114+0000 7ff4913ad700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff47806e680 0x7ff478070b40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0fcee0 tx=0x7ff48001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:33.115 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.118+0000 7ff47effd700 1 -- 172.21.15.42:0/4173692152 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff48805b4d0 con 0x7ff48c100ea0 2024-09-17T00:03:33.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.337+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff48c199e60 con 0x7ff47806e680 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.347+0000 7ff47effd700 1 -- 172.21.15.42:0/4173692152 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7ff48c199e60 con 0x7ff47806e680 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (12m) 5m ago 22m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (23m) 5m ago 23m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (22m) 5m ago 22m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (9m) 5m ago 23m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (9m) 5m ago 22m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:03:33.345 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (11m) 5m ago 22m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (5m) 5m ago 17m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (6m) 5m ago 17m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (5m) 5m ago 17m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (5m) 5m ago 17m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (14m) 5m ago 24m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (13m) 5m ago 21m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (10m) 5m ago 24m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (10m) 5m ago 21m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (12m) 5m ago 23m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (12m) 5m ago 22m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (9m) 5m ago 21m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (8m) 5m ago 20m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (8m) 5m ago 20m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:03:33.346 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (7m) 5m ago 19m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:03:33.347 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (7m) 5m ago 19m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:03:33.347 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (6m) 5m ago 18m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:03:33.347 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (12m) 5m ago 22m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:03:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff47806e680 msgr2=0x7ff478070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff47806e680 0x7ff478070b40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0fcee0 tx=0x7ff48001a040 comp rx=0 tx=0).stop 2024-09-17T00:03:33.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c100ea0 msgr2=0x7ff48c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c100ea0 0x7ff48c19e8d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff48800c930 tx=0x7ff48800ccf0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 shutdown_connections 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff47806e680 0x7ff478070b40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff48c100ea0 0x7ff48c19e8d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 --2- 172.21.15.42:0/4173692152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff48c101850 0x7ff48c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 >> 172.21.15.42:0/4173692152 conn(0x7ff48c0fc750 msgr2=0x7ff48c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 shutdown_connections 2024-09-17T00:03:33.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.349+0000 7ff493e12700 1 -- 172.21.15.42:0/4173692152 wait complete. 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 -- 172.21.15.42:0/3969928399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314100ea0 msgr2=0x7ff314101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 --2- 172.21.15.42:0/3969928399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314100ea0 0x7ff314101280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff310009910 tx=0x7ff310009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 -- 172.21.15.42:0/3969928399 shutdown_connections 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 --2- 172.21.15.42:0/3969928399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff314101850 0x7ff3141058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 --2- 172.21.15.42:0/3969928399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314100ea0 0x7ff314101280 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 -- 172.21.15.42:0/3969928399 >> 172.21.15.42:0/3969928399 conn(0x7ff3140fc750 msgr2=0x7ff3140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 -- 172.21.15.42:0/3969928399 shutdown_connections 2024-09-17T00:03:33.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.456+0000 7ff31b37b700 1 -- 172.21.15.42:0/3969928399 wait complete. 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff31b37b700 1 Processor -- start 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff31b37b700 1 -- start start 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff31b37b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff314100ea0 0x7ff314073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff31b37b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314101850 0x7ff3140735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff31b37b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314073b10 con 0x7ff314101850 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff31b37b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff314073c50 con 0x7ff314100ea0 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff318916700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314101850 0x7ff3140735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff318916700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314101850 0x7ff3140735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60112/0 (socket says 172.21.15.42:60112) 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff318916700 1 -- 172.21.15.42:0/4140181564 learned_addr learned my addr 172.21.15.42:0/4140181564 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:33.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff319117700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff314100ea0 0x7ff314073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff318916700 1 -- 172.21.15.42:0/4140181564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff314100ea0 msgr2=0x7ff314073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff318916700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff314100ea0 0x7ff314073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.457+0000 7ff318916700 1 -- 172.21.15.42:0/4140181564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100095f0 con 0x7ff314101850 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.458+0000 7ff318916700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314101850 0x7ff3140735d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e8f0 tx=0x7ff30800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.458+0000 7ff3067fc700 1 -- 172.21.15.42:0/4140181564 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff308009790 con 0x7ff314101850 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.458+0000 7ff3067fc700 1 -- 172.21.15.42:0/4140181564 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff30800c810 con 0x7ff314101850 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.458+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3141a2b10 con 0x7ff314101850 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.458+0000 7ff3067fc700 1 -- 172.21.15.42:0/4140181564 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff308005580 con 0x7ff314101850 2024-09-17T00:03:33.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.458+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141a2f20 con 0x7ff314101850 2024-09-17T00:03:33.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.459+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff314109240 con 0x7ff314101850 2024-09-17T00:03:33.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.459+0000 7ff3067fc700 1 -- 172.21.15.42:0/4140181564 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff30800c980 con 0x7ff314101850 2024-09-17T00:03:33.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.460+0000 7ff3067fc700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff30006e680 0x7ff300070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.460+0000 7ff3067fc700 1 -- 172.21.15.42:0/4140181564 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff308014070 con 0x7ff314101850 2024-09-17T00:03:33.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.460+0000 7ff319117700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff30006e680 0x7ff300070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.460+0000 7ff319117700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff30006e680 0x7ff300070b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3100059d0 tx=0x7ff310005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:33.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.464+0000 7ff3067fc700 1 -- 172.21.15.42:0/4140181564 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff308057a60 con 0x7ff314101850 2024-09-17T00:03:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:33 smithi042 ceph-mon[111056]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:33.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:33 smithi043 ceph-mon[81510]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:33.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.767+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff31404f2e0 con 0x7ff314101850 2024-09-17T00:03:33.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.767+0000 7ff3067fc700 1 -- 172.21.15.42:0/4140181564 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff30805b120 con 0x7ff314101850 2024-09-17T00:03:33.765 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:03:33.765 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:03:33.765 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:03:33.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:33.765 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:03:33.765 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:03:33.766 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:03:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff30006e680 msgr2=0x7ff300070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff30006e680 0x7ff300070b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff3100059d0 tx=0x7ff310005940 comp rx=0 tx=0).stop 2024-09-17T00:03:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314101850 msgr2=0x7ff3140735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314101850 0x7ff3140735d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff30800e8f0 tx=0x7ff30800ec00 comp rx=0 tx=0).stop 2024-09-17T00:03:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 shutdown_connections 2024-09-17T00:03:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff30006e680 0x7ff300070b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff314100ea0 0x7ff314073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 --2- 172.21.15.42:0/4140181564 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff314101850 0x7ff3140735d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.769+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 >> 172.21.15.42:0/4140181564 conn(0x7ff3140fc750 msgr2=0x7ff3140fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:33.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.770+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 shutdown_connections 2024-09-17T00:03:33.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.770+0000 7ff31b37b700 1 -- 172.21.15.42:0/4140181564 wait complete. 2024-09-17T00:03:33.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.877+0000 7f9ec4104700 1 -- 172.21.15.42:0/3485257332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 msgr2=0x7f9ebc0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.877+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3485257332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc0fffc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009910 tx=0x7f9eac009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:33.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 -- 172.21.15.42:0/3485257332 shutdown_connections 2024-09-17T00:03:33.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3485257332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc0fffc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3485257332 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc105bf0 0x7f9ebc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 -- 172.21.15.42:0/3485257332 >> 172.21.15.42:0/3485257332 conn(0x7f9ebc0fb650 msgr2=0x7f9ebc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:33.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 -- 172.21.15.42:0/3485257332 shutdown_connections 2024-09-17T00:03:33.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 -- 172.21.15.42:0/3485257332 wait complete. 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 Processor -- start 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.878+0000 7f9ec4104700 1 -- start start 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec4104700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec4104700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc105bf0 0x7f9ebc19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec4104700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc19d180 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec4104700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc196630 con 0x7f9ebc105bf0 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec1ea0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec1ea0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60136/0 (socket says 172.21.15.42:60136) 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec1ea0700 1 -- 172.21.15.42:0/3110928404 learned_addr learned my addr 172.21.15.42:0/3110928404 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec169f700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc105bf0 0x7f9ebc19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec1ea0700 1 -- 172.21.15.42:0/3110928404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc105bf0 msgr2=0x7f9ebc19caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec1ea0700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc105bf0 0x7f9ebc19caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:33.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec1ea0700 1 -- 172.21.15.42:0/3110928404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0095f0 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec1ea0700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc19c5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c8a0 tx=0x7f9eb800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:33.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9eb2ffd700 1 -- 172.21.15.42:0/3110928404 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb800cea0 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9eb2ffd700 1 -- 172.21.15.42:0/3110928404 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9eb8004830 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc196910 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9eb2ffd700 1 -- 172.21.15.42:0/3110928404 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb80055b0 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.879+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc196e60 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.880+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc04f2e0 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.881+0000 7f9eb2ffd700 1 -- 172.21.15.42:0/3110928404 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9eb80049a0 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.881+0000 7f9eb2ffd700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea806e680 0x7f9ea8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:33.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.882+0000 7f9eb2ffd700 1 -- 172.21.15.42:0/3110928404 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9eb808e3c0 con 0x7f9ebc0ffb40 2024-09-17T00:03:33.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.882+0000 7f9ec169f700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea806e680 0x7f9ea8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:33.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.882+0000 7f9ec169f700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea806e680 0x7f9ea8070b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0febe0 tx=0x7f9eac01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:33.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:33.886+0000 7f9eb2ffd700 1 -- 172.21.15.42:0/3110928404 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9eb805aff0 con 0x7f9ebc0ffb40 2024-09-17T00:03:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.141+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9ebc197280 con 0x7f9ebc0ffb40 2024-09-17T00:03:34.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.142+0000 7f9eb2ffd700 1 -- 172.21.15.42:0/3110928404 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f9eb8004c50 con 0x7f9ebc0ffb40 2024-09-17T00:03:34.139 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:03:34.140 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:03:34.141 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:34.142 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:03:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea806e680 msgr2=0x7f9ea8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea806e680 0x7f9ea8070b40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0febe0 tx=0x7f9eac01a040 comp rx=0 tx=0).stop 2024-09-17T00:03:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 msgr2=0x7f9ebc19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc19c5b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800c8a0 tx=0x7f9eb800cc60 comp rx=0 tx=0).stop 2024-09-17T00:03:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 shutdown_connections 2024-09-17T00:03:34.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea806e680 0x7f9ea8070b40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ebc0ffb40 0x7f9ebc19c5b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 --2- 172.21.15.42:0/3110928404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ebc105bf0 0x7f9ebc19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 >> 172.21.15.42:0/3110928404 conn(0x7f9ebc0fb650 msgr2=0x7f9ebc068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 shutdown_connections 2024-09-17T00:03:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.144+0000 7f9ec4104700 1 -- 172.21.15.42:0/3110928404 wait complete. 2024-09-17T00:03:34.144 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:03:34.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/3967131220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a00ffda0 msgr2=0x7fb4a0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/3967131220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a00ffda0 0x7fb4a0100180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb490009910 tx=0x7fb490009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:34.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/3967131220 shutdown_connections 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/3967131220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a0100750 0x7fb4a01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/3967131220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a00ffda0 0x7fb4a0100180 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/3967131220 >> 172.21.15.42:0/3967131220 conn(0x7fb4a00fb650 msgr2=0x7fb4a00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/3967131220 shutdown_connections 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/3967131220 wait complete. 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.253+0000 7fb4a5d9a700 1 Processor -- start 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb4a5d9a700 1 -- start start 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb4a5d9a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a00ffda0 0x7fb4a019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb4a5d9a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a0100750 0x7fb4a019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb4a5d9a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a019d2b0 con 0x7fb4a0100750 2024-09-17T00:03:34.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb4a5d9a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a0196760 con 0x7fb4a00ffda0 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb49effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a0100750 0x7fb4a019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb49effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a0100750 0x7fb4a019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60158/0 (socket says 172.21.15.42:60158) 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb49effd700 1 -- 172.21.15.42:0/883172576 learned_addr learned my addr 172.21.15.42:0/883172576 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb49f7fe700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a00ffda0 0x7fb4a019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb49effd700 1 -- 172.21.15.42:0/883172576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a00ffda0 msgr2=0x7fb4a019c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb49effd700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a00ffda0 0x7fb4a019c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.254+0000 7fb49effd700 1 -- 172.21.15.42:0/883172576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900095f0 con 0x7fb4a0100750 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.255+0000 7fb49effd700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a0100750 0x7fb4a019cc20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c8f0 tx=0x7fb49400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.255+0000 7fb49cff9700 1 -- 172.21.15.42:0/883172576 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb494007a50 con 0x7fb4a0100750 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.255+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a0196a40 con 0x7fb4a0100750 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.255+0000 7fb49cff9700 1 -- 172.21.15.42:0/883172576 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb494007bb0 con 0x7fb4a0100750 2024-09-17T00:03:34.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.255+0000 7fb49cff9700 1 -- 172.21.15.42:0/883172576 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb494018630 con 0x7fb4a0100750 2024-09-17T00:03:34.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.255+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a0196f90 con 0x7fb4a0100750 2024-09-17T00:03:34.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.256+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a004f2e0 con 0x7fb4a0100750 2024-09-17T00:03:34.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.257+0000 7fb49cff9700 1 -- 172.21.15.42:0/883172576 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb494007d20 con 0x7fb4a0100750 2024-09-17T00:03:34.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.257+0000 7fb49cff9700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb48806e680 0x7fb488070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:34.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.257+0000 7fb49cff9700 1 -- 172.21.15.42:0/883172576 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb49408e920 con 0x7fb4a0100750 2024-09-17T00:03:34.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.257+0000 7fb49f7fe700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb48806e680 0x7fb488070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:34.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.258+0000 7fb49f7fe700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb48806e680 0x7fb488070b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb490006010 tx=0x7fb49001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:34.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.261+0000 7fb49cff9700 1 -- 172.21.15.42:0/883172576 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb49405b710 con 0x7fb4a0100750 2024-09-17T00:03:34.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:34 smithi042 ceph-mon[111056]: pgmap v359: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:34.428 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:34 smithi042 ceph-mon[111056]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:34.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:34 smithi042 ceph-mon[111056]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:34.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:34 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4140181564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:03:34.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:34 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3110928404' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:03:34.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.494+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4a01972b0 con 0x7fb48806e680 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.495+0000 7fb49cff9700 1 -- 172.21.15.42:0/883172576 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb4a01972b0 con 0x7fb48806e680 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:03:34.493 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:03:34.494 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:03:34.494 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:03:34.494 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:03:34.494 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:03:34.494 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:03:34.494 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:03:34.494 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:03:34.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.497+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb48806e680 msgr2=0x7fb488070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.497+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb48806e680 0x7fb488070b40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb490006010 tx=0x7fb49001a040 comp rx=0 tx=0).stop 2024-09-17T00:03:34.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.497+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a0100750 msgr2=0x7fb4a019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.497+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a0100750 0x7fb4a019cc20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c8f0 tx=0x7fb49400ccb0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.498+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 shutdown_connections 2024-09-17T00:03:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.498+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb48806e680 0x7fb488070b40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.498+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a00ffda0 0x7fb4a019c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.498+0000 7fb4a5d9a700 1 --2- 172.21.15.42:0/883172576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4a0100750 0x7fb4a019cc20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.498+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 >> 172.21.15.42:0/883172576 conn(0x7fb4a00fb650 msgr2=0x7fb4a0103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.498+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 shutdown_connections 2024-09-17T00:03:34.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.498+0000 7fb4a5d9a700 1 -- 172.21.15.42:0/883172576 wait complete. 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 -- 172.21.15.42:0/2976471523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 msgr2=0x7fc9080689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 --2- 172.21.15.42:0/2976471523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080689e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0009910 tx=0x7fc8f0009c20 comp rx=0 tx=0).stop 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 -- 172.21.15.42:0/2976471523 shutdown_connections 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 --2- 172.21.15.42:0/2976471523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080689e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 --2- 172.21.15.42:0/2976471523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc908106cd0 0x7fc9081070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 -- 172.21.15.42:0/2976471523 >> 172.21.15.42:0/2976471523 conn(0x7fc908075670 msgr2=0x7fc908075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 -- 172.21.15.42:0/2976471523 shutdown_connections 2024-09-17T00:03:34.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.606+0000 7fc90cd14700 1 -- 172.21.15.42:0/2976471523 wait complete. 2024-09-17T00:03:34.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90cd14700 1 Processor -- start 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90cd14700 1 -- start start 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90cd14700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90cd14700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc908106cd0 0x7fc908100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90cd14700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc908100770 con 0x7fc908068580 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90cd14700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9081008b0 con 0x7fc908106cd0 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60174/0 (socket says 172.21.15.42:60174) 2024-09-17T00:03:34.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.607+0000 7fc90659c700 1 -- 172.21.15.42:0/1732324435 learned_addr learned my addr 172.21.15.42:0/1732324435 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc905d9b700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc908106cd0 0x7fc908100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc90659c700 1 -- 172.21.15.42:0/1732324435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc908106cd0 msgr2=0x7fc908100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc90659c700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc908106cd0 0x7fc908100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc90659c700 1 -- 172.21.15.42:0/1732324435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00095f0 con 0x7fc908068580 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc90659c700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080ffcf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800ef10 tx=0x7fc8f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc8ff7fe700 1 -- 172.21.15.42:0/1732324435 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f800ca50 con 0x7fc908068580 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc8ff7fe700 1 -- 172.21.15.42:0/1732324435 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f800f040 con 0x7fc908068580 2024-09-17T00:03:34.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc908100b30 con 0x7fc908068580 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc8ff7fe700 1 -- 172.21.15.42:0/1732324435 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f80146b0 con 0x7fc908068580 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.608+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9081a2dc0 con 0x7fc908068580 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.609+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9081091d0 con 0x7fc908068580 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.610+0000 7fc8ff7fe700 1 -- 172.21.15.42:0/1732324435 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc8f800cbb0 con 0x7fc908068580 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.610+0000 7fc8ff7fe700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8f406e680 0x7fc8f4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.611+0000 7fc8ff7fe700 1 -- 172.21.15.42:0/1732324435 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc8f808e9b0 con 0x7fc908068580 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.611+0000 7fc905d9b700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8f406e680 0x7fc8f4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:03:34.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.611+0000 7fc905d9b700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8f406e680 0x7fc8f4070b40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000b760 tx=0x7fc8f0009f90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:03:34.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.615+0000 7fc8ff7fe700 1 -- 172.21.15.42:0/1732324435 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc8f8057bd0 con 0x7fc908068580 2024-09-17T00:03:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:34 smithi043 ceph-mon[81510]: pgmap v359: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:34 smithi043 ceph-mon[81510]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:34 smithi043 ceph-mon[81510]: from='client.34720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:34 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4140181564' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:03:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:34 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3110928404' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:03:34.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.911+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc90802cc70 con 0x7fc908068580 2024-09-17T00:03:34.909 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.912+0000 7fc8ff7fe700 1 -- 172.21.15.42:0/1732324435 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc8f805b290 con 0x7fc908068580 2024-09-17T00:03:34.910 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:03:34.910 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:03:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:03:34.910 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:03:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:03:34.910 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:03:34.910 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:03:34.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8f406e680 msgr2=0x7fc8f4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8f406e680 0x7fc8f4070b40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000b760 tx=0x7fc8f0009f90 comp rx=0 tx=0).stop 2024-09-17T00:03:34.911 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 msgr2=0x7fc9080ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080ffcf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800ef10 tx=0x7fc8f800c3a0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 shutdown_connections 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8f406e680 0x7fc8f4070b40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc908068580 0x7fc9080ffcf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 --2- 172.21.15.42:0/1732324435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc908106cd0 0x7fc908100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 >> 172.21.15.42:0/1732324435 conn(0x7fc908075670 msgr2=0x7fc9080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 shutdown_connections 2024-09-17T00:03:34.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:03:34.914+0000 7fc90cd14700 1 -- 172.21.15.42:0/1732324435 wait complete. 2024-09-17T00:03:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:35 smithi043 ceph-mon[81510]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:35.699 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:35 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1732324435' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:03:35.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:35 smithi042 ceph-mon[111056]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:03:35.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:35 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1732324435' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:03:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:36 smithi043 ceph-mon[81510]: pgmap v360: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:36.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:36 smithi042 ceph-mon[111056]: pgmap v360: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:38.533 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:38 smithi043 ceph-mon[81510]: pgmap v361: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:38.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:38 smithi042 ceph-mon[111056]: pgmap v361: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:40.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:40 smithi043 ceph-mon[81510]: pgmap v362: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:40.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:40 smithi042 ceph-mon[111056]: pgmap v362: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:42.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:42 smithi043 ceph-mon[81510]: pgmap v363: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:42.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:42 smithi042 ceph-mon[111056]: pgmap v363: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:44 smithi043 ceph-mon[81510]: pgmap v364: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:44 smithi042 ceph-mon[111056]: pgmap v364: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:46.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:46 smithi043 ceph-mon[81510]: pgmap v365: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:46 smithi042 ceph-mon[111056]: pgmap v365: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:48.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:48 smithi043 ceph-mon[81510]: pgmap v366: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:48.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:48 smithi042 ceph-mon[111056]: pgmap v366: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:50.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:50 smithi043 ceph-mon[81510]: pgmap v367: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:50 smithi042 ceph-mon[111056]: pgmap v367: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:51.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:03:52.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:52 smithi043 ceph-mon[81510]: pgmap v368: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:52.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:52 smithi042 ceph-mon[111056]: pgmap v368: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:54.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:54 smithi042 ceph-mon[111056]: pgmap v369: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:54 smithi043 ceph-mon[81510]: pgmap v369: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:56.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:56 smithi043 ceph-mon[81510]: pgmap v370: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:56 smithi042 ceph-mon[111056]: pgmap v370: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:58.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:03:58 smithi043 ceph-mon[81510]: pgmap v371: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:03:58.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:03:58 smithi042 ceph-mon[111056]: pgmap v371: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:00.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:00 smithi043 ceph-mon[81510]: pgmap v372: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:00.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:00 smithi042 ceph-mon[111056]: pgmap v372: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:02.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:02 smithi043 ceph-mon[81510]: pgmap v373: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:02.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:02 smithi042 ceph-mon[111056]: pgmap v373: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:04.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:04 smithi043 ceph-mon[81510]: pgmap v374: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:04.740 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:04 smithi042 ceph-mon[111056]: pgmap v374: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.030+0000 7f8df51c7700 1 -- 172.21.15.42:0/2565398024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 msgr2=0x7f8df00ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.030+0000 7f8df51c7700 1 --2- 172.21.15.42:0/2565398024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df00ff980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009910 tx=0x7f8de0009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.030+0000 7f8df51c7700 1 -- 172.21.15.42:0/2565398024 shutdown_connections 2024-09-17T00:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.030+0000 7f8df51c7700 1 --2- 172.21.15.42:0/2565398024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df00fff50 0x7f8df0105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.030+0000 7f8df51c7700 1 --2- 172.21.15.42:0/2565398024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df00ff980 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.030+0000 7f8df51c7700 1 -- 172.21.15.42:0/2565398024 >> 172.21.15.42:0/2565398024 conn(0x7f8df0074e30 msgr2=0x7f8df0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.031+0000 7f8df51c7700 1 -- 172.21.15.42:0/2565398024 shutdown_connections 2024-09-17T00:04:05.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.031+0000 7f8df51c7700 1 -- 172.21.15.42:0/2565398024 wait complete. 2024-09-17T00:04:05.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.031+0000 7f8df51c7700 1 Processor -- start 2024-09-17T00:04:05.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.031+0000 7f8df51c7700 1 -- start start 2024-09-17T00:04:05.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8df51c7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8df51c7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df00fff50 0x7f8df019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8df51c7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df019d3b0 con 0x7f8df00ff5a0 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8df51c7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df0196860 con 0x7f8df00fff50 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8deed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8deed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36784/0 (socket says 172.21.15.42:36784) 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8deed9d700 1 -- 172.21.15.42:0/252563901 learned_addr learned my addr 172.21.15.42:0/252563901 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8dee59c700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df00fff50 0x7f8df019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8deed9d700 1 -- 172.21.15.42:0/252563901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df00fff50 msgr2=0x7f8df019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8deed9d700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df00fff50 0x7f8df019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8deed9d700 1 -- 172.21.15.42:0/252563901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00095f0 con 0x7f8df00ff5a0 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.032+0000 7f8deed9d700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df019c7e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8de0006010 tx=0x7f8de000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.033+0000 7f8dd7fff700 1 -- 172.21.15.42:0/252563901 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de001d070 con 0x7f8df00ff5a0 2024-09-17T00:04:05.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.033+0000 7f8dd7fff700 1 -- 172.21.15.42:0/252563901 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8de000f460 con 0x7f8df00ff5a0 2024-09-17T00:04:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.033+0000 7f8dd7fff700 1 -- 172.21.15.42:0/252563901 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8de0017610 con 0x7f8df00ff5a0 2024-09-17T00:04:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.033+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df0196ae0 con 0x7f8df00ff5a0 2024-09-17T00:04:05.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.033+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df0196fd0 con 0x7f8df00ff5a0 2024-09-17T00:04:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.034+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df0109270 con 0x7f8df00ff5a0 2024-09-17T00:04:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.034+0000 7f8dd7fff700 1 -- 172.21.15.42:0/252563901 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8de000f5d0 con 0x7f8df00ff5a0 2024-09-17T00:04:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.035+0000 7f8dd7fff700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd806e680 0x7f8dd8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.035+0000 7f8dd7fff700 1 -- 172.21.15.42:0/252563901 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8de008fb60 con 0x7f8df00ff5a0 2024-09-17T00:04:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.035+0000 7f8dee59c700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd806e680 0x7f8dd8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.035+0000 7f8dee59c700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd806e680 0x7f8dd8070b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8df0107790 tx=0x7f8de4009190 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:05.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.040+0000 7f8dd7fff700 1 -- 172.21.15.42:0/252563901 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8de005c8b0 con 0x7f8df00ff5a0 2024-09-17T00:04:05.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.271+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8df0197e30 con 0x7f8dd806e680 2024-09-17T00:04:05.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.272+0000 7f8dd7fff700 1 -- 172.21.15.42:0/252563901 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8df0197e30 con 0x7f8dd806e680 2024-09-17T00:04:05.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd806e680 msgr2=0x7f8dd8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd806e680 0x7f8dd8070b40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8df0107790 tx=0x7f8de4009190 comp rx=0 tx=0).stop 2024-09-17T00:04:05.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 msgr2=0x7f8df019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df019c7e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8de0006010 tx=0x7f8de000bd70 comp rx=0 tx=0).stop 2024-09-17T00:04:05.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 shutdown_connections 2024-09-17T00:04:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd806e680 0x7f8dd8070b40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8df00ff5a0 0x7f8df019c7e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 --2- 172.21.15.42:0/252563901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df00fff50 0x7f8df019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.274+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 >> 172.21.15.42:0/252563901 conn(0x7f8df0074e30 msgr2=0x7f8df00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.275+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 shutdown_connections 2024-09-17T00:04:05.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.275+0000 7f8df51c7700 1 -- 172.21.15.42:0/252563901 wait complete. 2024-09-17T00:04:05.283 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:04:05.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 -- 172.21.15.42:0/3075717778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 msgr2=0x7f729c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/3075717778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c0fffc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7284009910 tx=0x7f7284009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:05.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 -- 172.21.15.42:0/3075717778 shutdown_connections 2024-09-17T00:04:05.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/3075717778 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c0fffc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/3075717778 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f729c105bf0 0x7f729c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 -- 172.21.15.42:0/3075717778 >> 172.21.15.42:0/3075717778 conn(0x7f729c0fb610 msgr2=0x7f729c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:05.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 -- 172.21.15.42:0/3075717778 shutdown_connections 2024-09-17T00:04:05.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.385+0000 7f72a2e3d700 1 -- 172.21.15.42:0/3075717778 wait complete. 2024-09-17T00:04:05.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a2e3d700 1 Processor -- start 2024-09-17T00:04:05.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a2e3d700 1 -- start start 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a2e3d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a2e3d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f729c105bf0 0x7f729c196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a2e3d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c197530 con 0x7f729c0ffb40 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a2e3d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c19b340 con 0x7f729c105bf0 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a0bd9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a0bd9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36802/0 (socket says 172.21.15.42:36802) 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.386+0000 7f72a0bd9700 1 -- 172.21.15.42:0/1153191266 learned_addr learned my addr 172.21.15.42:0/1153191266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f729bfff700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f729c105bf0 0x7f729c196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f72a0bd9700 1 -- 172.21.15.42:0/1153191266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f729c105bf0 msgr2=0x7f729c196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f72a0bd9700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f729c105bf0 0x7f729c196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f72a0bd9700 1 -- 172.21.15.42:0/1153191266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840095f0 con 0x7f729c0ffb40 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f72a0bd9700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c196960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f728c00b4b0 tx=0x7f728c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:05.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f7299ffb700 1 -- 172.21.15.42:0/1153191266 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c011800 con 0x7f729c0ffb40 2024-09-17T00:04:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f7299ffb700 1 -- 172.21.15.42:0/1153191266 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f728c011e20 con 0x7f729c0ffb40 2024-09-17T00:04:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c19b620 con 0x7f729c0ffb40 2024-09-17T00:04:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f7299ffb700 1 -- 172.21.15.42:0/1153191266 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728c00f2e0 con 0x7f729c0ffb40 2024-09-17T00:04:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.387+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c19bb70 con 0x7f729c0ffb40 2024-09-17T00:04:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.388+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c04f2e0 con 0x7f729c0ffb40 2024-09-17T00:04:05.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.389+0000 7f7299ffb700 1 -- 172.21.15.42:0/1153191266 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f728c010380 con 0x7f729c0ffb40 2024-09-17T00:04:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.389+0000 7f7299ffb700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f728806e680 0x7f7288070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.390+0000 7f729bfff700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f728806e680 0x7f7288070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.390+0000 7f7299ffb700 1 -- 172.21.15.42:0/1153191266 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f728c08e180 con 0x7f729c0ffb40 2024-09-17T00:04:05.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.390+0000 7f729bfff700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f728806e680 0x7f7288070b40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f729c197f80 tx=0x7f728401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:05.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.394+0000 7f7299ffb700 1 -- 172.21.15.42:0/1153191266 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f728c05af80 con 0x7f729c0ffb40 2024-09-17T00:04:05.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.624+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f729c19beb0 con 0x7f728806e680 2024-09-17T00:04:05.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.626+0000 7f7299ffb700 1 -- 172.21.15.42:0/1153191266 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f729c19beb0 con 0x7f728806e680 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f728806e680 msgr2=0x7f7288070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f728806e680 0x7f7288070b40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f729c197f80 tx=0x7f728401a040 comp rx=0 tx=0).stop 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 msgr2=0x7f729c196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c196960 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f728c00b4b0 tx=0x7f728c00b870 comp rx=0 tx=0).stop 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 shutdown_connections 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f728806e680 0x7f7288070b40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c0ffb40 0x7f729c196960 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 --2- 172.21.15.42:0/1153191266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f729c105bf0 0x7f729c196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 >> 172.21.15.42:0/1153191266 conn(0x7f729c0fb610 msgr2=0x7f729c068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 shutdown_connections 2024-09-17T00:04:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.628+0000 7f72a2e3d700 1 -- 172.21.15.42:0/1153191266 wait complete. 2024-09-17T00:04:05.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 -- 172.21.15.42:0/1128127769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 msgr2=0x7f33801047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1128127769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f33801047a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3374009910 tx=0x7f3374009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 -- 172.21.15.42:0/1128127769 shutdown_connections 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1128127769 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f33801047a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1128127769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33800ffda0 0x7f3380100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 -- 172.21.15.42:0/1128127769 >> 172.21.15.42:0/1128127769 conn(0x7f33800fb650 msgr2=0x7f33800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 -- 172.21.15.42:0/1128127769 shutdown_connections 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.737+0000 7f3385ac1700 1 -- 172.21.15.42:0/1128127769 wait complete. 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f3385ac1700 1 Processor -- start 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f3385ac1700 1 -- start start 2024-09-17T00:04:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f3385ac1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33800ffda0 0x7f338019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f3385ac1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f338019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f3385ac1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338019d240 con 0x7f3380100750 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f3385ac1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33801966f0 con 0x7f33800ffda0 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f337effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f338019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f337effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f338019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36816/0 (socket says 172.21.15.42:36816) 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.738+0000 7f337effd700 1 -- 172.21.15.42:0/1704961385 learned_addr learned my addr 172.21.15.42:0/1704961385 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337f7fe700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33800ffda0 0x7f338019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337effd700 1 -- 172.21.15.42:0/1704961385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33800ffda0 msgr2=0x7f338019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337effd700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33800ffda0 0x7f338019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337effd700 1 -- 172.21.15.42:0/1704961385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33740095f0 con 0x7f3380100750 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337effd700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f338019cbb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33740059d0 tx=0x7f337400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337cff9700 1 -- 172.21.15.42:0/1704961385 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f337401d070 con 0x7f3380100750 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3380196970 con 0x7f3380100750 2024-09-17T00:04:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337cff9700 1 -- 172.21.15.42:0/1704961385 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f337400b610 con 0x7f3380100750 2024-09-17T00:04:05.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f337cff9700 1 -- 172.21.15.42:0/1704961385 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3374022470 con 0x7f3380100750 2024-09-17T00:04:05.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.739+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3380196e60 con 0x7f3380100750 2024-09-17T00:04:05.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.740+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33801091b0 con 0x7f3380100750 2024-09-17T00:04:05.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.741+0000 7f337cff9700 1 -- 172.21.15.42:0/1704961385 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f337400fc70 con 0x7f3380100750 2024-09-17T00:04:05.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.741+0000 7f337cff9700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f336806e680 0x7f3368070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:05.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.741+0000 7f337cff9700 1 -- 172.21.15.42:0/1704961385 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f337408faf0 con 0x7f3380100750 2024-09-17T00:04:05.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.741+0000 7f337f7fe700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f336806e680 0x7f3368070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:05.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.742+0000 7f337f7fe700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f336806e680 0x7f3368070b40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3370009bf0 tx=0x7f3370009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:05.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.746+0000 7f337cff9700 1 -- 172.21.15.42:0/1704961385 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f337405c7c0 con 0x7f3380100750 2024-09-17T00:04:05.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.961+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3380197140 con 0x7f336806e680 2024-09-17T00:04:05.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.971+0000 7f337cff9700 1 -- 172.21.15.42:0/1704961385 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f3380197140 con 0x7f336806e680 2024-09-17T00:04:05.968 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:04:05.968 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (12m) 6m ago 23m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:04:05.968 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (23m) 6m ago 23m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:04:05.968 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (22m) 5m ago 22m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (10m) 6m ago 23m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (10m) 5m ago 22m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (12m) 6m ago 23m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (6m) 6m ago 17m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (6m) 6m ago 17m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (6m) 5m ago 17m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (5m) 5m ago 17m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (14m) 6m ago 24m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (13m) 5m ago 22m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (11m) 6m ago 25m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (11m) 5m ago 22m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (13m) 6m ago 23m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:04:05.969 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (13m) 5m ago 22m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (9m) 6m ago 21m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (9m) 6m ago 21m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (8m) 6m ago 20m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (8m) 5m ago 20m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (7m) 5m ago 19m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (7m) 5m ago 19m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (13m) 6m ago 23m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:04:05.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f336806e680 msgr2=0x7f3368070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f336806e680 0x7f3368070b40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3370009bf0 tx=0x7f3370009260 comp rx=0 tx=0).stop 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 msgr2=0x7f338019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f338019cbb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f33740059d0 tx=0x7f337400f740 comp rx=0 tx=0).stop 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 shutdown_connections 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f336806e680 0x7f3368070b40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33800ffda0 0x7f338019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 --2- 172.21.15.42:0/1704961385 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3380100750 0x7f338019cbb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 >> 172.21.15.42:0/1704961385 conn(0x7f33800fb650 msgr2=0x7f3380069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 shutdown_connections 2024-09-17T00:04:05.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:05.973+0000 7f3385ac1700 1 -- 172.21.15.42:0/1704961385 wait complete. 2024-09-17T00:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 -- 172.21.15.42:0/1514023527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c0ffda0 msgr2=0x7f058c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 --2- 172.21.15.42:0/1514023527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c0ffda0 0x7f058c100180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f057c009910 tx=0x7f057c009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 -- 172.21.15.42:0/1514023527 shutdown_connections 2024-09-17T00:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 --2- 172.21.15.42:0/1514023527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c100750 0x7f058c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 --2- 172.21.15.42:0/1514023527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c0ffda0 0x7f058c100180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 -- 172.21.15.42:0/1514023527 >> 172.21.15.42:0/1514023527 conn(0x7f058c0fb650 msgr2=0x7f058c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:06.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 -- 172.21.15.42:0/1514023527 shutdown_connections 2024-09-17T00:04:06.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.080+0000 7f059150d700 1 -- 172.21.15.42:0/1514023527 wait complete. 2024-09-17T00:04:06.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.081+0000 7f059150d700 1 Processor -- start 2024-09-17T00:04:06.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.081+0000 7f059150d700 1 -- start start 2024-09-17T00:04:06.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.081+0000 7f059150d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c0ffda0 0x7f058c19c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.081+0000 7f059150d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c100750 0x7f058c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.081+0000 7f059150d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c19d2b0 con 0x7f058c100750 2024-09-17T00:04:06.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.081+0000 7f059150d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c196760 con 0x7f058c0ffda0 2024-09-17T00:04:06.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c100750 0x7f058c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c100750 0x7f058c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36844/0 (socket says 172.21.15.42:36844) 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058a7fc700 1 -- 172.21.15.42:0/2949874525 learned_addr learned my addr 172.21.15.42:0/2949874525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058affd700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c0ffda0 0x7f058c19c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058a7fc700 1 -- 172.21.15.42:0/2949874525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c0ffda0 msgr2=0x7f058c19c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058a7fc700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c0ffda0 0x7f058c19c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058a7fc700 1 -- 172.21.15.42:0/2949874525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f057c0095f0 con 0x7f058c100750 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f058a7fc700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c100750 0x7f058c19cc20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f058000d900 tx=0x7f058000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f0573fff700 1 -- 172.21.15.42:0/2949874525 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0580009900 con 0x7f058c100750 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f0573fff700 1 -- 172.21.15.42:0/2949874525 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0580010460 con 0x7f058c100750 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f0573fff700 1 -- 172.21.15.42:0/2949874525 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058000f5f0 con 0x7f058c100750 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f058c196a40 con 0x7f058c100750 2024-09-17T00:04:06.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.082+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c196f90 con 0x7f058c100750 2024-09-17T00:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.083+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c04f2e0 con 0x7f058c100750 2024-09-17T00:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.084+0000 7f0573fff700 1 -- 172.21.15.42:0/2949874525 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f05800105d0 con 0x7f058c100750 2024-09-17T00:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.084+0000 7f0573fff700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f057406e680 0x7f0574070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.084+0000 7f0573fff700 1 -- 172.21.15.42:0/2949874525 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f058008e500 con 0x7f058c100750 2024-09-17T00:04:06.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.085+0000 7f058affd700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f057406e680 0x7f0574070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.085+0000 7f058affd700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f057406e680 0x7f0574070b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f057c006010 tx=0x7f057c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:06.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.089+0000 7f0573fff700 1 -- 172.21.15.42:0/2949874525 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f058005b1d0 con 0x7f058c100750 2024-09-17T00:04:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.383+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f058c197c70 con 0x7f058c100750 2024-09-17T00:04:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.384+0000 7f0573fff700 1 -- 172.21.15.42:0/2949874525 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f058005ad60 con 0x7f058c100750 2024-09-17T00:04:06.382 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:04:06.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:04:06.382 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:04:06.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:06.382 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:04:06.382 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:04:06.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:04:06.383 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:04:06.384 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:04:06.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f057406e680 msgr2=0x7f0574070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f057406e680 0x7f0574070b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f057c006010 tx=0x7f057c00b540 comp rx=0 tx=0).stop 2024-09-17T00:04:06.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c100750 msgr2=0x7f058c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c100750 0x7f058c19cc20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f058000d900 tx=0x7f058000dcc0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 shutdown_connections 2024-09-17T00:04:06.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f057406e680 0x7f0574070b40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f058c0ffda0 0x7f058c19c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 --2- 172.21.15.42:0/2949874525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f058c100750 0x7f058c19cc20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 >> 172.21.15.42:0/2949874525 conn(0x7f058c0fb650 msgr2=0x7f058c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:06.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 shutdown_connections 2024-09-17T00:04:06.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.386+0000 7f059150d700 1 -- 172.21.15.42:0/2949874525 wait complete. 2024-09-17T00:04:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 -- 172.21.15.42:0/3018895646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 msgr2=0x7f6eac100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 --2- 172.21.15.42:0/3018895646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac100180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009910 tx=0x7f6e9c009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 -- 172.21.15.42:0/3018895646 shutdown_connections 2024-09-17T00:04:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 --2- 172.21.15.42:0/3018895646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eac100750 0x7f6eac1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 --2- 172.21.15.42:0/3018895646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac100180 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 -- 172.21.15.42:0/3018895646 >> 172.21.15.42:0/3018895646 conn(0x7f6eac0fb650 msgr2=0x7f6eac0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 -- 172.21.15.42:0/3018895646 shutdown_connections 2024-09-17T00:04:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.494+0000 7f6eb2598700 1 -- 172.21.15.42:0/3018895646 wait complete. 2024-09-17T00:04:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eb2598700 1 Processor -- start 2024-09-17T00:04:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eb2598700 1 -- start start 2024-09-17T00:04:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eb2598700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eb2598700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eac100750 0x7f6eac197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eb2598700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac198250 con 0x7f6eac0ffda0 2024-09-17T00:04:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eb2598700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1983c0 con 0x7f6eac100750 2024-09-17T00:04:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eabfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eabfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36860/0 (socket says 172.21.15.42:36860) 2024-09-17T00:04:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eabfff700 1 -- 172.21.15.42:0/1579218770 learned_addr learned my addr 172.21.15.42:0/1579218770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.495+0000 7f6eab7fe700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eac100750 0x7f6eac197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6eabfff700 1 -- 172.21.15.42:0/1579218770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eac100750 msgr2=0x7f6eac197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6eabfff700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eac100750 0x7f6eac197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6eabfff700 1 -- 172.21.15.42:0/1579218770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0095f0 con 0x7f6eac0ffda0 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6eabfff700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac1976f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c005f50 tx=0x7f6e9c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6ea97fa700 1 -- 172.21.15.42:0/1579218770 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e9c01d070 con 0x7f6eac0ffda0 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6ea97fa700 1 -- 172.21.15.42:0/1579218770 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e9c00b610 con 0x7f6eac0ffda0 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6ea97fa700 1 -- 172.21.15.42:0/1579218770 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e9c022470 con 0x7f6eac0ffda0 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac1a0760 con 0x7f6eac0ffda0 2024-09-17T00:04:06.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.496+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac1a0bd0 con 0x7f6eac0ffda0 2024-09-17T00:04:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.497+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac108140 con 0x7f6eac0ffda0 2024-09-17T00:04:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.498+0000 7f6ea97fa700 1 -- 172.21.15.42:0/1579218770 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6e9c00fca0 con 0x7f6eac0ffda0 2024-09-17T00:04:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.498+0000 7f6ea97fa700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9406e680 0x7f6e94070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.498+0000 7f6ea97fa700 1 -- 172.21.15.42:0/1579218770 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6e9c08fa40 con 0x7f6eac0ffda0 2024-09-17T00:04:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.498+0000 7f6eab7fe700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9406e680 0x7f6e94070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.499+0000 7f6eab7fe700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9406e680 0x7f6e94070b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0696e0 tx=0x7f6ea0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:06.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.502+0000 7f6ea97fa700 1 -- 172.21.15.42:0/1579218770 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6e9c058c60 con 0x7f6eac0ffda0 2024-09-17T00:04:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:06 smithi043 ceph-mon[81510]: pgmap v375: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:06 smithi043 ceph-mon[81510]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:06 smithi043 ceph-mon[81510]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:06.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.758+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6eac107b20 con 0x7f6eac0ffda0 2024-09-17T00:04:06.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:06 smithi042 ceph-mon[111056]: pgmap v375: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:06.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:06 smithi042 ceph-mon[111056]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:06.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:06 smithi042 ceph-mon[111056]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:06.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:06.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.758+0000 7f6ea97fa700 1 -- 172.21.15.42:0/1579218770 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f6e9c05c320 con 0x7f6eac0ffda0 2024-09-17T00:04:06.757 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:04:06.757 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:04:06.757 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:04:06.757 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:04:06.757 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:04:06.758 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:04:06.759 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:04:06.760 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:04:06.760 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:04:06.760 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:04:06.760 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:04:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:06.760 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:06.761 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:04:06.761 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:06.761 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:06.761 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:06.761 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:06.762 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9406e680 msgr2=0x7f6e94070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9406e680 0x7f6e94070b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0696e0 tx=0x7f6ea0009260 comp rx=0 tx=0).stop 2024-09-17T00:04:06.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 msgr2=0x7f6eac1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac1976f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c005f50 tx=0x7f6e9c00f740 comp rx=0 tx=0).stop 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 shutdown_connections 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9406e680 0x7f6e94070b40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eac0ffda0 0x7f6eac1976f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 --2- 172.21.15.42:0/1579218770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eac100750 0x7f6eac197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.760+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 >> 172.21.15.42:0/1579218770 conn(0x7f6eac0fb650 msgr2=0x7f6eac0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.761+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 shutdown_connections 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.761+0000 7f6eb2598700 1 -- 172.21.15.42:0/1579218770 wait complete. 2024-09-17T00:04:06.764 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 -- 172.21.15.42:0/1294226102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0068580 msgr2=0x7f05f00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1294226102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0068580 0x7f05f00689e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009910 tx=0x7f05e0009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 -- 172.21.15.42:0/1294226102 shutdown_connections 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1294226102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0068580 0x7f05f00689e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1294226102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f0106cd0 0x7f05f01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 -- 172.21.15.42:0/1294226102 >> 172.21.15.42:0/1294226102 conn(0x7f05f0075670 msgr2=0x7f05f0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 -- 172.21.15.42:0/1294226102 shutdown_connections 2024-09-17T00:04:06.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.870+0000 7f05f73ae700 1 -- 172.21.15.42:0/1294226102 wait complete. 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f73ae700 1 Processor -- start 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f73ae700 1 -- start start 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f73ae700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f0068580 0x7f05f019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f73ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0106cd0 0x7f05f019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f73ae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f019d170 con 0x7f05f0106cd0 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f73ae700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f01966b0 con 0x7f05f0068580 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f4949700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0106cd0 0x7f05f019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f4949700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0106cd0 0x7f05f019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36882/0 (socket says 172.21.15.42:36882) 2024-09-17T00:04:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f4949700 1 -- 172.21.15.42:0/1897786845 learned_addr learned my addr 172.21.15.42:0/1897786845 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.871+0000 7f05f514a700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f0068580 0x7f05f019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05f4949700 1 -- 172.21.15.42:0/1897786845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f0068580 msgr2=0x7f05f019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05f4949700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f0068580 0x7f05f019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05f4949700 1 -- 172.21.15.42:0/1897786845 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00095f0 con 0x7f05f0106cd0 2024-09-17T00:04:06.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05f4949700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0106cd0 0x7f05f019cb70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006010 tx=0x7f05e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05e67fc700 1 -- 172.21.15.42:0/1897786845 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e001d070 con 0x7f05f0106cd0 2024-09-17T00:04:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05e67fc700 1 -- 172.21.15.42:0/1897786845 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05e000f460 con 0x7f05f0106cd0 2024-09-17T00:04:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f0196930 con 0x7f05f0106cd0 2024-09-17T00:04:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05e67fc700 1 -- 172.21.15.42:0/1897786845 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e0017630 con 0x7f05f0106cd0 2024-09-17T00:04:06.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.872+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f0196e20 con 0x7f05f0106cd0 2024-09-17T00:04:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.873+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f01091d0 con 0x7f05f0106cd0 2024-09-17T00:04:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.874+0000 7f05e67fc700 1 -- 172.21.15.42:0/1897786845 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f05e000f5d0 con 0x7f05f0106cd0 2024-09-17T00:04:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.874+0000 7f05e67fc700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f05dc06e680 0x7f05dc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:06.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.874+0000 7f05e67fc700 1 -- 172.21.15.42:0/1897786845 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f05e008faf0 con 0x7f05f0106cd0 2024-09-17T00:04:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.874+0000 7f05f514a700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f05dc06e680 0x7f05dc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:06.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.875+0000 7f05f514a700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f05dc06e680 0x7f05dc070b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00ac50 tx=0x7f05ec00a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:06.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:06.879+0000 7f05e67fc700 1 -- 172.21.15.42:0/1897786845 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f05e005c840 con 0x7f05f0106cd0 2024-09-17T00:04:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.109+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f05f0197090 con 0x7f05dc06e680 2024-09-17T00:04:07.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.110+0000 7f05e67fc700 1 -- 172.21.15.42:0/1897786845 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f05f0197090 con 0x7f05dc06e680 2024-09-17T00:04:07.108 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:04:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:04:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:04:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:04:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:04:07.108 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:04:07.109 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f05dc06e680 msgr2=0x7f05dc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f05dc06e680 0x7f05dc070b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00ac50 tx=0x7f05ec00a380 comp rx=0 tx=0).stop 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0106cd0 msgr2=0x7f05f019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0106cd0 0x7f05f019cb70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f05e0006010 tx=0x7f05e000bd70 comp rx=0 tx=0).stop 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 shutdown_connections 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f05dc06e680 0x7f05dc070b40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05f0068580 0x7f05f019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 --2- 172.21.15.42:0/1897786845 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f0106cd0 0x7f05f019cb70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 >> 172.21.15.42:0/1897786845 conn(0x7f05f0075670 msgr2=0x7f05f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 shutdown_connections 2024-09-17T00:04:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.113+0000 7f05f73ae700 1 -- 172.21.15.42:0/1897786845 wait complete. 2024-09-17T00:04:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 -- 172.21.15.42:0/3919685869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c0ffc20 msgr2=0x7f1b5c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 --2- 172.21.15.42:0/3919685869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c0ffc20 0x7f1b5c100080 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009910 tx=0x7f1b50009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 -- 172.21.15.42:0/3919685869 shutdown_connections 2024-09-17T00:04:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 --2- 172.21.15.42:0/3919685869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c0ffc20 0x7f1b5c100080 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 --2- 172.21.15.42:0/3919685869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b5c105bd0 0x7f1b5c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 -- 172.21.15.42:0/3919685869 >> 172.21.15.42:0/3919685869 conn(0x7f1b5c075670 msgr2=0x7f1b5c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 -- 172.21.15.42:0/3919685869 shutdown_connections 2024-09-17T00:04:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.221+0000 7f1b61edc700 1 -- 172.21.15.42:0/3919685869 wait complete. 2024-09-17T00:04:07.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b61edc700 1 Processor -- start 2024-09-17T00:04:07.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b61edc700 1 -- start start 2024-09-17T00:04:07.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b61edc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b5c0ffc20 0x7f1b5c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b61edc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c105bd0 0x7f1b5c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b61edc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c198380 con 0x7f1b5c105bd0 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b61edc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c1984f0 con 0x7f1b5c0ffc20 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b5affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c105bd0 0x7f1b5c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b5affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c105bd0 0x7f1b5c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36912/0 (socket says 172.21.15.42:36912) 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b5affd700 1 -- 172.21.15.42:0/1941800301 learned_addr learned my addr 172.21.15.42:0/1941800301 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.222+0000 7f1b5b7fe700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b5c0ffc20 0x7f1b5c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b5affd700 1 -- 172.21.15.42:0/1941800301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b5c0ffc20 msgr2=0x7f1b5c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b5affd700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b5c0ffc20 0x7f1b5c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b5affd700 1 -- 172.21.15.42:0/1941800301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b500095f0 con 0x7f1b5c105bd0 2024-09-17T00:04:07.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b5affd700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c105bd0 0x7f1b5c197ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000b5c0 tx=0x7f1b5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:07.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b58ff9700 1 -- 172.21.15.42:0/1941800301 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b5001d070 con 0x7f1b5c105bd0 2024-09-17T00:04:07.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b58ff9700 1 -- 172.21.15.42:0/1941800301 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b50005e00 con 0x7f1b5c105bd0 2024-09-17T00:04:07.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c1a07d0 con 0x7f1b5c105bd0 2024-09-17T00:04:07.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b58ff9700 1 -- 172.21.15.42:0/1941800301 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b50017400 con 0x7f1b5c105bd0 2024-09-17T00:04:07.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.223+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c1a0c40 con 0x7f1b5c105bd0 2024-09-17T00:04:07.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.224+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c1080d0 con 0x7f1b5c105bd0 2024-09-17T00:04:07.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.224+0000 7f1b58ff9700 1 -- 172.21.15.42:0/1941800301 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1b5000f6f0 con 0x7f1b5c105bd0 2024-09-17T00:04:07.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.225+0000 7f1b58ff9700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1b4406e680 0x7f1b44070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:07.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.225+0000 7f1b58ff9700 1 -- 172.21.15.42:0/1941800301 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1b5008fb20 con 0x7f1b5c105bd0 2024-09-17T00:04:07.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.225+0000 7f1b5b7fe700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1b4406e680 0x7f1b44070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:07.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.225+0000 7f1b5b7fe700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1b4406e680 0x7f1b44070b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005950 tx=0x7f1b4c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:07.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.229+0000 7f1b58ff9700 1 -- 172.21.15.42:0/1941800301 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1b50058cc0 con 0x7f1b5c105bd0 2024-09-17T00:04:07.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:07 smithi042 ceph-mon[111056]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:07.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:07 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2949874525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:04:07.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:07 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1579218770' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:04:07.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.531+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1b5c04f2e0 con 0x7f1b5c105bd0 2024-09-17T00:04:07.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.531+0000 7f1b58ff9700 1 -- 172.21.15.42:0/1941800301 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f1b5005c380 con 0x7f1b5c105bd0 2024-09-17T00:04:07.528 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:04:07.529 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:04:07.529 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:04:07.529 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:04:07.529 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:04:07.529 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:04:07.529 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:04:07.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1b4406e680 msgr2=0x7f1b44070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:07.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1b4406e680 0x7f1b44070b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005950 tx=0x7f1b4c00b410 comp rx=0 tx=0).stop 2024-09-17T00:04:07.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c105bd0 msgr2=0x7f1b5c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c105bd0 0x7f1b5c197ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000b5c0 tx=0x7f1b5000bd70 comp rx=0 tx=0).stop 2024-09-17T00:04:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 shutdown_connections 2024-09-17T00:04:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1b4406e680 0x7f1b44070b40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b5c0ffc20 0x7f1b5c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 --2- 172.21.15.42:0/1941800301 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b5c105bd0 0x7f1b5c197ca0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:07.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 >> 172.21.15.42:0/1941800301 conn(0x7f1b5c075670 msgr2=0x7f1b5c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:07.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 shutdown_connections 2024-09-17T00:04:07.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:07.533+0000 7f1b61edc700 1 -- 172.21.15.42:0/1941800301 wait complete. 2024-09-17T00:04:07.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:07 smithi043 ceph-mon[81510]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:07.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:07 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2949874525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:04:07.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:07 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1579218770' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:04:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:08 smithi043 ceph-mon[81510]: pgmap v376: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:08 smithi043 ceph-mon[81510]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:08.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:08 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1941800301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:04:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:08 smithi042 ceph-mon[111056]: pgmap v376: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:08 smithi042 ceph-mon[111056]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:08 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1941800301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:04:10.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:10 smithi043 ceph-mon[81510]: pgmap v377: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:10.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:10 smithi042 ceph-mon[111056]: pgmap v377: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:12 smithi043 ceph-mon[81510]: pgmap v378: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:12.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:12 smithi042 ceph-mon[111056]: pgmap v378: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:14 smithi043 ceph-mon[81510]: pgmap v379: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:14.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:14 smithi042 ceph-mon[111056]: pgmap v379: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:16.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:16 smithi043 ceph-mon[81510]: pgmap v380: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:16.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:16 smithi042 ceph-mon[111056]: pgmap v380: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:18 smithi043 ceph-mon[81510]: pgmap v381: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:18.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:18 smithi042 ceph-mon[111056]: pgmap v381: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:20.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:20 smithi043 ceph-mon[81510]: pgmap v382: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:20.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:20 smithi042 ceph-mon[111056]: pgmap v382: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:21.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:22.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:22 smithi043 ceph-mon[81510]: pgmap v383: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:22.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:22 smithi042 ceph-mon[111056]: pgmap v383: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:24.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:24 smithi042 ceph-mon[111056]: pgmap v384: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:24.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:24 smithi043 ceph-mon[81510]: pgmap v384: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:26.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:26 smithi043 ceph-mon[81510]: pgmap v385: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:26.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:26 smithi042 ceph-mon[111056]: pgmap v385: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:28.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:28 smithi043 ceph-mon[81510]: pgmap v386: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:28 smithi042 ceph-mon[111056]: pgmap v386: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:30.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:30 smithi043 ceph-mon[81510]: pgmap v387: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:30.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:30 smithi042 ceph-mon[111056]: pgmap v387: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:32 smithi043 ceph-mon[81510]: pgmap v388: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:32 smithi042 ceph-mon[111056]: pgmap v388: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:34 smithi043 ceph-mon[81510]: pgmap v389: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:34.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:34 smithi042 ceph-mon[111056]: pgmap v389: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:36 smithi043 ceph-mon[81510]: pgmap v390: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:36 smithi042 ceph-mon[111056]: pgmap v390: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 -- 172.21.15.42:0/289000315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0fffb0 msgr2=0x7fed3c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 --2- 172.21.15.42:0/289000315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0fffb0 0x7fed3c1058e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fed30009910 tx=0x7fed30009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 -- 172.21.15.42:0/289000315 shutdown_connections 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 --2- 172.21.15.42:0/289000315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0fffb0 0x7fed3c1058e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 --2- 172.21.15.42:0/289000315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed3c0ff600 0x7fed3c0ff9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 -- 172.21.15.42:0/289000315 >> 172.21.15.42:0/289000315 conn(0x7fed3c074df0 msgr2=0x7fed3c075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 -- 172.21.15.42:0/289000315 shutdown_connections 2024-09-17T00:04:37.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.657+0000 7fed416e5700 1 -- 172.21.15.42:0/289000315 wait complete. 2024-09-17T00:04:37.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed416e5700 1 Processor -- start 2024-09-17T00:04:37.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed416e5700 1 -- start start 2024-09-17T00:04:37.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed416e5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0ff600 0x7fed3c19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:37.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed416e5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed3c0fffb0 0x7fed3c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:37.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed416e5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c19d290 con 0x7fed3c0ff600 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed416e5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c196740 con 0x7fed3c0fffb0 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0ff600 0x7fed3c19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0ff600 0x7fed3c19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58744/0 (socket says 172.21.15.42:58744) 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.658+0000 7fed3affd700 1 -- 172.21.15.42:0/949806536 learned_addr learned my addr 172.21.15.42:0/949806536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed3a7fc700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed3c0fffb0 0x7fed3c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed3affd700 1 -- 172.21.15.42:0/949806536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed3c0fffb0 msgr2=0x7fed3c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed3affd700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed3c0fffb0 0x7fed3c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed3affd700 1 -- 172.21.15.42:0/949806536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300095f0 con 0x7fed3c0ff600 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed3affd700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0ff600 0x7fed3c19c6c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00e920 tx=0x7fed2c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed23fff700 1 -- 172.21.15.42:0/949806536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2c009790 con 0x7fed3c0ff600 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed23fff700 1 -- 172.21.15.42:0/949806536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed2c00cba0 con 0x7fed3c0ff600 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c196a20 con 0x7fed3c0ff600 2024-09-17T00:04:37.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed23fff700 1 -- 172.21.15.42:0/949806536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2c018650 con 0x7fed3c0ff600 2024-09-17T00:04:37.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.659+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c196f70 con 0x7fed3c0ff600 2024-09-17T00:04:37.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.660+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3c1091a0 con 0x7fed3c0ff600 2024-09-17T00:04:37.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.661+0000 7fed23fff700 1 -- 172.21.15.42:0/949806536 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fed2c00cd10 con 0x7fed3c0ff600 2024-09-17T00:04:37.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.661+0000 7fed23fff700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed2406e680 0x7fed24070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:37.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.662+0000 7fed23fff700 1 -- 172.21.15.42:0/949806536 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fed2c014070 con 0x7fed3c0ff600 2024-09-17T00:04:37.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.662+0000 7fed3a7fc700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed2406e680 0x7fed24070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:37.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.662+0000 7fed3a7fc700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed2406e680 0x7fed24070b40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed30009fd0 tx=0x7fed30005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:37.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.666+0000 7fed23fff700 1 -- 172.21.15.42:0/949806536 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fed2c0541b0 con 0x7fed3c0ff600 2024-09-17T00:04:37.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.897+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed3c1971e0 con 0x7fed2406e680 2024-09-17T00:04:37.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.898+0000 7fed23fff700 1 -- 172.21.15.42:0/949806536 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fed3c1971e0 con 0x7fed2406e680 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed2406e680 msgr2=0x7fed24070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed2406e680 0x7fed24070b40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed30009fd0 tx=0x7fed30005940 comp rx=0 tx=0).stop 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0ff600 msgr2=0x7fed3c19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0ff600 0x7fed3c19c6c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00e920 tx=0x7fed2c00ece0 comp rx=0 tx=0).stop 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 shutdown_connections 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed2406e680 0x7fed24070b40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed3c0ff600 0x7fed3c19c6c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 --2- 172.21.15.42:0/949806536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed3c0fffb0 0x7fed3c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 >> 172.21.15.42:0/949806536 conn(0x7fed3c074df0 msgr2=0x7fed3c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 shutdown_connections 2024-09-17T00:04:37.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:37.900+0000 7fed416e5700 1 -- 172.21.15.42:0/949806536 wait complete. 2024-09-17T00:04:37.909 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:04:38.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 -- 172.21.15.42:0/1080238128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 msgr2=0x7f919c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 --2- 172.21.15.42:0/1080238128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c101270 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f918c009910 tx=0x7f918c009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 -- 172.21.15.42:0/1080238128 shutdown_connections 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 --2- 172.21.15.42:0/1080238128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f919c101840 0x7f919c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 --2- 172.21.15.42:0/1080238128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c101270 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 -- 172.21.15.42:0/1080238128 >> 172.21.15.42:0/1080238128 conn(0x7f919c0fc740 msgr2=0x7f919c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 -- 172.21.15.42:0/1080238128 shutdown_connections 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.009+0000 7f91a1393700 1 -- 172.21.15.42:0/1080238128 wait complete. 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f91a1393700 1 Processor -- start 2024-09-17T00:04:38.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f91a1393700 1 -- start start 2024-09-17T00:04:38.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f91a1393700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f91a1393700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f919c101840 0x7f919c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f91a1393700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c073b10 con 0x7f919c100e90 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f91a1393700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c073c50 con 0x7f919c101840 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f919affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f919affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58768/0 (socket says 172.21.15.42:58768) 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f919affd700 1 -- 172.21.15.42:0/951638061 learned_addr learned my addr 172.21.15.42:0/951638061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.010+0000 7f919a7fc700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f919c101840 0x7f919c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f919affd700 1 -- 172.21.15.42:0/951638061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f919c101840 msgr2=0x7f919c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f919affd700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f919c101840 0x7f919c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f919affd700 1 -- 172.21.15.42:0/951638061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918c0095f0 con 0x7f919c100e90 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f919affd700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c073090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f918c006010 tx=0x7f918c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:38.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f9183fff700 1 -- 172.21.15.42:0/951638061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f918c01d070 con 0x7f919c100e90 2024-09-17T00:04:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f9183fff700 1 -- 172.21.15.42:0/951638061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f918c005dc0 con 0x7f919c100e90 2024-09-17T00:04:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c073ed0 con 0x7f919c100e90 2024-09-17T00:04:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f9183fff700 1 -- 172.21.15.42:0/951638061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f918c017400 con 0x7f919c100e90 2024-09-17T00:04:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.011+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c1a2db0 con 0x7f919c100e90 2024-09-17T00:04:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.012+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c109230 con 0x7f919c100e90 2024-09-17T00:04:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.013+0000 7f9183fff700 1 -- 172.21.15.42:0/951638061 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f918c003a60 con 0x7f919c100e90 2024-09-17T00:04:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.013+0000 7f9183fff700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f918406e680 0x7f9184070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.013+0000 7f9183fff700 1 -- 172.21.15.42:0/951638061 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f918c08fa00 con 0x7f919c100e90 2024-09-17T00:04:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.013+0000 7f919a7fc700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f918406e680 0x7f9184070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.014+0000 7f919a7fc700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f918406e680 0x7f9184070b40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f919c074880 tx=0x7f919000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.018+0000 7f9183fff700 1 -- 172.21.15.42:0/951638061 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f918c058c20 con 0x7f919c100e90 2024-09-17T00:04:38.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.246+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f919c0611d0 con 0x7f918406e680 2024-09-17T00:04:38.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.247+0000 7f9183fff700 1 -- 172.21.15.42:0/951638061 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f919c0611d0 con 0x7f918406e680 2024-09-17T00:04:38.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f918406e680 msgr2=0x7f9184070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f918406e680 0x7f9184070b40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f919c074880 tx=0x7f919000b410 comp rx=0 tx=0).stop 2024-09-17T00:04:38.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 msgr2=0x7f919c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c073090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f918c006010 tx=0x7f918c00bd70 comp rx=0 tx=0).stop 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 shutdown_connections 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f918406e680 0x7f9184070b40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f919c100e90 0x7f919c073090 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 --2- 172.21.15.42:0/951638061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f919c101840 0x7f919c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 >> 172.21.15.42:0/951638061 conn(0x7f919c0fc740 msgr2=0x7f919c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 shutdown_connections 2024-09-17T00:04:38.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.250+0000 7f91a1393700 1 -- 172.21.15.42:0/951638061 wait complete. 2024-09-17T00:04:38.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 -- 172.21.15.42:0/1367688421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 msgr2=0x7f6ebc0ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/1367688421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc0ff9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009910 tx=0x7f6eac009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:38.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 -- 172.21.15.42:0/1367688421 shutdown_connections 2024-09-17T00:04:38.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/1367688421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc0fffb0 0x7f6ebc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/1367688421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc0ff9e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 -- 172.21.15.42:0/1367688421 >> 172.21.15.42:0/1367688421 conn(0x7f6ebc074df0 msgr2=0x7f6ebc075200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 -- 172.21.15.42:0/1367688421 shutdown_connections 2024-09-17T00:04:38.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.358+0000 7f6ec0e19700 1 -- 172.21.15.42:0/1367688421 wait complete. 2024-09-17T00:04:38.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.359+0000 7f6ec0e19700 1 Processor -- start 2024-09-17T00:04:38.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.359+0000 7f6ec0e19700 1 -- start start 2024-09-17T00:04:38.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.359+0000 7f6ec0e19700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.359+0000 7f6ec0e19700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc0fffb0 0x7f6ebc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.359+0000 7f6ec0e19700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc19f410 con 0x7f6ebc0ff600 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.359+0000 7f6ec0e19700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1988c0 con 0x7f6ebc0fffb0 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58790/0 (socket says 172.21.15.42:58790) 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eba59c700 1 -- 172.21.15.42:0/3503077059 learned_addr learned my addr 172.21.15.42:0/3503077059 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eb1d9b700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc0fffb0 0x7f6ebc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eba59c700 1 -- 172.21.15.42:0/3503077059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc0fffb0 msgr2=0x7f6ebc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eba59c700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc0fffb0 0x7f6ebc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eba59c700 1 -- 172.21.15.42:0/3503077059 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac0095f0 con 0x7f6ebc0ff600 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eb1d9b700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc0fffb0 0x7f6ebc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eba59c700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc19e840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0092e0 tx=0x7f6eac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eb3fff700 1 -- 172.21.15.42:0/3503077059 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eac01d070 con 0x7f6ebc0ff600 2024-09-17T00:04:38.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc198b40 con 0x7f6ebc0ff600 2024-09-17T00:04:38.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eb3fff700 1 -- 172.21.15.42:0/3503077059 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6eac005e00 con 0x7f6ebc0ff600 2024-09-17T00:04:38.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6eb3fff700 1 -- 172.21.15.42:0/3503077059 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6eac017400 con 0x7f6ebc0ff600 2024-09-17T00:04:38.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.360+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc199030 con 0x7f6ebc0ff600 2024-09-17T00:04:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.361+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc105060 con 0x7f6ebc0ff600 2024-09-17T00:04:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.362+0000 7f6eb3fff700 1 -- 172.21.15.42:0/3503077059 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6eac00f460 con 0x7f6ebc0ff600 2024-09-17T00:04:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.362+0000 7f6eb3fff700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ea806e680 0x7f6ea8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.363+0000 7f6eb3fff700 1 -- 172.21.15.42:0/3503077059 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6eac08fbb0 con 0x7f6ebc0ff600 2024-09-17T00:04:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.363+0000 7f6eb1d9b700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ea806e680 0x7f6ea8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.363+0000 7f6eb1d9b700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ea806e680 0x7f6ea8070b40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005fd0 tx=0x7f6ea4005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:38.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.367+0000 7f6eb3fff700 1 -- 172.21.15.42:0/3503077059 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6eac096080 con 0x7f6ebc0ff600 2024-09-17T00:04:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:38 smithi042 ceph-mon[111056]: pgmap v391: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:38.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.585+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ebc199250 con 0x7f6ea806e680 2024-09-17T00:04:38.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.595+0000 7f6eb3fff700 1 -- 172.21.15.42:0/3503077059 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f6ebc199250 con 0x7f6ea806e680 2024-09-17T00:04:38.592 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:04:38.592 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (13m) 6m ago 24m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:04:38.592 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (24m) 6m ago 24m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (23m) 6m ago 23m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (10m) 6m ago 24m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (10m) 6m ago 23m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (12m) 6m ago 23m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (7m) 6m ago 18m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (7m) 6m ago 18m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (6m) 6m ago 18m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (6m) 6m ago 18m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (15m) 6m ago 25m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (14m) 6m ago 23m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (11m) 6m ago 25m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (11m) 6m ago 22m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:04:38.593 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (13m) 6m ago 24m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (13m) 6m ago 23m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (10m) 6m ago 22m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (9m) 6m ago 21m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (9m) 6m ago 21m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (8m) 6m ago 20m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (8m) 6m ago 20m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (7m) 6m ago 19m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:04:38.594 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (13m) 6m ago 23m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ea806e680 msgr2=0x7f6ea8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ea806e680 0x7f6ea8070b40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005fd0 tx=0x7f6ea4005ee0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 msgr2=0x7f6ebc19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc19e840 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0092e0 tx=0x7f6eac00be20 comp rx=0 tx=0).stop 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 shutdown_connections 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6ea806e680 0x7f6ea8070b40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ebc0ff600 0x7f6ebc19e840 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 --2- 172.21.15.42:0/3503077059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ebc0fffb0 0x7f6ebc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 >> 172.21.15.42:0/3503077059 conn(0x7f6ebc074df0 msgr2=0x7f6ebc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 shutdown_connections 2024-09-17T00:04:38.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.597+0000 7f6ec0e19700 1 -- 172.21.15.42:0/3503077059 wait complete. 2024-09-17T00:04:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:38 smithi043 ceph-mon[81510]: pgmap v391: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 -- 172.21.15.42:0/2565054283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce90105950 msgr2=0x7fce90105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 --2- 172.21.15.42:0/2565054283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce90105950 0x7fce90105d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fce80009910 tx=0x7fce80009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 -- 172.21.15.42:0/2565054283 shutdown_connections 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 --2- 172.21.15.42:0/2565054283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce900ffa20 0x7fce900ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 --2- 172.21.15.42:0/2565054283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce90105950 0x7fce90105d30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 -- 172.21.15.42:0/2565054283 >> 172.21.15.42:0/2565054283 conn(0x7fce90075460 msgr2=0x7fce90075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 -- 172.21.15.42:0/2565054283 shutdown_connections 2024-09-17T00:04:38.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.704+0000 7fce94a02700 1 -- 172.21.15.42:0/2565054283 wait complete. 2024-09-17T00:04:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.705+0000 7fce94a02700 1 Processor -- start 2024-09-17T00:04:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.705+0000 7fce94a02700 1 -- start start 2024-09-17T00:04:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.705+0000 7fce94a02700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce900ffa20 0x7fce90197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.705+0000 7fce94a02700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce90105950 0x7fce901979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.705+0000 7fce94a02700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce90197f60 con 0x7fce900ffa20 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.705+0000 7fce94a02700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce901980a0 con 0x7fce90105950 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce900ffa20 0x7fce90197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce900ffa20 0x7fce90197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58822/0 (socket says 172.21.15.42:58822) 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8ed9d700 1 -- 172.21.15.42:0/3526826146 learned_addr learned my addr 172.21.15.42:0/3526826146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8e59c700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce90105950 0x7fce901979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8ed9d700 1 -- 172.21.15.42:0/3526826146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce90105950 msgr2=0x7fce901979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8ed9d700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce90105950 0x7fce901979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8ed9d700 1 -- 172.21.15.42:0/3526826146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800095f0 con 0x7fce900ffa20 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8e59c700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce90105950 0x7fce901979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce8ed9d700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce900ffa20 0x7fce90197490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fce8000b5c0 tx=0x7fce8000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce77fff700 1 -- 172.21.15.42:0/3526826146 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce8001d070 con 0x7fce900ffa20 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce77fff700 1 -- 172.21.15.42:0/3526826146 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce8000f460 con 0x7fce900ffa20 2024-09-17T00:04:38.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce901a0500 con 0x7fce900ffa20 2024-09-17T00:04:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce77fff700 1 -- 172.21.15.42:0/3526826146 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce80017610 con 0x7fce900ffa20 2024-09-17T00:04:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.706+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce901a08b0 con 0x7fce900ffa20 2024-09-17T00:04:38.706 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.707+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce90107e90 con 0x7fce900ffa20 2024-09-17T00:04:38.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.708+0000 7fce77fff700 1 -- 172.21.15.42:0/3526826146 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fce8000f5d0 con 0x7fce900ffa20 2024-09-17T00:04:38.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.708+0000 7fce77fff700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7806e680 0x7fce78070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:38.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.709+0000 7fce77fff700 1 -- 172.21.15.42:0/3526826146 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce8008fbb0 con 0x7fce900ffa20 2024-09-17T00:04:38.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.709+0000 7fce8e59c700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7806e680 0x7fce78070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:38.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.709+0000 7fce8e59c700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7806e680 0x7fce78070b40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce84005fd0 tx=0x7fce84005e40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:38.710 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:38.713+0000 7fce77fff700 1 -- 172.21.15.42:0/3526826146 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fce80096080 con 0x7fce900ffa20 2024-09-17T00:04:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.008+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce900623c0 con 0x7fce900ffa20 2024-09-17T00:04:39.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.009+0000 7fce77fff700 1 -- 172.21.15.42:0/3526826146 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fce80058dd0 con 0x7fce900ffa20 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:39.007 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:04:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:04:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:04:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:04:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:04:39.008 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:04:39.008 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:04:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7806e680 msgr2=0x7fce78070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7806e680 0x7fce78070b40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce84005fd0 tx=0x7fce84005e40 comp rx=0 tx=0).stop 2024-09-17T00:04:39.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce900ffa20 msgr2=0x7fce90197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce900ffa20 0x7fce90197490 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fce8000b5c0 tx=0x7fce8000be00 comp rx=0 tx=0).stop 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 shutdown_connections 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7806e680 0x7fce78070b40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce900ffa20 0x7fce90197490 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 --2- 172.21.15.42:0/3526826146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce90105950 0x7fce901979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.012+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 >> 172.21.15.42:0/3526826146 conn(0x7fce90075460 msgr2=0x7fce900fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.013+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 shutdown_connections 2024-09-17T00:04:39.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.013+0000 7fce94a02700 1 -- 172.21.15.42:0/3526826146 wait complete. 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/532360656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78100ea0 msgr2=0x7f8d78101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/532360656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78100ea0 0x7f8d78101280 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009910 tx=0x7f8d68009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/532360656 shutdown_connections 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/532360656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d78101850 0x7f8d781058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/532360656 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78100ea0 0x7f8d78101280 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/532360656 >> 172.21.15.42:0/532360656 conn(0x7f8d780fc750 msgr2=0x7f8d780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/532360656 shutdown_connections 2024-09-17T00:04:39.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.122+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/532360656 wait complete. 2024-09-17T00:04:39.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.123+0000 7f8d7e4dd700 1 Processor -- start 2024-09-17T00:04:39.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.123+0000 7f8d7e4dd700 1 -- start start 2024-09-17T00:04:39.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.123+0000 7f8d7e4dd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d78100ea0 0x7f8d78073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.123+0000 7f8d7e4dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78101850 0x7f8d780735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.123+0000 7f8d7e4dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78073b10 con 0x7f8d78101850 2024-09-17T00:04:39.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.123+0000 7f8d7e4dd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d78073c50 con 0x7f8d78100ea0 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.123+0000 7f8d777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78101850 0x7f8d780735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78101850 0x7f8d780735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58842/0 (socket says 172.21.15.42:58842) 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d777fe700 1 -- 172.21.15.42:0/3054505018 learned_addr learned my addr 172.21.15.42:0/3054505018 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d77fff700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d78100ea0 0x7f8d78073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d777fe700 1 -- 172.21.15.42:0/3054505018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d78100ea0 msgr2=0x7f8d78073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d777fe700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d78100ea0 0x7f8d78073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d777fe700 1 -- 172.21.15.42:0/3054505018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680095f0 con 0x7f8d78101850 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d777fe700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78101850 0x7f8d780735d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00d900 tx=0x7f8d6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d757fa700 1 -- 172.21.15.42:0/3054505018 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d6c009900 con 0x7f8d78101850 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d757fa700 1 -- 172.21.15.42:0/3054505018 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d6c010460 con 0x7f8d78101850 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d781a2af0 con 0x7f8d78101850 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d757fa700 1 -- 172.21.15.42:0/3054505018 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d6c00f5f0 con 0x7f8d78101850 2024-09-17T00:04:39.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.124+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d781a2f00 con 0x7f8d78101850 2024-09-17T00:04:39.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.125+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d78109240 con 0x7f8d78101850 2024-09-17T00:04:39.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.126+0000 7f8d757fa700 1 -- 172.21.15.42:0/3054505018 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8d6c0105d0 con 0x7f8d78101850 2024-09-17T00:04:39.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.126+0000 7f8d757fa700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6006e680 0x7f8d60070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.126+0000 7f8d757fa700 1 -- 172.21.15.42:0/3054505018 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d6c08e440 con 0x7f8d78101850 2024-09-17T00:04:39.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.126+0000 7f8d77fff700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6006e680 0x7f8d60070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.127+0000 7f8d77fff700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6006e680 0x7f8d60070b40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800b5c0 tx=0x7f8d6801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:39.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.131+0000 7f8d757fa700 1 -- 172.21.15.42:0/3054505018 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8d6c057660 con 0x7f8d78101850 2024-09-17T00:04:39.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.389+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d7804f2e0 con 0x7f8d78101850 2024-09-17T00:04:39.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.389+0000 7f8d757fa700 1 -- 172.21.15.42:0/3054505018 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f8d6c05ad20 con 0x7f8d78101850 2024-09-17T00:04:39.387 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:04:39.387 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:04:39.387 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:04:39.387 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:04:39.387 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:04:39.388 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:04:39.389 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:39.390 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6006e680 msgr2=0x7f8d60070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6006e680 0x7f8d60070b40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800b5c0 tx=0x7f8d6801a040 comp rx=0 tx=0).stop 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78101850 msgr2=0x7f8d780735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78101850 0x7f8d780735d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00d900 tx=0x7f8d6c00dcc0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 shutdown_connections 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d6006e680 0x7f8d60070b40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d78100ea0 0x7f8d78073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 --2- 172.21.15.42:0/3054505018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d78101850 0x7f8d780735d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 >> 172.21.15.42:0/3054505018 conn(0x7f8d780fc750 msgr2=0x7f8d780fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:39.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 shutdown_connections 2024-09-17T00:04:39.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.391+0000 7f8d7e4dd700 1 -- 172.21.15.42:0/3054505018 wait complete. 2024-09-17T00:04:39.392 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:04:39.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 -- 172.21.15.42:0/3428822117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 msgr2=0x7f77741021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 --2- 172.21.15.42:0/3428822117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77741021e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7770009910 tx=0x7f7770009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:39.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 -- 172.21.15.42:0/3428822117 shutdown_connections 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 --2- 172.21.15.42:0/3428822117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7774102720 0x7f777410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 --2- 172.21.15.42:0/3428822117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77741021e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 -- 172.21.15.42:0/3428822117 >> 172.21.15.42:0/3428822117 conn(0x7f77740fb650 msgr2=0x7f77740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 -- 172.21.15.42:0/3428822117 shutdown_connections 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.500+0000 7f777c6e3700 1 -- 172.21.15.42:0/3428822117 wait complete. 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777c6e3700 1 Processor -- start 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777c6e3700 1 -- start start 2024-09-17T00:04:39.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777c6e3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77740ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777c6e3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7774102720 0x7f77740fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777c6e3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77741004e0 con 0x7f7774101e00 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777c6e3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7774100620 con 0x7f7774102720 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777a47f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77740ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777a47f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77740ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58858/0 (socket says 172.21.15.42:58858) 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777a47f700 1 -- 172.21.15.42:0/844163680 learned_addr learned my addr 172.21.15.42:0/844163680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f7779c7e700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7774102720 0x7f77740fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777a47f700 1 -- 172.21.15.42:0/844163680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7774102720 msgr2=0x7f77740fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777a47f700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7774102720 0x7f77740fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f777a47f700 1 -- 172.21.15.42:0/844163680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700095f0 con 0x7f7774101e00 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.501+0000 7f7779c7e700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7774102720 0x7f77740fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.502+0000 7f777a47f700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77740ffa60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7770000c00 tx=0x7f777000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:39.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.502+0000 7f776b7fe700 1 -- 172.21.15.42:0/844163680 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777001d070 con 0x7f7774101e00 2024-09-17T00:04:39.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.502+0000 7f776b7fe700 1 -- 172.21.15.42:0/844163680 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7770022470 con 0x7f7774101e00 2024-09-17T00:04:39.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.502+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741008a0 con 0x7f7774101e00 2024-09-17T00:04:39.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.502+0000 7f776b7fe700 1 -- 172.21.15.42:0/844163680 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777000f690 con 0x7f7774101e00 2024-09-17T00:04:39.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.502+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77741a2e90 con 0x7f7774101e00 2024-09-17T00:04:39.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.503+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7774108310 con 0x7f7774101e00 2024-09-17T00:04:39.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.503+0000 7f776b7fe700 1 -- 172.21.15.42:0/844163680 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f77700225e0 con 0x7f7774101e00 2024-09-17T00:04:39.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.504+0000 7f776b7fe700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f776006e680 0x7f7760070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.504+0000 7f776b7fe700 1 -- 172.21.15.42:0/844163680 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f777008fb00 con 0x7f7774101e00 2024-09-17T00:04:39.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.504+0000 7f7779c7e700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f776006e680 0x7f7760070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.504+0000 7f7779c7e700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f776006e680 0x7f7760070b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7774101200 tx=0x7f7764008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.508+0000 7f776b7fe700 1 -- 172.21.15.42:0/844163680 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7770058dd0 con 0x7f7774101e00 2024-09-17T00:04:39.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:39 smithi043 ceph-mon[81510]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:39.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:39 smithi043 ceph-mon[81510]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:39.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:39 smithi043 ceph-mon[81510]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:39.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:39 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3526826146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:04:39.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.736+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77740611d0 con 0x7f776006e680 2024-09-17T00:04:39.735 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:39 smithi042 ceph-mon[111056]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:39.735 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:39 smithi042 ceph-mon[111056]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:39.735 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:39 smithi042 ceph-mon[111056]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:39.735 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:39 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3526826146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:04:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.738+0000 7f776b7fe700 1 -- 172.21.15.42:0/844163680 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f77740611d0 con 0x7f776006e680 2024-09-17T00:04:39.735 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:04:39.735 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:04:39.736 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f776006e680 msgr2=0x7f7760070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f776006e680 0x7f7760070b40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7774101200 tx=0x7f7764008040 comp rx=0 tx=0).stop 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 msgr2=0x7f77740ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77740ffa60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7770000c00 tx=0x7f777000be20 comp rx=0 tx=0).stop 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 shutdown_connections 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f776006e680 0x7f7760070b40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7774101e00 0x7f77740ffa60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 --2- 172.21.15.42:0/844163680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7774102720 0x7f77740fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 >> 172.21.15.42:0/844163680 conn(0x7f77740fb650 msgr2=0x7f7774105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 shutdown_connections 2024-09-17T00:04:39.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.740+0000 7f777c6e3700 1 -- 172.21.15.42:0/844163680 wait complete. 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1344363783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 msgr2=0x7f73d0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1344363783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d0100180 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009910 tx=0x7f73c0009c20 comp rx=0 tx=0).stop 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1344363783 shutdown_connections 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1344363783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73d0100750 0x7f73d01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1344363783 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d0100180 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1344363783 >> 172.21.15.42:0/1344363783 conn(0x7f73d00fb650 msgr2=0x7f73d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1344363783 shutdown_connections 2024-09-17T00:04:39.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.848+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1344363783 wait complete. 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73d4c7a700 1 Processor -- start 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73d4c7a700 1 -- start start 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73d4c7a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d01954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73d4c7a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73d0100750 0x7f73d0195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73d4c7a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0195f60 con 0x7f73d00ffda0 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73d4c7a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d01960d0 con 0x7f73d0100750 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73ce59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d01954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73ce59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d01954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58878/0 (socket says 172.21.15.42:58878) 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73ce59c700 1 -- 172.21.15.42:0/1910912754 learned_addr learned my addr 172.21.15.42:0/1910912754 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:04:39.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73c7fff700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73d0100750 0x7f73d0195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73ce59c700 1 -- 172.21.15.42:0/1910912754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73d0100750 msgr2=0x7f73d0195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73ce59c700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73d0100750 0x7f73d0195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.849+0000 7f73ce59c700 1 -- 172.21.15.42:0/1910912754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c00095f0 con 0x7f73d00ffda0 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.850+0000 7f73ce59c700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d01954e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73c00092e0 tx=0x7f73c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.850+0000 7f73c77fe700 1 -- 172.21.15.42:0/1910912754 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c001d070 con 0x7f73d00ffda0 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.850+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d0196350 con 0x7f73d00ffda0 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.850+0000 7f73c77fe700 1 -- 172.21.15.42:0/1910912754 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73c0005e00 con 0x7f73d00ffda0 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.850+0000 7f73c77fe700 1 -- 172.21.15.42:0/1910912754 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c0017400 con 0x7f73d00ffda0 2024-09-17T00:04:39.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.850+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d01a2dd0 con 0x7f73d00ffda0 2024-09-17T00:04:39.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.851+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d0108140 con 0x7f73d00ffda0 2024-09-17T00:04:39.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.851+0000 7f73c77fe700 1 -- 172.21.15.42:0/1910912754 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f73c000f6f0 con 0x7f73d00ffda0 2024-09-17T00:04:39.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.852+0000 7f73c77fe700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f73bc06e680 0x7f73bc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:04:39.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.852+0000 7f73c77fe700 1 -- 172.21.15.42:0/1910912754 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f73c008f970 con 0x7f73d00ffda0 2024-09-17T00:04:39.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.852+0000 7f73c7fff700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f73bc06e680 0x7f73bc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:04:39.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.852+0000 7f73c7fff700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f73bc06e680 0x7f73bc070b40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73d0069770 tx=0x7f73b800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:04:39.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:39.856+0000 7f73c77fe700 1 -- 172.21.15.42:0/1910912754 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f73c005c640 con 0x7f73d00ffda0 2024-09-17T00:04:40.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.153+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f73d004f2e0 con 0x7f73d00ffda0 2024-09-17T00:04:40.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.154+0000 7f73c77fe700 1 -- 172.21.15.42:0/1910912754 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f73c005c1d0 con 0x7f73d00ffda0 2024-09-17T00:04:40.152 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:04:40.152 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:04:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:04:40.152 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:04:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:04:40.152 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:04:40.152 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f73bc06e680 msgr2=0x7f73bc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f73bc06e680 0x7f73bc070b40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f73d0069770 tx=0x7f73b800b410 comp rx=0 tx=0).stop 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 msgr2=0x7f73d01954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d01954e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f73c00092e0 tx=0x7f73c000bd70 comp rx=0 tx=0).stop 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 shutdown_connections 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f73bc06e680 0x7f73bc070b40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73d00ffda0 0x7f73d01954e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 --2- 172.21.15.42:0/1910912754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73d0100750 0x7f73d0195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.156+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 >> 172.21.15.42:0/1910912754 conn(0x7f73d00fb650 msgr2=0x7f73d00fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.157+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 shutdown_connections 2024-09-17T00:04:40.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:04:40.157+0000 7f73d4c7a700 1 -- 172.21.15.42:0/1910912754 wait complete. 2024-09-17T00:04:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:40 smithi042 ceph-mon[111056]: pgmap v392: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:40 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3054505018' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:04:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:40 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1910912754' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:04:40.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:40 smithi043 ceph-mon[81510]: pgmap v392: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:40.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:40 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3054505018' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:04:40.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:40 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1910912754' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:04:41.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:41 smithi042 ceph-mon[111056]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:41.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:41 smithi043 ceph-mon[81510]: from='client.34788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:04:42.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:42 smithi042 ceph-mon[111056]: pgmap v393: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:42.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:42 smithi043 ceph-mon[81510]: pgmap v393: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:44 smithi043 ceph-mon[81510]: pgmap v394: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:44 smithi042 ceph-mon[111056]: pgmap v394: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:46 smithi042 ceph-mon[111056]: pgmap v395: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:46.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:46 smithi043 ceph-mon[81510]: pgmap v395: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:48 smithi042 ceph-mon[111056]: pgmap v396: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:48 smithi043 ceph-mon[81510]: pgmap v396: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:50.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:50 smithi042 ceph-mon[111056]: pgmap v397: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:50 smithi043 ceph-mon[81510]: pgmap v397: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:51.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:04:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:52 smithi042 ceph-mon[111056]: pgmap v398: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:52 smithi043 ceph-mon[81510]: pgmap v398: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:54.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:54 smithi042 ceph-mon[111056]: pgmap v399: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:54.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:54 smithi043 ceph-mon[81510]: pgmap v399: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:56.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:56 smithi042 ceph-mon[111056]: pgmap v400: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:56.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:56 smithi043 ceph-mon[81510]: pgmap v400: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:58.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:04:58 smithi042 ceph-mon[111056]: pgmap v401: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:04:58.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:04:58 smithi043 ceph-mon[81510]: pgmap v401: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:00.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:00 smithi042 ceph-mon[111056]: pgmap v402: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:00 smithi043 ceph-mon[81510]: pgmap v402: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:02 smithi042 ceph-mon[111056]: pgmap v403: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:02.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:02 smithi043 ceph-mon[81510]: pgmap v403: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:04 smithi042 ceph-mon[111056]: pgmap v404: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:04.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:04 smithi043 ceph-mon[81510]: pgmap v404: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:06 smithi042 ceph-mon[111056]: pgmap v405: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:06.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:06 smithi043 ceph-mon[81510]: pgmap v405: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:08.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:08 smithi042 ceph-mon[111056]: pgmap v406: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:08.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:08 smithi043 ceph-mon[81510]: pgmap v406: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:10.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 -- 172.21.15.42:0/2343844412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 msgr2=0x7f72e8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/2343844412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e8100180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009910 tx=0x7f72d8009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:10.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 -- 172.21.15.42:0/2343844412 shutdown_connections 2024-09-17T00:05:10.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/2343844412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8100750 0x7f72e81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/2343844412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e8100180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 -- 172.21.15.42:0/2343844412 >> 172.21.15.42:0/2343844412 conn(0x7f72e80fb650 msgr2=0x7f72e80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:10.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 -- 172.21.15.42:0/2343844412 shutdown_connections 2024-09-17T00:05:10.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.280+0000 7f72ecfa3700 1 -- 172.21.15.42:0/2343844412 wait complete. 2024-09-17T00:05:10.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72ecfa3700 1 Processor -- start 2024-09-17T00:05:10.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72ecfa3700 1 -- start start 2024-09-17T00:05:10.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72ecfa3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72ecfa3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8100750 0x7f72e8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72ecfa3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e8198250 con 0x7f72e80ffda0 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72ecfa3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e8198390 con 0x7f72e8100750 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72e659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72e659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e81976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47194/0 (socket says 172.21.15.42:47194) 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.281+0000 7f72e659c700 1 -- 172.21.15.42:0/1129176313 learned_addr learned my addr 172.21.15.42:0/1129176313 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72e5d9b700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8100750 0x7f72e8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72e659c700 1 -- 172.21.15.42:0/1129176313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8100750 msgr2=0x7f72e8197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72e659c700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8100750 0x7f72e8197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72e659c700 1 -- 172.21.15.42:0/1129176313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d80095f0 con 0x7f72e80ffda0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72e659c700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e81976f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f72d8006010 tx=0x7f72d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72d77fe700 1 -- 172.21.15.42:0/1129176313 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d801d070 con 0x7f72e80ffda0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e81a0760 con 0x7f72e80ffda0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72d77fe700 1 -- 172.21.15.42:0/1129176313 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72d800f460 con 0x7f72e80ffda0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72d77fe700 1 -- 172.21.15.42:0/1129176313 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d8017630 con 0x7f72e80ffda0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.282+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e81a0b70 con 0x7f72e80ffda0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.283+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e8108140 con 0x7f72e80ffda0 2024-09-17T00:05:10.284 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.284+0000 7f72d77fe700 1 -- 172.21.15.42:0/1129176313 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f72d800f5d0 con 0x7f72e80ffda0 2024-09-17T00:05:10.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.284+0000 7f72d77fe700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72d006e680 0x7f72d0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.284+0000 7f72d77fe700 1 -- 172.21.15.42:0/1129176313 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f72d808fb20 con 0x7f72e80ffda0 2024-09-17T00:05:10.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.284+0000 7f72e5d9b700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72d006e680 0x7f72d0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.285+0000 7f72e5d9b700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72d006e680 0x7f72d0070b40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005950 tx=0x7f72dc0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:10.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.289+0000 7f72d77fe700 1 -- 172.21.15.42:0/1129176313 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f72d8058cc0 con 0x7f72e80ffda0 2024-09-17T00:05:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.520+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72e80611d0 con 0x7f72d006e680 2024-09-17T00:05:10.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.522+0000 7f72d77fe700 1 -- 172.21.15.42:0/1129176313 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f72e80611d0 con 0x7f72d006e680 2024-09-17T00:05:10.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72d006e680 msgr2=0x7f72d0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72d006e680 0x7f72d0070b40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005950 tx=0x7f72dc0058e0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 msgr2=0x7f72e81976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e81976f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f72d8006010 tx=0x7f72d800be20 comp rx=0 tx=0).stop 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 shutdown_connections 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72d006e680 0x7f72d0070b40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72e80ffda0 0x7f72e81976f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 --2- 172.21.15.42:0/1129176313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72e8100750 0x7f72e8197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 >> 172.21.15.42:0/1129176313 conn(0x7f72e80fb650 msgr2=0x7f72e80fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 shutdown_connections 2024-09-17T00:05:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.524+0000 7f72ecfa3700 1 -- 172.21.15.42:0/1129176313 wait complete. 2024-09-17T00:05:10.532 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:05:10.533 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:10 smithi042 ceph-mon[111056]: pgmap v407: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.633+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3926268099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 msgr2=0x7ffa48101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.633+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3926268099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa48101280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009910 tx=0x7ffa38009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.633+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3926268099 shutdown_connections 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.633+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3926268099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48101850 0x7ffa481058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.633+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3926268099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa48101280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.633+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3926268099 >> 172.21.15.42:0/3926268099 conn(0x7ffa480fc750 msgr2=0x7ffa480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.634+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3926268099 shutdown_connections 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.634+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3926268099 wait complete. 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.634+0000 7ffa4cd71700 1 Processor -- start 2024-09-17T00:05:10.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.634+0000 7ffa4cd71700 1 -- start start 2024-09-17T00:05:10.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4cd71700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa4819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4cd71700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48101850 0x7ffa4819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4cd71700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4819d2c0 con 0x7ffa48100ea0 2024-09-17T00:05:10.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4cd71700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48196770 con 0x7ffa48101850 2024-09-17T00:05:10.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa4819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa4819c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47208/0 (socket says 172.21.15.42:47208) 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4659c700 1 -- 172.21.15.42:0/3607210669 learned_addr learned my addr 172.21.15.42:0/3607210669 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa45d9b700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48101850 0x7ffa4819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4659c700 1 -- 172.21.15.42:0/3607210669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48101850 msgr2=0x7ffa4819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4659c700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48101850 0x7ffa4819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4659c700 1 -- 172.21.15.42:0/3607210669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa380095f0 con 0x7ffa48100ea0 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa45d9b700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48101850 0x7ffa4819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4659c700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa4819c6f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000c00 tx=0x7ffa3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:10.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa377fe700 1 -- 172.21.15.42:0/3607210669 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3801d070 con 0x7ffa48100ea0 2024-09-17T00:05:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa377fe700 1 -- 172.21.15.42:0/3607210669 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa3800f460 con 0x7ffa48100ea0 2024-09-17T00:05:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa481969f0 con 0x7ffa48100ea0 2024-09-17T00:05:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.635+0000 7ffa377fe700 1 -- 172.21.15.42:0/3607210669 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa38017630 con 0x7ffa48100ea0 2024-09-17T00:05:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.636+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa48196ee0 con 0x7ffa48100ea0 2024-09-17T00:05:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.636+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa48109240 con 0x7ffa48100ea0 2024-09-17T00:05:10.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.637+0000 7ffa377fe700 1 -- 172.21.15.42:0/3607210669 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ffa3800f5d0 con 0x7ffa48100ea0 2024-09-17T00:05:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.637+0000 7ffa377fe700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.638+0000 7ffa377fe700 1 -- 172.21.15.42:0/3607210669 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffa3808fc60 con 0x7ffa48100ea0 2024-09-17T00:05:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.638+0000 7ffa45d9b700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.638+0000 7ffa45d9b700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa48197be0 tx=0x7ffa3c005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:10.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.642+0000 7ffa377fe700 1 -- 172.21.15.42:0/3607210669 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ffa3805c930 con 0x7ffa48100ea0 2024-09-17T00:05:10.779 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:10 smithi043 ceph-mon[81510]: pgmap v407: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:10.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.875+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa48197100 con 0x7ffa3006e680 2024-09-17T00:05:10.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.876+0000 7ffa377fe700 1 -- 172.21.15.42:0/3607210669 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffa48197100 con 0x7ffa3006e680 2024-09-17T00:05:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.878+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 msgr2=0x7ffa30070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.878+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa48197be0 tx=0x7ffa3c005f50 comp rx=0 tx=0).stop 2024-09-17T00:05:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.878+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 msgr2=0x7ffa4819c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.878+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa4819c6f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000c00 tx=0x7ffa3800be20 comp rx=0 tx=0).stop 2024-09-17T00:05:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.879+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 shutdown_connections 2024-09-17T00:05:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.879+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.879+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa48100ea0 0x7ffa4819c6f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.879+0000 7ffa4cd71700 1 --2- 172.21.15.42:0/3607210669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa48101850 0x7ffa4819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.879+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 >> 172.21.15.42:0/3607210669 conn(0x7ffa480fc750 msgr2=0x7ffa480fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.879+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 shutdown_connections 2024-09-17T00:05:10.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.879+0000 7ffa4cd71700 1 -- 172.21.15.42:0/3607210669 wait complete. 2024-09-17T00:05:10.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 -- 172.21.15.42:0/3052761142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 msgr2=0x7f82141058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 --2- 172.21.15.42:0/3052761142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82141058a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009910 tx=0x7f81fc009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:10.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 -- 172.21.15.42:0/3052761142 shutdown_connections 2024-09-17T00:05:10.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 --2- 172.21.15.42:0/3052761142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82141058a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 --2- 172.21.15.42:0/3052761142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214100ea0 0x7f8214101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 -- 172.21.15.42:0/3052761142 >> 172.21.15.42:0/3052761142 conn(0x7f82140fc750 msgr2=0x7f82140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 -- 172.21.15.42:0/3052761142 shutdown_connections 2024-09-17T00:05:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.987+0000 7f821986c700 1 -- 172.21.15.42:0/3052761142 wait complete. 2024-09-17T00:05:10.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f821986c700 1 Processor -- start 2024-09-17T00:05:10.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f821986c700 1 -- start start 2024-09-17T00:05:10.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f821986c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214100ea0 0x7f8214073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f821986c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82140735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f821986c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214073b10 con 0x7f8214101850 2024-09-17T00:05:10.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f821986c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214073c50 con 0x7f8214100ea0 2024-09-17T00:05:10.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f82127fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82140735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f82127fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82140735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47240/0 (socket says 172.21.15.42:47240) 2024-09-17T00:05:10.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f82127fc700 1 -- 172.21.15.42:0/3811539664 learned_addr learned my addr 172.21.15.42:0/3811539664 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:10.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.988+0000 7f8212ffd700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214100ea0 0x7f8214073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f82127fc700 1 -- 172.21.15.42:0/3811539664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214100ea0 msgr2=0x7f8214073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:10.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f82127fc700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214100ea0 0x7f8214073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:10.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f82127fc700 1 -- 172.21.15.42:0/3811539664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0095f0 con 0x7f8214101850 2024-09-17T00:05:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f8212ffd700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214100ea0 0x7f8214073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:05:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f82127fc700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82140735d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00bda0 tx=0x7f81fc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f821886a700 1 -- 172.21.15.42:0/3811539664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81fc01d070 con 0x7f8214101850 2024-09-17T00:05:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f821886a700 1 -- 172.21.15.42:0/3811539664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81fc004fb0 con 0x7f8214101850 2024-09-17T00:05:10.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8214073ed0 con 0x7f8214101850 2024-09-17T00:05:10.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f821886a700 1 -- 172.21.15.42:0/3811539664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81fc00f460 con 0x7f8214101850 2024-09-17T00:05:10.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.989+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a2e30 con 0x7f8214101850 2024-09-17T00:05:10.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.990+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82141092a0 con 0x7f8214101850 2024-09-17T00:05:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.991+0000 7f821886a700 1 -- 172.21.15.42:0/3811539664 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f81fc022800 con 0x7f8214101850 2024-09-17T00:05:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.991+0000 7f821886a700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f820006e680 0x7f8200070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.991+0000 7f821886a700 1 -- 172.21.15.42:0/3811539664 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f81fc08fc10 con 0x7f8214101850 2024-09-17T00:05:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.992+0000 7f8212ffd700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f820006e680 0x7f8200070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.992+0000 7f8212ffd700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f820006e680 0x7f8200070b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8204005950 tx=0x7f8204005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:10.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:10.996+0000 7f821886a700 1 -- 172.21.15.42:0/3811539664 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f81fc058e60 con 0x7f8214101850 2024-09-17T00:05:11.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.221+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f82140746d0 con 0x7f820006e680 2024-09-17T00:05:11.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.231+0000 7f821886a700 1 -- 172.21.15.42:0/3811539664 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f82140746d0 con 0x7f820006e680 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (13m) 7m ago 24m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (24m) 7m ago 24m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (23m) 7m ago 23m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (11m) 7m ago 24m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (11m) 7m ago 23m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (13m) 7m ago 24m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (7m) 7m ago 18m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (7m) 7m ago 18m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (7m) 7m ago 18m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (7m) 7m ago 18m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (15m) 7m ago 26m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:05:11.229 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (14m) 7m ago 23m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (12m) 7m ago 26m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (12m) 7m ago 23m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (14m) 7m ago 24m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (14m) 7m ago 23m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (11m) 7m ago 22m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (10m) 7m ago 22m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (9m) 7m ago 21m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (9m) 7m ago 21m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (8m) 7m ago 20m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:05:11.230 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (8m) 7m ago 20m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:05:11.231 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (14m) 7m ago 24m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:05:11.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f820006e680 msgr2=0x7f8200070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f820006e680 0x7f8200070b40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8204005950 tx=0x7f8204005f50 comp rx=0 tx=0).stop 2024-09-17T00:05:11.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 msgr2=0x7f82140735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82140735d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00bda0 tx=0x7f81fc00be80 comp rx=0 tx=0).stop 2024-09-17T00:05:11.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 shutdown_connections 2024-09-17T00:05:11.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f820006e680 0x7f8200070b40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214100ea0 0x7f8214073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 --2- 172.21.15.42:0/3811539664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8214101850 0x7f82140735d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 >> 172.21.15.42:0/3811539664 conn(0x7f82140fc750 msgr2=0x7f82140fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:11.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 shutdown_connections 2024-09-17T00:05:11.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.233+0000 7f821986c700 1 -- 172.21.15.42:0/3811539664 wait complete. 2024-09-17T00:05:11.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.340+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/3168036543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88100ea0 msgr2=0x7f1c88101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.340+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/3168036543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88100ea0 0x7f1c88101280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009910 tx=0x7f1c84009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:11.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.341+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/3168036543 shutdown_connections 2024-09-17T00:05:11.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.341+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/3168036543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c88101850 0x7f1c881058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.341+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/3168036543 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88100ea0 0x7f1c88101280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.341+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/3168036543 >> 172.21.15.42:0/3168036543 conn(0x7f1c880fc750 msgr2=0x7f1c880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:11.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.341+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/3168036543 shutdown_connections 2024-09-17T00:05:11.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.341+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/3168036543 wait complete. 2024-09-17T00:05:11.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.341+0000 7f1c8fa7c700 1 Processor -- start 2024-09-17T00:05:11.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8fa7c700 1 -- start start 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8fa7c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c88100ea0 0x7f1c8819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8fa7c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88101850 0x7f1c8819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8fa7c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8819d2c0 con 0x7f1c88101850 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8fa7c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c88196770 con 0x7f1c88100ea0 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d017700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88101850 0x7f1c8819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d017700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88101850 0x7f1c8819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47244/0 (socket says 172.21.15.42:47244) 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d017700 1 -- 172.21.15.42:0/1095038394 learned_addr learned my addr 172.21.15.42:0/1095038394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d818700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c88100ea0 0x7f1c8819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d017700 1 -- 172.21.15.42:0/1095038394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c88100ea0 msgr2=0x7f1c8819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d017700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c88100ea0 0x7f1c8819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d017700 1 -- 172.21.15.42:0/1095038394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c840095f0 con 0x7f1c88101850 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.342+0000 7f1c8d017700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88101850 0x7f1c8819cc30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e8f0 tx=0x7f1c7800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:11.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.343+0000 7f1c7effd700 1 -- 172.21.15.42:0/1095038394 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c78009790 con 0x7f1c88101850 2024-09-17T00:05:11.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.343+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c88196a50 con 0x7f1c88101850 2024-09-17T00:05:11.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.343+0000 7f1c7effd700 1 -- 172.21.15.42:0/1095038394 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c7800c810 con 0x7f1c88101850 2024-09-17T00:05:11.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.343+0000 7f1c7effd700 1 -- 172.21.15.42:0/1095038394 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c78005580 con 0x7f1c88101850 2024-09-17T00:05:11.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.343+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c88196fa0 con 0x7f1c88101850 2024-09-17T00:05:11.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.344+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c88109240 con 0x7f1c88101850 2024-09-17T00:05:11.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.344+0000 7f1c7effd700 1 -- 172.21.15.42:0/1095038394 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1c7800c980 con 0x7f1c88101850 2024-09-17T00:05:11.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.345+0000 7f1c7effd700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c7406e680 0x7f1c74070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:11.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.345+0000 7f1c7effd700 1 -- 172.21.15.42:0/1095038394 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1c78014070 con 0x7f1c88101850 2024-09-17T00:05:11.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.345+0000 7f1c8d818700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c7406e680 0x7f1c74070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:11.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.345+0000 7f1c8d818700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c7406e680 0x7f1c74070b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1c840059d0 tx=0x7f1c84005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:11.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.349+0000 7f1c7effd700 1 -- 172.21.15.42:0/1095038394 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1c7805b5e0 con 0x7f1c88101850 2024-09-17T00:05:11.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:11 smithi042 ceph-mon[111056]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:11.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.647+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1c88197d20 con 0x7f1c88101850 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.648+0000 7f1c7effd700 1 -- 172.21.15.42:0/1095038394 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1c7805b170 con 0x7f1c88101850 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:11.646 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:05:11.647 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:05:11.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c7406e680 msgr2=0x7f1c74070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c7406e680 0x7f1c74070b40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1c840059d0 tx=0x7f1c84005940 comp rx=0 tx=0).stop 2024-09-17T00:05:11.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88101850 msgr2=0x7f1c8819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88101850 0x7f1c8819cc30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e8f0 tx=0x7f1c7800ec00 comp rx=0 tx=0).stop 2024-09-17T00:05:11.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 shutdown_connections 2024-09-17T00:05:11.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1c7406e680 0x7f1c74070b40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c88100ea0 0x7f1c8819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 --2- 172.21.15.42:0/1095038394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1c88101850 0x7f1c8819cc30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 >> 172.21.15.42:0/1095038394 conn(0x7f1c880fc750 msgr2=0x7f1c880fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:11.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 shutdown_connections 2024-09-17T00:05:11.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.650+0000 7f1c8fa7c700 1 -- 172.21.15.42:0/1095038394 wait complete. 2024-09-17T00:05:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.757+0000 7f518d75c700 1 -- 172.21.15.42:0/2432617549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 msgr2=0x7f51881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.757+0000 7f518d75c700 1 --2- 172.21.15.42:0/2432617549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f51881021c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5178009910 tx=0x7f5178009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 -- 172.21.15.42:0/2432617549 shutdown_connections 2024-09-17T00:05:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 --2- 172.21.15.42:0/2432617549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5188102700 0x7f518810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 --2- 172.21.15.42:0/2432617549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f51881021c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 -- 172.21.15.42:0/2432617549 >> 172.21.15.42:0/2432617549 conn(0x7f51880fb650 msgr2=0x7f51880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 -- 172.21.15.42:0/2432617549 shutdown_connections 2024-09-17T00:05:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 -- 172.21.15.42:0/2432617549 wait complete. 2024-09-17T00:05:11.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 Processor -- start 2024-09-17T00:05:11.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.758+0000 7f518d75c700 1 -- start start 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f518d75c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f518819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f518d75c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5188102700 0x7f518819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f518d75c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518819d370 con 0x7f5188101de0 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f518d75c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5188196820 con 0x7f5188102700 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f5186ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f518819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f5186ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f518819c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47268/0 (socket says 172.21.15.42:47268) 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f5186ffd700 1 -- 172.21.15.42:0/1192620510 learned_addr learned my addr 172.21.15.42:0/1192620510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f51867fc700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5188102700 0x7f518819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f5186ffd700 1 -- 172.21.15.42:0/1192620510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5188102700 msgr2=0x7f518819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f5186ffd700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5188102700 0x7f518819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f5186ffd700 1 -- 172.21.15.42:0/1192620510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51780095f0 con 0x7f5188101de0 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f5186ffd700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f518819c7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5178006010 tx=0x7f517800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f516ffff700 1 -- 172.21.15.42:0/1192620510 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517801d070 con 0x7f5188101de0 2024-09-17T00:05:11.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f516ffff700 1 -- 172.21.15.42:0/1192620510 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5178004fb0 con 0x7f5188101de0 2024-09-17T00:05:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f516ffff700 1 -- 172.21.15.42:0/1192620510 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517800f690 con 0x7f5188101de0 2024-09-17T00:05:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.759+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5188196aa0 con 0x7f5188101de0 2024-09-17T00:05:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.760+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5188196f90 con 0x7f5188101de0 2024-09-17T00:05:11.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.760+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51881082f0 con 0x7f5188101de0 2024-09-17T00:05:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.761+0000 7f516ffff700 1 -- 172.21.15.42:0/1192620510 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f51780228a0 con 0x7f5188101de0 2024-09-17T00:05:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.762+0000 7f516ffff700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f517006e680 0x7f5170070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.762+0000 7f51867fc700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f517006e680 0x7f5170070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.762+0000 7f516ffff700 1 -- 172.21.15.42:0/1192620510 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f517808ff70 con 0x7f5188101de0 2024-09-17T00:05:11.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.762+0000 7f51867fc700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f517006e680 0x7f5170070b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51880fcf30 tx=0x7f517c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:11.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:11.766+0000 7f516ffff700 1 -- 172.21.15.42:0/1192620510 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f517805cba0 con 0x7f5188101de0 2024-09-17T00:05:11.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:11 smithi043 ceph-mon[81510]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.017+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f518804f2e0 con 0x7f5188101de0 2024-09-17T00:05:12.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.018+0000 7f516ffff700 1 -- 172.21.15.42:0/1192620510 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f5178027790 con 0x7f5188101de0 2024-09-17T00:05:12.015 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:05:12.016 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:05:12.017 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:12.019 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:12.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f517006e680 msgr2=0x7f5170070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f517006e680 0x7f5170070b40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f51880fcf30 tx=0x7f517c008040 comp rx=0 tx=0).stop 2024-09-17T00:05:12.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 msgr2=0x7f518819c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f518819c7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5178006010 tx=0x7f517800be20 comp rx=0 tx=0).stop 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 shutdown_connections 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f517006e680 0x7f5170070b40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5188101de0 0x7f518819c7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 --2- 172.21.15.42:0/1192620510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5188102700 0x7f518819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 >> 172.21.15.42:0/1192620510 conn(0x7f51880fb650 msgr2=0x7f5188105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 shutdown_connections 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.020+0000 7f518d75c700 1 -- 172.21.15.42:0/1192620510 wait complete. 2024-09-17T00:05:12.021 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:05:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 -- 172.21.15.42:0/2723688577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 msgr2=0x7f322810ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 --2- 172.21.15.42:0/2723688577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f322810ac70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f321c009910 tx=0x7f321c009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 -- 172.21.15.42:0/2723688577 shutdown_connections 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 --2- 172.21.15.42:0/2723688577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f322810ac70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 --2- 172.21.15.42:0/2723688577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3228101e60 0x7f3228102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 -- 172.21.15.42:0/2723688577 >> 172.21.15.42:0/2723688577 conn(0x7f32280fb650 msgr2=0x7f32280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 -- 172.21.15.42:0/2723688577 shutdown_connections 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.129+0000 7f322da6a700 1 -- 172.21.15.42:0/2723688577 wait complete. 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f322da6a700 1 Processor -- start 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f322da6a700 1 -- start start 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f322da6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3228101e60 0x7f32280ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:12.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f322da6a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f3228100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f322da6a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3228100540 con 0x7f3228102780 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f322da6a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3228100680 con 0x7f3228101e60 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f3226ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f3228100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f3226ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f3228100000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47284/0 (socket says 172.21.15.42:47284) 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f3226ffd700 1 -- 172.21.15.42:0/2890868396 learned_addr learned my addr 172.21.15.42:0/2890868396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.130+0000 7f32277fe700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3228101e60 0x7f32280ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f3226ffd700 1 -- 172.21.15.42:0/2890868396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3228101e60 msgr2=0x7f32280ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f3226ffd700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3228101e60 0x7f32280ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f3226ffd700 1 -- 172.21.15.42:0/2890868396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0095f0 con 0x7f3228102780 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f32277fe700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3228101e60 0x7f32280ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f3226ffd700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f3228100000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f321c009fd0 tx=0x7f321c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f3224ff9700 1 -- 172.21.15.42:0/2890868396 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f321c01d070 con 0x7f3228102780 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3228100900 con 0x7f3228102780 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f3224ff9700 1 -- 172.21.15.42:0/2890868396 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f321c022470 con 0x7f3228102780 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f3224ff9700 1 -- 172.21.15.42:0/2890868396 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f321c017610 con 0x7f3228102780 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.131+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32281a2cc0 con 0x7f3228102780 2024-09-17T00:05:12.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.132+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3228108300 con 0x7f3228102780 2024-09-17T00:05:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.133+0000 7f3224ff9700 1 -- 172.21.15.42:0/2890868396 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f321c005080 con 0x7f3228102780 2024-09-17T00:05:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.133+0000 7f3224ff9700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f321006e680 0x7f3210070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.133+0000 7f3224ff9700 1 -- 172.21.15.42:0/2890868396 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f321c08fd10 con 0x7f3228102780 2024-09-17T00:05:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.133+0000 7f32277fe700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f321006e680 0x7f3210070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:12.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.134+0000 7f32277fe700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f321006e680 0x7f3210070b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3218009be0 tx=0x7f3218009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:12.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.138+0000 7f3224ff9700 1 -- 172.21.15.42:0/2890868396 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f321c058eb0 con 0x7f3228102780 2024-09-17T00:05:12.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.370+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32280611d0 con 0x7f321006e680 2024-09-17T00:05:12.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.371+0000 7f3224ff9700 1 -- 172.21.15.42:0/2890868396 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f32280611d0 con 0x7f321006e680 2024-09-17T00:05:12.369 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:05:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:05:12.369 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:05:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:05:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:05:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:05:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:05:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:05:12.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:05:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:05:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:05:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:05:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:05:12.371 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:05:12.371 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:05:12.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f321006e680 msgr2=0x7f3210070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f321006e680 0x7f3210070b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3218009be0 tx=0x7f3218009260 comp rx=0 tx=0).stop 2024-09-17T00:05:12.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 msgr2=0x7f3228100000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f3228100000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f321c009fd0 tx=0x7f321c00f690 comp rx=0 tx=0).stop 2024-09-17T00:05:12.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 shutdown_connections 2024-09-17T00:05:12.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f321006e680 0x7f3210070b40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3228101e60 0x7f32280ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 --2- 172.21.15.42:0/2890868396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3228102780 0x7f3228100000 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 >> 172.21.15.42:0/2890868396 conn(0x7f32280fb650 msgr2=0x7f3228105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:12.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 shutdown_connections 2024-09-17T00:05:12.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.374+0000 7f322da6a700 1 -- 172.21.15.42:0/2890868396 wait complete. 2024-09-17T00:05:12.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 -- 172.21.15.42:0/672747700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d8100750 msgr2=0x7f53d81047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/672747700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d8100750 0x7f53d81047a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009910 tx=0x7f53c0009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:12.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 -- 172.21.15.42:0/672747700 shutdown_connections 2024-09-17T00:05:12.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/672747700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d8100750 0x7f53d81047a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/672747700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d80ffda0 0x7f53d8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 -- 172.21.15.42:0/672747700 >> 172.21.15.42:0/672747700 conn(0x7f53d80fb650 msgr2=0x7f53d80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:12.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 -- 172.21.15.42:0/672747700 shutdown_connections 2024-09-17T00:05:12.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.482+0000 7f53dd2e7700 1 -- 172.21.15.42:0/672747700 wait complete. 2024-09-17T00:05:12.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53dd2e7700 1 Processor -- start 2024-09-17T00:05:12.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53dd2e7700 1 -- start start 2024-09-17T00:05:12.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53dd2e7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d80ffda0 0x7f53d8196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:12.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53dd2e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d8100750 0x7f53d8196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:12.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53dd2e7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8197520 con 0x7f53d80ffda0 2024-09-17T00:05:12.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53dd2e7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d819b330 con 0x7f53d8100750 2024-09-17T00:05:12.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d80ffda0 0x7f53d8196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53d6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d80ffda0 0x7f53d8196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47292/0 (socket says 172.21.15.42:47292) 2024-09-17T00:05:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.483+0000 7f53d6ffd700 1 -- 172.21.15.42:0/1796243260 learned_addr learned my addr 172.21.15.42:0/1796243260 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53d6ffd700 1 -- 172.21.15.42:0/1796243260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d8100750 msgr2=0x7f53d8196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53d6ffd700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d8100750 0x7f53d8196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53d6ffd700 1 -- 172.21.15.42:0/1796243260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00095f0 con 0x7f53d80ffda0 2024-09-17T00:05:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53d6ffd700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d80ffda0 0x7f53d8196950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f53c800d8d0 tx=0x7f53c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:12.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53d4ff9700 1 -- 172.21.15.42:0/1796243260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53c8009940 con 0x7f53d80ffda0 2024-09-17T00:05:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53d4ff9700 1 -- 172.21.15.42:0/1796243260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53c8010460 con 0x7f53d80ffda0 2024-09-17T00:05:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d819b610 con 0x7f53d80ffda0 2024-09-17T00:05:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53d4ff9700 1 -- 172.21.15.42:0/1796243260 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53c800f5f0 con 0x7f53d80ffda0 2024-09-17T00:05:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.484+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d819bb60 con 0x7f53d80ffda0 2024-09-17T00:05:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.485+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d804f2e0 con 0x7f53d80ffda0 2024-09-17T00:05:12.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.486+0000 7f53d4ff9700 1 -- 172.21.15.42:0/1796243260 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f53c80105d0 con 0x7f53d80ffda0 2024-09-17T00:05:12.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.486+0000 7f53d4ff9700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f53c406e680 0x7f53c4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:12.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.486+0000 7f53d4ff9700 1 -- 172.21.15.42:0/1796243260 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f53c808e680 con 0x7f53d80ffda0 2024-09-17T00:05:12.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.487+0000 7f53cffff700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f53c406e680 0x7f53c4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:12.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.487+0000 7f53cffff700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f53c406e680 0x7f53c4070b40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006010 tx=0x7f53c000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:12.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.491+0000 7f53d4ff9700 1 -- 172.21.15.42:0/1796243260 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f53c805b350 con 0x7f53d80ffda0 2024-09-17T00:05:12.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:12 smithi042 ceph-mon[111056]: pgmap v408: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:12 smithi042 ceph-mon[111056]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:12 smithi042 ceph-mon[111056]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:12 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1095038394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:05:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:12 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1192620510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:05:12.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.788+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f53d819bd80 con 0x7f53d80ffda0 2024-09-17T00:05:12.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.789+0000 7f53d4ff9700 1 -- 172.21.15.42:0/1796243260 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f53c805aee0 con 0x7f53d80ffda0 2024-09-17T00:05:12.786 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:05:12.787 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:05:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:05:12.787 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:05:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:05:12.787 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:05:12.787 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:05:12.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f53c406e680 msgr2=0x7f53c4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f53c406e680 0x7f53c4070b40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006010 tx=0x7f53c000b540 comp rx=0 tx=0).stop 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d80ffda0 msgr2=0x7f53d8196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d80ffda0 0x7f53d8196950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f53c800d8d0 tx=0x7f53c800dc90 comp rx=0 tx=0).stop 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 shutdown_connections 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f53c406e680 0x7f53c4070b40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53d80ffda0 0x7f53d8196950 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 --2- 172.21.15.42:0/1796243260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53d8100750 0x7f53d8196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 >> 172.21.15.42:0/1796243260 conn(0x7f53d80fb650 msgr2=0x7f53d8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 shutdown_connections 2024-09-17T00:05:12.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:12.791+0000 7f53dd2e7700 1 -- 172.21.15.42:0/1796243260 wait complete. 2024-09-17T00:05:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:12 smithi043 ceph-mon[81510]: pgmap v408: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:12 smithi043 ceph-mon[81510]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:12 smithi043 ceph-mon[81510]: from='client.34804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:12 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1095038394' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:05:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:12 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1192620510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:05:13.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:13 smithi042 ceph-mon[111056]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:13.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:13 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1796243260' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:05:13.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:13 smithi043 ceph-mon[81510]: from='client.34816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:13.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:13 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1796243260' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:05:14.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:14 smithi042 ceph-mon[111056]: pgmap v409: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:14 smithi043 ceph-mon[81510]: pgmap v409: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:16.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:16 smithi042 ceph-mon[111056]: pgmap v410: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:16.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:16 smithi043 ceph-mon[81510]: pgmap v410: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:18.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:18 smithi042 ceph-mon[111056]: pgmap v411: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:18.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:18 smithi043 ceph-mon[81510]: pgmap v411: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:20.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:20 smithi042 ceph-mon[111056]: pgmap v412: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:20.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:20 smithi043 ceph-mon[81510]: pgmap v412: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:21.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:22.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:22 smithi042 ceph-mon[111056]: pgmap v413: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:22.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:22 smithi043 ceph-mon[81510]: pgmap v413: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:24.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:24 smithi042 ceph-mon[111056]: pgmap v414: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:24.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:24 smithi043 ceph-mon[81510]: pgmap v414: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:26.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:26 smithi042 ceph-mon[111056]: pgmap v415: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:26.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:26 smithi043 ceph-mon[81510]: pgmap v415: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:28 smithi042 ceph-mon[111056]: pgmap v416: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:28.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:28 smithi043 ceph-mon[81510]: pgmap v416: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:30.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:30 smithi042 ceph-mon[111056]: pgmap v417: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:30.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:30 smithi043 ceph-mon[81510]: pgmap v417: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:32.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:32 smithi042 ceph-mon[111056]: pgmap v418: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:32.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:32 smithi043 ceph-mon[81510]: pgmap v418: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:34.735 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:34 smithi043 ceph-mon[81510]: pgmap v419: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:34.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:34 smithi042 ceph-mon[111056]: pgmap v419: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:36 smithi042 ceph-mon[111056]: pgmap v420: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:36.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:36 smithi043 ceph-mon[81510]: pgmap v420: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:38.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:38 smithi042 ceph-mon[111056]: pgmap v421: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:38 smithi043 ceph-mon[81510]: pgmap v421: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:40.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:40 smithi042 ceph-mon[111056]: pgmap v422: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:40.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:40 smithi043 ceph-mon[81510]: pgmap v422: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:42.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:42 smithi042 ceph-mon[111056]: pgmap v423: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:42.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:42 smithi043 ceph-mon[81510]: pgmap v423: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:42.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 -- 172.21.15.42:0/1808337590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0100ea0 msgr2=0x7fb3a0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:42.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1808337590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0100ea0 0x7fb3a0101280 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb390009910 tx=0x7fb390009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:42.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 -- 172.21.15.42:0/1808337590 shutdown_connections 2024-09-17T00:05:42.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1808337590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a0101850 0x7fb3a01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:42.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1808337590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0100ea0 0x7fb3a0101280 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:42.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 -- 172.21.15.42:0/1808337590 >> 172.21.15.42:0/1808337590 conn(0x7fb3a00fc750 msgr2=0x7fb3a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 -- 172.21.15.42:0/1808337590 shutdown_connections 2024-09-17T00:05:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.914+0000 7fb3a6785700 1 -- 172.21.15.42:0/1808337590 wait complete. 2024-09-17T00:05:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.915+0000 7fb3a6785700 1 Processor -- start 2024-09-17T00:05:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.915+0000 7fb3a6785700 1 -- start start 2024-09-17T00:05:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb3a6785700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a0100ea0 0x7fb3a019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:42.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb3a6785700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0101850 0x7fb3a019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:42.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb3a6785700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a019d2e0 con 0x7fb3a0101850 2024-09-17T00:05:42.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb3a6785700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a0196790 con 0x7fb3a0100ea0 2024-09-17T00:05:42.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0101850 0x7fb3a019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:42.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0101850 0x7fb3a019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41892/0 (socket says 172.21.15.42:41892) 2024-09-17T00:05:42.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39f7fe700 1 -- 172.21.15.42:0/1756919204 learned_addr learned my addr 172.21.15.42:0/1756919204 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:42.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39ffff700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a0100ea0 0x7fb3a019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:42.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39f7fe700 1 -- 172.21.15.42:0/1756919204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a0100ea0 msgr2=0x7fb3a019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:42.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39f7fe700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a0100ea0 0x7fb3a019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:42.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39f7fe700 1 -- 172.21.15.42:0/1756919204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900095f0 con 0x7fb3a0101850 2024-09-17T00:05:42.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.916+0000 7fb39f7fe700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0101850 0x7fb3a019cc50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d900 tx=0x7fb38800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:42.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.917+0000 7fb39d7fa700 1 -- 172.21.15.42:0/1756919204 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb388009900 con 0x7fb3a0101850 2024-09-17T00:05:42.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.917+0000 7fb39d7fa700 1 -- 172.21.15.42:0/1756919204 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb388010460 con 0x7fb3a0101850 2024-09-17T00:05:42.919 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.917+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a0196a70 con 0x7fb3a0101850 2024-09-17T00:05:42.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.917+0000 7fb39d7fa700 1 -- 172.21.15.42:0/1756919204 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38800f5f0 con 0x7fb3a0101850 2024-09-17T00:05:42.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.917+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a0196fc0 con 0x7fb3a0101850 2024-09-17T00:05:42.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.918+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a0109240 con 0x7fb3a0101850 2024-09-17T00:05:42.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.918+0000 7fb39d7fa700 1 -- 172.21.15.42:0/1756919204 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb3880105d0 con 0x7fb3a0101850 2024-09-17T00:05:42.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.919+0000 7fb39d7fa700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb38c06e680 0x7fb38c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:42.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.919+0000 7fb39d7fa700 1 -- 172.21.15.42:0/1756919204 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb38808e240 con 0x7fb3a0101850 2024-09-17T00:05:42.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.919+0000 7fb39ffff700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb38c06e680 0x7fb38c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:42.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.919+0000 7fb39ffff700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb38c06e680 0x7fb38c070b40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb39000b5c0 tx=0x7fb39001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:42.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:42.923+0000 7fb39d7fa700 1 -- 172.21.15.42:0/1756919204 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb38805af10 con 0x7fb3a0101850 2024-09-17T00:05:43.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.155+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3a01971e0 con 0x7fb38c06e680 2024-09-17T00:05:43.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.157+0000 7fb39d7fa700 1 -- 172.21.15.42:0/1756919204 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb3a01971e0 con 0x7fb38c06e680 2024-09-17T00:05:43.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb38c06e680 msgr2=0x7fb38c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb38c06e680 0x7fb38c070b40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb39000b5c0 tx=0x7fb39001a040 comp rx=0 tx=0).stop 2024-09-17T00:05:43.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0101850 msgr2=0x7fb3a019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0101850 0x7fb3a019cc50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d900 tx=0x7fb38800dc10 comp rx=0 tx=0).stop 2024-09-17T00:05:43.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 shutdown_connections 2024-09-17T00:05:43.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb38c06e680 0x7fb38c070b40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3a0100ea0 0x7fb3a019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 --2- 172.21.15.42:0/1756919204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3a0101850 0x7fb3a019cc50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 >> 172.21.15.42:0/1756919204 conn(0x7fb3a00fc750 msgr2=0x7fb3a00fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:43.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 shutdown_connections 2024-09-17T00:05:43.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.159+0000 7fb3a6785700 1 -- 172.21.15.42:0/1756919204 wait complete. 2024-09-17T00:05:43.167 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:05:43.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/376568287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a58105bd0 msgr2=0x7f8a58105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/376568287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a58105bd0 0x7f8a58105fb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009910 tx=0x7f8a54009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:43.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/376568287 shutdown_connections 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/376568287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a580ffc20 0x7f8a58100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/376568287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a58105bd0 0x7f8a58105fb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/376568287 >> 172.21.15.42:0/376568287 conn(0x7f8a58075670 msgr2=0x7f8a58075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/376568287 shutdown_connections 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.269+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/376568287 wait complete. 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5f2b4700 1 Processor -- start 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5f2b4700 1 -- start start 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5f2b4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a580ffc20 0x7f8a581976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5f2b4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a58105bd0 0x7f8a58197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5f2b4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a581982f0 con 0x7f8a580ffc20 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5f2b4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a58198460 con 0x7f8a58105bd0 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5d050700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a580ffc20 0x7f8a581976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5d050700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a580ffc20 0x7f8a581976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41916/0 (socket says 172.21.15.42:41916) 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.270+0000 7f8a5d050700 1 -- 172.21.15.42:0/2204647191 learned_addr learned my addr 172.21.15.42:0/2204647191 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a5c84f700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a58105bd0 0x7f8a58197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a5d050700 1 -- 172.21.15.42:0/2204647191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a58105bd0 msgr2=0x7f8a58197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a5d050700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a58105bd0 0x7f8a58197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a5d050700 1 -- 172.21.15.42:0/2204647191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540095f0 con 0x7f8a580ffc20 2024-09-17T00:05:43.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a5d050700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a580ffc20 0x7f8a581976d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a54006010 tx=0x7f8a5400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a4e7fc700 1 -- 172.21.15.42:0/2204647191 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a5401d070 con 0x7f8a580ffc20 2024-09-17T00:05:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a4e7fc700 1 -- 172.21.15.42:0/2204647191 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a54005dc0 con 0x7f8a580ffc20 2024-09-17T00:05:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a581a0740 con 0x7f8a580ffc20 2024-09-17T00:05:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a4e7fc700 1 -- 172.21.15.42:0/2204647191 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a54017400 con 0x7f8a580ffc20 2024-09-17T00:05:43.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.271+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a581a0bb0 con 0x7f8a580ffc20 2024-09-17T00:05:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.272+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a581080f0 con 0x7f8a580ffc20 2024-09-17T00:05:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.273+0000 7f8a4e7fc700 1 -- 172.21.15.42:0/2204647191 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8a54003a60 con 0x7f8a580ffc20 2024-09-17T00:05:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.273+0000 7f8a4e7fc700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a4406e680 0x7f8a44070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.273+0000 7f8a5c84f700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a4406e680 0x7f8a44070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.273+0000 7f8a4e7fc700 1 -- 172.21.15.42:0/2204647191 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8a5408fa00 con 0x7f8a580ffc20 2024-09-17T00:05:43.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.274+0000 7f8a5c84f700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a4406e680 0x7f8a44070b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a58068e30 tx=0x7f8a480073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:43.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.278+0000 7f8a4e7fc700 1 -- 172.21.15.42:0/2204647191 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8a54058c20 con 0x7f8a580ffc20 2024-09-17T00:05:43.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.507+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a580611d0 con 0x7f8a4406e680 2024-09-17T00:05:43.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.508+0000 7f8a4e7fc700 1 -- 172.21.15.42:0/2204647191 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a580611d0 con 0x7f8a4406e680 2024-09-17T00:05:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a4406e680 msgr2=0x7f8a44070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a4406e680 0x7f8a44070b40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8a58068e30 tx=0x7f8a480073d0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a580ffc20 msgr2=0x7f8a581976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a580ffc20 0x7f8a581976d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8a54006010 tx=0x7f8a5400bd70 comp rx=0 tx=0).stop 2024-09-17T00:05:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 shutdown_connections 2024-09-17T00:05:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a4406e680 0x7f8a44070b40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a580ffc20 0x7f8a581976d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 --2- 172.21.15.42:0/2204647191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a58105bd0 0x7f8a58197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 >> 172.21.15.42:0/2204647191 conn(0x7f8a58075670 msgr2=0x7f8a580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 shutdown_connections 2024-09-17T00:05:43.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.510+0000 7f8a5f2b4700 1 -- 172.21.15.42:0/2204647191 wait complete. 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 -- 172.21.15.42:0/1473383792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 msgr2=0x7f0f3c10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 --2- 172.21.15.42:0/1473383792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c10ac10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009910 tx=0x7f0f30009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 -- 172.21.15.42:0/1473383792 shutdown_connections 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 --2- 172.21.15.42:0/1473383792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c10ac10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 --2- 172.21.15.42:0/1473383792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c101e00 0x7f0f3c1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 -- 172.21.15.42:0/1473383792 >> 172.21.15.42:0/1473383792 conn(0x7f0f3c0fb650 msgr2=0x7f0f3c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 -- 172.21.15.42:0/1473383792 shutdown_connections 2024-09-17T00:05:43.617 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.619+0000 7f0f41d40700 1 -- 172.21.15.42:0/1473383792 wait complete. 2024-09-17T00:05:43.619 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f41d40700 1 Processor -- start 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f41d40700 1 -- start start 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f41d40700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c101e00 0x7f0f3c0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f41d40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c0fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f41d40700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c1004e0 con 0x7f0f3c102720 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f41d40700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c100620 con 0x7f0f3c101e00 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c0fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c0fffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41948/0 (socket says 172.21.15.42:41948) 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f3affd700 1 -- 172.21.15.42:0/724706770 learned_addr learned my addr 172.21.15.42:0/724706770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:43.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.620+0000 7f0f3b7fe700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c101e00 0x7f0f3c0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f3affd700 1 -- 172.21.15.42:0/724706770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c101e00 msgr2=0x7f0f3c0ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f3affd700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c101e00 0x7f0f3c0ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f3affd700 1 -- 172.21.15.42:0/724706770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f300095f0 con 0x7f0f3c102720 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f3affd700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c0fffa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000bd70 tx=0x7f0f3000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f38ff9700 1 -- 172.21.15.42:0/724706770 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f3001d070 con 0x7f0f3c102720 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f38ff9700 1 -- 172.21.15.42:0/724706770 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f3000f460 con 0x7f0f3c102720 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3c1008a0 con 0x7f0f3c102720 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f38ff9700 1 -- 172.21.15.42:0/724706770 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f30017630 con 0x7f0f3c102720 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.621+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c1a2dc0 con 0x7f0f3c102720 2024-09-17T00:05:43.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.622+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c108310 con 0x7f0f3c102720 2024-09-17T00:05:43.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.623+0000 7f0f38ff9700 1 -- 172.21.15.42:0/724706770 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f0f3000fab0 con 0x7f0f3c102720 2024-09-17T00:05:43.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.623+0000 7f0f38ff9700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0f2406e680 0x7f0f24070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.623+0000 7f0f38ff9700 1 -- 172.21.15.42:0/724706770 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0f3008fb60 con 0x7f0f3c102720 2024-09-17T00:05:43.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.623+0000 7f0f3b7fe700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0f2406e680 0x7f0f24070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.624+0000 7f0f3b7fe700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0f2406e680 0x7f0f24070b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005fd0 tx=0x7f0f2c00a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:43.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.628+0000 7f0f38ff9700 1 -- 172.21.15.42:0/724706770 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0f30058d80 con 0x7f0f3c102720 2024-09-17T00:05:43.847 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.849+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3c0611d0 con 0x7f0f2406e680 2024-09-17T00:05:43.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.859+0000 7f0f38ff9700 1 -- 172.21.15.42:0/724706770 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f0f3c0611d0 con 0x7f0f2406e680 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (14m) 8m ago 25m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (25m) 8m ago 25m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (24m) 7m ago 24m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (11m) 8m ago 25m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (11m) 7m ago 24m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (13m) 8m ago 24m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (8m) 8m ago 19m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (8m) 8m ago 19m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (7m) 7m ago 19m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (7m) 7m ago 19m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:05:43.857 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (16m) 8m ago 26m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (15m) 7m ago 24m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (13m) 8m ago 26m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (12m) 7m ago 24m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (15m) 8m ago 25m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (14m) 7m ago 24m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (11m) 8m ago 23m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (10m) 8m ago 22m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (10m) 8m ago 22m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (9m) 7m ago 21m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (9m) 7m ago 21m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (8m) 7m ago 21m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:05:43.858 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (14m) 8m ago 24m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:05:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0f2406e680 msgr2=0x7f0f24070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0f2406e680 0x7f0f24070b40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005fd0 tx=0x7f0f2c00a560 comp rx=0 tx=0).stop 2024-09-17T00:05:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 msgr2=0x7f0f3c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c0fffa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000bd70 tx=0x7f0f3000be50 comp rx=0 tx=0).stop 2024-09-17T00:05:43.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 shutdown_connections 2024-09-17T00:05:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f0f2406e680 0x7f0f24070b40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3c101e00 0x7f0f3c0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 --2- 172.21.15.42:0/724706770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0f3c102720 0x7f0f3c0fffa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 >> 172.21.15.42:0/724706770 conn(0x7f0f3c0fb650 msgr2=0x7f0f3c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 shutdown_connections 2024-09-17T00:05:43.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.861+0000 7f0f41d40700 1 -- 172.21.15.42:0/724706770 wait complete. 2024-09-17T00:05:43.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 -- 172.21.15.42:0/506707249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb68100380 msgr2=0x7fcb6810d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 --2- 172.21.15.42:0/506707249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb68100380 0x7fcb6810d040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009910 tx=0x7fcb5c009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:43.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 -- 172.21.15.42:0/506707249 shutdown_connections 2024-09-17T00:05:43.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 --2- 172.21.15.42:0/506707249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb68100380 0x7fcb6810d040 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 --2- 172.21.15.42:0/506707249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb680ff9d0 0x7fcb680ffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 -- 172.21.15.42:0/506707249 >> 172.21.15.42:0/506707249 conn(0x7fcb680fb650 msgr2=0x7fcb680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:43.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 -- 172.21.15.42:0/506707249 shutdown_connections 2024-09-17T00:05:43.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 -- 172.21.15.42:0/506707249 wait complete. 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.970+0000 7fcb7056d700 1 Processor -- start 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb7056d700 1 -- start start 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb7056d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb680ff9d0 0x7fcb6819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb7056d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb68100380 0x7fcb6819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb7056d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6819f3b0 con 0x7fcb680ff9d0 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb7056d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb68198860 con 0x7fcb68100380 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6e309700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb680ff9d0 0x7fcb6819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6e309700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb680ff9d0 0x7fcb6819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41966/0 (socket says 172.21.15.42:41966) 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6e309700 1 -- 172.21.15.42:0/2826807337 learned_addr learned my addr 172.21.15.42:0/2826807337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:43.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6db08700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb68100380 0x7fcb6819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6e309700 1 -- 172.21.15.42:0/2826807337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb68100380 msgr2=0x7fcb6819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6e309700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb68100380 0x7fcb6819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6e309700 1 -- 172.21.15.42:0/2826807337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c0095f0 con 0x7fcb680ff9d0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.971+0000 7fcb6e309700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb680ff9d0 0x7fcb6819e7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d8d0 tx=0x7fcb6400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.972+0000 7fcb5b7fe700 1 -- 172.21.15.42:0/2826807337 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb640098a0 con 0x7fcb680ff9d0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.972+0000 7fcb5b7fe700 1 -- 172.21.15.42:0/2826807337 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb64010460 con 0x7fcb680ff9d0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.972+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb68198b40 con 0x7fcb680ff9d0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.972+0000 7fcb5b7fe700 1 -- 172.21.15.42:0/2826807337 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb6400f5f0 con 0x7fcb680ff9d0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.972+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb68199090 con 0x7fcb680ff9d0 2024-09-17T00:05:43.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.973+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6810a7b0 con 0x7fcb680ff9d0 2024-09-17T00:05:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.973+0000 7fcb5b7fe700 1 -- 172.21.15.42:0/2826807337 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fcb64009a00 con 0x7fcb680ff9d0 2024-09-17T00:05:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.974+0000 7fcb5b7fe700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcb5406e680 0x7fcb54070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.974+0000 7fcb5b7fe700 1 -- 172.21.15.42:0/2826807337 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcb6408e550 con 0x7fcb680ff9d0 2024-09-17T00:05:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.974+0000 7fcb6db08700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcb5406e680 0x7fcb54070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.974+0000 7fcb6db08700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcb5406e680 0x7fcb54070b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb680fbe00 tx=0x7fcb5c00b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:43.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:43.978+0000 7fcb5b7fe700 1 -- 172.21.15.42:0/2826807337 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcb6405b220 con 0x7fcb680ff9d0 2024-09-17T00:05:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.273+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb681992b0 con 0x7fcb680ff9d0 2024-09-17T00:05:44.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.273+0000 7fcb5b7fe700 1 -- 172.21.15.42:0/2826807337 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fcb6405adb0 con 0x7fcb680ff9d0 2024-09-17T00:05:44.271 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:05:44.272 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:05:44.273 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:05:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcb5406e680 msgr2=0x7fcb54070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcb5406e680 0x7fcb54070b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcb680fbe00 tx=0x7fcb5c00b540 comp rx=0 tx=0).stop 2024-09-17T00:05:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb680ff9d0 msgr2=0x7fcb6819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb680ff9d0 0x7fcb6819e7e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d8d0 tx=0x7fcb6400dbe0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 shutdown_connections 2024-09-17T00:05:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcb5406e680 0x7fcb54070b40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcb680ff9d0 0x7fcb6819e7e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 --2- 172.21.15.42:0/2826807337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb68100380 0x7fcb6819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 >> 172.21.15.42:0/2826807337 conn(0x7fcb680fb650 msgr2=0x7fcb680fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 shutdown_connections 2024-09-17T00:05:44.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.275+0000 7fcb7056d700 1 -- 172.21.15.42:0/2826807337 wait complete. 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/226557003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa440ff990 msgr2=0x7ffa440ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/226557003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa440ff990 0x7ffa440ffd70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009910 tx=0x7ffa40009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/226557003 shutdown_connections 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/226557003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa44100340 0x7ffa4410d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/226557003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa440ff990 0x7ffa440ffd70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/226557003 >> 172.21.15.42:0/226557003 conn(0x7ffa440fb650 msgr2=0x7ffa440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/226557003 shutdown_connections 2024-09-17T00:05:44.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.385+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/226557003 wait complete. 2024-09-17T00:05:44.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4bcd3700 1 Processor -- start 2024-09-17T00:05:44.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4bcd3700 1 -- start start 2024-09-17T00:05:44.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4bcd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa440ff990 0x7ffa4419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4bcd3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa44100340 0x7ffa4419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4bcd3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4419f430 con 0x7ffa44100340 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4bcd3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa441988e0 con 0x7ffa440ff990 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4926e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa44100340 0x7ffa4419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4926e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa44100340 0x7ffa4419eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41984/0 (socket says 172.21.15.42:41984) 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa4926e700 1 -- 172.21.15.42:0/3632045300 learned_addr learned my addr 172.21.15.42:0/3632045300 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.386+0000 7ffa49a6f700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa440ff990 0x7ffa4419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa4926e700 1 -- 172.21.15.42:0/3632045300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa440ff990 msgr2=0x7ffa4419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa4926e700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa440ff990 0x7ffa4419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa4926e700 1 -- 172.21.15.42:0/3632045300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa44100340 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa4926e700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa44100340 0x7ffa4419eda0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400e920 tx=0x7ffa3400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa3affd700 1 -- 172.21.15.42:0/3632045300 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3400c9d0 con 0x7ffa44100340 2024-09-17T00:05:44.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa3affd700 1 -- 172.21.15.42:0/3632045300 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa34010430 con 0x7ffa44100340 2024-09-17T00:05:44.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa44198bc0 con 0x7ffa44100340 2024-09-17T00:05:44.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa3affd700 1 -- 172.21.15.42:0/3632045300 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa34018630 con 0x7ffa44100340 2024-09-17T00:05:44.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.387+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa44199110 con 0x7ffa44100340 2024-09-17T00:05:44.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.388+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4410a770 con 0x7ffa44100340 2024-09-17T00:05:44.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.389+0000 7ffa3affd700 1 -- 172.21.15.42:0/3632045300 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ffa340105a0 con 0x7ffa44100340 2024-09-17T00:05:44.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.389+0000 7ffa3affd700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:44.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.389+0000 7ffa49a6f700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:44.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.389+0000 7ffa3affd700 1 -- 172.21.15.42:0/3632045300 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffa34014070 con 0x7ffa44100340 2024-09-17T00:05:44.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.390+0000 7ffa49a6f700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa40000c00 tx=0x7ffa40009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.394+0000 7ffa3affd700 1 -- 172.21.15.42:0/3632045300 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ffa340541b0 con 0x7ffa44100340 2024-09-17T00:05:44.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:44 smithi042 ceph-mon[111056]: pgmap v424: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:44.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:44 smithi042 ceph-mon[111056]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:44.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:44 smithi042 ceph-mon[111056]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:44.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:44 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2826807337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:05:44.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.649+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffa44199df0 con 0x7ffa44100340 2024-09-17T00:05:44.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.649+0000 7ffa3affd700 1 -- 172.21.15.42:0/3632045300 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7ffa3405b5b0 con 0x7ffa44100340 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:05:44.647 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:05:44.648 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:44.649 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:44.650 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:05:44.650 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:05:44.650 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:44.650 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:44.650 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:44.650 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 msgr2=0x7ffa30070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffa40000c00 tx=0x7ffa40009f70 comp rx=0 tx=0).stop 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa44100340 msgr2=0x7ffa4419eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa44100340 0x7ffa4419eda0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400e920 tx=0x7ffa3400ec30 comp rx=0 tx=0).stop 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 shutdown_connections 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ffa3006e680 0x7ffa30070b40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa440ff990 0x7ffa4419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 --2- 172.21.15.42:0/3632045300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa44100340 0x7ffa4419eda0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 >> 172.21.15.42:0/3632045300 conn(0x7ffa440fb650 msgr2=0x7ffa440fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 shutdown_connections 2024-09-17T00:05:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.651+0000 7ffa4bcd3700 1 -- 172.21.15.42:0/3632045300 wait complete. 2024-09-17T00:05:44.652 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:05:44.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 -- 172.21.15.42:0/62367132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680ff5a0 msgr2=0x7fbf680ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 --2- 172.21.15.42:0/62367132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680ff5a0 0x7fbf680ff980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009910 tx=0x7fbf50009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:44.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 -- 172.21.15.42:0/62367132 shutdown_connections 2024-09-17T00:05:44.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 --2- 172.21.15.42:0/62367132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680fff50 0x7fbf68105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 --2- 172.21.15.42:0/62367132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680ff5a0 0x7fbf680ff980 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 -- 172.21.15.42:0/62367132 >> 172.21.15.42:0/62367132 conn(0x7fbf68074e30 msgr2=0x7fbf68075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 -- 172.21.15.42:0/62367132 shutdown_connections 2024-09-17T00:05:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.765+0000 7fbf6f150700 1 -- 172.21.15.42:0/62367132 wait complete. 2024-09-17T00:05:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf6f150700 1 Processor -- start 2024-09-17T00:05:44.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf6f150700 1 -- start start 2024-09-17T00:05:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf6f150700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680ff5a0 0x7fbf6819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf6f150700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680fff50 0x7fbf6819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf6f150700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6819d250 con 0x7fbf680fff50 2024-09-17T00:05:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf6f150700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf68196700 con 0x7fbf680ff5a0 2024-09-17T00:05:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf67fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680fff50 0x7fbf6819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf67fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680fff50 0x7fbf6819cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42000/0 (socket says 172.21.15.42:42000) 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf67fff700 1 -- 172.21.15.42:0/3741546983 learned_addr learned my addr 172.21.15.42:0/3741546983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf6ceec700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680ff5a0 0x7fbf6819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf67fff700 1 -- 172.21.15.42:0/3741546983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680ff5a0 msgr2=0x7fbf6819c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf67fff700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680ff5a0 0x7fbf6819c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.766+0000 7fbf67fff700 1 -- 172.21.15.42:0/3741546983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500095f0 con 0x7fbf680fff50 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.767+0000 7fbf67fff700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680fff50 0x7fbf6819cbc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf58007e90 tx=0x7fbf5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.767+0000 7fbf65ffb700 1 -- 172.21.15.42:0/3741546983 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf5800dda0 con 0x7fbf680fff50 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.767+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf681969e0 con 0x7fbf680fff50 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.767+0000 7fbf65ffb700 1 -- 172.21.15.42:0/3741546983 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf5800f040 con 0x7fbf680fff50 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.767+0000 7fbf65ffb700 1 -- 172.21.15.42:0/3741546983 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf580125f0 con 0x7fbf680fff50 2024-09-17T00:05:44.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.767+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf68196f30 con 0x7fbf680fff50 2024-09-17T00:05:44.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.768+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf68109270 con 0x7fbf680fff50 2024-09-17T00:05:44.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.769+0000 7fbf65ffb700 1 -- 172.21.15.42:0/3741546983 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fbf580133f0 con 0x7fbf680fff50 2024-09-17T00:05:44.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.769+0000 7fbf65ffb700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbf5406e680 0x7fbf54070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.769+0000 7fbf65ffb700 1 -- 172.21.15.42:0/3741546983 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbf5808df40 con 0x7fbf680fff50 2024-09-17T00:05:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.769+0000 7fbf6ceec700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbf5406e680 0x7fbf54070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:44.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.770+0000 7fbf6ceec700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbf5406e680 0x7fbf54070b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf50006010 tx=0x7fbf5001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:44.774+0000 7fbf65ffb700 1 -- 172.21.15.42:0/3741546983 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbf580c31b0 con 0x7fbf680fff50 2024-09-17T00:05:44.839 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:44 smithi043 ceph-mon[81510]: pgmap v424: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:44.839 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:44 smithi043 ceph-mon[81510]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:44.839 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:44 smithi043 ceph-mon[81510]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:44.839 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:44 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2826807337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:05:45.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.006+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf68197150 con 0x7fbf5406e680 2024-09-17T00:05:45.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.008+0000 7fbf65ffb700 1 -- 172.21.15.42:0/3741546983 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbf68197150 con 0x7fbf5406e680 2024-09-17T00:05:45.005 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:05:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:05:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:05:45.007 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbf5406e680 msgr2=0x7fbf54070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbf5406e680 0x7fbf54070b40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf50006010 tx=0x7fbf5001a040 comp rx=0 tx=0).stop 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680fff50 msgr2=0x7fbf6819cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680fff50 0x7fbf6819cbc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf58007e90 tx=0x7fbf5800d3b0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 shutdown_connections 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbf5406e680 0x7fbf54070b40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680ff5a0 0x7fbf6819c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 --2- 172.21.15.42:0/3741546983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf680fff50 0x7fbf6819cbc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 >> 172.21.15.42:0/3741546983 conn(0x7fbf68074e30 msgr2=0x7fbf680fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 shutdown_connections 2024-09-17T00:05:45.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.010+0000 7fbf6f150700 1 -- 172.21.15.42:0/3741546983 wait complete. 2024-09-17T00:05:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.118+0000 7f6e52af9700 1 -- 172.21.15.42:0/2453893398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 msgr2=0x7f6e4c105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.118+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2453893398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c105840 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009910 tx=0x7f6e38009c20 comp rx=0 tx=0).stop 2024-09-17T00:05:45.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.118+0000 7f6e52af9700 1 -- 172.21.15.42:0/2453893398 shutdown_connections 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.118+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2453893398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c105840 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.118+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2453893398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e4c0ff5a0 0x7f6e4c0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.118+0000 7f6e52af9700 1 -- 172.21.15.42:0/2453893398 >> 172.21.15.42:0/2453893398 conn(0x7f6e4c074e30 msgr2=0x7f6e4c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.118+0000 7f6e52af9700 1 -- 172.21.15.42:0/2453893398 shutdown_connections 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e52af9700 1 -- 172.21.15.42:0/2453893398 wait complete. 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e52af9700 1 Processor -- start 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e52af9700 1 -- start start 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e52af9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e4c0ff5a0 0x7f6e4c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e52af9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e52af9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c19d210 con 0x7f6e4c0fff50 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e52af9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c1966c0 con 0x7f6e4c0ff5a0 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:45.118 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.119+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42016/0 (socket says 172.21.15.42:42016) 2024-09-17T00:05:45.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e4bfff700 1 -- 172.21.15.42:0/2923238370 learned_addr learned my addr 172.21.15.42:0/2923238370 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:05:45.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e50895700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e4c0ff5a0 0x7f6e4c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:45.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e4bfff700 1 -- 172.21.15.42:0/2923238370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e4c0ff5a0 msgr2=0x7f6e4c19c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:45.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e4bfff700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e4c0ff5a0 0x7f6e4c19c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e4bfff700 1 -- 172.21.15.42:0/2923238370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380095f0 con 0x7f6e4c0fff50 2024-09-17T00:05:45.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e4bfff700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c19cb80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e380092e0 tx=0x7f6e3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e49ffb700 1 -- 172.21.15.42:0/2923238370 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3801d070 con 0x7f6e4c0fff50 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e49ffb700 1 -- 172.21.15.42:0/2923238370 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e38005dc0 con 0x7f6e4c0fff50 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c196940 con 0x7f6e4c0fff50 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e49ffb700 1 -- 172.21.15.42:0/2923238370 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e38017400 con 0x7f6e4c0fff50 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.120+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c196e30 con 0x7f6e4c0fff50 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.121+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c109200 con 0x7f6e4c0fff50 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.122+0000 7f6e49ffb700 1 -- 172.21.15.42:0/2923238370 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6e3800f460 con 0x7f6e4c0fff50 2024-09-17T00:05:45.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.122+0000 7f6e49ffb700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e3c06e680 0x7f6e3c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:05:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.123+0000 7f6e49ffb700 1 -- 172.21.15.42:0/2923238370 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6e3808fb10 con 0x7f6e4c0fff50 2024-09-17T00:05:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.123+0000 7f6e50895700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e3c06e680 0x7f6e3c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:05:45.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.123+0000 7f6e50895700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e3c06e680 0x7f6e3c070b40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005950 tx=0x7f6e400058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:05:45.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.127+0000 7f6e49ffb700 1 -- 172.21.15.42:0/2923238370 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6e3805c800 con 0x7f6e4c0fff50 2024-09-17T00:05:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.425+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6e4c04f2e0 con 0x7f6e4c0fff50 2024-09-17T00:05:45.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.425+0000 7f6e49ffb700 1 -- 172.21.15.42:0/2923238370 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f6e3805c390 con 0x7f6e4c0fff50 2024-09-17T00:05:45.423 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:05:45.423 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:05:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:05:45.423 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:05:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:05:45.423 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:05:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e3c06e680 msgr2=0x7f6e3c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e3c06e680 0x7f6e3c070b40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6e40005950 tx=0x7f6e400058e0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 msgr2=0x7f6e4c19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c19cb80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6e380092e0 tx=0x7f6e3800be20 comp rx=0 tx=0).stop 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 shutdown_connections 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e3c06e680 0x7f6e3c070b40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e4c0ff5a0 0x7f6e4c19c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 --2- 172.21.15.42:0/2923238370 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e4c0fff50 0x7f6e4c19cb80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 >> 172.21.15.42:0/2923238370 conn(0x7f6e4c074e30 msgr2=0x7f6e4c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 shutdown_connections 2024-09-17T00:05:45.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:05:45.427+0000 7f6e52af9700 1 -- 172.21.15.42:0/2923238370 wait complete. 2024-09-17T00:05:45.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:45 smithi042 ceph-mon[111056]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:45.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:45 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3632045300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:05:45.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:45 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2923238370' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:05:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:45 smithi043 ceph-mon[81510]: from='client.34832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:45 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3632045300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:05:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:45 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2923238370' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:05:46.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:46 smithi042 ceph-mon[111056]: pgmap v425: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:46.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:46 smithi042 ceph-mon[111056]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:46 smithi043 ceph-mon[81510]: pgmap v425: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:46 smithi043 ceph-mon[81510]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:05:48.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:48 smithi042 ceph-mon[111056]: pgmap v426: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:48 smithi043 ceph-mon[81510]: pgmap v426: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:50 smithi042 ceph-mon[111056]: pgmap v427: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:50 smithi043 ceph-mon[81510]: pgmap v427: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:05:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:52 smithi042 ceph-mon[111056]: pgmap v428: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:52 smithi043 ceph-mon[81510]: pgmap v428: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:54.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:54 smithi042 ceph-mon[111056]: pgmap v429: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:54.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:54 smithi043 ceph-mon[81510]: pgmap v429: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:56.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:56 smithi042 ceph-mon[111056]: pgmap v430: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:56.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:56 smithi043 ceph-mon[81510]: pgmap v430: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:58.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:05:58 smithi042 ceph-mon[111056]: pgmap v431: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:05:58.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:05:58 smithi043 ceph-mon[81510]: pgmap v431: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:00.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:00 smithi043 ceph-mon[81510]: pgmap v432: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:00.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:00 smithi042 ceph-mon[111056]: pgmap v432: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:02.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:02 smithi043 ceph-mon[81510]: pgmap v433: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:03.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:02 smithi042 ceph-mon[111056]: pgmap v433: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:04.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:04 smithi042 ceph-mon[111056]: pgmap v434: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:04.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:04 smithi043 ceph-mon[81510]: pgmap v434: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:06 smithi043 ceph-mon[81510]: pgmap v435: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:06 smithi042 ceph-mon[111056]: pgmap v435: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:07.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:08.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:08 smithi043 ceph-mon[81510]: pgmap v436: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:08 smithi042 ceph-mon[111056]: pgmap v436: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:10.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:10 smithi043 ceph-mon[81510]: pgmap v437: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:10 smithi042 ceph-mon[111056]: pgmap v437: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:12.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:12 smithi043 ceph-mon[81510]: pgmap v438: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:13.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:12 smithi042 ceph-mon[111056]: pgmap v438: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:14.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:14 smithi042 ceph-mon[111056]: pgmap v439: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:14.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:14 smithi043 ceph-mon[81510]: pgmap v439: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 -- 172.21.15.42:0/1180719032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 msgr2=0x7fb2340ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 --2- 172.21.15.42:0/1180719032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb2340ffd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb224009910 tx=0x7fb224009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 -- 172.21.15.42:0/1180719032 shutdown_connections 2024-09-17T00:06:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 --2- 172.21.15.42:0/1180719032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb234100320 0x7fb23410cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 --2- 172.21.15.42:0/1180719032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb2340ffd50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 -- 172.21.15.42:0/1180719032 >> 172.21.15.42:0/1180719032 conn(0x7fb2340fb650 msgr2=0x7fb2340fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 -- 172.21.15.42:0/1180719032 shutdown_connections 2024-09-17T00:06:15.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.550+0000 7fb238b13700 1 -- 172.21.15.42:0/1180719032 wait complete. 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb238b13700 1 Processor -- start 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb238b13700 1 -- start start 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb238b13700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb23419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb238b13700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb234100320 0x7fb23419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb238b13700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23419f440 con 0x7fb2340ff970 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb238b13700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2341988f0 con 0x7fb234100320 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb23259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb23419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb23259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb23419e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57058/0 (socket says 172.21.15.42:57058) 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb23259c700 1 -- 172.21.15.42:0/3952577716 learned_addr learned my addr 172.21.15.42:0/3952577716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.551+0000 7fb231d9b700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb234100320 0x7fb23419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb23259c700 1 -- 172.21.15.42:0/3952577716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb234100320 msgr2=0x7fb23419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:15.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb23259c700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb234100320 0x7fb23419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb23259c700 1 -- 172.21.15.42:0/3952577716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240095f0 con 0x7fb2340ff970 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb231d9b700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb234100320 0x7fb23419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb23259c700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb23419e870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb2240092e0 tx=0x7fb2240056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb22b7fe700 1 -- 172.21.15.42:0/3952577716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb22401d070 con 0x7fb2340ff970 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb234198b70 con 0x7fb2340ff970 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb22b7fe700 1 -- 172.21.15.42:0/3952577716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb22400f460 con 0x7fb2340ff970 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2341990c0 con 0x7fb2340ff970 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.552+0000 7fb22b7fe700 1 -- 172.21.15.42:0/3952577716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb22402b610 con 0x7fb2340ff970 2024-09-17T00:06:15.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.553+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23410a750 con 0x7fb2340ff970 2024-09-17T00:06:15.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.555+0000 7fb22b7fe700 1 -- 172.21.15.42:0/3952577716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb22400f5d0 con 0x7fb2340ff970 2024-09-17T00:06:15.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.555+0000 7fb22b7fe700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb22006e680 0x7fb220070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:15.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.555+0000 7fb22b7fe700 1 -- 172.21.15.42:0/3952577716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb22408f7e0 con 0x7fb2340ff970 2024-09-17T00:06:15.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.555+0000 7fb231d9b700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb22006e680 0x7fb220070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:15.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.556+0000 7fb231d9b700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb22006e680 0x7fb220070b40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb21c005950 tx=0x7fb21c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:15.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.561+0000 7fb22b7fe700 1 -- 172.21.15.42:0/3952577716 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb22405c560 con 0x7fb2340ff970 2024-09-17T00:06:15.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.791+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2341993f0 con 0x7fb22006e680 2024-09-17T00:06:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.792+0000 7fb22b7fe700 1 -- 172.21.15.42:0/3952577716 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb2341993f0 con 0x7fb22006e680 2024-09-17T00:06:15.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb22006e680 msgr2=0x7fb220070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb22006e680 0x7fb220070b40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb21c005950 tx=0x7fb21c0058e0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 msgr2=0x7fb23419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb23419e870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb2240092e0 tx=0x7fb2240056a0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 shutdown_connections 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb22006e680 0x7fb220070b40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2340ff970 0x7fb23419e870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 --2- 172.21.15.42:0/3952577716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb234100320 0x7fb23419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 >> 172.21.15.42:0/3952577716 conn(0x7fb2340fb650 msgr2=0x7fb2340fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:15.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 shutdown_connections 2024-09-17T00:06:15.794 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.794+0000 7fb238b13700 1 -- 172.21.15.42:0/3952577716 wait complete. 2024-09-17T00:06:15.803 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:06:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3801058948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484100ea0 msgr2=0x7fc484101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3801058948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484100ea0 0x7fc484101280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc47400b0a0 tx=0x7fc47400b3b0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3801058948 shutdown_connections 2024-09-17T00:06:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3801058948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc484101850 0x7fc4841058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3801058948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484100ea0 0x7fc484101280 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3801058948 >> 172.21.15.42:0/3801058948 conn(0x7fc4840fc750 msgr2=0x7fc4840feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:15.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3801058948 shutdown_connections 2024-09-17T00:06:15.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.903+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3801058948 wait complete. 2024-09-17T00:06:15.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc48c1a7700 1 Processor -- start 2024-09-17T00:06:15.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc48c1a7700 1 -- start start 2024-09-17T00:06:15.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc48c1a7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc484100ea0 0x7fc48419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:15.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc48c1a7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484101850 0x7fc48419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:15.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc48c1a7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48419d2c0 con 0x7fc484101850 2024-09-17T00:06:15.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc48c1a7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484196770 con 0x7fc484100ea0 2024-09-17T00:06:15.906 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc489742700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484101850 0x7fc48419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc489742700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484101850 0x7fc48419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57070/0 (socket says 172.21.15.42:57070) 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc489742700 1 -- 172.21.15.42:0/3906354539 learned_addr learned my addr 172.21.15.42:0/3906354539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc489f43700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc484100ea0 0x7fc48419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc489742700 1 -- 172.21.15.42:0/3906354539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc484100ea0 msgr2=0x7fc48419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc489742700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc484100ea0 0x7fc48419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.904+0000 7fc489742700 1 -- 172.21.15.42:0/3906354539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc474009d00 con 0x7fc484101850 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.905+0000 7fc489742700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484101850 0x7fc48419cc30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc47800d900 tx=0x7fc47800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.905+0000 7fc482ffd700 1 -- 172.21.15.42:0/3906354539 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc478009900 con 0x7fc484101850 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.905+0000 7fc482ffd700 1 -- 172.21.15.42:0/3906354539 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc478010460 con 0x7fc484101850 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.905+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc484196a50 con 0x7fc484101850 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.905+0000 7fc482ffd700 1 -- 172.21.15.42:0/3906354539 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc47800f5f0 con 0x7fc484101850 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.905+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc484196fa0 con 0x7fc484101850 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.906+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc484109240 con 0x7fc484101850 2024-09-17T00:06:15.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.906+0000 7fc482ffd700 1 -- 172.21.15.42:0/3906354539 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc4780105d0 con 0x7fc484101850 2024-09-17T00:06:15.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.907+0000 7fc482ffd700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc46c06e680 0x7fc46c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:15.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.907+0000 7fc482ffd700 1 -- 172.21.15.42:0/3906354539 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc47808e240 con 0x7fc484101850 2024-09-17T00:06:15.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.907+0000 7fc489f43700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc46c06e680 0x7fc46c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:15.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.907+0000 7fc489f43700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc46c06e680 0x7fc46c070b40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc474000f80 tx=0x7fc4740060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:15.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:15.911+0000 7fc482ffd700 1 -- 172.21.15.42:0/3906354539 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc47805af10 con 0x7fc484101850 2024-09-17T00:06:16.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.145+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc484197c80 con 0x7fc46c06e680 2024-09-17T00:06:16.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.146+0000 7fc482ffd700 1 -- 172.21.15.42:0/3906354539 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc484197c80 con 0x7fc46c06e680 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc46c06e680 msgr2=0x7fc46c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc46c06e680 0x7fc46c070b40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc474000f80 tx=0x7fc4740060c0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484101850 msgr2=0x7fc48419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484101850 0x7fc48419cc30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc47800d900 tx=0x7fc47800dc10 comp rx=0 tx=0).stop 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 shutdown_connections 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc46c06e680 0x7fc46c070b40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc484100ea0 0x7fc48419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 --2- 172.21.15.42:0/3906354539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc484101850 0x7fc48419cc30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 >> 172.21.15.42:0/3906354539 conn(0x7fc4840fc750 msgr2=0x7fc4840fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:16.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 shutdown_connections 2024-09-17T00:06:16.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.148+0000 7fc48c1a7700 1 -- 172.21.15.42:0/3906354539 wait complete. 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.255+0000 7fa659617700 1 -- 172.21.15.42:0/4118545815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 msgr2=0x7fa6541058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.255+0000 7fa659617700 1 --2- 172.21.15.42:0/4118545815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa6541058a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa648009910 tx=0x7fa648009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 -- 172.21.15.42:0/4118545815 shutdown_connections 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 --2- 172.21.15.42:0/4118545815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa6541058a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 --2- 172.21.15.42:0/4118545815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa654100ea0 0x7fa654101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 -- 172.21.15.42:0/4118545815 >> 172.21.15.42:0/4118545815 conn(0x7fa6540fc750 msgr2=0x7fa6540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 -- 172.21.15.42:0/4118545815 shutdown_connections 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 -- 172.21.15.42:0/4118545815 wait complete. 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 Processor -- start 2024-09-17T00:06:16.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.256+0000 7fa659617700 1 -- start start 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa659617700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa654100ea0 0x7fa65419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa659617700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa65419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa659617700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65419f4a0 con 0x7fa654101850 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa659617700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa654198950 con 0x7fa654100ea0 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa6527fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa65419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa6527fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa65419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57080/0 (socket says 172.21.15.42:57080) 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa6527fc700 1 -- 172.21.15.42:0/3197012976 learned_addr learned my addr 172.21.15.42:0/3197012976 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa652ffd700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa654100ea0 0x7fa65419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa6527fc700 1 -- 172.21.15.42:0/3197012976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa654100ea0 msgr2=0x7fa65419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa6527fc700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa654100ea0 0x7fa65419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa6527fc700 1 -- 172.21.15.42:0/3197012976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6480095f0 con 0x7fa654101850 2024-09-17T00:06:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa6527fc700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa65419ee10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa64800bd70 tx=0x7fa64800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:16.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.257+0000 7fa63bfff700 1 -- 172.21.15.42:0/3197012976 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa64801d070 con 0x7fa654101850 2024-09-17T00:06:16.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.258+0000 7fa63bfff700 1 -- 172.21.15.42:0/3197012976 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa648005dc0 con 0x7fa654101850 2024-09-17T00:06:16.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.258+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa654198bd0 con 0x7fa654101850 2024-09-17T00:06:16.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.258+0000 7fa63bfff700 1 -- 172.21.15.42:0/3197012976 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa648017400 con 0x7fa654101850 2024-09-17T00:06:16.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.258+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6541990c0 con 0x7fa654101850 2024-09-17T00:06:16.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.259+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa654105020 con 0x7fa654101850 2024-09-17T00:06:16.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.259+0000 7fa63bfff700 1 -- 172.21.15.42:0/3197012976 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa648003a60 con 0x7fa654101850 2024-09-17T00:06:16.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.259+0000 7fa63bfff700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa63c06e680 0x7fa63c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:16.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.260+0000 7fa652ffd700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa63c06e680 0x7fa63c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:16.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.260+0000 7fa63bfff700 1 -- 172.21.15.42:0/3197012976 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa64808fb20 con 0x7fa654101850 2024-09-17T00:06:16.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.260+0000 7fa652ffd700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa63c06e680 0x7fa63c070b40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa644005950 tx=0x7fa64400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.264+0000 7fa63bfff700 1 -- 172.21.15.42:0/3197012976 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa64805c7f0 con 0x7fa654101850 2024-09-17T00:06:16.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.482+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa654199ed0 con 0x7fa63c06e680 2024-09-17T00:06:16.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.492+0000 7fa63bfff700 1 -- 172.21.15.42:0/3197012976 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fa654199ed0 con 0x7fa63c06e680 2024-09-17T00:06:16.491 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:06:16.491 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (14m) 8m ago 25m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:06:16.491 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (25m) 8m ago 25m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:06:16.492 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (24m) 8m ago 24m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:06:16.492 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (12m) 8m ago 25m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:06:16.492 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (12m) 8m ago 24m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:06:16.492 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (14m) 8m ago 25m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:06:16.492 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (8m) 8m ago 19m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:06:16.492 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (8m) 8m ago 20m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:06:16.493 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (8m) 8m ago 20m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:06:16.493 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (8m) 8m ago 19m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:06:16.493 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (16m) 8m ago 27m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:06:16.493 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (15m) 8m ago 24m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:06:16.493 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (13m) 8m ago 27m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:06:16.493 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (13m) 8m ago 24m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:06:16.493 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (15m) 8m ago 25m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:06:16.494 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (15m) 8m ago 24m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:06:16.494 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (12m) 8m ago 23m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:06:16.495 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (11m) 8m ago 23m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:06:16.495 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (10m) 8m ago 22m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:06:16.495 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (10m) 8m ago 22m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:06:16.495 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (9m) 8m ago 22m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:06:16.495 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (9m) 8m ago 21m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:06:16.495 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (15m) 8m ago 25m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:06:16.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa63c06e680 msgr2=0x7fa63c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa63c06e680 0x7fa63c070b40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa644005950 tx=0x7fa64400b410 comp rx=0 tx=0).stop 2024-09-17T00:06:16.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 msgr2=0x7fa65419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa65419ee10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa64800bd70 tx=0x7fa64800bda0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 shutdown_connections 2024-09-17T00:06:16.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa63c06e680 0x7fa63c070b40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa654100ea0 0x7fa65419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 --2- 172.21.15.42:0/3197012976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa654101850 0x7fa65419ee10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.494+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 >> 172.21.15.42:0/3197012976 conn(0x7fa6540fc750 msgr2=0x7fa6540feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.495+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 shutdown_connections 2024-09-17T00:06:16.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.495+0000 7fa659617700 1 -- 172.21.15.42:0/3197012976 wait complete. 2024-09-17T00:06:16.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/543588617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 msgr2=0x7fe0e41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/543588617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e41047a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009910 tx=0x7fe0d8009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:16.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/543588617 shutdown_connections 2024-09-17T00:06:16.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/543588617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e41047a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/543588617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e40ffda0 0x7fe0e4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/543588617 >> 172.21.15.42:0/543588617 conn(0x7fe0e40fb650 msgr2=0x7fe0e40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:16.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/543588617 shutdown_connections 2024-09-17T00:06:16.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/543588617 wait complete. 2024-09-17T00:06:16.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.602+0000 7fe0eaa0c700 1 Processor -- start 2024-09-17T00:06:16.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0eaa0c700 1 -- start start 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0eaa0c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e40ffda0 0x7fe0e4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0eaa0c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0eaa0c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e41022c0 con 0x7fe0e4100750 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0eaa0c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e4102400 con 0x7fe0e40ffda0 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0e37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0e37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e4101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57110/0 (socket says 172.21.15.42:57110) 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0e37fe700 1 -- 172.21.15.42:0/1622977271 learned_addr learned my addr 172.21.15.42:0/1622977271 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0e3fff700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e40ffda0 0x7fe0e4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0e37fe700 1 -- 172.21.15.42:0/1622977271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e40ffda0 msgr2=0x7fe0e4101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0e37fe700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e40ffda0 0x7fe0e4101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.603+0000 7fe0e37fe700 1 -- 172.21.15.42:0/1622977271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d80095f0 con 0x7fe0e4100750 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.604+0000 7fe0e37fe700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e4101d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80092e0 tx=0x7fe0d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:16.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.604+0000 7fe0e17fa700 1 -- 172.21.15.42:0/1622977271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d801d070 con 0x7fe0e4100750 2024-09-17T00:06:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.604+0000 7fe0e17fa700 1 -- 172.21.15.42:0/1622977271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0d800f460 con 0x7fe0e4100750 2024-09-17T00:06:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.604+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e4102680 con 0x7fe0e4100750 2024-09-17T00:06:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.604+0000 7fe0e17fa700 1 -- 172.21.15.42:0/1622977271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d80175d0 con 0x7fe0e4100750 2024-09-17T00:06:16.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.604+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e41a2da0 con 0x7fe0e4100750 2024-09-17T00:06:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.605+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e41091b0 con 0x7fe0e4100750 2024-09-17T00:06:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.605+0000 7fe0e17fa700 1 -- 172.21.15.42:0/1622977271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe0d800f5d0 con 0x7fe0e4100750 2024-09-17T00:06:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.606+0000 7fe0e17fa700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0cc06e680 0x7fe0cc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.606+0000 7fe0e17fa700 1 -- 172.21.15.42:0/1622977271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe0d808fad0 con 0x7fe0e4100750 2024-09-17T00:06:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.606+0000 7fe0e3fff700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0cc06e680 0x7fe0cc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:16.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.606+0000 7fe0e3fff700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0cc06e680 0x7fe0cc070b40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005950 tx=0x7fe0d400a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:16.608 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.610+0000 7fe0e17fa700 1 -- 172.21.15.42:0/1622977271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe0d8058c70 con 0x7fe0e4100750 2024-09-17T00:06:16.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:16 smithi042 ceph-mon[111056]: pgmap v440: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:16 smithi043 ceph-mon[81510]: pgmap v440: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:16.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.906+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe0e404f2e0 con 0x7fe0e4100750 2024-09-17T00:06:16.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.907+0000 7fe0e17fa700 1 -- 172.21.15.42:0/1622977271 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe0d805c330 con 0x7fe0e4100750 2024-09-17T00:06:16.905 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:06:16.905 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:06:16.906 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:06:16.907 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0cc06e680 msgr2=0x7fe0cc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0cc06e680 0x7fe0cc070b40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4005950 tx=0x7fe0d400a300 comp rx=0 tx=0).stop 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 msgr2=0x7fe0e4101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e4101d80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80092e0 tx=0x7fe0d800bd70 comp rx=0 tx=0).stop 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 shutdown_connections 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe0cc06e680 0x7fe0cc070b40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e40ffda0 0x7fe0e4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 --2- 172.21.15.42:0/1622977271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0e4100750 0x7fe0e4101d80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 >> 172.21.15.42:0/1622977271 conn(0x7fe0e40fb650 msgr2=0x7fe0e4069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 shutdown_connections 2024-09-17T00:06:16.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:16.909+0000 7fe0eaa0c700 1 -- 172.21.15.42:0/1622977271 wait complete. 2024-09-17T00:06:17.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 -- 172.21.15.42:0/352188654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8105bd0 msgr2=0x7f12a8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 --2- 172.21.15.42:0/352188654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8105bd0 0x7f12a8105fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1298009910 tx=0x7f1298009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:17.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 -- 172.21.15.42:0/352188654 shutdown_connections 2024-09-17T00:06:17.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 --2- 172.21.15.42:0/352188654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a80ffc20 0x7f12a8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 --2- 172.21.15.42:0/352188654 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8105bd0 0x7f12a8105fb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 -- 172.21.15.42:0/352188654 >> 172.21.15.42:0/352188654 conn(0x7f12a8075670 msgr2=0x7f12a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:17.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 -- 172.21.15.42:0/352188654 shutdown_connections 2024-09-17T00:06:17.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.016+0000 7f12ac863700 1 -- 172.21.15.42:0/352188654 wait complete. 2024-09-17T00:06:17.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12ac863700 1 Processor -- start 2024-09-17T00:06:17.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12ac863700 1 -- start start 2024-09-17T00:06:17.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12ac863700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80ffc20 0x7f12a819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12ac863700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8105bd0 0x7f12a819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12ac863700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8195590 con 0x7f12a80ffc20 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12ac863700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8195700 con 0x7f12a8105bd0 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12a659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80ffc20 0x7f12a819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12a659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80ffc20 0x7f12a819b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57134/0 (socket says 172.21.15.42:57134) 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12a659c700 1 -- 172.21.15.42:0/1338955532 learned_addr learned my addr 172.21.15.42:0/1338955532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.017+0000 7f12a5d9b700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8105bd0 0x7f12a819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f12a659c700 1 -- 172.21.15.42:0/1338955532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8105bd0 msgr2=0x7f12a819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f12a659c700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8105bd0 0x7f12a819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f12a659c700 1 -- 172.21.15.42:0/1338955532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980095f0 con 0x7f12a80ffc20 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f12a659c700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80ffc20 0x7f12a819b5a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1298006010 tx=0x7f129800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:17.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f129f7fe700 1 -- 172.21.15.42:0/1338955532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129801d070 con 0x7f12a80ffc20 2024-09-17T00:06:17.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a8195980 con 0x7f12a80ffc20 2024-09-17T00:06:17.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f129f7fe700 1 -- 172.21.15.42:0/1338955532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1298005dc0 con 0x7f12a80ffc20 2024-09-17T00:06:17.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f129f7fe700 1 -- 172.21.15.42:0/1338955532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1298017400 con 0x7f12a80ffc20 2024-09-17T00:06:17.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.018+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a8195e70 con 0x7f12a80ffc20 2024-09-17T00:06:17.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.019+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a81080f0 con 0x7f12a80ffc20 2024-09-17T00:06:17.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.020+0000 7f129f7fe700 1 -- 172.21.15.42:0/1338955532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1298003a60 con 0x7f12a80ffc20 2024-09-17T00:06:17.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.020+0000 7f129f7fe700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129406e680 0x7f1294070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.020+0000 7f129f7fe700 1 -- 172.21.15.42:0/1338955532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f129808fad0 con 0x7f12a80ffc20 2024-09-17T00:06:17.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.020+0000 7f12a5d9b700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129406e680 0x7f1294070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.020+0000 7f12a5d9b700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129406e680 0x7f1294070b40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12a8196e30 tx=0x7f129000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:17.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.024+0000 7f129f7fe700 1 -- 172.21.15.42:0/1338955532 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1298058c70 con 0x7f12a80ffc20 2024-09-17T00:06:17.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.275+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f12a8196ac0 con 0x7f12a80ffc20 2024-09-17T00:06:17.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.276+0000 7f129f7fe700 1 -- 172.21.15.42:0/1338955532 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f129805c330 con 0x7f12a80ffc20 2024-09-17T00:06:17.274 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:06:17.275 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:06:17.276 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:17.277 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:17.278 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:17.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129406e680 msgr2=0x7f1294070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129406e680 0x7f1294070b40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12a8196e30 tx=0x7f129000b410 comp rx=0 tx=0).stop 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80ffc20 msgr2=0x7f12a819b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80ffc20 0x7f12a819b5a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1298006010 tx=0x7f129800bd70 comp rx=0 tx=0).stop 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 shutdown_connections 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129406e680 0x7f1294070b40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a80ffc20 0x7f12a819b5a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 --2- 172.21.15.42:0/1338955532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8105bd0 0x7f12a819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 >> 172.21.15.42:0/1338955532 conn(0x7f12a8075670 msgr2=0x7f12a80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 shutdown_connections 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.278+0000 7f12ac863700 1 -- 172.21.15.42:0/1338955532 wait complete. 2024-09-17T00:06:17.279 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:06:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 -- 172.21.15.42:0/3275591128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 msgr2=0x7fadd8102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 --2- 172.21.15.42:0/3275591128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd8102240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009940 tx=0x7fadc8009c50 comp rx=0 tx=0).stop 2024-09-17T00:06:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 -- 172.21.15.42:0/3275591128 shutdown_connections 2024-09-17T00:06:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 --2- 172.21.15.42:0/3275591128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8102780 0x7fadd810ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 --2- 172.21.15.42:0/3275591128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd8102240 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 -- 172.21.15.42:0/3275591128 >> 172.21.15.42:0/3275591128 conn(0x7fadd80fb610 msgr2=0x7fadd80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:17.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 -- 172.21.15.42:0/3275591128 shutdown_connections 2024-09-17T00:06:17.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.386+0000 7faddf0cf700 1 -- 172.21.15.42:0/3275591128 wait complete. 2024-09-17T00:06:17.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddf0cf700 1 Processor -- start 2024-09-17T00:06:17.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddf0cf700 1 -- start start 2024-09-17T00:06:17.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddf0cf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddf0cf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8102780 0x7fadd819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddf0cf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd819d240 con 0x7fadd8101e60 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddf0cf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd81966f0 con 0x7fadd8102780 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddce6b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddce6b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd819c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57166/0 (socket says 172.21.15.42:57166) 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddce6b700 1 -- 172.21.15.42:0/635508213 learned_addr learned my addr 172.21.15.42:0/635508213 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7fadd7fff700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8102780 0x7fadd819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddce6b700 1 -- 172.21.15.42:0/635508213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8102780 msgr2=0x7fadd819cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddce6b700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8102780 0x7fadd819cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.387+0000 7faddce6b700 1 -- 172.21.15.42:0/635508213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80095f0 con 0x7fadd8101e60 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.388+0000 7faddce6b700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd819c670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fadc80055f0 tx=0x7fadc8005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:17.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.388+0000 7fadd5ffb700 1 -- 172.21.15.42:0/635508213 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc801d070 con 0x7fadd8101e60 2024-09-17T00:06:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.388+0000 7fadd5ffb700 1 -- 172.21.15.42:0/635508213 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc800f460 con 0x7fadd8101e60 2024-09-17T00:06:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.388+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd8196970 con 0x7fadd8101e60 2024-09-17T00:06:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.388+0000 7fadd5ffb700 1 -- 172.21.15.42:0/635508213 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc8021620 con 0x7fadd8101e60 2024-09-17T00:06:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.388+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd8196e60 con 0x7fadd8101e60 2024-09-17T00:06:17.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.389+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd8108370 con 0x7fadd8101e60 2024-09-17T00:06:17.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.389+0000 7fadd5ffb700 1 -- 172.21.15.42:0/635508213 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fadc800fad0 con 0x7fadd8101e60 2024-09-17T00:06:17.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.390+0000 7fadd5ffb700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadc006e680 0x7fadc0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.390+0000 7fadd5ffb700 1 -- 172.21.15.42:0/635508213 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fadc808f7f0 con 0x7fadd8101e60 2024-09-17T00:06:17.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.390+0000 7fadd7fff700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadc006e680 0x7fadc0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.391+0000 7fadd7fff700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadc006e680 0x7fadc0070b40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fadd8197b60 tx=0x7fadcc00a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:17.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.395+0000 7fadd5ffb700 1 -- 172.21.15.42:0/635508213 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fadc805c540 con 0x7fadd8101e60 2024-09-17T00:06:17.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.626+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadd8197190 con 0x7fadc006e680 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.627+0000 7fadd5ffb700 1 -- 172.21.15.42:0/635508213 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fadd8197190 con 0x7fadc006e680 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:06:17.626 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:06:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:06:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:06:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:06:17.627 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:06:17.627 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:06:17.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadc006e680 msgr2=0x7fadc0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadc006e680 0x7fadc0070b40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fadd8197b60 tx=0x7fadcc00a300 comp rx=0 tx=0).stop 2024-09-17T00:06:17.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 msgr2=0x7fadd819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd819c670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fadc80055f0 tx=0x7fadc8005620 comp rx=0 tx=0).stop 2024-09-17T00:06:17.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 shutdown_connections 2024-09-17T00:06:17.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fadc006e680 0x7fadc0070b40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadd8101e60 0x7fadd819c670 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 --2- 172.21.15.42:0/635508213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8102780 0x7fadd819cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.629+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 >> 172.21.15.42:0/635508213 conn(0x7fadd80fb610 msgr2=0x7fadd81054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:17.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.630+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 shutdown_connections 2024-09-17T00:06:17.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.630+0000 7faddf0cf700 1 -- 172.21.15.42:0/635508213 wait complete. 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 -- 172.21.15.42:0/451785628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 msgr2=0x7faecc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 --2- 172.21.15.42:0/451785628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc105fd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faebc009910 tx=0x7faebc009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 -- 172.21.15.42:0/451785628 shutdown_connections 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 --2- 172.21.15.42:0/451785628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faecc0ffb40 0x7faecc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 --2- 172.21.15.42:0/451785628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc105fd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 -- 172.21.15.42:0/451785628 >> 172.21.15.42:0/451785628 conn(0x7faecc0fb610 msgr2=0x7faecc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 -- 172.21.15.42:0/451785628 shutdown_connections 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 -- 172.21.15.42:0/451785628 wait complete. 2024-09-17T00:06:17.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.738+0000 7faed1ae7700 1 Processor -- start 2024-09-17T00:06:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faed1ae7700 1 -- start start 2024-09-17T00:06:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faed1ae7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faecc0ffb40 0x7faecc19c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faed1ae7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faed1ae7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc19d1b0 con 0x7faecc105bf0 2024-09-17T00:06:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faed1ae7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc196660 con 0x7faecc0ffb40 2024-09-17T00:06:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57178/0 (socket says 172.21.15.42:57178) 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecaffd700 1 -- 172.21.15.42:0/922453579 learned_addr learned my addr 172.21.15.42:0/922453579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecb7fe700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faecc0ffb40 0x7faecc19c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecaffd700 1 -- 172.21.15.42:0/922453579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faecc0ffb40 msgr2=0x7faecc19c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecaffd700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faecc0ffb40 0x7faecc19c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecaffd700 1 -- 172.21.15.42:0/922453579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc0095f0 con 0x7faecc105bf0 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.739+0000 7faecaffd700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc19cb20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faeb400d8d0 tx=0x7faeb400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.740+0000 7faec8ff9700 1 -- 172.21.15.42:0/922453579 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeb40098a0 con 0x7faecc105bf0 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.740+0000 7faec8ff9700 1 -- 172.21.15.42:0/922453579 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faeb4010460 con 0x7faecc105bf0 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.740+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc196940 con 0x7faecc105bf0 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.740+0000 7faec8ff9700 1 -- 172.21.15.42:0/922453579 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeb400f5f0 con 0x7faecc105bf0 2024-09-17T00:06:17.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.740+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc196e90 con 0x7faecc105bf0 2024-09-17T00:06:17.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.741+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc04f2e0 con 0x7faecc105bf0 2024-09-17T00:06:17.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.741+0000 7faec8ff9700 1 -- 172.21.15.42:0/922453579 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7faeb4009a00 con 0x7faecc105bf0 2024-09-17T00:06:17.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.742+0000 7faec8ff9700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faeb806e680 0x7faeb8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:17.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.742+0000 7faec8ff9700 1 -- 172.21.15.42:0/922453579 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faeb408e450 con 0x7faecc105bf0 2024-09-17T00:06:17.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.742+0000 7faecb7fe700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faeb806e680 0x7faeb8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:17.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.742+0000 7faecb7fe700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faeb806e680 0x7faeb8070b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b5c0 tx=0x7faebc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:17.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:17.746+0000 7faec8ff9700 1 -- 172.21.15.42:0/922453579 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faeb405b1a0 con 0x7faecc105bf0 2024-09-17T00:06:17.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:17 smithi043 ceph-mon[81510]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:17.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:17 smithi043 ceph-mon[81510]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:17.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:17 smithi043 ceph-mon[81510]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:17.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:17 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1622977271' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:06:17.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:17 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1338955532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:06:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:17 smithi042 ceph-mon[111056]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:17 smithi042 ceph-mon[111056]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:17 smithi042 ceph-mon[111056]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:17 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1622977271' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:06:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:17 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1338955532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:06:18.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.047+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faecc197b70 con 0x7faecc105bf0 2024-09-17T00:06:18.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.047+0000 7faec8ff9700 1 -- 172.21.15.42:0/922453579 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7faeb405ad30 con 0x7faecc105bf0 2024-09-17T00:06:18.046 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:06:18.047 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:06:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:06:18.047 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:06:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:06:18.047 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:06:18.047 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:06:18.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.049+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faeb806e680 msgr2=0x7faeb8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.049+0000 7faed1ae7700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faeb806e680 0x7faeb8070b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b5c0 tx=0x7faebc01a040 comp rx=0 tx=0).stop 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.049+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 msgr2=0x7faecc19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.049+0000 7faed1ae7700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc19cb20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faeb400d8d0 tx=0x7faeb400dbe0 comp rx=0 tx=0).stop 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.049+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 shutdown_connections 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.049+0000 7faed1ae7700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faeb806e680 0x7faeb8070b40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.050+0000 7faed1ae7700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faecc0ffb40 0x7faecc19c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.050+0000 7faed1ae7700 1 --2- 172.21.15.42:0/922453579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faecc105bf0 0x7faecc19cb20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.050+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 >> 172.21.15.42:0/922453579 conn(0x7faecc0fb610 msgr2=0x7faecc068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.050+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 shutdown_connections 2024-09-17T00:06:18.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:18.050+0000 7faed1ae7700 1 -- 172.21.15.42:0/922453579 wait complete. 2024-09-17T00:06:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:18 smithi043 ceph-mon[81510]: pgmap v441: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:18 smithi043 ceph-mon[81510]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:18 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/922453579' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:06:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:18 smithi042 ceph-mon[111056]: pgmap v441: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:18 smithi042 ceph-mon[111056]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:18 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/922453579' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:06:20.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:20 smithi043 ceph-mon[81510]: pgmap v442: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:21.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:20 smithi042 ceph-mon[111056]: pgmap v442: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:22.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:22 smithi043 ceph-mon[81510]: pgmap v443: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:23.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:22 smithi042 ceph-mon[111056]: pgmap v443: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:24.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:24 smithi043 ceph-mon[81510]: pgmap v444: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:25.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:24 smithi042 ceph-mon[111056]: pgmap v444: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:26.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:26 smithi043 ceph-mon[81510]: pgmap v445: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:27.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:26 smithi042 ceph-mon[111056]: pgmap v445: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:28.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:28 smithi043 ceph-mon[81510]: pgmap v446: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:29.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:28 smithi042 ceph-mon[111056]: pgmap v446: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:30.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:30 smithi043 ceph-mon[81510]: pgmap v447: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:31.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:30 smithi042 ceph-mon[111056]: pgmap v447: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:32.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:32 smithi043 ceph-mon[81510]: pgmap v448: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:33.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:32 smithi042 ceph-mon[111056]: pgmap v448: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:34.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:34 smithi043 ceph-mon[81510]: pgmap v449: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:35.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:34 smithi042 ceph-mon[111056]: pgmap v449: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:36 smithi043 ceph-mon[81510]: pgmap v450: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:36 smithi042 ceph-mon[111056]: pgmap v450: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:38.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:38 smithi043 ceph-mon[81510]: pgmap v451: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:39.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:38 smithi042 ceph-mon[111056]: pgmap v451: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:40.680 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:40 smithi043 ceph-mon[81510]: pgmap v452: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:41.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:40 smithi042 ceph-mon[111056]: pgmap v452: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:42.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:42 smithi043 ceph-mon[81510]: pgmap v453: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:43.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:42 smithi042 ceph-mon[111056]: pgmap v453: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:44.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:44 smithi043 ceph-mon[81510]: pgmap v454: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:45.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:44 smithi042 ceph-mon[111056]: pgmap v454: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:46.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:46 smithi042 ceph-mon[111056]: pgmap v455: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:46 smithi043 ceph-mon[81510]: pgmap v455: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:48.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 -- 172.21.15.42:0/3365524809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48068580 msgr2=0x7f8a480689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 --2- 172.21.15.42:0/3365524809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48068580 0x7f8a480689e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009910 tx=0x7f8a30009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 -- 172.21.15.42:0/3365524809 shutdown_connections 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 --2- 172.21.15.42:0/3365524809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48068580 0x7f8a480689e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 --2- 172.21.15.42:0/3365524809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a48106cd0 0x7f8a481070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 -- 172.21.15.42:0/3365524809 >> 172.21.15.42:0/3365524809 conn(0x7f8a48075670 msgr2=0x7f8a48075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 -- 172.21.15.42:0/3365524809 shutdown_connections 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 -- 172.21.15.42:0/3365524809 wait complete. 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 Processor -- start 2024-09-17T00:06:48.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.174+0000 7f8a4e217700 1 -- start start 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a4e217700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a48068580 0x7f8a4819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a4e217700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48106cd0 0x7f8a4819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a4e217700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819d200 con 0x7f8a48106cd0 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a4e217700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481966b0 con 0x7f8a48068580 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a3ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48106cd0 0x7f8a4819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a3ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48106cd0 0x7f8a4819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50792/0 (socket says 172.21.15.42:50792) 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a3ffff700 1 -- 172.21.15.42:0/843975454 learned_addr learned my addr 172.21.15.42:0/843975454 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a477fe700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a48068580 0x7f8a4819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a3ffff700 1 -- 172.21.15.42:0/843975454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a48068580 msgr2=0x7f8a4819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a3ffff700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a48068580 0x7f8a4819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a3ffff700 1 -- 172.21.15.42:0/843975454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a300095f0 con 0x7f8a48106cd0 2024-09-17T00:06:48.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.175+0000 7f8a3ffff700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48106cd0 0x7f8a4819cb70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005f50 tx=0x7f8a3000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:48.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.176+0000 7f8a457fa700 1 -- 172.21.15.42:0/843975454 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3001d070 con 0x7f8a48106cd0 2024-09-17T00:06:48.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.176+0000 7f8a457fa700 1 -- 172.21.15.42:0/843975454 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a3000b610 con 0x7f8a48106cd0 2024-09-17T00:06:48.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.176+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a48196930 con 0x7f8a48106cd0 2024-09-17T00:06:48.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.176+0000 7f8a457fa700 1 -- 172.21.15.42:0/843975454 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a30022470 con 0x7f8a48106cd0 2024-09-17T00:06:48.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.176+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a48196e20 con 0x7f8a48106cd0 2024-09-17T00:06:48.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.177+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a481091d0 con 0x7f8a48106cd0 2024-09-17T00:06:48.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.177+0000 7f8a457fa700 1 -- 172.21.15.42:0/843975454 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8a30003780 con 0x7f8a48106cd0 2024-09-17T00:06:48.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.178+0000 7f8a457fa700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a3406e680 0x7f8a34070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.178+0000 7f8a457fa700 1 -- 172.21.15.42:0/843975454 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8a3008f930 con 0x7f8a48106cd0 2024-09-17T00:06:48.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.178+0000 7f8a477fe700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a3406e680 0x7f8a34070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.178+0000 7f8a477fe700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a3406e680 0x7f8a34070b40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009c60 tx=0x7f8a38009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:48.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.182+0000 7f8a457fa700 1 -- 172.21.15.42:0/843975454 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8a3005c680 con 0x7f8a48106cd0 2024-09-17T00:06:48.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.415+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a48197cb0 con 0x7f8a3406e680 2024-09-17T00:06:48.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.416+0000 7f8a457fa700 1 -- 172.21.15.42:0/843975454 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a48197cb0 con 0x7f8a3406e680 2024-09-17T00:06:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a3406e680 msgr2=0x7f8a34070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a3406e680 0x7f8a34070b40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009c60 tx=0x7f8a38009260 comp rx=0 tx=0).stop 2024-09-17T00:06:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48106cd0 msgr2=0x7f8a4819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48106cd0 0x7f8a4819cb70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005f50 tx=0x7f8a3000f690 comp rx=0 tx=0).stop 2024-09-17T00:06:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 shutdown_connections 2024-09-17T00:06:48.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a3406e680 0x7f8a34070b40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a48068580 0x7f8a4819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 --2- 172.21.15.42:0/843975454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48106cd0 0x7f8a4819cb70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 >> 172.21.15.42:0/843975454 conn(0x7f8a48075670 msgr2=0x7f8a480feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:48.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 shutdown_connections 2024-09-17T00:06:48.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.419+0000 7f8a4e217700 1 -- 172.21.15.42:0/843975454 wait complete. 2024-09-17T00:06:48.427 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:06:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 -- 172.21.15.42:0/1222228573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 msgr2=0x7fec30105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 --2- 172.21.15.42:0/1222228573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec30105730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec20009910 tx=0x7fec20009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 -- 172.21.15.42:0/1222228573 shutdown_connections 2024-09-17T00:06:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 --2- 172.21.15.42:0/1222228573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec300ff340 0x7fec300ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 --2- 172.21.15.42:0/1222228573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec30105730 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 -- 172.21.15.42:0/1222228573 >> 172.21.15.42:0/1222228573 conn(0x7fec30074e30 msgr2=0x7fec30075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 -- 172.21.15.42:0/1222228573 shutdown_connections 2024-09-17T00:06:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.527+0000 7fec36823700 1 -- 172.21.15.42:0/1222228573 wait complete. 2024-09-17T00:06:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.528+0000 7fec36823700 1 Processor -- start 2024-09-17T00:06:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.528+0000 7fec36823700 1 -- start start 2024-09-17T00:06:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.528+0000 7fec36823700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec300ff340 0x7fec3019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.528+0000 7fec36823700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec3019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.528+0000 7fec36823700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec301955b0 con 0x7fec30105350 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.528+0000 7fec36823700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec30195720 con 0x7fec300ff340 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec3019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec3019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50822/0 (socket says 172.21.15.42:50822) 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2f7fe700 1 -- 172.21.15.42:0/400355478 learned_addr learned my addr 172.21.15.42:0/400355478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2ffff700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec300ff340 0x7fec3019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2f7fe700 1 -- 172.21.15.42:0/400355478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec300ff340 msgr2=0x7fec3019b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2f7fe700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec300ff340 0x7fec3019b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2f7fe700 1 -- 172.21.15.42:0/400355478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec200095f0 con 0x7fec30105350 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2f7fe700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec3019bb00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec24009970 tx=0x7fec24009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2d7fa700 1 -- 172.21.15.42:0/400355478 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec24012070 con 0x7fec30105350 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2d7fa700 1 -- 172.21.15.42:0/400355478 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec24007af0 con 0x7fec30105350 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec301959a0 con 0x7fec30105350 2024-09-17T00:06:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec2d7fa700 1 -- 172.21.15.42:0/400355478 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec24017bf0 con 0x7fec30105350 2024-09-17T00:06:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.529+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec30195ef0 con 0x7fec30105350 2024-09-17T00:06:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.530+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3004f2e0 con 0x7fec30105350 2024-09-17T00:06:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.531+0000 7fec2d7fa700 1 -- 172.21.15.42:0/400355478 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fec24007c60 con 0x7fec30105350 2024-09-17T00:06:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.531+0000 7fec2d7fa700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fec1806e680 0x7fec18070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.531+0000 7fec2d7fa700 1 -- 172.21.15.42:0/400355478 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fec2408faf0 con 0x7fec30105350 2024-09-17T00:06:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.531+0000 7fec2ffff700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fec1806e680 0x7fec18070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.532+0000 7fec2ffff700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fec1806e680 0x7fec18070b40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec20006010 tx=0x7fec2001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:48.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.536+0000 7fec2d7fa700 1 -- 172.21.15.42:0/400355478 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fec24058dc0 con 0x7fec30105350 2024-09-17T00:06:48.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.770+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec3010cda0 con 0x7fec1806e680 2024-09-17T00:06:48.768 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:48 smithi042 ceph-mon[111056]: pgmap v456: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:48.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.772+0000 7fec2d7fa700 1 -- 172.21.15.42:0/400355478 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fec3010cda0 con 0x7fec1806e680 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fec1806e680 msgr2=0x7fec18070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fec1806e680 0x7fec18070b40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec20006010 tx=0x7fec2001a040 comp rx=0 tx=0).stop 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 msgr2=0x7fec3019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec3019bb00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec24009970 tx=0x7fec24009c80 comp rx=0 tx=0).stop 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 shutdown_connections 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fec1806e680 0x7fec18070b40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec300ff340 0x7fec3019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 --2- 172.21.15.42:0/400355478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec30105350 0x7fec3019bb00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 >> 172.21.15.42:0/400355478 conn(0x7fec30074e30 msgr2=0x7fec300fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:48.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 shutdown_connections 2024-09-17T00:06:48.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.774+0000 7fec36823700 1 -- 172.21.15.42:0/400355478 wait complete. 2024-09-17T00:06:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:48 smithi043 ceph-mon[81510]: pgmap v456: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:48.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1815575840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 msgr2=0x7fd6bc100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1815575840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc100180 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00b0a0 tx=0x7fd6ac00b3b0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1815575840 shutdown_connections 2024-09-17T00:06:48.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1815575840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc100750 0x7fd6bc1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1815575840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc100180 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1815575840 >> 172.21.15.42:0/1815575840 conn(0x7fd6bc0fb650 msgr2=0x7fd6bc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:48.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1815575840 shutdown_connections 2024-09-17T00:06:48.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.883+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1815575840 wait complete. 2024-09-17T00:06:48.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6c0f92700 1 Processor -- start 2024-09-17T00:06:48.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6c0f92700 1 -- start start 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6c0f92700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6c0f92700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc100750 0x7fd6bc101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6c0f92700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc1022c0 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6c0f92700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6bc102400 con 0x7fd6bc100750 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6ba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6ba59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50834/0 (socket says 172.21.15.42:50834) 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6ba59c700 1 -- 172.21.15.42:0/1124302372 learned_addr learned my addr 172.21.15.42:0/1124302372 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6b9d9b700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc100750 0x7fd6bc101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6ba59c700 1 -- 172.21.15.42:0/1124302372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc100750 msgr2=0x7fd6bc101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6ba59c700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc100750 0x7fd6bc101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.884+0000 7fd6ba59c700 1 -- 172.21.15.42:0/1124302372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac009d00 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.885+0000 7fd6ba59c700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc101840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000f80 tx=0x7fd6ac00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:48.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.885+0000 7fd6b37fe700 1 -- 172.21.15.42:0/1124302372 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6ac00e040 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.885+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6bc102680 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.885+0000 7fd6b37fe700 1 -- 172.21.15.42:0/1124302372 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6ac003e60 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.885+0000 7fd6b37fe700 1 -- 172.21.15.42:0/1124302372 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6ac01d910 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.885+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6bc1a2e10 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.886+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6bc04f2e0 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.886+0000 7fd6b37fe700 1 -- 172.21.15.42:0/1124302372 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd6ac004820 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.887+0000 7fd6b37fe700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd6a806e680 0x7fd6a8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.887+0000 7fd6b37fe700 1 -- 172.21.15.42:0/1124302372 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd6ac08fe70 con 0x7fd6bc0ffda0 2024-09-17T00:06:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.887+0000 7fd6b9d9b700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd6a806e680 0x7fd6a8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.887+0000 7fd6b9d9b700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd6a806e680 0x7fd6a8070b40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005fd0 tx=0x7fd6a400a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:48.891+0000 7fd6b37fe700 1 -- 172.21.15.42:0/1124302372 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd6ac059090 con 0x7fd6bc0ffda0 2024-09-17T00:06:49.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.110+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6bc0fc440 con 0x7fd6a806e680 2024-09-17T00:06:49.117 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.119+0000 7fd6b37fe700 1 -- 172.21.15.42:0/1124302372 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fd6bc0fc440 con 0x7fd6a806e680 2024-09-17T00:06:49.117 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:06:49.117 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (15m) 9m ago 26m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:06:49.117 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (26m) 9m ago 26m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:06:49.117 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (25m) 8m ago 25m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:06:49.117 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (13m) 9m ago 26m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:06:49.117 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (12m) 8m ago 25m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (15m) 9m ago 25m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (9m) 9m ago 20m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (9m) 9m ago 20m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (8m) 8m ago 20m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (8m) 8m ago 20m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (17m) 9m ago 27m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (16m) 8m ago 25m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (14m) 9m ago 27m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (13m) 8m ago 25m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (16m) 9m ago 26m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (16m) 8m ago 25m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (12m) 9m ago 24m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (12m) 9m ago 23m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:06:49.118 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (11m) 9m ago 23m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:06:49.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (10m) 8m ago 22m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:06:49.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (10m) 8m ago 22m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:06:49.119 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (9m) 8m ago 22m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:06:49.119 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (15m) 9m ago 25m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd6a806e680 msgr2=0x7fd6a8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd6a806e680 0x7fd6a8070b40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005fd0 tx=0x7fd6a400a560 comp rx=0 tx=0).stop 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 msgr2=0x7fd6bc101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc101840 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000f80 tx=0x7fd6ac00cdd0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 shutdown_connections 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd6a806e680 0x7fd6a8070b40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6bc0ffda0 0x7fd6bc101840 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 --2- 172.21.15.42:0/1124302372 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6bc100750 0x7fd6bc101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 >> 172.21.15.42:0/1124302372 conn(0x7fd6bc0fb650 msgr2=0x7fd6bc103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 shutdown_connections 2024-09-17T00:06:49.120 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.122+0000 7fd6c0f92700 1 -- 172.21.15.42:0/1124302372 wait complete. 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2441841356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c100750 msgr2=0x7fdd1c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2441841356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c100750 0x7fdd1c1047a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009910 tx=0x7fdd10009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2441841356 shutdown_connections 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2441841356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c100750 0x7fdd1c1047a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2441841356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c0ffda0 0x7fdd1c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2441841356 >> 172.21.15.42:0/2441841356 conn(0x7fdd1c0fb650 msgr2=0x7fdd1c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2441841356 shutdown_connections 2024-09-17T00:06:49.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.229+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2441841356 wait complete. 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd20bf5700 1 Processor -- start 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd20bf5700 1 -- start start 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd20bf5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c0ffda0 0x7fdd1c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd20bf5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c100750 0x7fdd1c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd20bf5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19d1b0 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd20bf5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c1966f0 con 0x7fdd1c100750 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c0ffda0 0x7fdd1c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c0ffda0 0x7fdd1c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50850/0 (socket says 172.21.15.42:50850) 2024-09-17T00:06:49.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd1a59c700 1 -- 172.21.15.42:0/2706472742 learned_addr learned my addr 172.21.15.42:0/2706472742 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.230+0000 7fdd19d9b700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c100750 0x7fdd1c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd1a59c700 1 -- 172.21.15.42:0/2706472742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c100750 msgr2=0x7fdd1c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd1a59c700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c100750 0x7fdd1c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd1a59c700 1 -- 172.21.15.42:0/2706472742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100095f0 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd19d9b700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c100750 0x7fdd1c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd1a59c700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c0ffda0 0x7fdd1c19c670 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00c930 tx=0x7fdd0c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd0b7fe700 1 -- 172.21.15.42:0/2706472742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c007ad0 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c1969d0 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd0b7fe700 1 -- 172.21.15.42:0/2706472742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd0c007c30 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd0b7fe700 1 -- 172.21.15.42:0/2706472742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c018610 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.231+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c196f20 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.232+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c1091b0 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.233+0000 7fdd0b7fe700 1 -- 172.21.15.42:0/2706472742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fdd0c00f450 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.233+0000 7fdd0b7fe700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd0406e680 0x7fdd04070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.233+0000 7fdd0b7fe700 1 -- 172.21.15.42:0/2706472742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdd0c08e880 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.233+0000 7fdd19d9b700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd0406e680 0x7fdd04070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:49.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.234+0000 7fdd19d9b700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd0406e680 0x7fdd04070b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c197b60 tx=0x7fdd1001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:49.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.237+0000 7fdd0b7fe700 1 -- 172.21.15.42:0/2706472742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdd0c05b550 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.534+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdd1c197c30 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.534+0000 7fdd0b7fe700 1 -- 172.21.15.42:0/2706472742 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fdd0c05b0e0 con 0x7fdd1c0ffda0 2024-09-17T00:06:49.532 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:06:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:06:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:06:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:49.532 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:06:49.533 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:06:49.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.536+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd0406e680 msgr2=0x7fdd04070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.536+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd0406e680 0x7fdd04070b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c197b60 tx=0x7fdd1001a040 comp rx=0 tx=0).stop 2024-09-17T00:06:49.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.536+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c0ffda0 msgr2=0x7fdd1c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.536+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c0ffda0 0x7fdd1c19c670 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00c930 tx=0x7fdd0c00ccf0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.537+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 shutdown_connections 2024-09-17T00:06:49.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.537+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd0406e680 0x7fdd04070b40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.537+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd1c0ffda0 0x7fdd1c19c670 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.537+0000 7fdd20bf5700 1 --2- 172.21.15.42:0/2706472742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd1c100750 0x7fdd1c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.537+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 >> 172.21.15.42:0/2706472742 conn(0x7fdd1c0fb650 msgr2=0x7fdd1c069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:49.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.537+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 shutdown_connections 2024-09-17T00:06:49.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.537+0000 7fdd20bf5700 1 -- 172.21.15.42:0/2706472742 wait complete. 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 -- 172.21.15.42:0/4175619766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 msgr2=0x7f55c00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 --2- 172.21.15.42:0/4175619766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c00689e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009910 tx=0x7f55b4009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 -- 172.21.15.42:0/4175619766 shutdown_connections 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 --2- 172.21.15.42:0/4175619766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c00689e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 --2- 172.21.15.42:0/4175619766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c0106cd0 0x7f55c01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 -- 172.21.15.42:0/4175619766 >> 172.21.15.42:0/4175619766 conn(0x7f55c0075670 msgr2=0x7f55c0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 -- 172.21.15.42:0/4175619766 shutdown_connections 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.645+0000 7f55c5c95700 1 -- 172.21.15.42:0/4175619766 wait complete. 2024-09-17T00:06:49.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55c5c95700 1 Processor -- start 2024-09-17T00:06:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55c5c95700 1 -- start start 2024-09-17T00:06:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55c5c95700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55c5c95700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c0106cd0 0x7f55c019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55c5c95700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c019d200 con 0x7f55c0068580 2024-09-17T00:06:49.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55c5c95700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c01966b0 con 0x7f55c0106cd0 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55bf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55bf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50876/0 (socket says 172.21.15.42:50876) 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55bf7fe700 1 -- 172.21.15.42:0/1256525734 learned_addr learned my addr 172.21.15.42:0/1256525734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.646+0000 7f55beffd700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c0106cd0 0x7f55c019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55bf7fe700 1 -- 172.21.15.42:0/1256525734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c0106cd0 msgr2=0x7f55c019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55bf7fe700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c0106cd0 0x7f55c019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55bf7fe700 1 -- 172.21.15.42:0/1256525734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b40095f0 con 0x7f55c0068580 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55bf7fe700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c019c630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f55b000d8d0 tx=0x7f55b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55bcff9700 1 -- 172.21.15.42:0/1256525734 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55b0009940 con 0x7f55c0068580 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55bcff9700 1 -- 172.21.15.42:0/1256525734 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55b0010460 con 0x7f55c0068580 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c0196990 con 0x7f55c0068580 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55bcff9700 1 -- 172.21.15.42:0/1256525734 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55b000f5f0 con 0x7f55c0068580 2024-09-17T00:06:49.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.647+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c0196ee0 con 0x7f55c0068580 2024-09-17T00:06:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.648+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c01091d0 con 0x7f55c0068580 2024-09-17T00:06:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.649+0000 7f55bcff9700 1 -- 172.21.15.42:0/1256525734 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f55b00105d0 con 0x7f55c0068580 2024-09-17T00:06:49.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.649+0000 7f55bcff9700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f55a806e680 0x7f55a8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.649+0000 7f55bcff9700 1 -- 172.21.15.42:0/1256525734 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f55b008e680 con 0x7f55c0068580 2024-09-17T00:06:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.649+0000 7f55beffd700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f55a806e680 0x7f55a8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:49.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.650+0000 7f55beffd700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f55a806e680 0x7f55a8070b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005b40 tx=0x7f55b401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:49.651 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.654+0000 7f55bcff9700 1 -- 172.21.15.42:0/1256525734 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f55b005b350 con 0x7f55c0068580 2024-09-17T00:06:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:49 smithi043 ceph-mon[81510]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:49 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2706472742' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:06:49.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.909+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f55c004f2e0 con 0x7f55c0068580 2024-09-17T00:06:49.907 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:49 smithi042 ceph-mon[111056]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:49.907 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:49 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2706472742' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:06:49.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.910+0000 7f55bcff9700 1 -- 172.21.15.42:0/1256525734 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f55b005aee0 con 0x7f55c0068580 2024-09-17T00:06:49.908 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:06:49.908 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:06:49.908 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:06:49.908 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:06:49.908 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:06:49.908 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:06:49.909 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:06:49.910 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:06:49.911 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:49.912 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:49.912 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:49.912 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.912+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f55a806e680 msgr2=0x7f55a8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.912+0000 7f55c5c95700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f55a806e680 0x7f55a8070b40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005b40 tx=0x7f55b401a040 comp rx=0 tx=0).stop 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.912+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 msgr2=0x7f55c019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.912+0000 7f55c5c95700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c019c630 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f55b000d8d0 tx=0x7f55b000dc90 comp rx=0 tx=0).stop 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.912+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 shutdown_connections 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.912+0000 7f55c5c95700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f55a806e680 0x7f55a8070b40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.912+0000 7f55c5c95700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55c0068580 0x7f55c019c630 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.913+0000 7f55c5c95700 1 --2- 172.21.15.42:0/1256525734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c0106cd0 0x7f55c019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.913+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 >> 172.21.15.42:0/1256525734 conn(0x7f55c0075670 msgr2=0x7f55c00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:49.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.913+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 shutdown_connections 2024-09-17T00:06:49.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:49.913+0000 7f55c5c95700 1 -- 172.21.15.42:0/1256525734 wait complete. 2024-09-17T00:06:49.914 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:06:50.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 -- 172.21.15.42:0/63002863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9298100750 msgr2=0x7f92981047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 --2- 172.21.15.42:0/63002863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9298100750 0x7f92981047a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9284009910 tx=0x7f9284009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:50.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 -- 172.21.15.42:0/63002863 shutdown_connections 2024-09-17T00:06:50.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 --2- 172.21.15.42:0/63002863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9298100750 0x7f92981047a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 --2- 172.21.15.42:0/63002863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92980ffda0 0x7f9298100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 -- 172.21.15.42:0/63002863 >> 172.21.15.42:0/63002863 conn(0x7f92980fb650 msgr2=0x7f92980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:50.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 -- 172.21.15.42:0/63002863 shutdown_connections 2024-09-17T00:06:50.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 -- 172.21.15.42:0/63002863 wait complete. 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 Processor -- start 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.022+0000 7f929ef26700 1 -- start start 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ef26700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92980ffda0 0x7f9298101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ef26700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9298100750 0x7f9298101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ef26700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92981022c0 con 0x7f92980ffda0 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ef26700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9298102400 con 0x7f9298100750 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ccc2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92980ffda0 0x7f9298101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ccc2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92980ffda0 0x7f9298101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50888/0 (socket says 172.21.15.42:50888) 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ccc2700 1 -- 172.21.15.42:0/134631353 learned_addr learned my addr 172.21.15.42:0/134631353 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:50.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f9297fff700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9298100750 0x7f9298101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ccc2700 1 -- 172.21.15.42:0/134631353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9298100750 msgr2=0x7f9298101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ccc2700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9298100750 0x7f9298101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ccc2700 1 -- 172.21.15.42:0/134631353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840095f0 con 0x7f92980ffda0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f929ccc2700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92980ffda0 0x7f9298101840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f928c00d8d0 tx=0x7f928c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.023+0000 7f9295ffb700 1 -- 172.21.15.42:0/134631353 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f928c0098a0 con 0x7f92980ffda0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.024+0000 7f9295ffb700 1 -- 172.21.15.42:0/134631353 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f928c010460 con 0x7f92980ffda0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.024+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92981a2a70 con 0x7f92980ffda0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.024+0000 7f9295ffb700 1 -- 172.21.15.42:0/134631353 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f928c00f5f0 con 0x7f92980ffda0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.024+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92981a2e20 con 0x7f92980ffda0 2024-09-17T00:06:50.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.025+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92981091b0 con 0x7f92980ffda0 2024-09-17T00:06:50.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.025+0000 7f9295ffb700 1 -- 172.21.15.42:0/134631353 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f928c009a00 con 0x7f92980ffda0 2024-09-17T00:06:50.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.026+0000 7f9295ffb700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f928806e680 0x7f9288070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:50.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.026+0000 7f9295ffb700 1 -- 172.21.15.42:0/134631353 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f928c08e460 con 0x7f92980ffda0 2024-09-17T00:06:50.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.026+0000 7f9297fff700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f928806e680 0x7f9288070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:50.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.026+0000 7f9297fff700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f928806e680 0x7f9288070b40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9284006010 tx=0x7f928401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:50.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.030+0000 7f9295ffb700 1 -- 172.21.15.42:0/134631353 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f928c057680 con 0x7f92980ffda0 2024-09-17T00:06:50.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.260+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92981026b0 con 0x7f928806e680 2024-09-17T00:06:50.258 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.261+0000 7f9295ffb700 1 -- 172.21.15.42:0/134631353 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f92981026b0 con 0x7f928806e680 2024-09-17T00:06:50.258 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:06:50.258 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:06:50.258 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:06:50.259 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:06:50.260 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f928806e680 msgr2=0x7f9288070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f928806e680 0x7f9288070b40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9284006010 tx=0x7f928401a040 comp rx=0 tx=0).stop 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92980ffda0 msgr2=0x7f9298101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92980ffda0 0x7f9298101840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f928c00d8d0 tx=0x7f928c00dbe0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 shutdown_connections 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f928806e680 0x7f9288070b40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92980ffda0 0x7f9298101840 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 --2- 172.21.15.42:0/134631353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9298100750 0x7f9298101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 >> 172.21.15.42:0/134631353 conn(0x7f92980fb650 msgr2=0x7f9298069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 shutdown_connections 2024-09-17T00:06:50.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.263+0000 7f929ef26700 1 -- 172.21.15.42:0/134631353 wait complete. 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 -- 172.21.15.42:0/2435178187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14800ff340 msgr2=0x7f14800ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 --2- 172.21.15.42:0/2435178187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14800ff340 0x7f14800ff7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1474009910 tx=0x7f1474009c20 comp rx=0 tx=0).stop 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 -- 172.21.15.42:0/2435178187 shutdown_connections 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 --2- 172.21.15.42:0/2435178187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14800ff340 0x7f14800ff7a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 --2- 172.21.15.42:0/2435178187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1480105350 0x7f1480105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 -- 172.21.15.42:0/2435178187 >> 172.21.15.42:0/2435178187 conn(0x7f1480074e30 msgr2=0x7f1480075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 -- 172.21.15.42:0/2435178187 shutdown_connections 2024-09-17T00:06:50.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.371+0000 7f148548d700 1 -- 172.21.15.42:0/2435178187 wait complete. 2024-09-17T00:06:50.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.372+0000 7f148548d700 1 Processor -- start 2024-09-17T00:06:50.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.372+0000 7f148548d700 1 -- start start 2024-09-17T00:06:50.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.372+0000 7f148548d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800ff340 0x7f148019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:50.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.372+0000 7f148548d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1480105350 0x7f148019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:50.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.372+0000 7f148548d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148019d220 con 0x7f1480105350 2024-09-17T00:06:50.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.372+0000 7f148548d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801966d0 con 0x7f14800ff340 2024-09-17T00:06:50.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.372+0000 7f147e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1480105350 0x7f148019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f147e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1480105350 0x7f148019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50916/0 (socket says 172.21.15.42:50916) 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f147e7fc700 1 -- 172.21.15.42:0/646853479 learned_addr learned my addr 172.21.15.42:0/646853479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f147effd700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800ff340 0x7f148019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f147e7fc700 1 -- 172.21.15.42:0/646853479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800ff340 msgr2=0x7f148019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f147e7fc700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800ff340 0x7f148019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f147e7fc700 1 -- 172.21.15.42:0/646853479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14740095f0 con 0x7f1480105350 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f147e7fc700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1480105350 0x7f148019cb90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f14740092e0 tx=0x7f147400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f1467fff700 1 -- 172.21.15.42:0/646853479 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f147401d070 con 0x7f1480105350 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1480196950 con 0x7f1480105350 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f1467fff700 1 -- 172.21.15.42:0/646853479 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1474005e00 con 0x7f1480105350 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f1467fff700 1 -- 172.21.15.42:0/646853479 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1474017400 con 0x7f1480105350 2024-09-17T00:06:50.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.373+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1480196e40 con 0x7f1480105350 2024-09-17T00:06:50.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.374+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1480109860 con 0x7f1480105350 2024-09-17T00:06:50.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.375+0000 7f1467fff700 1 -- 172.21.15.42:0/646853479 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f147400f6f0 con 0x7f1480105350 2024-09-17T00:06:50.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.375+0000 7f1467fff700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f146806e680 0x7f1468070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:06:50.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.375+0000 7f1467fff700 1 -- 172.21.15.42:0/646853479 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f147408fbc0 con 0x7f1480105350 2024-09-17T00:06:50.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.375+0000 7f147effd700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f146806e680 0x7f1468070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:06:50.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.376+0000 7f147effd700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f146806e680 0x7f1468070b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1470005950 tx=0x7f147000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:06:50.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.380+0000 7f1467fff700 1 -- 172.21.15.42:0/646853479 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f147405c7f0 con 0x7f1480105350 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.677+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f148004f2e0 con 0x7f1480105350 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.677+0000 7f1467fff700 1 -- 172.21.15.42:0/646853479 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f147405c380 con 0x7f1480105350 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:06:50.675 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f146806e680 msgr2=0x7f1468070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f146806e680 0x7f1468070b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1470005950 tx=0x7f147000b410 comp rx=0 tx=0).stop 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1480105350 msgr2=0x7f148019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1480105350 0x7f148019cb90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f14740092e0 tx=0x7f147400bd70 comp rx=0 tx=0).stop 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 shutdown_connections 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f146806e680 0x7f1468070b40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800ff340 0x7f148019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 --2- 172.21.15.42:0/646853479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1480105350 0x7f148019cb90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:06:50.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 >> 172.21.15.42:0/646853479 conn(0x7f1480074e30 msgr2=0x7f1480103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:06:50.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 shutdown_connections 2024-09-17T00:06:50.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:06:50.679+0000 7f148548d700 1 -- 172.21.15.42:0/646853479 wait complete. 2024-09-17T00:06:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:50 smithi043 ceph-mon[81510]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:50 smithi043 ceph-mon[81510]: pgmap v457: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:50 smithi043 ceph-mon[81510]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:50 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1256525734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:06:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:50 smithi042 ceph-mon[111056]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:50 smithi042 ceph-mon[111056]: pgmap v457: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:50 smithi042 ceph-mon[111056]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:50 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1256525734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:06:51.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:51 smithi042 ceph-mon[111056]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:51.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:51.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:51 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/646853479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:06:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:51 smithi043 ceph-mon[81510]: from='client.34900 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:06:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:06:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:51 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/646853479' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:06:52.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:52 smithi043 ceph-mon[81510]: pgmap v458: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:53.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:52 smithi042 ceph-mon[111056]: pgmap v458: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:54.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:54 smithi043 ceph-mon[81510]: pgmap v459: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:54 smithi042 ceph-mon[111056]: pgmap v459: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:56.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:56 smithi042 ceph-mon[111056]: pgmap v460: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:56 smithi043 ceph-mon[81510]: pgmap v460: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:59.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:06:58 smithi042 ceph-mon[111056]: pgmap v461: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:06:59.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:06:58 smithi043 ceph-mon[81510]: pgmap v461: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:00.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:00 smithi043 ceph-mon[81510]: pgmap v462: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:01.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:00 smithi042 ceph-mon[111056]: pgmap v462: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:03.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:02 smithi042 ceph-mon[111056]: pgmap v463: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:03.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:02 smithi043 ceph-mon[81510]: pgmap v463: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:05.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:04 smithi042 ceph-mon[111056]: pgmap v464: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:05.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:04 smithi043 ceph-mon[81510]: pgmap v464: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:06.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:06.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:06.934 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:06 smithi043 ceph-mon[81510]: pgmap v465: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:06 smithi042 ceph-mon[111056]: pgmap v465: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:08 smithi042 ceph-mon[111056]: pgmap v466: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:09.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:08 smithi043 ceph-mon[81510]: pgmap v466: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:10.930 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:10 smithi043 ceph-mon[81510]: pgmap v467: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:11.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:10 smithi042 ceph-mon[111056]: pgmap v467: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:13.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:12 smithi042 ceph-mon[111056]: pgmap v468: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:13.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:12 smithi043 ceph-mon[81510]: pgmap v468: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:15.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:14 smithi042 ceph-mon[111056]: pgmap v469: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:15.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:14 smithi043 ceph-mon[81510]: pgmap v469: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:16 smithi042 ceph-mon[111056]: pgmap v470: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:17.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:16 smithi043 ceph-mon[81510]: pgmap v470: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:18 smithi042 ceph-mon[111056]: pgmap v471: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:19.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:18 smithi043 ceph-mon[81510]: pgmap v471: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:20.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 -- 172.21.15.42:0/2828174829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 msgr2=0x7f0548100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:20.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 --2- 172.21.15.42:0/2828174829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548100080 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f053c009910 tx=0x7f053c009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:20.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 -- 172.21.15.42:0/2828174829 shutdown_connections 2024-09-17T00:07:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 --2- 172.21.15.42:0/2828174829 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548100080 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 --2- 172.21.15.42:0/2828174829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0548105bd0 0x7f0548105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 -- 172.21.15.42:0/2828174829 >> 172.21.15.42:0/2828174829 conn(0x7f0548075670 msgr2=0x7f0548075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 -- 172.21.15.42:0/2828174829 shutdown_connections 2024-09-17T00:07:20.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.801+0000 7f054c8a1700 1 -- 172.21.15.42:0/2828174829 wait complete. 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.802+0000 7f054c8a1700 1 Processor -- start 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.802+0000 7f054c8a1700 1 -- start start 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.802+0000 7f054c8a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.802+0000 7f054c8a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0548105bd0 0x7f0548197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.802+0000 7f054c8a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548198380 con 0x7f05480ffc20 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.802+0000 7f054c8a1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481984f0 con 0x7f0548105bd0 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46778/0 (socket says 172.21.15.42:46778) 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054659c700 1 -- 172.21.15.42:0/766213726 learned_addr learned my addr 172.21.15.42:0/766213726 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f0545d9b700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0548105bd0 0x7f0548197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054659c700 1 -- 172.21.15.42:0/766213726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0548105bd0 msgr2=0x7f0548197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:20.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054659c700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0548105bd0 0x7f0548197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054659c700 1 -- 172.21.15.42:0/766213726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c0095f0 con 0x7f05480ffc20 2024-09-17T00:07:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054659c700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548197760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f053800d8d0 tx=0x7f053800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f05377fe700 1 -- 172.21.15.42:0/766213726 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05380098a0 con 0x7f05480ffc20 2024-09-17T00:07:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05481a07d0 con 0x7f05480ffc20 2024-09-17T00:07:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f05377fe700 1 -- 172.21.15.42:0/766213726 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0538010460 con 0x7f05480ffc20 2024-09-17T00:07:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f05377fe700 1 -- 172.21.15.42:0/766213726 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f053800f5f0 con 0x7f05480ffc20 2024-09-17T00:07:20.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.803+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05481a0d20 con 0x7f05480ffc20 2024-09-17T00:07:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.805+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05481080d0 con 0x7f05480ffc20 2024-09-17T00:07:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.805+0000 7f05377fe700 1 -- 172.21.15.42:0/766213726 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f0538009a00 con 0x7f05480ffc20 2024-09-17T00:07:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.806+0000 7f05377fe700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f053006e680 0x7f0530070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.806+0000 7f05377fe700 1 -- 172.21.15.42:0/766213726 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f053808e5f0 con 0x7f05480ffc20 2024-09-17T00:07:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.806+0000 7f0545d9b700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f053006e680 0x7f0530070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:20.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.806+0000 7f0545d9b700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f053006e680 0x7f0530070b40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0548068e30 tx=0x7f053c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:20.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:20.810+0000 7f05377fe700 1 -- 172.21.15.42:0/766213726 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f05380576f0 con 0x7f05480ffc20 2024-09-17T00:07:21.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:20 smithi042 ceph-mon[111056]: pgmap v472: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:21.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:21.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.040+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0548068c80 con 0x7f053006e680 2024-09-17T00:07:21.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.041+0000 7f05377fe700 1 -- 172.21.15.42:0/766213726 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0548068c80 con 0x7f053006e680 2024-09-17T00:07:21.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.043+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f053006e680 msgr2=0x7f0530070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f053006e680 0x7f0530070b40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0548068e30 tx=0x7f053c01a040 comp rx=0 tx=0).stop 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 msgr2=0x7f0548197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548197760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f053800d8d0 tx=0x7f053800dbe0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 shutdown_connections 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f053006e680 0x7f0530070b40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05480ffc20 0x7f0548197760 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 --2- 172.21.15.42:0/766213726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0548105bd0 0x7f0548197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 >> 172.21.15.42:0/766213726 conn(0x7f0548075670 msgr2=0x7f05480fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 shutdown_connections 2024-09-17T00:07:21.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.044+0000 7f054c8a1700 1 -- 172.21.15.42:0/766213726 wait complete. 2024-09-17T00:07:21.052 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:07:21.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:20 smithi043 ceph-mon[81510]: pgmap v472: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:21.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:21.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 -- 172.21.15.42:0/2313942814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 msgr2=0x7f2ee0102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 --2- 172.21.15.42:0/2313942814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee0102280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009910 tx=0x7f2ed0009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:21.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 -- 172.21.15.42:0/2313942814 shutdown_connections 2024-09-17T00:07:21.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 --2- 172.21.15.42:0/2313942814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee01027c0 0x7f2ee010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 --2- 172.21.15.42:0/2313942814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee0102280 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 -- 172.21.15.42:0/2313942814 >> 172.21.15.42:0/2313942814 conn(0x7f2ee00fb650 msgr2=0x7f2ee00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:21.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 -- 172.21.15.42:0/2313942814 shutdown_connections 2024-09-17T00:07:21.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.153+0000 7f2ee6101700 1 -- 172.21.15.42:0/2313942814 wait complete. 2024-09-17T00:07:21.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2ee6101700 1 Processor -- start 2024-09-17T00:07:21.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2ee6101700 1 -- start start 2024-09-17T00:07:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2ee6101700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee00ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2ee6101700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee01027c0 0x7f2ee0100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2ee6101700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee0100580 con 0x7f2ee0101ea0 2024-09-17T00:07:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2ee6101700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee01006c0 con 0x7f2ee01027c0 2024-09-17T00:07:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2edf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee00ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2edf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee00ffb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46798/0 (socket says 172.21.15.42:46798) 2024-09-17T00:07:21.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2edf7fe700 1 -- 172.21.15.42:0/4203266788 learned_addr learned my addr 172.21.15.42:0/4203266788 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2edf7fe700 1 -- 172.21.15.42:0/4203266788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee01027c0 msgr2=0x7f2ee0100040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2edeffd700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee01027c0 0x7f2ee0100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.154+0000 7f2edf7fe700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee01027c0 0x7f2ee0100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.155+0000 7f2edf7fe700 1 -- 172.21.15.42:0/4203266788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed00095f0 con 0x7f2ee0101ea0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.155+0000 7f2edf7fe700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee00ffb00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0000c00 tx=0x7f2ed000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.155+0000 7f2edcff9700 1 -- 172.21.15.42:0/4203266788 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ed001d070 con 0x7f2ee0101ea0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.155+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee0100940 con 0x7f2ee0101ea0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.155+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee01a2d90 con 0x7f2ee0101ea0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.155+0000 7f2edcff9700 1 -- 172.21.15.42:0/4203266788 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ed0005dc0 con 0x7f2ee0101ea0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.155+0000 7f2edcff9700 1 -- 172.21.15.42:0/4203266788 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ed0017400 con 0x7f2ee0101ea0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.156+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc005320 con 0x7f2ee0101ea0 2024-09-17T00:07:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.156+0000 7f2edcff9700 1 -- 172.21.15.42:0/4203266788 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f2ed0017660 con 0x7f2ee0101ea0 2024-09-17T00:07:21.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.157+0000 7f2edcff9700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2ec806e680 0x7f2ec8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.157+0000 7f2edcff9700 1 -- 172.21.15.42:0/4203266788 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2ed008faa0 con 0x7f2ee0101ea0 2024-09-17T00:07:21.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.157+0000 7f2edeffd700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2ec806e680 0x7f2ec8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.157+0000 7f2edeffd700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2ec806e680 0x7f2ec8070b40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ee01012f0 tx=0x7f2ed40094f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:21.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.161+0000 7f2edcff9700 1 -- 172.21.15.42:0/4203266788 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2ed0058d30 con 0x7f2ee0101ea0 2024-09-17T00:07:21.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.389+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ecc000bf0 con 0x7f2ec806e680 2024-09-17T00:07:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.391+0000 7f2edcff9700 1 -- 172.21.15.42:0/4203266788 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2ecc000bf0 con 0x7f2ec806e680 2024-09-17T00:07:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2ec806e680 msgr2=0x7f2ec8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2ec806e680 0x7f2ec8070b40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2ee01012f0 tx=0x7f2ed40094f0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 msgr2=0x7f2ee00ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee00ffb00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0000c00 tx=0x7f2ed000be20 comp rx=0 tx=0).stop 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 shutdown_connections 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2ec806e680 0x7f2ec8070b40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee0101ea0 0x7f2ee00ffb00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 --2- 172.21.15.42:0/4203266788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee01027c0 0x7f2ee0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 >> 172.21.15.42:0/4203266788 conn(0x7f2ee00fb650 msgr2=0x7f2ee0105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 shutdown_connections 2024-09-17T00:07:21.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.394+0000 7f2ee6101700 1 -- 172.21.15.42:0/4203266788 wait complete. 2024-09-17T00:07:21.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 -- 172.21.15.42:0/2158058335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f40ffda0 msgr2=0x7f00f4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 --2- 172.21.15.42:0/2158058335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f40ffda0 0x7f00f4100180 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009910 tx=0x7f00f0009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:21.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 -- 172.21.15.42:0/2158058335 shutdown_connections 2024-09-17T00:07:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 --2- 172.21.15.42:0/2158058335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00f4100750 0x7f00f41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 --2- 172.21.15.42:0/2158058335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f40ffda0 0x7f00f4100180 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 -- 172.21.15.42:0/2158058335 >> 172.21.15.42:0/2158058335 conn(0x7f00f40fb650 msgr2=0x7f00f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 -- 172.21.15.42:0/2158058335 shutdown_connections 2024-09-17T00:07:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.503+0000 7f00fc775700 1 -- 172.21.15.42:0/2158058335 wait complete. 2024-09-17T00:07:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00fc775700 1 Processor -- start 2024-09-17T00:07:21.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00fc775700 1 -- start start 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00fc775700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00f40ffda0 0x7f00f419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00fc775700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f4100750 0x7f00f419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00fc775700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f419d2c0 con 0x7f00f4100750 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00fc775700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4196770 con 0x7f00f40ffda0 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00f9d10700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f4100750 0x7f00f419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.504+0000 7f00f9d10700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f4100750 0x7f00f419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46820/0 (socket says 172.21.15.42:46820) 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00f9d10700 1 -- 172.21.15.42:0/2195031749 learned_addr learned my addr 172.21.15.42:0/2195031749 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:21.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00fa511700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00f40ffda0 0x7f00f419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00f9d10700 1 -- 172.21.15.42:0/2195031749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00f40ffda0 msgr2=0x7f00f419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00f9d10700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00f40ffda0 0x7f00f419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00f9d10700 1 -- 172.21.15.42:0/2195031749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00095f0 con 0x7f00f4100750 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00f9d10700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f4100750 0x7f00f419cc30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f00e400d900 tx=0x7f00e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00eb7fe700 1 -- 172.21.15.42:0/2195031749 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e4009900 con 0x7f00f4100750 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00eb7fe700 1 -- 172.21.15.42:0/2195031749 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00e4010460 con 0x7f00f4100750 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f4196a50 con 0x7f00f4100750 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00eb7fe700 1 -- 172.21.15.42:0/2195031749 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e400f5f0 con 0x7f00f4100750 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.505+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f4196fa0 con 0x7f00f4100750 2024-09-17T00:07:21.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.506+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f404f2e0 con 0x7f00f4100750 2024-09-17T00:07:21.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.507+0000 7f00eb7fe700 1 -- 172.21.15.42:0/2195031749 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f00e40105d0 con 0x7f00f4100750 2024-09-17T00:07:21.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.507+0000 7f00eb7fe700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f00e006e680 0x7f00e0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.507+0000 7f00eb7fe700 1 -- 172.21.15.42:0/2195031749 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f00e408e440 con 0x7f00f4100750 2024-09-17T00:07:21.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.507+0000 7f00fa511700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f00e006e680 0x7f00e0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.508+0000 7f00fa511700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f00e006e680 0x7f00e0070b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b5c0 tx=0x7f00f001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:21.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.511+0000 7f00eb7fe700 1 -- 172.21.15.42:0/2195031749 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f00e405b190 con 0x7f00f4100750 2024-09-17T00:07:21.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.733+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f00f4197280 con 0x7f00e006e680 2024-09-17T00:07:21.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.744+0000 7f00eb7fe700 1 -- 172.21.15.42:0/2195031749 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f00f4197280 con 0x7f00e006e680 2024-09-17T00:07:21.741 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:07:21.741 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (16m) 9m ago 26m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (27m) 9m ago 27m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (25m) 9m ago 25m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (13m) 9m ago 26m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (13m) 9m ago 25m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (15m) 9m ago 26m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (9m) 9m ago 21m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (10m) 9m ago 21m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (9m) 9m ago 21m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (9m) 9m ago 20m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (17m) 9m ago 28m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (16m) 9m ago 25m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:07:21.742 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (14m) 9m ago 28m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (14m) 9m ago 25m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (16m) 9m ago 26m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (16m) 9m ago 25m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (13m) 9m ago 24m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (12m) 9m ago 24m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (12m) 9m ago 23m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (11m) 9m ago 23m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (10m) 9m ago 23m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (10m) 9m ago 22m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:07:21.743 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (16m) 9m ago 26m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:07:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f00e006e680 msgr2=0x7f00e0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f00e006e680 0x7f00e0070b40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b5c0 tx=0x7f00f001a040 comp rx=0 tx=0).stop 2024-09-17T00:07:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f4100750 msgr2=0x7f00f419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f4100750 0x7f00f419cc30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f00e400d900 tx=0x7f00e400dcc0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 shutdown_connections 2024-09-17T00:07:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f00e006e680 0x7f00e0070b40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00f40ffda0 0x7f00f419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 --2- 172.21.15.42:0/2195031749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00f4100750 0x7f00f419cc30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 >> 172.21.15.42:0/2195031749 conn(0x7f00f40fb650 msgr2=0x7f00f4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 shutdown_connections 2024-09-17T00:07:21.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.746+0000 7f00fc775700 1 -- 172.21.15.42:0/2195031749 wait complete. 2024-09-17T00:07:21.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 -- 172.21.15.42:0/2906547644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 msgr2=0x7fc91410acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 --2- 172.21.15.42:0/2906547644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc91410acb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc908009910 tx=0x7fc908009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:21.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 -- 172.21.15.42:0/2906547644 shutdown_connections 2024-09-17T00:07:21.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 --2- 172.21.15.42:0/2906547644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc91410acb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 --2- 172.21.15.42:0/2906547644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914101ea0 0x7fc914102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 -- 172.21.15.42:0/2906547644 >> 172.21.15.42:0/2906547644 conn(0x7fc9140fb650 msgr2=0x7fc9140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:21.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 -- 172.21.15.42:0/2906547644 shutdown_connections 2024-09-17T00:07:21.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.861+0000 7fc9196fa700 1 -- 172.21.15.42:0/2906547644 wait complete. 2024-09-17T00:07:21.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9196fa700 1 Processor -- start 2024-09-17T00:07:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9196fa700 1 -- start start 2024-09-17T00:07:21.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9196fa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914101ea0 0x7fc9140ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9196fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc914100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9196fa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc914100580 con 0x7fc9141027c0 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9196fa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9141006c0 con 0x7fc914101ea0 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9127fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc914100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9127fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc914100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46846/0 (socket says 172.21.15.42:46846) 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.862+0000 7fc9127fc700 1 -- 172.21.15.42:0/500192719 learned_addr learned my addr 172.21.15.42:0/500192719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc912ffd700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914101ea0 0x7fc9140ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc9127fc700 1 -- 172.21.15.42:0/500192719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914101ea0 msgr2=0x7fc9140ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc9127fc700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914101ea0 0x7fc9140ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc9127fc700 1 -- 172.21.15.42:0/500192719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9080095f0 con 0x7fc9141027c0 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc9127fc700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc914100040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bd70 tx=0x7fc90800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:21.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc8fbfff700 1 -- 172.21.15.42:0/500192719 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc90801d070 con 0x7fc9141027c0 2024-09-17T00:07:21.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc914100940 con 0x7fc9141027c0 2024-09-17T00:07:21.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc8fbfff700 1 -- 172.21.15.42:0/500192719 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc908005dc0 con 0x7fc9141027c0 2024-09-17T00:07:21.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc8fbfff700 1 -- 172.21.15.42:0/500192719 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc908017400 con 0x7fc9141027c0 2024-09-17T00:07:21.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.863+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9141a2d90 con 0x7fc9141027c0 2024-09-17T00:07:21.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.864+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc914108340 con 0x7fc9141027c0 2024-09-17T00:07:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.865+0000 7fc8fbfff700 1 -- 172.21.15.42:0/500192719 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc90800f7f0 con 0x7fc9141027c0 2024-09-17T00:07:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.865+0000 7fc8fbfff700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8fc06e680 0x7fc8fc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.865+0000 7fc8fbfff700 1 -- 172.21.15.42:0/500192719 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc90808fb50 con 0x7fc9141027c0 2024-09-17T00:07:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.865+0000 7fc912ffd700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8fc06e680 0x7fc8fc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:21.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.866+0000 7fc912ffd700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8fc06e680 0x7fc8fc070b40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc904005950 tx=0x7fc9040058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:21.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:21.870+0000 7fc8fbfff700 1 -- 172.21.15.42:0/500192719 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc908058cf0 con 0x7fc9141027c0 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.169+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc91404f2e0 con 0x7fc9141027c0 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.169+0000 7fc8fbfff700 1 -- 172.21.15.42:0/500192719 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc90805c3b0 con 0x7fc9141027c0 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:07:22.167 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:07:22.168 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.171+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8fc06e680 msgr2=0x7fc8fc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.171+0000 7fc9196fa700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8fc06e680 0x7fc8fc070b40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc904005950 tx=0x7fc9040058e0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.171+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 msgr2=0x7fc914100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.171+0000 7fc9196fa700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc914100040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc90800bd70 tx=0x7fc90800be50 comp rx=0 tx=0).stop 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.172+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 shutdown_connections 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.172+0000 7fc9196fa700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc8fc06e680 0x7fc8fc070b40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.172+0000 7fc9196fa700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc914101ea0 0x7fc9140ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.172+0000 7fc9196fa700 1 --2- 172.21.15.42:0/500192719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9141027c0 0x7fc914100040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.172+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 >> 172.21.15.42:0/500192719 conn(0x7fc9140fb650 msgr2=0x7fc914105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.172+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 shutdown_connections 2024-09-17T00:07:22.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.172+0000 7fc9196fa700 1 -- 172.21.15.42:0/500192719 wait complete. 2024-09-17T00:07:22.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 -- 172.21.15.42:0/2039800944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d00ffda0 msgr2=0x7ff6d0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2039800944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d00ffda0 0x7ff6d0100180 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc009910 tx=0x7ff6cc009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 -- 172.21.15.42:0/2039800944 shutdown_connections 2024-09-17T00:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2039800944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d0100750 0x7ff6d01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2039800944 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d00ffda0 0x7ff6d0100180 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 -- 172.21.15.42:0/2039800944 >> 172.21.15.42:0/2039800944 conn(0x7ff6d00fb650 msgr2=0x7ff6d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 -- 172.21.15.42:0/2039800944 shutdown_connections 2024-09-17T00:07:22.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 -- 172.21.15.42:0/2039800944 wait complete. 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.280+0000 7ff6d7311700 1 Processor -- start 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d7311700 1 -- start start 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d7311700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d00ffda0 0x7ff6d0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d7311700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d0100750 0x7ff6d0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d7311700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01022c0 con 0x7ff6d0100750 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d7311700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0102400 con 0x7ff6d00ffda0 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d48ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d0100750 0x7ff6d0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d48ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d0100750 0x7ff6d0101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46864/0 (socket says 172.21.15.42:46864) 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d48ac700 1 -- 172.21.15.42:0/2514216094 learned_addr learned my addr 172.21.15.42:0/2514216094 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d50ad700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d00ffda0 0x7ff6d0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d48ac700 1 -- 172.21.15.42:0/2514216094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d00ffda0 msgr2=0x7ff6d0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d48ac700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d00ffda0 0x7ff6d0101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d48ac700 1 -- 172.21.15.42:0/2514216094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0095f0 con 0x7ff6d0100750 2024-09-17T00:07:22.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.281+0000 7ff6d48ac700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d0100750 0x7ff6d0101d80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000c8f0 tx=0x7ff6c000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:22.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.282+0000 7ff6c67fc700 1 -- 172.21.15.42:0/2514216094 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c00078a0 con 0x7ff6d0100750 2024-09-17T00:07:22.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.282+0000 7ff6c67fc700 1 -- 172.21.15.42:0/2514216094 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6c000f450 con 0x7ff6d0100750 2024-09-17T00:07:22.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.282+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d01a2ae0 con 0x7ff6d0100750 2024-09-17T00:07:22.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.282+0000 7ff6c67fc700 1 -- 172.21.15.42:0/2514216094 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c0018610 con 0x7ff6d0100750 2024-09-17T00:07:22.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.282+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d01a2ef0 con 0x7ff6d0100750 2024-09-17T00:07:22.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.283+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d004f2e0 con 0x7ff6d0100750 2024-09-17T00:07:22.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.283+0000 7ff6c67fc700 1 -- 172.21.15.42:0/2514216094 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff6c000f5c0 con 0x7ff6d0100750 2024-09-17T00:07:22.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.284+0000 7ff6c67fc700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff6bc06e680 0x7ff6bc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:22.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.284+0000 7ff6d50ad700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff6bc06e680 0x7ff6bc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:22.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.284+0000 7ff6c67fc700 1 -- 172.21.15.42:0/2514216094 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff6c008e540 con 0x7ff6d0100750 2024-09-17T00:07:22.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.285+0000 7ff6d50ad700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff6bc06e680 0x7ff6bc070b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006010 tx=0x7ff6cc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:22.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.288+0000 7ff6c67fc700 1 -- 172.21.15.42:0/2514216094 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff6c0057760 con 0x7ff6d0100750 2024-09-17T00:07:22.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.546+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff6d0102d70 con 0x7ff6d0100750 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.546+0000 7ff6c67fc700 1 -- 172.21.15.42:0/2514216094 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7ff6c005ae20 con 0x7ff6d0100750 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:07:22.544 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:07:22.545 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:22.546 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:22.547 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:07:22.547 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:22.547 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:22.547 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:22.547 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:22.547 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:22.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff6bc06e680 msgr2=0x7ff6bc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff6bc06e680 0x7ff6bc070b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006010 tx=0x7ff6cc01a040 comp rx=0 tx=0).stop 2024-09-17T00:07:22.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d0100750 msgr2=0x7ff6d0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d0100750 0x7ff6d0101d80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000c8f0 tx=0x7ff6c000cc00 comp rx=0 tx=0).stop 2024-09-17T00:07:22.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 shutdown_connections 2024-09-17T00:07:22.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff6bc06e680 0x7ff6bc070b40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6d00ffda0 0x7ff6d0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 --2- 172.21.15.42:0/2514216094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6d0100750 0x7ff6d0101d80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.548+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 >> 172.21.15.42:0/2514216094 conn(0x7ff6d00fb650 msgr2=0x7ff6d0103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:22.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.549+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 shutdown_connections 2024-09-17T00:07:22.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.549+0000 7ff6d7311700 1 -- 172.21.15.42:0/2514216094 wait complete. 2024-09-17T00:07:22.549 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 -- 172.21.15.42:0/2892980040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940100ea0 msgr2=0x7f4940101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 --2- 172.21.15.42:0/2892980040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940100ea0 0x7f4940101280 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4934009910 tx=0x7f4934009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 -- 172.21.15.42:0/2892980040 shutdown_connections 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 --2- 172.21.15.42:0/2892980040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940101850 0x7f49401058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 --2- 172.21.15.42:0/2892980040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940100ea0 0x7f4940101280 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 -- 172.21.15.42:0/2892980040 >> 172.21.15.42:0/2892980040 conn(0x7f49400fc750 msgr2=0x7f49400feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 -- 172.21.15.42:0/2892980040 shutdown_connections 2024-09-17T00:07:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.657+0000 7f4946d0d700 1 -- 172.21.15.42:0/2892980040 wait complete. 2024-09-17T00:07:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f4946d0d700 1 Processor -- start 2024-09-17T00:07:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f4946d0d700 1 -- start start 2024-09-17T00:07:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f4946d0d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100ea0 0x7f494019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f4946d0d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940101850 0x7f494019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f4946d0d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494019f360 con 0x7f4940101850 2024-09-17T00:07:22.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f4946d0d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49401988f0 con 0x7f4940100ea0 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f493ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940101850 0x7f494019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f493ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940101850 0x7f494019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60600/0 (socket says 172.21.15.42:60600) 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f493ffff700 1 -- 172.21.15.42:0/1967828546 learned_addr learned my addr 172.21.15.42:0/1967828546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.658+0000 7f4944aa9700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100ea0 0x7f494019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f493ffff700 1 -- 172.21.15.42:0/1967828546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100ea0 msgr2=0x7f494019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f493ffff700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100ea0 0x7f494019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f493ffff700 1 -- 172.21.15.42:0/1967828546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340095f0 con 0x7f4940101850 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f493ffff700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940101850 0x7f494019ed40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f492c00d8d0 tx=0x7f492c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f493dffb700 1 -- 172.21.15.42:0/1967828546 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f492c0041f0 con 0x7f4940101850 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f493dffb700 1 -- 172.21.15.42:0/1967828546 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f492c007620 con 0x7f4940101850 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4940198bd0 con 0x7f4940101850 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f493dffb700 1 -- 172.21.15.42:0/1967828546 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f492c0056b0 con 0x7f4940101850 2024-09-17T00:07:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.659+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4940199120 con 0x7f4940101850 2024-09-17T00:07:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.660+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4940105020 con 0x7f4940101850 2024-09-17T00:07:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.661+0000 7f493dffb700 1 -- 172.21.15.42:0/1967828546 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f492c004750 con 0x7f4940101850 2024-09-17T00:07:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.661+0000 7f493dffb700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f493006e680 0x7f4930070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:22.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.661+0000 7f493dffb700 1 -- 172.21.15.42:0/1967828546 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f492c08e420 con 0x7f4940101850 2024-09-17T00:07:22.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.661+0000 7f4944aa9700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f493006e680 0x7f4930070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:22.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.662+0000 7f4944aa9700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f493006e680 0x7f4930070b40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4934006010 tx=0x7f493401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:22.663 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.666+0000 7f493dffb700 1 -- 172.21.15.42:0/1967828546 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f492c05b170 con 0x7f4940101850 2024-09-17T00:07:22.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.898+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4940199340 con 0x7f493006e680 2024-09-17T00:07:22.895 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:22 smithi042 ceph-mon[111056]: pgmap v473: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:22.896 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:22 smithi042 ceph-mon[111056]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:22.896 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:22 smithi042 ceph-mon[111056]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:22.896 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:22 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/500192719' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:07:22.896 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:22 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2514216094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:07:22.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.899+0000 7f493dffb700 1 -- 172.21.15.42:0/1967828546 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4940199340 con 0x7f493006e680 2024-09-17T00:07:22.896 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:07:22.897 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f493006e680 msgr2=0x7f4930070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f493006e680 0x7f4930070b40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4934006010 tx=0x7f493401a040 comp rx=0 tx=0).stop 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940101850 msgr2=0x7f494019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940101850 0x7f494019ed40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f492c00d8d0 tx=0x7f492c00dbe0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 shutdown_connections 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f493006e680 0x7f4930070b40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4940100ea0 0x7f494019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 --2- 172.21.15.42:0/1967828546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4940101850 0x7f494019ed40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 >> 172.21.15.42:0/1967828546 conn(0x7f49400fc750 msgr2=0x7f49400fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 shutdown_connections 2024-09-17T00:07:22.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:22.901+0000 7f4946d0d700 1 -- 172.21.15.42:0/1967828546 wait complete. 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.010+0000 7f478ebd7700 1 -- 172.21.15.42:0/2607557848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47880ffb40 msgr2=0x7f47880fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.010+0000 7f478ebd7700 1 --2- 172.21.15.42:0/2607557848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47880ffb40 0x7f47880fffc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4774009910 tx=0x7f4774009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 -- 172.21.15.42:0/2607557848 shutdown_connections 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 --2- 172.21.15.42:0/2607557848 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47880ffb40 0x7f47880fffc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 --2- 172.21.15.42:0/2607557848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4788105bf0 0x7f4788105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 -- 172.21.15.42:0/2607557848 >> 172.21.15.42:0/2607557848 conn(0x7f47880fb650 msgr2=0x7f47880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 -- 172.21.15.42:0/2607557848 shutdown_connections 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 -- 172.21.15.42:0/2607557848 wait complete. 2024-09-17T00:07:23.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 Processor -- start 2024-09-17T00:07:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 -- start start 2024-09-17T00:07:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47880ffb40 0x7f47881017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105bf0 0x7f4788101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788102230 con 0x7f4788105bf0 2024-09-17T00:07:23.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.011+0000 7f478ebd7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788102370 con 0x7f47880ffb40 2024-09-17T00:07:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4787fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105bf0 0x7f4788101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4787fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105bf0 0x7f4788101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60612/0 (socket says 172.21.15.42:60612) 2024-09-17T00:07:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4787fff700 1 -- 172.21.15.42:0/1878570686 learned_addr learned my addr 172.21.15.42:0/1878570686 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f478c973700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47880ffb40 0x7f47881017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:23.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4787fff700 1 -- 172.21.15.42:0/1878570686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47880ffb40 msgr2=0x7f47881017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4787fff700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47880ffb40 0x7f47881017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4787fff700 1 -- 172.21.15.42:0/1878570686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47740095f0 con 0x7f4788105bf0 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4787fff700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105bf0 0x7f4788101cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4774006010 tx=0x7f477400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4785ffb700 1 -- 172.21.15.42:0/1878570686 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f477401d070 con 0x7f4788105bf0 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4785ffb700 1 -- 172.21.15.42:0/1878570686 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4774004fb0 con 0x7f4788105bf0 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f4785ffb700 1 -- 172.21.15.42:0/1878570686 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f477400f690 con 0x7f4788105bf0 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.012+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47881025f0 con 0x7f4788105bf0 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.013+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47881a2ce0 con 0x7f4788105bf0 2024-09-17T00:07:23.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.013+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478804f2e0 con 0x7f4788105bf0 2024-09-17T00:07:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.014+0000 7f4785ffb700 1 -- 172.21.15.42:0/1878570686 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f47740228a0 con 0x7f4788105bf0 2024-09-17T00:07:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.015+0000 7f4785ffb700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f477806e680 0x7f4778070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.015+0000 7f4785ffb700 1 -- 172.21.15.42:0/1878570686 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f477408fe10 con 0x7f4788105bf0 2024-09-17T00:07:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.015+0000 7f478c973700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f477806e680 0x7f4778070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:23.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.015+0000 7f478c973700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f477806e680 0x7f4778070b40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f477c005950 tx=0x7f477c005cf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:23.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.019+0000 7f4785ffb700 1 -- 172.21.15.42:0/1878570686 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4774059030 con 0x7f4788105bf0 2024-09-17T00:07:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:22 smithi043 ceph-mon[81510]: pgmap v473: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:22 smithi043 ceph-mon[81510]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:22 smithi043 ceph-mon[81510]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:22 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/500192719' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:07:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:22 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2514216094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:07:23.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.313+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f47881a30e0 con 0x7f4788105bf0 2024-09-17T00:07:23.311 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.314+0000 7f4785ffb700 1 -- 172.21.15.42:0/1878570686 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f4774027020 con 0x7f4788105bf0 2024-09-17T00:07:23.312 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:07:23.312 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:07:23.312 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:07:23.312 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:07:23.312 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:07:23.312 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:07:23.312 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f477806e680 msgr2=0x7f4778070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f477806e680 0x7f4778070b40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f477c005950 tx=0x7f477c005cf0 comp rx=0 tx=0).stop 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105bf0 msgr2=0x7f4788101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105bf0 0x7f4788101cf0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4774006010 tx=0x7f477400be20 comp rx=0 tx=0).stop 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 shutdown_connections 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f477806e680 0x7f4778070b40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47880ffb40 0x7f47881017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 --2- 172.21.15.42:0/1878570686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105bf0 0x7f4788101cf0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 >> 172.21.15.42:0/1878570686 conn(0x7f47880fb650 msgr2=0x7f4788068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 shutdown_connections 2024-09-17T00:07:23.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:23.316+0000 7f478ebd7700 1 -- 172.21.15.42:0/1878570686 wait complete. 2024-09-17T00:07:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:23 smithi042 ceph-mon[111056]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:24.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:23 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1878570686' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:07:24.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:23 smithi043 ceph-mon[81510]: from='client.34916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:23 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1878570686' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:07:25.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:24 smithi042 ceph-mon[111056]: pgmap v474: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:25.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:24 smithi042 ceph-mon[111056]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:24 smithi043 ceph-mon[81510]: pgmap v474: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:24 smithi043 ceph-mon[81510]: from='client.34928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:27.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:26 smithi042 ceph-mon[111056]: pgmap v475: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:27.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:26 smithi043 ceph-mon[81510]: pgmap v475: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:29.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:28 smithi042 ceph-mon[111056]: pgmap v476: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:29.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:28 smithi043 ceph-mon[81510]: pgmap v476: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:30.937 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:30 smithi043 ceph-mon[81510]: pgmap v477: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:31.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:30 smithi042 ceph-mon[111056]: pgmap v477: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:33.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:32 smithi042 ceph-mon[111056]: pgmap v478: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:33.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:32 smithi043 ceph-mon[81510]: pgmap v478: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:35.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:34 smithi042 ceph-mon[111056]: pgmap v479: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:35.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:34 smithi043 ceph-mon[81510]: pgmap v479: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:35.991 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:36.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:36 smithi042 ceph-mon[111056]: pgmap v480: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:36 smithi043 ceph-mon[81510]: pgmap v480: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:39.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:38 smithi042 ceph-mon[111056]: pgmap v481: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:39.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:38 smithi043 ceph-mon[81510]: pgmap v481: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:41.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:40 smithi042 ceph-mon[111056]: pgmap v482: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:41.042 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:40 smithi043 ceph-mon[81510]: pgmap v482: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:43.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:42 smithi042 ceph-mon[111056]: pgmap v483: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:43.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:42 smithi043 ceph-mon[81510]: pgmap v483: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:45.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:44 smithi042 ceph-mon[111056]: pgmap v484: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:45.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:44 smithi043 ceph-mon[81510]: pgmap v484: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:47.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:46 smithi042 ceph-mon[111056]: pgmap v485: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:47.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:46 smithi043 ceph-mon[81510]: pgmap v485: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:49.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:48 smithi042 ceph-mon[111056]: pgmap v486: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:49.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:48 smithi043 ceph-mon[81510]: pgmap v486: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:50 smithi042 ceph-mon[111056]: pgmap v487: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:51.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:50 smithi043 ceph-mon[81510]: pgmap v487: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:07:53.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:52 smithi042 ceph-mon[111056]: pgmap v488: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:53.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:52 smithi043 ceph-mon[81510]: pgmap v488: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 -- 172.21.15.42:0/1173962777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 msgr2=0x7f2a541070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1173962777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a541070b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009910 tx=0x7f2a44009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 -- 172.21.15.42:0/1173962777 shutdown_connections 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1173962777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a54068580 0x7f2a540689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1173962777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a541070b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 -- 172.21.15.42:0/1173962777 >> 172.21.15.42:0/1173962777 conn(0x7f2a54075670 msgr2=0x7f2a54075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 -- 172.21.15.42:0/1173962777 shutdown_connections 2024-09-17T00:07:53.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.441+0000 7f2a59d15700 1 -- 172.21.15.42:0/1173962777 wait complete. 2024-09-17T00:07:53.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.442+0000 7f2a59d15700 1 Processor -- start 2024-09-17T00:07:53.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.442+0000 7f2a59d15700 1 -- start start 2024-09-17T00:07:53.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a59d15700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:53.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a59d15700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:53.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a59d15700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54100770 con 0x7f2a54106cd0 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a59d15700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a541008b0 con 0x7f2a54068580 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a4affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a4affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46712/0 (socket says 172.21.15.42:46712) 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a4affd700 1 -- 172.21.15.42:0/1249909212 learned_addr learned my addr 172.21.15.42:0/1249909212 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a537fe700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a4affd700 1 -- 172.21.15.42:0/1249909212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a54068580 msgr2=0x7f2a540ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a4affd700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a4affd700 1 -- 172.21.15.42:0/1249909212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440095f0 con 0x7f2a54106cd0 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a4affd700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e920 tx=0x7f2a3c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a517fa700 1 -- 172.21.15.42:0/1249909212 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c009790 con 0x7f2a54106cd0 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a517fa700 1 -- 172.21.15.42:0/1249909212 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a3c00cba0 con 0x7f2a54106cd0 2024-09-17T00:07:53.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a541a2a60 con 0x7f2a54106cd0 2024-09-17T00:07:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.443+0000 7f2a517fa700 1 -- 172.21.15.42:0/1249909212 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c018650 con 0x7f2a54106cd0 2024-09-17T00:07:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.444+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a541a2e70 con 0x7f2a54106cd0 2024-09-17T00:07:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.445+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a541091d0 con 0x7f2a54106cd0 2024-09-17T00:07:53.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.446+0000 7f2a517fa700 1 -- 172.21.15.42:0/1249909212 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f2a3c00cd10 con 0x7f2a54106cd0 2024-09-17T00:07:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.446+0000 7f2a517fa700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2a4006e680 0x7f2a40070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.446+0000 7f2a517fa700 1 -- 172.21.15.42:0/1249909212 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2a3c014070 con 0x7f2a54106cd0 2024-09-17T00:07:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.446+0000 7f2a537fe700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2a4006e680 0x7f2a40070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:53.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.447+0000 7f2a537fe700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2a4006e680 0x7f2a40070b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a440059d0 tx=0x7f2a44005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:53.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.451+0000 7f2a517fa700 1 -- 172.21.15.42:0/1249909212 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2a3c0541b0 con 0x7f2a54106cd0 2024-09-17T00:07:53.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.685+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a5402ccc0 con 0x7f2a4006e680 2024-09-17T00:07:53.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.686+0000 7f2a517fa700 1 -- 172.21.15.42:0/1249909212 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2a5402ccc0 con 0x7f2a4006e680 2024-09-17T00:07:53.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2a4006e680 msgr2=0x7f2a40070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2a4006e680 0x7f2a40070b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2a440059d0 tx=0x7f2a44005940 comp rx=0 tx=0).stop 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 msgr2=0x7f2a54100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00e920 tx=0x7f2a3c00ece0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 shutdown_connections 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2a4006e680 0x7f2a40070b40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 --2- 172.21.15.42:0/1249909212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 >> 172.21.15.42:0/1249909212 conn(0x7f2a54075670 msgr2=0x7f2a540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 shutdown_connections 2024-09-17T00:07:53.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.688+0000 7f2a59d15700 1 -- 172.21.15.42:0/1249909212 wait complete. 2024-09-17T00:07:53.697 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:07:53.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 -- 172.21.15.42:0/1239319966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 msgr2=0x7fbe60101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:53.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 --2- 172.21.15.42:0/1239319966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe60101280 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009910 tx=0x7fbe50009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:53.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 -- 172.21.15.42:0/1239319966 shutdown_connections 2024-09-17T00:07:53.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 --2- 172.21.15.42:0/1239319966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe60101850 0x7fbe601058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 --2- 172.21.15.42:0/1239319966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe60101280 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 -- 172.21.15.42:0/1239319966 >> 172.21.15.42:0/1239319966 conn(0x7fbe600fc750 msgr2=0x7fbe600feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:53.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 -- 172.21.15.42:0/1239319966 shutdown_connections 2024-09-17T00:07:53.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.800+0000 7fbe65034700 1 -- 172.21.15.42:0/1239319966 wait complete. 2024-09-17T00:07:53.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe65034700 1 Processor -- start 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe65034700 1 -- start start 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe65034700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe6019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe65034700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe60101850 0x7fbe6019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe65034700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6019f4c0 con 0x7fbe60100ea0 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe65034700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe60198970 con 0x7fbe60101850 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe5ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe6019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe5ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe6019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46724/0 (socket says 172.21.15.42:46724) 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe5ed9d700 1 -- 172.21.15.42:0/1457144316 learned_addr learned my addr 172.21.15.42:0/1457144316 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.801+0000 7fbe5e59c700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe60101850 0x7fbe6019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe5ed9d700 1 -- 172.21.15.42:0/1457144316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe60101850 msgr2=0x7fbe6019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe5ed9d700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe60101850 0x7fbe6019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:53.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe5ed9d700 1 -- 172.21.15.42:0/1457144316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500095f0 con 0x7fbe60100ea0 2024-09-17T00:07:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe5e59c700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe60101850 0x7fbe6019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:07:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe5ed9d700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe6019e8f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbe50000c00 tx=0x7fbe5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe47fff700 1 -- 172.21.15.42:0/1457144316 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5001d070 con 0x7fbe60100ea0 2024-09-17T00:07:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe60198bf0 con 0x7fbe60100ea0 2024-09-17T00:07:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe47fff700 1 -- 172.21.15.42:0/1457144316 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe50005dc0 con 0x7fbe60100ea0 2024-09-17T00:07:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe47fff700 1 -- 172.21.15.42:0/1457144316 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe50017400 con 0x7fbe60100ea0 2024-09-17T00:07:53.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.802+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe601990e0 con 0x7fbe60100ea0 2024-09-17T00:07:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.803+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe60105020 con 0x7fbe60100ea0 2024-09-17T00:07:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.804+0000 7fbe47fff700 1 -- 172.21.15.42:0/1457144316 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fbe5000f7f0 con 0x7fbe60100ea0 2024-09-17T00:07:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.804+0000 7fbe47fff700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbe4806e680 0x7fbe48070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.804+0000 7fbe47fff700 1 -- 172.21.15.42:0/1457144316 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe5008fbe0 con 0x7fbe60100ea0 2024-09-17T00:07:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.804+0000 7fbe5e59c700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbe4806e680 0x7fbe48070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:53.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.805+0000 7fbe5e59c700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbe4806e680 0x7fbe48070b40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe54005950 tx=0x7fbe540058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:53.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:53.808+0000 7fbe47fff700 1 -- 172.21.15.42:0/1457144316 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbe5005c8b0 con 0x7fbe60100ea0 2024-09-17T00:07:54.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:53 smithi042 ceph-mon[111056]: pgmap v489: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:54.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:53 smithi042 ceph-mon[111056]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:54.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.039+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe60199f40 con 0x7fbe4806e680 2024-09-17T00:07:54.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.041+0000 7fbe47fff700 1 -- 172.21.15.42:0/1457144316 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbe60199f40 con 0x7fbe4806e680 2024-09-17T00:07:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbe4806e680 msgr2=0x7fbe48070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbe4806e680 0x7fbe48070b40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbe54005950 tx=0x7fbe540058e0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 msgr2=0x7fbe6019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe6019e8f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbe50000c00 tx=0x7fbe5000be20 comp rx=0 tx=0).stop 2024-09-17T00:07:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 shutdown_connections 2024-09-17T00:07:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fbe4806e680 0x7fbe48070b40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe60100ea0 0x7fbe6019e8f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 --2- 172.21.15.42:0/1457144316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe60101850 0x7fbe6019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 >> 172.21.15.42:0/1457144316 conn(0x7fbe600fc750 msgr2=0x7fbe600fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:54.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 shutdown_connections 2024-09-17T00:07:54.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.043+0000 7fbe65034700 1 -- 172.21.15.42:0/1457144316 wait complete. 2024-09-17T00:07:54.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:53 smithi043 ceph-mon[81510]: pgmap v489: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:54.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:53 smithi043 ceph-mon[81510]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:54.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 -- 172.21.15.42:0/362613242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c101850 msgr2=0x7fe59c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 --2- 172.21.15.42:0/362613242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c101850 0x7fe59c1058a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe590009910 tx=0x7fe590009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:54.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 -- 172.21.15.42:0/362613242 shutdown_connections 2024-09-17T00:07:54.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 --2- 172.21.15.42:0/362613242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c101850 0x7fe59c1058a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 --2- 172.21.15.42:0/362613242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe59c100ea0 0x7fe59c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 -- 172.21.15.42:0/362613242 >> 172.21.15.42:0/362613242 conn(0x7fe59c0fc750 msgr2=0x7fe59c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:54.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 -- 172.21.15.42:0/362613242 shutdown_connections 2024-09-17T00:07:54.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.152+0000 7fe5a1547700 1 -- 172.21.15.42:0/362613242 wait complete. 2024-09-17T00:07:54.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe5a1547700 1 Processor -- start 2024-09-17T00:07:54.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe5a1547700 1 -- start start 2024-09-17T00:07:54.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe5a1547700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c100ea0 0x7fe59c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe5a1547700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe59c101850 0x7fe59c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe5a1547700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c073b10 con 0x7fe59c100ea0 2024-09-17T00:07:54.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe5a1547700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c073c50 con 0x7fe59c101850 2024-09-17T00:07:54.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe59affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c100ea0 0x7fe59c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe59affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c100ea0 0x7fe59c073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46730/0 (socket says 172.21.15.42:46730) 2024-09-17T00:07:54.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe59affd700 1 -- 172.21.15.42:0/4279520337 learned_addr learned my addr 172.21.15.42:0/4279520337 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe59a7fc700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe59c101850 0x7fe59c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe59affd700 1 -- 172.21.15.42:0/4279520337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe59c101850 msgr2=0x7fe59c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe59affd700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe59c101850 0x7fe59c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.153+0000 7fe59affd700 1 -- 172.21.15.42:0/4279520337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900095f0 con 0x7fe59c100ea0 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.154+0000 7fe59affd700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c100ea0 0x7fe59c073090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d8d0 tx=0x7fe58c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.154+0000 7fe583fff700 1 -- 172.21.15.42:0/4279520337 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe58c0098a0 con 0x7fe59c100ea0 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.154+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c1a2b00 con 0x7fe59c100ea0 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.154+0000 7fe583fff700 1 -- 172.21.15.42:0/4279520337 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe58c010460 con 0x7fe59c100ea0 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.154+0000 7fe583fff700 1 -- 172.21.15.42:0/4279520337 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe58c00f5f0 con 0x7fe59c100ea0 2024-09-17T00:07:54.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.154+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c1a2f10 con 0x7fe59c100ea0 2024-09-17T00:07:54.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.155+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c1092a0 con 0x7fe59c100ea0 2024-09-17T00:07:54.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.156+0000 7fe583fff700 1 -- 172.21.15.42:0/4279520337 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe58c009a00 con 0x7fe59c100ea0 2024-09-17T00:07:54.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.156+0000 7fe583fff700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe58406e680 0x7fe584070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.156+0000 7fe583fff700 1 -- 172.21.15.42:0/4279520337 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe58c08e550 con 0x7fe59c100ea0 2024-09-17T00:07:54.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.156+0000 7fe59a7fc700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe58406e680 0x7fe584070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.157+0000 7fe59a7fc700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe58406e680 0x7fe584070b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe59c074880 tx=0x7fe59001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:54.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.160+0000 7fe583fff700 1 -- 172.21.15.42:0/4279520337 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe58c0576f0 con 0x7fe59c100ea0 2024-09-17T00:07:54.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.380+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe59c073e10 con 0x7fe58406e680 2024-09-17T00:07:54.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.390+0000 7fe583fff700 1 -- 172.21.15.42:0/4279520337 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fe59c073e10 con 0x7fe58406e680 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (16m) 10m ago 27m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (27m) 10m ago 27m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (26m) 9m ago 26m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (14m) 10m ago 27m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (14m) 9m ago 26m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (16m) 10m ago 27m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (10m) 10m ago 21m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (10m) 10m ago 21m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (10m) 9m ago 21m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:07:54.388 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (9m) 9m ago 21m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (18m) 10m ago 28m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (17m) 9m ago 26m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (15m) 10m ago 28m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (14m) 9m ago 26m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (17m) 10m ago 27m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (17m) 9m ago 26m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (13m) 10m ago 25m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (13m) 10m ago 24m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (12m) 10m ago 24m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (11m) 9m ago 24m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (11m) 9m ago 23m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (10m) 9m ago 23m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:07:54.389 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (16m) 10m ago 26m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:07:54.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe58406e680 msgr2=0x7fe584070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe58406e680 0x7fe584070b40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe59c074880 tx=0x7fe59001a040 comp rx=0 tx=0).stop 2024-09-17T00:07:54.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c100ea0 msgr2=0x7fe59c073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c100ea0 0x7fe59c073090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00d8d0 tx=0x7fe58c00dbe0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 shutdown_connections 2024-09-17T00:07:54.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe58406e680 0x7fe584070b40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe59c100ea0 0x7fe59c073090 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 --2- 172.21.15.42:0/4279520337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe59c101850 0x7fe59c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 >> 172.21.15.42:0/4279520337 conn(0x7fe59c0fc750 msgr2=0x7fe59c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:54.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.392+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 shutdown_connections 2024-09-17T00:07:54.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.393+0000 7fe5a1547700 1 -- 172.21.15.42:0/4279520337 wait complete. 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 -- 172.21.15.42:0/522294002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa620101ec0 msgr2=0x7fa6201022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 --2- 172.21.15.42:0/522294002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa620101ec0 0x7fa6201022a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa610009940 tx=0x7fa610009c50 comp rx=0 tx=0).stop 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 -- 172.21.15.42:0/522294002 shutdown_connections 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 --2- 172.21.15.42:0/522294002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6201027e0 0x7fa62010acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 --2- 172.21.15.42:0/522294002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa620101ec0 0x7fa6201022a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 -- 172.21.15.42:0/522294002 >> 172.21.15.42:0/522294002 conn(0x7fa6200fb650 msgr2=0x7fa6200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 -- 172.21.15.42:0/522294002 shutdown_connections 2024-09-17T00:07:54.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.499+0000 7fa62509b700 1 -- 172.21.15.42:0/522294002 wait complete. 2024-09-17T00:07:54.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.500+0000 7fa62509b700 1 Processor -- start 2024-09-17T00:07:54.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.500+0000 7fa62509b700 1 -- start start 2024-09-17T00:07:54.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.500+0000 7fa62509b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa620101ec0 0x7fa6200ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.500+0000 7fa62509b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6201027e0 0x7fa620100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.500+0000 7fa62509b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6201005a0 con 0x7fa6201027e0 2024-09-17T00:07:54.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.500+0000 7fa62509b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6201006e0 con 0x7fa620101ec0 2024-09-17T00:07:54.499 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6201027e0 0x7fa620100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6201027e0 0x7fa620100060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46752/0 (socket says 172.21.15.42:46752) 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61e59c700 1 -- 172.21.15.42:0/1945916562 learned_addr learned my addr 172.21.15.42:0/1945916562 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61ed9d700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa620101ec0 0x7fa6200ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61e59c700 1 -- 172.21.15.42:0/1945916562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa620101ec0 msgr2=0x7fa6200ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61e59c700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa620101ec0 0x7fa6200ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61e59c700 1 -- 172.21.15.42:0/1945916562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100095f0 con 0x7fa6201027e0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa61e59c700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6201027e0 0x7fa620100060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa61400b480 tx=0x7fa61400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa607fff700 1 -- 172.21.15.42:0/1945916562 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6140118c0 con 0x7fa6201027e0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6201a2b90 con 0x7fa6201027e0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa607fff700 1 -- 172.21.15.42:0/1945916562 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa614010380 con 0x7fa6201027e0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa607fff700 1 -- 172.21.15.42:0/1945916562 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa61400f4f0 con 0x7fa6201027e0 2024-09-17T00:07:54.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.501+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6201a2fa0 con 0x7fa6201027e0 2024-09-17T00:07:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.502+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6201083a0 con 0x7fa6201027e0 2024-09-17T00:07:54.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.503+0000 7fa607fff700 1 -- 172.21.15.42:0/1945916562 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa6140104f0 con 0x7fa6201027e0 2024-09-17T00:07:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.503+0000 7fa607fff700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa60806e680 0x7fa608070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.504+0000 7fa607fff700 1 -- 172.21.15.42:0/1945916562 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa61408e1b0 con 0x7fa6201027e0 2024-09-17T00:07:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.504+0000 7fa61ed9d700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa60806e680 0x7fa608070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.504+0000 7fa61ed9d700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa60806e680 0x7fa608070b40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa61000b870 tx=0x7fa610005fb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:54.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.508+0000 7fa607fff700 1 -- 172.21.15.42:0/1945916562 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa6140573d0 con 0x7fa6201027e0 2024-09-17T00:07:54.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.805+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa6201008a0 con 0x7fa6201027e0 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.805+0000 7fa607fff700 1 -- 172.21.15.42:0/1945916562 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa61405aa90 con 0x7fa6201027e0 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:07:54.803 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:07:54.804 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:07:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa60806e680 msgr2=0x7fa608070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa60806e680 0x7fa608070b40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa61000b870 tx=0x7fa610005fb0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6201027e0 msgr2=0x7fa620100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6201027e0 0x7fa620100060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa61400b480 tx=0x7fa61400b840 comp rx=0 tx=0).stop 2024-09-17T00:07:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 shutdown_connections 2024-09-17T00:07:54.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa60806e680 0x7fa608070b40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa620101ec0 0x7fa6200ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 --2- 172.21.15.42:0/1945916562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6201027e0 0x7fa620100060 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 >> 172.21.15.42:0/1945916562 conn(0x7fa6200fb650 msgr2=0x7fa6201054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 shutdown_connections 2024-09-17T00:07:54.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.807+0000 7fa62509b700 1 -- 172.21.15.42:0/1945916562 wait complete. 2024-09-17T00:07:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 -- 172.21.15.42:0/3994894686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc100ea0 msgr2=0x7f82cc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 --2- 172.21.15.42:0/3994894686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc100ea0 0x7f82cc101280 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009910 tx=0x7f82c0009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:54.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 -- 172.21.15.42:0/3994894686 shutdown_connections 2024-09-17T00:07:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 --2- 172.21.15.42:0/3994894686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82cc101850 0x7f82cc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 --2- 172.21.15.42:0/3994894686 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc100ea0 0x7f82cc101280 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 -- 172.21.15.42:0/3994894686 >> 172.21.15.42:0/3994894686 conn(0x7f82cc0fc750 msgr2=0x7f82cc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 -- 172.21.15.42:0/3994894686 shutdown_connections 2024-09-17T00:07:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 -- 172.21.15.42:0/3994894686 wait complete. 2024-09-17T00:07:54.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 Processor -- start 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.916+0000 7f82d3033700 1 -- start start 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82d3033700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82cc100ea0 0x7f82cc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82d3033700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc101850 0x7f82cc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82d3033700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc19d2c0 con 0x7f82cc101850 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82d3033700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc196770 con 0x7f82cc100ea0 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82cbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc101850 0x7f82cc19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82cbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc101850 0x7f82cc19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46774/0 (socket says 172.21.15.42:46774) 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82cbfff700 1 -- 172.21.15.42:0/2337534695 learned_addr learned my addr 172.21.15.42:0/2337534695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82d0dcf700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82cc100ea0 0x7f82cc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82cbfff700 1 -- 172.21.15.42:0/2337534695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82cc100ea0 msgr2=0x7f82cc19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82cbfff700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82cc100ea0 0x7f82cc19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82cbfff700 1 -- 172.21.15.42:0/2337534695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c00095f0 con 0x7f82cc101850 2024-09-17T00:07:54.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82cbfff700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc101850 0x7f82cc19cc30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f82b800d900 tx=0x7f82b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82c9ffb700 1 -- 172.21.15.42:0/2337534695 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b8009900 con 0x7f82cc101850 2024-09-17T00:07:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82c9ffb700 1 -- 172.21.15.42:0/2337534695 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82b8010460 con 0x7f82cc101850 2024-09-17T00:07:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc196a50 con 0x7f82cc101850 2024-09-17T00:07:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.917+0000 7f82c9ffb700 1 -- 172.21.15.42:0/2337534695 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b800f5f0 con 0x7f82cc101850 2024-09-17T00:07:54.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.918+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc196fa0 con 0x7f82cc101850 2024-09-17T00:07:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.919+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc109240 con 0x7f82cc101850 2024-09-17T00:07:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.919+0000 7f82c9ffb700 1 -- 172.21.15.42:0/2337534695 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f82b80105d0 con 0x7f82cc101850 2024-09-17T00:07:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.920+0000 7f82c9ffb700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f82bc06e680 0x7f82bc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.920+0000 7f82c9ffb700 1 -- 172.21.15.42:0/2337534695 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f82b808e560 con 0x7f82cc101850 2024-09-17T00:07:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.920+0000 7f82d0dcf700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f82bc06e680 0x7f82bc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:54.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.920+0000 7f82d0dcf700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f82bc06e680 0x7f82bc070b40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f82c0006010 tx=0x7f82c001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:54.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:54.924+0000 7f82c9ffb700 1 -- 172.21.15.42:0/2337534695 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f82b805b1d0 con 0x7f82cc101850 2024-09-17T00:07:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:54 smithi042 ceph-mon[111056]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:55.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:54 smithi042 ceph-mon[111056]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:55.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:54 smithi043 ceph-mon[81510]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:55.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:54 smithi043 ceph-mon[81510]: from='client.34944 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.177+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f82cc197d20 con 0x7f82cc101850 2024-09-17T00:07:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.178+0000 7f82c9ffb700 1 -- 172.21.15.42:0/2337534695 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f82b805ad60 con 0x7f82cc101850 2024-09-17T00:07:55.175 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:07:55.176 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:07:55.177 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:55.178 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:07:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f82bc06e680 msgr2=0x7f82bc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f82bc06e680 0x7f82bc070b40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f82c0006010 tx=0x7f82c001a040 comp rx=0 tx=0).stop 2024-09-17T00:07:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc101850 msgr2=0x7f82cc19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc101850 0x7f82cc19cc30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f82b800d900 tx=0x7f82b800dcc0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 shutdown_connections 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f82bc06e680 0x7f82bc070b40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82cc100ea0 0x7f82cc19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 --2- 172.21.15.42:0/2337534695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82cc101850 0x7f82cc19cc30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 >> 172.21.15.42:0/2337534695 conn(0x7f82cc0fc750 msgr2=0x7f82cc0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 shutdown_connections 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.180+0000 7f82d3033700 1 -- 172.21.15.42:0/2337534695 wait complete. 2024-09-17T00:07:55.180 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:07:55.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.292+0000 7fed37f79700 1 -- 172.21.15.42:0/1474445900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30102700 msgr2=0x7fed3010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.292+0000 7fed37f79700 1 --2- 172.21.15.42:0/1474445900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30102700 0x7fed3010abf0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fed20009910 tx=0x7fed20009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:55.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 -- 172.21.15.42:0/1474445900 shutdown_connections 2024-09-17T00:07:55.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 --2- 172.21.15.42:0/1474445900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30102700 0x7fed3010abf0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 --2- 172.21.15.42:0/1474445900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed30101de0 0x7fed301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 -- 172.21.15.42:0/1474445900 >> 172.21.15.42:0/1474445900 conn(0x7fed300fb610 msgr2=0x7fed300fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 -- 172.21.15.42:0/1474445900 shutdown_connections 2024-09-17T00:07:55.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 -- 172.21.15.42:0/1474445900 wait complete. 2024-09-17T00:07:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 Processor -- start 2024-09-17T00:07:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.293+0000 7fed37f79700 1 -- start start 2024-09-17T00:07:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed37f79700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30101de0 0x7fed300ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:55.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed37f79700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed30102700 0x7fed300fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed37f79700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301004c0 con 0x7fed30101de0 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed37f79700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed30100600 con 0x7fed30102700 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35d15700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30101de0 0x7fed300ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35d15700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30101de0 0x7fed300ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46794/0 (socket says 172.21.15.42:46794) 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35d15700 1 -- 172.21.15.42:0/36479743 learned_addr learned my addr 172.21.15.42:0/36479743 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35514700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed30102700 0x7fed300fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35d15700 1 -- 172.21.15.42:0/36479743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed30102700 msgr2=0x7fed300fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35d15700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed30102700 0x7fed300fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35d15700 1 -- 172.21.15.42:0/36479743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200095f0 con 0x7fed30101de0 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed35d15700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30101de0 0x7fed300ffa40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00d8d0 tx=0x7fed2c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.294+0000 7fed26ffd700 1 -- 172.21.15.42:0/36479743 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2c0098a0 con 0x7fed30101de0 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.295+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed301a2aa0 con 0x7fed30101de0 2024-09-17T00:07:55.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.295+0000 7fed26ffd700 1 -- 172.21.15.42:0/36479743 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed2c010460 con 0x7fed30101de0 2024-09-17T00:07:55.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.295+0000 7fed26ffd700 1 -- 172.21.15.42:0/36479743 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed2c00f5f0 con 0x7fed30101de0 2024-09-17T00:07:55.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.295+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed301a2e50 con 0x7fed30101de0 2024-09-17T00:07:55.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.296+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed301082f0 con 0x7fed30101de0 2024-09-17T00:07:55.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.296+0000 7fed26ffd700 1 -- 172.21.15.42:0/36479743 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fed2c009a00 con 0x7fed30101de0 2024-09-17T00:07:55.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.297+0000 7fed26ffd700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed1c06e680 0x7fed1c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:55.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.297+0000 7fed26ffd700 1 -- 172.21.15.42:0/36479743 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fed2c08e3c0 con 0x7fed30101de0 2024-09-17T00:07:55.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.297+0000 7fed35514700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed1c06e680 0x7fed1c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:55.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.297+0000 7fed35514700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed1c06e680 0x7fed1c070b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed301011e0 tx=0x7fed2001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:55.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.301+0000 7fed26ffd700 1 -- 172.21.15.42:0/36479743 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fed2c0575e0 con 0x7fed30101de0 2024-09-17T00:07:55.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.534+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed300611d0 con 0x7fed1c06e680 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.536+0000 7fed26ffd700 1 -- 172.21.15.42:0/36479743 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fed300611d0 con 0x7fed1c06e680 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:07:55.533 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:07:55.534 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.539+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed1c06e680 msgr2=0x7fed1c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.539+0000 7fed37f79700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed1c06e680 0x7fed1c070b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed301011e0 tx=0x7fed2001a040 comp rx=0 tx=0).stop 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.539+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30101de0 msgr2=0x7fed300ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.539+0000 7fed37f79700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30101de0 0x7fed300ffa40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00d8d0 tx=0x7fed2c00dbe0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.540+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 shutdown_connections 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.540+0000 7fed37f79700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed1c06e680 0x7fed1c070b40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.540+0000 7fed37f79700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed30101de0 0x7fed300ffa40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.540+0000 7fed37f79700 1 --2- 172.21.15.42:0/36479743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed30102700 0x7fed300fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.540+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 >> 172.21.15.42:0/36479743 conn(0x7fed300fb610 msgr2=0x7fed30105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.540+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 shutdown_connections 2024-09-17T00:07:55.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.540+0000 7fed37f79700 1 -- 172.21.15.42:0/36479743 wait complete. 2024-09-17T00:07:55.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 -- 172.21.15.42:0/4150546366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 msgr2=0x7f34c810d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 --2- 172.21.15.42:0/4150546366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c810d000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f34b8009910 tx=0x7f34b8009c20 comp rx=0 tx=0).stop 2024-09-17T00:07:55.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 -- 172.21.15.42:0/4150546366 shutdown_connections 2024-09-17T00:07:55.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 --2- 172.21.15.42:0/4150546366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c810d000 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 --2- 172.21.15.42:0/4150546366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80ff990 0x7f34c80ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 -- 172.21.15.42:0/4150546366 >> 172.21.15.42:0/4150546366 conn(0x7f34c80fb650 msgr2=0x7f34c80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:55.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 -- 172.21.15.42:0/4150546366 shutdown_connections 2024-09-17T00:07:55.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 -- 172.21.15.42:0/4150546366 wait complete. 2024-09-17T00:07:55.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.655+0000 7f34cf461700 1 Processor -- start 2024-09-17T00:07:55.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cf461700 1 -- start start 2024-09-17T00:07:55.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cf461700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80ff990 0x7f34c819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:55.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cf461700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:55.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cf461700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c819f430 con 0x7f34c8100340 2024-09-17T00:07:55.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cf461700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c81988e0 con 0x7f34c80ff990 2024-09-17T00:07:55.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cc9fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:55.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cc9fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46812/0 (socket says 172.21.15.42:46812) 2024-09-17T00:07:55.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cc9fc700 1 -- 172.21.15.42:0/4080301275 learned_addr learned my addr 172.21.15.42:0/4080301275 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:07:55.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cd1fd700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80ff990 0x7f34c819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:55.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cc9fc700 1 -- 172.21.15.42:0/4080301275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80ff990 msgr2=0x7f34c819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cc9fc700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80ff990 0x7f34c819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cc9fc700 1 -- 172.21.15.42:0/4080301275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80095f0 con 0x7f34c8100340 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.656+0000 7f34cc9fc700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c819eda0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f34b800b770 tx=0x7f34b800bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.657+0000 7f34be7fc700 1 -- 172.21.15.42:0/4080301275 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b801d070 con 0x7f34c8100340 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.657+0000 7f34be7fc700 1 -- 172.21.15.42:0/4080301275 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34b80052a0 con 0x7f34c8100340 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.657+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c8198b60 con 0x7f34c8100340 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.657+0000 7f34be7fc700 1 -- 172.21.15.42:0/4080301275 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34b8022470 con 0x7f34c8100340 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.657+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c8199050 con 0x7f34c8100340 2024-09-17T00:07:55.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.658+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c810a770 con 0x7f34c8100340 2024-09-17T00:07:55.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.659+0000 7f34be7fc700 1 -- 172.21.15.42:0/4080301275 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f34b800f850 con 0x7f34c8100340 2024-09-17T00:07:55.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.659+0000 7f34be7fc700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f34b406e680 0x7f34b4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:07:55.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.659+0000 7f34be7fc700 1 -- 172.21.15.42:0/4080301275 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f34b808fb60 con 0x7f34c8100340 2024-09-17T00:07:55.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.660+0000 7f34cd1fd700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f34b406e680 0x7f34b4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:07:55.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.660+0000 7f34cd1fd700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f34b406e680 0x7f34b4070b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009c60 tx=0x7f34c4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:07:55.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.664+0000 7f34be7fc700 1 -- 172.21.15.42:0/4080301275 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f34b805c830 con 0x7f34c8100340 2024-09-17T00:07:55.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.966+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f34c804f2e0 con 0x7f34c8100340 2024-09-17T00:07:55.965 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:55 smithi042 ceph-mon[111056]: pgmap v490: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:55.965 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:55 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1945916562' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:07:55.965 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:55 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2337534695' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:07:55.965 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:55 smithi042 ceph-mon[111056]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:55.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.967+0000 7f34be7fc700 1 -- 172.21.15.42:0/4080301275 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f34b805c3c0 con 0x7f34c8100340 2024-09-17T00:07:55.966 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:07:55.966 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:07:55.966 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:07:55.966 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:07:55.966 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:07:55.966 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:07:55.966 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:07:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f34b406e680 msgr2=0x7f34b4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f34b406e680 0x7f34b4070b40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009c60 tx=0x7f34c4009260 comp rx=0 tx=0).stop 2024-09-17T00:07:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 msgr2=0x7f34c819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:07:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c819eda0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f34b800b770 tx=0x7f34b800bf30 comp rx=0 tx=0).stop 2024-09-17T00:07:55.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 shutdown_connections 2024-09-17T00:07:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f34b406e680 0x7f34b4070b40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34c80ff990 0x7f34c819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 --2- 172.21.15.42:0/4080301275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34c8100340 0x7f34c819eda0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:07:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 >> 172.21.15.42:0/4080301275 conn(0x7f34c80fb650 msgr2=0x7f34c80fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:07:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 shutdown_connections 2024-09-17T00:07:55.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:07:55.970+0000 7f34cf461700 1 -- 172.21.15.42:0/4080301275 wait complete. 2024-09-17T00:07:56.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:55 smithi043 ceph-mon[81510]: pgmap v490: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:56.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:55 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1945916562' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:07:56.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:55 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2337534695' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:07:56.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:55 smithi043 ceph-mon[81510]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:07:57.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:56 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4080301275' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:07:57.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:56 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4080301275' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:07:58.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:57 smithi042 ceph-mon[111056]: pgmap v491: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:07:58.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:57 smithi043 ceph-mon[81510]: pgmap v491: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:00.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:07:59 smithi043 ceph-mon[81510]: pgmap v492: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:00.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:07:59 smithi042 ceph-mon[111056]: pgmap v492: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:02.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:01 smithi043 ceph-mon[81510]: pgmap v493: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:02.195 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:01 smithi042 ceph-mon[111056]: pgmap v493: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:04.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:03 smithi043 ceph-mon[81510]: pgmap v494: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:04.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:03 smithi042 ceph-mon[111056]: pgmap v494: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:06.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:05 smithi043 ceph-mon[81510]: pgmap v495: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:06.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:05 smithi042 ceph-mon[111056]: pgmap v495: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:06.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:08.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:07 smithi043 ceph-mon[81510]: pgmap v496: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:08.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:07 smithi042 ceph-mon[111056]: pgmap v496: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:10.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:09 smithi043 ceph-mon[81510]: pgmap v497: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:10.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:09 smithi042 ceph-mon[111056]: pgmap v497: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:11 smithi043 ceph-mon[81510]: pgmap v498: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:12.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:11 smithi042 ceph-mon[111056]: pgmap v498: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:13 smithi043 ceph-mon[81510]: pgmap v499: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:14.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:13 smithi042 ceph-mon[111056]: pgmap v499: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:15 smithi043 ceph-mon[81510]: pgmap v500: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:16.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:15 smithi042 ceph-mon[111056]: pgmap v500: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:18.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:17 smithi042 ceph-mon[111056]: pgmap v501: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:18.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:17 smithi043 ceph-mon[81510]: pgmap v501: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:20.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:19 smithi042 ceph-mon[111056]: pgmap v502: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:20.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:19 smithi043 ceph-mon[81510]: pgmap v502: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:21.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:21.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:22.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:21 smithi042 ceph-mon[111056]: pgmap v503: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:22.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:21 smithi043 ceph-mon[81510]: pgmap v503: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:24.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:23 smithi042 ceph-mon[111056]: pgmap v504: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:24.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:23 smithi043 ceph-mon[81510]: pgmap v504: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:26.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 -- 172.21.15.42:0/4176057775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0101850 msgr2=0x7f6eb01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/4176057775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0101850 0x7f6eb01058a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009910 tx=0x7f6ea4009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:26.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 -- 172.21.15.42:0/4176057775 shutdown_connections 2024-09-17T00:08:26.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/4176057775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0101850 0x7f6eb01058a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/4176057775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb0100ea0 0x7f6eb0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 -- 172.21.15.42:0/4176057775 >> 172.21.15.42:0/4176057775 conn(0x7f6eb00fc750 msgr2=0x7f6eb00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:26.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 -- 172.21.15.42:0/4176057775 shutdown_connections 2024-09-17T00:08:26.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 -- 172.21.15.42:0/4176057775 wait complete. 2024-09-17T00:08:26.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.094+0000 7f6eb52f2700 1 Processor -- start 2024-09-17T00:08:26.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eb52f2700 1 -- start start 2024-09-17T00:08:26.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eb52f2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0100ea0 0x7f6eb019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eb52f2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb0101850 0x7f6eb019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eb52f2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb019d2d0 con 0x7f6eb0100ea0 2024-09-17T00:08:26.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eb52f2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0196780 con 0x7f6eb0101850 2024-09-17T00:08:26.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eaeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0100ea0 0x7f6eb019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eaeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0100ea0 0x7f6eb019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43314/0 (socket says 172.21.15.42:43314) 2024-09-17T00:08:26.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eaeffd700 1 -- 172.21.15.42:0/2828520748 learned_addr learned my addr 172.21.15.42:0/2828520748 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eae7fc700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb0101850 0x7f6eb019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eaeffd700 1 -- 172.21.15.42:0/2828520748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb0101850 msgr2=0x7f6eb019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eaeffd700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb0101850 0x7f6eb019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eaeffd700 1 -- 172.21.15.42:0/2828520748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40095f0 con 0x7f6eb0100ea0 2024-09-17T00:08:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.095+0000 7f6eae7fc700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb0101850 0x7f6eb019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T00:08:26.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.096+0000 7f6eaeffd700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0100ea0 0x7f6eb019c700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000e920 tx=0x7f6ea000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:26.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.096+0000 7f6e97fff700 1 -- 172.21.15.42:0/2828520748 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ea000c9d0 con 0x7f6eb0100ea0 2024-09-17T00:08:26.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.096+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb0196a60 con 0x7f6eb0100ea0 2024-09-17T00:08:26.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.096+0000 7f6e97fff700 1 -- 172.21.15.42:0/2828520748 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ea0010430 con 0x7f6eb0100ea0 2024-09-17T00:08:26.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.096+0000 7f6e97fff700 1 -- 172.21.15.42:0/2828520748 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ea0018630 con 0x7f6eb0100ea0 2024-09-17T00:08:26.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.096+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb0196f80 con 0x7f6eb0100ea0 2024-09-17T00:08:26.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.097+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb01092a0 con 0x7f6eb0100ea0 2024-09-17T00:08:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.098+0000 7f6e97fff700 1 -- 172.21.15.42:0/2828520748 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6ea00105a0 con 0x7f6eb0100ea0 2024-09-17T00:08:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.098+0000 7f6e97fff700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9806e680 0x7f6e98070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.098+0000 7f6e97fff700 1 -- 172.21.15.42:0/2828520748 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6ea0014070 con 0x7f6eb0100ea0 2024-09-17T00:08:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.098+0000 7f6eae7fc700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9806e680 0x7f6e98070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.099+0000 7f6eae7fc700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9806e680 0x7f6e98070b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005b40 tx=0x7f6ea4005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:26.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.103+0000 7f6e97fff700 1 -- 172.21.15.42:0/2828520748 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6ea00541b0 con 0x7f6eb0100ea0 2024-09-17T00:08:26.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:25 smithi042 ceph-mon[111056]: pgmap v505: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:26.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.335+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb0197d00 con 0x7f6e9806e680 2024-09-17T00:08:26.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.337+0000 7f6e97fff700 1 -- 172.21.15.42:0/2828520748 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6eb0197d00 con 0x7f6e9806e680 2024-09-17T00:08:26.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9806e680 msgr2=0x7f6e98070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9806e680 0x7f6e98070b40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4005b40 tx=0x7f6ea4005a90 comp rx=0 tx=0).stop 2024-09-17T00:08:26.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0100ea0 msgr2=0x7f6eb019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0100ea0 0x7f6eb019c700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000e920 tx=0x7f6ea000ec30 comp rx=0 tx=0).stop 2024-09-17T00:08:26.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 shutdown_connections 2024-09-17T00:08:26.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e9806e680 0x7f6e98070b40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6eb0100ea0 0x7f6eb019c700 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 --2- 172.21.15.42:0/2828520748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6eb0101850 0x7f6eb019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.339+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 >> 172.21.15.42:0/2828520748 conn(0x7f6eb00fc750 msgr2=0x7f6eb00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:26.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.340+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 shutdown_connections 2024-09-17T00:08:26.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.340+0000 7f6eb52f2700 1 -- 172.21.15.42:0/2828520748 wait complete. 2024-09-17T00:08:26.347 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:08:26.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:25 smithi043 ceph-mon[81510]: pgmap v505: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 -- 172.21.15.42:0/1695211221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 msgr2=0x7febc01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 --2- 172.21.15.42:0/1695211221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc01058a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7febb4009910 tx=0x7febb4009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 -- 172.21.15.42:0/1695211221 shutdown_connections 2024-09-17T00:08:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 --2- 172.21.15.42:0/1695211221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc01058a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 --2- 172.21.15.42:0/1695211221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0100ea0 0x7febc0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 -- 172.21.15.42:0/1695211221 >> 172.21.15.42:0/1695211221 conn(0x7febc00fc750 msgr2=0x7febc00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:26.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 -- 172.21.15.42:0/1695211221 shutdown_connections 2024-09-17T00:08:26.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.448+0000 7febc641e700 1 -- 172.21.15.42:0/1695211221 wait complete. 2024-09-17T00:08:26.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febc641e700 1 Processor -- start 2024-09-17T00:08:26.447 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febc641e700 1 -- start start 2024-09-17T00:08:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febc641e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0100ea0 0x7febc019c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febc641e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febc641e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019d210 con 0x7febc0101850 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febc641e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc01966c0 con 0x7febc0100ea0 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febbf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febbf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43330/0 (socket says 172.21.15.42:43330) 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febbf7fe700 1 -- 172.21.15.42:0/3361938730 learned_addr learned my addr 172.21.15.42:0/3361938730 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.449+0000 7febbffff700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0100ea0 0x7febc019c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febbf7fe700 1 -- 172.21.15.42:0/3361938730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0100ea0 msgr2=0x7febc019c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febbf7fe700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0100ea0 0x7febc019c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febbf7fe700 1 -- 172.21.15.42:0/3361938730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb40095f0 con 0x7febc0101850 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febbf7fe700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc019cb80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7febb400bda0 tx=0x7febb400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febbd7fa700 1 -- 172.21.15.42:0/3361938730 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febb401d070 con 0x7febc0101850 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febbd7fa700 1 -- 172.21.15.42:0/3361938730 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febb4005dc0 con 0x7febc0101850 2024-09-17T00:08:26.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc0196940 con 0x7febc0101850 2024-09-17T00:08:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febbd7fa700 1 -- 172.21.15.42:0/3361938730 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febb4017400 con 0x7febc0101850 2024-09-17T00:08:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.450+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc0196e30 con 0x7febc0101850 2024-09-17T00:08:26.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.451+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc01092a0 con 0x7febc0101850 2024-09-17T00:08:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.452+0000 7febbd7fa700 1 -- 172.21.15.42:0/3361938730 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7febb400f6f0 con 0x7febc0101850 2024-09-17T00:08:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.452+0000 7febbd7fa700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7feba806e680 0x7feba8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.452+0000 7febbd7fa700 1 -- 172.21.15.42:0/3361938730 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7febb408fa30 con 0x7febc0101850 2024-09-17T00:08:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.452+0000 7febbffff700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7feba806e680 0x7feba8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.453+0000 7febbffff700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7feba806e680 0x7feba8070b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7febb0005950 tx=0x7febb000b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:26.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.457+0000 7febbd7fa700 1 -- 172.21.15.42:0/3361938730 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7febb405c700 con 0x7febc0101850 2024-09-17T00:08:26.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.689+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febc0197050 con 0x7feba806e680 2024-09-17T00:08:26.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.690+0000 7febbd7fa700 1 -- 172.21.15.42:0/3361938730 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7febc0197050 con 0x7feba806e680 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7feba806e680 msgr2=0x7feba8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7feba806e680 0x7feba8070b40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7febb0005950 tx=0x7febb000b410 comp rx=0 tx=0).stop 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 msgr2=0x7febc019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc019cb80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7febb400bda0 tx=0x7febb400bdd0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 shutdown_connections 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7feba806e680 0x7feba8070b40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7febc0100ea0 0x7febc019c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 --2- 172.21.15.42:0/3361938730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febc0101850 0x7febc019cb80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.693+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 >> 172.21.15.42:0/3361938730 conn(0x7febc00fc750 msgr2=0x7febc00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:26.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.694+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 shutdown_connections 2024-09-17T00:08:26.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.694+0000 7febc641e700 1 -- 172.21.15.42:0/3361938730 wait complete. 2024-09-17T00:08:26.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 -- 172.21.15.42:0/596269563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 msgr2=0x7fcc301047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/596269563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc301047a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009910 tx=0x7fcc20009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:26.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 -- 172.21.15.42:0/596269563 shutdown_connections 2024-09-17T00:08:26.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/596269563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc301047a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/596269563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc300ffda0 0x7fcc30100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 -- 172.21.15.42:0/596269563 >> 172.21.15.42:0/596269563 conn(0x7fcc300fb650 msgr2=0x7fcc300fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:26.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 -- 172.21.15.42:0/596269563 shutdown_connections 2024-09-17T00:08:26.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 -- 172.21.15.42:0/596269563 wait complete. 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 Processor -- start 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.802+0000 7fcc37fe4700 1 -- start start 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc37fe4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc300ffda0 0x7fcc30101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc37fe4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc30101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc37fe4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc301022c0 con 0x7fcc30100750 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc37fe4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc30102400 con 0x7fcc300ffda0 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc3557f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc30101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc3557f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc30101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43346/0 (socket says 172.21.15.42:43346) 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc3557f700 1 -- 172.21.15.42:0/3623303467 learned_addr learned my addr 172.21.15.42:0/3623303467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc35d80700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc300ffda0 0x7fcc30101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc3557f700 1 -- 172.21.15.42:0/3623303467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc300ffda0 msgr2=0x7fcc30101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc3557f700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc300ffda0 0x7fcc30101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:26.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc3557f700 1 -- 172.21.15.42:0/3623303467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc200095f0 con 0x7fcc30100750 2024-09-17T00:08:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc3557f700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc30101d80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc200092e0 tx=0x7fcc2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc26ffd700 1 -- 172.21.15.42:0/3623303467 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc2001d070 con 0x7fcc30100750 2024-09-17T00:08:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc26ffd700 1 -- 172.21.15.42:0/3623303467 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc2000f460 con 0x7fcc30100750 2024-09-17T00:08:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc30102680 con 0x7fcc30100750 2024-09-17T00:08:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.803+0000 7fcc26ffd700 1 -- 172.21.15.42:0/3623303467 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc200175f0 con 0x7fcc30100750 2024-09-17T00:08:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.804+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc301a2da0 con 0x7fcc30100750 2024-09-17T00:08:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.804+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc301091b0 con 0x7fcc30100750 2024-09-17T00:08:26.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.805+0000 7fcc26ffd700 1 -- 172.21.15.42:0/3623303467 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fcc2000f5d0 con 0x7fcc30100750 2024-09-17T00:08:26.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.805+0000 7fcc26ffd700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc1c06e680 0x7fcc1c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:26.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.806+0000 7fcc26ffd700 1 -- 172.21.15.42:0/3623303467 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcc2008fb70 con 0x7fcc30100750 2024-09-17T00:08:26.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.806+0000 7fcc35d80700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc1c06e680 0x7fcc1c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:26.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.806+0000 7fcc35d80700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc1c06e680 0x7fcc1c070b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005fd0 tx=0x7fcc2c00a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:26.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:26.810+0000 7fcc26ffd700 1 -- 172.21.15.42:0/3623303467 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcc20058d10 con 0x7fcc30100750 2024-09-17T00:08:27.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:26 smithi042 ceph-mon[111056]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:27.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:26 smithi042 ceph-mon[111056]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:27.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.029+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc300611d0 con 0x7fcc1c06e680 2024-09-17T00:08:27.036 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.038+0000 7fcc26ffd700 1 -- 172.21.15.42:0/3623303467 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fcc300611d0 con 0x7fcc1c06e680 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (17m) 10m ago 27m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (28m) 10m ago 28m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (27m) 10m ago 27m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (14m) 10m ago 28m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (14m) 10m ago 26m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (16m) 10m ago 27m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:08:27.037 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (10m) 10m ago 22m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (11m) 10m ago 22m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (10m) 10m ago 22m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (10m) 10m ago 22m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (18m) 10m ago 29m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (17m) 10m ago 26m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (15m) 10m ago 29m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (15m) 10m ago 26m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (17m) 10m ago 27m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (17m) 10m ago 26m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (14m) 10m ago 25m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (13m) 10m ago 25m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (13m) 10m ago 25m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:08:27.038 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (12m) 10m ago 24m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:08:27.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (11m) 10m ago 24m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:08:27.039 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (11m) 10m ago 23m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:08:27.039 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (17m) 10m ago 27m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:08:27.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc1c06e680 msgr2=0x7fcc1c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc1c06e680 0x7fcc1c070b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c005fd0 tx=0x7fcc2c00a560 comp rx=0 tx=0).stop 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 msgr2=0x7fcc30101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc30101d80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc200092e0 tx=0x7fcc2000be20 comp rx=0 tx=0).stop 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 shutdown_connections 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcc1c06e680 0x7fcc1c070b40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc300ffda0 0x7fcc30101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 --2- 172.21.15.42:0/3623303467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcc30100750 0x7fcc30101d80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.040+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 >> 172.21.15.42:0/3623303467 conn(0x7fcc300fb650 msgr2=0x7fcc30069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.041+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 shutdown_connections 2024-09-17T00:08:27.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.041+0000 7fcc37fe4700 1 -- 172.21.15.42:0/3623303467 wait complete. 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 -- 172.21.15.42:0/1750009779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 msgr2=0x7fe698100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 --2- 172.21.15.42:0/1750009779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe698100080 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe680009910 tx=0x7fe680009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 -- 172.21.15.42:0/1750009779 shutdown_connections 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 --2- 172.21.15.42:0/1750009779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe698100080 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 --2- 172.21.15.42:0/1750009779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe698105bd0 0x7fe698105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 -- 172.21.15.42:0/1750009779 >> 172.21.15.42:0/1750009779 conn(0x7fe698075670 msgr2=0x7fe698075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 -- 172.21.15.42:0/1750009779 shutdown_connections 2024-09-17T00:08:27.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.148+0000 7fe69e72f700 1 -- 172.21.15.42:0/1750009779 wait complete. 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe69e72f700 1 Processor -- start 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe69e72f700 1 -- start start 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe69e72f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe69819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe69e72f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe698105bd0 0x7fe69819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe69e72f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69819d200 con 0x7fe6980ffc20 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe69e72f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6981966b0 con 0x7fe698105bd0 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe697fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe69819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe697fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe69819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43364/0 (socket says 172.21.15.42:43364) 2024-09-17T00:08:27.148 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.149+0000 7fe697fff700 1 -- 172.21.15.42:0/2994758836 learned_addr learned my addr 172.21.15.42:0/2994758836 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe68ffff700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe698105bd0 0x7fe69819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe697fff700 1 -- 172.21.15.42:0/2994758836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe698105bd0 msgr2=0x7fe69819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe697fff700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe698105bd0 0x7fe69819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe697fff700 1 -- 172.21.15.42:0/2994758836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6800095f0 con 0x7fe6980ffc20 2024-09-17T00:08:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe697fff700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe69819c630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c8a0 tx=0x7fe68800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe695ffb700 1 -- 172.21.15.42:0/2994758836 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe68800cea0 con 0x7fe6980ffc20 2024-09-17T00:08:27.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe695ffb700 1 -- 172.21.15.42:0/2994758836 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe688004830 con 0x7fe6980ffc20 2024-09-17T00:08:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe698196990 con 0x7fe6980ffc20 2024-09-17T00:08:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe695ffb700 1 -- 172.21.15.42:0/2994758836 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6880055b0 con 0x7fe6980ffc20 2024-09-17T00:08:27.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.150+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe698196ee0 con 0x7fe6980ffc20 2024-09-17T00:08:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.151+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69804f2e0 con 0x7fe6980ffc20 2024-09-17T00:08:27.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.151+0000 7fe695ffb700 1 -- 172.21.15.42:0/2994758836 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fe6880049a0 con 0x7fe6980ffc20 2024-09-17T00:08:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.152+0000 7fe695ffb700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe68406e680 0x7fe684070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.152+0000 7fe695ffb700 1 -- 172.21.15.42:0/2994758836 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe68808e320 con 0x7fe6980ffc20 2024-09-17T00:08:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.152+0000 7fe68ffff700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe68406e680 0x7fe684070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.152+0000 7fe68ffff700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe68406e680 0x7fe684070b40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe680006010 tx=0x7fe68000b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:27.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.156+0000 7fe695ffb700 1 -- 172.21.15.42:0/2994758836 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe68805aff0 con 0x7fe6980ffc20 2024-09-17T00:08:27.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:26 smithi043 ceph-mon[81510]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:27.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:26 smithi043 ceph-mon[81510]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:27.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.453+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe698197190 con 0x7fe6980ffc20 2024-09-17T00:08:27.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.454+0000 7fe695ffb700 1 -- 172.21.15.42:0/2994758836 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe688004c50 con 0x7fe6980ffc20 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:08:27.452 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:08:27.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:08:27.453 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:08:27.453 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:08:27.453 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:08:27.453 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe68406e680 msgr2=0x7fe684070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe68406e680 0x7fe684070b40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe680006010 tx=0x7fe68000b540 comp rx=0 tx=0).stop 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 msgr2=0x7fe69819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe69819c630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c8a0 tx=0x7fe68800cc60 comp rx=0 tx=0).stop 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 shutdown_connections 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fe68406e680 0x7fe684070b40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe6980ffc20 0x7fe69819c630 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 --2- 172.21.15.42:0/2994758836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe698105bd0 0x7fe69819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 >> 172.21.15.42:0/2994758836 conn(0x7fe698075670 msgr2=0x7fe698068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:27.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 shutdown_connections 2024-09-17T00:08:27.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.457+0000 7fe69e72f700 1 -- 172.21.15.42:0/2994758836 wait complete. 2024-09-17T00:08:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.570+0000 7fa74da75700 1 -- 172.21.15.42:0/3505396731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748106cd0 msgr2=0x7fa7481070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.570+0000 7fa74da75700 1 --2- 172.21.15.42:0/3505396731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748106cd0 0x7fa7481070b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009910 tx=0x7fa73c009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 -- 172.21.15.42:0/3505396731 shutdown_connections 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 --2- 172.21.15.42:0/3505396731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa748068580 0x7fa7480689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 --2- 172.21.15.42:0/3505396731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748106cd0 0x7fa7481070b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 -- 172.21.15.42:0/3505396731 >> 172.21.15.42:0/3505396731 conn(0x7fa748075670 msgr2=0x7fa748075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 -- 172.21.15.42:0/3505396731 shutdown_connections 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 -- 172.21.15.42:0/3505396731 wait complete. 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 Processor -- start 2024-09-17T00:08:27.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.571+0000 7fa74da75700 1 -- start start 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa74da75700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748068580 0x7fa7480ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa74da75700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa748106cd0 0x7fa748100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa74da75700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa748100770 con 0x7fa748068580 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa74da75700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7481008b0 con 0x7fa748106cd0 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa7477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748068580 0x7fa7480ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa7477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748068580 0x7fa7480ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43386/0 (socket says 172.21.15.42:43386) 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa7477fe700 1 -- 172.21.15.42:0/899716416 learned_addr learned my addr 172.21.15.42:0/899716416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:27.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa7477fe700 1 -- 172.21.15.42:0/899716416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa748106cd0 msgr2=0x7fa748100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa746ffd700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa748106cd0 0x7fa748100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa7477fe700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa748106cd0 0x7fa748100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa7477fe700 1 -- 172.21.15.42:0/899716416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0095f0 con 0x7fa748068580 2024-09-17T00:08:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa7477fe700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748068580 0x7fa7480ffcf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa73c006010 tx=0x7fa73c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.572+0000 7fa744ff9700 1 -- 172.21.15.42:0/899716416 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73c01d070 con 0x7fa748068580 2024-09-17T00:08:27.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.573+0000 7fa744ff9700 1 -- 172.21.15.42:0/899716416 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa73c00f460 con 0x7fa748068580 2024-09-17T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.573+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa748100b30 con 0x7fa748068580 2024-09-17T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.573+0000 7fa744ff9700 1 -- 172.21.15.42:0/899716416 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73c017630 con 0x7fa748068580 2024-09-17T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.573+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7481a2d90 con 0x7fa748068580 2024-09-17T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.574+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7481091f0 con 0x7fa748068580 2024-09-17T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.574+0000 7fa744ff9700 1 -- 172.21.15.42:0/899716416 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa73c00f5d0 con 0x7fa748068580 2024-09-17T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.575+0000 7fa744ff9700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa73006e680 0x7fa730070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.575+0000 7fa744ff9700 1 -- 172.21.15.42:0/899716416 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa73c08fad0 con 0x7fa748068580 2024-09-17T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.575+0000 7fa746ffd700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa73006e680 0x7fa730070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.575+0000 7fa746ffd700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa73006e680 0x7fa730070b40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa738005950 tx=0x7fa7380058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:27.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.579+0000 7fa744ff9700 1 -- 172.21.15.42:0/899716416 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa73c058c70 con 0x7fa748068580 2024-09-17T00:08:27.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.838+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa748108bd0 con 0x7fa748068580 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.838+0000 7fa744ff9700 1 -- 172.21.15.42:0/899716416 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fa73c05c330 con 0x7fa748068580 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:08:27.836 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:08:27.837 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:08:27.838 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:08:27.839 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:08:27.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa73006e680 msgr2=0x7fa730070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa73006e680 0x7fa730070b40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa738005950 tx=0x7fa7380058e0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748068580 msgr2=0x7fa7480ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748068580 0x7fa7480ffcf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa73c006010 tx=0x7fa73c00be20 comp rx=0 tx=0).stop 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 shutdown_connections 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa73006e680 0x7fa730070b40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa748068580 0x7fa7480ffcf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 --2- 172.21.15.42:0/899716416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa748106cd0 0x7fa748100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.840+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 >> 172.21.15.42:0/899716416 conn(0x7fa748075670 msgr2=0x7fa7480fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.841+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 shutdown_connections 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.841+0000 7fa74da75700 1 -- 172.21.15.42:0/899716416 wait complete. 2024-09-17T00:08:27.841 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:08:27.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 -- 172.21.15.42:0/1294937424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0101840 msgr2=0x7f25d0105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 --2- 172.21.15.42:0/1294937424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0101840 0x7f25d0105890 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009910 tx=0x7f25cc009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 -- 172.21.15.42:0/1294937424 shutdown_connections 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 --2- 172.21.15.42:0/1294937424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0101840 0x7f25d0105890 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 --2- 172.21.15.42:0/1294937424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25d0100e90 0x7f25d0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 -- 172.21.15.42:0/1294937424 >> 172.21.15.42:0/1294937424 conn(0x7f25d00fc740 msgr2=0x7f25d00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 -- 172.21.15.42:0/1294937424 shutdown_connections 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.950+0000 7f25d7657700 1 -- 172.21.15.42:0/1294937424 wait complete. 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d7657700 1 Processor -- start 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d7657700 1 -- start start 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d7657700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0100e90 0x7f25d019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d7657700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25d0101840 0x7f25d019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d7657700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d019d2e0 con 0x7f25d0100e90 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d7657700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d0196790 con 0x7f25d0101840 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d53f3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0100e90 0x7f25d019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d53f3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0100e90 0x7f25d019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43408/0 (socket says 172.21.15.42:43408) 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.951+0000 7f25d53f3700 1 -- 172.21.15.42:0/4188225388 learned_addr learned my addr 172.21.15.42:0/4188225388 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25d53f3700 1 -- 172.21.15.42:0/4188225388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25d0101840 msgr2=0x7f25d019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25d53f3700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25d0101840 0x7f25d019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25d53f3700 1 -- 172.21.15.42:0/4188225388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0095f0 con 0x7f25d0100e90 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25d53f3700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0100e90 0x7f25d019c710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d8d0 tx=0x7f25c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25c27fc700 1 -- 172.21.15.42:0/4188225388 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c4009940 con 0x7f25d0100e90 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25c27fc700 1 -- 172.21.15.42:0/4188225388 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25c4010460 con 0x7f25d0100e90 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d0196a70 con 0x7f25d0100e90 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25c27fc700 1 -- 172.21.15.42:0/4188225388 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25c400f5f0 con 0x7f25d0100e90 2024-09-17T00:08:27.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.952+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d0196fc0 con 0x7f25d0100e90 2024-09-17T00:08:27.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.953+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d0109290 con 0x7f25d0100e90 2024-09-17T00:08:27.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.954+0000 7f25c27fc700 1 -- 172.21.15.42:0/4188225388 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f25c40105d0 con 0x7f25d0100e90 2024-09-17T00:08:27.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.954+0000 7f25c27fc700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f25bc06e680 0x7f25bc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:27.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.954+0000 7f25c27fc700 1 -- 172.21.15.42:0/4188225388 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f25c408e680 con 0x7f25d0100e90 2024-09-17T00:08:27.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.954+0000 7f25d4bf2700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f25bc06e680 0x7f25bc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:27.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.955+0000 7f25d4bf2700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f25bc06e680 0x7f25bc070b40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00b5c0 tx=0x7f25cc005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:27.956 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:27.959+0000 7f25c27fc700 1 -- 172.21.15.42:0/4188225388 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f25c405b350 con 0x7f25d0100e90 2024-09-17T00:08:28.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.188+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25d0197150 con 0x7f25bc06e680 2024-09-17T00:08:28.186 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:27 smithi042 ceph-mon[111056]: pgmap v506: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:28.186 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:27 smithi042 ceph-mon[111056]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:28.186 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:27 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2994758836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:08:28.186 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:27 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/899716416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.189+0000 7f25c27fc700 1 -- 172.21.15.42:0/4188225388 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f25d0197150 con 0x7f25bc06e680 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:08:28.187 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:08:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:08:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:08:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:08:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:08:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:08:28.188 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:08:28.188 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:08:28.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f25bc06e680 msgr2=0x7f25bc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:28.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f25bc06e680 0x7f25bc070b40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00b5c0 tx=0x7f25cc005960 comp rx=0 tx=0).stop 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0100e90 msgr2=0x7f25d019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0100e90 0x7f25d019c710 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f25c400d8d0 tx=0x7f25c400dc90 comp rx=0 tx=0).stop 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 shutdown_connections 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f25bc06e680 0x7f25bc070b40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25d0100e90 0x7f25d019c710 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 --2- 172.21.15.42:0/4188225388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25d0101840 0x7f25d019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 >> 172.21.15.42:0/4188225388 conn(0x7f25d00fc740 msgr2=0x7f25d00fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 shutdown_connections 2024-09-17T00:08:28.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.192+0000 7f25d7657700 1 -- 172.21.15.42:0/4188225388 wait complete. 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 -- 172.21.15.42:0/692862984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a0105bd0 msgr2=0x7f76a0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 --2- 172.21.15.42:0/692862984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a0105bd0 0x7f76a0105fb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7690009910 tx=0x7f7690009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 -- 172.21.15.42:0/692862984 shutdown_connections 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 --2- 172.21.15.42:0/692862984 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a00ffc20 0x7f76a0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 --2- 172.21.15.42:0/692862984 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a0105bd0 0x7f76a0105fb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 -- 172.21.15.42:0/692862984 >> 172.21.15.42:0/692862984 conn(0x7f76a0075670 msgr2=0x7f76a0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 -- 172.21.15.42:0/692862984 shutdown_connections 2024-09-17T00:08:28.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.300+0000 7f76a69b6700 1 -- 172.21.15.42:0/692862984 wait complete. 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f76a69b6700 1 Processor -- start 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f76a69b6700 1 -- start start 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f76a69b6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a00ffc20 0x7f76a01976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f76a69b6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a0105bd0 0x7f76a0197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f76a69b6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a01982f0 con 0x7f76a00ffc20 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f76a69b6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0198460 con 0x7f76a0105bd0 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f769ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a00ffc20 0x7f76a01976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:28.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f769ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a00ffc20 0x7f76a01976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43420/0 (socket says 172.21.15.42:43420) 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.301+0000 7f769ffff700 1 -- 172.21.15.42:0/1564871809 learned_addr learned my addr 172.21.15.42:0/1564871809 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f7697fff700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a0105bd0 0x7f76a0197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f769ffff700 1 -- 172.21.15.42:0/1564871809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a0105bd0 msgr2=0x7f76a0197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f769ffff700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a0105bd0 0x7f76a0197c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f769ffff700 1 -- 172.21.15.42:0/1564871809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76900095f0 con 0x7f76a00ffc20 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f769ffff700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a00ffc20 0x7f76a01976d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f769000b5c0 tx=0x7f769000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f769dffb700 1 -- 172.21.15.42:0/1564871809 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f769001d070 con 0x7f76a00ffc20 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a01a0740 con 0x7f76a00ffc20 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f769dffb700 1 -- 172.21.15.42:0/1564871809 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f769000f460 con 0x7f76a00ffc20 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f769dffb700 1 -- 172.21.15.42:0/1564871809 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7690017610 con 0x7f76a00ffc20 2024-09-17T00:08:28.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.302+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a01a0bb0 con 0x7f76a00ffc20 2024-09-17T00:08:28.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.303+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a01080f0 con 0x7f76a00ffc20 2024-09-17T00:08:28.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.304+0000 7f769dffb700 1 -- 172.21.15.42:0/1564871809 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f769000f5d0 con 0x7f76a00ffc20 2024-09-17T00:08:28.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.304+0000 7f769dffb700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f768c06e680 0x7f768c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:28.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.304+0000 7f769dffb700 1 -- 172.21.15.42:0/1564871809 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f769008fbd0 con 0x7f76a00ffc20 2024-09-17T00:08:28.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.304+0000 7f7697fff700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f768c06e680 0x7f768c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:28.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.305+0000 7f7697fff700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f768c06e680 0x7f768c070b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f76a0068e30 tx=0x7f7688005cf0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:28.306 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.309+0000 7f769dffb700 1 -- 172.21.15.42:0/1564871809 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7690058df0 con 0x7f76a00ffc20 2024-09-17T00:08:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:27 smithi043 ceph-mon[81510]: pgmap v506: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:27 smithi043 ceph-mon[81510]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:27 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2994758836' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:08:28.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:27 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/899716416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:08:28.600 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.603+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f76a0107ad0 con 0x7f76a00ffc20 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.603+0000 7f769dffb700 1 -- 172.21.15.42:0/1564871809 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f769005c4b0 con 0x7f76a00ffc20 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:08:28.601 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:08:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f768c06e680 msgr2=0x7f768c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f768c06e680 0x7f768c070b40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f76a0068e30 tx=0x7f7688005cf0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a00ffc20 msgr2=0x7f76a01976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a00ffc20 0x7f76a01976d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f769000b5c0 tx=0x7f769000be00 comp rx=0 tx=0).stop 2024-09-17T00:08:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 shutdown_connections 2024-09-17T00:08:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f768c06e680 0x7f768c070b40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76a00ffc20 0x7f76a01976d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 --2- 172.21.15.42:0/1564871809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76a0105bd0 0x7f76a0197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:28.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 >> 172.21.15.42:0/1564871809 conn(0x7f76a0075670 msgr2=0x7f76a00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:28.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 shutdown_connections 2024-09-17T00:08:28.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:28.605+0000 7f76a69b6700 1 -- 172.21.15.42:0/1564871809 wait complete. 2024-09-17T00:08:29.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:28 smithi042 ceph-mon[111056]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:29.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:28 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1564871809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:08:29.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:28 smithi043 ceph-mon[81510]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:08:29.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:28 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1564871809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:08:30.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:29 smithi042 ceph-mon[111056]: pgmap v507: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:30.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:29 smithi043 ceph-mon[81510]: pgmap v507: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:32.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:31 smithi042 ceph-mon[111056]: pgmap v508: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:32.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:31 smithi043 ceph-mon[81510]: pgmap v508: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:33 smithi042 ceph-mon[111056]: pgmap v509: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:33 smithi043 ceph-mon[81510]: pgmap v509: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:36.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:35 smithi042 ceph-mon[111056]: pgmap v510: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:36.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:36.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:35 smithi043 ceph-mon[81510]: pgmap v510: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:36.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:38.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:37 smithi042 ceph-mon[111056]: pgmap v511: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:38.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:37 smithi043 ceph-mon[81510]: pgmap v511: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:40.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:39 smithi042 ceph-mon[111056]: pgmap v512: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:40.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:39 smithi043 ceph-mon[81510]: pgmap v512: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:42.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:41 smithi042 ceph-mon[111056]: pgmap v513: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:42.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:41 smithi043 ceph-mon[81510]: pgmap v513: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:44.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:43 smithi042 ceph-mon[111056]: pgmap v514: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:44.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:43 smithi043 ceph-mon[81510]: pgmap v514: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:46.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:45 smithi042 ceph-mon[111056]: pgmap v515: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:46.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:45 smithi043 ceph-mon[81510]: pgmap v515: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:48.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:47 smithi042 ceph-mon[111056]: pgmap v516: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:48.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:47 smithi043 ceph-mon[81510]: pgmap v516: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:50.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:49 smithi042 ceph-mon[111056]: pgmap v517: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:50.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:49 smithi043 ceph-mon[81510]: pgmap v517: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:51.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:51.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:08:52.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:51 smithi043 ceph-mon[81510]: pgmap v518: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:52.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:51 smithi042 ceph-mon[111056]: pgmap v518: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:54.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:53 smithi042 ceph-mon[111056]: pgmap v519: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:54.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:53 smithi043 ceph-mon[81510]: pgmap v519: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:56.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:55 smithi042 ceph-mon[111056]: pgmap v520: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:56.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:55 smithi043 ceph-mon[81510]: pgmap v520: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:58.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:57 smithi042 ceph-mon[111056]: pgmap v521: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:58.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:57 smithi043 ceph-mon[81510]: pgmap v521: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:08:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 -- 172.21.15.42:0/1896644052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64101de0 msgr2=0x7f5e641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/1896644052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64101de0 0x7f5e641021c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009910 tx=0x7f5e58009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 -- 172.21.15.42:0/1896644052 shutdown_connections 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/1896644052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e64102700 0x7f5e6410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/1896644052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64101de0 0x7f5e641021c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 -- 172.21.15.42:0/1896644052 >> 172.21.15.42:0/1896644052 conn(0x7f5e640fb650 msgr2=0x7f5e640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 -- 172.21.15.42:0/1896644052 shutdown_connections 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 -- 172.21.15.42:0/1896644052 wait complete. 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.729+0000 7f5e6ae23700 1 Processor -- start 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e6ae23700 1 -- start start 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e6ae23700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e64101de0 0x7f5e6419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e6ae23700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64102700 0x7f5e6419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e6ae23700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6419f460 con 0x7f5e64102700 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e6ae23700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e64198910 con 0x7f5e64101de0 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64102700 0x7f5e6419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:58.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e63fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64102700 0x7f5e6419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56742/0 (socket says 172.21.15.42:56742) 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e63fff700 1 -- 172.21.15.42:0/954986264 learned_addr learned my addr 172.21.15.42:0/954986264 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e68bbf700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e64101de0 0x7f5e6419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e63fff700 1 -- 172.21.15.42:0/954986264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e64101de0 msgr2=0x7f5e6419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e63fff700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e64101de0 0x7f5e6419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.730+0000 7f5e63fff700 1 -- 172.21.15.42:0/954986264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e580095f0 con 0x7f5e64102700 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.731+0000 7f5e63fff700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64102700 0x7f5e6419edd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d6d0 tx=0x7f5e5000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.731+0000 7f5e61ffb700 1 -- 172.21.15.42:0/954986264 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e5000f860 con 0x7f5e64102700 2024-09-17T00:08:58.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.731+0000 7f5e61ffb700 1 -- 172.21.15.42:0/954986264 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e5000fe80 con 0x7f5e64102700 2024-09-17T00:08:58.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.731+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e64198bf0 con 0x7f5e64102700 2024-09-17T00:08:58.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.731+0000 7f5e61ffb700 1 -- 172.21.15.42:0/954986264 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e5000e5e0 con 0x7f5e64102700 2024-09-17T00:08:58.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.731+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e64199140 con 0x7f5e64102700 2024-09-17T00:08:58.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.732+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6410a750 con 0x7f5e64102700 2024-09-17T00:08:58.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.733+0000 7f5e61ffb700 1 -- 172.21.15.42:0/954986264 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5e50010460 con 0x7f5e64102700 2024-09-17T00:08:58.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.733+0000 7f5e61ffb700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e5406e680 0x7f5e54070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:58.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.733+0000 7f5e61ffb700 1 -- 172.21.15.42:0/954986264 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5e5008e310 con 0x7f5e64102700 2024-09-17T00:08:58.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.733+0000 7f5e68bbf700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e5406e680 0x7f5e54070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:58.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.734+0000 7f5e68bbf700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e5406e680 0x7f5e54070b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e5801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:58.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.738+0000 7f5e61ffb700 1 -- 172.21.15.42:0/954986264 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5e5005afe0 con 0x7f5e64102700 2024-09-17T00:08:58.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.968+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e641993c0 con 0x7f5e5406e680 2024-09-17T00:08:58.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.970+0000 7f5e61ffb700 1 -- 172.21.15.42:0/954986264 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5e641993c0 con 0x7f5e5406e680 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e5406e680 msgr2=0x7f5e54070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e5406e680 0x7f5e54070b40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006010 tx=0x7f5e5801a040 comp rx=0 tx=0).stop 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64102700 msgr2=0x7f5e6419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64102700 0x7f5e6419edd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000d6d0 tx=0x7f5e5000d9e0 comp rx=0 tx=0).stop 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 shutdown_connections 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5e5406e680 0x7f5e54070b40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e64101de0 0x7f5e6419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 --2- 172.21.15.42:0/954986264 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e64102700 0x7f5e6419edd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:58.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 >> 172.21.15.42:0/954986264 conn(0x7f5e640fb650 msgr2=0x7f5e64100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:58.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 shutdown_connections 2024-09-17T00:08:58.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:58.972+0000 7f5e6ae23700 1 -- 172.21.15.42:0/954986264 wait complete. 2024-09-17T00:08:58.980 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:08:59.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 -- 172.21.15.42:0/1474901401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424100ea0 msgr2=0x7f7424101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 --2- 172.21.15.42:0/1474901401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424100ea0 0x7f7424101280 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7418009910 tx=0x7f7418009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 -- 172.21.15.42:0/1474901401 shutdown_connections 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 --2- 172.21.15.42:0/1474901401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7424101850 0x7f74241058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 --2- 172.21.15.42:0/1474901401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424100ea0 0x7f7424101280 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 -- 172.21.15.42:0/1474901401 >> 172.21.15.42:0/1474901401 conn(0x7f74240fc750 msgr2=0x7f74240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 -- 172.21.15.42:0/1474901401 shutdown_connections 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.082+0000 7f742b0de700 1 -- 172.21.15.42:0/1474901401 wait complete. 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f742b0de700 1 Processor -- start 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f742b0de700 1 -- start start 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f742b0de700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7424100ea0 0x7f7424073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f742b0de700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424101850 0x7f74240735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f742b0de700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7424073b10 con 0x7f7424101850 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f742b0de700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7424073c50 con 0x7f7424100ea0 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f7423fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424101850 0x7f74240735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f7423fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424101850 0x7f74240735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56754/0 (socket says 172.21.15.42:56754) 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f7423fff700 1 -- 172.21.15.42:0/1664039916 learned_addr learned my addr 172.21.15.42:0/1664039916 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.083+0000 7f7428e7a700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7424100ea0 0x7f7424073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f7423fff700 1 -- 172.21.15.42:0/1664039916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7424100ea0 msgr2=0x7f7424073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f7423fff700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7424100ea0 0x7f7424073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f7423fff700 1 -- 172.21.15.42:0/1664039916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74180095f0 con 0x7f7424101850 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f7423fff700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424101850 0x7f74240735d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f741000d900 tx=0x7f741000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f7421ffb700 1 -- 172.21.15.42:0/1664039916 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7410009900 con 0x7f7424101850 2024-09-17T00:08:59.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f7421ffb700 1 -- 172.21.15.42:0/1664039916 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7410010460 con 0x7f7424101850 2024-09-17T00:08:59.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74241a2b10 con 0x7f7424101850 2024-09-17T00:08:59.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f7421ffb700 1 -- 172.21.15.42:0/1664039916 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f741000f5f0 con 0x7f7424101850 2024-09-17T00:08:59.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.084+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74241a2f20 con 0x7f7424101850 2024-09-17T00:08:59.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.085+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7424109240 con 0x7f7424101850 2024-09-17T00:08:59.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.086+0000 7f7421ffb700 1 -- 172.21.15.42:0/1664039916 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f74100105d0 con 0x7f7424101850 2024-09-17T00:08:59.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.086+0000 7f7421ffb700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f741406e680 0x7f7414070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.086+0000 7f7421ffb700 1 -- 172.21.15.42:0/1664039916 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f741008e1d0 con 0x7f7424101850 2024-09-17T00:08:59.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.086+0000 7f7428e7a700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f741406e680 0x7f7414070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.087+0000 7f7428e7a700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f741406e680 0x7f7414070b40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7418006010 tx=0x7f741801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:59.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.090+0000 7f7421ffb700 1 -- 172.21.15.42:0/1664039916 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7410057350 con 0x7f7424101850 2024-09-17T00:08:59.317 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.320+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74240611d0 con 0x7f741406e680 2024-09-17T00:08:59.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.321+0000 7f7421ffb700 1 -- 172.21.15.42:0/1664039916 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f74240611d0 con 0x7f741406e680 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f741406e680 msgr2=0x7f7414070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f741406e680 0x7f7414070b40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7418006010 tx=0x7f741801a040 comp rx=0 tx=0).stop 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424101850 msgr2=0x7f74240735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424101850 0x7f74240735d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f741000d900 tx=0x7f741000dc10 comp rx=0 tx=0).stop 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 shutdown_connections 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f741406e680 0x7f7414070b40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7424100ea0 0x7f7424073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 --2- 172.21.15.42:0/1664039916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7424101850 0x7f74240735d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 >> 172.21.15.42:0/1664039916 conn(0x7f74240fc750 msgr2=0x7f74240fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:59.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 shutdown_connections 2024-09-17T00:08:59.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.323+0000 7f742b0de700 1 -- 172.21.15.42:0/1664039916 wait complete. 2024-09-17T00:08:59.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 -- 172.21.15.42:0/2907021255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 msgr2=0x7f6e940ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 --2- 172.21.15.42:0/2907021255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e940ff980 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009910 tx=0x7f6e84009c20 comp rx=0 tx=0).stop 2024-09-17T00:08:59.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 -- 172.21.15.42:0/2907021255 shutdown_connections 2024-09-17T00:08:59.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 --2- 172.21.15.42:0/2907021255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e940fff50 0x7f6e94105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 --2- 172.21.15.42:0/2907021255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e940ff980 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 -- 172.21.15.42:0/2907021255 >> 172.21.15.42:0/2907021255 conn(0x7f6e94074e30 msgr2=0x7f6e94075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:59.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 -- 172.21.15.42:0/2907021255 shutdown_connections 2024-09-17T00:08:59.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.432+0000 7f6e996b3700 1 -- 172.21.15.42:0/2907021255 wait complete. 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e996b3700 1 Processor -- start 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e996b3700 1 -- start start 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e996b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e94100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e996b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e940fff50 0x7f6e94100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e996b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e94100fa0 con 0x7f6e940ff5a0 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e996b3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e941010e0 con 0x7f6e940fff50 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e92ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e94100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e92ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e94100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56768/0 (socket says 172.21.15.42:56768) 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e92ffd700 1 -- 172.21.15.42:0/1980838265 learned_addr learned my addr 172.21.15.42:0/1980838265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.433+0000 7f6e927fc700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e940fff50 0x7f6e94100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e92ffd700 1 -- 172.21.15.42:0/1980838265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e940fff50 msgr2=0x7f6e94100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e92ffd700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e940fff50 0x7f6e94100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e92ffd700 1 -- 172.21.15.42:0/1980838265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840095f0 con 0x7f6e940ff5a0 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e927fc700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e940fff50 0x7f6e94100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e92ffd700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e94100520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6e84006010 tx=0x7f6e8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e7bfff700 1 -- 172.21.15.42:0/1980838265 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e8401d070 con 0x7f6e940ff5a0 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e94101360 con 0x7f6e940ff5a0 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e7bfff700 1 -- 172.21.15.42:0/1980838265 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e84004fb0 con 0x7f6e940ff5a0 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e7bfff700 1 -- 172.21.15.42:0/1980838265 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e84022ea0 con 0x7f6e940ff5a0 2024-09-17T00:08:59.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.434+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e941a2de0 con 0x7f6e940ff5a0 2024-09-17T00:08:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.435+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e94109270 con 0x7f6e940ff5a0 2024-09-17T00:08:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.436+0000 7f6e7bfff700 1 -- 172.21.15.42:0/1980838265 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6e84004af0 con 0x7f6e940ff5a0 2024-09-17T00:08:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.436+0000 7f6e7bfff700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e7c06e680 0x7f6e7c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.436+0000 7f6e927fc700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e7c06e680 0x7f6e7c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.436+0000 7f6e7bfff700 1 -- 172.21.15.42:0/1980838265 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6e8408fbc0 con 0x7f6e940ff5a0 2024-09-17T00:08:59.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.437+0000 7f6e927fc700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e7c06e680 0x7f6e7c070b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e88005fd0 tx=0x7f6e88005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:59.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.441+0000 7f6e7bfff700 1 -- 172.21.15.42:0/1980838265 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6e84058f30 con 0x7f6e940ff5a0 2024-09-17T00:08:59.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.661+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e94101b80 con 0x7f6e7c06e680 2024-09-17T00:08:59.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.671+0000 7f6e7bfff700 1 -- 172.21.15.42:0/1980838265 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f6e94101b80 con 0x7f6e7c06e680 2024-09-17T00:08:59.668 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:08:59.668 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (17m) 11m ago 28m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (28m) 11m ago 28m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (27m) 10m ago 27m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (15m) 11m ago 28m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (15m) 10m ago 27m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (17m) 11m ago 28m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (11m) 11m ago 22m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (11m) 11m ago 22m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (11m) 10m ago 22m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (10m) 10m ago 22m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (19m) 11m ago 29m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (18m) 10m ago 27m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (16m) 11m ago 29m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (15m) 10m ago 27m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:08:59.669 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (18m) 11m ago 28m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (18m) 10m ago 27m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (14m) 11m ago 26m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (14m) 11m ago 26m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (13m) 11m ago 25m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (13m) 10m ago 25m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (12m) 10m ago 24m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (11m) 10m ago 24m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:08:59.670 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (17m) 11m ago 27m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e7c06e680 msgr2=0x7f6e7c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e7c06e680 0x7f6e7c070b40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e88005fd0 tx=0x7f6e88005e30 comp rx=0 tx=0).stop 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 msgr2=0x7f6e94100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e94100520 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6e84006010 tx=0x7f6e8400be20 comp rx=0 tx=0).stop 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 shutdown_connections 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6e7c06e680 0x7f6e7c070b40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6e940ff5a0 0x7f6e94100520 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 --2- 172.21.15.42:0/1980838265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e940fff50 0x7f6e94100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 >> 172.21.15.42:0/1980838265 conn(0x7f6e94074e30 msgr2=0x7f6e940fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:59.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 shutdown_connections 2024-09-17T00:08:59.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.673+0000 7f6e996b3700 1 -- 172.21.15.42:0/1980838265 wait complete. 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 -- 172.21.15.42:0/3302658102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 msgr2=0x7f82b0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 --2- 172.21.15.42:0/3302658102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b0105fd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f82a000b0a0 tx=0x7f82a000b3b0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 -- 172.21.15.42:0/3302658102 shutdown_connections 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 --2- 172.21.15.42:0/3302658102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b00ffb40 0x7f82b00fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 --2- 172.21.15.42:0/3302658102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b0105fd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 -- 172.21.15.42:0/3302658102 >> 172.21.15.42:0/3302658102 conn(0x7f82b00fb610 msgr2=0x7f82b00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 -- 172.21.15.42:0/3302658102 shutdown_connections 2024-09-17T00:08:59.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.780+0000 7f82b7037700 1 -- 172.21.15.42:0/3302658102 wait complete. 2024-09-17T00:08:59.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82b7037700 1 Processor -- start 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82b7037700 1 -- start start 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82b7037700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b00ffb40 0x7f82b019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82b7037700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82b7037700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b019c0f0 con 0x7f82b0105bf0 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82b7037700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01955a0 con 0x7f82b00ffb40 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82affff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56798/0 (socket says 172.21.15.42:56798) 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82affff700 1 -- 172.21.15.42:0/4235932243 learned_addr learned my addr 172.21.15.42:0/4235932243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.781+0000 7f82b4dd3700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b00ffb40 0x7f82b019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82affff700 1 -- 172.21.15.42:0/4235932243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b00ffb40 msgr2=0x7f82b019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82affff700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b00ffb40 0x7f82b019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:08:59.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82affff700 1 -- 172.21.15.42:0/4235932243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0009d00 con 0x7f82b0105bf0 2024-09-17T00:08:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82affff700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b019ba60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f829800d8d0 tx=0x7f829800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82adffb700 1 -- 172.21.15.42:0/4235932243 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82980098a0 con 0x7f82b0105bf0 2024-09-17T00:08:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82adffb700 1 -- 172.21.15.42:0/4235932243 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8298010460 con 0x7f82b0105bf0 2024-09-17T00:08:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b0195880 con 0x7f82b0105bf0 2024-09-17T00:08:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82adffb700 1 -- 172.21.15.42:0/4235932243 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f829800f5f0 con 0x7f82b0105bf0 2024-09-17T00:08:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.782+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b0195dd0 con 0x7f82b0105bf0 2024-09-17T00:08:59.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.783+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b004f2e0 con 0x7f82b0105bf0 2024-09-17T00:08:59.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.784+0000 7f82adffb700 1 -- 172.21.15.42:0/4235932243 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8298009a00 con 0x7f82b0105bf0 2024-09-17T00:08:59.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.784+0000 7f82adffb700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f829c070890 0x7f829c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:08:59.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.784+0000 7f82adffb700 1 -- 172.21.15.42:0/4235932243 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f829808e550 con 0x7f82b0105bf0 2024-09-17T00:08:59.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.784+0000 7f82b4dd3700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f829c070890 0x7f829c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:08:59.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.785+0000 7f82b4dd3700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f829c070890 0x7f829c072d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000f80 tx=0x7f82a00060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:08:59.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:08:59.789+0000 7f82adffb700 1 -- 172.21.15.42:0/4235932243 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f829805b220 con 0x7f82b0105bf0 2024-09-17T00:09:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.087+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f82b0196ab0 con 0x7f82b0105bf0 2024-09-17T00:09:00.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.088+0000 7f82adffb700 1 -- 172.21.15.42:0/4235932243 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f829805adb0 con 0x7f82b0105bf0 2024-09-17T00:09:00.085 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:09:00.086 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:09:00.087 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:09:00.087 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:09:00.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f829c070890 msgr2=0x7f829c072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f829c070890 0x7f829c072d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000f80 tx=0x7f82a00060c0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 msgr2=0x7f82b019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b019ba60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f829800d8d0 tx=0x7f829800dbe0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 shutdown_connections 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f829c070890 0x7f829c072d50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b00ffb40 0x7f82b019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 --2- 172.21.15.42:0/4235932243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b0105bf0 0x7f82b019ba60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 >> 172.21.15.42:0/4235932243 conn(0x7f82b00fb610 msgr2=0x7f82b00fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 shutdown_connections 2024-09-17T00:09:00.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.090+0000 7f82b7037700 1 -- 172.21.15.42:0/4235932243 wait complete. 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.198+0000 7f7531b13700 1 -- 172.21.15.42:0/1178527440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c101130 msgr2=0x7f752c1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.198+0000 7f7531b13700 1 --2- 172.21.15.42:0/1178527440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c101130 0x7f752c1047b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f751c009910 tx=0x7f751c009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 -- 172.21.15.42:0/1178527440 shutdown_connections 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 --2- 172.21.15.42:0/1178527440 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c101130 0x7f752c1047b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 --2- 172.21.15.42:0/1178527440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f752c100780 0x7f752c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 -- 172.21.15.42:0/1178527440 >> 172.21.15.42:0/1178527440 conn(0x7f752c074e30 msgr2=0x7f752c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 -- 172.21.15.42:0/1178527440 shutdown_connections 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 -- 172.21.15.42:0/1178527440 wait complete. 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 Processor -- start 2024-09-17T00:09:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.199+0000 7f7531b13700 1 -- start start 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f7531b13700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c100780 0x7f752c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f7531b13700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f752c101130 0x7f752c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f7531b13700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c196880 con 0x7f752c100780 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f7531b13700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c1969f0 con 0x7f752c101130 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c100780 0x7f752c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c100780 0x7f752c19c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56812/0 (socket says 172.21.15.42:56812) 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752b7fe700 1 -- 172.21.15.42:0/2915641869 learned_addr learned my addr 172.21.15.42:0/2915641869 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752affd700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f752c101130 0x7f752c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752b7fe700 1 -- 172.21.15.42:0/2915641869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f752c101130 msgr2=0x7f752c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752b7fe700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f752c101130 0x7f752c19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752b7fe700 1 -- 172.21.15.42:0/2915641869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0095f0 con 0x7f752c100780 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f752b7fe700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c100780 0x7f752c19c890 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f751400b4b0 tx=0x7f751400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.200+0000 7f7528ff9700 1 -- 172.21.15.42:0/2915641869 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7514011800 con 0x7f752c100780 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.201+0000 7f7528ff9700 1 -- 172.21.15.42:0/2915641869 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7514011e20 con 0x7f752c100780 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.201+0000 7f7528ff9700 1 -- 172.21.15.42:0/2915641869 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f751400f2e0 con 0x7f752c100780 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.201+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c196cd0 con 0x7f752c100780 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.201+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c1971f0 con 0x7f752c100780 2024-09-17T00:09:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.202+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c108140 con 0x7f752c100780 2024-09-17T00:09:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.202+0000 7f7528ff9700 1 -- 172.21.15.42:0/2915641869 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f7514010380 con 0x7f752c100780 2024-09-17T00:09:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.203+0000 7f7528ff9700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f751806e680 0x7f7518070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.203+0000 7f7528ff9700 1 -- 172.21.15.42:0/2915641869 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f751408e090 con 0x7f752c100780 2024-09-17T00:09:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.203+0000 7f752affd700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f751806e680 0x7f7518070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.203+0000 7f752affd700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f751806e680 0x7f7518070b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f752c198140 tx=0x7f751c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:00.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.207+0000 7f7528ff9700 1 -- 172.21.15.42:0/2915641869 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f75140572b0 con 0x7f752c100780 2024-09-17T00:09:00.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:59 smithi042 ceph-mon[111056]: pgmap v522: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:59 smithi042 ceph-mon[111056]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:59 smithi042 ceph-mon[111056]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:08:59 smithi042 ceph-mon[111056]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:00.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:59 smithi043 ceph-mon[81510]: pgmap v522: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:00.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:59 smithi043 ceph-mon[81510]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:00.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:59 smithi043 ceph-mon[81510]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:00.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:08:59 smithi043 ceph-mon[81510]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:00.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.463+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f752c197e50 con 0x7f752c100780 2024-09-17T00:09:00.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.464+0000 7f7528ff9700 1 -- 172.21.15.42:0/2915641869 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f751405a970 con 0x7f752c100780 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:09:00.462 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:09:00.463 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:00.464 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:00.465 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:00.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f751806e680 msgr2=0x7f7518070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f751806e680 0x7f7518070b40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f752c198140 tx=0x7f751c01a040 comp rx=0 tx=0).stop 2024-09-17T00:09:00.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c100780 msgr2=0x7f752c19c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c100780 0x7f752c19c890 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f751400b4b0 tx=0x7f751400b7c0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 shutdown_connections 2024-09-17T00:09:00.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f751806e680 0x7f7518070b40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f752c100780 0x7f752c19c890 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 --2- 172.21.15.42:0/2915641869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f752c101130 0x7f752c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 >> 172.21.15.42:0/2915641869 conn(0x7f752c074e30 msgr2=0x7f752c109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:00.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 shutdown_connections 2024-09-17T00:09:00.467 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.467+0000 7f7531b13700 1 -- 172.21.15.42:0/2915641869 wait complete. 2024-09-17T00:09:00.467 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:09:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 -- 172.21.15.42:0/3652513652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940ff5a0 msgr2=0x7f58940ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 --2- 172.21.15.42:0/3652513652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940ff5a0 0x7f58940ff980 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5884009910 tx=0x7f5884009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 -- 172.21.15.42:0/3652513652 shutdown_connections 2024-09-17T00:09:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 --2- 172.21.15.42:0/3652513652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58940fff50 0x7f5894105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 --2- 172.21.15.42:0/3652513652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940ff5a0 0x7f58940ff980 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 -- 172.21.15.42:0/3652513652 >> 172.21.15.42:0/3652513652 conn(0x7f5894074e30 msgr2=0x7f5894075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:00.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 -- 172.21.15.42:0/3652513652 shutdown_connections 2024-09-17T00:09:00.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.576+0000 7f589894f700 1 -- 172.21.15.42:0/3652513652 wait complete. 2024-09-17T00:09:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f589894f700 1 Processor -- start 2024-09-17T00:09:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f589894f700 1 -- start start 2024-09-17T00:09:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f589894f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58940ff5a0 0x7f589419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f589894f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940fff50 0x7f589419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f589894f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589419d250 con 0x7f58940fff50 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f589894f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5894196700 con 0x7f58940ff5a0 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f5891d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940fff50 0x7f589419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f5891d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940fff50 0x7f589419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56828/0 (socket says 172.21.15.42:56828) 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f5891d9b700 1 -- 172.21.15.42:0/3761387004 learned_addr learned my addr 172.21.15.42:0/3761387004 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.577+0000 7f589259c700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58940ff5a0 0x7f589419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f5891d9b700 1 -- 172.21.15.42:0/3761387004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58940ff5a0 msgr2=0x7f589419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f5891d9b700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58940ff5a0 0x7f589419c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f5891d9b700 1 -- 172.21.15.42:0/3761387004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58840095f0 con 0x7f58940fff50 2024-09-17T00:09:00.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f5891d9b700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940fff50 0x7f589419cbc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f588800d900 tx=0x7f588800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:00.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f58837fe700 1 -- 172.21.15.42:0/3761387004 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5888009900 con 0x7f58940fff50 2024-09-17T00:09:00.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58941969e0 con 0x7f58940fff50 2024-09-17T00:09:00.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f58837fe700 1 -- 172.21.15.42:0/3761387004 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5888010460 con 0x7f58940fff50 2024-09-17T00:09:00.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f58837fe700 1 -- 172.21.15.42:0/3761387004 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f588800f5f0 con 0x7f58940fff50 2024-09-17T00:09:00.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.578+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5894196f30 con 0x7f58940fff50 2024-09-17T00:09:00.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.579+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5894109270 con 0x7f58940fff50 2024-09-17T00:09:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.580+0000 7f58837fe700 1 -- 172.21.15.42:0/3761387004 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f58880105d0 con 0x7f58940fff50 2024-09-17T00:09:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.580+0000 7f58837fe700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f587c06e680 0x7f587c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.580+0000 7f58837fe700 1 -- 172.21.15.42:0/3761387004 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f588808e240 con 0x7f58940fff50 2024-09-17T00:09:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.580+0000 7f589259c700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f587c06e680 0x7f587c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.581+0000 7f589259c700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f587c06e680 0x7f587c070b40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5884006010 tx=0x7f588401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:00.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.585+0000 7f58837fe700 1 -- 172.21.15.42:0/3761387004 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f588805af10 con 0x7f58940fff50 2024-09-17T00:09:00.812 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.815+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58941971a0 con 0x7f587c06e680 2024-09-17T00:09:00.813 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.816+0000 7f58837fe700 1 -- 172.21.15.42:0/3761387004 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f58941971a0 con 0x7f587c06e680 2024-09-17T00:09:00.813 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:09:00.813 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:09:00.813 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:09:00.813 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:09:00.814 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:09:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f587c06e680 msgr2=0x7f587c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f587c06e680 0x7f587c070b40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5884006010 tx=0x7f588401a040 comp rx=0 tx=0).stop 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940fff50 msgr2=0x7f589419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940fff50 0x7f589419cbc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f588800d900 tx=0x7f588800dc10 comp rx=0 tx=0).stop 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 shutdown_connections 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f587c06e680 0x7f587c070b40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58940ff5a0 0x7f589419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 --2- 172.21.15.42:0/3761387004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58940fff50 0x7f589419cbc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 >> 172.21.15.42:0/3761387004 conn(0x7f5894074e30 msgr2=0x7f58940fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 shutdown_connections 2024-09-17T00:09:00.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.818+0000 7f589894f700 1 -- 172.21.15.42:0/3761387004 wait complete. 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 -- 172.21.15.42:0/1450336085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc0ffc20 msgr2=0x7f5bbc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/1450336085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc0ffc20 0x7f5bbc100080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009910 tx=0x7f5ba4009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 -- 172.21.15.42:0/1450336085 shutdown_connections 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/1450336085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc0ffc20 0x7f5bbc100080 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/1450336085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bbc105bd0 0x7f5bbc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 -- 172.21.15.42:0/1450336085 >> 172.21.15.42:0/1450336085 conn(0x7f5bbc075670 msgr2=0x7f5bbc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 -- 172.21.15.42:0/1450336085 shutdown_connections 2024-09-17T00:09:00.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.926+0000 7f5bc31cc700 1 -- 172.21.15.42:0/1450336085 wait complete. 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bc31cc700 1 Processor -- start 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bc31cc700 1 -- start start 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bc31cc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bbc0ffc20 0x7f5bbc101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bc31cc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc105bd0 0x7f5bbc101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bc31cc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc1022c0 con 0x7f5bbc105bd0 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bc31cc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc102400 con 0x7f5bbc0ffc20 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bbbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc105bd0 0x7f5bbc101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bbbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc105bd0 0x7f5bbc101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56846/0 (socket says 172.21.15.42:56846) 2024-09-17T00:09:00.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.927+0000 7f5bbbfff700 1 -- 172.21.15.42:0/87120000 learned_addr learned my addr 172.21.15.42:0/87120000 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bc0f68700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bbc0ffc20 0x7f5bbc101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bbbfff700 1 -- 172.21.15.42:0/87120000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bbc0ffc20 msgr2=0x7f5bbc101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bbbfff700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bbc0ffc20 0x7f5bbc101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bbbfff700 1 -- 172.21.15.42:0/87120000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40095f0 con 0x7f5bbc105bd0 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bbbfff700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc105bd0 0x7f5bbc101d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400b5c0 tx=0x7f5ba400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bb9ffb700 1 -- 172.21.15.42:0/87120000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ba401d070 con 0x7f5bbc105bd0 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bb9ffb700 1 -- 172.21.15.42:0/87120000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ba4005e00 con 0x7f5bbc105bd0 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc102680 con 0x7f5bbc105bd0 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bb9ffb700 1 -- 172.21.15.42:0/87120000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ba4017400 con 0x7f5bbc105bd0 2024-09-17T00:09:00.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.928+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc1a2d90 con 0x7f5bbc105bd0 2024-09-17T00:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.929+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc04f2e0 con 0x7f5bbc105bd0 2024-09-17T00:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.930+0000 7f5bb9ffb700 1 -- 172.21.15.42:0/87120000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5ba400f460 con 0x7f5bbc105bd0 2024-09-17T00:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.931+0000 7f5bb9ffb700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ba806e680 0x7f5ba8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.931+0000 7f5bb9ffb700 1 -- 172.21.15.42:0/87120000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5ba408fb40 con 0x7f5bbc105bd0 2024-09-17T00:09:00.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.931+0000 7f5bc0f68700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ba806e680 0x7f5ba8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:00.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.931+0000 7f5bc0f68700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ba806e680 0x7f5ba8070b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5bac005950 tx=0x7f5bac0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:00.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:00.935+0000 7f5bb9ffb700 1 -- 172.21.15.42:0/87120000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5ba4058cc0 con 0x7f5bbc105bd0 2024-09-17T00:09:01.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.233+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5bbc102d70 con 0x7f5bbc105bd0 2024-09-17T00:09:01.231 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:00 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4235932243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:09:01.231 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:00 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2915641869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.234+0000 7f5bb9ffb700 1 -- 172.21.15.42:0/87120000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5ba405c380 con 0x7f5bbc105bd0 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:09:01.232 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:09:01.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ba806e680 msgr2=0x7f5ba8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ba806e680 0x7f5ba8070b40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5bac005950 tx=0x7f5bac0058e0 comp rx=0 tx=0).stop 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc105bd0 msgr2=0x7f5bbc101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc105bd0 0x7f5bbc101d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5ba400b5c0 tx=0x7f5ba400be20 comp rx=0 tx=0).stop 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 shutdown_connections 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5ba806e680 0x7f5ba8070b40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bbc0ffc20 0x7f5bbc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 --2- 172.21.15.42:0/87120000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bbc105bd0 0x7f5bbc101d80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 >> 172.21.15.42:0/87120000 conn(0x7f5bbc075670 msgr2=0x7f5bbc0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 shutdown_connections 2024-09-17T00:09:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:01.236+0000 7f5bc31cc700 1 -- 172.21.15.42:0/87120000 wait complete. 2024-09-17T00:09:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:00 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4235932243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:09:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:00 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2915641869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:09:02.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:01 smithi043 ceph-mon[81510]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:02.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:01 smithi043 ceph-mon[81510]: pgmap v523: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:02.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:01 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/87120000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:09:02.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:01 smithi042 ceph-mon[111056]: from='client.35012 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:01 smithi042 ceph-mon[111056]: pgmap v523: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:01 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/87120000' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:09:04.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:03 smithi042 ceph-mon[111056]: pgmap v524: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:04.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:03 smithi043 ceph-mon[81510]: pgmap v524: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:06.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:05 smithi042 ceph-mon[111056]: pgmap v525: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:06.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:06.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:05 smithi043 ceph-mon[81510]: pgmap v525: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:08.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:07 smithi042 ceph-mon[111056]: pgmap v526: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:08.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:07 smithi043 ceph-mon[81510]: pgmap v526: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:10.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:09 smithi042 ceph-mon[111056]: pgmap v527: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:09 smithi043 ceph-mon[81510]: pgmap v527: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:12.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:11 smithi042 ceph-mon[111056]: pgmap v528: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:12.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:11 smithi043 ceph-mon[81510]: pgmap v528: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:14.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:13 smithi042 ceph-mon[111056]: pgmap v529: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:14.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:13 smithi043 ceph-mon[81510]: pgmap v529: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:16.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:15 smithi042 ceph-mon[111056]: pgmap v530: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:16.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:15 smithi043 ceph-mon[81510]: pgmap v530: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:18.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:17 smithi042 ceph-mon[111056]: pgmap v531: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:18.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:17 smithi043 ceph-mon[81510]: pgmap v531: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:20.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:19 smithi042 ceph-mon[111056]: pgmap v532: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:20.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:19 smithi043 ceph-mon[81510]: pgmap v532: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:21.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:21.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:22.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:22 smithi042 ceph-mon[111056]: pgmap v533: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:22.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:22 smithi043 ceph-mon[81510]: pgmap v533: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:24.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:24 smithi042 ceph-mon[111056]: pgmap v534: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:24.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:24 smithi043 ceph-mon[81510]: pgmap v534: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:26.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:26 smithi042 ceph-mon[111056]: pgmap v535: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:26.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:26 smithi043 ceph-mon[81510]: pgmap v535: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:28.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:28 smithi042 ceph-mon[111056]: pgmap v536: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:28.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:28 smithi043 ceph-mon[81510]: pgmap v536: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:30.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:30 smithi042 ceph-mon[111056]: pgmap v537: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:30.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:30 smithi043 ceph-mon[81510]: pgmap v537: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.359+0000 7ff9e2e28700 1 -- 172.21.15.42:0/1833189974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 msgr2=0x7ff9dc10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/1833189974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc10abf0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009910 tx=0x7ff9cc009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 -- 172.21.15.42:0/1833189974 shutdown_connections 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/1833189974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc10abf0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/1833189974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9dc101de0 0x7ff9dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 -- 172.21.15.42:0/1833189974 >> 172.21.15.42:0/1833189974 conn(0x7ff9dc0fb5f0 msgr2=0x7ff9dc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 -- 172.21.15.42:0/1833189974 shutdown_connections 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 -- 172.21.15.42:0/1833189974 wait complete. 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 Processor -- start 2024-09-17T00:09:31.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.360+0000 7ff9e2e28700 1 -- start start 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9e2e28700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9dc101de0 0x7ff9dc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9e2e28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9e2e28700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc19f470 con 0x7ff9dc102700 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9e2e28700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc198920 con 0x7ff9dc101de0 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9dbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46344/0 (socket says 172.21.15.42:46344) 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9dbfff700 1 -- 172.21.15.42:0/149045163 learned_addr learned my addr 172.21.15.42:0/149045163 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9e0bc4700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9dc101de0 0x7ff9dc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:31.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9dbfff700 1 -- 172.21.15.42:0/149045163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9dc101de0 msgr2=0x7ff9dc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9dbfff700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9dc101de0 0x7ff9dc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9dbfff700 1 -- 172.21.15.42:0/149045163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc0095f0 con 0x7ff9dc102700 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9dbfff700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc19ede0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0092e0 tx=0x7ff9cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.361+0000 7ff9d9ffb700 1 -- 172.21.15.42:0/149045163 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9cc01d070 con 0x7ff9dc102700 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.362+0000 7ff9d9ffb700 1 -- 172.21.15.42:0/149045163 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9cc00f460 con 0x7ff9dc102700 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.362+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9dc198ba0 con 0x7ff9dc102700 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.362+0000 7ff9d9ffb700 1 -- 172.21.15.42:0/149045163 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9cc017610 con 0x7ff9dc102700 2024-09-17T00:09:31.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.362+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9dc199090 con 0x7ff9dc102700 2024-09-17T00:09:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.363+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9dc10a750 con 0x7ff9dc102700 2024-09-17T00:09:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.363+0000 7ff9d9ffb700 1 -- 172.21.15.42:0/149045163 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff9cc00f5d0 con 0x7ff9dc102700 2024-09-17T00:09:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.364+0000 7ff9d9ffb700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff9c806e680 0x7ff9c8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.364+0000 7ff9d9ffb700 1 -- 172.21.15.42:0/149045163 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff9cc08fb00 con 0x7ff9dc102700 2024-09-17T00:09:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.364+0000 7ff9e0bc4700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff9c806e680 0x7ff9c8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:31.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.364+0000 7ff9e0bc4700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff9c806e680 0x7ff9c8070b40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005950 tx=0x7ff9c4005eb0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:31.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.369+0000 7ff9d9ffb700 1 -- 172.21.15.42:0/149045163 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff9cc05c7b0 con 0x7ff9dc102700 2024-09-17T00:09:31.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.597+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9dc199300 con 0x7ff9c806e680 2024-09-17T00:09:31.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.598+0000 7ff9d9ffb700 1 -- 172.21.15.42:0/149045163 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff9dc199300 con 0x7ff9c806e680 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff9c806e680 msgr2=0x7ff9c8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff9c806e680 0x7ff9c8070b40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4005950 tx=0x7ff9c4005eb0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 msgr2=0x7ff9dc19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc19ede0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0092e0 tx=0x7ff9cc00be20 comp rx=0 tx=0).stop 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 shutdown_connections 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff9c806e680 0x7ff9c8070b40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9dc101de0 0x7ff9dc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 --2- 172.21.15.42:0/149045163 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff9dc102700 0x7ff9dc19ede0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 >> 172.21.15.42:0/149045163 conn(0x7ff9dc0fb5f0 msgr2=0x7ff9dc100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:31.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 shutdown_connections 2024-09-17T00:09:31.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.600+0000 7ff9e2e28700 1 -- 172.21.15.42:0/149045163 wait complete. 2024-09-17T00:09:31.609 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:09:31.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 -- 172.21.15.42:0/2376970113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 msgr2=0x7f3e541047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/2376970113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e541047a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009910 tx=0x7f3e3c009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:31.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 -- 172.21.15.42:0/2376970113 shutdown_connections 2024-09-17T00:09:31.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/2376970113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e541047a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/2376970113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e540ffda0 0x7f3e54100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 -- 172.21.15.42:0/2376970113 >> 172.21.15.42:0/2376970113 conn(0x7f3e540fb650 msgr2=0x7f3e540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:31.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 -- 172.21.15.42:0/2376970113 shutdown_connections 2024-09-17T00:09:31.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 -- 172.21.15.42:0/2376970113 wait complete. 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 Processor -- start 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.711+0000 7f3e5a17d700 1 -- start start 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e5a17d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e540ffda0 0x7f3e54196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e5a17d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e54196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e5a17d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54197520 con 0x7f3e54100750 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e5a17d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5419b330 con 0x7f3e540ffda0 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e52ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e54196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e52ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e54196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46358/0 (socket says 172.21.15.42:46358) 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e52ffd700 1 -- 172.21.15.42:0/1652855876 learned_addr learned my addr 172.21.15.42:0/1652855876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e537fe700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e540ffda0 0x7f3e54196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:31.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e52ffd700 1 -- 172.21.15.42:0/1652855876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e540ffda0 msgr2=0x7f3e54196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e52ffd700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e540ffda0 0x7f3e54196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e52ffd700 1 -- 172.21.15.42:0/1652855876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c0095f0 con 0x7f3e54100750 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.712+0000 7f3e52ffd700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e54196e90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00bd70 tx=0x7f3e3c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.713+0000 7f3e50ff9700 1 -- 172.21.15.42:0/1652855876 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c01d070 con 0x7f3e54100750 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.713+0000 7f3e50ff9700 1 -- 172.21.15.42:0/1652855876 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e3c005dc0 con 0x7f3e54100750 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.713+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5419b5b0 con 0x7f3e54100750 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.713+0000 7f3e50ff9700 1 -- 172.21.15.42:0/1652855876 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c017400 con 0x7f3e54100750 2024-09-17T00:09:31.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.713+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5419baa0 con 0x7f3e54100750 2024-09-17T00:09:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.714+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5404f2e0 con 0x7f3e54100750 2024-09-17T00:09:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.714+0000 7f3e50ff9700 1 -- 172.21.15.42:0/1652855876 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3e3c003a60 con 0x7f3e54100750 2024-09-17T00:09:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.715+0000 7f3e50ff9700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3e4006e680 0x7f3e40070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.715+0000 7f3e50ff9700 1 -- 172.21.15.42:0/1652855876 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3e3c08fad0 con 0x7f3e54100750 2024-09-17T00:09:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.715+0000 7f3e537fe700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3e4006e680 0x7f3e40070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:31.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.715+0000 7f3e537fe700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3e4006e680 0x7f3e40070b40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005950 tx=0x7f3e4400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:31.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.719+0000 7f3e50ff9700 1 -- 172.21.15.42:0/1652855876 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3e3c05c7a0 con 0x7f3e54100750 2024-09-17T00:09:31.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.949+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e5419bcc0 con 0x7f3e4006e680 2024-09-17T00:09:31.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.950+0000 7f3e50ff9700 1 -- 172.21.15.42:0/1652855876 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3e5419bcc0 con 0x7f3e4006e680 2024-09-17T00:09:31.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.952+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3e4006e680 msgr2=0x7f3e40070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.952+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3e4006e680 0x7f3e40070b40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005950 tx=0x7f3e4400b410 comp rx=0 tx=0).stop 2024-09-17T00:09:31.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.952+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 msgr2=0x7f3e54196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:31.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.952+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e54196e90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00bd70 tx=0x7f3e3c00bda0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.953+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 shutdown_connections 2024-09-17T00:09:31.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.953+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3e4006e680 0x7f3e40070b40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.953+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e540ffda0 0x7f3e54196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.953+0000 7f3e5a17d700 1 --2- 172.21.15.42:0/1652855876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3e54100750 0x7f3e54196e90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:31.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.953+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 >> 172.21.15.42:0/1652855876 conn(0x7f3e540fb650 msgr2=0x7f3e54103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:31.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.953+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 shutdown_connections 2024-09-17T00:09:31.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:31.953+0000 7f3e5a17d700 1 -- 172.21.15.42:0/1652855876 wait complete. 2024-09-17T00:09:32.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 -- 172.21.15.42:0/3201928361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 msgr2=0x7f1de4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 --2- 172.21.15.42:0/3201928361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de4100180 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009910 tx=0x7f1dd4009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:32.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 -- 172.21.15.42:0/3201928361 shutdown_connections 2024-09-17T00:09:32.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 --2- 172.21.15.42:0/3201928361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de4100750 0x7f1de41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 --2- 172.21.15.42:0/3201928361 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de4100180 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 -- 172.21.15.42:0/3201928361 >> 172.21.15.42:0/3201928361 conn(0x7f1de40fb650 msgr2=0x7f1de40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:32.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 -- 172.21.15.42:0/3201928361 shutdown_connections 2024-09-17T00:09:32.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.062+0000 7f1de9746700 1 -- 172.21.15.42:0/3201928361 wait complete. 2024-09-17T00:09:32.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de9746700 1 Processor -- start 2024-09-17T00:09:32.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de9746700 1 -- start start 2024-09-17T00:09:32.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de9746700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de419b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de9746700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de4100750 0x7f1de419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de9746700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de419c120 con 0x7f1de40ffda0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de9746700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de41955d0 con 0x7f1de4100750 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de419b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de419b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46382/0 (socket says 172.21.15.42:46382) 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.063+0000 7f1de2ffd700 1 -- 172.21.15.42:0/1748002496 learned_addr learned my addr 172.21.15.42:0/1748002496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1de27fc700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de4100750 0x7f1de419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1de2ffd700 1 -- 172.21.15.42:0/1748002496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de4100750 msgr2=0x7f1de419ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1de2ffd700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de4100750 0x7f1de419ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1de2ffd700 1 -- 172.21.15.42:0/1748002496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40095f0 con 0x7f1de40ffda0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1de2ffd700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de419b550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4006010 tx=0x7f1dd400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1ddbfff700 1 -- 172.21.15.42:0/1748002496 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd401d070 con 0x7f1de40ffda0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1ddbfff700 1 -- 172.21.15.42:0/1748002496 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dd4004fb0 con 0x7f1de40ffda0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de4195850 con 0x7f1de40ffda0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1ddbfff700 1 -- 172.21.15.42:0/1748002496 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dd400f690 con 0x7f1de40ffda0 2024-09-17T00:09:32.063 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.064+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de4195d40 con 0x7f1de40ffda0 2024-09-17T00:09:32.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.065+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de4108140 con 0x7f1de40ffda0 2024-09-17T00:09:32.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.066+0000 7f1ddbfff700 1 -- 172.21.15.42:0/1748002496 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1dd40228a0 con 0x7f1de40ffda0 2024-09-17T00:09:32.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.066+0000 7f1ddbfff700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dd006e680 0x7f1dd0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.066+0000 7f1ddbfff700 1 -- 172.21.15.42:0/1748002496 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1dd408fe50 con 0x7f1de40ffda0 2024-09-17T00:09:32.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.066+0000 7f1de27fc700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dd006e680 0x7f1dd0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.067+0000 7f1de27fc700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dd006e680 0x7f1dd0070b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc005fd0 tx=0x7f1dcc00a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.071+0000 7f1ddbfff700 1 -- 172.21.15.42:0/1748002496 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1dd405cba0 con 0x7f1de40ffda0 2024-09-17T00:09:32.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:32 smithi042 ceph-mon[111056]: pgmap v538: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:32.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:32 smithi042 ceph-mon[111056]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:32.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.289+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1de4196c00 con 0x7f1dd006e680 2024-09-17T00:09:32.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.299+0000 7f1ddbfff700 1 -- 172.21.15.42:0/1748002496 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f1dcc00a2d0 con 0x7f1dd006e680 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (18m) 11m ago 28m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (29m) 11m ago 29m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (28m) 11m ago 28m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (15m) 11m ago 29m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (15m) 11m ago 28m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (17m) 11m ago 28m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (11m) 11m ago 23m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (12m) 11m ago 23m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (11m) 11m ago 23m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (11m) 11m ago 23m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:09:32.297 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (20m) 11m ago 30m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (19m) 11m ago 27m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (16m) 11m ago 30m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (16m) 11m ago 27m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (18m) 11m ago 29m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (18m) 11m ago 27m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (15m) 11m ago 27m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (14m) 11m ago 26m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (14m) 11m ago 26m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (13m) 11m ago 25m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (13m) 11m ago 25m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (12m) 11m ago 24m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:09:32.298 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (18m) 11m ago 28m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:09:32.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dd006e680 msgr2=0x7f1dd0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dd006e680 0x7f1dd0070b40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc005fd0 tx=0x7f1dcc00a560 comp rx=0 tx=0).stop 2024-09-17T00:09:32.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 msgr2=0x7f1de419b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de419b550 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4006010 tx=0x7f1dd400be20 comp rx=0 tx=0).stop 2024-09-17T00:09:32.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 shutdown_connections 2024-09-17T00:09:32.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1dd006e680 0x7f1dd0070b40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1de40ffda0 0x7f1de419b550 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 --2- 172.21.15.42:0/1748002496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1de4100750 0x7f1de419ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 >> 172.21.15.42:0/1748002496 conn(0x7f1de40fb650 msgr2=0x7f1de40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:32.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 shutdown_connections 2024-09-17T00:09:32.300 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.301+0000 7f1de9746700 1 -- 172.21.15.42:0/1748002496 wait complete. 2024-09-17T00:09:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:32 smithi043 ceph-mon[81510]: pgmap v538: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:32.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:32 smithi043 ceph-mon[81510]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 -- 172.21.15.42:0/2729341196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 msgr2=0x7f9050100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 --2- 172.21.15.42:0/2729341196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050100080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9040009910 tx=0x7f9040009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 -- 172.21.15.42:0/2729341196 shutdown_connections 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 --2- 172.21.15.42:0/2729341196 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050100080 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 --2- 172.21.15.42:0/2729341196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9050105bd0 0x7f9050105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 -- 172.21.15.42:0/2729341196 >> 172.21.15.42:0/2729341196 conn(0x7f9050075670 msgr2=0x7f9050075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 -- 172.21.15.42:0/2729341196 shutdown_connections 2024-09-17T00:09:32.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.409+0000 7f90573a1700 1 -- 172.21.15.42:0/2729341196 wait complete. 2024-09-17T00:09:32.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f90573a1700 1 Processor -- start 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f90573a1700 1 -- start start 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f90573a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f90573a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9050105bd0 0x7f9050101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f90573a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90501022c0 con 0x7f90500ffc20 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f90573a1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9050102400 con 0x7f9050105bd0 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f905513d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f905513d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46402/0 (socket says 172.21.15.42:46402) 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.410+0000 7f905513d700 1 -- 172.21.15.42:0/537140221 learned_addr learned my addr 172.21.15.42:0/537140221 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f905493c700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9050105bd0 0x7f9050101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f905513d700 1 -- 172.21.15.42:0/537140221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9050105bd0 msgr2=0x7f9050101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f905513d700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9050105bd0 0x7f9050101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f905513d700 1 -- 172.21.15.42:0/537140221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90400095f0 con 0x7f90500ffc20 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f905513d700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050101840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d8d0 tx=0x7f904c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:32.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f90467fc700 1 -- 172.21.15.42:0/537140221 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f904c009940 con 0x7f90500ffc20 2024-09-17T00:09:32.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f90467fc700 1 -- 172.21.15.42:0/537140221 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f904c010460 con 0x7f90500ffc20 2024-09-17T00:09:32.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90501a2a30 con 0x7f90500ffc20 2024-09-17T00:09:32.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f90467fc700 1 -- 172.21.15.42:0/537140221 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f904c00f5f0 con 0x7f90500ffc20 2024-09-17T00:09:32.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.411+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90501a2e40 con 0x7f90500ffc20 2024-09-17T00:09:32.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.412+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905004f2e0 con 0x7f90500ffc20 2024-09-17T00:09:32.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.413+0000 7f90467fc700 1 -- 172.21.15.42:0/537140221 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f904c0105d0 con 0x7f90500ffc20 2024-09-17T00:09:32.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.413+0000 7f90467fc700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f903c06e680 0x7f903c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.413+0000 7f90467fc700 1 -- 172.21.15.42:0/537140221 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f904c08e680 con 0x7f90500ffc20 2024-09-17T00:09:32.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.413+0000 7f905493c700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f903c06e680 0x7f903c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.414+0000 7f905493c700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f903c06e680 0x7f903c070b40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f904000b5c0 tx=0x7f9040005960 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:32.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.418+0000 7f90467fc700 1 -- 172.21.15.42:0/537140221 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f904c057820 con 0x7f90500ffc20 2024-09-17T00:09:32.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.714+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f90501a3100 con 0x7f90500ffc20 2024-09-17T00:09:32.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.714+0000 7f90467fc700 1 -- 172.21.15.42:0/537140221 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f904c05aee0 con 0x7f90500ffc20 2024-09-17T00:09:32.712 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:09:32.712 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:09:32.712 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:09:32.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:32.712 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:09:32.712 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:09:32.713 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:09:32.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f903c06e680 msgr2=0x7f903c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f903c06e680 0x7f903c070b40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f904000b5c0 tx=0x7f9040005960 comp rx=0 tx=0).stop 2024-09-17T00:09:32.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 msgr2=0x7f9050101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050101840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d8d0 tx=0x7f904c00dc90 comp rx=0 tx=0).stop 2024-09-17T00:09:32.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 shutdown_connections 2024-09-17T00:09:32.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f903c06e680 0x7f903c070b40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f90500ffc20 0x7f9050101840 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 --2- 172.21.15.42:0/537140221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9050105bd0 0x7f9050101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 >> 172.21.15.42:0/537140221 conn(0x7f9050075670 msgr2=0x7f9050068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:32.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 shutdown_connections 2024-09-17T00:09:32.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.717+0000 7f90573a1700 1 -- 172.21.15.42:0/537140221 wait complete. 2024-09-17T00:09:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 -- 172.21.15.42:0/1364862447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49700ffc20 msgr2=0x7f4970100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 --2- 172.21.15.42:0/1364862447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49700ffc20 0x7f4970100080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4958009910 tx=0x7f4958009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 -- 172.21.15.42:0/1364862447 shutdown_connections 2024-09-17T00:09:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 --2- 172.21.15.42:0/1364862447 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49700ffc20 0x7f4970100080 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 --2- 172.21.15.42:0/1364862447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4970105bd0 0x7f4970105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 -- 172.21.15.42:0/1364862447 >> 172.21.15.42:0/1364862447 conn(0x7f4970075670 msgr2=0x7f4970075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:32.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 -- 172.21.15.42:0/1364862447 shutdown_connections 2024-09-17T00:09:32.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.825+0000 7f4975b8e700 1 -- 172.21.15.42:0/1364862447 wait complete. 2024-09-17T00:09:32.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f4975b8e700 1 Processor -- start 2024-09-17T00:09:32.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f4975b8e700 1 -- start start 2024-09-17T00:09:32.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f4975b8e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49700ffc20 0x7f497019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f4975b8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4970105bd0 0x7f497019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f4975b8e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970198900 con 0x7f4970105bd0 2024-09-17T00:09:32.825 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f4975b8e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970198a70 con 0x7f49700ffc20 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f496effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4970105bd0 0x7f497019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f496effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4970105bd0 0x7f497019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43764/0 (socket says 172.21.15.42:43764) 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.826+0000 7f496effd700 1 -- 172.21.15.42:0/3089932514 learned_addr learned my addr 172.21.15.42:0/3089932514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496f7fe700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49700ffc20 0x7f497019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496effd700 1 -- 172.21.15.42:0/3089932514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49700ffc20 msgr2=0x7f497019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496effd700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49700ffc20 0x7f497019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496effd700 1 -- 172.21.15.42:0/3089932514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49580095f0 con 0x7f4970105bd0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496effd700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4970105bd0 0x7f497019ee50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f495800f690 tx=0x7f495800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496cff9700 1 -- 172.21.15.42:0/3089932514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495801d070 con 0x7f4970105bd0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496cff9700 1 -- 172.21.15.42:0/3089932514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49580050d0 con 0x7f4970105bd0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4970198cf0 con 0x7f4970105bd0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f496cff9700 1 -- 172.21.15.42:0/3089932514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4958022470 con 0x7f4970105bd0 2024-09-17T00:09:32.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.827+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49701991e0 con 0x7f4970105bd0 2024-09-17T00:09:32.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.828+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497004f2e0 con 0x7f4970105bd0 2024-09-17T00:09:32.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.829+0000 7f496cff9700 1 -- 172.21.15.42:0/3089932514 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f4958003680 con 0x7f4970105bd0 2024-09-17T00:09:32.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.829+0000 7f496cff9700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f495c06e680 0x7f495c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:32.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.829+0000 7f496cff9700 1 -- 172.21.15.42:0/3089932514 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f495808fa80 con 0x7f4970105bd0 2024-09-17T00:09:32.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.829+0000 7f496f7fe700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f495c06e680 0x7f495c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:32.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.830+0000 7f496f7fe700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f495c06e680 0x7f495c070b40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4960009bf0 tx=0x7f4960009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:32.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:32.834+0000 7f496cff9700 1 -- 172.21.15.42:0/3089932514 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4958058ca0 con 0x7f4970105bd0 2024-09-17T00:09:33.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.092+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4970199450 con 0x7f4970105bd0 2024-09-17T00:09:33.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.093+0000 7f496cff9700 1 -- 172.21.15.42:0/3089932514 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f495805c360 con 0x7f4970105bd0 2024-09-17T00:09:33.092 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:09:33.093 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:09:33.093 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:09:33.093 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:09:33.093 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:09:33.094 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:09:33.095 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:33.096 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:09:33.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f495c06e680 msgr2=0x7f495c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f495c06e680 0x7f495c070b40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4960009bf0 tx=0x7f4960009260 comp rx=0 tx=0).stop 2024-09-17T00:09:33.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4970105bd0 msgr2=0x7f497019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4970105bd0 0x7f497019ee50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f495800f690 tx=0x7f495800f770 comp rx=0 tx=0).stop 2024-09-17T00:09:33.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 shutdown_connections 2024-09-17T00:09:33.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f495c06e680 0x7f495c070b40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49700ffc20 0x7f497019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 --2- 172.21.15.42:0/3089932514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4970105bd0 0x7f497019ee50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 >> 172.21.15.42:0/3089932514 conn(0x7f4970075670 msgr2=0x7f4970068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:33.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 shutdown_connections 2024-09-17T00:09:33.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.095+0000 7f4975b8e700 1 -- 172.21.15.42:0/3089932514 wait complete. 2024-09-17T00:09:33.098 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:09:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.208+0000 7fddf946b700 1 -- 172.21.15.42:0/2922190545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 msgr2=0x7fddf41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.208+0000 7fddf946b700 1 --2- 172.21.15.42:0/2922190545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf41058a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009910 tx=0x7fdde8009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:33.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.209+0000 7fddf946b700 1 -- 172.21.15.42:0/2922190545 shutdown_connections 2024-09-17T00:09:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.209+0000 7fddf946b700 1 --2- 172.21.15.42:0/2922190545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf41058a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.209+0000 7fddf946b700 1 --2- 172.21.15.42:0/2922190545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4100ea0 0x7fddf4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.209+0000 7fddf946b700 1 -- 172.21.15.42:0/2922190545 >> 172.21.15.42:0/2922190545 conn(0x7fddf40fc750 msgr2=0x7fddf40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.209+0000 7fddf946b700 1 -- 172.21.15.42:0/2922190545 shutdown_connections 2024-09-17T00:09:33.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.209+0000 7fddf946b700 1 -- 172.21.15.42:0/2922190545 wait complete. 2024-09-17T00:09:33.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.209+0000 7fddf946b700 1 Processor -- start 2024-09-17T00:09:33.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf946b700 1 -- start start 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf946b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4100ea0 0x7fddf419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf946b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf946b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf419d2d0 con 0x7fddf4101850 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf946b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4196780 con 0x7fddf4100ea0 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43778/0 (socket says 172.21.15.42:43778) 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf27fc700 1 -- 172.21.15.42:0/2631015725 learned_addr learned my addr 172.21.15.42:0/2631015725 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf2ffd700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4100ea0 0x7fddf419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf27fc700 1 -- 172.21.15.42:0/2631015725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4100ea0 msgr2=0x7fddf419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf27fc700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4100ea0 0x7fddf419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf27fc700 1 -- 172.21.15.42:0/2631015725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde80095f0 con 0x7fddf4101850 2024-09-17T00:09:33.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.210+0000 7fddf2ffd700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4100ea0 0x7fddf419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:09:33.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.211+0000 7fddf27fc700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf419cc40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdde8000c00 tx=0x7fdde800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:33.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.211+0000 7fdddbfff700 1 -- 172.21.15.42:0/2631015725 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde801d070 con 0x7fddf4101850 2024-09-17T00:09:33.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.211+0000 7fdddbfff700 1 -- 172.21.15.42:0/2631015725 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdde8005dc0 con 0x7fddf4101850 2024-09-17T00:09:33.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.211+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf4196a00 con 0x7fddf4101850 2024-09-17T00:09:33.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.211+0000 7fdddbfff700 1 -- 172.21.15.42:0/2631015725 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdde8017400 con 0x7fddf4101850 2024-09-17T00:09:33.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.211+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf4196ef0 con 0x7fddf4101850 2024-09-17T00:09:33.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.212+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf41092a0 con 0x7fddf4101850 2024-09-17T00:09:33.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.212+0000 7fdddbfff700 1 -- 172.21.15.42:0/2631015725 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fdde800f7f0 con 0x7fddf4101850 2024-09-17T00:09:33.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.213+0000 7fdddbfff700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdddc06e680 0x7fdddc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:33.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.213+0000 7fdddbfff700 1 -- 172.21.15.42:0/2631015725 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdde808fb90 con 0x7fddf4101850 2024-09-17T00:09:33.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.213+0000 7fddf2ffd700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdddc06e680 0x7fdddc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:33.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.213+0000 7fddf2ffd700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdddc06e680 0x7fdddc070b40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdde4005fd0 tx=0x7fdde4005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:33.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.217+0000 7fdddbfff700 1 -- 172.21.15.42:0/2631015725 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdde805c860 con 0x7fddf4101850 2024-09-17T00:09:33.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:33 smithi042 ceph-mon[111056]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:33 smithi042 ceph-mon[111056]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:33 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/537140221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:09:33.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:33 smithi043 ceph-mon[81510]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:33 smithi043 ceph-mon[81510]: from='client.35028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:33 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/537140221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:09:33.445 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.447+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddf4197110 con 0x7fdddc06e680 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.448+0000 7fdddbfff700 1 -- 172.21.15.42:0/2631015725 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fddf4197110 con 0x7fdddc06e680 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:09:33.446 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:09:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:09:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:09:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:09:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:09:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:09:33.447 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:09:33.447 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdddc06e680 msgr2=0x7fdddc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdddc06e680 0x7fdddc070b40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdde4005fd0 tx=0x7fdde4005ee0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 msgr2=0x7fddf419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf419cc40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdde8000c00 tx=0x7fdde800be20 comp rx=0 tx=0).stop 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 shutdown_connections 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdddc06e680 0x7fdddc070b40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf4100ea0 0x7fddf419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 --2- 172.21.15.42:0/2631015725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fddf4101850 0x7fddf419cc40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 >> 172.21.15.42:0/2631015725 conn(0x7fddf40fc750 msgr2=0x7fddf40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 shutdown_connections 2024-09-17T00:09:33.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.451+0000 7fddf946b700 1 -- 172.21.15.42:0/2631015725 wait complete. 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 -- 172.21.15.42:0/3641709381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3400ffb40 msgr2=0x7fb3400fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 --2- 172.21.15.42:0/3641709381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3400ffb40 0x7fb3400fffc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb330009910 tx=0x7fb330009c20 comp rx=0 tx=0).stop 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 -- 172.21.15.42:0/3641709381 shutdown_connections 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 --2- 172.21.15.42:0/3641709381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3400ffb40 0x7fb3400fffc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 --2- 172.21.15.42:0/3641709381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb340105bf0 0x7fb340105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 -- 172.21.15.42:0/3641709381 >> 172.21.15.42:0/3641709381 conn(0x7fb3400fb610 msgr2=0x7fb3400fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 -- 172.21.15.42:0/3641709381 shutdown_connections 2024-09-17T00:09:33.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.559+0000 7fb347321700 1 -- 172.21.15.42:0/3641709381 wait complete. 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb347321700 1 Processor -- start 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb347321700 1 -- start start 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb347321700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3400ffb40 0x7fb34019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb347321700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340105bf0 0x7fb34019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb347321700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3401988e0 con 0x7fb340105bf0 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb347321700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340198a50 con 0x7fb3400ffb40 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb3448bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340105bf0 0x7fb34019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb3448bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340105bf0 0x7fb34019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43788/0 (socket says 172.21.15.42:43788) 2024-09-17T00:09:33.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb3448bc700 1 -- 172.21.15.42:0/1942107016 learned_addr learned my addr 172.21.15.42:0/1942107016 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.560+0000 7fb3450bd700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3400ffb40 0x7fb34019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb3448bc700 1 -- 172.21.15.42:0/1942107016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3400ffb40 msgr2=0x7fb34019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb3448bc700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3400ffb40 0x7fb34019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb3448bc700 1 -- 172.21.15.42:0/1942107016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300095f0 con 0x7fb340105bf0 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb3448bc700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340105bf0 0x7fb34019ee30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb330006010 tx=0x7fb33000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb3367fc700 1 -- 172.21.15.42:0/1942107016 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33001d070 con 0x7fb340105bf0 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb3367fc700 1 -- 172.21.15.42:0/1942107016 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb33000f460 con 0x7fb340105bf0 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb340198cd0 con 0x7fb340105bf0 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb3367fc700 1 -- 172.21.15.42:0/1942107016 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb330017630 con 0x7fb340105bf0 2024-09-17T00:09:33.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.561+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3401991c0 con 0x7fb340105bf0 2024-09-17T00:09:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.562+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3400679c0 con 0x7fb340105bf0 2024-09-17T00:09:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.563+0000 7fb3367fc700 1 -- 172.21.15.42:0/1942107016 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb33000f5d0 con 0x7fb340105bf0 2024-09-17T00:09:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.563+0000 7fb3367fc700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb32c06e680 0x7fb32c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:09:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.563+0000 7fb3367fc700 1 -- 172.21.15.42:0/1942107016 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb33008fab0 con 0x7fb340105bf0 2024-09-17T00:09:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.563+0000 7fb3450bd700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb32c06e680 0x7fb32c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:09:33.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.564+0000 7fb3450bd700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb32c06e680 0x7fb32c070b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0060b0 tx=0x7fb33c00c040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:09:33.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.568+0000 7fb3367fc700 1 -- 172.21.15.42:0/1942107016 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb330058cd0 con 0x7fb340105bf0 2024-09-17T00:09:33.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.863+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb34004f2e0 con 0x7fb340105bf0 2024-09-17T00:09:33.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.864+0000 7fb3367fc700 1 -- 172.21.15.42:0/1942107016 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb33005c390 con 0x7fb340105bf0 2024-09-17T00:09:33.861 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:09:33.861 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:09:33.861 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:09:33.862 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:09:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:09:33.862 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:09:33.862 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:09:33.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb32c06e680 msgr2=0x7fb32c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb32c06e680 0x7fb32c070b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0060b0 tx=0x7fb33c00c040 comp rx=0 tx=0).stop 2024-09-17T00:09:33.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340105bf0 msgr2=0x7fb34019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340105bf0 0x7fb34019ee30 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb330006010 tx=0x7fb33000bd70 comp rx=0 tx=0).stop 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 shutdown_connections 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb32c06e680 0x7fb32c070b40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3400ffb40 0x7fb34019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 --2- 172.21.15.42:0/1942107016 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb340105bf0 0x7fb34019ee30 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 >> 172.21.15.42:0/1942107016 conn(0x7fb3400fb610 msgr2=0x7fb340068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 shutdown_connections 2024-09-17T00:09:33.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:09:33.866+0000 7fb347321700 1 -- 172.21.15.42:0/1942107016 wait complete. 2024-09-17T00:09:34.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:34 smithi042 ceph-mon[111056]: pgmap v539: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:34 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3089932514' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:09:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:34 smithi042 ceph-mon[111056]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:34 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1942107016' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:09:34.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:34 smithi043 ceph-mon[81510]: pgmap v539: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:34 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3089932514' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:09:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:34 smithi043 ceph-mon[81510]: from='client.35040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:09:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:34 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1942107016' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:09:36.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:36 smithi042 ceph-mon[111056]: pgmap v540: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:36.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:36.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:36 smithi043 ceph-mon[81510]: pgmap v540: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:36.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:38.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:38 smithi043 ceph-mon[81510]: pgmap v541: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:38 smithi042 ceph-mon[111056]: pgmap v541: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:40.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:40 smithi043 ceph-mon[81510]: pgmap v542: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:40.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:40 smithi042 ceph-mon[111056]: pgmap v542: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:42.295 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:42 smithi043 ceph-mon[81510]: pgmap v543: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:42.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:42 smithi042 ceph-mon[111056]: pgmap v543: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:44.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:44 smithi042 ceph-mon[111056]: pgmap v544: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:44.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:44 smithi043 ceph-mon[81510]: pgmap v544: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:46.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:46 smithi043 ceph-mon[81510]: pgmap v545: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:46.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:46 smithi042 ceph-mon[111056]: pgmap v545: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:48.300 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:48 smithi042 ceph-mon[111056]: pgmap v546: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:48.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:48 smithi043 ceph-mon[81510]: pgmap v546: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:50.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:50 smithi043 ceph-mon[81510]: pgmap v547: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:50.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:50 smithi042 ceph-mon[111056]: pgmap v547: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:51.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:51.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:09:52.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:52 smithi043 ceph-mon[81510]: pgmap v548: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:52.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:52 smithi042 ceph-mon[111056]: pgmap v548: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:54.317 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:54 smithi042 ceph-mon[111056]: pgmap v549: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:54.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:54 smithi043 ceph-mon[81510]: pgmap v549: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:56.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:56 smithi043 ceph-mon[81510]: pgmap v550: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:56.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:56 smithi042 ceph-mon[111056]: pgmap v550: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:58.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:09:58 smithi043 ceph-mon[81510]: pgmap v551: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:09:58.405 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:09:58 smithi042 ceph-mon[111056]: pgmap v551: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:00 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:09:59.999+0000 7fa278873640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:10:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:00 smithi042 ceph-mon[111056]: pgmap v552: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:00.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:00 smithi042 ceph-mon[111056]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:10:00.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:00 smithi043 ceph-mon[81510]: pgmap v552: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:00.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:00 smithi043 ceph-mon[81510]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:10:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:02 smithi043 ceph-mon[81510]: pgmap v553: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:02.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:02 smithi042 ceph-mon[111056]: pgmap v553: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 -- 172.21.15.42:0/1305106662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 msgr2=0x7fcdb4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 --2- 172.21.15.42:0/1305106662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb4100180 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009910 tx=0x7fcdb0009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 -- 172.21.15.42:0/1305106662 shutdown_connections 2024-09-17T00:10:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 --2- 172.21.15.42:0/1305106662 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdb4100750 0x7fcdb41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 --2- 172.21.15.42:0/1305106662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb4100180 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 -- 172.21.15.42:0/1305106662 >> 172.21.15.42:0/1305106662 conn(0x7fcdb40fb650 msgr2=0x7fcdb40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:03.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 -- 172.21.15.42:0/1305106662 shutdown_connections 2024-09-17T00:10:03.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.989+0000 7fcdbb493700 1 -- 172.21.15.42:0/1305106662 wait complete. 2024-09-17T00:10:03.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdbb493700 1 Processor -- start 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdbb493700 1 -- start start 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdbb493700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdbb493700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdb4100750 0x7fcdb419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdbb493700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb419d280 con 0x7fcdb40ffda0 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdbb493700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4196700 con 0x7fcdb4100750 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdb922f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdb922f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb419c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51284/0 (socket says 172.21.15.42:51284) 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.990+0000 7fcdb922f700 1 -- 172.21.15.42:0/3533371393 learned_addr learned my addr 172.21.15.42:0/3533371393 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdb8a2e700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdb4100750 0x7fcdb419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdb922f700 1 -- 172.21.15.42:0/3533371393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdb4100750 msgr2=0x7fcdb419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdb922f700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdb4100750 0x7fcdb419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdb922f700 1 -- 172.21.15.42:0/3533371393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb00095f0 con 0x7fcdb40ffda0 2024-09-17T00:10:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdb922f700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb419c660 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0006010 tx=0x7fcdb000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdaa7fc700 1 -- 172.21.15.42:0/3533371393 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb001d070 con 0x7fcdb40ffda0 2024-09-17T00:10:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdaa7fc700 1 -- 172.21.15.42:0/3533371393 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdb000f460 con 0x7fcdb40ffda0 2024-09-17T00:10:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb4196980 con 0x7fcdb40ffda0 2024-09-17T00:10:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdaa7fc700 1 -- 172.21.15.42:0/3533371393 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb0017630 con 0x7fcdb40ffda0 2024-09-17T00:10:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.991+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb4196e70 con 0x7fcdb40ffda0 2024-09-17T00:10:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.992+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb404f2e0 con 0x7fcdb40ffda0 2024-09-17T00:10:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.993+0000 7fcdaa7fc700 1 -- 172.21.15.42:0/3533371393 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fcdb000f5d0 con 0x7fcdb40ffda0 2024-09-17T00:10:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.993+0000 7fcdaa7fc700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcda006e680 0x7fcda0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.993+0000 7fcdaa7fc700 1 -- 172.21.15.42:0/3533371393 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcdb008fab0 con 0x7fcdb40ffda0 2024-09-17T00:10:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.993+0000 7fcdb8a2e700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcda006e680 0x7fcda0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.994+0000 7fcdb8a2e700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcda006e680 0x7fcda0070b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005950 tx=0x7fcda40058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:03.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:03.998+0000 7fcdaa7fc700 1 -- 172.21.15.42:0/3533371393 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcdb005c800 con 0x7fcdb40ffda0 2024-09-17T00:10:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.231+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdb41971a0 con 0x7fcda006e680 2024-09-17T00:10:04.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.232+0000 7fcdaa7fc700 1 -- 172.21.15.42:0/3533371393 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcdb41971a0 con 0x7fcda006e680 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcda006e680 msgr2=0x7fcda0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcda006e680 0x7fcda0070b40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005950 tx=0x7fcda40058e0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 msgr2=0x7fcdb419c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb419c660 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0006010 tx=0x7fcdb000be20 comp rx=0 tx=0).stop 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 shutdown_connections 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcda006e680 0x7fcda0070b40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcdb40ffda0 0x7fcdb419c660 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 --2- 172.21.15.42:0/3533371393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdb4100750 0x7fcdb419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 >> 172.21.15.42:0/3533371393 conn(0x7fcdb40fb650 msgr2=0x7fcdb4103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:04.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 shutdown_connections 2024-09-17T00:10:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.234+0000 7fcdbb493700 1 -- 172.21.15.42:0/3533371393 wait complete. 2024-09-17T00:10:04.242 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:10:04.243 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:04 smithi042 ceph-mon[111056]: pgmap v554: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 -- 172.21.15.42:0/1520785664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 msgr2=0x7f72c0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 --2- 172.21.15.42:0/1520785664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c0100180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009910 tx=0x7f72b0009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 -- 172.21.15.42:0/1520785664 shutdown_connections 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 --2- 172.21.15.42:0/1520785664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72c0100750 0x7f72c01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 --2- 172.21.15.42:0/1520785664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c0100180 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 -- 172.21.15.42:0/1520785664 >> 172.21.15.42:0/1520785664 conn(0x7f72c00fb650 msgr2=0x7f72c00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 -- 172.21.15.42:0/1520785664 shutdown_connections 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.345+0000 7f72c684e700 1 -- 172.21.15.42:0/1520785664 wait complete. 2024-09-17T00:10:04.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72c684e700 1 Processor -- start 2024-09-17T00:10:04.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72c684e700 1 -- start start 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72c684e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72c684e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72c0100750 0x7f72c019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72c684e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c019d2c0 con 0x7f72c00ffda0 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72c684e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0196770 con 0x7f72c0100750 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72bffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72bffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51300/0 (socket says 172.21.15.42:51300) 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72bffff700 1 -- 172.21.15.42:0/2416127208 learned_addr learned my addr 172.21.15.42:0/2416127208 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:04.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72bf7fe700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72c0100750 0x7f72c019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72bffff700 1 -- 172.21.15.42:0/2416127208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72c0100750 msgr2=0x7f72c019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72bffff700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72c0100750 0x7f72c019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.346+0000 7f72bffff700 1 -- 172.21.15.42:0/2416127208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00095f0 con 0x7f72c00ffda0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.347+0000 7f72bffff700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c019c6f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005f50 tx=0x7f72b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.347+0000 7f72bd7fa700 1 -- 172.21.15.42:0/2416127208 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b001d070 con 0x7f72c00ffda0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.347+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c01969f0 con 0x7f72c00ffda0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.347+0000 7f72bd7fa700 1 -- 172.21.15.42:0/2416127208 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b000b610 con 0x7f72c00ffda0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.347+0000 7f72bd7fa700 1 -- 172.21.15.42:0/2416127208 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b0022470 con 0x7f72c00ffda0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.347+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c0196ee0 con 0x7f72c00ffda0 2024-09-17T00:10:04.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.348+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c004f2e0 con 0x7f72c00ffda0 2024-09-17T00:10:04.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.348+0000 7f72bd7fa700 1 -- 172.21.15.42:0/2416127208 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f72b0003780 con 0x7f72c00ffda0 2024-09-17T00:10:04.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.349+0000 7f72bd7fa700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72a806e680 0x7f72a8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:04.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.349+0000 7f72bd7fa700 1 -- 172.21.15.42:0/2416127208 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f72b008fa40 con 0x7f72c00ffda0 2024-09-17T00:10:04.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.349+0000 7f72bf7fe700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72a806e680 0x7f72a8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:04.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.349+0000 7f72bf7fe700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72a806e680 0x7f72a8070b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f72c0197be0 tx=0x7f72b4009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:04.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.353+0000 7f72bd7fa700 1 -- 172.21.15.42:0/2416127208 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f72b005c710 con 0x7f72c00ffda0 2024-09-17T00:10:04.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:04 smithi043 ceph-mon[81510]: pgmap v554: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:04.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.584+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72c01971c0 con 0x7f72a806e680 2024-09-17T00:10:04.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.585+0000 7f72bd7fa700 1 -- 172.21.15.42:0/2416127208 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f72c01971c0 con 0x7f72a806e680 2024-09-17T00:10:04.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72a806e680 msgr2=0x7f72a8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72a806e680 0x7f72a8070b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f72c0197be0 tx=0x7f72b4009260 comp rx=0 tx=0).stop 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 msgr2=0x7f72c019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c019c6f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f72b0005f50 tx=0x7f72b000f690 comp rx=0 tx=0).stop 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 shutdown_connections 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f72a806e680 0x7f72a8070b40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72c00ffda0 0x7f72c019c6f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 --2- 172.21.15.42:0/2416127208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72c0100750 0x7f72c019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 >> 172.21.15.42:0/2416127208 conn(0x7f72c00fb650 msgr2=0x7f72c0103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 shutdown_connections 2024-09-17T00:10:04.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.587+0000 7f72c684e700 1 -- 172.21.15.42:0/2416127208 wait complete. 2024-09-17T00:10:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 -- 172.21.15.42:0/1525929574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 msgr2=0x7f7a241047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/1525929574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a241047a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009910 tx=0x7f7a10009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 -- 172.21.15.42:0/1525929574 shutdown_connections 2024-09-17T00:10:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/1525929574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a241047a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/1525929574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a240ffda0 0x7f7a24100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 -- 172.21.15.42:0/1525929574 >> 172.21.15.42:0/1525929574 conn(0x7f7a240fb650 msgr2=0x7f7a240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:04.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 -- 172.21.15.42:0/1525929574 shutdown_connections 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 -- 172.21.15.42:0/1525929574 wait complete. 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 Processor -- start 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.696+0000 7f7a2b14a700 1 -- start start 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a2b14a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a240ffda0 0x7f7a2419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a2b14a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a2419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a2b14a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2419d240 con 0x7f7a24100750 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a2b14a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a241966f0 con 0x7f7a240ffda0 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a23fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a2419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a23fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a2419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51316/0 (socket says 172.21.15.42:51316) 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a23fff700 1 -- 172.21.15.42:0/4075900675 learned_addr learned my addr 172.21.15.42:0/4075900675 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a28ee6700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a240ffda0 0x7f7a2419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a23fff700 1 -- 172.21.15.42:0/4075900675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a240ffda0 msgr2=0x7f7a2419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a23fff700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a240ffda0 0x7f7a2419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a23fff700 1 -- 172.21.15.42:0/4075900675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a100095f0 con 0x7f7a24100750 2024-09-17T00:10:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a23fff700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a2419cbb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7a100092e0 tx=0x7f7a1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a21ffb700 1 -- 172.21.15.42:0/4075900675 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a1001d070 con 0x7f7a24100750 2024-09-17T00:10:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a21ffb700 1 -- 172.21.15.42:0/4075900675 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a1000f460 con 0x7f7a24100750 2024-09-17T00:10:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a24196970 con 0x7f7a24100750 2024-09-17T00:10:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.697+0000 7f7a21ffb700 1 -- 172.21.15.42:0/4075900675 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a100175f0 con 0x7f7a24100750 2024-09-17T00:10:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.698+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a24196e60 con 0x7f7a24100750 2024-09-17T00:10:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.699+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2404f2e0 con 0x7f7a24100750 2024-09-17T00:10:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.699+0000 7f7a21ffb700 1 -- 172.21.15.42:0/4075900675 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f7a1000f5d0 con 0x7f7a24100750 2024-09-17T00:10:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.700+0000 7f7a21ffb700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7a1406e680 0x7f7a14070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.700+0000 7f7a21ffb700 1 -- 172.21.15.42:0/4075900675 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7a1008fb90 con 0x7f7a24100750 2024-09-17T00:10:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.700+0000 7f7a28ee6700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7a1406e680 0x7f7a14070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.700+0000 7f7a28ee6700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7a1406e680 0x7f7a14070b40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005fd0 tx=0x7f7a1800a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:04.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.704+0000 7f7a21ffb700 1 -- 172.21.15.42:0/4075900675 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7a1005c860 con 0x7f7a24100750 2024-09-17T00:10:04.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.923+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a241970f0 con 0x7f7a1406e680 2024-09-17T00:10:04.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.933+0000 7f7a21ffb700 1 -- 172.21.15.42:0/4075900675 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f7a241970f0 con 0x7f7a1406e680 2024-09-17T00:10:04.932 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:10:04.932 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (18m) 12m ago 29m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:10:04.932 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (29m) 12m ago 29m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:10:04.932 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (28m) 11m ago 28m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:10:04.932 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (16m) 12m ago 29m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:10:04.932 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (16m) 11m ago 28m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (18m) 12m ago 29m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (12m) 12m ago 23m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (12m) 12m ago 23m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (12m) 11m ago 23m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (11m) 11m ago 23m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (20m) 12m ago 30m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (19m) 11m ago 28m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (17m) 12m ago 31m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (16m) 11m ago 28m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (19m) 12m ago 29m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (19m) 11m ago 28m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (15m) 12m ago 27m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (15m) 12m ago 27m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:10:04.933 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (14m) 12m ago 26m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:10:04.934 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (14m) 11m ago 26m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:10:04.934 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (13m) 11m ago 25m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:10:04.934 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (13m) 11m ago 25m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:10:04.934 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (19m) 12m ago 29m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:10:04.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7a1406e680 msgr2=0x7f7a14070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7a1406e680 0x7f7a14070b40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005fd0 tx=0x7f7a1800a560 comp rx=0 tx=0).stop 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 msgr2=0x7f7a2419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a2419cbb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7a100092e0 tx=0x7f7a1000be20 comp rx=0 tx=0).stop 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 shutdown_connections 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7a1406e680 0x7f7a14070b40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a240ffda0 0x7f7a2419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 --2- 172.21.15.42:0/4075900675 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7a24100750 0x7f7a2419cbb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 >> 172.21.15.42:0/4075900675 conn(0x7f7a240fb650 msgr2=0x7f7a24103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 shutdown_connections 2024-09-17T00:10:04.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:04.936+0000 7f7a2b14a700 1 -- 172.21.15.42:0/4075900675 wait complete. 2024-09-17T00:10:05.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 -- 172.21.15.42:0/1271834413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b0106db0 msgr2=0x7f03b0107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 --2- 172.21.15.42:0/1271834413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b0106db0 0x7f03b0107190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009910 tx=0x7f03a0009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 -- 172.21.15.42:0/1271834413 shutdown_connections 2024-09-17T00:10:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 --2- 172.21.15.42:0/1271834413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b00ff410 0x7f03b00ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 --2- 172.21.15.42:0/1271834413 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b0106db0 0x7f03b0107190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 -- 172.21.15.42:0/1271834413 >> 172.21.15.42:0/1271834413 conn(0x7f03b0074e30 msgr2=0x7f03b0075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 -- 172.21.15.42:0/1271834413 shutdown_connections 2024-09-17T00:10:05.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 -- 172.21.15.42:0/1271834413 wait complete. 2024-09-17T00:10:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.043+0000 7f03b5005700 1 Processor -- start 2024-09-17T00:10:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03b5005700 1 -- start start 2024-09-17T00:10:05.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03b5005700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b00ff410 0x7f03b019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03b5005700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b0106db0 0x7f03b019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03b5005700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b019d2e0 con 0x7f03b00ff410 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03b5005700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0196820 con 0x7f03b0106db0 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03aed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b00ff410 0x7f03b019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03aed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b00ff410 0x7f03b019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51326/0 (socket says 172.21.15.42:51326) 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03aed9d700 1 -- 172.21.15.42:0/3940893297 learned_addr learned my addr 172.21.15.42:0/3940893297 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03a659c700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b0106db0 0x7f03b019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03aed9d700 1 -- 172.21.15.42:0/3940893297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b0106db0 msgr2=0x7f03b019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03aed9d700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b0106db0 0x7f03b019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.044+0000 7f03aed9d700 1 -- 172.21.15.42:0/3940893297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a00095f0 con 0x7f03b00ff410 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.045+0000 7f03aed9d700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b00ff410 0x7f03b019c7a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f03a0006010 tx=0x7f03a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.045+0000 7f03acd99700 1 -- 172.21.15.42:0/3940893297 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a001d070 con 0x7f03b00ff410 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.045+0000 7f03acd99700 1 -- 172.21.15.42:0/3940893297 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03a0004fb0 con 0x7f03b00ff410 2024-09-17T00:10:05.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.045+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b0196aa0 con 0x7f03b00ff410 2024-09-17T00:10:05.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.045+0000 7f03acd99700 1 -- 172.21.15.42:0/3940893297 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03a0022ea0 con 0x7f03b00ff410 2024-09-17T00:10:05.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.045+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b0196f90 con 0x7f03b00ff410 2024-09-17T00:10:05.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.046+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b0109300 con 0x7f03b00ff410 2024-09-17T00:10:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.047+0000 7f03acd99700 1 -- 172.21.15.42:0/3940893297 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f03a0022470 con 0x7f03b00ff410 2024-09-17T00:10:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.047+0000 7f03acd99700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f039c06e680 0x7f039c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.047+0000 7f03acd99700 1 -- 172.21.15.42:0/3940893297 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f03a008fbf0 con 0x7f03b00ff410 2024-09-17T00:10:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.047+0000 7f03a659c700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f039c06e680 0x7f039c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.048+0000 7f03a659c700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f039c06e680 0x7f039c070b40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0398005ea0 tx=0x7f0398005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:05.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.052+0000 7f03acd99700 1 -- 172.21.15.42:0/3940893297 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f03a005c970 con 0x7f03b00ff410 2024-09-17T00:10:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:05 smithi042 ceph-mon[111056]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:05 smithi042 ceph-mon[111056]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:05.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.349+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f03b0197d30 con 0x7f03b00ff410 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.349+0000 7f03acd99700 1 -- 172.21.15.42:0/3940893297 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f03a0027290 con 0x7f03b00ff410 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:10:05.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:05.349 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:10:05.349 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:10:05.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:05.349 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:10:05.349 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:10:05.349 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:10:05.349 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:10:05.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f039c06e680 msgr2=0x7f039c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f039c06e680 0x7f039c070b40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0398005ea0 tx=0x7f0398005e30 comp rx=0 tx=0).stop 2024-09-17T00:10:05.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b00ff410 msgr2=0x7f03b019c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b00ff410 0x7f03b019c7a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f03a0006010 tx=0x7f03a000be20 comp rx=0 tx=0).stop 2024-09-17T00:10:05.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 shutdown_connections 2024-09-17T00:10:05.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f039c06e680 0x7f039c070b40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03b00ff410 0x7f03b019c7a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 --2- 172.21.15.42:0/3940893297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b0106db0 0x7f03b019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.351+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 >> 172.21.15.42:0/3940893297 conn(0x7f03b0074e30 msgr2=0x7f03b00fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:05.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.352+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 shutdown_connections 2024-09-17T00:10:05.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.352+0000 7f03b5005700 1 -- 172.21.15.42:0/3940893297 wait complete. 2024-09-17T00:10:05.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:05 smithi043 ceph-mon[81510]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:05.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:05 smithi043 ceph-mon[81510]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.459+0000 7f93c3c32700 1 -- 172.21.15.42:0/1214684495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 msgr2=0x7f93bc100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.459+0000 7f93c3c32700 1 --2- 172.21.15.42:0/1214684495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc100180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009910 tx=0x7f93b8009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:05.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 -- 172.21.15.42:0/1214684495 shutdown_connections 2024-09-17T00:10:05.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 --2- 172.21.15.42:0/1214684495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93bc100750 0x7f93bc1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 --2- 172.21.15.42:0/1214684495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc100180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 -- 172.21.15.42:0/1214684495 >> 172.21.15.42:0/1214684495 conn(0x7f93bc0fb650 msgr2=0x7f93bc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 -- 172.21.15.42:0/1214684495 shutdown_connections 2024-09-17T00:10:05.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 -- 172.21.15.42:0/1214684495 wait complete. 2024-09-17T00:10:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 Processor -- start 2024-09-17T00:10:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.460+0000 7f93c3c32700 1 -- start start 2024-09-17T00:10:05.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c3c32700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c3c32700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93bc100750 0x7f93bc19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c3c32700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc19d230 con 0x7f93bc0ffda0 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c3c32700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1966b0 con 0x7f93bc100750 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c19ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c19ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc19c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51342/0 (socket says 172.21.15.42:51342) 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c19ce700 1 -- 172.21.15.42:0/3599513951 learned_addr learned my addr 172.21.15.42:0/3599513951 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c11cd700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93bc100750 0x7f93bc19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c19ce700 1 -- 172.21.15.42:0/3599513951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93bc100750 msgr2=0x7f93bc19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c19ce700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93bc100750 0x7f93bc19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c19ce700 1 -- 172.21.15.42:0/3599513951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80095f0 con 0x7f93bc0ffda0 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c11cd700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93bc100750 0x7f93bc19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c19ce700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc19c610 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f93b8006010 tx=0x7f93b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:05.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93b2ffd700 1 -- 172.21.15.42:0/3599513951 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93b801d070 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93b2ffd700 1 -- 172.21.15.42:0/3599513951 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93b8004fb0 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc196930 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.461+0000 7f93b2ffd700 1 -- 172.21.15.42:0/3599513951 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93b8022ea0 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.462+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc196e20 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.462+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc04f2e0 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.463+0000 7f93b2ffd700 1 -- 172.21.15.42:0/3599513951 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f93b8022470 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.463+0000 7f93b2ffd700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93a806e680 0x7f93a8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.464+0000 7f93b2ffd700 1 -- 172.21.15.42:0/3599513951 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f93b808fb40 con 0x7f93bc0ffda0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.464+0000 7f93c11cd700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93a806e680 0x7f93a8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.464+0000 7f93c11cd700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93a806e680 0x7f93a8070b40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f93bc197ea0 tx=0x7f93ac005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:05.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.468+0000 7f93b2ffd700 1 -- 172.21.15.42:0/3599513951 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f93b805ca60 con 0x7f93bc0ffda0 2024-09-17T00:10:05.733 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.735+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f93bc1971f0 con 0x7f93bc0ffda0 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.735+0000 7f93b2ffd700 1 -- 172.21.15.42:0/3599513951 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f93b8027070 con 0x7f93bc0ffda0 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:10:05.734 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:10:05.735 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:10:05.736 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:05.737 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.737+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93a806e680 msgr2=0x7f93a8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93a806e680 0x7f93a8070b40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f93bc197ea0 tx=0x7f93ac005f50 comp rx=0 tx=0).stop 2024-09-17T00:10:05.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 msgr2=0x7f93bc19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc19c610 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f93b8006010 tx=0x7f93b800be20 comp rx=0 tx=0).stop 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 shutdown_connections 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93a806e680 0x7f93a8070b40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93bc0ffda0 0x7f93bc19c610 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 --2- 172.21.15.42:0/3599513951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93bc100750 0x7f93bc19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 >> 172.21.15.42:0/3599513951 conn(0x7f93bc0fb650 msgr2=0x7f93bc0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 shutdown_connections 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.738+0000 7f93c3c32700 1 -- 172.21.15.42:0/3599513951 wait complete. 2024-09-17T00:10:05.739 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:10:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 -- 172.21.15.42:0/669506612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6340ffc20 msgr2=0x7fa634100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 --2- 172.21.15.42:0/669506612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6340ffc20 0x7fa634100080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa628009910 tx=0x7fa628009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:05.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 -- 172.21.15.42:0/669506612 shutdown_connections 2024-09-17T00:10:05.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 --2- 172.21.15.42:0/669506612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa6340ffc20 0x7fa634100080 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 --2- 172.21.15.42:0/669506612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa634105bd0 0x7fa634105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 -- 172.21.15.42:0/669506612 >> 172.21.15.42:0/669506612 conn(0x7fa634075670 msgr2=0x7fa634075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:05.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 -- 172.21.15.42:0/669506612 shutdown_connections 2024-09-17T00:10:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.847+0000 7fa63b241700 1 -- 172.21.15.42:0/669506612 wait complete. 2024-09-17T00:10:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa63b241700 1 Processor -- start 2024-09-17T00:10:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa63b241700 1 -- start start 2024-09-17T00:10:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa63b241700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6340ffc20 0x7fa634101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa63b241700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa634105bd0 0x7fa634101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa63b241700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6341022c0 con 0x7fa634105bd0 2024-09-17T00:10:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa63b241700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa634102400 con 0x7fa6340ffc20 2024-09-17T00:10:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa633fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa634105bd0 0x7fa634101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa633fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa634105bd0 0x7fa634101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51350/0 (socket says 172.21.15.42:51350) 2024-09-17T00:10:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.848+0000 7fa633fff700 1 -- 172.21.15.42:0/2389961411 learned_addr learned my addr 172.21.15.42:0/2389961411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa638fdd700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6340ffc20 0x7fa634101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa633fff700 1 -- 172.21.15.42:0/2389961411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6340ffc20 msgr2=0x7fa634101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa633fff700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6340ffc20 0x7fa634101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa633fff700 1 -- 172.21.15.42:0/2389961411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6280095f0 con 0x7fa634105bd0 2024-09-17T00:10:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa638fdd700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6340ffc20 0x7fa634101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:10:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa633fff700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa634105bd0 0x7fa634101d80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa628006010 tx=0x7fa62800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa631ffb700 1 -- 172.21.15.42:0/2389961411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa62801d070 con 0x7fa634105bd0 2024-09-17T00:10:05.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa634102680 con 0x7fa634105bd0 2024-09-17T00:10:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa631ffb700 1 -- 172.21.15.42:0/2389961411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa628004fb0 con 0x7fa634105bd0 2024-09-17T00:10:05.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa631ffb700 1 -- 172.21.15.42:0/2389961411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa628022ea0 con 0x7fa634105bd0 2024-09-17T00:10:05.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.849+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6341a2eb0 con 0x7fa634105bd0 2024-09-17T00:10:05.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.850+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63404f2e0 con 0x7fa634105bd0 2024-09-17T00:10:05.858 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.851+0000 7fa631ffb700 1 -- 172.21.15.42:0/2389961411 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa628022470 con 0x7fa634105bd0 2024-09-17T00:10:05.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.851+0000 7fa631ffb700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa61c06e680 0x7fa61c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:05.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.851+0000 7fa631ffb700 1 -- 172.21.15.42:0/2389961411 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa62808fc70 con 0x7fa634105bd0 2024-09-17T00:10:05.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.852+0000 7fa638fdd700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa61c06e680 0x7fa61c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:05.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.852+0000 7fa638fdd700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa61c06e680 0x7fa61c070b40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa624007900 tx=0x7fa624008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:05.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:05.856+0000 7fa631ffb700 1 -- 172.21.15.42:0/2389961411 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa628096080 con 0x7fa634105bd0 2024-09-17T00:10:06.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.092+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa634104990 con 0x7fa61c06e680 2024-09-17T00:10:06.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.094+0000 7fa631ffb700 1 -- 172.21.15.42:0/2389961411 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa634104990 con 0x7fa61c06e680 2024-09-17T00:10:06.092 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:10:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:10:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:10:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:10:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:10:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:10:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:10:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:10:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:10:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:10:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:10:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:10:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:10:06.094 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:10:06.094 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:10:06.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa61c06e680 msgr2=0x7fa61c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:06.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa61c06e680 0x7fa61c070b40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa624007900 tx=0x7fa624008040 comp rx=0 tx=0).stop 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa634105bd0 msgr2=0x7fa634101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa634105bd0 0x7fa634101d80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa628006010 tx=0x7fa62800be20 comp rx=0 tx=0).stop 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 shutdown_connections 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa61c06e680 0x7fa61c070b40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6340ffc20 0x7fa634101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 --2- 172.21.15.42:0/2389961411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa634105bd0 0x7fa634101d80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 >> 172.21.15.42:0/2389961411 conn(0x7fa634075670 msgr2=0x7fa6340686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:06.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 shutdown_connections 2024-09-17T00:10:06.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.096+0000 7fa63b241700 1 -- 172.21.15.42:0/2389961411 wait complete. 2024-09-17T00:10:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 -- 172.21.15.42:0/1836238761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 msgr2=0x7f59e0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 --2- 172.21.15.42:0/1836238761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0105fb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f59dc009910 tx=0x7f59dc009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 -- 172.21.15.42:0/1836238761 shutdown_connections 2024-09-17T00:10:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 --2- 172.21.15.42:0/1836238761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e00ffc20 0x7f59e0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 --2- 172.21.15.42:0/1836238761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0105fb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 -- 172.21.15.42:0/1836238761 >> 172.21.15.42:0/1836238761 conn(0x7f59e0075670 msgr2=0x7f59e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:06.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 -- 172.21.15.42:0/1836238761 shutdown_connections 2024-09-17T00:10:06.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.204+0000 7f59e84c4700 1 -- 172.21.15.42:0/1836238761 wait complete. 2024-09-17T00:10:06.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e84c4700 1 Processor -- start 2024-09-17T00:10:06.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e84c4700 1 -- start start 2024-09-17T00:10:06.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e84c4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e00ffc20 0x7f59e0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:06.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e84c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:06.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e84c4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e01022c0 con 0x7f59e0105bd0 2024-09-17T00:10:06.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e84c4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e0102400 con 0x7f59e00ffc20 2024-09-17T00:10:06.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e5a5f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e5a5f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51374/0 (socket says 172.21.15.42:51374) 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.205+0000 7f59e5a5f700 1 -- 172.21.15.42:0/640666929 learned_addr learned my addr 172.21.15.42:0/640666929 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e6260700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e00ffc20 0x7f59e0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e5a5f700 1 -- 172.21.15.42:0/640666929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e00ffc20 msgr2=0x7f59e0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e5a5f700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e00ffc20 0x7f59e0101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e5a5f700 1 -- 172.21.15.42:0/640666929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc0095f0 con 0x7f59e0105bd0 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e6260700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e00ffc20 0x7f59e0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e5a5f700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0101d80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f59d000e920 tx=0x7f59d000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59d77fe700 1 -- 172.21.15.42:0/640666929 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59d0009790 con 0x7f59e0105bd0 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e01a2a20 con 0x7f59e0105bd0 2024-09-17T00:10:06.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59d77fe700 1 -- 172.21.15.42:0/640666929 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59d000cba0 con 0x7f59e0105bd0 2024-09-17T00:10:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59d77fe700 1 -- 172.21.15.42:0/640666929 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59d0018650 con 0x7f59e0105bd0 2024-09-17T00:10:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.206+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e01a2e30 con 0x7f59e0105bd0 2024-09-17T00:10:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.207+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e004f2e0 con 0x7f59e0105bd0 2024-09-17T00:10:06.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.208+0000 7f59d77fe700 1 -- 172.21.15.42:0/640666929 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f59d000cd10 con 0x7f59e0105bd0 2024-09-17T00:10:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.208+0000 7f59d77fe700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f59cc06e6d0 0x7f59cc070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.208+0000 7f59d77fe700 1 -- 172.21.15.42:0/640666929 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f59d0014070 con 0x7f59e0105bd0 2024-09-17T00:10:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.208+0000 7f59e6260700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f59cc06e6d0 0x7f59cc070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:06.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.209+0000 7f59e6260700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f59cc06e6d0 0x7f59cc070b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b760 tx=0x7f59dc005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:06.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.213+0000 7f59d77fe700 1 -- 172.21.15.42:0/640666929 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f59d00541b0 con 0x7f59e0105bd0 2024-09-17T00:10:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:06 smithi043 ceph-mon[81510]: pgmap v555: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:06 smithi043 ceph-mon[81510]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:06 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3940893297' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:10:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:06.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:06 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3599513951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:10:06.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.510+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f59e0102d70 con 0x7f59e0105bd0 2024-09-17T00:10:06.509 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:06 smithi042 ceph-mon[111056]: pgmap v555: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:06.509 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:06 smithi042 ceph-mon[111056]: from='client.35056 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:06.509 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:06 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3940893297' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:10:06.509 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:06.509 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:06 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3599513951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:10:06.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.510+0000 7f59d77fe700 1 -- 172.21.15.42:0/640666929 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f59d005b730 con 0x7f59e0105bd0 2024-09-17T00:10:06.509 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:10:06.510 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:10:06.510 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:10:06.510 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:10:06.510 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:10:06.510 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:10:06.510 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f59cc06e6d0 msgr2=0x7f59cc070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f59cc06e6d0 0x7f59cc070b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00b760 tx=0x7f59dc005940 comp rx=0 tx=0).stop 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 msgr2=0x7f59e0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0101d80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f59d000e920 tx=0x7f59d000ece0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 shutdown_connections 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f59cc06e6d0 0x7f59cc070b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59e00ffc20 0x7f59e0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 --2- 172.21.15.42:0/640666929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59e0105bd0 0x7f59e0101d80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 >> 172.21.15.42:0/640666929 conn(0x7f59e0075670 msgr2=0x7f59e00686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 shutdown_connections 2024-09-17T00:10:06.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:06.512+0000 7f59e84c4700 1 -- 172.21.15.42:0/640666929 wait complete. 2024-09-17T00:10:07.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:07 smithi043 ceph-mon[81510]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:07.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:07 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/640666929' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:10:07.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:07 smithi042 ceph-mon[111056]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:07.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:07 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/640666929' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:10:08.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:08 smithi043 ceph-mon[81510]: pgmap v556: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:08.513 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:08 smithi042 ceph-mon[111056]: pgmap v556: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:10.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:10 smithi043 ceph-mon[81510]: pgmap v557: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:10.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:10 smithi042 ceph-mon[111056]: pgmap v557: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:12.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:12 smithi043 ceph-mon[81510]: pgmap v558: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:12.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:12 smithi042 ceph-mon[111056]: pgmap v558: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:14.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:14 smithi043 ceph-mon[81510]: pgmap v559: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:14.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:14 smithi042 ceph-mon[111056]: pgmap v559: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:16.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:16 smithi043 ceph-mon[81510]: pgmap v560: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:16.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:16 smithi042 ceph-mon[111056]: pgmap v560: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:18.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:18 smithi043 ceph-mon[81510]: pgmap v561: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:18.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:18 smithi042 ceph-mon[111056]: pgmap v561: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:20.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:20 smithi043 ceph-mon[81510]: pgmap v562: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:20.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:20 smithi042 ceph-mon[111056]: pgmap v562: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:21.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:22.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:22 smithi043 ceph-mon[81510]: pgmap v563: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:22.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:22 smithi042 ceph-mon[111056]: pgmap v563: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:24.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:24 smithi042 ceph-mon[111056]: pgmap v564: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:24.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:24 smithi043 ceph-mon[81510]: pgmap v564: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:26.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:26 smithi043 ceph-mon[81510]: pgmap v565: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:26.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:26 smithi042 ceph-mon[111056]: pgmap v565: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:28.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:28 smithi043 ceph-mon[81510]: pgmap v566: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:28.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:28 smithi042 ceph-mon[111056]: pgmap v566: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:30.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:30 smithi043 ceph-mon[81510]: pgmap v567: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:30.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:30 smithi042 ceph-mon[111056]: pgmap v567: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:32.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:32 smithi043 ceph-mon[81510]: pgmap v568: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:32.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:32 smithi042 ceph-mon[111056]: pgmap v568: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:34.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:34 smithi043 ceph-mon[81510]: pgmap v569: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:34.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:34 smithi042 ceph-mon[111056]: pgmap v569: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:36.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:36 smithi042 ceph-mon[111056]: pgmap v570: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:36.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:36 smithi043 ceph-mon[81510]: pgmap v570: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:36.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 -- 172.21.15.42:0/2595786291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 msgr2=0x7fcfe0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:36.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 --2- 172.21.15.42:0/2595786291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe0101280 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00b0a0 tx=0x7fcfdc00b3b0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 -- 172.21.15.42:0/2595786291 shutdown_connections 2024-09-17T00:10:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 --2- 172.21.15.42:0/2595786291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe0101850 0x7fcfe01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 --2- 172.21.15.42:0/2595786291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe0101280 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 -- 172.21.15.42:0/2595786291 >> 172.21.15.42:0/2595786291 conn(0x7fcfe00fc750 msgr2=0x7fcfe00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 -- 172.21.15.42:0/2595786291 shutdown_connections 2024-09-17T00:10:36.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.636+0000 7fcfe75db700 1 -- 172.21.15.42:0/2595786291 wait complete. 2024-09-17T00:10:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe75db700 1 Processor -- start 2024-09-17T00:10:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe75db700 1 -- start start 2024-09-17T00:10:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe75db700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:36.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe75db700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe0101850 0x7fcfe019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe75db700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe019f4f0 con 0x7fcfe0100ea0 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe75db700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01989a0 con 0x7fcfe0101850 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe5377700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe5377700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47010/0 (socket says 172.21.15.42:47010) 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe5377700 1 -- 172.21.15.42:0/7057091 learned_addr learned my addr 172.21.15.42:0/7057091 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe4b76700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe0101850 0x7fcfe019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe5377700 1 -- 172.21.15.42:0/7057091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe0101850 msgr2=0x7fcfe019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe5377700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe0101850 0x7fcfe019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe5377700 1 -- 172.21.15.42:0/7057091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc009d00 con 0x7fcfe0100ea0 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.637+0000 7fcfe4b76700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe0101850 0x7fcfe019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.638+0000 7fcfe5377700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe019e920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00b070 tx=0x7fcfdc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.638+0000 7fcfd67fc700 1 -- 172.21.15.42:0/7057091 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfdc00e040 con 0x7fcfe0100ea0 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.638+0000 7fcfd67fc700 1 -- 172.21.15.42:0/7057091 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfdc003de0 con 0x7fcfe0100ea0 2024-09-17T00:10:36.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.638+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe0198c20 con 0x7fcfe0100ea0 2024-09-17T00:10:36.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.638+0000 7fcfd67fc700 1 -- 172.21.15.42:0/7057091 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfdc01d8e0 con 0x7fcfe0100ea0 2024-09-17T00:10:36.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.638+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe0199110 con 0x7fcfe0100ea0 2024-09-17T00:10:36.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.639+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe0105020 con 0x7fcfe0100ea0 2024-09-17T00:10:36.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.640+0000 7fcfd67fc700 1 -- 172.21.15.42:0/7057091 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fcfdc004040 con 0x7fcfe0100ea0 2024-09-17T00:10:36.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.640+0000 7fcfd67fc700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcfcc06e680 0x7fcfcc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:36.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.640+0000 7fcfe4b76700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcfcc06e680 0x7fcfcc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:36.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.640+0000 7fcfd67fc700 1 -- 172.21.15.42:0/7057091 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcfdc08feb0 con 0x7fcfe0100ea0 2024-09-17T00:10:36.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.641+0000 7fcfe4b76700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcfcc06e680 0x7fcfcc070b40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0005950 tx=0x7fcfd00058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:36.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.645+0000 7fcfd67fc700 1 -- 172.21.15.42:0/7057091 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcfdc05cb80 con 0x7fcfe0100ea0 2024-09-17T00:10:36.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.873+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfe0199f70 con 0x7fcfcc06e680 2024-09-17T00:10:36.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.874+0000 7fcfd67fc700 1 -- 172.21.15.42:0/7057091 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcfe0199f70 con 0x7fcfcc06e680 2024-09-17T00:10:36.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcfcc06e680 msgr2=0x7fcfcc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcfcc06e680 0x7fcfcc070b40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0005950 tx=0x7fcfd00058e0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 msgr2=0x7fcfe019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe019e920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00b070 tx=0x7fcfdc00cdd0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 shutdown_connections 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fcfcc06e680 0x7fcfcc070b40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe0100ea0 0x7fcfe019e920 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 --2- 172.21.15.42:0/7057091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfe0101850 0x7fcfe019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 >> 172.21.15.42:0/7057091 conn(0x7fcfe00fc750 msgr2=0x7fcfe00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 shutdown_connections 2024-09-17T00:10:36.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.877+0000 7fcfe75db700 1 -- 172.21.15.42:0/7057091 wait complete. 2024-09-17T00:10:36.885 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:10:36.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 -- 172.21.15.42:0/4090802130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 msgr2=0x7f5c781047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 --2- 172.21.15.42:0/4090802130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c781047a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009910 tx=0x7f5c6c009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 -- 172.21.15.42:0/4090802130 shutdown_connections 2024-09-17T00:10:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 --2- 172.21.15.42:0/4090802130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c781047a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 --2- 172.21.15.42:0/4090802130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c780ffda0 0x7f5c78100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 -- 172.21.15.42:0/4090802130 >> 172.21.15.42:0/4090802130 conn(0x7f5c780fb650 msgr2=0x7f5c780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 -- 172.21.15.42:0/4090802130 shutdown_connections 2024-09-17T00:10:36.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.987+0000 7f5c7e744700 1 -- 172.21.15.42:0/4090802130 wait complete. 2024-09-17T00:10:36.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c7e744700 1 Processor -- start 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c7e744700 1 -- start start 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c7e744700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c780ffda0 0x7f5c78101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c7e744700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c78101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c7e744700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781022c0 con 0x7f5c78100750 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c7e744700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78102400 con 0x7f5c780ffda0 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c78101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c777fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c78101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47028/0 (socket says 172.21.15.42:47028) 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c777fe700 1 -- 172.21.15.42:0/923279735 learned_addr learned my addr 172.21.15.42:0/923279735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.988+0000 7f5c77fff700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c780ffda0 0x7f5c78101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c777fe700 1 -- 172.21.15.42:0/923279735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c780ffda0 msgr2=0x7f5c78101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c777fe700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c780ffda0 0x7f5c78101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:36.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c777fe700 1 -- 172.21.15.42:0/923279735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0095f0 con 0x7f5c78100750 2024-09-17T00:10:36.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c777fe700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c78101d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0059d0 tx=0x7f5c6c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:36.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c757fa700 1 -- 172.21.15.42:0/923279735 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6c01d070 con 0x7f5c78100750 2024-09-17T00:10:36.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c757fa700 1 -- 172.21.15.42:0/923279735 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c6c00b610 con 0x7f5c78100750 2024-09-17T00:10:36.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c78102680 con 0x7f5c78100750 2024-09-17T00:10:36.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c757fa700 1 -- 172.21.15.42:0/923279735 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6c022470 con 0x7f5c78100750 2024-09-17T00:10:36.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.989+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c781a2da0 con 0x7f5c78100750 2024-09-17T00:10:36.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.990+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c781091b0 con 0x7f5c78100750 2024-09-17T00:10:36.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.991+0000 7f5c757fa700 1 -- 172.21.15.42:0/923279735 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5c6c003710 con 0x7f5c78100750 2024-09-17T00:10:36.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.991+0000 7f5c757fa700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5c6006e680 0x7f5c60070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:36.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.991+0000 7f5c757fa700 1 -- 172.21.15.42:0/923279735 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5c6c08fa10 con 0x7f5c78100750 2024-09-17T00:10:36.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.991+0000 7f5c77fff700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5c6006e680 0x7f5c60070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:36.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.992+0000 7f5c77fff700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5c6006e680 0x7f5c60070b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009c60 tx=0x7f5c68009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:36.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:36.996+0000 7f5c757fa700 1 -- 172.21.15.42:0/923279735 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5c6c058c30 con 0x7f5c78100750 2024-09-17T00:10:37.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.229+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c780611d0 con 0x7f5c6006e680 2024-09-17T00:10:37.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.230+0000 7f5c757fa700 1 -- 172.21.15.42:0/923279735 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5c780611d0 con 0x7f5c6006e680 2024-09-17T00:10:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5c6006e680 msgr2=0x7f5c60070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5c6006e680 0x7f5c60070b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009c60 tx=0x7f5c68009260 comp rx=0 tx=0).stop 2024-09-17T00:10:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 msgr2=0x7f5c78101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c78101d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0059d0 tx=0x7f5c6c00f690 comp rx=0 tx=0).stop 2024-09-17T00:10:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 shutdown_connections 2024-09-17T00:10:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5c6006e680 0x7f5c60070b40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c780ffda0 0x7f5c78101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.232+0000 7f5c7e744700 1 --2- 172.21.15.42:0/923279735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c78100750 0x7f5c78101d80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.233+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 >> 172.21.15.42:0/923279735 conn(0x7f5c780fb650 msgr2=0x7f5c78069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.233+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 shutdown_connections 2024-09-17T00:10:37.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.233+0000 7f5c7e744700 1 -- 172.21.15.42:0/923279735 wait complete. 2024-09-17T00:10:37.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.340+0000 7fa157dd3700 1 -- 172.21.15.42:0/648311244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150100ea0 msgr2=0x7fa150101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.340+0000 7fa157dd3700 1 --2- 172.21.15.42:0/648311244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150100ea0 0x7fa150101280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa14000b0a0 tx=0x7fa14000b3b0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.340+0000 7fa157dd3700 1 -- 172.21.15.42:0/648311244 shutdown_connections 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.340+0000 7fa157dd3700 1 --2- 172.21.15.42:0/648311244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa150101850 0x7fa1501058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.340+0000 7fa157dd3700 1 --2- 172.21.15.42:0/648311244 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150100ea0 0x7fa150101280 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.340+0000 7fa157dd3700 1 -- 172.21.15.42:0/648311244 >> 172.21.15.42:0/648311244 conn(0x7fa1500fc750 msgr2=0x7fa1500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 -- 172.21.15.42:0/648311244 shutdown_connections 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 -- 172.21.15.42:0/648311244 wait complete. 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 Processor -- start 2024-09-17T00:10:37.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 -- start start 2024-09-17T00:10:37.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa150100ea0 0x7fa150073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:37.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150101850 0x7fa1500735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150073b10 con 0x7fa150101850 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.341+0000 7fa157dd3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa150073c50 con 0x7fa150100ea0 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa15536e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150101850 0x7fa1500735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa15536e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150101850 0x7fa1500735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47034/0 (socket says 172.21.15.42:47034) 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa15536e700 1 -- 172.21.15.42:0/1871856002 learned_addr learned my addr 172.21.15.42:0/1871856002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa155b6f700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa150100ea0 0x7fa150073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa15536e700 1 -- 172.21.15.42:0/1871856002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa150100ea0 msgr2=0x7fa150073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa15536e700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa150100ea0 0x7fa150073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa15536e700 1 -- 172.21.15.42:0/1871856002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140009d00 con 0x7fa150101850 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa15536e700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150101850 0x7fa1500735d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d900 tx=0x7fa14400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa14effd700 1 -- 172.21.15.42:0/1871856002 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa144009900 con 0x7fa150101850 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa14effd700 1 -- 172.21.15.42:0/1871856002 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa144010460 con 0x7fa150101850 2024-09-17T00:10:37.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa14effd700 1 -- 172.21.15.42:0/1871856002 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa14400f5f0 con 0x7fa150101850 2024-09-17T00:10:37.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1501a2b30 con 0x7fa150101850 2024-09-17T00:10:37.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.342+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1501a2f40 con 0x7fa150101850 2024-09-17T00:10:37.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.343+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa150109240 con 0x7fa150101850 2024-09-17T00:10:37.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.344+0000 7fa14effd700 1 -- 172.21.15.42:0/1871856002 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa1440105d0 con 0x7fa150101850 2024-09-17T00:10:37.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.345+0000 7fa14effd700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa13806e680 0x7fa138070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:37.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.345+0000 7fa155b6f700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa13806e680 0x7fa138070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:37.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.345+0000 7fa14effd700 1 -- 172.21.15.42:0/1871856002 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa14408e240 con 0x7fa150101850 2024-09-17T00:10:37.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.345+0000 7fa155b6f700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa13806e680 0x7fa138070b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa140000f80 tx=0x7fa1400061f0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:37.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.349+0000 7fa14effd700 1 -- 172.21.15.42:0/1871856002 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa1440573e0 con 0x7fa150101850 2024-09-17T00:10:37.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.567+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1500611d0 con 0x7fa13806e680 2024-09-17T00:10:37.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.577+0000 7fa14effd700 1 -- 172.21.15.42:0/1871856002 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fa1500611d0 con 0x7fa13806e680 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (19m) 12m ago 29m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (30m) 12m ago 30m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (29m) 12m ago 29m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (16m) 12m ago 30m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (16m) 12m ago 29m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (18m) 12m ago 29m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (13m) 12m ago 24m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (13m) 12m ago 24m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (12m) 12m ago 24m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (12m) 12m ago 24m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (21m) 12m ago 31m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:10:37.575 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (20m) 12m ago 29m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (17m) 12m ago 31m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (17m) 12m ago 28m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (19m) 12m ago 30m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (19m) 12m ago 29m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (16m) 12m ago 28m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (15m) 12m ago 27m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (15m) 12m ago 27m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (14m) 12m ago 26m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (14m) 12m ago 26m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (13m) 12m ago 25m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:10:37.576 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (19m) 12m ago 29m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:10:37.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa13806e680 msgr2=0x7fa138070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa13806e680 0x7fa138070b40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa140000f80 tx=0x7fa1400061f0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150101850 msgr2=0x7fa1500735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150101850 0x7fa1500735d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa14400d900 tx=0x7fa14400dc10 comp rx=0 tx=0).stop 2024-09-17T00:10:37.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 shutdown_connections 2024-09-17T00:10:37.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa13806e680 0x7fa138070b40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa150100ea0 0x7fa150073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 --2- 172.21.15.42:0/1871856002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa150101850 0x7fa1500735d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 >> 172.21.15.42:0/1871856002 conn(0x7fa1500fc750 msgr2=0x7fa1500fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:37.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 shutdown_connections 2024-09-17T00:10:37.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.579+0000 7fa157dd3700 1 -- 172.21.15.42:0/1871856002 wait complete. 2024-09-17T00:10:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.685+0000 7f3b4a152700 1 -- 172.21.15.42:0/3165525623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 msgr2=0x7f3b44100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.685+0000 7f3b4a152700 1 --2- 172.21.15.42:0/3165525623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b44100180 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009910 tx=0x7f3b34009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 -- 172.21.15.42:0/3165525623 shutdown_connections 2024-09-17T00:10:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 --2- 172.21.15.42:0/3165525623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b44100750 0x7f3b441047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 --2- 172.21.15.42:0/3165525623 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b44100180 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 -- 172.21.15.42:0/3165525623 >> 172.21.15.42:0/3165525623 conn(0x7f3b440fb650 msgr2=0x7f3b440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 -- 172.21.15.42:0/3165525623 shutdown_connections 2024-09-17T00:10:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 -- 172.21.15.42:0/3165525623 wait complete. 2024-09-17T00:10:37.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 Processor -- start 2024-09-17T00:10:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.686+0000 7f3b4a152700 1 -- start start 2024-09-17T00:10:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b4a152700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b4419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b4a152700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b44100750 0x7f3b4419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:37.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b4a152700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4419d210 con 0x7f3b440ffda0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b4a152700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b441966c0 con 0x7f3b44100750 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b4419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b4419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47056/0 (socket says 172.21.15.42:47056) 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b437fe700 1 -- 172.21.15.42:0/1562283754 learned_addr learned my addr 172.21.15.42:0/1562283754 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b42ffd700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b44100750 0x7f3b4419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b437fe700 1 -- 172.21.15.42:0/1562283754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b44100750 msgr2=0x7f3b4419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b437fe700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b44100750 0x7f3b4419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b437fe700 1 -- 172.21.15.42:0/1562283754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340095f0 con 0x7f3b440ffda0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b437fe700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b4419c640 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006010 tx=0x7f3b3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b40ff9700 1 -- 172.21.15.42:0/1562283754 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3401d070 con 0x7f3b440ffda0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b40ff9700 1 -- 172.21.15.42:0/1562283754 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b3400f460 con 0x7f3b440ffda0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.687+0000 7f3b40ff9700 1 -- 172.21.15.42:0/1562283754 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b34017630 con 0x7f3b440ffda0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.688+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b44196940 con 0x7f3b440ffda0 2024-09-17T00:10:37.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.688+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b44196e30 con 0x7f3b440ffda0 2024-09-17T00:10:37.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.689+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4404f2e0 con 0x7f3b440ffda0 2024-09-17T00:10:37.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.689+0000 7f3b40ff9700 1 -- 172.21.15.42:0/1562283754 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3b3400f5d0 con 0x7f3b440ffda0 2024-09-17T00:10:37.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.690+0000 7f3b40ff9700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b2c06e680 0x7f3b2c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:37.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.690+0000 7f3b42ffd700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b2c06e680 0x7f3b2c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:37.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.690+0000 7f3b40ff9700 1 -- 172.21.15.42:0/1562283754 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3b3408fa80 con 0x7f3b440ffda0 2024-09-17T00:10:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.690+0000 7f3b42ffd700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b2c06e680 0x7f3b2c070b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3b38005950 tx=0x7f3b380058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.694+0000 7f3b40ff9700 1 -- 172.21.15.42:0/1562283754 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3b3405c7d0 con 0x7f3b440ffda0 2024-09-17T00:10:37.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.990+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3b44197110 con 0x7f3b440ffda0 2024-09-17T00:10:37.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.991+0000 7f3b40ff9700 1 -- 172.21.15.42:0/1562283754 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3b3405c360 con 0x7f3b440ffda0 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:10:37.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:10:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:10:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:10:37.990 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:10:37.990 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b2c06e680 msgr2=0x7f3b2c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b2c06e680 0x7f3b2c070b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3b38005950 tx=0x7f3b380058e0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 msgr2=0x7f3b4419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b4419c640 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006010 tx=0x7f3b3400be20 comp rx=0 tx=0).stop 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 shutdown_connections 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3b2c06e680 0x7f3b2c070b40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b440ffda0 0x7f3b4419c640 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 --2- 172.21.15.42:0/1562283754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b44100750 0x7f3b4419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 >> 172.21.15.42:0/1562283754 conn(0x7f3b440fb650 msgr2=0x7f3b44103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.994+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 shutdown_connections 2024-09-17T00:10:37.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:37.995+0000 7f3b4a152700 1 -- 172.21.15.42:0/1562283754 wait complete. 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 -- 172.21.15.42:0/3976449676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de40ffc20 msgr2=0x7f8de4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 --2- 172.21.15.42:0/3976449676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de40ffc20 0x7f8de4100080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4009910 tx=0x7f8dd4009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 -- 172.21.15.42:0/3976449676 shutdown_connections 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 --2- 172.21.15.42:0/3976449676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de40ffc20 0x7f8de4100080 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 --2- 172.21.15.42:0/3976449676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de4105bd0 0x7f8de4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 -- 172.21.15.42:0/3976449676 >> 172.21.15.42:0/3976449676 conn(0x7f8de4075670 msgr2=0x7f8de4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 -- 172.21.15.42:0/3976449676 shutdown_connections 2024-09-17T00:10:38.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.103+0000 7f8deb344700 1 -- 172.21.15.42:0/3976449676 wait complete. 2024-09-17T00:10:38.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8deb344700 1 Processor -- start 2024-09-17T00:10:38.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8deb344700 1 -- start start 2024-09-17T00:10:38.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8deb344700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40ffc20 0x7f8de419c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8deb344700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de4105bd0 0x7f8de419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8deb344700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de419d230 con 0x7f8de4105bd0 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8deb344700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de41966e0 con 0x7f8de40ffc20 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de88df700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de4105bd0 0x7f8de419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de88df700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de4105bd0 0x7f8de419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47072/0 (socket says 172.21.15.42:47072) 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de88df700 1 -- 172.21.15.42:0/723869997 learned_addr learned my addr 172.21.15.42:0/723869997 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de90e0700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40ffc20 0x7f8de419c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de88df700 1 -- 172.21.15.42:0/723869997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40ffc20 msgr2=0x7f8de419c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de88df700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40ffc20 0x7f8de419c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de88df700 1 -- 172.21.15.42:0/723869997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40095f0 con 0x7f8de4105bd0 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.104+0000 7f8de90e0700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40ffc20 0x7f8de419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.105+0000 7f8de88df700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de4105bd0 0x7f8de419cba0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006010 tx=0x7f8dd400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.105+0000 7f8dda7fc700 1 -- 172.21.15.42:0/723869997 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd401d070 con 0x7f8de4105bd0 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.105+0000 7f8dda7fc700 1 -- 172.21.15.42:0/723869997 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8dd4004fb0 con 0x7f8de4105bd0 2024-09-17T00:10:38.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.105+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de4196960 con 0x7f8de4105bd0 2024-09-17T00:10:38.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.105+0000 7f8dda7fc700 1 -- 172.21.15.42:0/723869997 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dd400f690 con 0x7f8de4105bd0 2024-09-17T00:10:38.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.105+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de4196e50 con 0x7f8de4105bd0 2024-09-17T00:10:38.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.106+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de404f2e0 con 0x7f8de4105bd0 2024-09-17T00:10:38.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.107+0000 7f8dda7fc700 1 -- 172.21.15.42:0/723869997 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8dd40228a0 con 0x7f8de4105bd0 2024-09-17T00:10:38.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.107+0000 7f8dda7fc700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd006e680 0x7f8dd0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.107+0000 7f8dda7fc700 1 -- 172.21.15.42:0/723869997 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8dd408fe50 con 0x7f8de4105bd0 2024-09-17T00:10:38.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.107+0000 7f8de90e0700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd006e680 0x7f8dd0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.108+0000 7f8de90e0700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd006e680 0x7f8dd0070b40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8de00060b0 tx=0x7f8de00076a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:38.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.112+0000 7f8dda7fc700 1 -- 172.21.15.42:0/723869997 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8dd405cba0 con 0x7f8de4105bd0 2024-09-17T00:10:38.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:38 smithi042 ceph-mon[111056]: pgmap v571: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:38.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:38 smithi042 ceph-mon[111056]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:38.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:38 smithi042 ceph-mon[111056]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:38.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:38 smithi042 ceph-mon[111056]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:38.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:38 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1562283754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:10:38.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.369+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8de4197bf0 con 0x7f8de4105bd0 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.369+0000 7f8dda7fc700 1 -- 172.21.15.42:0/723869997 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f8dd4027020 con 0x7f8de4105bd0 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:10:38.367 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:10:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:10:38.369 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:38.370 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd006e680 msgr2=0x7f8dd0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd006e680 0x7f8dd0070b40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8de00060b0 tx=0x7f8de00076a0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de4105bd0 msgr2=0x7f8de419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de4105bd0 0x7f8de419cba0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4006010 tx=0x7f8dd400be20 comp rx=0 tx=0).stop 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 shutdown_connections 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8dd006e680 0x7f8dd0070b40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40ffc20 0x7f8de419c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 --2- 172.21.15.42:0/723869997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8de4105bd0 0x7f8de419cba0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.371+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 >> 172.21.15.42:0/723869997 conn(0x7f8de4075670 msgr2=0x7f8de4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.372+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 shutdown_connections 2024-09-17T00:10:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.372+0000 7f8deb344700 1 -- 172.21.15.42:0/723869997 wait complete. 2024-09-17T00:10:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:10:38.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 -- 172.21.15.42:0/3280324487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 msgr2=0x7f3d9c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 --2- 172.21.15.42:0/3280324487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c105890 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009910 tx=0x7f3d8c009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:38.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 -- 172.21.15.42:0/3280324487 shutdown_connections 2024-09-17T00:10:38.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 --2- 172.21.15.42:0/3280324487 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c105890 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 --2- 172.21.15.42:0/3280324487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c100e90 0x7f3d9c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 -- 172.21.15.42:0/3280324487 >> 172.21.15.42:0/3280324487 conn(0x7f3d9c0fc740 msgr2=0x7f3d9c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 -- 172.21.15.42:0/3280324487 shutdown_connections 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 -- 172.21.15.42:0/3280324487 wait complete. 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.481+0000 7f3da3424700 1 Processor -- start 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da3424700 1 -- start start 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da3424700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c100e90 0x7f3d9c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da3424700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da3424700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c073b10 con 0x7f3d9c101840 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da3424700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c073c50 con 0x7f3d9c100e90 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da09bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da09bf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47084/0 (socket says 172.21.15.42:47084) 2024-09-17T00:10:38.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da09bf700 1 -- 172.21.15.42:0/1919939707 learned_addr learned my addr 172.21.15.42:0/1919939707 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:38.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da11c0700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c100e90 0x7f3d9c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da09bf700 1 -- 172.21.15.42:0/1919939707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c100e90 msgr2=0x7f3d9c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da09bf700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c100e90 0x7f3d9c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da09bf700 1 -- 172.21.15.42:0/1919939707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c0095f0 con 0x7f3d9c101840 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.482+0000 7f3da09bf700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c0735d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00bd70 tx=0x7f3d8c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.483+0000 7f3d927fc700 1 -- 172.21.15.42:0/1919939707 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d8c01d070 con 0x7f3d9c101840 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.483+0000 7f3d927fc700 1 -- 172.21.15.42:0/1919939707 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d8c005dc0 con 0x7f3d9c101840 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.483+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c073ed0 con 0x7f3d9c101840 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.483+0000 7f3d927fc700 1 -- 172.21.15.42:0/1919939707 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d8c017400 con 0x7f3d9c101840 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.483+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c1a2e40 con 0x7f3d9c101840 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.484+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c109290 con 0x7f3d9c101840 2024-09-17T00:10:38.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.484+0000 7f3d927fc700 1 -- 172.21.15.42:0/1919939707 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f3d8c00f460 con 0x7f3d9c101840 2024-09-17T00:10:38.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.485+0000 7f3d927fc700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d8806e680 0x7f3d88070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.485+0000 7f3d927fc700 1 -- 172.21.15.42:0/1919939707 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d8c08fa30 con 0x7f3d9c101840 2024-09-17T00:10:38.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.485+0000 7f3da11c0700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d8806e680 0x7f3d88070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.485+0000 7f3da11c0700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d8806e680 0x7f3d88070b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005950 tx=0x7f3d9800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:38.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.489+0000 7f3d927fc700 1 -- 172.21.15.42:0/1919939707 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3d8c058bf0 con 0x7f3d9c101840 2024-09-17T00:10:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:38 smithi043 ceph-mon[81510]: pgmap v571: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:38 smithi043 ceph-mon[81510]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:38 smithi043 ceph-mon[81510]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:38 smithi043 ceph-mon[81510]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:38.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:38 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1562283754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:10:38.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.722+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d9c074630 con 0x7f3d8806e680 2024-09-17T00:10:38.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.723+0000 7f3d927fc700 1 -- 172.21.15.42:0/1919939707 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3d9c074630 con 0x7f3d8806e680 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:10:38.721 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:10:38.722 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:10:38.722 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:10:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d8806e680 msgr2=0x7f3d88070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d8806e680 0x7f3d88070b40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005950 tx=0x7f3d9800b410 comp rx=0 tx=0).stop 2024-09-17T00:10:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 msgr2=0x7f3d9c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c0735d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00bd70 tx=0x7f3d8c00bda0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 shutdown_connections 2024-09-17T00:10:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f3d8806e680 0x7f3d88070b40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d9c100e90 0x7f3d9c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 --2- 172.21.15.42:0/1919939707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3d9c101840 0x7f3d9c0735d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.726+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 >> 172.21.15.42:0/1919939707 conn(0x7f3d9c0fc740 msgr2=0x7f3d9c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:38.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.727+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 shutdown_connections 2024-09-17T00:10:38.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.727+0000 7f3da3424700 1 -- 172.21.15.42:0/1919939707 wait complete. 2024-09-17T00:10:38.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 -- 172.21.15.42:0/3408992166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 msgr2=0x7f47c8102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/3408992166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c8102260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009910 tx=0x7f47c4009c20 comp rx=0 tx=0).stop 2024-09-17T00:10:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 -- 172.21.15.42:0/3408992166 shutdown_connections 2024-09-17T00:10:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/3408992166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c81027a0 0x7f47c810ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/3408992166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c8102260 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 -- 172.21.15.42:0/3408992166 >> 172.21.15.42:0/3408992166 conn(0x7f47c80fb610 msgr2=0x7f47c80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 -- 172.21.15.42:0/3408992166 shutdown_connections 2024-09-17T00:10:38.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.836+0000 7f47cfd4a700 1 -- 172.21.15.42:0/3408992166 wait complete. 2024-09-17T00:10:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cfd4a700 1 Processor -- start 2024-09-17T00:10:38.835 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cfd4a700 1 -- start start 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cfd4a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c80ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cfd4a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c81027a0 0x7f47c8100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cfd4a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8100560 con 0x7f47c8101e80 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cfd4a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c81006a0 con 0x7f47c81027a0 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cdae6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c80ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cdae6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c80ffae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47102/0 (socket says 172.21.15.42:47102) 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.837+0000 7f47cdae6700 1 -- 172.21.15.42:0/1945473350 learned_addr learned my addr 172.21.15.42:0/1945473350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cd2e5700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c81027a0 0x7f47c8100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cdae6700 1 -- 172.21.15.42:0/1945473350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c81027a0 msgr2=0x7f47c8100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cdae6700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c81027a0 0x7f47c8100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cdae6700 1 -- 172.21.15.42:0/1945473350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40095f0 con 0x7f47c8101e80 2024-09-17T00:10:38.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cd2e5700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c81027a0 0x7f47c8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cdae6700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c80ffae0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f47c40092e0 tx=0x7f47c400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47baffd700 1 -- 172.21.15.42:0/1945473350 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c401d070 con 0x7f47c8101e80 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47baffd700 1 -- 172.21.15.42:0/1945473350 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47c400f460 con 0x7f47c8101e80 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c8100920 con 0x7f47c8101e80 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47baffd700 1 -- 172.21.15.42:0/1945473350 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c4017630 con 0x7f47c8101e80 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.838+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c81a2d50 con 0x7f47c8101e80 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.839+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c8108390 con 0x7f47c8101e80 2024-09-17T00:10:38.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.840+0000 7f47baffd700 1 -- 172.21.15.42:0/1945473350 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f47c400f5d0 con 0x7f47c8101e80 2024-09-17T00:10:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.841+0000 7f47baffd700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b406e680 0x7f47b4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:10:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.841+0000 7f47baffd700 1 -- 172.21.15.42:0/1945473350 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f47c408fb80 con 0x7f47c8101e80 2024-09-17T00:10:38.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.841+0000 7f47cd2e5700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b406e680 0x7f47b4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:10:38.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.841+0000 7f47cd2e5700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b406e680 0x7f47b4070b40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005fd0 tx=0x7f47bc005e20 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:10:38.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:38.845+0000 7f47baffd700 1 -- 172.21.15.42:0/1945473350 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f47c4058da0 con 0x7f47c8101e80 2024-09-17T00:10:39.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.139+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f47c8107d70 con 0x7f47c8101e80 2024-09-17T00:10:39.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.140+0000 7f47baffd700 1 -- 172.21.15.42:0/1945473350 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f47c405c460 con 0x7f47c8101e80 2024-09-17T00:10:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:10:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:10:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:10:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:10:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:10:39.138 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:10:39.138 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:10:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b406e680 msgr2=0x7f47b4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b406e680 0x7f47b4070b40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005fd0 tx=0x7f47bc005e20 comp rx=0 tx=0).stop 2024-09-17T00:10:39.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 msgr2=0x7f47c80ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c80ffae0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f47c40092e0 tx=0x7f47c400be00 comp rx=0 tx=0).stop 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 shutdown_connections 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f47b406e680 0x7f47b4070b40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47c8101e80 0x7f47c80ffae0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 --2- 172.21.15.42:0/1945473350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c81027a0 0x7f47c8100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 >> 172.21.15.42:0/1945473350 conn(0x7f47c80fb610 msgr2=0x7f47c81054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 shutdown_connections 2024-09-17T00:10:39.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:10:39.142+0000 7f47cfd4a700 1 -- 172.21.15.42:0/1945473350 wait complete. 2024-09-17T00:10:39.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:39 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/723869997' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:10:39.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:39 smithi042 ceph-mon[111056]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:39.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:39 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/723869997' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:10:39.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:39 smithi043 ceph-mon[81510]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:10:40.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:40 smithi042 ceph-mon[111056]: pgmap v572: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:40.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:40 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1945473350' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:10:40.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:40 smithi043 ceph-mon[81510]: pgmap v572: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:40.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:40 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1945473350' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:10:42.430 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:42 smithi043 ceph-mon[81510]: pgmap v573: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:42.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:42 smithi042 ceph-mon[111056]: pgmap v573: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:44.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:44 smithi042 ceph-mon[111056]: pgmap v574: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:44.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:44 smithi043 ceph-mon[81510]: pgmap v574: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:46.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:46 smithi042 ceph-mon[111056]: pgmap v575: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:46.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:46 smithi043 ceph-mon[81510]: pgmap v575: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:48.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:48 smithi043 ceph-mon[81510]: pgmap v576: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:48.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:48 smithi042 ceph-mon[111056]: pgmap v576: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:50.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:50 smithi042 ceph-mon[111056]: pgmap v577: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:50.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:50 smithi043 ceph-mon[81510]: pgmap v577: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:51.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:51.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:10:52.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:52 smithi042 ceph-mon[111056]: pgmap v578: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:52.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:52 smithi043 ceph-mon[81510]: pgmap v578: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:54.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:54 smithi042 ceph-mon[111056]: pgmap v579: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:54.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:54 smithi043 ceph-mon[81510]: pgmap v579: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:56.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:56 smithi042 ceph-mon[111056]: pgmap v580: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:56.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:56 smithi043 ceph-mon[81510]: pgmap v580: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:58.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:10:58 smithi043 ceph-mon[81510]: pgmap v581: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:10:58.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:10:58 smithi042 ceph-mon[111056]: pgmap v581: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:00.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:00 smithi042 ceph-mon[111056]: pgmap v582: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:00.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:00 smithi043 ceph-mon[81510]: pgmap v582: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:02.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:02 smithi042 ceph-mon[111056]: pgmap v583: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:02.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:02 smithi043 ceph-mon[81510]: pgmap v583: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:04.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:04 smithi042 ceph-mon[111056]: pgmap v584: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:04.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:04 smithi043 ceph-mon[81510]: pgmap v584: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:06 smithi043 ceph-mon[81510]: pgmap v585: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:06.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:06.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:06 smithi042 ceph-mon[111056]: pgmap v585: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:06.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:08.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:08 smithi043 ceph-mon[81510]: pgmap v586: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:08.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:08 smithi042 ceph-mon[111056]: pgmap v586: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:09.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 -- 172.21.15.42:0/2333140909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a40ffda0 msgr2=0x7f25a4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 --2- 172.21.15.42:0/2333140909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a40ffda0 0x7f25a4100180 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2594009910 tx=0x7f2594009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:09.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 -- 172.21.15.42:0/2333140909 shutdown_connections 2024-09-17T00:11:09.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 --2- 172.21.15.42:0/2333140909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a4100750 0x7f25a41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 --2- 172.21.15.42:0/2333140909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a40ffda0 0x7f25a4100180 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 -- 172.21.15.42:0/2333140909 >> 172.21.15.42:0/2333140909 conn(0x7f25a40fb650 msgr2=0x7f25a40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:09.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 -- 172.21.15.42:0/2333140909 shutdown_connections 2024-09-17T00:11:09.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.274+0000 7f25a93ee700 1 -- 172.21.15.42:0/2333140909 wait complete. 2024-09-17T00:11:09.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.275+0000 7f25a93ee700 1 Processor -- start 2024-09-17T00:11:09.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.275+0000 7f25a93ee700 1 -- start start 2024-09-17T00:11:09.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.275+0000 7f25a93ee700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a40ffda0 0x7f25a419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.275+0000 7f25a93ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a4100750 0x7f25a419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.275+0000 7f25a93ee700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a419f4f0 con 0x7f25a4100750 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.275+0000 7f25a93ee700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41989a0 con 0x7f25a40ffda0 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.275+0000 7f25a27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a4100750 0x7f25a419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a4100750 0x7f25a419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51508/0 (socket says 172.21.15.42:51508) 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a27fc700 1 -- 172.21.15.42:0/3445895536 learned_addr learned my addr 172.21.15.42:0/3445895536 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a2ffd700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a40ffda0 0x7f25a419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a27fc700 1 -- 172.21.15.42:0/3445895536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a40ffda0 msgr2=0x7f25a419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a27fc700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a40ffda0 0x7f25a419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a27fc700 1 -- 172.21.15.42:0/3445895536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25940095f0 con 0x7f25a4100750 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a2ffd700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a40ffda0 0x7f25a419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a27fc700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a4100750 0x7f25a419ee60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f259800d900 tx=0x7f259800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f258bfff700 1 -- 172.21.15.42:0/3445895536 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2598009900 con 0x7f25a4100750 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a4198c80 con 0x7f25a4100750 2024-09-17T00:11:09.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f258bfff700 1 -- 172.21.15.42:0/3445895536 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2598010460 con 0x7f25a4100750 2024-09-17T00:11:09.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f258bfff700 1 -- 172.21.15.42:0/3445895536 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f259800f5f0 con 0x7f25a4100750 2024-09-17T00:11:09.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.276+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a41991d0 con 0x7f25a4100750 2024-09-17T00:11:09.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.277+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a404f2e0 con 0x7f25a4100750 2024-09-17T00:11:09.276 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.278+0000 7f258bfff700 1 -- 172.21.15.42:0/3445895536 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f25980105d0 con 0x7f25a4100750 2024-09-17T00:11:09.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.278+0000 7f258bfff700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f258c06e680 0x7f258c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.278+0000 7f258bfff700 1 -- 172.21.15.42:0/3445895536 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f259808e570 con 0x7f25a4100750 2024-09-17T00:11:09.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.279+0000 7f25a2ffd700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f258c06e680 0x7f258c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.277 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.279+0000 7f25a2ffd700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f258c06e680 0x7f258c070b40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2594005b40 tx=0x7f259401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:09.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.283+0000 7f258bfff700 1 -- 172.21.15.42:0/3445895536 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f259805b1a0 con 0x7f25a4100750 2024-09-17T00:11:09.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.515+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25a41993f0 con 0x7f258c06e680 2024-09-17T00:11:09.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.517+0000 7f258bfff700 1 -- 172.21.15.42:0/3445895536 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f25a41993f0 con 0x7f258c06e680 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f258c06e680 msgr2=0x7f258c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f258c06e680 0x7f258c070b40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2594005b40 tx=0x7f259401a040 comp rx=0 tx=0).stop 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a4100750 msgr2=0x7f25a419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a4100750 0x7f25a419ee60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f259800d900 tx=0x7f259800dcc0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 shutdown_connections 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f258c06e680 0x7f258c070b40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25a40ffda0 0x7f25a419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 --2- 172.21.15.42:0/3445895536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f25a4100750 0x7f25a419ee60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 >> 172.21.15.42:0/3445895536 conn(0x7f25a40fb650 msgr2=0x7f25a4103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:09.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 shutdown_connections 2024-09-17T00:11:09.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.519+0000 7f25a93ee700 1 -- 172.21.15.42:0/3445895536 wait complete. 2024-09-17T00:11:09.527 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:11:09.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 -- 172.21.15.42:0/3892468055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c100320 msgr2=0x7fd21c10cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 --2- 172.21.15.42:0/3892468055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c100320 0x7fd21c10cfe0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd204009910 tx=0x7fd204009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 -- 172.21.15.42:0/3892468055 shutdown_connections 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 --2- 172.21.15.42:0/3892468055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c100320 0x7fd21c10cfe0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 --2- 172.21.15.42:0/3892468055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd21c0ff970 0x7fd21c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 -- 172.21.15.42:0/3892468055 >> 172.21.15.42:0/3892468055 conn(0x7fd21c0fb650 msgr2=0x7fd21c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 -- 172.21.15.42:0/3892468055 shutdown_connections 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.629+0000 7fd222a07700 1 -- 172.21.15.42:0/3892468055 wait complete. 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd222a07700 1 Processor -- start 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd222a07700 1 -- start start 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd222a07700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c0ff970 0x7fd21c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd222a07700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd21c100320 0x7fd21c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd222a07700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c19f3f0 con 0x7fd21c0ff970 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd222a07700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c1988a0 con 0x7fd21c100320 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd21bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c0ff970 0x7fd21c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd21bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c0ff970 0x7fd21c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51530/0 (socket says 172.21.15.42:51530) 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd21bfff700 1 -- 172.21.15.42:0/1642578999 learned_addr learned my addr 172.21.15.42:0/1642578999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd21bfff700 1 -- 172.21.15.42:0/1642578999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd21c100320 msgr2=0x7fd21c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd213fff700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd21c100320 0x7fd21c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd21bfff700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd21c100320 0x7fd21c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd21bfff700 1 -- 172.21.15.42:0/1642578999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040095f0 con 0x7fd21c0ff970 2024-09-17T00:11:09.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.630+0000 7fd21bfff700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c0ff970 0x7fd21c19e820 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00d8d0 tx=0x7fd20c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:09.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.631+0000 7fd219ffb700 1 -- 172.21.15.42:0/1642578999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20c009940 con 0x7fd21c0ff970 2024-09-17T00:11:09.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.631+0000 7fd219ffb700 1 -- 172.21.15.42:0/1642578999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd20c010460 con 0x7fd21c0ff970 2024-09-17T00:11:09.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.631+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c198b80 con 0x7fd21c0ff970 2024-09-17T00:11:09.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.631+0000 7fd219ffb700 1 -- 172.21.15.42:0/1642578999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd20c00f5f0 con 0x7fd21c0ff970 2024-09-17T00:11:09.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.631+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c1990d0 con 0x7fd21c0ff970 2024-09-17T00:11:09.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.632+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c10a750 con 0x7fd21c0ff970 2024-09-17T00:11:09.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.632+0000 7fd219ffb700 1 -- 172.21.15.42:0/1642578999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd20c0105d0 con 0x7fd21c0ff970 2024-09-17T00:11:09.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.633+0000 7fd219ffb700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd20806e680 0x7fd208070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.633+0000 7fd213fff700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd20806e680 0x7fd208070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.633+0000 7fd219ffb700 1 -- 172.21.15.42:0/1642578999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd20c08e570 con 0x7fd21c0ff970 2024-09-17T00:11:09.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.633+0000 7fd213fff700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd20806e680 0x7fd208070b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd204006010 tx=0x7fd20400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:09.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.638+0000 7fd219ffb700 1 -- 172.21.15.42:0/1642578999 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd20c05b2c0 con 0x7fd21c0ff970 2024-09-17T00:11:09.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.869+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd21c199340 con 0x7fd20806e680 2024-09-17T00:11:09.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.870+0000 7fd219ffb700 1 -- 172.21.15.42:0/1642578999 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd21c199340 con 0x7fd20806e680 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd20806e680 msgr2=0x7fd208070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd20806e680 0x7fd208070b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd204006010 tx=0x7fd20400b540 comp rx=0 tx=0).stop 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c0ff970 msgr2=0x7fd21c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c0ff970 0x7fd21c19e820 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00d8d0 tx=0x7fd20c00dc90 comp rx=0 tx=0).stop 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 shutdown_connections 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd20806e680 0x7fd208070b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21c0ff970 0x7fd21c19e820 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 --2- 172.21.15.42:0/1642578999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd21c100320 0x7fd21c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 >> 172.21.15.42:0/1642578999 conn(0x7fd21c0fb650 msgr2=0x7fd21c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 shutdown_connections 2024-09-17T00:11:09.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.872+0000 7fd222a07700 1 -- 172.21.15.42:0/1642578999 wait complete. 2024-09-17T00:11:09.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.980+0000 7f6f421ef700 1 -- 172.21.15.42:0/3219853719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 msgr2=0x7f6f3c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.980+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3219853719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c1058a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009910 tx=0x7f6f30009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 -- 172.21.15.42:0/3219853719 shutdown_connections 2024-09-17T00:11:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3219853719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c1058a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3219853719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c100ea0 0x7f6f3c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 -- 172.21.15.42:0/3219853719 >> 172.21.15.42:0/3219853719 conn(0x7f6f3c0fc750 msgr2=0x7f6f3c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 -- 172.21.15.42:0/3219853719 shutdown_connections 2024-09-17T00:11:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 -- 172.21.15.42:0/3219853719 wait complete. 2024-09-17T00:11:09.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 Processor -- start 2024-09-17T00:11:09.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.981+0000 7f6f421ef700 1 -- start start 2024-09-17T00:11:09.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f421ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c100ea0 0x7f6f3c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f421ef700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f421ef700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c19d2d0 con 0x7f6f3c101850 2024-09-17T00:11:09.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f421ef700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c196780 con 0x7f6f3c100ea0 2024-09-17T00:11:09.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51542/0 (socket says 172.21.15.42:51542) 2024-09-17T00:11:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3affd700 1 -- 172.21.15.42:0/3599581651 learned_addr learned my addr 172.21.15.42:0/3599581651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3b7fe700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c100ea0 0x7f6f3c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3affd700 1 -- 172.21.15.42:0/3599581651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c100ea0 msgr2=0x7f6f3c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3affd700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c100ea0 0x7f6f3c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3affd700 1 -- 172.21.15.42:0/3599581651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300095f0 con 0x7f6f3c101850 2024-09-17T00:11:09.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.982+0000 7f6f3affd700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c19cc40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f300092e0 tx=0x7f6f3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:09.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.983+0000 7f6f38ff9700 1 -- 172.21.15.42:0/3599581651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f3001d070 con 0x7f6f3c101850 2024-09-17T00:11:09.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.983+0000 7f6f38ff9700 1 -- 172.21.15.42:0/3599581651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f30005dc0 con 0x7f6f3c101850 2024-09-17T00:11:09.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.983+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c196a00 con 0x7f6f3c101850 2024-09-17T00:11:09.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.983+0000 7f6f38ff9700 1 -- 172.21.15.42:0/3599581651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f30017400 con 0x7f6f3c101850 2024-09-17T00:11:09.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.983+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c196ef0 con 0x7f6f3c101850 2024-09-17T00:11:09.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.984+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c1092a0 con 0x7f6f3c101850 2024-09-17T00:11:09.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.984+0000 7f6f38ff9700 1 -- 172.21.15.42:0/3599581651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6f3000f460 con 0x7f6f3c101850 2024-09-17T00:11:09.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.985+0000 7f6f38ff9700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6f2406e680 0x7f6f24070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:09.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.985+0000 7f6f38ff9700 1 -- 172.21.15.42:0/3599581651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6f3008fc10 con 0x7f6f3c101850 2024-09-17T00:11:09.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.985+0000 7f6f3b7fe700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6f2406e680 0x7f6f24070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:09.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.985+0000 7f6f3b7fe700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6f2406e680 0x7f6f24070b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005950 tx=0x7f6f2c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:09.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:09.989+0000 7f6f38ff9700 1 -- 172.21.15.42:0/3599581651 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6f3005c840 con 0x7f6f3c101850 2024-09-17T00:11:10.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.211+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f3c197110 con 0x7f6f2406e680 2024-09-17T00:11:10.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.221+0000 7f6f38ff9700 1 -- 172.21.15.42:0/3599581651 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f6f3c197110 con 0x7f6f2406e680 2024-09-17T00:11:10.218 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:11:10.218 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (19m) 13m ago 30m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:11:10.218 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (30m) 13m ago 30m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:11:10.218 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (29m) 12m ago 29m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:11:10.218 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (17m) 13m ago 30m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:11:10.218 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (17m) 12m ago 29m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (19m) 13m ago 30m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (13m) 13m ago 24m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (13m) 13m ago 24m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (13m) 12m ago 24m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (13m) 12m ago 24m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (21m) 13m ago 32m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (20m) 12m ago 29m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (18m) 13m ago 32m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (18m) 12m ago 29m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (20m) 13m ago 30m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (20m) 12m ago 29m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (17m) 13m ago 28m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (16m) 13m ago 28m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:11:10.219 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (15m) 13m ago 27m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:11:10.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (15m) 12m ago 27m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:11:10.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (14m) 12m ago 26m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:11:10.220 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (14m) 12m ago 26m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:11:10.220 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (20m) 13m ago 30m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:11:10.220 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6f2406e680 msgr2=0x7f6f24070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6f2406e680 0x7f6f24070b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c005950 tx=0x7f6f2c0058e0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 msgr2=0x7f6f3c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c19cc40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f300092e0 tx=0x7f6f3000be20 comp rx=0 tx=0).stop 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 shutdown_connections 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6f2406e680 0x7f6f24070b40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f3c100ea0 0x7f6f3c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 --2- 172.21.15.42:0/3599581651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f3c101850 0x7f6f3c19cc40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 >> 172.21.15.42:0/3599581651 conn(0x7f6f3c0fc750 msgr2=0x7f6f3c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:10.221 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 shutdown_connections 2024-09-17T00:11:10.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.223+0000 7f6f421ef700 1 -- 172.21.15.42:0/3599581651 wait complete. 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 -- 172.21.15.42:0/2367275761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 msgr2=0x7efc181058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 --2- 172.21.15.42:0/2367275761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc181058a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efc08009910 tx=0x7efc08009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 -- 172.21.15.42:0/2367275761 shutdown_connections 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 --2- 172.21.15.42:0/2367275761 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc181058a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 --2- 172.21.15.42:0/2367275761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc18100ea0 0x7efc18101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 -- 172.21.15.42:0/2367275761 >> 172.21.15.42:0/2367275761 conn(0x7efc180fc750 msgr2=0x7efc180feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 -- 172.21.15.42:0/2367275761 shutdown_connections 2024-09-17T00:11:10.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.330+0000 7efc205fe700 1 -- 172.21.15.42:0/2367275761 wait complete. 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc205fe700 1 Processor -- start 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc205fe700 1 -- start start 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc205fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc18100ea0 0x7efc1819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc205fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc1819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc205fe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1819f4a0 con 0x7efc18101850 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc205fe700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc18198950 con 0x7efc18100ea0 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc1db99700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc1819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc1db99700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc1819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51562/0 (socket says 172.21.15.42:51562) 2024-09-17T00:11:10.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc1db99700 1 -- 172.21.15.42:0/3443891251 learned_addr learned my addr 172.21.15.42:0/3443891251 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.331+0000 7efc1e39a700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc18100ea0 0x7efc1819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc1db99700 1 -- 172.21.15.42:0/3443891251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc18100ea0 msgr2=0x7efc1819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc1db99700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc18100ea0 0x7efc1819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc1db99700 1 -- 172.21.15.42:0/3443891251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc080095f0 con 0x7efc18101850 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc1db99700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc1819ee10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efc08005f50 tx=0x7efc0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc0f7fe700 1 -- 172.21.15.42:0/3443891251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0801d070 con 0x7efc18101850 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc18198bd0 con 0x7efc18101850 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc0f7fe700 1 -- 172.21.15.42:0/3443891251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc0800f460 con 0x7efc18101850 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc0f7fe700 1 -- 172.21.15.42:0/3443891251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc08017630 con 0x7efc18101850 2024-09-17T00:11:10.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.332+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc181990c0 con 0x7efc18101850 2024-09-17T00:11:10.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.333+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc18105020 con 0x7efc18101850 2024-09-17T00:11:10.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.334+0000 7efc0f7fe700 1 -- 172.21.15.42:0/3443891251 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7efc0800f5d0 con 0x7efc18101850 2024-09-17T00:11:10.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.334+0000 7efc0f7fe700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efc0406e680 0x7efc04070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:10.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.334+0000 7efc0f7fe700 1 -- 172.21.15.42:0/3443891251 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efc0808fc80 con 0x7efc18101850 2024-09-17T00:11:10.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.334+0000 7efc1e39a700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efc0406e680 0x7efc04070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:10.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.335+0000 7efc1e39a700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efc0406e680 0x7efc04070b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc14005fd0 tx=0x7efc1400a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:10.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.339+0000 7efc0f7fe700 1 -- 172.21.15.42:0/3443891251 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efc0805c950 con 0x7efc18101850 2024-09-17T00:11:10.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:10 smithi042 ceph-mon[111056]: pgmap v587: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:10.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:10 smithi042 ceph-mon[111056]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:10.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:10 smithi043 ceph-mon[81510]: pgmap v587: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:10.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:10 smithi043 ceph-mon[81510]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:10.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.636+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efc18199e90 con 0x7efc18101850 2024-09-17T00:11:10.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.636+0000 7efc0f7fe700 1 -- 172.21.15.42:0/3443891251 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7efc0805c4e0 con 0x7efc18101850 2024-09-17T00:11:10.634 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:11:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:11:10.634 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:11:10.635 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:11:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efc0406e680 msgr2=0x7efc04070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efc0406e680 0x7efc04070b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc14005fd0 tx=0x7efc1400a560 comp rx=0 tx=0).stop 2024-09-17T00:11:10.636 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 msgr2=0x7efc1819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc1819ee10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efc08005f50 tx=0x7efc0800be20 comp rx=0 tx=0).stop 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 shutdown_connections 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7efc0406e680 0x7efc04070b40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc18100ea0 0x7efc1819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 --2- 172.21.15.42:0/3443891251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc18101850 0x7efc1819ee10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 >> 172.21.15.42:0/3443891251 conn(0x7efc180fc750 msgr2=0x7efc180feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 shutdown_connections 2024-09-17T00:11:10.637 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.638+0000 7efc205fe700 1 -- 172.21.15.42:0/3443891251 wait complete. 2024-09-17T00:11:10.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 -- 172.21.15.42:0/3381467276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88100ea0 msgr2=0x7fab88101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 --2- 172.21.15.42:0/3381467276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88100ea0 0x7fab88101280 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fab78009910 tx=0x7fab78009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:10.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 -- 172.21.15.42:0/3381467276 shutdown_connections 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 --2- 172.21.15.42:0/3381467276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab88101850 0x7fab881058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 --2- 172.21.15.42:0/3381467276 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88100ea0 0x7fab88101280 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 -- 172.21.15.42:0/3381467276 >> 172.21.15.42:0/3381467276 conn(0x7fab880fc750 msgr2=0x7fab880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 -- 172.21.15.42:0/3381467276 shutdown_connections 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.746+0000 7fab8ef34700 1 -- 172.21.15.42:0/3381467276 wait complete. 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.747+0000 7fab8ef34700 1 Processor -- start 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.747+0000 7fab8ef34700 1 -- start start 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.747+0000 7fab8ef34700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab88100ea0 0x7fab88196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.747+0000 7fab8ef34700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88101850 0x7fab88196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:10.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.747+0000 7fab8ef34700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab88197560 con 0x7fab88101850 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.747+0000 7fab8ef34700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8819b370 con 0x7fab88100ea0 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.747+0000 7fab87fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88101850 0x7fab88196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab87fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88101850 0x7fab88196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51578/0 (socket says 172.21.15.42:51578) 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab87fff700 1 -- 172.21.15.42:0/936026785 learned_addr learned my addr 172.21.15.42:0/936026785 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab8ccd0700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab88100ea0 0x7fab88196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab87fff700 1 -- 172.21.15.42:0/936026785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab88100ea0 msgr2=0x7fab88196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab87fff700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab88100ea0 0x7fab88196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab87fff700 1 -- 172.21.15.42:0/936026785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab780095f0 con 0x7fab88101850 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab8ccd0700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab88100ea0 0x7fab88196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab87fff700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88101850 0x7fab88196ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b480 tx=0x7fab7c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab85ffb700 1 -- 172.21.15.42:0/936026785 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab7c0118c0 con 0x7fab88101850 2024-09-17T00:11:10.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab85ffb700 1 -- 172.21.15.42:0/936026785 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab7c010380 con 0x7fab88101850 2024-09-17T00:11:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8819b650 con 0x7fab88101850 2024-09-17T00:11:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab85ffb700 1 -- 172.21.15.42:0/936026785 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab7c00f4f0 con 0x7fab88101850 2024-09-17T00:11:10.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.748+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8819bba0 con 0x7fab88101850 2024-09-17T00:11:10.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.749+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab88109240 con 0x7fab88101850 2024-09-17T00:11:10.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.750+0000 7fab85ffb700 1 -- 172.21.15.42:0/936026785 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fab7c0104f0 con 0x7fab88101850 2024-09-17T00:11:10.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.750+0000 7fab85ffb700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab7006e680 0x7fab70070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:10.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.750+0000 7fab85ffb700 1 -- 172.21.15.42:0/936026785 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fab7c08e2c0 con 0x7fab88101850 2024-09-17T00:11:10.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.750+0000 7fab8ccd0700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab7006e680 0x7fab70070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:10.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.751+0000 7fab8ccd0700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab7006e680 0x7fab70070b40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab78006010 tx=0x7fab7800b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:10.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:10.755+0000 7fab85ffb700 1 -- 172.21.15.42:0/936026785 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fab7c05af90 con 0x7fab88101850 2024-09-17T00:11:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.006+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab8804f2e0 con 0x7fab88101850 2024-09-17T00:11:11.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.007+0000 7fab85ffb700 1 -- 172.21.15.42:0/936026785 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7fab7c05ab20 con 0x7fab88101850 2024-09-17T00:11:11.004 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:11:11.005 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:11:11.006 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:11:11.007 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:11.008 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:11.008 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:11.008 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:11.008 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:11.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab7006e680 msgr2=0x7fab70070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab7006e680 0x7fab70070b40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab78006010 tx=0x7fab7800b540 comp rx=0 tx=0).stop 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88101850 msgr2=0x7fab88196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88101850 0x7fab88196ed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b480 tx=0x7fab7c00b840 comp rx=0 tx=0).stop 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 shutdown_connections 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fab7006e680 0x7fab70070b40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab88100ea0 0x7fab88196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 --2- 172.21.15.42:0/936026785 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab88101850 0x7fab88196ed0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 >> 172.21.15.42:0/936026785 conn(0x7fab880fc750 msgr2=0x7fab880fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 shutdown_connections 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.009+0000 7fab8ef34700 1 -- 172.21.15.42:0/936026785 wait complete. 2024-09-17T00:11:11.009 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:11:11.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 -- 172.21.15.42:0/4248947869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 msgr2=0x7f12a81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/4248947869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a81021c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1298009910 tx=0x7f1298009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:11.121 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 -- 172.21.15.42:0/4248947869 shutdown_connections 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/4248947869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8102700 0x7f12a810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/4248947869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a81021c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 -- 172.21.15.42:0/4248947869 >> 172.21.15.42:0/4248947869 conn(0x7f12a80fb650 msgr2=0x7f12a80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 -- 172.21.15.42:0/4248947869 shutdown_connections 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 -- 172.21.15.42:0/4248947869 wait complete. 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 Processor -- start 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.118+0000 7f12ae9a3700 1 -- start start 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12ae9a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a80ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12ae9a3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8102700 0x7f12a80fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12ae9a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81004c0 con 0x7f12a8101de0 2024-09-17T00:11:11.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12ae9a3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8100600 con 0x7f12a8102700 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a80ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a80ffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51594/0 (socket says 172.21.15.42:51594) 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a7fff700 1 -- 172.21.15.42:0/2649965386 learned_addr learned my addr 172.21.15.42:0/2649965386 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a77fe700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8102700 0x7f12a80fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a7fff700 1 -- 172.21.15.42:0/2649965386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8102700 msgr2=0x7f12a80fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a7fff700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8102700 0x7f12a80fff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a7fff700 1 -- 172.21.15.42:0/2649965386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980095f0 con 0x7f12a8101de0 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.119+0000 7f12a7fff700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a80ffa40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f12980092e0 tx=0x7f129800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.120+0000 7f12a57fa700 1 -- 172.21.15.42:0/2649965386 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129801d070 con 0x7f12a8101de0 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.120+0000 7f12a57fa700 1 -- 172.21.15.42:0/2649965386 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1298005e00 con 0x7f12a8101de0 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.120+0000 7f12a57fa700 1 -- 172.21.15.42:0/2649965386 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1298017400 con 0x7f12a8101de0 2024-09-17T00:11:11.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.120+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a8100880 con 0x7f12a8101de0 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.120+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a81a2dd0 con 0x7f12a8101de0 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.121+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a81082f0 con 0x7f12a8101de0 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.121+0000 7f12a57fa700 1 -- 172.21.15.42:0/2649965386 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f129800f6f0 con 0x7f12a8101de0 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.122+0000 7f12a57fa700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129006e680 0x7f1290070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.122+0000 7f12a57fa700 1 -- 172.21.15.42:0/2649965386 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f129808fa50 con 0x7f12a8101de0 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.122+0000 7f12a77fe700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129006e680 0x7f1290070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.122+0000 7f12a77fe700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129006e680 0x7f1290070b40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f12a8101230 tx=0x7f129c00b360 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:11.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.126+0000 7f12a57fa700 1 -- 172.21.15.42:0/2649965386 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1298058c70 con 0x7f12a8101de0 2024-09-17T00:11:11.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.361+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12a80611d0 con 0x7f129006e680 2024-09-17T00:11:11.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.362+0000 7f12a57fa700 1 -- 172.21.15.42:0/2649965386 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f12a80611d0 con 0x7f129006e680 2024-09-17T00:11:11.359 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:11:11.359 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:11:11.360 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:11:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129006e680 msgr2=0x7f1290070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129006e680 0x7f1290070b40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f12a8101230 tx=0x7f129c00b360 comp rx=0 tx=0).stop 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 msgr2=0x7f12a80ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a80ffa40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f12980092e0 tx=0x7f129800bd70 comp rx=0 tx=0).stop 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 shutdown_connections 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f129006e680 0x7f1290070b40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f12a8101de0 0x7f12a80ffa40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 --2- 172.21.15.42:0/2649965386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12a8102700 0x7f12a80fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 >> 172.21.15.42:0/2649965386 conn(0x7f12a80fb650 msgr2=0x7f12a8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 shutdown_connections 2024-09-17T00:11:11.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.364+0000 7f12ae9a3700 1 -- 172.21.15.42:0/2649965386 wait complete. 2024-09-17T00:11:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 -- 172.21.15.42:0/2291816484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d440ffc20 msgr2=0x7f8d44100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 --2- 172.21.15.42:0/2291816484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d440ffc20 0x7f8d44100080 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009910 tx=0x7f8d38009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:11.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 -- 172.21.15.42:0/2291816484 shutdown_connections 2024-09-17T00:11:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 --2- 172.21.15.42:0/2291816484 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d440ffc20 0x7f8d44100080 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 --2- 172.21.15.42:0/2291816484 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d44105bd0 0x7f8d44105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 -- 172.21.15.42:0/2291816484 >> 172.21.15.42:0/2291816484 conn(0x7f8d44075670 msgr2=0x7f8d44075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 -- 172.21.15.42:0/2291816484 shutdown_connections 2024-09-17T00:11:11.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 -- 172.21.15.42:0/2291816484 wait complete. 2024-09-17T00:11:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.472+0000 7f8d49511700 1 Processor -- start 2024-09-17T00:11:11.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d49511700 1 -- start start 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d49511700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d49511700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d49511700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d441022c0 con 0x7f8d44105bd0 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d49511700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44102400 con 0x7f8d440ffc20 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d427fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d427fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51612/0 (socket says 172.21.15.42:51612) 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d427fc700 1 -- 172.21.15.42:0/3302598466 learned_addr learned my addr 172.21.15.42:0/3302598466 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d42ffd700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d427fc700 1 -- 172.21.15.42:0/3302598466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d440ffc20 msgr2=0x7f8d44101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d427fc700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d427fc700 1 -- 172.21.15.42:0/3302598466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380095f0 con 0x7f8d44105bd0 2024-09-17T00:11:11.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.473+0000 7f8d427fc700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d3800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:11.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.474+0000 7f8d2bfff700 1 -- 172.21.15.42:0/3302598466 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3801d070 con 0x7f8d44105bd0 2024-09-17T00:11:11.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.474+0000 7f8d2bfff700 1 -- 172.21.15.42:0/3302598466 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d3800f460 con 0x7f8d44105bd0 2024-09-17T00:11:11.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.474+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d44102680 con 0x7f8d44105bd0 2024-09-17T00:11:11.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.474+0000 7f8d2bfff700 1 -- 172.21.15.42:0/3302598466 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d38017610 con 0x7f8d44105bd0 2024-09-17T00:11:11.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.474+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d441a2d60 con 0x7f8d44105bd0 2024-09-17T00:11:11.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.475+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4404f2e0 con 0x7f8d44105bd0 2024-09-17T00:11:11.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.475+0000 7f8d2bfff700 1 -- 172.21.15.42:0/3302598466 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8d3800f5d0 con 0x7f8d44105bd0 2024-09-17T00:11:11.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.476+0000 7f8d2bfff700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d2c06e680 0x7f8d2c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:11.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.476+0000 7f8d2bfff700 1 -- 172.21.15.42:0/3302598466 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d3808fb40 con 0x7f8d44105bd0 2024-09-17T00:11:11.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.476+0000 7f8d42ffd700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d2c06e680 0x7f8d2c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:11.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.476+0000 7f8d42ffd700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d2c06e680 0x7f8d2c070b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005950 tx=0x7f8d3400a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:11.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.480+0000 7f8d2bfff700 1 -- 172.21.15.42:0/3302598466 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8d38058d60 con 0x7f8d44105bd0 2024-09-17T00:11:11.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:11 smithi043 ceph-mon[81510]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:11.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:11 smithi043 ceph-mon[81510]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:11.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:11 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3443891251' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:11:11.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:11 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/936026785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:11:11.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:11 smithi042 ceph-mon[111056]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:11 smithi042 ceph-mon[111056]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:11 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3443891251' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:11:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:11 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/936026785' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:11:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.777+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8d441a2fd0 con 0x7f8d44105bd0 2024-09-17T00:11:11.774 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.777+0000 7f8d2bfff700 1 -- 172.21.15.42:0/3302598466 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f8d3805c420 con 0x7f8d44105bd0 2024-09-17T00:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:11:11.775 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:11:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.779+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d2c06e680 msgr2=0x7f8d2c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.779+0000 7f8d49511700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d2c06e680 0x7f8d2c070b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005950 tx=0x7f8d3400a300 comp rx=0 tx=0).stop 2024-09-17T00:11:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.779+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44105bd0 msgr2=0x7f8d44101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.779+0000 7f8d49511700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d3800bd70 comp rx=0 tx=0).stop 2024-09-17T00:11:11.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.780+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 shutdown_connections 2024-09-17T00:11:11.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.780+0000 7f8d49511700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8d2c06e680 0x7f8d2c070b40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.780+0000 7f8d49511700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.780+0000 7f8d49511700 1 --2- 172.21.15.42:0/3302598466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:11.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.780+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 >> 172.21.15.42:0/3302598466 conn(0x7f8d44075670 msgr2=0x7f8d44068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:11.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.780+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 shutdown_connections 2024-09-17T00:11:11.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:11.780+0000 7f8d49511700 1 -- 172.21.15.42:0/3302598466 wait complete. 2024-09-17T00:11:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:12 smithi043 ceph-mon[81510]: pgmap v588: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:12 smithi043 ceph-mon[81510]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:12.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:12 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3302598466' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:11:12.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:12 smithi042 ceph-mon[111056]: pgmap v588: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:12 smithi042 ceph-mon[111056]: from='client.35124 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:12.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:12 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3302598466' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:11:14.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:14 smithi042 ceph-mon[111056]: pgmap v589: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:14.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:14 smithi043 ceph-mon[81510]: pgmap v589: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:16.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:16 smithi043 ceph-mon[81510]: pgmap v590: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:16.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:16 smithi042 ceph-mon[111056]: pgmap v590: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:18.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:18 smithi043 ceph-mon[81510]: pgmap v591: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:18.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:18 smithi042 ceph-mon[111056]: pgmap v591: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:20.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:20 smithi043 ceph-mon[81510]: pgmap v592: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:20.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:20 smithi042 ceph-mon[111056]: pgmap v592: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:21.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:21.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:22.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:22 smithi043 ceph-mon[81510]: pgmap v593: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:22.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:22 smithi042 ceph-mon[111056]: pgmap v593: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:24.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:24 smithi042 ceph-mon[111056]: pgmap v594: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:24.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:24 smithi043 ceph-mon[81510]: pgmap v594: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:26.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:26 smithi043 ceph-mon[81510]: pgmap v595: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:26 smithi042 ceph-mon[111056]: pgmap v595: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:28.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:28 smithi043 ceph-mon[81510]: pgmap v596: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:28 smithi042 ceph-mon[111056]: pgmap v596: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:30.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:30 smithi043 ceph-mon[81510]: pgmap v597: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:30.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:30 smithi042 ceph-mon[111056]: pgmap v597: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:32.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:32 smithi043 ceph-mon[81510]: pgmap v598: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:32.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:32 smithi042 ceph-mon[111056]: pgmap v598: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:34.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:34 smithi043 ceph-mon[81510]: pgmap v599: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:34.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:34 smithi042 ceph-mon[111056]: pgmap v599: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:36.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:36 smithi043 ceph-mon[81510]: pgmap v600: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:36.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:36 smithi042 ceph-mon[111056]: pgmap v600: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:38.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:38 smithi043 ceph-mon[81510]: pgmap v601: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:38.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:38 smithi042 ceph-mon[111056]: pgmap v601: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:40.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:40 smithi043 ceph-mon[81510]: pgmap v602: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:40.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:40 smithi042 ceph-mon[111056]: pgmap v602: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:41.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.902+0000 7f4adec18700 1 -- 172.21.15.42:0/426513071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8102720 msgr2=0x7f4ad810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:41.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.902+0000 7f4adec18700 1 --2- 172.21.15.42:0/426513071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8102720 0x7f4ad810ac10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009910 tx=0x7f4acc009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:41.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.902+0000 7f4adec18700 1 -- 172.21.15.42:0/426513071 shutdown_connections 2024-09-17T00:11:41.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.902+0000 7f4adec18700 1 --2- 172.21.15.42:0/426513071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8102720 0x7f4ad810ac10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:41.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.902+0000 7f4adec18700 1 --2- 172.21.15.42:0/426513071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad8101e00 0x7f4ad81021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.902+0000 7f4adec18700 1 -- 172.21.15.42:0/426513071 >> 172.21.15.42:0/426513071 conn(0x7f4ad80fb650 msgr2=0x7f4ad80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 -- 172.21.15.42:0/426513071 shutdown_connections 2024-09-17T00:11:41.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 -- 172.21.15.42:0/426513071 wait complete. 2024-09-17T00:11:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 Processor -- start 2024-09-17T00:11:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 -- start start 2024-09-17T00:11:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8101e00 0x7f4ad80ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:41.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad8102720 0x7f4ad80fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad81004e0 con 0x7f4ad8101e00 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.903+0000 7f4adec18700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad8100620 con 0x7f4ad8102720 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adc9b4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8101e00 0x7f4ad80ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adc9b4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8101e00 0x7f4ad80ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45212/0 (socket says 172.21.15.42:45212) 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adc9b4700 1 -- 172.21.15.42:0/4127758723 learned_addr learned my addr 172.21.15.42:0/4127758723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4ad7fff700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad8102720 0x7f4ad80fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adc9b4700 1 -- 172.21.15.42:0/4127758723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad8102720 msgr2=0x7f4ad80fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adc9b4700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad8102720 0x7f4ad80fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adc9b4700 1 -- 172.21.15.42:0/4127758723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4acc0095f0 con 0x7f4ad8101e00 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adc9b4700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8101e00 0x7f4ad80ffa60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800d900 tx=0x7f4ac800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4ad5ffb700 1 -- 172.21.15.42:0/4127758723 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac8004a00 con 0x7f4ad8101e00 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad81a2aa0 con 0x7f4ad8101e00 2024-09-17T00:11:41.903 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4ad5ffb700 1 -- 172.21.15.42:0/4127758723 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ac80054e0 con 0x7f4ad8101e00 2024-09-17T00:11:41.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.904+0000 7f4ad5ffb700 1 -- 172.21.15.42:0/4127758723 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ac8009d00 con 0x7f4ad8101e00 2024-09-17T00:11:41.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.905+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad81a2e50 con 0x7f4ad8101e00 2024-09-17T00:11:41.904 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.906+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad8108310 con 0x7f4ad8101e00 2024-09-17T00:11:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.906+0000 7f4ad5ffb700 1 -- 172.21.15.42:0/4127758723 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f4ac8005020 con 0x7f4ad8101e00 2024-09-17T00:11:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.907+0000 7f4ad5ffb700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4ac006e680 0x7f4ac0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.907+0000 7f4ad5ffb700 1 -- 172.21.15.42:0/4127758723 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ac808e680 con 0x7f4ad8101e00 2024-09-17T00:11:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.907+0000 7f4ad7fff700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4ac006e680 0x7f4ac0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:41.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.907+0000 7f4ad7fff700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4ac006e680 0x7f4ac0070b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006010 tx=0x7f4acc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:41.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:41.911+0000 7f4ad5ffb700 1 -- 172.21.15.42:0/4127758723 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4ac8057780 con 0x7f4ad8101e00 2024-09-17T00:11:42.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.141+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ad80611d0 con 0x7f4ac006e680 2024-09-17T00:11:42.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.142+0000 7f4ad5ffb700 1 -- 172.21.15.42:0/4127758723 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4ad80611d0 con 0x7f4ac006e680 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4ac006e680 msgr2=0x7f4ac0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4ac006e680 0x7f4ac0070b40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4acc006010 tx=0x7f4acc01a040 comp rx=0 tx=0).stop 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8101e00 msgr2=0x7f4ad80ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8101e00 0x7f4ad80ffa60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800d900 tx=0x7f4ac800dc10 comp rx=0 tx=0).stop 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 shutdown_connections 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4ac006e680 0x7f4ac0070b40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ad8101e00 0x7f4ad80ffa60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 --2- 172.21.15.42:0/4127758723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad8102720 0x7f4ad80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.144+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 >> 172.21.15.42:0/4127758723 conn(0x7f4ad80fb650 msgr2=0x7f4ad8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:42.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.145+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 shutdown_connections 2024-09-17T00:11:42.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.145+0000 7f4adec18700 1 -- 172.21.15.42:0/4127758723 wait complete. 2024-09-17T00:11:42.153 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:11:42.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 -- 172.21.15.42:0/3828551558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 msgr2=0x7fa0fc1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 --2- 172.21.15.42:0/3828551558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc1047a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009910 tx=0x7fa0f0009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 -- 172.21.15.42:0/3828551558 shutdown_connections 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 --2- 172.21.15.42:0/3828551558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc1047a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 --2- 172.21.15.42:0/3828551558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc0ffda0 0x7fa0fc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 -- 172.21.15.42:0/3828551558 >> 172.21.15.42:0/3828551558 conn(0x7fa0fc0fb650 msgr2=0x7fa0fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 -- 172.21.15.42:0/3828551558 shutdown_connections 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 -- 172.21.15.42:0/3828551558 wait complete. 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 Processor -- start 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.254+0000 7fa1038b1700 1 -- start start 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa1038b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc0ffda0 0x7fa0fc196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa1038b1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa1038b1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc197520 con 0x7fa0fc100750 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa1038b1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19b330 con 0x7fa0fc0ffda0 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa100e4c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa100e4c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45232/0 (socket says 172.21.15.42:45232) 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa100e4c700 1 -- 172.21.15.42:0/87658877 learned_addr learned my addr 172.21.15.42:0/87658877 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa10164d700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc0ffda0 0x7fa0fc196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa100e4c700 1 -- 172.21.15.42:0/87658877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc0ffda0 msgr2=0x7fa0fc196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa100e4c700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc0ffda0 0x7fa0fc196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa100e4c700 1 -- 172.21.15.42:0/87658877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00095f0 con 0x7fa0fc100750 2024-09-17T00:11:42.254 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa100e4c700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc196e90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000bd70 tx=0x7fa0f000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:42.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa0ee7fc700 1 -- 172.21.15.42:0/87658877 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f001d070 con 0x7fa0fc100750 2024-09-17T00:11:42.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.255+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc19b5b0 con 0x7fa0fc100750 2024-09-17T00:11:42.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.256+0000 7fa0ee7fc700 1 -- 172.21.15.42:0/87658877 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0f0005dc0 con 0x7fa0fc100750 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.256+0000 7fa0ee7fc700 1 -- 172.21.15.42:0/87658877 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f0017400 con 0x7fa0fc100750 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.256+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc19baa0 con 0x7fa0fc100750 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.257+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc04f2e0 con 0x7fa0fc100750 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.257+0000 7fa0ee7fc700 1 -- 172.21.15.42:0/87658877 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa0f000f460 con 0x7fa0fc100750 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.257+0000 7fa0ee7fc700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa0e806e680 0x7fa0e8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.258+0000 7fa0ee7fc700 1 -- 172.21.15.42:0/87658877 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa0f008fb00 con 0x7fa0fc100750 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.258+0000 7fa10164d700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa0e806e680 0x7fa0e8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.258+0000 7fa10164d700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa0e806e680 0x7fa0e8070b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8005950 tx=0x7fa0f800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:42.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.262+0000 7fa0ee7fc700 1 -- 172.21.15.42:0/87658877 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa0f005c7d0 con 0x7fa0fc100750 2024-09-17T00:11:42.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.492+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0fc19bcc0 con 0x7fa0e806e680 2024-09-17T00:11:42.490 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:42 smithi042 ceph-mon[111056]: pgmap v603: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:42.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.493+0000 7fa0ee7fc700 1 -- 172.21.15.42:0/87658877 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa0fc19bcc0 con 0x7fa0e806e680 2024-09-17T00:11:42.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.495+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa0e806e680 msgr2=0x7fa0e8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.495+0000 7fa1038b1700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa0e806e680 0x7fa0e8070b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8005950 tx=0x7fa0f800b410 comp rx=0 tx=0).stop 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.495+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 msgr2=0x7fa0fc196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.495+0000 7fa1038b1700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc196e90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000bd70 tx=0x7fa0f000bda0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.496+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 shutdown_connections 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.496+0000 7fa1038b1700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa0e806e680 0x7fa0e8070b40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.496+0000 7fa1038b1700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0fc0ffda0 0x7fa0fc196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.496+0000 7fa1038b1700 1 --2- 172.21.15.42:0/87658877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0fc100750 0x7fa0fc196e90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.496+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 >> 172.21.15.42:0/87658877 conn(0x7fa0fc0fb650 msgr2=0x7fa0fc103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.496+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 shutdown_connections 2024-09-17T00:11:42.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.496+0000 7fa1038b1700 1 -- 172.21.15.42:0/87658877 wait complete. 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 -- 172.21.15.42:0/2308163153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 msgr2=0x7faa581058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 --2- 172.21.15.42:0/2308163153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa581058a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009910 tx=0x7faa4c009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 -- 172.21.15.42:0/2308163153 shutdown_connections 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 --2- 172.21.15.42:0/2308163153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa581058a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 --2- 172.21.15.42:0/2308163153 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa58100ea0 0x7faa58101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 -- 172.21.15.42:0/2308163153 >> 172.21.15.42:0/2308163153 conn(0x7faa580fc750 msgr2=0x7faa580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 -- 172.21.15.42:0/2308163153 shutdown_connections 2024-09-17T00:11:42.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.604+0000 7faa5cef8700 1 -- 172.21.15.42:0/2308163153 wait complete. 2024-09-17T00:11:42.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa5cef8700 1 Processor -- start 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa5cef8700 1 -- start start 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa5cef8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa58100ea0 0x7faa58073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa5cef8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa580735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa5cef8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58073b10 con 0x7faa58101850 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa5cef8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58073c50 con 0x7faa58100ea0 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa55d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa580735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa55d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa580735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43780/0 (socket says 172.21.15.42:43780) 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa55d9b700 1 -- 172.21.15.42:0/3283209396 learned_addr learned my addr 172.21.15.42:0/3283209396 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa5659c700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa58100ea0 0x7faa58073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa55d9b700 1 -- 172.21.15.42:0/3283209396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa58100ea0 msgr2=0x7faa58073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa55d9b700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa58100ea0 0x7faa58073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.605+0000 7faa55d9b700 1 -- 172.21.15.42:0/3283209396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4c0095f0 con 0x7faa58101850 2024-09-17T00:11:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.606+0000 7faa55d9b700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa580735d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00bd70 tx=0x7faa4c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.606+0000 7faa477fe700 1 -- 172.21.15.42:0/3283209396 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4c01d070 con 0x7faa58101850 2024-09-17T00:11:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.606+0000 7faa477fe700 1 -- 172.21.15.42:0/3283209396 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa4c005dc0 con 0x7faa58101850 2024-09-17T00:11:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.606+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa58073ed0 con 0x7faa58101850 2024-09-17T00:11:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.606+0000 7faa477fe700 1 -- 172.21.15.42:0/3283209396 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa4c017400 con 0x7faa58101850 2024-09-17T00:11:42.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.606+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa581a2db0 con 0x7faa58101850 2024-09-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.607+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa581091b0 con 0x7faa58101850 2024-09-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.607+0000 7faa477fe700 1 -- 172.21.15.42:0/3283209396 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7faa4c00f460 con 0x7faa58101850 2024-09-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.608+0000 7faa477fe700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faa4006e680 0x7faa40070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.608+0000 7faa477fe700 1 -- 172.21.15.42:0/3283209396 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faa4c08fab0 con 0x7faa58101850 2024-09-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.608+0000 7faa5659c700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faa4006e680 0x7faa40070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.608+0000 7faa5659c700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faa4006e680 0x7faa40070b40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faa48004630 tx=0x7faa4800b3c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:42.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.612+0000 7faa477fe700 1 -- 172.21.15.42:0/3283209396 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faa4c058c50 con 0x7faa58101850 2024-09-17T00:11:42.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:42 smithi043 ceph-mon[81510]: pgmap v603: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:42.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.831+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faa580611d0 con 0x7faa4006e680 2024-09-17T00:11:42.838 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.841+0000 7faa477fe700 1 -- 172.21.15.42:0/3283209396 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7faa580611d0 con 0x7faa4006e680 2024-09-17T00:11:42.838 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:11:42.838 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (20m) 13m ago 31m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:11:42.838 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (31m) 13m ago 31m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:11:42.838 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (30m) 13m ago 30m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (17m) 13m ago 31m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (17m) 13m ago 30m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (19m) 13m ago 30m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (14m) 13m ago 25m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (14m) 13m ago 25m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (13m) 13m ago 25m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (13m) 13m ago 25m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (22m) 13m ago 32m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (21m) 13m ago 30m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (18m) 13m ago 32m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (18m) 13m ago 29m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (21m) 13m ago 31m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:11:42.839 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (20m) 13m ago 30m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:11:42.840 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (17m) 13m ago 29m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:11:42.840 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (16m) 13m ago 28m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:11:42.840 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (16m) 13m ago 28m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:11:42.840 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (15m) 13m ago 27m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:11:42.840 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (15m) 13m ago 27m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:11:42.840 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (14m) 13m ago 26m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:11:42.840 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (20m) 13m ago 30m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:11:42.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faa4006e680 msgr2=0x7faa40070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faa4006e680 0x7faa40070b40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7faa48004630 tx=0x7faa4800b3c0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 msgr2=0x7faa580735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa580735d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00bd70 tx=0x7faa4c00bda0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 shutdown_connections 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7faa4006e680 0x7faa40070b40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa58100ea0 0x7faa58073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 --2- 172.21.15.42:0/3283209396 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa58101850 0x7faa580735d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 >> 172.21.15.42:0/3283209396 conn(0x7faa580fc750 msgr2=0x7faa58105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 shutdown_connections 2024-09-17T00:11:42.842 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.844+0000 7faa5cef8700 1 -- 172.21.15.42:0/3283209396 wait complete. 2024-09-17T00:11:42.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.951+0000 7fd32ba6f700 1 -- 172.21.15.42:0/3702455323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 msgr2=0x7fd3241047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.951+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/3702455323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd3241047a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd314009910 tx=0x7fd314009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.951+0000 7fd32ba6f700 1 -- 172.21.15.42:0/3702455323 shutdown_connections 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.951+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/3702455323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd3241047a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.951+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/3702455323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3240ffda0 0x7fd324100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.951+0000 7fd32ba6f700 1 -- 172.21.15.42:0/3702455323 >> 172.21.15.42:0/3702455323 conn(0x7fd3240fb650 msgr2=0x7fd3240fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 -- 172.21.15.42:0/3702455323 shutdown_connections 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 -- 172.21.15.42:0/3702455323 wait complete. 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 Processor -- start 2024-09-17T00:11:42.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 -- start start 2024-09-17T00:11:42.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3240ffda0 0x7fd32419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd32419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32419d240 con 0x7fd324100750 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.952+0000 7fd32ba6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3241966f0 con 0x7fd3240ffda0 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32900a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd32419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32900a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd32419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43800/0 (socket says 172.21.15.42:43800) 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32900a700 1 -- 172.21.15.42:0/1467770352 learned_addr learned my addr 172.21.15.42:0/1467770352 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32980b700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3240ffda0 0x7fd32419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32900a700 1 -- 172.21.15.42:0/1467770352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3240ffda0 msgr2=0x7fd32419c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32900a700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3240ffda0 0x7fd32419c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32900a700 1 -- 172.21.15.42:0/1467770352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3140095f0 con 0x7fd324100750 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32900a700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd32419cbb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd314009fd0 tx=0x7fd31400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd31affd700 1 -- 172.21.15.42:0/1467770352 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd31401d070 con 0x7fd324100750 2024-09-17T00:11:42.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd31affd700 1 -- 172.21.15.42:0/1467770352 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd314022470 con 0x7fd324100750 2024-09-17T00:11:42.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd324196970 con 0x7fd324100750 2024-09-17T00:11:42.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd31affd700 1 -- 172.21.15.42:0/1467770352 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd314017650 con 0x7fd324100750 2024-09-17T00:11:42.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.953+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd324196e00 con 0x7fd324100750 2024-09-17T00:11:42.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.954+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3241091b0 con 0x7fd324100750 2024-09-17T00:11:42.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.955+0000 7fd31affd700 1 -- 172.21.15.42:0/1467770352 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd314005080 con 0x7fd324100750 2024-09-17T00:11:42.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.955+0000 7fd31affd700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd31006e680 0x7fd310070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:42.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.955+0000 7fd31affd700 1 -- 172.21.15.42:0/1467770352 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd31408fbb0 con 0x7fd324100750 2024-09-17T00:11:42.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.955+0000 7fd32980b700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd31006e680 0x7fd310070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:42.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.956+0000 7fd32980b700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd31006e680 0x7fd310070b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd320009c60 tx=0x7fd320009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:42.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:42.960+0000 7fd31affd700 1 -- 172.21.15.42:0/1467770352 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd31405c860 con 0x7fd324100750 2024-09-17T00:11:43.255 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.258+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd32404f2e0 con 0x7fd324100750 2024-09-17T00:11:43.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.258+0000 7fd31affd700 1 -- 172.21.15.42:0/1467770352 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fd31405c3f0 con 0x7fd324100750 2024-09-17T00:11:43.257 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:11:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:11:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:11:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:11:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:11:43.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:11:43.258 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:11:43.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd31006e680 msgr2=0x7fd310070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd31006e680 0x7fd310070b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd320009c60 tx=0x7fd320009260 comp rx=0 tx=0).stop 2024-09-17T00:11:43.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 msgr2=0x7fd32419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd32419cbb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd314009fd0 tx=0x7fd31400f690 comp rx=0 tx=0).stop 2024-09-17T00:11:43.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 shutdown_connections 2024-09-17T00:11:43.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd31006e680 0x7fd310070b40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3240ffda0 0x7fd32419c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 --2- 172.21.15.42:0/1467770352 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd324100750 0x7fd32419cbb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 >> 172.21.15.42:0/1467770352 conn(0x7fd3240fb650 msgr2=0x7fd324069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:43.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 shutdown_connections 2024-09-17T00:11:43.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.261+0000 7fd32ba6f700 1 -- 172.21.15.42:0/1467770352 wait complete. 2024-09-17T00:11:43.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 -- 172.21.15.42:0/1751078696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 msgr2=0x7ff1dc103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 --2- 172.21.15.42:0/1751078696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc103f10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009910 tx=0x7ff1d0009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:43.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 -- 172.21.15.42:0/1751078696 shutdown_connections 2024-09-17T00:11:43.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 --2- 172.21.15.42:0/1751078696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc103f10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 --2- 172.21.15.42:0/1751078696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1dc0ff670 0x7ff1dc0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 -- 172.21.15.42:0/1751078696 >> 172.21.15.42:0/1751078696 conn(0x7ff1dc074e30 msgr2=0x7ff1dc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:43.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 -- 172.21.15.42:0/1751078696 shutdown_connections 2024-09-17T00:11:43.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 -- 172.21.15.42:0/1751078696 wait complete. 2024-09-17T00:11:43.368 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 Processor -- start 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.369+0000 7ff1e2481700 1 -- start start 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1e2481700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1dc0ff670 0x7ff1dc19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1e2481700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1e2481700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc19f480 con 0x7ff1dc100020 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1e2481700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc198930 con 0x7ff1dc0ff670 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1db7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43820/0 (socket says 172.21.15.42:43820) 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1db7fe700 1 -- 172.21.15.42:0/2406456423 learned_addr learned my addr 172.21.15.42:0/2406456423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1dbfff700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1dc0ff670 0x7ff1dc19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1db7fe700 1 -- 172.21.15.42:0/2406456423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1dc0ff670 msgr2=0x7ff1dc19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1db7fe700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1dc0ff670 0x7ff1dc19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1db7fe700 1 -- 172.21.15.42:0/2406456423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d00095f0 con 0x7ff1dc100020 2024-09-17T00:11:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.370+0000 7ff1db7fe700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc19edf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005b40 tx=0x7ff1d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.371+0000 7ff1d97fa700 1 -- 172.21.15.42:0/2406456423 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d001d070 con 0x7ff1dc100020 2024-09-17T00:11:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.371+0000 7ff1d97fa700 1 -- 172.21.15.42:0/2406456423 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1d000bed0 con 0x7ff1dc100020 2024-09-17T00:11:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.371+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc198bb0 con 0x7ff1dc100020 2024-09-17T00:11:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.371+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc1990a0 con 0x7ff1dc100020 2024-09-17T00:11:43.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.371+0000 7ff1d97fa700 1 -- 172.21.15.42:0/2406456423 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1d0003e70 con 0x7ff1dc100020 2024-09-17T00:11:43.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.372+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc107910 con 0x7ff1dc100020 2024-09-17T00:11:43.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.372+0000 7ff1d97fa700 1 -- 172.21.15.42:0/2406456423 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff1d0017500 con 0x7ff1dc100020 2024-09-17T00:11:43.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.373+0000 7ff1d97fa700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff1c406e680 0x7ff1c4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:43.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.373+0000 7ff1dbfff700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff1c406e680 0x7ff1c4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:43.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.373+0000 7ff1dbfff700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff1c406e680 0x7ff1c4070b40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00acd0 tx=0x7ff1cc010040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:43.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.374+0000 7ff1d97fa700 1 -- 172.21.15.42:0/2406456423 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff1d005cfe0 con 0x7ff1dc100020 2024-09-17T00:11:43.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.378+0000 7ff1d97fa700 1 -- 172.21.15.42:0/2406456423 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff1d0091030 con 0x7ff1dc100020 2024-09-17T00:11:43.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:43 smithi042 ceph-mon[111056]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:43.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:43 smithi042 ceph-mon[111056]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:43.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:43 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1467770352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:11:43.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:43 smithi043 ceph-mon[81510]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:43.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:43 smithi043 ceph-mon[81510]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:43.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:43 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1467770352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:11:43.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.635+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff1dc199e40 con 0x7ff1dc100020 2024-09-17T00:11:43.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.636+0000 7ff1d97fa700 1 -- 172.21.15.42:0/2406456423 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7ff1d005c6b0 con 0x7ff1dc100020 2024-09-17T00:11:43.633 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:11:43.633 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:11:43.633 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:11:43.633 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:11:43.634 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:11:43.634 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:43.634 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:11:43.634 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:11:43.634 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:11:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:11:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:11:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:11:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:11:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:11:43.635 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:11:43.636 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:43.637 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:43.638 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:11:43.638 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:11:43.638 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:43.638 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:43.638 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:43.638 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:11:43.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff1c406e680 msgr2=0x7ff1c4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff1c406e680 0x7ff1c4070b40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00acd0 tx=0x7ff1cc010040 comp rx=0 tx=0).stop 2024-09-17T00:11:43.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 msgr2=0x7ff1dc19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc19edf0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005b40 tx=0x7ff1d000f690 comp rx=0 tx=0).stop 2024-09-17T00:11:43.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 shutdown_connections 2024-09-17T00:11:43.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff1c406e680 0x7ff1c4070b40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1dc0ff670 0x7ff1dc19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 --2- 172.21.15.42:0/2406456423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1dc100020 0x7ff1dc19edf0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 >> 172.21.15.42:0/2406456423 conn(0x7ff1dc074e30 msgr2=0x7ff1dc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 shutdown_connections 2024-09-17T00:11:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.638+0000 7ff1e2481700 1 -- 172.21.15.42:0/2406456423 wait complete. 2024-09-17T00:11:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:11:43.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.746+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2120898856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0101850 msgr2=0x7f8ea01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.746+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2120898856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0101850 0x7f8ea01058a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009910 tx=0x7f8e90009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2120898856 shutdown_connections 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2120898856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0101850 0x7f8ea01058a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2120898856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2120898856 >> 172.21.15.42:0/2120898856 conn(0x7f8ea00fc750 msgr2=0x7f8ea00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2120898856 shutdown_connections 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2120898856 wait complete. 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 Processor -- start 2024-09-17T00:11:43.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.747+0000 7f8ea77d6700 1 -- start start 2024-09-17T00:11:43.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea77d6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea77d6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ea0101850 0x7f8ea00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea77d6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0073b10 con 0x7f8ea0100ea0 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea77d6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0073c50 con 0x7f8ea0101850 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea5572700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea5572700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43844/0 (socket says 172.21.15.42:43844) 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea5572700 1 -- 172.21.15.42:0/2499974659 learned_addr learned my addr 172.21.15.42:0/2499974659 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea4d71700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ea0101850 0x7f8ea00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea5572700 1 -- 172.21.15.42:0/2499974659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ea0101850 msgr2=0x7f8ea00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea5572700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ea0101850 0x7f8ea00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea5572700 1 -- 172.21.15.42:0/2499974659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e900095f0 con 0x7f8ea0100ea0 2024-09-17T00:11:43.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea5572700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0073090 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00c8a0 tx=0x7f8e9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8e967fc700 1 -- 172.21.15.42:0/2499974659 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c00cea0 con 0x7f8ea0100ea0 2024-09-17T00:11:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8e967fc700 1 -- 172.21.15.42:0/2499974659 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e9c004830 con 0x7f8ea0100ea0 2024-09-17T00:11:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea01a2b00 con 0x7f8ea0100ea0 2024-09-17T00:11:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8e967fc700 1 -- 172.21.15.42:0/2499974659 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e9c0055b0 con 0x7f8ea0100ea0 2024-09-17T00:11:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.748+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea01a2f10 con 0x7f8ea0100ea0 2024-09-17T00:11:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.749+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea01092a0 con 0x7f8ea0100ea0 2024-09-17T00:11:43.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.750+0000 7f8e967fc700 1 -- 172.21.15.42:0/2499974659 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8e9c0049a0 con 0x7f8ea0100ea0 2024-09-17T00:11:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.750+0000 7f8e967fc700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8e8c06e680 0x7f8e8c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.750+0000 7f8e967fc700 1 -- 172.21.15.42:0/2499974659 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8e9c08e340 con 0x7f8ea0100ea0 2024-09-17T00:11:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.750+0000 7f8ea4d71700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8e8c06e680 0x7f8e8c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:43.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.751+0000 7f8ea4d71700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8e8c06e680 0x7f8e8c070b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0074880 tx=0x7f8e9001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:43.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.755+0000 7f8e967fc700 1 -- 172.21.15.42:0/2499974659 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8e9c0574c0 con 0x7f8ea0100ea0 2024-09-17T00:11:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.987+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ea00611d0 con 0x7f8e8c06e680 2024-09-17T00:11:43.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.988+0000 7f8e967fc700 1 -- 172.21.15.42:0/2499974659 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8ea00611d0 con 0x7f8e8c06e680 2024-09-17T00:11:43.986 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:11:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:11:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:11:43.986 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:11:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:11:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:11:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:11:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:11:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:11:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:11:43.987 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:11:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:11:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:11:43.988 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:11:43.988 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:11:43.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8e8c06e680 msgr2=0x7f8e8c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8e8c06e680 0x7f8e8c070b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0074880 tx=0x7f8e9001a040 comp rx=0 tx=0).stop 2024-09-17T00:11:43.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0100ea0 msgr2=0x7f8ea0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:43.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0073090 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00c8a0 tx=0x7f8e9c00cc60 comp rx=0 tx=0).stop 2024-09-17T00:11:43.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 shutdown_connections 2024-09-17T00:11:43.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8e8c06e680 0x7f8e8c070b40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ea0100ea0 0x7f8ea0073090 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 --2- 172.21.15.42:0/2499974659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ea0101850 0x7f8ea00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:43.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.990+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 >> 172.21.15.42:0/2499974659 conn(0x7f8ea00fc750 msgr2=0x7f8ea00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:43.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.991+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 shutdown_connections 2024-09-17T00:11:43.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:43.991+0000 7f8ea77d6700 1 -- 172.21.15.42:0/2499974659 wait complete. 2024-09-17T00:11:44.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.098+0000 7fd446136700 1 -- 172.21.15.42:0/3657814028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 msgr2=0x7fd440100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:44.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.098+0000 7fd446136700 1 --2- 172.21.15.42:0/3657814028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd440100180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd430009910 tx=0x7fd430009c20 comp rx=0 tx=0).stop 2024-09-17T00:11:44.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.099+0000 7fd446136700 1 -- 172.21.15.42:0/3657814028 shutdown_connections 2024-09-17T00:11:44.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.099+0000 7fd446136700 1 --2- 172.21.15.42:0/3657814028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd440100750 0x7fd4401047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:44.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.099+0000 7fd446136700 1 --2- 172.21.15.42:0/3657814028 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd440100180 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:44.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.099+0000 7fd446136700 1 -- 172.21.15.42:0/3657814028 >> 172.21.15.42:0/3657814028 conn(0x7fd4400fb650 msgr2=0x7fd4400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:44.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.099+0000 7fd446136700 1 -- 172.21.15.42:0/3657814028 shutdown_connections 2024-09-17T00:11:44.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.099+0000 7fd446136700 1 -- 172.21.15.42:0/3657814028 wait complete. 2024-09-17T00:11:44.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.099+0000 7fd446136700 1 Processor -- start 2024-09-17T00:11:44.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd446136700 1 -- start start 2024-09-17T00:11:44.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd446136700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd44019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:44.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd446136700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd440100750 0x7fd44019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:44.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd446136700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44019c120 con 0x7fd4400ffda0 2024-09-17T00:11:44.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd446136700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4401955d0 con 0x7fd440100750 2024-09-17T00:11:44.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd43f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd44019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:44.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd43f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd44019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43866/0 (socket says 172.21.15.42:43866) 2024-09-17T00:11:44.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd43f7fe700 1 -- 172.21.15.42:0/1766031410 learned_addr learned my addr 172.21.15.42:0/1766031410 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:11:44.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd43effd700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd440100750 0x7fd44019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:44.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd43f7fe700 1 -- 172.21.15.42:0/1766031410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd440100750 msgr2=0x7fd44019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:44.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd43f7fe700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd440100750 0x7fd44019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:44.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.100+0000 7fd43f7fe700 1 -- 172.21.15.42:0/1766031410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4300095f0 con 0x7fd4400ffda0 2024-09-17T00:11:44.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.101+0000 7fd43f7fe700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd44019b550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd430006010 tx=0x7fd43000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:44.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.101+0000 7fd43cff9700 1 -- 172.21.15.42:0/1766031410 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd43001d070 con 0x7fd4400ffda0 2024-09-17T00:11:44.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.101+0000 7fd43cff9700 1 -- 172.21.15.42:0/1766031410 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd430005dc0 con 0x7fd4400ffda0 2024-09-17T00:11:44.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.101+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd440195850 con 0x7fd4400ffda0 2024-09-17T00:11:44.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.101+0000 7fd43cff9700 1 -- 172.21.15.42:0/1766031410 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd430017400 con 0x7fd4400ffda0 2024-09-17T00:11:44.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.101+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd440195d40 con 0x7fd4400ffda0 2024-09-17T00:11:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.102+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd440108140 con 0x7fd4400ffda0 2024-09-17T00:11:44.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.102+0000 7fd43cff9700 1 -- 172.21.15.42:0/1766031410 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd430003a60 con 0x7fd4400ffda0 2024-09-17T00:11:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.103+0000 7fd43cff9700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd42806e680 0x7fd428070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:11:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.103+0000 7fd43cff9700 1 -- 172.21.15.42:0/1766031410 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd43008fa30 con 0x7fd4400ffda0 2024-09-17T00:11:44.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.103+0000 7fd43effd700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd42806e680 0x7fd428070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:11:44.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.103+0000 7fd43effd700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd42806e680 0x7fd428070b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd440104a20 tx=0x7fd43400b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:11:44.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.107+0000 7fd43cff9700 1 -- 172.21.15.42:0/1766031410 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd43005c780 con 0x7fd4400ffda0 2024-09-17T00:11:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.401+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd44004f2e0 con 0x7fd4400ffda0 2024-09-17T00:11:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.402+0000 7fd43cff9700 1 -- 172.21.15.42:0/1766031410 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd43005c310 con 0x7fd4400ffda0 2024-09-17T00:11:44.399 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:11:44.399 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:11:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:11:44.400 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:11:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:11:44.400 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:11:44.400 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:11:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd42806e680 msgr2=0x7fd428070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd42806e680 0x7fd428070b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd440104a20 tx=0x7fd43400b410 comp rx=0 tx=0).stop 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 msgr2=0x7fd44019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd44019b550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd430006010 tx=0x7fd43000bd70 comp rx=0 tx=0).stop 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 shutdown_connections 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd42806e680 0x7fd428070b40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4400ffda0 0x7fd44019b550 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 --2- 172.21.15.42:0/1766031410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd440100750 0x7fd44019ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 >> 172.21.15.42:0/1766031410 conn(0x7fd4400fb650 msgr2=0x7fd4400fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 shutdown_connections 2024-09-17T00:11:44.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:11:44.404+0000 7fd446136700 1 -- 172.21.15.42:0/1766031410 wait complete. 2024-09-17T00:11:44.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:44 smithi042 ceph-mon[111056]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:44 smithi042 ceph-mon[111056]: pgmap v604: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:44 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2406456423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:11:44.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:44 smithi043 ceph-mon[81510]: from='client.35140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:44.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:44 smithi043 ceph-mon[81510]: pgmap v604: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:44.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:44 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2406456423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:11:45.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:45 smithi042 ceph-mon[111056]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:45.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:45 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1766031410' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:11:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:45 smithi043 ceph-mon[81510]: from='client.35152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:11:45.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:45 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1766031410' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:11:46.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:46 smithi042 ceph-mon[111056]: pgmap v605: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:46.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:46 smithi043 ceph-mon[81510]: pgmap v605: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:48.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:48 smithi042 ceph-mon[111056]: pgmap v606: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:48 smithi043 ceph-mon[81510]: pgmap v606: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:50 smithi042 ceph-mon[111056]: pgmap v607: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:50.880 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:50 smithi043 ceph-mon[81510]: pgmap v607: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:11:52.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:52 smithi042 ceph-mon[111056]: pgmap v608: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:52 smithi043 ceph-mon[81510]: pgmap v608: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:54.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:54 smithi042 ceph-mon[111056]: pgmap v609: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:54.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:54 smithi043 ceph-mon[81510]: pgmap v609: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:56.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:56 smithi042 ceph-mon[111056]: pgmap v610: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:56.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:56 smithi043 ceph-mon[81510]: pgmap v610: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:58.708 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:11:58 smithi043 ceph-mon[81510]: pgmap v611: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:11:58.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:11:58 smithi042 ceph-mon[111056]: pgmap v611: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:00.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:00 smithi042 ceph-mon[111056]: pgmap v612: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:00.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:00 smithi043 ceph-mon[81510]: pgmap v612: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:02.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:02 smithi042 ceph-mon[111056]: pgmap v613: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:02.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:02 smithi043 ceph-mon[81510]: pgmap v613: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:04.728 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:04 smithi042 ceph-mon[111056]: pgmap v614: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:04.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:04 smithi043 ceph-mon[81510]: pgmap v614: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:06 smithi042 ceph-mon[111056]: pgmap v615: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:06.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:06 smithi043 ceph-mon[81510]: pgmap v615: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:08.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:08 smithi042 ceph-mon[111056]: pgmap v616: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:08.812 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:08 smithi043 ceph-mon[81510]: pgmap v616: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:10.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:10 smithi042 ceph-mon[111056]: pgmap v617: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:10.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:10 smithi043 ceph-mon[81510]: pgmap v617: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:12.680 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:12 smithi043 ceph-mon[81510]: pgmap v618: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:12.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:12 smithi042 ceph-mon[111056]: pgmap v618: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1976587410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88101850 msgr2=0x7f9e881058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1976587410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88101850 0x7f9e881058a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009910 tx=0x7f9e70009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1976587410 shutdown_connections 2024-09-17T00:12:14.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1976587410 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88101850 0x7f9e881058a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1976587410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88100ea0 0x7f9e88101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1976587410 >> 172.21.15.42:0/1976587410 conn(0x7f9e880fc750 msgr2=0x7f9e880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1976587410 shutdown_connections 2024-09-17T00:12:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.528+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1976587410 wait complete. 2024-09-17T00:12:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e8dd34700 1 Processor -- start 2024-09-17T00:12:14.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e8dd34700 1 -- start start 2024-09-17T00:12:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e8dd34700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88100ea0 0x7f9e8819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e8dd34700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88101850 0x7f9e8819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e8dd34700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8819d2d0 con 0x7f9e88100ea0 2024-09-17T00:12:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e8dd34700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88196780 con 0x7f9e88101850 2024-09-17T00:12:14.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88100ea0 0x7f9e8819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88100ea0 0x7f9e8819c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40914/0 (socket says 172.21.15.42:40914) 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e877fe700 1 -- 172.21.15.42:0/1101402018 learned_addr learned my addr 172.21.15.42:0/1101402018 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.529+0000 7f9e7effd700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88101850 0x7f9e8819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e877fe700 1 -- 172.21.15.42:0/1101402018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88101850 msgr2=0x7f9e8819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e877fe700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88101850 0x7f9e8819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e877fe700 1 -- 172.21.15.42:0/1101402018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700095f0 con 0x7f9e88100ea0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e877fe700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88100ea0 0x7f9e8819c700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800d900 tx=0x7f9e7800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e857fa700 1 -- 172.21.15.42:0/1101402018 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e780041f0 con 0x7f9e88100ea0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e857fa700 1 -- 172.21.15.42:0/1101402018 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e78004cf0 con 0x7f9e88100ea0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e857fa700 1 -- 172.21.15.42:0/1101402018 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e7800b750 con 0x7f9e88100ea0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e88196a60 con 0x7f9e88100ea0 2024-09-17T00:12:14.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.530+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e88196f80 con 0x7f9e88100ea0 2024-09-17T00:12:14.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.531+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e881092a0 con 0x7f9e88100ea0 2024-09-17T00:12:14.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.532+0000 7f9e857fa700 1 -- 172.21.15.42:0/1101402018 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9e78004750 con 0x7f9e88100ea0 2024-09-17T00:12:14.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.532+0000 7f9e857fa700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e7406e680 0x7f9e74070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:14.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.532+0000 7f9e857fa700 1 -- 172.21.15.42:0/1101402018 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9e7808e490 con 0x7f9e88100ea0 2024-09-17T00:12:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.532+0000 7f9e7effd700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e7406e680 0x7f9e74070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:14.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.533+0000 7f9e7effd700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e7406e680 0x7f9e74070b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006010 tx=0x7f9e7001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:14.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.537+0000 7f9e857fa700 1 -- 172.21.15.42:0/1101402018 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9e7805b1e0 con 0x7f9e88100ea0 2024-09-17T00:12:14.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.767+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e88197da0 con 0x7f9e7406e680 2024-09-17T00:12:14.766 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:14 smithi042 ceph-mon[111056]: pgmap v619: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:14.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.768+0000 7f9e857fa700 1 -- 172.21.15.42:0/1101402018 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9e88197da0 con 0x7f9e7406e680 2024-09-17T00:12:14.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.770+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e7406e680 msgr2=0x7f9e74070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:14.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.770+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e7406e680 0x7f9e74070b40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9e70006010 tx=0x7f9e7001a040 comp rx=0 tx=0).stop 2024-09-17T00:12:14.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88100ea0 msgr2=0x7f9e8819c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:14.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88100ea0 0x7f9e8819c700 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800d900 tx=0x7f9e7800dc10 comp rx=0 tx=0).stop 2024-09-17T00:12:14.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 shutdown_connections 2024-09-17T00:12:14.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9e7406e680 0x7f9e74070b40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e88100ea0 0x7f9e8819c700 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 --2- 172.21.15.42:0/1101402018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e88101850 0x7f9e8819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 >> 172.21.15.42:0/1101402018 conn(0x7f9e880fc750 msgr2=0x7f9e880fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:14.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 shutdown_connections 2024-09-17T00:12:14.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.771+0000 7f9e8dd34700 1 -- 172.21.15.42:0/1101402018 wait complete. 2024-09-17T00:12:14.780 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:12:14.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:14 smithi043 ceph-mon[81510]: pgmap v619: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:14.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 -- 172.21.15.42:0/2636391881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 msgr2=0x7fdd9c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 --2- 172.21.15.42:0/2636391881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 0x7fdd9c101280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00b0a0 tx=0x7fdd8c00b3b0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 -- 172.21.15.42:0/2636391881 shutdown_connections 2024-09-17T00:12:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 --2- 172.21.15.42:0/2636391881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 0x7fdd9c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 --2- 172.21.15.42:0/2636391881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 0x7fdd9c101280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 -- 172.21.15.42:0/2636391881 >> 172.21.15.42:0/2636391881 conn(0x7fdd9c0fc750 msgr2=0x7fdd9c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 -- 172.21.15.42:0/2636391881 shutdown_connections 2024-09-17T00:12:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.881+0000 7fdda1352700 1 -- 172.21.15.42:0/2636391881 wait complete. 2024-09-17T00:12:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdda1352700 1 Processor -- start 2024-09-17T00:12:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdda1352700 1 -- start start 2024-09-17T00:12:14.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdda1352700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 0x7fdd9c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:14.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdda1352700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 0x7fdd9c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:14.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdda1352700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c073b10 con 0x7fdd9c101850 2024-09-17T00:12:14.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdda1352700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c073c50 con 0x7fdd9c100ea0 2024-09-17T00:12:14.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdd9a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 0x7fdd9c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:14.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdd9a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 0x7fdd9c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40930/0 (socket says 172.21.15.42:40930) 2024-09-17T00:12:14.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdd9a7fc700 1 -- 172.21.15.42:0/256240925 learned_addr learned my addr 172.21.15.42:0/256240925 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:14.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.882+0000 7fdd9affd700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 0x7fdd9c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:14.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd9a7fc700 1 -- 172.21.15.42:0/256240925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 msgr2=0x7fdd9c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:14.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd9a7fc700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 0x7fdd9c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:14.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd9a7fc700 1 -- 172.21.15.42:0/256240925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c009d00 con 0x7fdd9c101850 2024-09-17T00:12:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd9affd700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 0x7fdd9c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T00:12:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd9a7fc700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 0x7fdd9c0735d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e8f0 tx=0x7fdd8400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd93fff700 1 -- 172.21.15.42:0/256240925 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd84009790 con 0x7fdd9c101850 2024-09-17T00:12:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd93fff700 1 -- 172.21.15.42:0/256240925 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd8400c810 con 0x7fdd9c101850 2024-09-17T00:12:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdd93fff700 1 -- 172.21.15.42:0/256240925 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd84005580 con 0x7fdd9c101850 2024-09-17T00:12:14.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9c1a2ab0 con 0x7fdd9c101850 2024-09-17T00:12:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.883+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c1a2ec0 con 0x7fdd9c101850 2024-09-17T00:12:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.884+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c1091d0 con 0x7fdd9c101850 2024-09-17T00:12:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.885+0000 7fdd93fff700 1 -- 172.21.15.42:0/256240925 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fdd8400c980 con 0x7fdd9c101850 2024-09-17T00:12:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.885+0000 7fdd93fff700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd8806e680 0x7fdd88070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:14.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.885+0000 7fdd93fff700 1 -- 172.21.15.42:0/256240925 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdd84014070 con 0x7fdd9c101850 2024-09-17T00:12:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.885+0000 7fdd9affd700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd8806e680 0x7fdd88070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.886+0000 7fdd9affd700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd8806e680 0x7fdd88070b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000f80 tx=0x7fdd8c0060c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:14.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:14.890+0000 7fdd93fff700 1 -- 172.21.15.42:0/256240925 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdd84057a60 con 0x7fdd9c101850 2024-09-17T00:12:15.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.123+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd9c0611d0 con 0x7fdd8806e680 2024-09-17T00:12:15.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.125+0000 7fdd93fff700 1 -- 172.21.15.42:0/256240925 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdd9c0611d0 con 0x7fdd8806e680 2024-09-17T00:12:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd8806e680 msgr2=0x7fdd88070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd8806e680 0x7fdd88070b40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000f80 tx=0x7fdd8c0060c0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 msgr2=0x7fdd9c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 0x7fdd9c0735d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e8f0 tx=0x7fdd8400ec00 comp rx=0 tx=0).stop 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 shutdown_connections 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fdd8806e680 0x7fdd88070b40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c100ea0 0x7fdd9c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 --2- 172.21.15.42:0/256240925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9c101850 0x7fdd9c0735d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 >> 172.21.15.42:0/256240925 conn(0x7fdd9c0fc750 msgr2=0x7fdd9c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 shutdown_connections 2024-09-17T00:12:15.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.127+0000 7fdda1352700 1 -- 172.21.15.42:0/256240925 wait complete. 2024-09-17T00:12:15.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 -- 172.21.15.42:0/3923442274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 msgr2=0x7f57fc100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 --2- 172.21.15.42:0/3923442274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc100180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009910 tx=0x7f57f0009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:15.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 -- 172.21.15.42:0/3923442274 shutdown_connections 2024-09-17T00:12:15.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 --2- 172.21.15.42:0/3923442274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57fc100750 0x7f57fc1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 --2- 172.21.15.42:0/3923442274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc100180 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 -- 172.21.15.42:0/3923442274 >> 172.21.15.42:0/3923442274 conn(0x7f57fc0fb650 msgr2=0x7f57fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:15.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 -- 172.21.15.42:0/3923442274 shutdown_connections 2024-09-17T00:12:15.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.235+0000 7f5802ad5700 1 -- 172.21.15.42:0/3923442274 wait complete. 2024-09-17T00:12:15.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5802ad5700 1 Processor -- start 2024-09-17T00:12:15.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5802ad5700 1 -- start start 2024-09-17T00:12:15.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5802ad5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:15.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5802ad5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57fc100750 0x7f57fc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:15.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5802ad5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19d210 con 0x7f57fc0ffda0 2024-09-17T00:12:15.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5802ad5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1966c0 con 0x7f57fc100750 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5800871700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5800871700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40942/0 (socket says 172.21.15.42:40942) 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.236+0000 7f5800871700 1 -- 172.21.15.42:0/1104968949 learned_addr learned my addr 172.21.15.42:0/1104968949 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f57fbfff700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57fc100750 0x7f57fc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f5800871700 1 -- 172.21.15.42:0/1104968949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57fc100750 msgr2=0x7f57fc19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f5800871700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57fc100750 0x7f57fc19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f5800871700 1 -- 172.21.15.42:0/1104968949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00095f0 con 0x7f57fc0ffda0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f5800871700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc19c640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f57f0006010 tx=0x7f57f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f57f9ffb700 1 -- 172.21.15.42:0/1104968949 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f001d070 con 0x7f57fc0ffda0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f57f9ffb700 1 -- 172.21.15.42:0/1104968949 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57f0004fb0 con 0x7f57fc0ffda0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc196940 con 0x7f57fc0ffda0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f57f9ffb700 1 -- 172.21.15.42:0/1104968949 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57f0022ea0 con 0x7f57fc0ffda0 2024-09-17T00:12:15.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.237+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc196e30 con 0x7f57fc0ffda0 2024-09-17T00:12:15.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.238+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc04f2e0 con 0x7f57fc0ffda0 2024-09-17T00:12:15.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.239+0000 7f57f9ffb700 1 -- 172.21.15.42:0/1104968949 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f57f0022470 con 0x7f57fc0ffda0 2024-09-17T00:12:15.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.239+0000 7f57f9ffb700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f57ec06e680 0x7f57ec070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:15.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.239+0000 7f57f9ffb700 1 -- 172.21.15.42:0/1104968949 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f57f008fc90 con 0x7f57fc0ffda0 2024-09-17T00:12:15.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.240+0000 7f57fbfff700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f57ec06e680 0x7f57ec070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:15.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.240+0000 7f57fbfff700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f57ec06e680 0x7f57ec070b40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005fd0 tx=0x7f57e800a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:15.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.244+0000 7f57f9ffb700 1 -- 172.21.15.42:0/1104968949 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f57f005c970 con 0x7f57fc0ffda0 2024-09-17T00:12:15.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.464+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f57fc197110 con 0x7f57ec06e680 2024-09-17T00:12:15.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.474+0000 7f57f9ffb700 1 -- 172.21.15.42:0/1104968949 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f57fc197110 con 0x7f57ec06e680 2024-09-17T00:12:15.472 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:12:15.472 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (20m) 14m ago 31m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:12:15.472 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (31m) 14m ago 31m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:12:15.472 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (30m) 14m ago 30m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:12:15.472 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (18m) 14m ago 31m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (18m) 14m ago 30m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (20m) 14m ago 31m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (14m) 14m ago 25m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (14m) 14m ago 26m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (14m) 14m ago 26m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (14m) 14m ago 25m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (22m) 14m ago 33m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (21m) 14m ago 30m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (19m) 14m ago 33m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (19m) 14m ago 30m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (21m) 14m ago 31m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (21m) 14m ago 30m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:12:15.473 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (18m) 14m ago 29m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:12:15.474 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (17m) 14m ago 29m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:12:15.474 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (16m) 14m ago 28m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:12:15.474 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (16m) 14m ago 28m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:12:15.474 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (15m) 14m ago 27m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:12:15.474 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (15m) 14m ago 27m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:12:15.474 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (21m) 14m ago 31m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:12:15.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f57ec06e680 msgr2=0x7f57ec070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f57ec06e680 0x7f57ec070b40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f57e8005fd0 tx=0x7f57e800a560 comp rx=0 tx=0).stop 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 msgr2=0x7f57fc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc19c640 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f57f0006010 tx=0x7f57f000be20 comp rx=0 tx=0).stop 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 shutdown_connections 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f57ec06e680 0x7f57ec070b40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f57fc0ffda0 0x7f57fc19c640 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 --2- 172.21.15.42:0/1104968949 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57fc100750 0x7f57fc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 >> 172.21.15.42:0/1104968949 conn(0x7f57fc0fb650 msgr2=0x7f57fc103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:15.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 shutdown_connections 2024-09-17T00:12:15.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.476+0000 7f5802ad5700 1 -- 172.21.15.42:0/1104968949 wait complete. 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 -- 172.21.15.42:0/591001535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 msgr2=0x7f1f881058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 --2- 172.21.15.42:0/591001535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f881058a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009910 tx=0x7f1f78009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 -- 172.21.15.42:0/591001535 shutdown_connections 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 --2- 172.21.15.42:0/591001535 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f881058a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 --2- 172.21.15.42:0/591001535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f88100ea0 0x7f1f88101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 -- 172.21.15.42:0/591001535 >> 172.21.15.42:0/591001535 conn(0x7f1f880fc750 msgr2=0x7f1f880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 -- 172.21.15.42:0/591001535 shutdown_connections 2024-09-17T00:12:15.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.583+0000 7f1f907c5700 1 -- 172.21.15.42:0/591001535 wait complete. 2024-09-17T00:12:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f907c5700 1 Processor -- start 2024-09-17T00:12:15.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f907c5700 1 -- start start 2024-09-17T00:12:15.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f907c5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f88100ea0 0x7f1f8819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:15.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f907c5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f8819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:15.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f907c5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8819d240 con 0x7f1f88101850 2024-09-17T00:12:15.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f907c5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88196780 con 0x7f1f88100ea0 2024-09-17T00:12:15.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f8dd60700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f8819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f8dd60700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f8819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40964/0 (socket says 172.21.15.42:40964) 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f8dd60700 1 -- 172.21.15.42:0/2012953203 learned_addr learned my addr 172.21.15.42:0/2012953203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f8e561700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f88100ea0 0x7f1f8819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f8dd60700 1 -- 172.21.15.42:0/2012953203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f88100ea0 msgr2=0x7f1f8819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f8dd60700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f88100ea0 0x7f1f8819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.584+0000 7f1f8dd60700 1 -- 172.21.15.42:0/2012953203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f780095f0 con 0x7f1f88101850 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.585+0000 7f1f8dd60700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f8819cc40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800bda0 tx=0x7f1f7800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.585+0000 7f1f7f7fe700 1 -- 172.21.15.42:0/2012953203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f7801d070 con 0x7f1f88101850 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.585+0000 7f1f7f7fe700 1 -- 172.21.15.42:0/2012953203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f7800f460 con 0x7f1f88101850 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.585+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f88196a00 con 0x7f1f88101850 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.585+0000 7f1f7f7fe700 1 -- 172.21.15.42:0/2012953203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f78017630 con 0x7f1f88101850 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.585+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f88196ef0 con 0x7f1f88101850 2024-09-17T00:12:15.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.586+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f881092a0 con 0x7f1f88101850 2024-09-17T00:12:15.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.586+0000 7f1f7f7fe700 1 -- 172.21.15.42:0/2012953203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1f7800fab0 con 0x7f1f88101850 2024-09-17T00:12:15.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.587+0000 7f1f7f7fe700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f7406e680 0x7f1f74070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:15.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.587+0000 7f1f7f7fe700 1 -- 172.21.15.42:0/2012953203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1f7808fb20 con 0x7f1f88101850 2024-09-17T00:12:15.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.587+0000 7f1f8e561700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f7406e680 0x7f1f74070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:15.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.587+0000 7f1f8e561700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f7406e680 0x7f1f74070b40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400ac50 tx=0x7f1f8400a380 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:15.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.591+0000 7f1f7f7fe700 1 -- 172.21.15.42:0/2012953203 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1f7805c7d0 con 0x7f1f88101850 2024-09-17T00:12:15.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.888+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f88197d50 con 0x7f1f88101850 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.888+0000 7f1f7f7fe700 1 -- 172.21.15.42:0/2012953203 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1f7805c360 con 0x7f1f88101850 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:12:15.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:12:15.888 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:12:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.890+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f7406e680 msgr2=0x7f1f74070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.890+0000 7f1f907c5700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f7406e680 0x7f1f74070b40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400ac50 tx=0x7f1f8400a380 comp rx=0 tx=0).stop 2024-09-17T00:12:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.890+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 msgr2=0x7f1f8819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.890+0000 7f1f907c5700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f8819cc40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800bda0 tx=0x7f1f7800bdd0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.891+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 shutdown_connections 2024-09-17T00:12:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.891+0000 7f1f907c5700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1f7406e680 0x7f1f74070b40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.891+0000 7f1f907c5700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f88100ea0 0x7f1f8819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.891+0000 7f1f907c5700 1 --2- 172.21.15.42:0/2012953203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f88101850 0x7f1f8819cc40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.891+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 >> 172.21.15.42:0/2012953203 conn(0x7f1f880fc750 msgr2=0x7f1f880fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.891+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 shutdown_connections 2024-09-17T00:12:15.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.891+0000 7f1f907c5700 1 -- 172.21.15.42:0/2012953203 wait complete. 2024-09-17T00:12:15.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 -- 172.21.15.42:0/3522606302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48102760 msgr2=0x7f4d4810ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 --2- 172.21.15.42:0/3522606302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48102760 0x7f4d4810ac50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009940 tx=0x7f4d38009c50 comp rx=0 tx=0).stop 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 -- 172.21.15.42:0/3522606302 shutdown_connections 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 --2- 172.21.15.42:0/3522606302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48102760 0x7f4d4810ac50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 --2- 172.21.15.42:0/3522606302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d48101e40 0x7f4d48102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 -- 172.21.15.42:0/3522606302 >> 172.21.15.42:0/3522606302 conn(0x7f4d480fb650 msgr2=0x7f4d480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 -- 172.21.15.42:0/3522606302 shutdown_connections 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 -- 172.21.15.42:0/3522606302 wait complete. 2024-09-17T00:12:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:15.999+0000 7f4d4df44700 1 Processor -- start 2024-09-17T00:12:16.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d4df44700 1 -- start start 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d4df44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48101e40 0x7f4d480ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d4df44700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d48102760 0x7f4d480fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d4df44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48100520 con 0x7f4d48101e40 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d4df44700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48100660 con 0x7f4d48102760 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48101e40 0x7f4d480ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48101e40 0x7f4d480ffaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40988/0 (socket says 172.21.15.42:40988) 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d477fe700 1 -- 172.21.15.42:0/1073156137 learned_addr learned my addr 172.21.15.42:0/1073156137 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d46ffd700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d48102760 0x7f4d480fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d477fe700 1 -- 172.21.15.42:0/1073156137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d48102760 msgr2=0x7f4d480fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d477fe700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d48102760 0x7f4d480fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d477fe700 1 -- 172.21.15.42:0/1073156137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d380095f0 con 0x7f4d48101e40 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.000+0000 7f4d477fe700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48101e40 0x7f4d480ffaa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00e920 tx=0x7f4d3c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.001+0000 7f4d44ff9700 1 -- 172.21.15.42:0/1073156137 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d3c00c9d0 con 0x7f4d48101e40 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.001+0000 7f4d44ff9700 1 -- 172.21.15.42:0/1073156137 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d3c010430 con 0x7f4d48101e40 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.001+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d481a2a50 con 0x7f4d48101e40 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.001+0000 7f4d44ff9700 1 -- 172.21.15.42:0/1073156137 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d3c018630 con 0x7f4d48101e40 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.001+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d481a2e00 con 0x7f4d48101e40 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.002+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d48108350 con 0x7f4d48101e40 2024-09-17T00:12:16.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.002+0000 7f4d44ff9700 1 -- 172.21.15.42:0/1073156137 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f4d3c0105a0 con 0x7f4d48101e40 2024-09-17T00:12:16.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.003+0000 7f4d44ff9700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d3006e680 0x7f4d30070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.003+0000 7f4d44ff9700 1 -- 172.21.15.42:0/1073156137 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4d3c014070 con 0x7f4d48101e40 2024-09-17T00:12:16.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.003+0000 7f4d46ffd700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d3006e680 0x7f4d30070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.003+0000 7f4d46ffd700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d3006e680 0x7f4d30070b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009fa0 tx=0x7f4d38005ad0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:16.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.007+0000 7f4d44ff9700 1 -- 172.21.15.42:0/1073156137 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4d3c0541b0 con 0x7f4d48101e40 2024-09-17T00:12:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.258+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d48107d30 con 0x7f4d48101e40 2024-09-17T00:12:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.258+0000 7f4d44ff9700 1 -- 172.21.15.42:0/1073156137 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f4d3c057ad0 con 0x7f4d48101e40 2024-09-17T00:12:16.257 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:12:16.257 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:12:16.257 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:12:16.257 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:12:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:12:16.259 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:16.260 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d3006e680 msgr2=0x7f4d30070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d3006e680 0x7f4d30070b40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009fa0 tx=0x7f4d38005ad0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48101e40 msgr2=0x7f4d480ffaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48101e40 0x7f4d480ffaa0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00e920 tx=0x7f4d3c00ec30 comp rx=0 tx=0).stop 2024-09-17T00:12:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 shutdown_connections 2024-09-17T00:12:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d3006e680 0x7f4d30070b40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d48101e40 0x7f4d480ffaa0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 --2- 172.21.15.42:0/1073156137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d48102760 0x7f4d480fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 >> 172.21.15.42:0/1073156137 conn(0x7f4d480fb650 msgr2=0x7f4d48105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 shutdown_connections 2024-09-17T00:12:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.261+0000 7f4d4df44700 1 -- 172.21.15.42:0/1073156137 wait complete. 2024-09-17T00:12:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:12:16.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.370+0000 7f844659f700 1 -- 172.21.15.42:0/1195136973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84400ffc20 msgr2=0x7f8440100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.370 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.370+0000 7f844659f700 1 --2- 172.21.15.42:0/1195136973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84400ffc20 0x7f8440100080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8428009910 tx=0x7f8428009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 -- 172.21.15.42:0/1195136973 shutdown_connections 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 --2- 172.21.15.42:0/1195136973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84400ffc20 0x7f8440100080 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 --2- 172.21.15.42:0/1195136973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8440105bd0 0x7f8440105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 -- 172.21.15.42:0/1195136973 >> 172.21.15.42:0/1195136973 conn(0x7f8440075670 msgr2=0x7f8440075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 -- 172.21.15.42:0/1195136973 shutdown_connections 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 -- 172.21.15.42:0/1195136973 wait complete. 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 Processor -- start 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 -- start start 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.371+0000 7f844659f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84400ffc20 0x7f844019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f844659f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8440105bd0 0x7f844019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f844659f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844019f370 con 0x7f8440105bd0 2024-09-17T00:12:16.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f844659f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84401988b0 con 0x7f84400ffc20 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f8437fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8440105bd0 0x7f844019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f8437fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8440105bd0 0x7f844019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41004/0 (socket says 172.21.15.42:41004) 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f8437fff700 1 -- 172.21.15.42:0/85150056 learned_addr learned my addr 172.21.15.42:0/85150056 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f843ffff700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84400ffc20 0x7f844019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f8437fff700 1 -- 172.21.15.42:0/85150056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84400ffc20 msgr2=0x7f844019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f8437fff700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84400ffc20 0x7f844019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f8437fff700 1 -- 172.21.15.42:0/85150056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84280095f0 con 0x7f8440105bd0 2024-09-17T00:12:16.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f8437fff700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8440105bd0 0x7f844019ed70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8428006010 tx=0x7f842800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:16.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f843dffb700 1 -- 172.21.15.42:0/85150056 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842801d070 con 0x7f8440105bd0 2024-09-17T00:12:16.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f843dffb700 1 -- 172.21.15.42:0/85150056 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8428004fb0 con 0x7f8440105bd0 2024-09-17T00:12:16.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f843dffb700 1 -- 172.21.15.42:0/85150056 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8428022ea0 con 0x7f8440105bd0 2024-09-17T00:12:16.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.372+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8440198b30 con 0x7f8440105bd0 2024-09-17T00:12:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.373+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8440199020 con 0x7f8440105bd0 2024-09-17T00:12:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.373+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844004f2e0 con 0x7f8440105bd0 2024-09-17T00:12:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.374+0000 7f843dffb700 1 -- 172.21.15.42:0/85150056 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8428022470 con 0x7f8440105bd0 2024-09-17T00:12:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.375+0000 7f843dffb700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f842c06e680 0x7f842c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.375+0000 7f843dffb700 1 -- 172.21.15.42:0/85150056 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f842808fbe0 con 0x7f8440105bd0 2024-09-17T00:12:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.375+0000 7f843ffff700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f842c06e680 0x7f842c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.375+0000 7f843ffff700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f842c06e680 0x7f842c070b40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8430005ea0 tx=0x7f8430005e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:16.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.379+0000 7f843dffb700 1 -- 172.21.15.42:0/85150056 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f842805ca60 con 0x7f8440105bd0 2024-09-17T00:12:16.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:16 smithi042 ceph-mon[111056]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:16 smithi042 ceph-mon[111056]: pgmap v620: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:16 smithi042 ceph-mon[111056]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:16 smithi042 ceph-mon[111056]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:16 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2012953203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:12:16.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:16 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1073156137' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:12:16.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.612+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8440199290 con 0x7f842c06e680 2024-09-17T00:12:16.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.613+0000 7f843dffb700 1 -- 172.21.15.42:0/85150056 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8440199290 con 0x7f842c06e680 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:12:16.612 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:12:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:12:16.613 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:12:16.613 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:12:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f842c06e680 msgr2=0x7f842c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f842c06e680 0x7f842c070b40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8430005ea0 tx=0x7f8430005e30 comp rx=0 tx=0).stop 2024-09-17T00:12:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8440105bd0 msgr2=0x7f844019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8440105bd0 0x7f844019ed70 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8428006010 tx=0x7f842800be20 comp rx=0 tx=0).stop 2024-09-17T00:12:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 shutdown_connections 2024-09-17T00:12:16.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f842c06e680 0x7f842c070b40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84400ffc20 0x7f844019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 --2- 172.21.15.42:0/85150056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8440105bd0 0x7f844019ed70 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 >> 172.21.15.42:0/85150056 conn(0x7f8440075670 msgr2=0x7f8440068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:16.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 shutdown_connections 2024-09-17T00:12:16.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.616+0000 7f844659f700 1 -- 172.21.15.42:0/85150056 wait complete. 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 -- 172.21.15.42:0/3191974804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc0ffc20 msgr2=0x7f7fcc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/3191974804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc0ffc20 0x7f7fcc100080 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009910 tx=0x7f7fc0009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 -- 172.21.15.42:0/3191974804 shutdown_connections 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/3191974804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc0ffc20 0x7f7fcc100080 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/3191974804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc105bd0 0x7f7fcc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 -- 172.21.15.42:0/3191974804 >> 172.21.15.42:0/3191974804 conn(0x7f7fcc075670 msgr2=0x7f7fcc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 -- 172.21.15.42:0/3191974804 shutdown_connections 2024-09-17T00:12:16.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.724+0000 7f7fd23a6700 1 -- 172.21.15.42:0/3191974804 wait complete. 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.725+0000 7f7fd23a6700 1 Processor -- start 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.725+0000 7f7fd23a6700 1 -- start start 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.725+0000 7f7fd23a6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc0ffc20 0x7f7fcc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.725+0000 7f7fd23a6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc105bd0 0x7f7fcc197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.725+0000 7f7fd23a6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc198360 con 0x7f7fcc105bd0 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.725+0000 7f7fd23a6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1984d0 con 0x7f7fcc0ffc20 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc105bd0 0x7f7fcc197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc105bd0 0x7f7fcc197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41032/0 (socket says 172.21.15.42:41032) 2024-09-17T00:12:16.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcb7fe700 1 -- 172.21.15.42:0/4038918969 learned_addr learned my addr 172.21.15.42:0/4038918969 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcbfff700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc0ffc20 0x7f7fcc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcb7fe700 1 -- 172.21.15.42:0/4038918969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc0ffc20 msgr2=0x7f7fcc197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcb7fe700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc0ffc20 0x7f7fcc197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcb7fe700 1 -- 172.21.15.42:0/4038918969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc00095f0 con 0x7f7fcc105bd0 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fcb7fe700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc105bd0 0x7f7fcc197c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006010 tx=0x7f7fc000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fc97fa700 1 -- 172.21.15.42:0/4038918969 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc001d070 con 0x7f7fcc105bd0 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fc97fa700 1 -- 172.21.15.42:0/4038918969 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fc0005dc0 con 0x7f7fcc105bd0 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc1a07b0 con 0x7f7fcc105bd0 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fc97fa700 1 -- 172.21.15.42:0/4038918969 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc0017400 con 0x7f7fcc105bd0 2024-09-17T00:12:16.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.726+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc1a0c50 con 0x7f7fcc105bd0 2024-09-17T00:12:16.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.727+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc1080d0 con 0x7f7fcc105bd0 2024-09-17T00:12:16.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.728+0000 7f7fc97fa700 1 -- 172.21.15.42:0/4038918969 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f7fc0003a60 con 0x7f7fcc105bd0 2024-09-17T00:12:16.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.729+0000 7f7fc97fa700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7fb406e680 0x7f7fb4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:16.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.729+0000 7f7fcbfff700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7fb406e680 0x7f7fb4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:16.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.729+0000 7f7fc97fa700 1 -- 172.21.15.42:0/4038918969 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7fc008fad0 con 0x7f7fcc105bd0 2024-09-17T00:12:16.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.729+0000 7f7fcbfff700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7fb406e680 0x7f7fb4070b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005950 tx=0x7f7fbc00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:16.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:16.733+0000 7f7fc97fa700 1 -- 172.21.15.42:0/4038918969 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7fc0058c50 con 0x7f7fcc105bd0 2024-09-17T00:12:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:16 smithi043 ceph-mon[81510]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:16 smithi043 ceph-mon[81510]: pgmap v620: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:16 smithi043 ceph-mon[81510]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:16 smithi043 ceph-mon[81510]: from='client.35168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:16 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2012953203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:12:16.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:16 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1073156137' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:12:17.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.031+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7fcc068b90 con 0x7f7fcc105bd0 2024-09-17T00:12:17.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.032+0000 7f7fc97fa700 1 -- 172.21.15.42:0/4038918969 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f7fc005c310 con 0x7f7fcc105bd0 2024-09-17T00:12:17.030 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:12:17.030 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:12:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:12:17.031 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:12:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:12:17.031 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:12:17.031 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:12:17.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7fb406e680 msgr2=0x7f7fb4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:17.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7fb406e680 0x7f7fb4070b40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005950 tx=0x7f7fbc00b410 comp rx=0 tx=0).stop 2024-09-17T00:12:17.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc105bd0 msgr2=0x7f7fcc197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:17.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc105bd0 0x7f7fcc197c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006010 tx=0x7f7fc000bd70 comp rx=0 tx=0).stop 2024-09-17T00:12:17.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 shutdown_connections 2024-09-17T00:12:17.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7fb406e680 0x7f7fb4070b40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:17.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7fcc0ffc20 0x7f7fcc197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:17.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 --2- 172.21.15.42:0/4038918969 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7fcc105bd0 0x7f7fcc197c80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:17.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 >> 172.21.15.42:0/4038918969 conn(0x7f7fcc075670 msgr2=0x7f7fcc0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:17.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 shutdown_connections 2024-09-17T00:12:17.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:17.034+0000 7f7fd23a6700 1 -- 172.21.15.42:0/4038918969 wait complete. 2024-09-17T00:12:17.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:17 smithi042 ceph-mon[111056]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:17.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:17 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4038918969' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:12:17.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:17 smithi043 ceph-mon[81510]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:17.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:17 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4038918969' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:12:18.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:18 smithi042 ceph-mon[111056]: pgmap v621: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:18.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:18 smithi043 ceph-mon[81510]: pgmap v621: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:20.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:20 smithi042 ceph-mon[111056]: pgmap v622: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:20.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:20 smithi043 ceph-mon[81510]: pgmap v622: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:21.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:21.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:22.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:22 smithi042 ceph-mon[111056]: pgmap v623: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:22.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:22 smithi043 ceph-mon[81510]: pgmap v623: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:24.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:24 smithi042 ceph-mon[111056]: pgmap v624: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:24.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:24 smithi043 ceph-mon[81510]: pgmap v624: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:26.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:26 smithi042 ceph-mon[111056]: pgmap v625: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:26.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:26 smithi043 ceph-mon[81510]: pgmap v625: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:28 smithi042 ceph-mon[111056]: pgmap v626: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:28.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:28 smithi043 ceph-mon[81510]: pgmap v626: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:30.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:30 smithi042 ceph-mon[111056]: pgmap v627: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:30.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:30 smithi043 ceph-mon[81510]: pgmap v627: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:32.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:32 smithi042 ceph-mon[111056]: pgmap v628: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:32.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:32 smithi043 ceph-mon[81510]: pgmap v628: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:34.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:34 smithi042 ceph-mon[111056]: pgmap v629: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:34.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:34 smithi043 ceph-mon[81510]: pgmap v629: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:36 smithi042 ceph-mon[111056]: pgmap v630: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:36.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:36.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:36 smithi043 ceph-mon[81510]: pgmap v630: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:36.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:38.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:38 smithi042 ceph-mon[111056]: pgmap v631: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:38.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:38 smithi043 ceph-mon[81510]: pgmap v631: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:40.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:40 smithi042 ceph-mon[111056]: pgmap v632: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:40.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:40 smithi043 ceph-mon[81510]: pgmap v632: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:42.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:42 smithi042 ceph-mon[111056]: pgmap v633: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:42.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:42 smithi043 ceph-mon[81510]: pgmap v633: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:44.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:44 smithi042 ceph-mon[111056]: pgmap v634: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:44.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:44 smithi043 ceph-mon[81510]: pgmap v634: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:46.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:46 smithi042 ceph-mon[111056]: pgmap v635: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:46.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:46 smithi043 ceph-mon[81510]: pgmap v635: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:47.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 -- 172.21.15.42:0/155079774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 msgr2=0x7f8a3c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 --2- 172.21.15.42:0/155079774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c1058a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009910 tx=0x7f8a24009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 -- 172.21.15.42:0/155079774 shutdown_connections 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 --2- 172.21.15.42:0/155079774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c1058a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 --2- 172.21.15.42:0/155079774 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c100ea0 0x7f8a3c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 -- 172.21.15.42:0/155079774 >> 172.21.15.42:0/155079774 conn(0x7f8a3c0fc750 msgr2=0x7f8a3c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 -- 172.21.15.42:0/155079774 shutdown_connections 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.157+0000 7f8a41cef700 1 -- 172.21.15.42:0/155079774 wait complete. 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.158+0000 7f8a41cef700 1 Processor -- start 2024-09-17T00:12:47.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.158+0000 7f8a41cef700 1 -- start start 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a41cef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c100ea0 0x7f8a3c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a41cef700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a41cef700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c073b10 con 0x7f8a3c101850 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a41cef700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c073c50 con 0x7f8a3c100ea0 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a32ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a32ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33156/0 (socket says 172.21.15.42:33156) 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a32ffd700 1 -- 172.21.15.42:0/3862797035 learned_addr learned my addr 172.21.15.42:0/3862797035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:47.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a3b7fe700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c100ea0 0x7f8a3c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a32ffd700 1 -- 172.21.15.42:0/3862797035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c100ea0 msgr2=0x7f8a3c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a32ffd700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c100ea0 0x7f8a3c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a32ffd700 1 -- 172.21.15.42:0/3862797035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240095f0 con 0x7f8a3c101850 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a32ffd700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c0735d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8a240092e0 tx=0x7f8a2400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.159+0000 7f8a397fa700 1 -- 172.21.15.42:0/3862797035 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a2401d070 con 0x7f8a3c101850 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.160+0000 7f8a397fa700 1 -- 172.21.15.42:0/3862797035 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a2400f460 con 0x7f8a3c101850 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.160+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c073ed0 con 0x7f8a3c101850 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.160+0000 7f8a397fa700 1 -- 172.21.15.42:0/3862797035 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a240175d0 con 0x7f8a3c101850 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.160+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c1a2e30 con 0x7f8a3c101850 2024-09-17T00:12:47.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.161+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c1092a0 con 0x7f8a3c101850 2024-09-17T00:12:47.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.161+0000 7f8a397fa700 1 -- 172.21.15.42:0/3862797035 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f8a2400f5d0 con 0x7f8a3c101850 2024-09-17T00:12:47.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.162+0000 7f8a397fa700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a2806e680 0x7f8a28070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.162+0000 7f8a397fa700 1 -- 172.21.15.42:0/3862797035 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8a2408fa40 con 0x7f8a3c101850 2024-09-17T00:12:47.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.162+0000 7f8a3b7fe700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a2806e680 0x7f8a28070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.162+0000 7f8a3b7fe700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a2806e680 0x7f8a28070b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005950 tx=0x7f8a2c00a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:47.164 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.166+0000 7f8a397fa700 1 -- 172.21.15.42:0/3862797035 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8a24058c60 con 0x7f8a3c101850 2024-09-17T00:12:47.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.395+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a3c074630 con 0x7f8a2806e680 2024-09-17T00:12:47.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.397+0000 7f8a397fa700 1 -- 172.21.15.42:0/3862797035 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a3c074630 con 0x7f8a2806e680 2024-09-17T00:12:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a2806e680 msgr2=0x7f8a28070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a2806e680 0x7f8a28070b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c005950 tx=0x7f8a2c00a300 comp rx=0 tx=0).stop 2024-09-17T00:12:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 msgr2=0x7f8a3c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c0735d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8a240092e0 tx=0x7f8a2400bd70 comp rx=0 tx=0).stop 2024-09-17T00:12:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 shutdown_connections 2024-09-17T00:12:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f8a2806e680 0x7f8a28070b40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a3c100ea0 0x7f8a3c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 --2- 172.21.15.42:0/3862797035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a3c101850 0x7f8a3c0735d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 >> 172.21.15.42:0/3862797035 conn(0x7f8a3c0fc750 msgr2=0x7f8a3c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 shutdown_connections 2024-09-17T00:12:47.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.399+0000 7f8a41cef700 1 -- 172.21.15.42:0/3862797035 wait complete. 2024-09-17T00:12:47.407 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 -- 172.21.15.42:0/2230200913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d24105bd0 msgr2=0x7f4d24105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 --2- 172.21.15.42:0/2230200913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d24105bd0 0x7f4d24105fb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400b0a0 tx=0x7f4d1400b3b0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 -- 172.21.15.42:0/2230200913 shutdown_connections 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 --2- 172.21.15.42:0/2230200913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d240ffc20 0x7f4d24100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 --2- 172.21.15.42:0/2230200913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d24105bd0 0x7f4d24105fb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 -- 172.21.15.42:0/2230200913 >> 172.21.15.42:0/2230200913 conn(0x7f4d24075670 msgr2=0x7f4d24075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 -- 172.21.15.42:0/2230200913 shutdown_connections 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 -- 172.21.15.42:0/2230200913 wait complete. 2024-09-17T00:12:47.507 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.509+0000 7f4d29765700 1 Processor -- start 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d29765700 1 -- start start 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d29765700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d240ffc20 0x7f4d2419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d29765700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d24105bd0 0x7f4d2419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d29765700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2419d360 con 0x7f4d240ffc20 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d29765700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d24196810 con 0x7f4d24105bd0 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d22ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d240ffc20 0x7f4d2419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d22ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d240ffc20 0x7f4d2419c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33172/0 (socket says 172.21.15.42:33172) 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d22ffd700 1 -- 172.21.15.42:0/199592359 learned_addr learned my addr 172.21.15.42:0/199592359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:47.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d227fc700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d24105bd0 0x7f4d2419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d22ffd700 1 -- 172.21.15.42:0/199592359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d24105bd0 msgr2=0x7f4d2419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d22ffd700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d24105bd0 0x7f4d2419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.510+0000 7f4d22ffd700 1 -- 172.21.15.42:0/199592359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009d00 con 0x7f4d240ffc20 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.511+0000 7f4d22ffd700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d240ffc20 0x7f4d2419c790 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400b070 tx=0x7f4d1400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.511+0000 7f4d1bfff700 1 -- 172.21.15.42:0/199592359 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1400e040 con 0x7f4d240ffc20 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.511+0000 7f4d1bfff700 1 -- 172.21.15.42:0/199592359 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d14003e60 con 0x7f4d240ffc20 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.511+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d24196a90 con 0x7f4d240ffc20 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.511+0000 7f4d1bfff700 1 -- 172.21.15.42:0/199592359 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d1401d910 con 0x7f4d240ffc20 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.511+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d24196f80 con 0x7f4d240ffc20 2024-09-17T00:12:47.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.512+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2404f2e0 con 0x7f4d240ffc20 2024-09-17T00:12:47.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.512+0000 7f4d1bfff700 1 -- 172.21.15.42:0/199592359 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f4d14004820 con 0x7f4d240ffc20 2024-09-17T00:12:47.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.513+0000 7f4d1bfff700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d1006e680 0x7f4d10070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.513+0000 7f4d1bfff700 1 -- 172.21.15.42:0/199592359 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4d1408fee0 con 0x7f4d240ffc20 2024-09-17T00:12:47.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.513+0000 7f4d227fc700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d1006e680 0x7f4d10070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.512 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.513+0000 7f4d227fc700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d1006e680 0x7f4d10070b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:47.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.517+0000 7f4d1bfff700 1 -- 172.21.15.42:0/199592359 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4d1405cc30 con 0x7f4d240ffc20 2024-09-17T00:12:47.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.747+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d24197260 con 0x7f4d1006e680 2024-09-17T00:12:47.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.749+0000 7f4d1bfff700 1 -- 172.21.15.42:0/199592359 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4d24197260 con 0x7f4d1006e680 2024-09-17T00:12:47.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d1006e680 msgr2=0x7f4d10070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d1006e680 0x7f4d10070b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c006fd0 tx=0x7f4d0c008040 comp rx=0 tx=0).stop 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d240ffc20 msgr2=0x7f4d2419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d240ffc20 0x7f4d2419c790 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400b070 tx=0x7f4d1400cdd0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 shutdown_connections 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f4d1006e680 0x7f4d10070b40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4d240ffc20 0x7f4d2419c790 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 --2- 172.21.15.42:0/199592359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d24105bd0 0x7f4d2419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 >> 172.21.15.42:0/199592359 conn(0x7f4d24075670 msgr2=0x7f4d240686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 shutdown_connections 2024-09-17T00:12:47.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.751+0000 7f4d29765700 1 -- 172.21.15.42:0/199592359 wait complete. 2024-09-17T00:12:47.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1592288318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 msgr2=0x7f5bd40ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1592288318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40ffeb0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009910 tx=0x7f5bc4009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:47.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1592288318 shutdown_connections 2024-09-17T00:12:47.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1592288318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40ffeb0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1592288318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bd4105a30 0x7f5bd4105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1592288318 >> 172.21.15.42:0/1592288318 conn(0x7f5bd40fb520 msgr2=0x7f5bd40fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:47.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1592288318 shutdown_connections 2024-09-17T00:12:47.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.859+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1592288318 wait complete. 2024-09-17T00:12:47.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bdc7b7700 1 Processor -- start 2024-09-17T00:12:47.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bdc7b7700 1 -- start start 2024-09-17T00:12:47.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bdc7b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bdc7b7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bd4105a30 0x7f5bd40fc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bdc7b7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40fc8c0 con 0x7f5bd40ffa30 2024-09-17T00:12:47.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bdc7b7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40fca00 con 0x7f5bd4105a30 2024-09-17T00:12:47.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bda553700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bda553700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33192/0 (socket says 172.21.15.42:33192) 2024-09-17T00:12:47.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bda553700 1 -- 172.21.15.42:0/1609173131 learned_addr learned my addr 172.21.15.42:0/1609173131 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:47.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.860+0000 7f5bd9d52700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bd4105a30 0x7f5bd40fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bda553700 1 -- 172.21.15.42:0/1609173131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bd4105a30 msgr2=0x7f5bd40fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:47.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bda553700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bd4105a30 0x7f5bd40fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:47.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bda553700 1 -- 172.21.15.42:0/1609173131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40095f0 con 0x7f5bd40ffa30 2024-09-17T00:12:47.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bda553700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40fbe40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d8d0 tx=0x7f5bd000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:47.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bcb7fe700 1 -- 172.21.15.42:0/1609173131 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd00098a0 con 0x7f5bd40ffa30 2024-09-17T00:12:47.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bcb7fe700 1 -- 172.21.15.42:0/1609173131 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bd0010460 con 0x7f5bd40ffa30 2024-09-17T00:12:47.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bcb7fe700 1 -- 172.21.15.42:0/1609173131 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd000f5f0 con 0x7f5bd40ffa30 2024-09-17T00:12:47.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd41a2860 con 0x7f5bd40ffa30 2024-09-17T00:12:47.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.861+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd41a2c70 con 0x7f5bd40ffa30 2024-09-17T00:12:47.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.862+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd4108fd0 con 0x7f5bd40ffa30 2024-09-17T00:12:47.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.863+0000 7f5bcb7fe700 1 -- 172.21.15.42:0/1609173131 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f5bd0009a00 con 0x7f5bd40ffa30 2024-09-17T00:12:47.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.863+0000 7f5bcb7fe700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5bc006e680 0x7f5bc0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:47.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.863+0000 7f5bcb7fe700 1 -- 172.21.15.42:0/1609173131 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5bd008e450 con 0x7f5bd40ffa30 2024-09-17T00:12:47.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.863+0000 7f5bd9d52700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5bc006e680 0x7f5bc0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:47.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.864+0000 7f5bd9d52700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5bc006e680 0x7f5bc0070b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40fd630 tx=0x7f5bc401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:47.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:47.867+0000 7f5bcb7fe700 1 -- 172.21.15.42:0/1609173131 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5bd0057670 con 0x7f5bd40ffa30 2024-09-17T00:12:48.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.088+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bd402cdd0 con 0x7f5bc006e680 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.097+0000 7f5bcb7fe700 1 -- 172.21.15.42:0/1609173131 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f5bd402cdd0 con 0x7f5bc006e680 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (21m) 15m ago 32m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (32m) 15m ago 32m 8938k - 18.2.1 5be31c24972a 9dc3a39595b4 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (31m) 14m ago 31m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (19m) 15m ago 32m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (18m) 14m ago 31m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (21m) 15m ago 31m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:12:48.095 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (15m) 15m ago 26m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (15m) 15m ago 26m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (14m) 14m ago 26m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (14m) 14m ago 26m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (23m) 15m ago 33m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (22m) 14m ago 31m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (20m) 15m ago 33m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (19m) 14m ago 31m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (22m) 15m ago 32m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (22m) 14m ago 31m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (18m) 15m ago 30m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (18m) 15m ago 29m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:12:48.096 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (17m) 15m ago 29m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:12:48.097 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (16m) 14m ago 28m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:12:48.097 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (16m) 14m ago 28m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:12:48.097 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (15m) 14m ago 28m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:12:48.097 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (21m) 15m ago 31m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:12:48.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5bc006e680 msgr2=0x7f5bc0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5bc006e680 0x7f5bc0070b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40fd630 tx=0x7f5bc401a040 comp rx=0 tx=0).stop 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 msgr2=0x7f5bd40fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40fbe40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d8d0 tx=0x7f5bd000dbe0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 shutdown_connections 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f5bc006e680 0x7f5bc0070b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5bd40ffa30 0x7f5bd40fbe40 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 --2- 172.21.15.42:0/1609173131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bd4105a30 0x7f5bd40fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 >> 172.21.15.42:0/1609173131 conn(0x7f5bd40fb520 msgr2=0x7f5bd4068ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 shutdown_connections 2024-09-17T00:12:48.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.100+0000 7f5bdc7b7700 1 -- 172.21.15.42:0/1609173131 wait complete. 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 -- 172.21.15.42:0/3402729222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530102560 msgr2=0x7fc5301029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 --2- 172.21.15.42:0/3402729222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530102560 0x7fc5301029e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc524009910 tx=0x7fc524009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 -- 172.21.15.42:0/3402729222 shutdown_connections 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 --2- 172.21.15.42:0/3402729222 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530102560 0x7fc5301029e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 --2- 172.21.15.42:0/3402729222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530100e10 0x7fc5301011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 -- 172.21.15.42:0/3402729222 >> 172.21.15.42:0/3402729222 conn(0x7fc5300fc590 msgr2=0x7fc5300fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 -- 172.21.15.42:0/3402729222 shutdown_connections 2024-09-17T00:12:48.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.206+0000 7fc534f9e700 1 -- 172.21.15.42:0/3402729222 wait complete. 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc534f9e700 1 Processor -- start 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc534f9e700 1 -- start start 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc534f9e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530100e10 0x7fc530073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc534f9e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530102560 0x7fc5300735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc534f9e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc530073b10 con 0x7fc530100e10 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc534f9e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc530073c50 con 0x7fc530102560 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc52f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530100e10 0x7fc530073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc52f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530100e10 0x7fc530073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33220/0 (socket says 172.21.15.42:33220) 2024-09-17T00:12:48.206 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.207+0000 7fc52f7fe700 1 -- 172.21.15.42:0/4004947017 learned_addr learned my addr 172.21.15.42:0/4004947017 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52effd700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530102560 0x7fc5300735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52f7fe700 1 -- 172.21.15.42:0/4004947017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530102560 msgr2=0x7fc5300735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52f7fe700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530102560 0x7fc5300735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52f7fe700 1 -- 172.21.15.42:0/4004947017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240095f0 con 0x7fc530100e10 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52f7fe700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530100e10 0x7fc530073090 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc52000d900 tx=0x7fc52000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52cff9700 1 -- 172.21.15.42:0/4004947017 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5200041f0 con 0x7fc530100e10 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52cff9700 1 -- 172.21.15.42:0/4004947017 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc520004cf0 con 0x7fc530100e10 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5301a28a0 con 0x7fc530100e10 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc52cff9700 1 -- 172.21.15.42:0/4004947017 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52000b750 con 0x7fc530100e10 2024-09-17T00:12:48.207 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.208+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5301a2cb0 con 0x7fc530100e10 2024-09-17T00:12:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.209+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5301090f0 con 0x7fc530100e10 2024-09-17T00:12:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.210+0000 7fc52cff9700 1 -- 172.21.15.42:0/4004947017 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc520004350 con 0x7fc530100e10 2024-09-17T00:12:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.210+0000 7fc52cff9700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc51806e680 0x7fc518070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.211+0000 7fc52cff9700 1 -- 172.21.15.42:0/4004947017 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc52008e840 con 0x7fc530100e10 2024-09-17T00:12:48.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.211+0000 7fc52effd700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc51806e680 0x7fc518070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:48.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.211+0000 7fc52effd700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc51806e680 0x7fc518070b40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc524005b40 tx=0x7fc52401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:48.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.215+0000 7fc52cff9700 1 -- 172.21.15.42:0/4004947017 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc5200579e0 con 0x7fc530100e10 2024-09-17T00:12:48.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.513+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc530108ad0 con 0x7fc530100e10 2024-09-17T00:12:48.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.514+0000 7fc52cff9700 1 -- 172.21.15.42:0/4004947017 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc52005b0a0 con 0x7fc530100e10 2024-09-17T00:12:48.511 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:12:48.511 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:12:48.511 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:12:48.512 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:12:48.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc51806e680 msgr2=0x7fc518070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc51806e680 0x7fc518070b40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc524005b40 tx=0x7fc52401a040 comp rx=0 tx=0).stop 2024-09-17T00:12:48.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530100e10 msgr2=0x7fc530073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530100e10 0x7fc530073090 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc52000d900 tx=0x7fc52000dcc0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 shutdown_connections 2024-09-17T00:12:48.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc51806e680 0x7fc518070b40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc530100e10 0x7fc530073090 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 --2- 172.21.15.42:0/4004947017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc530102560 0x7fc5300735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 >> 172.21.15.42:0/4004947017 conn(0x7fc5300fc590 msgr2=0x7fc5300fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:48.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 shutdown_connections 2024-09-17T00:12:48.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.516+0000 7fc534f9e700 1 -- 172.21.15.42:0/4004947017 wait complete. 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 -- 172.21.15.42:0/1617873179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 msgr2=0x7f99881047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 --2- 172.21.15.42:0/1617873179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f99881047a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9978009910 tx=0x7f9978009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 -- 172.21.15.42:0/1617873179 shutdown_connections 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 --2- 172.21.15.42:0/1617873179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f99881047a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 --2- 172.21.15.42:0/1617873179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99880ffda0 0x7f9988100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 -- 172.21.15.42:0/1617873179 >> 172.21.15.42:0/1617873179 conn(0x7f99880fb650 msgr2=0x7f99880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 -- 172.21.15.42:0/1617873179 shutdown_connections 2024-09-17T00:12:48.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.623+0000 7f9990750700 1 -- 172.21.15.42:0/1617873179 wait complete. 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f9990750700 1 Processor -- start 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f9990750700 1 -- start start 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f9990750700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99880ffda0 0x7f99881955b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f9990750700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f9988195af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f9990750700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9988196030 con 0x7f9988100750 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f9990750700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99881961a0 con 0x7f99880ffda0 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f998dceb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f9988195af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f998dceb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f9988195af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33236/0 (socket says 172.21.15.42:33236) 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.624+0000 7f998dceb700 1 -- 172.21.15.42:0/2432979672 learned_addr learned my addr 172.21.15.42:0/2432979672 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f998e4ec700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99880ffda0 0x7f99881955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f998dceb700 1 -- 172.21.15.42:0/2432979672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99880ffda0 msgr2=0x7f99881955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f998dceb700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99880ffda0 0x7f99881955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f998dceb700 1 -- 172.21.15.42:0/2432979672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780095f0 con 0x7f9988100750 2024-09-17T00:12:48.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f998dceb700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f9988195af0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f997800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f997f7fe700 1 -- 172.21.15.42:0/2432979672 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f997801d070 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f997f7fe700 1 -- 172.21.15.42:0/2432979672 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9978022470 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9988196420 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f997f7fe700 1 -- 172.21.15.42:0/2432979672 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9978017610 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.625+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99881a2e40 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.626+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99881081a0 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.627+0000 7f997f7fe700 1 -- 172.21.15.42:0/2432979672 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f99780225e0 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.627+0000 7f997f7fe700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f997406e680 0x7f9974070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.627+0000 7f997f7fe700 1 -- 172.21.15.42:0/2432979672 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f997808fb20 con 0x7f9988100750 2024-09-17T00:12:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.627+0000 7f998e4ec700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f997406e680 0x7f9974070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:48.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.628+0000 7f998e4ec700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f997406e680 0x7f9974070b40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9984009bf0 tx=0x7f9984009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:48.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.632+0000 7f997f7fe700 1 -- 172.21.15.42:0/2432979672 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f997805c7f0 con 0x7f9988100750 2024-09-17T00:12:48.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:48 smithi042 ceph-mon[111056]: pgmap v636: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:48.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:48 smithi042 ceph-mon[111056]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:48.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:48 smithi042 ceph-mon[111056]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:48.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:48 smithi043 ceph-mon[81510]: pgmap v636: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:48 smithi043 ceph-mon[81510]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:48.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:48 smithi043 ceph-mon[81510]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.887+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9988196420 con 0x7f9988100750 2024-09-17T00:12:48.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.888+0000 7f997f7fe700 1 -- 172.21.15.42:0/2432979672 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1910 (secure 0 0 0) 0x7f9988196420 con 0x7f9988100750 2024-09-17T00:12:48.885 INFO:teuthology.orchestra.run.smithi042.stdout:e29 2024-09-17T00:12:48.885 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-16T23:58:07:827497+0000 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 20 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-16T23:57:12.183428+0000 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:12:48.886 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:up {} 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:failed 0,1 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:12:48.887 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 0 members: 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{-1:34420} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:48.888 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:12:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f997406e680 msgr2=0x7f9974070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f997406e680 0x7f9974070b40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9984009bf0 tx=0x7f9984009260 comp rx=0 tx=0).stop 2024-09-17T00:12:48.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 msgr2=0x7f9988195af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f9988195af0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f997800f690 comp rx=0 tx=0).stop 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 shutdown_connections 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f997406e680 0x7f9974070b40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99880ffda0 0x7f99881955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 --2- 172.21.15.42:0/2432979672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9988100750 0x7f9988195af0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 >> 172.21.15.42:0/2432979672 conn(0x7f99880fb650 msgr2=0x7f99880fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 shutdown_connections 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.890+0000 7f9990750700 1 -- 172.21.15.42:0/2432979672 wait complete. 2024-09-17T00:12:48.890 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:12:48.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 -- 172.21.15.42:0/2315112900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 msgr2=0x7fd178101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:48.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 --2- 172.21.15.42:0/2315112900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd178101270 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd168009910 tx=0x7fd168009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:48.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 -- 172.21.15.42:0/2315112900 shutdown_connections 2024-09-17T00:12:48.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 --2- 172.21.15.42:0/2315112900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd178101840 0x7fd178105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 --2- 172.21.15.42:0/2315112900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd178101270 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:48.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 -- 172.21.15.42:0/2315112900 >> 172.21.15.42:0/2315112900 conn(0x7fd1780fc740 msgr2=0x7fd1780feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:48.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 -- 172.21.15.42:0/2315112900 shutdown_connections 2024-09-17T00:12:48.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:48.999+0000 7fd17e87c700 1 -- 172.21.15.42:0/2315112900 wait complete. 2024-09-17T00:12:48.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd17e87c700 1 Processor -- start 2024-09-17T00:12:48.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd17e87c700 1 -- start start 2024-09-17T00:12:48.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd17e87c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd17819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd17e87c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd178101840 0x7fd17819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:48.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd17e87c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17819f430 con 0x7fd178100e90 2024-09-17T00:12:48.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd17e87c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781988e0 con 0x7fd178101840 2024-09-17T00:12:48.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd177fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd17819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:48.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd177fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd17819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33260/0 (socket says 172.21.15.42:33260) 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd177fff700 1 -- 172.21.15.42:0/882012770 learned_addr learned my addr 172.21.15.42:0/882012770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd1777fe700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd178101840 0x7fd17819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.000+0000 7fd177fff700 1 -- 172.21.15.42:0/882012770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd178101840 msgr2=0x7fd17819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd177fff700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd178101840 0x7fd17819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd177fff700 1 -- 172.21.15.42:0/882012770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680095f0 con 0x7fd178100e90 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd177fff700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd17819e860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd168006010 tx=0x7fd16800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd1757fa700 1 -- 172.21.15.42:0/882012770 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd16801d070 con 0x7fd178100e90 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178198b60 con 0x7fd178100e90 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd1757fa700 1 -- 172.21.15.42:0/882012770 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd16800f460 con 0x7fd178100e90 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd1757fa700 1 -- 172.21.15.42:0/882012770 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd168017630 con 0x7fd178100e90 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.001+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178199050 con 0x7fd178100e90 2024-09-17T00:12:49.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.002+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178105010 con 0x7fd178100e90 2024-09-17T00:12:49.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.003+0000 7fd1757fa700 1 -- 172.21.15.42:0/882012770 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd16800f5d0 con 0x7fd178100e90 2024-09-17T00:12:49.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.003+0000 7fd1757fa700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd16006e680 0x7fd160070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:49.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.003+0000 7fd1777fe700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd16006e680 0x7fd160070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:49.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.003+0000 7fd1757fa700 1 -- 172.21.15.42:0/882012770 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd16808fa80 con 0x7fd178100e90 2024-09-17T00:12:49.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.004+0000 7fd1777fe700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd16006e680 0x7fd160070b40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005950 tx=0x7fd16c0058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:49.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.008+0000 7fd1757fa700 1 -- 172.21.15.42:0/882012770 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd16805c7d0 con 0x7fd178100e90 2024-09-17T00:12:49.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.236+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd178199270 con 0x7fd16006e680 2024-09-17T00:12:49.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.237+0000 7fd1757fa700 1 -- 172.21.15.42:0/882012770 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd178199270 con 0x7fd16006e680 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:12:49.235 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:12:49.236 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:12:49.236 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading mds daemons", 2024-09-17T00:12:49.236 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:12:49.236 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd16006e680 msgr2=0x7fd160070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd16006e680 0x7fd160070b40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005950 tx=0x7fd16c0058e0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 msgr2=0x7fd17819e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd17819e860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd168006010 tx=0x7fd16800be20 comp rx=0 tx=0).stop 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 shutdown_connections 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd16006e680 0x7fd160070b40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd178100e90 0x7fd17819e860 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 --2- 172.21.15.42:0/882012770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd178101840 0x7fd17819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 >> 172.21.15.42:0/882012770 conn(0x7fd1780fc740 msgr2=0x7fd1780fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 shutdown_connections 2024-09-17T00:12:49.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.240+0000 7fd17e87c700 1 -- 172.21.15.42:0/882012770 wait complete. 2024-09-17T00:12:49.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 -- 172.21.15.42:0/179153751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc101850 msgr2=0x7fc9dc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:49.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 --2- 172.21.15.42:0/179153751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc101850 0x7fc9dc1058a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4009910 tx=0x7fc9c4009c20 comp rx=0 tx=0).stop 2024-09-17T00:12:49.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 -- 172.21.15.42:0/179153751 shutdown_connections 2024-09-17T00:12:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 --2- 172.21.15.42:0/179153751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc101850 0x7fc9dc1058a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 --2- 172.21.15.42:0/179153751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9dc100ea0 0x7fc9dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 -- 172.21.15.42:0/179153751 >> 172.21.15.42:0/179153751 conn(0x7fc9dc0fc750 msgr2=0x7fc9dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 -- 172.21.15.42:0/179153751 shutdown_connections 2024-09-17T00:12:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.349+0000 7fc9e1399700 1 -- 172.21.15.42:0/179153751 wait complete. 2024-09-17T00:12:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.350+0000 7fc9e1399700 1 Processor -- start 2024-09-17T00:12:49.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.350+0000 7fc9e1399700 1 -- start start 2024-09-17T00:12:49.349 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.350+0000 7fc9e1399700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc100ea0 0x7fc9dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.350+0000 7fc9e1399700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9dc101850 0x7fc9dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.350+0000 7fc9e1399700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc073b10 con 0x7fc9dc100ea0 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.350+0000 7fc9e1399700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc073c50 con 0x7fc9dc101850 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc100ea0 0x7fc9dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc100ea0 0x7fc9dc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33282/0 (socket says 172.21.15.42:33282) 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9daffd700 1 -- 172.21.15.42:0/2508967952 learned_addr learned my addr 172.21.15.42:0/2508967952 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9da7fc700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9dc101850 0x7fc9dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9daffd700 1 -- 172.21.15.42:0/2508967952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9dc101850 msgr2=0x7fc9dc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9daffd700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9dc101850 0x7fc9dc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9daffd700 1 -- 172.21.15.42:0/2508967952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c40095f0 con 0x7fc9dc100ea0 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9da7fc700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9dc101850 0x7fc9dc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:12:49.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9daffd700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc100ea0 0x7fc9dc073090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00d8d0 tx=0x7fc9cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9d3fff700 1 -- 172.21.15.42:0/2508967952 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9cc009940 con 0x7fc9dc100ea0 2024-09-17T00:12:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9d3fff700 1 -- 172.21.15.42:0/2508967952 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9cc010460 con 0x7fc9dc100ea0 2024-09-17T00:12:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc1a2b00 con 0x7fc9dc100ea0 2024-09-17T00:12:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9d3fff700 1 -- 172.21.15.42:0/2508967952 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9cc00f5f0 con 0x7fc9dc100ea0 2024-09-17T00:12:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.351+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc1a2f10 con 0x7fc9dc100ea0 2024-09-17T00:12:49.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.352+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc1092a0 con 0x7fc9dc100ea0 2024-09-17T00:12:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.353+0000 7fc9d3fff700 1 -- 172.21.15.42:0/2508967952 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc9cc0105d0 con 0x7fc9dc100ea0 2024-09-17T00:12:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.353+0000 7fc9d3fff700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc9c806e680 0x7fc9c8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:12:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.354+0000 7fc9da7fc700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc9c806e680 0x7fc9c8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:12:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.354+0000 7fc9d3fff700 1 -- 172.21.15.42:0/2508967952 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc9cc08e680 con 0x7fc9dc100ea0 2024-09-17T00:12:49.352 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.354+0000 7fc9da7fc700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc9c806e680 0x7fc9c8070b40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4006010 tx=0x7fc9c400b540 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:12:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.358+0000 7fc9d3fff700 1 -- 172.21.15.42:0/2508967952 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc9cc057820 con 0x7fc9dc100ea0 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.658+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc9dc108c80 con 0x7fc9dc100ea0 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.658+0000 7fc9d3fff700 1 -- 172.21.15.42:0/2508967952 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc9cc05aee0 con 0x7fc9dc100ea0 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is degraded 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:12:49.656 INFO:teuthology.orchestra.run.smithi042.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-17T00:12:49.657 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-17T00:12:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc9c806e680 msgr2=0x7fc9c8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc9c806e680 0x7fc9c8070b40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc9c4006010 tx=0x7fc9c400b540 comp rx=0 tx=0).stop 2024-09-17T00:12:49.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc100ea0 msgr2=0x7fc9dc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc100ea0 0x7fc9dc073090 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00d8d0 tx=0x7fc9cc00dc90 comp rx=0 tx=0).stop 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 shutdown_connections 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc9c806e680 0x7fc9c8070b40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9dc100ea0 0x7fc9dc073090 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 --2- 172.21.15.42:0/2508967952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9dc101850 0x7fc9dc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 >> 172.21.15.42:0/2508967952 conn(0x7fc9dc0fc750 msgr2=0x7fc9dc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 shutdown_connections 2024-09-17T00:12:49.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:12:49.661+0000 7fc9e1399700 1 -- 172.21.15.42:0/2508967952 wait complete. 2024-09-17T00:12:49.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:49 smithi042 ceph-mon[111056]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:49.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:49 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/4004947017' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:12:49.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:49 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2432979672' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:12:49.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:49 smithi043 ceph-mon[81510]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:49 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/4004947017' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:12:49.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:49 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2432979672' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:12:50.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:50 smithi042 ceph-mon[111056]: pgmap v637: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:50 smithi042 ceph-mon[111056]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:50.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:50 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2508967952' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:12:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:50 smithi043 ceph-mon[81510]: pgmap v637: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:50 smithi043 ceph-mon[81510]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:12:50.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:50 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2508967952' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:12:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:12:52.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:52 smithi042 ceph-mon[111056]: pgmap v638: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:52.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:52 smithi043 ceph-mon[81510]: pgmap v638: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:54.575 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:54 smithi042 ceph-mon[111056]: pgmap v639: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:54.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:54 smithi043 ceph-mon[81510]: pgmap v639: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:56.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:56 smithi042 ceph-mon[111056]: pgmap v640: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:56.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:56 smithi043 ceph-mon[81510]: pgmap v640: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:58.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:12:58 smithi042 ceph-mon[111056]: pgmap v641: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:12:58.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:12:58 smithi043 ceph-mon[81510]: pgmap v641: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:00.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:00 smithi042 ceph-mon[111056]: pgmap v642: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:00.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:00 smithi043 ceph-mon[81510]: pgmap v642: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:02.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:02 smithi042 ceph-mon[111056]: pgmap v643: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:02.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:02 smithi043 ceph-mon[81510]: pgmap v643: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:04.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:04 smithi042 ceph-mon[111056]: pgmap v644: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:04.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:04 smithi043 ceph-mon[81510]: pgmap v644: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:06 smithi042 ceph-mon[111056]: pgmap v645: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:06.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:06 smithi043 ceph-mon[81510]: pgmap v645: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: pgmap v646: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:08.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.anvvuy"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.anvvuy"}]': finished 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.ctckas"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.ctckas"}]': finished 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.pzcdzg"}]': finished 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.riwwhn"}]: dispatch 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.riwwhn"}]': finished 2024-09-17T00:13:08.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:08 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-17T00:13:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: pgmap v646: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:09.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.anvvuy"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.anvvuy"}]': finished 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.ctckas"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi042.ctckas"}]': finished 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.pzcdzg"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.pzcdzg"}]': finished 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.riwwhn"}]: dispatch 2024-09-17T00:13:09.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi043.riwwhn"}]': finished 2024-09-17T00:13:09.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:08 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-17T00:13:09.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: pgmap v647: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:09.774 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all mds 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: Upgrade: Setting filesystem cephfs Joinable 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: Standby daemon mds.cephfs.smithi042.ctckas assigned to filesystem cephfs as rank 0 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: Standby daemon mds.cephfs.smithi042.anvvuy assigned to filesystem cephfs as rank 1 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:replay,1=cephfs.smithi042.anvvuy=up:replay} 2 up:standby 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:09.775 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:09 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: pgmap v647: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all mds 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: Upgrade: Setting filesystem cephfs Joinable 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: Standby daemon mds.cephfs.smithi042.ctckas assigned to filesystem cephfs as rank 0 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: Standby daemon mds.cephfs.smithi042.anvvuy assigned to filesystem cephfs as rank 1 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:replay,1=cephfs.smithi042.anvvuy=up:replay} 2 up:standby 2024-09-17T00:13:09.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:09.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:09.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:09.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:09 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:10.682 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:10 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all rgw 2024-09-17T00:13:10.682 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:10 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all rbd-mirror 2024-09-17T00:13:10.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:10 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all rgw 2024-09-17T00:13:10.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:10 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all rbd-mirror 2024-09-17T00:13:11.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:11 smithi042 ceph-mon[111056]: pgmap v648: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 747 KiB/s rd, 0 op/s 2024-09-17T00:13:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:11 smithi042 ceph-mon[111056]: Upgrade: Updating ceph-exporter.smithi042 (1/2) 2024-09-17T00:13:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-17T00:13:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:11 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:11.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:11 smithi042 ceph-mon[111056]: Deploying daemon ceph-exporter.smithi042 on smithi042 2024-09-17T00:13:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:11 smithi043 ceph-mon[81510]: pgmap v648: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 747 KiB/s rd, 0 op/s 2024-09-17T00:13:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:11 smithi043 ceph-mon[81510]: Upgrade: Updating ceph-exporter.smithi042 (1/2) 2024-09-17T00:13:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi042", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-17T00:13:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:11 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:11 smithi043 ceph-mon[81510]: Deploying daemon ceph-exporter.smithi042 on smithi042 2024-09-17T00:13:13.873 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:13 smithi042 ceph-mon[111056]: pgmap v649: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-09-17T00:13:14.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:13 smithi043 ceph-mon[81510]: pgmap v649: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 3 op/s 2024-09-17T00:13:16.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:15 smithi042 ceph-mon[111056]: pgmap v650: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8 op/s 2024-09-17T00:13:16.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:15 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:resolve 2024-09-17T00:13:16.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:15 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:resolve 2024-09-17T00:13:16.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:15 smithi042 ceph-mon[111056]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:resolve,1=cephfs.smithi042.anvvuy=up:resolve} 2 up:standby 2024-09-17T00:13:16.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:16.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:15 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:15 smithi043 ceph-mon[81510]: pgmap v650: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8 op/s 2024-09-17T00:13:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:15 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:resolve 2024-09-17T00:13:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:15 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:resolve 2024-09-17T00:13:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:15 smithi043 ceph-mon[81510]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:resolve,1=cephfs.smithi042.anvvuy=up:resolve} 2 up:standby 2024-09-17T00:13:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:15 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:16.750 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: reconnect by client.14514 192.168.0.1:0/2919541394 after 0.000999992 2024-09-17T00:13:16.750 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: reconnect by client.14514 192.168.0.1:0/2919541394 after 0.000999992 2024-09-17T00:13:16.750 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: reconnect by client.24305 192.168.0.1:0/2491590655 after 0.000999992 2024-09-17T00:13:16.750 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: reconnect by client.24305 192.168.0.1:0/2491590655 after 0.000999992 2024-09-17T00:13:17.007 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:reconnect 2024-09-17T00:13:17.007 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:reconnect 2024-09-17T00:13:17.008 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:reconnect,1=cephfs.smithi042.anvvuy=up:reconnect} 2 up:standby 2024-09-17T00:13:17.008 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:17.008 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-17T00:13:17.008 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:16 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: reconnect by client.14514 192.168.0.1:0/2919541394 after 0.000999992 2024-09-17T00:13:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: reconnect by client.14514 192.168.0.1:0/2919541394 after 0.000999992 2024-09-17T00:13:17.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: reconnect by client.24305 192.168.0.1:0/2491590655 after 0.000999992 2024-09-17T00:13:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: reconnect by client.24305 192.168.0.1:0/2491590655 after 0.000999992 2024-09-17T00:13:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:reconnect 2024-09-17T00:13:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:reconnect 2024-09-17T00:13:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:reconnect,1=cephfs.smithi042.anvvuy=up:reconnect} 2 up:standby 2024-09-17T00:13:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi043", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-17T00:13:17.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:16 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:18.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: pgmap v651: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-09-17T00:13:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: Upgrade: Updating ceph-exporter.smithi043 (2/2) 2024-09-17T00:13:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: Deploying daemon ceph-exporter.smithi043 on smithi043 2024-09-17T00:13:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:rejoin 2024-09-17T00:13:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:rejoin 2024-09-17T00:13:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:rejoin,1=cephfs.smithi042.anvvuy=up:rejoin} 2 up:standby 2024-09-17T00:13:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: daemon mds.cephfs.smithi042.anvvuy is now active in filesystem cephfs as rank 1 2024-09-17T00:13:18.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:17 smithi042 ceph-mon[111056]: daemon mds.cephfs.smithi042.ctckas is now active in filesystem cephfs as rank 0 2024-09-17T00:13:18.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: pgmap v651: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-09-17T00:13:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: Upgrade: Updating ceph-exporter.smithi043 (2/2) 2024-09-17T00:13:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: Deploying daemon ceph-exporter.smithi043 on smithi043 2024-09-17T00:13:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:rejoin 2024-09-17T00:13:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:rejoin 2024-09-17T00:13:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: fsmap cephfs:2/2 {0=cephfs.smithi042.ctckas=up:rejoin,1=cephfs.smithi042.anvvuy=up:rejoin} 2 up:standby 2024-09-17T00:13:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: daemon mds.cephfs.smithi042.anvvuy is now active in filesystem cephfs as rank 1 2024-09-17T00:13:18.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:17 smithi043 ceph-mon[81510]: daemon mds.cephfs.smithi042.ctckas is now active in filesystem cephfs as rank 0 2024-09-17T00:13:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:18 smithi043 ceph-mon[81510]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-17T00:13:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:18 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:active 2024-09-17T00:13:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:18 smithi043 ceph-mon[81510]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:active 2024-09-17T00:13:18.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:18 smithi043 ceph-mon[81510]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi042.anvvuy=up:active} 2 up:standby 2024-09-17T00:13:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:18 smithi042 ceph-mon[111056]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-17T00:13:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:18 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] up:active 2024-09-17T00:13:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:18 smithi042 ceph-mon[111056]: mds.? [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] up:active 2024-09-17T00:13:19.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:18 smithi042 ceph-mon[111056]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi042.anvvuy=up:active} 2 up:standby 2024-09-17T00:13:19.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.775+0000 7fc87d5de700 1 -- 172.21.15.42:0/3024359240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 msgr2=0x7fc878101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.775+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3024359240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc878101280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc868009910 tx=0x7fc868009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.776+0000 7fc87d5de700 1 -- 172.21.15.42:0/3024359240 shutdown_connections 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.776+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3024359240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc878101850 0x7fc8781058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.776+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3024359240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc878101280 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.776+0000 7fc87d5de700 1 -- 172.21.15.42:0/3024359240 >> 172.21.15.42:0/3024359240 conn(0x7fc8780fc750 msgr2=0x7fc8780feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.776+0000 7fc87d5de700 1 -- 172.21.15.42:0/3024359240 shutdown_connections 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.776+0000 7fc87d5de700 1 -- 172.21.15.42:0/3024359240 wait complete. 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.776+0000 7fc87d5de700 1 Processor -- start 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc87d5de700 1 -- start start 2024-09-17T00:13:19.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc87d5de700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc87819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc87d5de700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc878101850 0x7fc87819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc87d5de700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87819d2e0 con 0x7fc878100ea0 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc87d5de700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc878196790 con 0x7fc878101850 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc876ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc87819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc876ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc87819c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53590/0 (socket says 172.21.15.42:53590) 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc876ffd700 1 -- 172.21.15.42:0/3665561009 learned_addr learned my addr 172.21.15.42:0/3665561009 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc8767fc700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc878101850 0x7fc87819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc876ffd700 1 -- 172.21.15.42:0/3665561009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc878101850 msgr2=0x7fc87819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc876ffd700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc878101850 0x7fc87819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc876ffd700 1 -- 172.21.15.42:0/3665561009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680095f0 con 0x7fc878100ea0 2024-09-17T00:13:19.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.777+0000 7fc876ffd700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc87819c710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc868005f50 tx=0x7fc86800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:19.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.778+0000 7fc85ffff700 1 -- 172.21.15.42:0/3665561009 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc86801d070 con 0x7fc878100ea0 2024-09-17T00:13:19.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.778+0000 7fc85ffff700 1 -- 172.21.15.42:0/3665561009 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc86800b610 con 0x7fc878100ea0 2024-09-17T00:13:19.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.778+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc878196a10 con 0x7fc878100ea0 2024-09-17T00:13:19.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.778+0000 7fc85ffff700 1 -- 172.21.15.42:0/3665561009 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc868022470 con 0x7fc878100ea0 2024-09-17T00:13:19.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.778+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc878196f00 con 0x7fc878100ea0 2024-09-17T00:13:19.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.779+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc878109240 con 0x7fc878100ea0 2024-09-17T00:13:19.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.779+0000 7fc85ffff700 1 -- 172.21.15.42:0/3665561009 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fc86800faf0 con 0x7fc878100ea0 2024-09-17T00:13:19.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.780+0000 7fc85ffff700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86006e680 0x7fc860070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:19.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.780+0000 7fc85ffff700 1 -- 172.21.15.42:0/3665561009 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc86808f9c0 con 0x7fc878100ea0 2024-09-17T00:13:19.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.780+0000 7fc8767fc700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86006e680 0x7fc860070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:19.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.780+0000 7fc8767fc700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86006e680 0x7fc860070b40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc878197c00 tx=0x7fc86c009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:19.782 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:19.785+0000 7fc85ffff700 1 -- 172.21.15.42:0/3665561009 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc86805c710 con 0x7fc878100ea0 2024-09-17T00:13:19.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:19 smithi043 ceph-mon[81510]: pgmap v652: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 93 B/s wr, 9 op/s 2024-09-17T00:13:20.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.016+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc878197170 con 0x7fc86006e680 2024-09-17T00:13:20.013 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:19 smithi042 ceph-mon[111056]: pgmap v652: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 93 B/s wr, 9 op/s 2024-09-17T00:13:20.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.017+0000 7fc85ffff700 1 -- 172.21.15.42:0/3665561009 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fc878197170 con 0x7fc86006e680 2024-09-17T00:13:20.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86006e680 msgr2=0x7fc860070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86006e680 0x7fc860070b40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc878197c00 tx=0x7fc86c009260 comp rx=0 tx=0).stop 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 msgr2=0x7fc87819c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc87819c710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc868005f50 tx=0x7fc86800f690 comp rx=0 tx=0).stop 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 shutdown_connections 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fc86006e680 0x7fc860070b40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc878100ea0 0x7fc87819c710 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 --2- 172.21.15.42:0/3665561009 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc878101850 0x7fc87819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 >> 172.21.15.42:0/3665561009 conn(0x7fc8780fc750 msgr2=0x7fc8780fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 shutdown_connections 2024-09-17T00:13:20.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.019+0000 7fc87d5de700 1 -- 172.21.15.42:0/3665561009 wait complete. 2024-09-17T00:13:20.028 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:13:20.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 -- 172.21.15.42:0/3127391431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 msgr2=0x7f7ae4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3127391431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae4100080 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7acc009910 tx=0x7f7acc009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 -- 172.21.15.42:0/3127391431 shutdown_connections 2024-09-17T00:13:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3127391431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae4100080 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3127391431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ae4105bd0 0x7f7ae4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 -- 172.21.15.42:0/3127391431 >> 172.21.15.42:0/3127391431 conn(0x7f7ae4075670 msgr2=0x7f7ae4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:20.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 -- 172.21.15.42:0/3127391431 shutdown_connections 2024-09-17T00:13:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.136+0000 7f7ae8872700 1 -- 172.21.15.42:0/3127391431 wait complete. 2024-09-17T00:13:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.137+0000 7f7ae8872700 1 Processor -- start 2024-09-17T00:13:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.137+0000 7f7ae8872700 1 -- start start 2024-09-17T00:13:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.137+0000 7f7ae8872700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.137+0000 7f7ae8872700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ae4105bd0 0x7f7ae419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.137+0000 7f7ae8872700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae419f400 con 0x7f7ae40ffc20 2024-09-17T00:13:20.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.137+0000 7f7ae8872700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae41988b0 con 0x7f7ae4105bd0 2024-09-17T00:13:20.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53612/0 (socket says 172.21.15.42:53612) 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae259c700 1 -- 172.21.15.42:0/3401453169 learned_addr learned my addr 172.21.15.42:0/3401453169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ad9d9b700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ae4105bd0 0x7f7ae419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae259c700 1 -- 172.21.15.42:0/3401453169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ae4105bd0 msgr2=0x7f7ae419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae259c700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ae4105bd0 0x7f7ae419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae259c700 1 -- 172.21.15.42:0/3401453169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc0095f0 con 0x7f7ae40ffc20 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae259c700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae419e830 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e920 tx=0x7f7ad400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7adbfff700 1 -- 172.21.15.42:0/3401453169 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ad400c9d0 con 0x7f7ae40ffc20 2024-09-17T00:13:20.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae4198b90 con 0x7f7ae40ffc20 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7adbfff700 1 -- 172.21.15.42:0/3401453169 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ad4010430 con 0x7f7ae40ffc20 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41990e0 con 0x7f7ae40ffc20 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.138+0000 7f7adbfff700 1 -- 172.21.15.42:0/3401453169 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ad4018630 con 0x7f7ae40ffc20 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.139+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac4005320 con 0x7f7ae40ffc20 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.141+0000 7f7adbfff700 1 -- 172.21.15.42:0/3401453169 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f7ad400cb30 con 0x7f7ae40ffc20 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.141+0000 7f7adbfff700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7ad006e680 0x7f7ad0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.141+0000 7f7adbfff700 1 -- 172.21.15.42:0/3401453169 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7ad4014070 con 0x7f7ae40ffc20 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.141+0000 7f7ad9d9b700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7ad006e680 0x7f7ad0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.142+0000 7f7ad9d9b700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7ad006e680 0x7f7ad0070b40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40ffa60 tx=0x7f7acc005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:20.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.146+0000 7f7adbfff700 1 -- 172.21.15.42:0/3401453169 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7ad40541b0 con 0x7f7ae40ffc20 2024-09-17T00:13:20.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.378+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ac4000bf0 con 0x7f7ad006e680 2024-09-17T00:13:20.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.379+0000 7f7adbfff700 1 -- 172.21.15.42:0/3401453169 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f7ac4000bf0 con 0x7f7ad006e680 2024-09-17T00:13:20.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7ad006e680 msgr2=0x7f7ad0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7ad006e680 0x7f7ad0070b40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40ffa60 tx=0x7f7acc005940 comp rx=0 tx=0).stop 2024-09-17T00:13:20.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 msgr2=0x7f7ae419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae419e830 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400e920 tx=0x7f7ad400ec30 comp rx=0 tx=0).stop 2024-09-17T00:13:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 shutdown_connections 2024-09-17T00:13:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7ad006e680 0x7f7ad0070b40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ae40ffc20 0x7f7ae419e830 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.381+0000 7f7ae8872700 1 --2- 172.21.15.42:0/3401453169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ae4105bd0 0x7f7ae419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.382+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 >> 172.21.15.42:0/3401453169 conn(0x7f7ae4075670 msgr2=0x7f7ae4068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.382+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 shutdown_connections 2024-09-17T00:13:20.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.382+0000 7f7ae8872700 1 -- 172.21.15.42:0/3401453169 wait complete. 2024-09-17T00:13:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.490+0000 7f0814506700 1 -- 172.21.15.42:0/1337383867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 msgr2=0x7f080c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 --2- 172.21.15.42:0/1337383867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c100080 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0800009910 tx=0x7f0800009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 -- 172.21.15.42:0/1337383867 shutdown_connections 2024-09-17T00:13:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 --2- 172.21.15.42:0/1337383867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c100080 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 --2- 172.21.15.42:0/1337383867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c105bd0 0x7f080c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 -- 172.21.15.42:0/1337383867 >> 172.21.15.42:0/1337383867 conn(0x7f080c075670 msgr2=0x7f080c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:20.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 -- 172.21.15.42:0/1337383867 shutdown_connections 2024-09-17T00:13:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 -- 172.21.15.42:0/1337383867 wait complete. 2024-09-17T00:13:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 Processor -- start 2024-09-17T00:13:20.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.491+0000 7f0814506700 1 -- start start 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f0814506700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f0814506700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c105bd0 0x7f080c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f0814506700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c1022c0 con 0x7f080c0ffc20 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f0814506700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c102400 con 0x7f080c105bd0 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f08122a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f08122a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53626/0 (socket says 172.21.15.42:53626) 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f08122a2700 1 -- 172.21.15.42:0/358460651 learned_addr learned my addr 172.21.15.42:0/358460651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f0811aa1700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c105bd0 0x7f080c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f08122a2700 1 -- 172.21.15.42:0/358460651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c105bd0 msgr2=0x7f080c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f08122a2700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c105bd0 0x7f080c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f08122a2700 1 -- 172.21.15.42:0/358460651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08000095f0 con 0x7f080c0ffc20 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f08122a2700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c101840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00e920 tx=0x7f07fc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f080b7fe700 1 -- 172.21.15.42:0/358460651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07fc00c9d0 con 0x7f080c0ffc20 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f080b7fe700 1 -- 172.21.15.42:0/358460651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07fc010430 con 0x7f080c0ffc20 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.492+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f080c1a2a30 con 0x7f080c0ffc20 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.493+0000 7f080b7fe700 1 -- 172.21.15.42:0/358460651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07fc018630 con 0x7f080c0ffc20 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.493+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c1a2e40 con 0x7f080c0ffc20 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.494+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f080c04f2e0 con 0x7f080c0ffc20 2024-09-17T00:13:20.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.494+0000 7f080b7fe700 1 -- 172.21.15.42:0/358460651 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f07fc0105a0 con 0x7f080c0ffc20 2024-09-17T00:13:20.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.495+0000 7f080b7fe700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f07f406e680 0x7f07f4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.495+0000 7f080b7fe700 1 -- 172.21.15.42:0/358460651 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f07fc014070 con 0x7f080c0ffc20 2024-09-17T00:13:20.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.495+0000 7f0811aa1700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f07f406e680 0x7f07f4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.495+0000 7f0811aa1700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f07f406e680 0x7f07f4070b40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f080c103030 tx=0x7f0800005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:20.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.499+0000 7f080b7fe700 1 -- 172.21.15.42:0/358460651 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f07fc0541b0 con 0x7f080c0ffc20 2024-09-17T00:13:20.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.730+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f080c068d90 con 0x7f07f406e680 2024-09-17T00:13:20.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.740+0000 7f080b7fe700 1 -- 172.21.15.42:0/358460651 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f080c068d90 con 0x7f07f406e680 2024-09-17T00:13:20.737 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (22m) 15m ago 32m 24.8M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 starting - - - - 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (31m) 15m ago 31m 8762k - 18.2.1 5be31c24972a 47b2370fb58c 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (19m) 15m ago 32m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (19m) 15m ago 31m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (21m) 15m ago 32m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (15m) 15m ago 27m 13.2M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (15m) 15m ago 27m 15.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (15m) 15m ago 27m 13.7M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (15m) 15m ago 26m 12.3M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (23m) 15m ago 34m 505M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (22m) 15m ago 31m 441M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (20m) 15m ago 34m 56.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:13:20.738 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (20m) 15m ago 31m 44.1M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (22m) 15m ago 32m 15.9M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (22m) 15m ago 31m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (19m) 15m ago 30m 88.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (18m) 15m ago 30m 85.7M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (17m) 15m ago 29m 80.8M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (17m) 15m ago 29m 89.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (16m) 15m ago 29m 92.6M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (16m) 15m ago 28m 86.7M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:13:20.739 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (22m) 15m ago 32m 76.8M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f07f406e680 msgr2=0x7f07f4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f07f406e680 0x7f07f4070b40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f080c103030 tx=0x7f0800005940 comp rx=0 tx=0).stop 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 msgr2=0x7f080c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c101840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00e920 tx=0x7f07fc00ec30 comp rx=0 tx=0).stop 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 shutdown_connections 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f07f406e680 0x7f07f4070b40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f080c0ffc20 0x7f080c101840 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 --2- 172.21.15.42:0/358460651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f080c105bd0 0x7f080c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.742+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 >> 172.21.15.42:0/358460651 conn(0x7f080c075670 msgr2=0x7f080c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:20.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.743+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 shutdown_connections 2024-09-17T00:13:20.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.743+0000 7f0814506700 1 -- 172.21.15.42:0/358460651 wait complete. 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 -- 172.21.15.42:0/3381362786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 msgr2=0x7f95e0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/3381362786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e0100080 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009910 tx=0x7f95d0009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 -- 172.21.15.42:0/3381362786 shutdown_connections 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/3381362786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e0100080 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/3381362786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95e0105bd0 0x7f95e0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 -- 172.21.15.42:0/3381362786 >> 172.21.15.42:0/3381362786 conn(0x7f95e0075670 msgr2=0x7f95e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 -- 172.21.15.42:0/3381362786 shutdown_connections 2024-09-17T00:13:20.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.851+0000 7f95e7c6f700 1 -- 172.21.15.42:0/3381362786 wait complete. 2024-09-17T00:13:20.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e7c6f700 1 Processor -- start 2024-09-17T00:13:20.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e7c6f700 1 -- start start 2024-09-17T00:13:20.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e7c6f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e7c6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95e0105bd0 0x7f95e019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e7c6f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0195620 con 0x7f95e00ffc20 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e7c6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0195790 con 0x7f95e0105bd0 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e5a0b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e5a0b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e019b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53642/0 (socket says 172.21.15.42:53642) 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.852+0000 7f95e5a0b700 1 -- 172.21.15.42:0/610186085 learned_addr learned my addr 172.21.15.42:0/610186085 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95e520a700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95e0105bd0 0x7f95e019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95e5a0b700 1 -- 172.21.15.42:0/610186085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95e0105bd0 msgr2=0x7f95e019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95e5a0b700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95e0105bd0 0x7f95e019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95e5a0b700 1 -- 172.21.15.42:0/610186085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d00095f0 con 0x7f95e00ffc20 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95e5a0b700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e019b630 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00d8d0 tx=0x7f95dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95d6ffd700 1 -- 172.21.15.42:0/610186085 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95dc0098a0 con 0x7f95e00ffc20 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95d6ffd700 1 -- 172.21.15.42:0/610186085 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95dc010460 con 0x7f95e00ffc20 2024-09-17T00:13:20.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e0195a70 con 0x7f95e00ffc20 2024-09-17T00:13:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95d6ffd700 1 -- 172.21.15.42:0/610186085 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95dc00f5f0 con 0x7f95e00ffc20 2024-09-17T00:13:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.853+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e0195f90 con 0x7f95e00ffc20 2024-09-17T00:13:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.854+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e01080d0 con 0x7f95e00ffc20 2024-09-17T00:13:20.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.855+0000 7f95d6ffd700 1 -- 172.21.15.42:0/610186085 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f95dc009a00 con 0x7f95e00ffc20 2024-09-17T00:13:20.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.855+0000 7f95d6ffd700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f95cc06e680 0x7f95cc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:20.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.855+0000 7f95d6ffd700 1 -- 172.21.15.42:0/610186085 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f95dc08e4b0 con 0x7f95e00ffc20 2024-09-17T00:13:20.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.855+0000 7f95e520a700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f95cc06e680 0x7f95cc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:20.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.856+0000 7f95e520a700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f95cc06e680 0x7f95cc070b40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b5c0 tx=0x7f95d001a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:20.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:20.860+0000 7f95d6ffd700 1 -- 172.21.15.42:0/610186085 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f95dc057670 con 0x7f95e00ffc20 2024-09-17T00:13:21.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.158+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f95e0196c60 con 0x7f95e00ffc20 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.158+0000 7f95d6ffd700 1 -- 172.21.15.42:0/610186085 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f95dc05ad30 con 0x7f95e00ffc20 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:13:21.156 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:13:21.157 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.160+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f95cc06e680 msgr2=0x7f95cc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.160+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f95cc06e680 0x7f95cc070b40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f95d000b5c0 tx=0x7f95d001a040 comp rx=0 tx=0).stop 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.160+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 msgr2=0x7f95e019b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.160+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e019b630 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00d8d0 tx=0x7f95dc00dbe0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.160+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 shutdown_connections 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.160+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f95cc06e680 0x7f95cc070b40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.160+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95e00ffc20 0x7f95e019b630 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.161+0000 7f95e7c6f700 1 --2- 172.21.15.42:0/610186085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95e0105bd0 0x7f95e019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.158 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.161+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 >> 172.21.15.42:0/610186085 conn(0x7f95e0075670 msgr2=0x7f95e00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:21.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.161+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 shutdown_connections 2024-09-17T00:13:21.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.161+0000 7f95e7c6f700 1 -- 172.21.15.42:0/610186085 wait complete. 2024-09-17T00:13:21.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 -- 172.21.15.42:0/2016319115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 msgr2=0x7fa7a410ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/2016319115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a410ac50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa794009940 tx=0x7fa794009c50 comp rx=0 tx=0).stop 2024-09-17T00:13:21.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 -- 172.21.15.42:0/2016319115 shutdown_connections 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/2016319115 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a410ac50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/2016319115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a4101e40 0x7fa7a4102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 -- 172.21.15.42:0/2016319115 >> 172.21.15.42:0/2016319115 conn(0x7fa7a40fb610 msgr2=0x7fa7a40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 -- 172.21.15.42:0/2016319115 shutdown_connections 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.273+0000 7fa7abbf9700 1 -- 172.21.15.42:0/2016319115 wait complete. 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7abbf9700 1 Processor -- start 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7abbf9700 1 -- start start 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7abbf9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a4101e40 0x7fa7a40ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7abbf9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a40fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7abbf9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4100520 con 0x7fa7a4102760 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7abbf9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4100660 con 0x7fa7a4101e40 2024-09-17T00:13:21.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7a9194700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a40fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:21.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7a9194700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a40fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53668/0 (socket says 172.21.15.42:53668) 2024-09-17T00:13:21.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7a9194700 1 -- 172.21.15.42:0/581459734 learned_addr learned my addr 172.21.15.42:0/581459734 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:21.273 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7a9995700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a4101e40 0x7fa7a40ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7a9194700 1 -- 172.21.15.42:0/581459734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a4101e40 msgr2=0x7fa7a40ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7a9194700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a4101e40 0x7fa7a40ffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.274+0000 7fa7a9194700 1 -- 172.21.15.42:0/581459734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7940095f0 con 0x7fa7a4102760 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.275+0000 7fa7a9194700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a40fffe0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa79400bb40 tx=0x7fa79400bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.275+0000 7fa79affd700 1 -- 172.21.15.42:0/581459734 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79401d070 con 0x7fa7a4102760 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.275+0000 7fa79affd700 1 -- 172.21.15.42:0/581459734 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa794003d10 con 0x7fa7a4102760 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.275+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a41008e0 con 0x7fa7a4102760 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.275+0000 7fa79affd700 1 -- 172.21.15.42:0/581459734 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa794021410 con 0x7fa7a4102760 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.275+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a41a2d40 con 0x7fa7a4102760 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.276+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4108350 con 0x7fa7a4102760 2024-09-17T00:13:21.274 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.277+0000 7fa79affd700 1 -- 172.21.15.42:0/581459734 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa79402b440 con 0x7fa7a4102760 2024-09-17T00:13:21.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.277+0000 7fa79affd700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa79006e680 0x7fa790070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:21.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.277+0000 7fa79affd700 1 -- 172.21.15.42:0/581459734 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa79408f740 con 0x7fa7a4102760 2024-09-17T00:13:21.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.277+0000 7fa7a9995700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa79006e680 0x7fa790070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:21.275 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.278+0000 7fa7a9995700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa79006e680 0x7fa790070b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009c60 tx=0x7fa7a0009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:21.280 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.283+0000 7fa79affd700 1 -- 172.21.15.42:0/581459734 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa794058b10 con 0x7fa7a4102760 2024-09-17T00:13:21.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.567+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa7a404f2e0 con 0x7fa7a4102760 2024-09-17T00:13:21.565 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.568+0000 7fa79affd700 1 -- 172.21.15.42:0/581459734 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1990 (secure 0 0 0) 0x7fa79405c1d0 con 0x7fa7a4102760 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:e35 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-17T00:13:17:752665+0000 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 35 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-17T00:13:21.566 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-17T00:13:17.752661+0000 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:13:21.567 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34388,1=34420} 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 34388 members: 34388,34420 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{0:34388} state up:active seq 243 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{1:34420} state up:active seq 239 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:13:21.568 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:13:21.569 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:13:21.569 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:13:21.569 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:13:21.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa79006e680 msgr2=0x7fa790070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.569 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa79006e680 0x7fa790070b40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009c60 tx=0x7fa7a0009260 comp rx=0 tx=0).stop 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 msgr2=0x7fa7a40fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a40fffe0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa79400bb40 tx=0x7fa79400bb70 comp rx=0 tx=0).stop 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 shutdown_connections 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa79006e680 0x7fa790070b40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7a4101e40 0x7fa7a40ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 --2- 172.21.15.42:0/581459734 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa7a4102760 0x7fa7a40fffe0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 >> 172.21.15.42:0/581459734 conn(0x7fa7a40fb610 msgr2=0x7fa7a4105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 shutdown_connections 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.570+0000 7fa7abbf9700 1 -- 172.21.15.42:0/581459734 wait complete. 2024-09-17T00:13:21.570 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 35 2024-09-17T00:13:21.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.685+0000 7f6cd38ea700 1 -- 172.21.15.42:0/1145741797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc106cd0 msgr2=0x7f6ccc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.685+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/1145741797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc106cd0 0x7f6ccc1070b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800b0a0 tx=0x7f6cc800b3b0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.685+0000 7f6cd38ea700 1 -- 172.21.15.42:0/1145741797 shutdown_connections 2024-09-17T00:13:21.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.685+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/1145741797 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ccc068580 0x7f6ccc0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.685+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/1145741797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc106cd0 0x7f6ccc1070b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.685+0000 7f6cd38ea700 1 -- 172.21.15.42:0/1145741797 >> 172.21.15.42:0/1145741797 conn(0x7f6ccc075670 msgr2=0x7f6ccc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:21.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.686+0000 7f6cd38ea700 1 -- 172.21.15.42:0/1145741797 shutdown_connections 2024-09-17T00:13:21.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.686+0000 7f6cd38ea700 1 -- 172.21.15.42:0/1145741797 wait complete. 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.686+0000 7f6cd38ea700 1 Processor -- start 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.686+0000 7f6cd38ea700 1 -- start start 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd38ea700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc068580 0x7f6ccc0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd38ea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ccc106cd0 0x7f6ccc100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd38ea700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc100770 con 0x7f6ccc068580 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd38ea700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc1008b0 con 0x7f6ccc106cd0 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd1686700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc068580 0x7f6ccc0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd1686700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc068580 0x7f6ccc0ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53684/0 (socket says 172.21.15.42:53684) 2024-09-17T00:13:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd0e85700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ccc106cd0 0x7f6ccc100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd1686700 1 -- 172.21.15.42:0/4024499151 learned_addr learned my addr 172.21.15.42:0/4024499151 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd1686700 1 -- 172.21.15.42:0/4024499151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ccc106cd0 msgr2=0x7f6ccc100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd1686700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ccc106cd0 0x7f6ccc100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd1686700 1 -- 172.21.15.42:0/4024499151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8009d00 con 0x7f6ccc068580 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd0e85700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ccc106cd0 0x7f6ccc100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cd1686700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc068580 0x7f6ccc0ffcf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8000f80 tx=0x7f6cc800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.687+0000 7f6cc27fc700 1 -- 172.21.15.42:0/4024499151 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc800e040 con 0x7f6ccc068580 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.688+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc100b30 con 0x7f6ccc068580 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.688+0000 7f6cc27fc700 1 -- 172.21.15.42:0/4024499151 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cc8003c10 con 0x7f6ccc068580 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.688+0000 7f6cc27fc700 1 -- 172.21.15.42:0/4024499151 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc801d680 con 0x7f6ccc068580 2024-09-17T00:13:21.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.688+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc1a2d90 con 0x7f6ccc068580 2024-09-17T00:13:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.689+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc1091f0 con 0x7f6ccc068580 2024-09-17T00:13:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.689+0000 7f6cc27fc700 1 -- 172.21.15.42:0/4024499151 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6cc8019070 con 0x7f6ccc068580 2024-09-17T00:13:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.690+0000 7f6cc27fc700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6cb806e680 0x7f6cb8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.690+0000 7f6cc27fc700 1 -- 172.21.15.42:0/4024499151 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6cc808fba0 con 0x7f6ccc068580 2024-09-17T00:13:21.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.690+0000 7f6cd0e85700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6cb806e680 0x7f6cb8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:21.688 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.690+0000 7f6cd0e85700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6cb806e680 0x7f6cb8070b40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc1014e0 tx=0x7f6cbc009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:21.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.695+0000 7f6cc27fc700 1 -- 172.21.15.42:0/4024499151 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6cc8058d40 con 0x7f6ccc068580 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: pgmap v653: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:21.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/610186085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:21.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.879 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:21 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/581459734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:13:21.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.937+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ccc0611d0 con 0x7f6cb806e680 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: pgmap v653: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:21.935 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/610186085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:21.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:21.936 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:21 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/581459734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:13:21.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.938+0000 7f6cc27fc700 1 -- 172.21.15.42:0/4024499151 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f6ccc0611d0 con 0x7f6cb806e680 2024-09-17T00:13:21.936 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": true, 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [ 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "mds", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "mon", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "osd", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr" 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": "16/23 daemons upgraded", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:13:21.937 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:13:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6cb806e680 msgr2=0x7f6cb8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6cb806e680 0x7f6cb8070b40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc1014e0 tx=0x7f6cbc009260 comp rx=0 tx=0).stop 2024-09-17T00:13:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc068580 msgr2=0x7f6ccc0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc068580 0x7f6ccc0ffcf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8000f80 tx=0x7f6cc800bc60 comp rx=0 tx=0).stop 2024-09-17T00:13:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 shutdown_connections 2024-09-17T00:13:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f6cb806e680 0x7f6cb8070b40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ccc068580 0x7f6ccc0ffcf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 --2- 172.21.15.42:0/4024499151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ccc106cd0 0x7f6ccc100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.940+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 >> 172.21.15.42:0/4024499151 conn(0x7f6ccc075670 msgr2=0x7f6ccc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.941+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 shutdown_connections 2024-09-17T00:13:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:21.941+0000 7f6cd38ea700 1 -- 172.21.15.42:0/4024499151 wait complete. 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 -- 172.21.15.42:0/908490917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 msgr2=0x7f462c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 --2- 172.21.15.42:0/908490917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c1070b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f461c009910 tx=0x7f461c009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 -- 172.21.15.42:0/908490917 shutdown_connections 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 --2- 172.21.15.42:0/908490917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f462c068580 0x7f462c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 --2- 172.21.15.42:0/908490917 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c1070b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 -- 172.21.15.42:0/908490917 >> 172.21.15.42:0/908490917 conn(0x7f462c075670 msgr2=0x7f462c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 -- 172.21.15.42:0/908490917 shutdown_connections 2024-09-17T00:13:22.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.048+0000 7f46324c4700 1 -- 172.21.15.42:0/908490917 wait complete. 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f46324c4700 1 Processor -- start 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f46324c4700 1 -- start start 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f46324c4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f462c068580 0x7f462c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f46324c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f46324c4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c19d230 con 0x7f462c106cd0 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f46324c4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c1966e0 con 0x7f462c068580 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f462b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:22.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f462b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53710/0 (socket says 172.21.15.42:53710) 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f462b7fe700 1 -- 172.21.15.42:0/1111138660 learned_addr learned my addr 172.21.15.42:0/1111138660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.049+0000 7f462bfff700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f462c068580 0x7f462c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f462b7fe700 1 -- 172.21.15.42:0/1111138660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f462c068580 msgr2=0x7f462c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f462b7fe700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f462c068580 0x7f462c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f462b7fe700 1 -- 172.21.15.42:0/1111138660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0095f0 con 0x7f462c106cd0 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f462b7fe700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c19cba0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f462000e920 tx=0x7f462000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f46297fa700 1 -- 172.21.15.42:0/1111138660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462000c9d0 con 0x7f462c106cd0 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c1969c0 con 0x7f462c106cd0 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f46297fa700 1 -- 172.21.15.42:0/1111138660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4620010430 con 0x7f462c106cd0 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f46297fa700 1 -- 172.21.15.42:0/1111138660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4620018630 con 0x7f462c106cd0 2024-09-17T00:13:22.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.050+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c196f10 con 0x7f462c106cd0 2024-09-17T00:13:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.051+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c1091d0 con 0x7f462c106cd0 2024-09-17T00:13:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.052+0000 7f46297fa700 1 -- 172.21.15.42:0/1111138660 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f46200105a0 con 0x7f462c106cd0 2024-09-17T00:13:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.052+0000 7f46297fa700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f461406e6d0 0x7f4614070b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.053+0000 7f46297fa700 1 -- 172.21.15.42:0/1111138660 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4620014070 con 0x7f462c106cd0 2024-09-17T00:13:22.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.053+0000 7f462bfff700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f461406e6d0 0x7f4614070b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:22.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.053+0000 7f462bfff700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f461406e6d0 0x7f4614070b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f461c0059d0 tx=0x7f461c005940 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:22.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.057+0000 7f46297fa700 1 -- 172.21.15.42:0/1111138660 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f46200541b0 con 0x7f462c106cd0 2024-09-17T00:13:22.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.353+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f462c04f2e0 con 0x7f462c106cd0 2024-09-17T00:13:22.351 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.354+0000 7f46297fa700 1 -- 172.21.15.42:0/1111138660 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3573 (secure 0 0 0) 0x7f462005b600 con 0x7f462c106cd0 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem with deprecated feature inline_data 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout: host smithi043 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi043/config 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 -e NODE_NAME=smithi043 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpo350o6bu:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 inventory --format=json-pretty --filter-for-batch 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout:Traceback (most recent call last): 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout: return _run_code(code, main_globals, None, 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-17T00:13:22.352 INFO:teuthology.orchestra.run.smithi042.stdout: exec(code, run_globals) 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/__main__.py", line 5579, in 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/__main__.py", line 5567, in main 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-17T00:13:22.353 INFO:teuthology.orchestra.run.smithi042.stdout: File "/tmp/tmpnroy8d0w.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-17T00:13:22.354 INFO:teuthology.orchestra.run.smithi042.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 -e NODE_NAME=smithi043 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968:/var/run/ceph:z -v /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpo350o6bu:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 inventory --format=json-pretty --filter-for-batch 2024-09-17T00:13:22.354 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:13:22.354 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:13:22.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f461406e6d0 msgr2=0x7f4614070b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:22.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f461406e6d0 0x7f4614070b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f461c0059d0 tx=0x7f461c005940 comp rx=0 tx=0).stop 2024-09-17T00:13:22.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 msgr2=0x7f462c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:22.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c19cba0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f462000e920 tx=0x7f462000ec30 comp rx=0 tx=0).stop 2024-09-17T00:13:22.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 shutdown_connections 2024-09-17T00:13:22.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f461406e6d0 0x7f4614070b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:22.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f462c068580 0x7f462c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:22.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 --2- 172.21.15.42:0/1111138660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f462c106cd0 0x7f462c19cba0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:22.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 >> 172.21.15.42:0/1111138660 conn(0x7f462c075670 msgr2=0x7f462c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:22.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 shutdown_connections 2024-09-17T00:13:22.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:22.356+0000 7f46324c4700 1 -- 172.21.15.42:0/1111138660 wait complete. 2024-09-17T00:13:22.934 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:22 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1111138660' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:13:23.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:22 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1111138660' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:13:23.957 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:23 smithi043 ceph-mon[81510]: pgmap v654: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-09-17T00:13:23.957 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:23 smithi043 ceph-mon[81510]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:23.957 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:23 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:23.957 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:23 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:23 smithi042 ceph-mon[111056]: pgmap v654: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 5.4 KiB/s wr, 17 op/s 2024-09-17T00:13:24.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:23 smithi042 ceph-mon[111056]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:24.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:23 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:24.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:23 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:25.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:25 smithi043 ceph-mon[81510]: pgmap v655: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-17T00:13:25.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:25 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:25.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:25 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:25 smithi042 ceph-mon[111056]: pgmap v655: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-17T00:13:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:25 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:25.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:25 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:27.336 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:27 smithi043 ceph-mon[81510]: pgmap v656: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-17T00:13:27.492 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:27 smithi042 ceph-mon[111056]: pgmap v656: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-17T00:13:28.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:28.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:28.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:28.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:28 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:28.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:28.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:28 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:30.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:29 smithi042 ceph-mon[111056]: pgmap v657: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-17T00:13:30.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:29 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:30.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:29 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:30.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:29 smithi043 ceph-mon[81510]: pgmap v657: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 11 KiB/s wr, 12 op/s 2024-09-17T00:13:30.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:29 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:30.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:29 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:31.875 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:31 smithi042 ceph-mon[111056]: pgmap v658: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-17T00:13:32.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:31 smithi043 ceph-mon[81510]: pgmap v658: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: pgmap v659: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: pgmap v660: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 3 op/s 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: Upgrade: Setting filesystem cephfs Joinable 2024-09-17T00:13:33.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-17T00:13:33.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:32 smithi042 ceph-mon[111056]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: pgmap v659: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.2 KiB/s rd, 5.3 KiB/s wr, 5 op/s 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: pgmap v660: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 3 op/s 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: Upgrade: Setting filesystem cephfs Joinable 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-17T00:13:33.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:32 smithi043 ceph-mon[81510]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-17T00:13:34.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi042.anvvuy=up:active} 2 up:standby 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all ceph-exporter 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi042"}]: dispatch 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi042"}]': finished 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi043"}]: dispatch 2024-09-17T00:13:34.274 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi043"}]': finished 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all iscsi 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all nfs 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: Upgrade: Setting container_image for all nvmeof 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.275 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: Upgrade: Finalizing container_image settings 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-17T00:13:34.276 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-17T00:13:34.277 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: Upgrade: Complete! 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-17T00:13:34.278 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:34.279 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-17T00:13:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: fsmap cephfs:2 {0=cephfs.smithi042.ctckas=up:active,1=cephfs.smithi042.anvvuy=up:active} 2 up:standby 2024-09-17T00:13:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all ceph-exporter 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi042"}]: dispatch 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi042"}]': finished 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi043"}]: dispatch 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi043"}]': finished 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all iscsi 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.379 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all nfs 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: Upgrade: Setting container_image for all nvmeof 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: Upgrade: Finalizing container_image settings 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-17T00:13:34.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-17T00:13:34.381 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-17T00:13:34.382 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: Upgrade: Complete! 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:13:34.383 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:13:34.384 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:13:34.384 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:35 smithi042 ceph-mon[111056]: pgmap v661: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:35 smithi043 ceph-mon[81510]: pgmap v661: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:37.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:37.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:13:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:37 smithi042 ceph-mon[111056]: pgmap v662: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:38.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:37 smithi043 ceph-mon[81510]: pgmap v662: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:40.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:39 smithi043 ceph-mon[81510]: pgmap v663: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:40.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:39 smithi042 ceph-mon[111056]: pgmap v663: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:42.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:41 smithi043 ceph-mon[81510]: pgmap v664: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:42.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:41 smithi042 ceph-mon[111056]: pgmap v664: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:44.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:43 smithi043 ceph-mon[81510]: pgmap v665: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:44.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:43 smithi042 ceph-mon[111056]: pgmap v665: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:45.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:44 smithi043 ceph-mon[81510]: pgmap v666: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:45.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:44 smithi042 ceph-mon[111056]: pgmap v666: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:47.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:46 smithi042 ceph-mon[111056]: pgmap v667: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:47.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:46 smithi043 ceph-mon[81510]: pgmap v667: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:49.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:48 smithi042 ceph-mon[111056]: pgmap v668: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:49.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:48 smithi043 ceph-mon[81510]: pgmap v668: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:51.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:50 smithi042 ceph-mon[111056]: pgmap v669: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:51.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:51.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:50 smithi043 ceph-mon[81510]: pgmap v669: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:51.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 -- 172.21.15.42:0/1084099060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78101e00 msgr2=0x7fca781021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 --2- 172.21.15.42:0/1084099060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78101e00 0x7fca781021e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fca68009910 tx=0x7fca68009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 -- 172.21.15.42:0/1084099060 shutdown_connections 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 --2- 172.21.15.42:0/1084099060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca78102720 0x7fca7810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 --2- 172.21.15.42:0/1084099060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78101e00 0x7fca781021e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 -- 172.21.15.42:0/1084099060 >> 172.21.15.42:0/1084099060 conn(0x7fca780fb650 msgr2=0x7fca780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 -- 172.21.15.42:0/1084099060 shutdown_connections 2024-09-17T00:13:52.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 -- 172.21.15.42:0/1084099060 wait complete. 2024-09-17T00:13:52.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 Processor -- start 2024-09-17T00:13:52.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.481+0000 7fca7da18700 1 -- start start 2024-09-17T00:13:52.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca7da18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca78101e00 0x7fca7819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:52.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca7da18700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78102720 0x7fca7819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:52.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca7da18700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7819d1d0 con 0x7fca78102720 2024-09-17T00:13:52.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca7da18700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca78196710 con 0x7fca78101e00 2024-09-17T00:13:52.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca6e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78102720 0x7fca7819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:52.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca6e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78102720 0x7fca7819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46558/0 (socket says 172.21.15.42:46558) 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca6e7fc700 1 -- 172.21.15.42:0/1647178671 learned_addr learned my addr 172.21.15.42:0/1647178671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca76ffd700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca78101e00 0x7fca7819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca6e7fc700 1 -- 172.21.15.42:0/1647178671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca78101e00 msgr2=0x7fca7819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca6e7fc700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca78101e00 0x7fca7819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.482+0000 7fca6e7fc700 1 -- 172.21.15.42:0/1647178671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca680095f0 con 0x7fca78102720 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.483+0000 7fca6e7fc700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78102720 0x7fca7819cbd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c8f0 tx=0x7fca6000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.483+0000 7fca7ca16700 1 -- 172.21.15.42:0/1647178671 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca600041f0 con 0x7fca78102720 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.483+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca781969f0 con 0x7fca78102720 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.483+0000 7fca7ca16700 1 -- 172.21.15.42:0/1647178671 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca60004350 con 0x7fca78102720 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.483+0000 7fca7ca16700 1 -- 172.21.15.42:0/1647178671 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca60003d00 con 0x7fca78102720 2024-09-17T00:13:52.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.483+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca78196f40 con 0x7fca78102720 2024-09-17T00:13:52.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.484+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca78108310 con 0x7fca78102720 2024-09-17T00:13:52.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.485+0000 7fca7ca16700 1 -- 172.21.15.42:0/1647178671 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fca60007740 con 0x7fca78102720 2024-09-17T00:13:52.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.485+0000 7fca7ca16700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca6406e680 0x7fca64070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:52.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.485+0000 7fca7ca16700 1 -- 172.21.15.42:0/1647178671 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca6008e330 con 0x7fca78102720 2024-09-17T00:13:52.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.485+0000 7fca76ffd700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca6406e680 0x7fca64070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:52.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.486+0000 7fca76ffd700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca6406e680 0x7fca64070b40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca68006010 tx=0x7fca6801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:52.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.490+0000 7fca7ca16700 1 -- 172.21.15.42:0/1647178671 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fca6005b000 con 0x7fca78102720 2024-09-17T00:13:52.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.719+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca781971c0 con 0x7fca6406e680 2024-09-17T00:13:52.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.720+0000 7fca7ca16700 1 -- 172.21.15.42:0/1647178671 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fca781971c0 con 0x7fca6406e680 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca6406e680 msgr2=0x7fca64070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca6406e680 0x7fca64070b40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca68006010 tx=0x7fca6801a040 comp rx=0 tx=0).stop 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78102720 msgr2=0x7fca7819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78102720 0x7fca7819cbd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fca6000c8f0 tx=0x7fca6000cc00 comp rx=0 tx=0).stop 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 shutdown_connections 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca6406e680 0x7fca64070b40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca78101e00 0x7fca7819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 --2- 172.21.15.42:0/1647178671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca78102720 0x7fca7819cbd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:52.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 >> 172.21.15.42:0/1647178671 conn(0x7fca780fb650 msgr2=0x7fca78105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:52.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 shutdown_connections 2024-09-17T00:13:52.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:52.722+0000 7fca7da18700 1 -- 172.21.15.42:0/1647178671 wait complete. 2024-09-17T00:13:53.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:52 smithi042 ceph-mon[111056]: pgmap v670: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:53.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:52 smithi042 ceph-mon[111056]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:53.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:52 smithi043 ceph-mon[81510]: pgmap v670: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:53.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:52 smithi043 ceph-mon[81510]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:53.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph orch ps' 2024-09-17T00:13:53.630 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 -- 172.21.15.42:0/379073136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54102720 msgr2=0x7fed5410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 --2- 172.21.15.42:0/379073136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54102720 0x7fed5410ac10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fed48009910 tx=0x7fed48009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 -- 172.21.15.42:0/379073136 shutdown_connections 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 --2- 172.21.15.42:0/379073136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54102720 0x7fed5410ac10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 --2- 172.21.15.42:0/379073136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed54101e00 0x7fed541021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 -- 172.21.15.42:0/379073136 >> 172.21.15.42:0/379073136 conn(0x7fed540fb650 msgr2=0x7fed540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 -- 172.21.15.42:0/379073136 shutdown_connections 2024-09-17T00:13:55.241 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.243+0000 7fed5989c700 1 -- 172.21.15.42:0/379073136 wait complete. 2024-09-17T00:13:55.242 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:54 smithi042 ceph-mon[111056]: pgmap v671: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:55.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed5989c700 1 Processor -- start 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed5989c700 1 -- start start 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed5989c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54101e00 0x7fed5419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed5989c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed54102720 0x7fed5419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed5989c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5419d260 con 0x7fed54101e00 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed5989c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed54196710 con 0x7fed54102720 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed52ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54101e00 0x7fed5419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed52ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54101e00 0x7fed5419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53574/0 (socket says 172.21.15.42:53574) 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.244+0000 7fed52ffd700 1 -- 172.21.15.42:0/2758947500 learned_addr learned my addr 172.21.15.42:0/2758947500 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed527fc700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed54102720 0x7fed5419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed52ffd700 1 -- 172.21.15.42:0/2758947500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed54102720 msgr2=0x7fed5419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:55.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed52ffd700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed54102720 0x7fed5419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:55.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed52ffd700 1 -- 172.21.15.42:0/2758947500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed480095f0 con 0x7fed54101e00 2024-09-17T00:13:55.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed52ffd700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54101e00 0x7fed5419c690 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d8d0 tx=0x7fed4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed5889a700 1 -- 172.21.15.42:0/2758947500 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed44009940 con 0x7fed54101e00 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed5889a700 1 -- 172.21.15.42:0/2758947500 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed44010460 con 0x7fed54101e00 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed541969f0 con 0x7fed54101e00 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed5889a700 1 -- 172.21.15.42:0/2758947500 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed4400f5f0 con 0x7fed54101e00 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.245+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54196f10 con 0x7fed54101e00 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.246+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed54108310 con 0x7fed54101e00 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.247+0000 7fed5889a700 1 -- 172.21.15.42:0/2758947500 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fed440105d0 con 0x7fed54101e00 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.248+0000 7fed5889a700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed3c06e680 0x7fed3c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:55.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.248+0000 7fed5889a700 1 -- 172.21.15.42:0/2758947500 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fed4408e680 con 0x7fed54101e00 2024-09-17T00:13:55.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.248+0000 7fed527fc700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed3c06e680 0x7fed3c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:55.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.248+0000 7fed527fc700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed3c06e680 0x7fed3c070b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed48006010 tx=0x7fed4801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:55.251 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.252+0000 7fed5889a700 1 -- 172.21.15.42:0/2758947500 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fed4405b350 con 0x7fed54101e00 2024-09-17T00:13:55.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:54 smithi043 ceph-mon[81510]: pgmap v671: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:55.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.472+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed541971f0 con 0x7fed3c06e680 2024-09-17T00:13:55.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.482+0000 7fed5889a700 1 -- 172.21.15.42:0/2758947500 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fed541971f0 con 0x7fed3c06e680 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (22m) 27s ago 33m 25.1M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (40s) 27s ago 33m 7684k - 19.1.1-330-gc2ddf972 ebb0615149de 21ee7c047725 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (34s) 31s ago 32m 7331k - 19.1.1-330-gc2ddf972 ebb0615149de af7fe87f7941 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (20m) 27s ago 33m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (20m) 31s ago 32m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (22m) 27s ago 33m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (16m) 27s ago 27m 63.5M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:13:55.480 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (16m) 27s ago 27m 65.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (16m) 31s ago 27m 20.3M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (15m) 31s ago 27m 19.6M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (24m) 27s ago 34m 511M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (23m) 31s ago 32m 448M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (21m) 27s ago 34m 76.9M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (20m) 31s ago 32m 52.3M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (23m) 27s ago 33m 16.2M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (23m) 31s ago 32m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (19m) 27s ago 31m 101M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (19m) 27s ago 30m 99.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:13:55.481 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (18m) 27s ago 30m 90.3M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:13:55.482 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (17m) 31s ago 30m 105M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:13:55.482 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (17m) 31s ago 29m 103M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:13:55.482 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (16m) 31s ago 29m 98.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:13:55.482 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (22m) 27s ago 32m 86.0M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:13:55.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed3c06e680 msgr2=0x7fed3c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed3c06e680 0x7fed3c070b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fed48006010 tx=0x7fed4801a040 comp rx=0 tx=0).stop 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54101e00 msgr2=0x7fed5419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54101e00 0x7fed5419c690 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d8d0 tx=0x7fed4400dc90 comp rx=0 tx=0).stop 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 shutdown_connections 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fed3c06e680 0x7fed3c070b40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed54101e00 0x7fed5419c690 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 --2- 172.21.15.42:0/2758947500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed54102720 0x7fed5419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:55.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 >> 172.21.15.42:0/2758947500 conn(0x7fed540fb650 msgr2=0x7fed54105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:55.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 shutdown_connections 2024-09-17T00:13:55.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:55.484+0000 7fed5989c700 1 -- 172.21.15.42:0/2758947500 wait complete. 2024-09-17T00:13:56.150 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:55 smithi042 ceph-mon[111056]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:56.172 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph orch upgrade status' 2024-09-17T00:13:56.287 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:55 smithi043 ceph-mon[81510]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:56.383 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:13:57.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:56 smithi042 ceph-mon[111056]: pgmap v672: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:57.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:56 smithi043 ceph-mon[81510]: pgmap v672: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:58.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 -- 172.21.15.42:0/1815725095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 msgr2=0x7fef40100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:58.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 --2- 172.21.15.42:0/1815725095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef40100180 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fef30009910 tx=0x7fef30009c20 comp rx=0 tx=0).stop 2024-09-17T00:13:58.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 -- 172.21.15.42:0/1815725095 shutdown_connections 2024-09-17T00:13:58.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 --2- 172.21.15.42:0/1815725095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef40100750 0x7fef401047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:58.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 --2- 172.21.15.42:0/1815725095 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef40100180 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:58.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 -- 172.21.15.42:0/1815725095 >> 172.21.15.42:0/1815725095 conn(0x7fef400fb650 msgr2=0x7fef400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:58.142 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 -- 172.21.15.42:0/1815725095 shutdown_connections 2024-09-17T00:13:58.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.144+0000 7fef44cc6700 1 -- 172.21.15.42:0/1815725095 wait complete. 2024-09-17T00:13:58.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef44cc6700 1 Processor -- start 2024-09-17T00:13:58.143 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef44cc6700 1 -- start start 2024-09-17T00:13:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef44cc6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef4019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef44cc6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef40100750 0x7fef4019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef44cc6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4019d2c0 con 0x7fef400ffda0 2024-09-17T00:13:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef44cc6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40196770 con 0x7fef40100750 2024-09-17T00:13:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef3e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef4019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:58.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef3e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef4019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53594/0 (socket says 172.21.15.42:53594) 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef3e59c700 1 -- 172.21.15.42:0/935070524 learned_addr learned my addr 172.21.15.42:0/935070524 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.145+0000 7fef3dd9b700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef40100750 0x7fef4019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef3e59c700 1 -- 172.21.15.42:0/935070524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef40100750 msgr2=0x7fef4019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef3e59c700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef40100750 0x7fef4019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef3e59c700 1 -- 172.21.15.42:0/935070524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300095f0 con 0x7fef400ffda0 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef3dd9b700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef40100750 0x7fef4019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef3e59c700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef4019c6f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fef30006010 tx=0x7fef3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef2f7fe700 1 -- 172.21.15.42:0/935070524 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef3001d070 con 0x7fef400ffda0 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef2f7fe700 1 -- 172.21.15.42:0/935070524 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef30004fb0 con 0x7fef400ffda0 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef401969f0 con 0x7fef400ffda0 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef2f7fe700 1 -- 172.21.15.42:0/935070524 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef30022ea0 con 0x7fef400ffda0 2024-09-17T00:13:58.145 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.146+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef40196ee0 con 0x7fef400ffda0 2024-09-17T00:13:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.147+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4004f2e0 con 0x7fef400ffda0 2024-09-17T00:13:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.148+0000 7fef2f7fe700 1 -- 172.21.15.42:0/935070524 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fef30022470 con 0x7fef400ffda0 2024-09-17T00:13:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.148+0000 7fef2f7fe700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fef2806e680 0x7fef28070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:13:58.146 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.148+0000 7fef2f7fe700 1 -- 172.21.15.42:0/935070524 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fef3008fbf0 con 0x7fef400ffda0 2024-09-17T00:13:58.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.148+0000 7fef3dd9b700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fef2806e680 0x7fef28070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:13:58.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.149+0000 7fef3dd9b700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fef2806e680 0x7fef28070b40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef40197be0 tx=0x7fef34005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:13:58.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.153+0000 7fef2f7fe700 1 -- 172.21.15.42:0/935070524 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fef3005c970 con 0x7fef400ffda0 2024-09-17T00:13:58.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.381+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef40197150 con 0x7fef2806e680 2024-09-17T00:13:58.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.382+0000 7fef2f7fe700 1 -- 172.21.15.42:0/935070524 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fef40197150 con 0x7fef2806e680 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "target_image": null, 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "in_progress": false, 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "which": "", 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "services_complete": [], 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "progress": null, 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "message": "", 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout: "is_paused": false 2024-09-17T00:13:58.380 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:13:58.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.384+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fef2806e680 msgr2=0x7fef28070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:58.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.384+0000 7fef44cc6700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fef2806e680 0x7fef28070b40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fef40197be0 tx=0x7fef34005f50 comp rx=0 tx=0).stop 2024-09-17T00:13:58.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.384+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 msgr2=0x7fef4019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:13:58.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.384+0000 7fef44cc6700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef4019c6f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fef30006010 tx=0x7fef3000be20 comp rx=0 tx=0).stop 2024-09-17T00:13:58.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.385+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 shutdown_connections 2024-09-17T00:13:58.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.385+0000 7fef44cc6700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fef2806e680 0x7fef28070b40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:58.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.385+0000 7fef44cc6700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef400ffda0 0x7fef4019c6f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:58.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.385+0000 7fef44cc6700 1 --2- 172.21.15.42:0/935070524 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef40100750 0x7fef4019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:13:58.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.385+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 >> 172.21.15.42:0/935070524 conn(0x7fef400fb650 msgr2=0x7fef40103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:13:58.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.385+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 shutdown_connections 2024-09-17T00:13:58.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:13:58.385+0000 7fef44cc6700 1 -- 172.21.15.42:0/935070524 wait complete. 2024-09-17T00:13:59.051 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph health detail' 2024-09-17T00:13:59.259 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:13:59.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:59 smithi042 ceph-mon[111056]: pgmap v673: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:59.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:13:59 smithi042 ceph-mon[111056]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:13:59.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:59 smithi043 ceph-mon[81510]: pgmap v673: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:13:59.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:13:59 smithi043 ceph-mon[81510]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:14:00.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 -- 172.21.15.42:0/301019713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254068580 msgr2=0x7f12540689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 --2- 172.21.15.42:0/301019713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254068580 0x7f12540689e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1248009910 tx=0x7f1248009c20 comp rx=0 tx=0).stop 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 -- 172.21.15.42:0/301019713 shutdown_connections 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 --2- 172.21.15.42:0/301019713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254068580 0x7f12540689e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 --2- 172.21.15.42:0/301019713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254106cd0 0x7f12541070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 -- 172.21.15.42:0/301019713 >> 172.21.15.42:0/301019713 conn(0x7f1254075670 msgr2=0x7f1254075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 -- 172.21.15.42:0/301019713 shutdown_connections 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.984+0000 7f125a2fe700 1 -- 172.21.15.42:0/301019713 wait complete. 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.985+0000 7f125a2fe700 1 Processor -- start 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.985+0000 7f125a2fe700 1 -- start start 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.985+0000 7f125a2fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254068580 0x7f125419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:14:00.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f125a2fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254106cd0 0x7f125419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f125a2fe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125419d200 con 0x7f1254106cd0 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f125a2fe700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541966b0 con 0x7f1254068580 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254106cd0 0x7f125419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254106cd0 0x7f125419cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53600/0 (socket says 172.21.15.42:53600) 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12537fe700 1 -- 172.21.15.42:0/1015364878 learned_addr learned my addr 172.21.15.42:0/1015364878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f1253fff700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254068580 0x7f125419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12537fe700 1 -- 172.21.15.42:0/1015364878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254068580 msgr2=0x7f125419c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12537fe700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254068580 0x7f125419c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12537fe700 1 -- 172.21.15.42:0/1015364878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480095f0 con 0x7f1254106cd0 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12537fe700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254106cd0 0x7f125419cb70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1248006010 tx=0x7f124800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12517fa700 1 -- 172.21.15.42:0/1015364878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f124801d070 con 0x7f1254106cd0 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12517fa700 1 -- 172.21.15.42:0/1015364878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f124800f460 con 0x7f1254106cd0 2024-09-17T00:14:00.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f12517fa700 1 -- 172.21.15.42:0/1015364878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1248017630 con 0x7f1254106cd0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1254196930 con 0x7f1254106cd0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.986+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1254196e20 con 0x7f1254106cd0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.987+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12541091d0 con 0x7f1254106cd0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.988+0000 7f12517fa700 1 -- 172.21.15.42:0/1015364878 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f124800f5d0 con 0x7f1254106cd0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.989+0000 7f12517fa700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f123c06e680 0x7f123c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.989+0000 7f1253fff700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f123c06e680 0x7f123c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.989+0000 7f12517fa700 1 -- 172.21.15.42:0/1015364878 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f124808fab0 con 0x7f1254106cd0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.989+0000 7f1253fff700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f123c06e680 0x7f123c070b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1244005950 tx=0x7f124400a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:14:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:00.993+0000 7f12517fa700 1 -- 172.21.15.42:0/1015364878 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f124805c800 con 0x7f1254106cd0 2024-09-17T00:14:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:01 smithi042 ceph-mon[111056]: pgmap v674: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:01.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.288+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f125404f2e0 con 0x7f1254106cd0 2024-09-17T00:14:01.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.288+0000 7f12517fa700 1 -- 172.21.15.42:0/1015364878 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f124805c390 con 0x7f1254106cd0 2024-09-17T00:14:01.285 INFO:teuthology.orchestra.run.smithi042.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-17T00:14:01.286 INFO:teuthology.orchestra.run.smithi042.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:14:01.286 INFO:teuthology.orchestra.run.smithi042.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:14:01.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f123c06e680 msgr2=0x7f123c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f123c06e680 0x7f123c070b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1244005950 tx=0x7f124400a300 comp rx=0 tx=0).stop 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254106cd0 msgr2=0x7f125419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254106cd0 0x7f125419cb70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1248006010 tx=0x7f124800bd70 comp rx=0 tx=0).stop 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 shutdown_connections 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f123c06e680 0x7f123c070b40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1254068580 0x7f125419c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 --2- 172.21.15.42:0/1015364878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1254106cd0 0x7f125419cb70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 >> 172.21.15.42:0/1015364878 conn(0x7f1254075670 msgr2=0x7f12540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 shutdown_connections 2024-09-17T00:14:01.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:01.290+0000 7f125a2fe700 1 -- 172.21.15.42:0/1015364878 wait complete. 2024-09-17T00:14:01.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:01 smithi043 ceph-mon[81510]: pgmap v674: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:01.991 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph versions' 2024-09-17T00:14:02.203 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:14:02.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:02 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1015364878' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:14:02.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:02 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1015364878' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-17T00:14:03.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:03 smithi042 ceph-mon[111056]: pgmap v675: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:03.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:03 smithi043 ceph-mon[81510]: pgmap v675: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:04.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 -- 172.21.15.42:0/1884235747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 msgr2=0x7f5978100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:04.005 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 --2- 172.21.15.42:0/1884235747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f5978100180 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5974009910 tx=0x7f5974009c20 comp rx=0 tx=0).stop 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 -- 172.21.15.42:0/1884235747 shutdown_connections 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 --2- 172.21.15.42:0/1884235747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5978100750 0x7f59781047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 --2- 172.21.15.42:0/1884235747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f5978100180 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 -- 172.21.15.42:0/1884235747 >> 172.21.15.42:0/1884235747 conn(0x7f59780fb650 msgr2=0x7f59780fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 -- 172.21.15.42:0/1884235747 shutdown_connections 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.006+0000 7f597fca8700 1 -- 172.21.15.42:0/1884235747 wait complete. 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.007+0000 7f597fca8700 1 Processor -- start 2024-09-17T00:14:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.007+0000 7f597fca8700 1 -- start start 2024-09-17T00:14:04.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.007+0000 7f597fca8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f597819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:14:04.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.007+0000 7f597fca8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5978100750 0x7f597819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.007+0000 7f597fca8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819d250 con 0x7f59780ffda0 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.007+0000 7f597fca8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978196700 con 0x7f5978100750 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597da44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f597819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597da44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f597819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40010/0 (socket says 172.21.15.42:40010) 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597da44700 1 -- 172.21.15.42:0/937415582 learned_addr learned my addr 172.21.15.42:0/937415582 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597d243700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5978100750 0x7f597819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597da44700 1 -- 172.21.15.42:0/937415582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5978100750 msgr2=0x7f597819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597da44700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5978100750 0x7f597819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597da44700 1 -- 172.21.15.42:0/937415582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740095f0 con 0x7f59780ffda0 2024-09-17T00:14:04.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.008+0000 7f597da44700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f597819c680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5974006010 tx=0x7f597400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:14:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.010+0000 7f596affd700 1 -- 172.21.15.42:0/937415582 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597401d070 con 0x7f59780ffda0 2024-09-17T00:14:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.010+0000 7f596affd700 1 -- 172.21.15.42:0/937415582 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f597400f460 con 0x7f59780ffda0 2024-09-17T00:14:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.010+0000 7f596affd700 1 -- 172.21.15.42:0/937415582 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5974017610 con 0x7f59780ffda0 2024-09-17T00:14:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.010+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5978196980 con 0x7f59780ffda0 2024-09-17T00:14:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.010+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978196e70 con 0x7f59780ffda0 2024-09-17T00:14:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.011+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5978108140 con 0x7f59780ffda0 2024-09-17T00:14:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.012+0000 7f596affd700 1 -- 172.21.15.42:0/937415582 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f597400f5d0 con 0x7f59780ffda0 2024-09-17T00:14:04.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.012+0000 7f596affd700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f596406e680 0x7f5964070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:14:04.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.012+0000 7f596affd700 1 -- 172.21.15.42:0/937415582 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f597408fb70 con 0x7f59780ffda0 2024-09-17T00:14:04.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.012+0000 7f597d243700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f596406e680 0x7f5964070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:14:04.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.013+0000 7f597d243700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f596406e680 0x7f5964070b40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5978197b70 tx=0x7f596c00a300 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:14:04.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.017+0000 7f596affd700 1 -- 172.21.15.42:0/937415582 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f597405c8c0 con 0x7f59780ffda0 2024-09-17T00:14:04.312 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.314+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5978197cb0 con 0x7f59780ffda0 2024-09-17T00:14:04.313 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.314+0000 7f596affd700 1 -- 172.21.15.42:0/937415582 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f597405c450 con 0x7f59780ffda0 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:14:04.317 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:14:04.318 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:14:04.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:14:04.318 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:14:04.318 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:14:04.318 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:14:04.318 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f596406e680 msgr2=0x7f5964070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f596406e680 0x7f5964070b40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5978197b70 tx=0x7f596c00a300 comp rx=0 tx=0).stop 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 msgr2=0x7f597819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f597819c680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f5974006010 tx=0x7f597400bd70 comp rx=0 tx=0).stop 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 shutdown_connections 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f596406e680 0x7f5964070b40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f59780ffda0 0x7f597819c680 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.316+0000 7f597fca8700 1 --2- 172.21.15.42:0/937415582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5978100750 0x7f597819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.317+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 >> 172.21.15.42:0/937415582 conn(0x7f59780fb650 msgr2=0x7f59780fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:14:04.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.317+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 shutdown_connections 2024-09-17T00:14:04.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:14:04.317+0000 7f597fca8700 1 -- 172.21.15.42:0/937415582 wait complete. 2024-09-17T00:14:04.975 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-17T00:14:05.185 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:14:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:05 smithi042 ceph-mon[111056]: pgmap v676: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:05.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:05 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/937415582' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:14:05.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:05 smithi043 ceph-mon[81510]: pgmap v676: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:05.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:05 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/937415582' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:14:06.214 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:06.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:06.913 INFO:teuthology.orchestra.run.smithi042.stdout:wait for servicemap items w/ changing names to refresh 2024-09-17T00:14:07.267 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:07 smithi042 ceph-mon[111056]: pgmap v677: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:07.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:07 smithi043 ceph-mon[81510]: pgmap v677: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:07.598 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'sleep 60' 2024-09-17T00:14:07.814 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:14:09.236 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:09 smithi042 ceph-mon[111056]: pgmap v678: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:09.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:09 smithi043 ceph-mon[81510]: pgmap v678: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:11.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:11 smithi042 ceph-mon[111056]: pgmap v679: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:11.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:11 smithi043 ceph-mon[81510]: pgmap v679: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:13.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:13 smithi042 ceph-mon[111056]: pgmap v680: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:13.305 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:13 smithi043 ceph-mon[81510]: pgmap v680: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:15.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:15 smithi043 ceph-mon[81510]: pgmap v681: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:15.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:15 smithi042 ceph-mon[111056]: pgmap v681: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:17.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:17 smithi043 ceph-mon[81510]: pgmap v682: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:17.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:17 smithi042 ceph-mon[111056]: pgmap v682: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:19.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:19 smithi043 ceph-mon[81510]: pgmap v683: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:19.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:19 smithi042 ceph-mon[111056]: pgmap v683: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:21.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:21 smithi043 ceph-mon[81510]: pgmap v684: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:21.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:21.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:21 smithi042 ceph-mon[111056]: pgmap v684: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:21.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:23.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:23 smithi043 ceph-mon[81510]: pgmap v685: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:23.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:23 smithi042 ceph-mon[111056]: pgmap v685: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:25.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:25 smithi043 ceph-mon[81510]: pgmap v686: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:25.707 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:25 smithi042 ceph-mon[111056]: pgmap v686: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:27.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:27 smithi043 ceph-mon[81510]: pgmap v687: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:27.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:27 smithi042 ceph-mon[111056]: pgmap v687: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:29.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:29 smithi043 ceph-mon[81510]: pgmap v688: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:29.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:29 smithi042 ceph-mon[111056]: pgmap v688: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:31.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:31 smithi043 ceph-mon[81510]: pgmap v689: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:31.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:31 smithi042 ceph-mon[111056]: pgmap v689: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:33.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:33 smithi043 ceph-mon[81510]: pgmap v690: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:33.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:33 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:14:33.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:33 smithi042 ceph-mon[111056]: pgmap v690: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:33.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:33 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:14:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:14:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:14:34.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:14:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:14:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:14:34.773 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:14:36.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:35 smithi042 ceph-mon[111056]: pgmap v691: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:36.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:35 smithi043 ceph-mon[81510]: pgmap v691: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:37 smithi042 ceph-mon[111056]: pgmap v692: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:38.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:37 smithi043 ceph-mon[81510]: pgmap v692: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:39 smithi042 ceph-mon[111056]: pgmap v693: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:40.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:39 smithi043 ceph-mon[81510]: pgmap v693: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:42.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:41 smithi042 ceph-mon[111056]: pgmap v694: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:42.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:41 smithi043 ceph-mon[81510]: pgmap v694: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:44.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:43 smithi042 ceph-mon[111056]: pgmap v695: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:44.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:43 smithi043 ceph-mon[81510]: pgmap v695: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:46.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:45 smithi042 ceph-mon[111056]: pgmap v696: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:46.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:45 smithi043 ceph-mon[81510]: pgmap v696: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:48.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:47 smithi042 ceph-mon[111056]: pgmap v697: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:48.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:47 smithi043 ceph-mon[81510]: pgmap v697: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:50.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:49 smithi042 ceph-mon[111056]: pgmap v698: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:50.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:49 smithi043 ceph-mon[81510]: pgmap v698: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:51.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:14:52.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:51 smithi042 ceph-mon[111056]: pgmap v699: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:52.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:51 smithi043 ceph-mon[81510]: pgmap v699: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:54.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:53 smithi042 ceph-mon[111056]: pgmap v700: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:54.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:53 smithi043 ceph-mon[81510]: pgmap v700: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:56.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:55 smithi042 ceph-mon[111056]: pgmap v701: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:56.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:55 smithi043 ceph-mon[81510]: pgmap v701: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:58.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:57 smithi042 ceph-mon[111056]: pgmap v702: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:14:58.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:57 smithi043 ceph-mon[81510]: pgmap v702: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:00.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:14:59 smithi042 ceph-mon[111056]: pgmap v703: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:14:59 smithi043 ceph-mon[81510]: pgmap v703: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:01 smithi042 ceph-mon[111056]: pgmap v704: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:02.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:01 smithi043 ceph-mon[81510]: pgmap v704: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:03 smithi042 ceph-mon[111056]: pgmap v705: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:04.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:03 smithi043 ceph-mon[81510]: pgmap v705: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:06.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:05 smithi042 ceph-mon[111056]: pgmap v706: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:06.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:06.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:05 smithi043 ceph-mon[81510]: pgmap v706: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:07.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:07 smithi043 ceph-mon[81510]: pgmap v707: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:08.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:07 smithi042 ceph-mon[111056]: pgmap v707: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:09 smithi042 ceph-mon[111056]: pgmap v708: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:10.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:09 smithi043 ceph-mon[81510]: pgmap v708: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:10.135 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph orch ps' 2024-09-17T00:15:10.355 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:12.059 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:11 smithi042 ceph-mon[111056]: pgmap v709: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:12.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:11 smithi043 ceph-mon[81510]: pgmap v709: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:12.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 -- 172.21.15.42:0/3338155727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 msgr2=0x7fa5e8105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:12.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/3338155727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e8105890 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009910 tx=0x7fa5d4009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:12.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 -- 172.21.15.42:0/3338155727 shutdown_connections 2024-09-17T00:15:12.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/3338155727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e8105890 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/3338155727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8100e90 0x7fa5e8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 -- 172.21.15.42:0/3338155727 >> 172.21.15.42:0/3338155727 conn(0x7fa5e80fc740 msgr2=0x7fa5e80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 -- 172.21.15.42:0/3338155727 shutdown_connections 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 -- 172.21.15.42:0/3338155727 wait complete. 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.177+0000 7fa5eecb0700 1 Processor -- start 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5eecb0700 1 -- start start 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5eecb0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8100e90 0x7fa5e8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5eecb0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5eecb0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8073b10 con 0x7fa5e8101840 2024-09-17T00:15:12.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5eecb0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8073c50 con 0x7fa5e8100e90 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5e7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5e7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53514/0 (socket says 172.21.15.42:53514) 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5e7fff700 1 -- 172.21.15.42:0/2417679474 learned_addr learned my addr 172.21.15.42:0/2417679474 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5eca4c700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8100e90 0x7fa5e8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5e7fff700 1 -- 172.21.15.42:0/2417679474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8100e90 msgr2=0x7fa5e8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5e7fff700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8100e90 0x7fa5e8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.178+0000 7fa5e7fff700 1 -- 172.21.15.42:0/2417679474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40095f0 con 0x7fa5e8101840 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.179+0000 7fa5e7fff700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e80735d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400bda0 tx=0x7fa5d400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.179+0000 7fa5e5ffb700 1 -- 172.21.15.42:0/2417679474 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d401d070 con 0x7fa5e8101840 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.179+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e8073ed0 con 0x7fa5e8101840 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.179+0000 7fa5e5ffb700 1 -- 172.21.15.42:0/2417679474 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5d4005dc0 con 0x7fa5e8101840 2024-09-17T00:15:12.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.179+0000 7fa5e5ffb700 1 -- 172.21.15.42:0/2417679474 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5d4017400 con 0x7fa5e8101840 2024-09-17T00:15:12.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.179+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e81a2da0 con 0x7fa5e8101840 2024-09-17T00:15:12.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.180+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e8109290 con 0x7fa5e8101840 2024-09-17T00:15:12.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.181+0000 7fa5e5ffb700 1 -- 172.21.15.42:0/2417679474 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fa5d400f460 con 0x7fa5e8101840 2024-09-17T00:15:12.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.181+0000 7fa5e5ffb700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5d806e680 0x7fa5d8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:12.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.181+0000 7fa5e5ffb700 1 -- 172.21.15.42:0/2417679474 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa5d408fa30 con 0x7fa5e8101840 2024-09-17T00:15:12.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.181+0000 7fa5eca4c700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5d806e680 0x7fa5d8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:12.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.182+0000 7fa5eca4c700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5d806e680 0x7fa5d8070b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc005950 tx=0x7fa5dc00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.190+0000 7fa5e5ffb700 1 -- 172.21.15.42:0/2417679474 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa5d4058bd0 con 0x7fa5e8101840 2024-09-17T00:15:12.458 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.460+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 --> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5e80611d0 con 0x7fa5d806e680 2024-09-17T00:15:12.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.470+0000 7fa5e5ffb700 1 -- 172.21.15.42:0/2417679474 <== mgr.34102 v2:172.21.15.42:6800/970153898 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa5e80611d0 con 0x7fa5d806e680 2024-09-17T00:15:12.468 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-17T00:15:12.468 INFO:teuthology.orchestra.run.smithi042.stdout:alertmanager.smithi042 smithi042 *:9093,9094 running (23m) 104s ago 34m 25.1M - 0.25.0 c8568f914cd2 e033e5132bea 2024-09-17T00:15:12.468 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi042 smithi042 running (117s) 104s ago 34m 7684k - 19.1.1-330-gc2ddf972 ebb0615149de 21ee7c047725 2024-09-17T00:15:12.468 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-exporter.smithi043 smithi043 running (111s) 108s ago 33m 7331k - 19.1.1-330-gc2ddf972 ebb0615149de af7fe87f7941 2024-09-17T00:15:12.468 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi042 smithi042 running (21m) 104s ago 34m 6895k - 19.1.1-330-gc2ddf972 ebb0615149de 14a8e2e0ad95 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:crash.smithi043 smithi043 running (21m) 108s ago 33m 6899k - 19.1.1-330-gc2ddf972 ebb0615149de cecace00ebd6 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:grafana.smithi042 smithi042 *:3000 running (23m) 104s ago 34m 80.4M - 9.4.12 2bacad6d85d8 c2b535cf1431 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.anvvuy smithi042 running (17m) 104s ago 28m 63.5M - 19.1.1-330-gc2ddf972 ebb0615149de f32013b40467 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi042.ctckas smithi042 running (17m) 104s ago 29m 65.5M - 19.1.1-330-gc2ddf972 ebb0615149de fdefd2135134 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.pzcdzg smithi043 running (17m) 108s ago 28m 20.3M - 19.1.1-330-gc2ddf972 ebb0615149de fe0c2777242f 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mds.cephfs.smithi043.riwwhn smithi043 running (17m) 108s ago 28m 19.6M - 19.1.1-330-gc2ddf972 ebb0615149de 4c76dabd2e7b 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi042.zrkumz smithi042 *:8443,9283,8765 running (25m) 104s ago 36m 511M - 19.1.1-330-gc2ddf972 ebb0615149de a94913ffe7ef 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.smithi043.xveyou smithi043 *:8443,9283,8765 running (24m) 108s ago 33m 448M - 19.1.1-330-gc2ddf972 ebb0615149de 96531cec8d96 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi042 smithi042 running (22m) 104s ago 36m 76.9M 2048M 19.1.1-330-gc2ddf972 ebb0615149de 7ace5d09425f 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:mon.smithi043 smithi043 running (22m) 108s ago 33m 52.3M 2048M 19.1.1-330-gc2ddf972 ebb0615149de a860b22006ae 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi042 smithi042 *:9100 running (24m) 104s ago 34m 16.2M - 1.5.0 0da6a335fe13 a3bf13d9af6f 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:node-exporter.smithi043 smithi043 *:9100 running (24m) 108s ago 33m 15.3M - 1.5.0 0da6a335fe13 7067aedce447 2024-09-17T00:15:12.469 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (21m) 104s ago 32m 101M 1217M 19.1.1-330-gc2ddf972 ebb0615149de e3e56c422ec5 2024-09-17T00:15:12.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi042 running (20m) 104s ago 32m 99.5M 1217M 19.1.1-330-gc2ddf972 ebb0615149de c378ed398a4d 2024-09-17T00:15:12.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi042 running (19m) 104s ago 31m 90.3M 1217M 19.1.1-330-gc2ddf972 ebb0615149de 61db938df01d 2024-09-17T00:15:12.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi043 running (19m) 108s ago 31m 105M 2241M 19.1.1-330-gc2ddf972 ebb0615149de 47a2dcb81243 2024-09-17T00:15:12.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi043 running (18m) 108s ago 30m 103M 2241M 19.1.1-330-gc2ddf972 ebb0615149de bd894ca62c9c 2024-09-17T00:15:12.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi043 running (18m) 108s ago 30m 98.9M 2241M 19.1.1-330-gc2ddf972 ebb0615149de dc14c8f3f72c 2024-09-17T00:15:12.470 INFO:teuthology.orchestra.run.smithi042.stdout:prometheus.smithi042 smithi042 *:9095 running (24m) 104s ago 34m 86.0M - 2.43.0 a07b618ecd1d a08efc1fc007 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5d806e680 msgr2=0x7fa5d8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5d806e680 0x7fa5d8070b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc005950 tx=0x7fa5dc00b410 comp rx=0 tx=0).stop 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 msgr2=0x7fa5e80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e80735d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa5d400bda0 tx=0x7fa5d400bdd0 comp rx=0 tx=0).stop 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 shutdown_connections 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fa5d806e680 0x7fa5d8070b40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5e8100e90 0x7fa5e8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:12.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 --2- 172.21.15.42:0/2417679474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa5e8101840 0x7fa5e80735d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:12.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 >> 172.21.15.42:0/2417679474 conn(0x7fa5e80fc740 msgr2=0x7fa5e80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:12.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 shutdown_connections 2024-09-17T00:15:12.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:12.473+0000 7fa5eecb0700 1 -- 172.21.15.42:0/2417679474 wait complete. 2024-09-17T00:15:13.169 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph versions' 2024-09-17T00:15:13.381 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:14.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:13 smithi042 ceph-mon[111056]: pgmap v710: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:14.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:13 smithi042 ceph-mon[111056]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:15:14.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:13 smithi043 ceph-mon[81510]: pgmap v710: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:14.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:13 smithi043 ceph-mon[81510]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-17T00:15:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.992+0000 7fce90767700 1 -- 172.21.15.42:0/2150316958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 msgr2=0x7fce88100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.992+0000 7fce90767700 1 --2- 172.21.15.42:0/2150316958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce88100180 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fce8400b0a0 tx=0x7fce8400b3b0 comp rx=0 tx=0).stop 2024-09-17T00:15:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 -- 172.21.15.42:0/2150316958 shutdown_connections 2024-09-17T00:15:14.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 --2- 172.21.15.42:0/2150316958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88100750 0x7fce881047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 --2- 172.21.15.42:0/2150316958 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce88100180 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 -- 172.21.15.42:0/2150316958 >> 172.21.15.42:0/2150316958 conn(0x7fce880fb650 msgr2=0x7fce880fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 -- 172.21.15.42:0/2150316958 shutdown_connections 2024-09-17T00:15:14.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 -- 172.21.15.42:0/2150316958 wait complete. 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 Processor -- start 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.993+0000 7fce90767700 1 -- start start 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce90767700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce8819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce90767700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88100750 0x7fce8819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce90767700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8819d230 con 0x7fce880ffda0 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce90767700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce881966e0 con 0x7fce88100750 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8e503700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce8819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:14.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8e503700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce8819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58632/0 (socket says 172.21.15.42:58632) 2024-09-17T00:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8e503700 1 -- 172.21.15.42:0/2920197854 learned_addr learned my addr 172.21.15.42:0/2920197854 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8dd02700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88100750 0x7fce8819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8e503700 1 -- 172.21.15.42:0/2920197854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88100750 msgr2=0x7fce8819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8e503700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88100750 0x7fce8819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8e503700 1 -- 172.21.15.42:0/2920197854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce84009d00 con 0x7fce880ffda0 2024-09-17T00:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.994+0000 7fce8e503700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce8819c660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fce84000f80 tx=0x7fce8400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:14.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.995+0000 7fce7f7fe700 1 -- 172.21.15.42:0/2920197854 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce8400e040 con 0x7fce880ffda0 2024-09-17T00:15:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.995+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce88196960 con 0x7fce880ffda0 2024-09-17T00:15:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.995+0000 7fce7f7fe700 1 -- 172.21.15.42:0/2920197854 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce84003c10 con 0x7fce880ffda0 2024-09-17T00:15:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.995+0000 7fce7f7fe700 1 -- 172.21.15.42:0/2920197854 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce8401d6b0 con 0x7fce880ffda0 2024-09-17T00:15:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.995+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce88196e50 con 0x7fce880ffda0 2024-09-17T00:15:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.996+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce88108140 con 0x7fce880ffda0 2024-09-17T00:15:14.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.996+0000 7fce7f7fe700 1 -- 172.21.15.42:0/2920197854 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fce84004040 con 0x7fce880ffda0 2024-09-17T00:15:14.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.997+0000 7fce7f7fe700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7406e680 0x7fce74070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:14.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.997+0000 7fce7f7fe700 1 -- 172.21.15.42:0/2920197854 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce8408fe00 con 0x7fce880ffda0 2024-09-17T00:15:14.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.997+0000 7fce8dd02700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7406e680 0x7fce74070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:14.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:14.997+0000 7fce8dd02700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7406e680 0x7fce74070b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fce88197b50 tx=0x7fce7800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:14.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.002+0000 7fce7f7fe700 1 -- 172.21.15.42:0/2920197854 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fce8405cad0 con 0x7fce880ffda0 2024-09-17T00:15:15.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.300+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce88197cf0 con 0x7fce880ffda0 2024-09-17T00:15:15.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.301+0000 7fce7f7fe700 1 -- 172.21.15.42:0/2920197854 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fce88197cf0 con 0x7fce880ffda0 2024-09-17T00:15:15.298 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-09-17T00:15:15.298 INFO:teuthology.orchestra.run.smithi042.stdout: "mon": { 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr": { 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 2 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "osd": { 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 6 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 4 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "overall": { 2024-09-17T00:15:15.299 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)": 14 2024-09-17T00:15:15.300 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-09-17T00:15:15.300 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7406e680 msgr2=0x7fce74070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7406e680 0x7fce74070b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fce88197b50 tx=0x7fce7800b410 comp rx=0 tx=0).stop 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 msgr2=0x7fce8819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce8819c660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fce84000f80 tx=0x7fce8400cd20 comp rx=0 tx=0).stop 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 shutdown_connections 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fce7406e680 0x7fce74070b40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce880ffda0 0x7fce8819c660 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 --2- 172.21.15.42:0/2920197854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce88100750 0x7fce8819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 >> 172.21.15.42:0/2920197854 conn(0x7fce880fb650 msgr2=0x7fce880fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 shutdown_connections 2024-09-17T00:15:15.301 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:15.303+0000 7fce90767700 1 -- 172.21.15.42:0/2920197854 wait complete. 2024-09-17T00:15:15.920 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:15 smithi042 ceph-mon[111056]: pgmap v711: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:15.921 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:15 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2920197854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:15:15.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-17T00:15:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:15 smithi043 ceph-mon[81510]: pgmap v711: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:16.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:15 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2920197854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:15:16.156 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 -- 172.21.15.42:0/1305947881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964102720 msgr2=0x7f396410ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 --2- 172.21.15.42:0/1305947881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964102720 0x7f396410ac10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3958009910 tx=0x7f3958009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 -- 172.21.15.42:0/1305947881 shutdown_connections 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 --2- 172.21.15.42:0/1305947881 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964102720 0x7f396410ac10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 --2- 172.21.15.42:0/1305947881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3964101e00 0x7f39641021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 -- 172.21.15.42:0/1305947881 >> 172.21.15.42:0/1305947881 conn(0x7f39640fb650 msgr2=0x7f39640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 -- 172.21.15.42:0/1305947881 shutdown_connections 2024-09-17T00:15:17.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 -- 172.21.15.42:0/1305947881 wait complete. 2024-09-17T00:15:17.849 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:17 smithi042 ceph-mon[111056]: pgmap v712: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:17.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 Processor -- start 2024-09-17T00:15:17.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.850+0000 7f396a664700 1 -- start start 2024-09-17T00:15:17.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f396a664700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964101e00 0x7f396419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:17.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f396a664700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3964102720 0x7f39641944e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:17.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f396a664700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3964194a20 con 0x7f3964101e00 2024-09-17T00:15:17.850 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f396a664700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3964194b90 con 0x7f3964102720 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f3963fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964101e00 0x7f396419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f3963fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964101e00 0x7f396419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58640/0 (socket says 172.21.15.42:58640) 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f3963fff700 1 -- 172.21.15.42:0/270819055 learned_addr learned my addr 172.21.15.42:0/270819055 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f39637fe700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3964102720 0x7f39641944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f3963fff700 1 -- 172.21.15.42:0/270819055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3964102720 msgr2=0x7f39641944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f3963fff700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3964102720 0x7f39641944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.851+0000 7f3963fff700 1 -- 172.21.15.42:0/270819055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580095f0 con 0x7f3964101e00 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.852+0000 7f3963fff700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964101e00 0x7f396419b4c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f395400e920 tx=0x7f395400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.852+0000 7f39617fa700 1 -- 172.21.15.42:0/270819055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f395400c9d0 con 0x7f3964101e00 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.852+0000 7f39617fa700 1 -- 172.21.15.42:0/270819055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3954010430 con 0x7f3964101e00 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.852+0000 7f39617fa700 1 -- 172.21.15.42:0/270819055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3954018630 con 0x7f3964101e00 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.852+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3964194e70 con 0x7f3964101e00 2024-09-17T00:15:17.851 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.852+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3964199060 con 0x7f3964101e00 2024-09-17T00:15:17.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.853+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396404f2e0 con 0x7f3964101e00 2024-09-17T00:15:17.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.854+0000 7f39617fa700 1 -- 172.21.15.42:0/270819055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f39540105a0 con 0x7f3964101e00 2024-09-17T00:15:17.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.854+0000 7f39617fa700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f394c06e680 0x7f394c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:17.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.854+0000 7f39617fa700 1 -- 172.21.15.42:0/270819055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3954014070 con 0x7f3964101e00 2024-09-17T00:15:17.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.854+0000 7f39637fe700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f394c06e680 0x7f394c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:17.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.855+0000 7f39637fe700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f394c06e680 0x7f394c070b40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3958009fd0 tx=0x7f3958005a90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:17.856 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:17.859+0000 7f39617fa700 1 -- 172.21.15.42:0/270819055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f39540541b0 con 0x7f3964101e00 2024-09-17T00:15:18.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:17 smithi043 ceph-mon[81510]: pgmap v712: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:18.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.154+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3964195a10 con 0x7f3964101e00 2024-09-17T00:15:18.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.155+0000 7f39617fa700 1 -- 172.21.15.42:0/270819055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3954057af0 con 0x7f3964101e00 2024-09-17T00:15:18.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f394c06e680 msgr2=0x7f394c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:18.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f394c06e680 0x7f394c070b40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3958009fd0 tx=0x7f3958005a90 comp rx=0 tx=0).stop 2024-09-17T00:15:18.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964101e00 msgr2=0x7f396419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964101e00 0x7f396419b4c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f395400e920 tx=0x7f395400ec30 comp rx=0 tx=0).stop 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 shutdown_connections 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f394c06e680 0x7f394c070b40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964101e00 0x7f396419b4c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 --2- 172.21.15.42:0/270819055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3964102720 0x7f39641944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 >> 172.21.15.42:0/270819055 conn(0x7f39640fb650 msgr2=0x7f3964100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 shutdown_connections 2024-09-17T00:15:18.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:18.157+0000 7f396a664700 1 -- 172.21.15.42:0/270819055 wait complete. 2024-09-17T00:15:18.165 INFO:teuthology.orchestra.run.smithi042.stdout:true 2024-09-17T00:15:18.796 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -e sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-09-17T00:15:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:19.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:18 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/270819055' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:15:19.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:18 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/270819055' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:15:20.093 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:19 smithi042 ceph-mon[111056]: pgmap v713: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:19 smithi043 ceph-mon[81510]: pgmap v713: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 -- 172.21.15.42:0/2966868689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 msgr2=0x7fca4c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 --2- 172.21.15.42:0/2966868689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c100180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009910 tx=0x7fca3c009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 -- 172.21.15.42:0/2966868689 shutdown_connections 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 --2- 172.21.15.42:0/2966868689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c100750 0x7fca4c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 --2- 172.21.15.42:0/2966868689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c100180 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 -- 172.21.15.42:0/2966868689 >> 172.21.15.42:0/2966868689 conn(0x7fca4c0fb650 msgr2=0x7fca4c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 -- 172.21.15.42:0/2966868689 shutdown_connections 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 -- 172.21.15.42:0/2966868689 wait complete. 2024-09-17T00:15:20.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.622+0000 7fca521b3700 1 Processor -- start 2024-09-17T00:15:20.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca521b3700 1 -- start start 2024-09-17T00:15:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca521b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca521b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c100750 0x7fca4c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca521b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c102350 con 0x7fca4c0ffda0 2024-09-17T00:15:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca521b3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1024c0 con 0x7fca4c100750 2024-09-17T00:15:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca4b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca4b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58650/0 (socket says 172.21.15.42:58650) 2024-09-17T00:15:20.622 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca4b7fe700 1 -- 172.21.15.42:0/3425101065 learned_addr learned my addr 172.21.15.42:0/3425101065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca4affd700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c100750 0x7fca4c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca4b7fe700 1 -- 172.21.15.42:0/3425101065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c100750 msgr2=0x7fca4c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca4b7fe700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c100750 0x7fca4c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.623+0000 7fca4b7fe700 1 -- 172.21.15.42:0/3425101065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0095f0 con 0x7fca4c0ffda0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.624+0000 7fca4b7fe700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c101840 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006010 tx=0x7fca3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.624+0000 7fca48ff9700 1 -- 172.21.15.42:0/3425101065 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3c01d070 con 0x7fca4c0ffda0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.624+0000 7fca48ff9700 1 -- 172.21.15.42:0/3425101065 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca3c00f460 con 0x7fca4c0ffda0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.624+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c1a2a50 con 0x7fca4c0ffda0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.624+0000 7fca48ff9700 1 -- 172.21.15.42:0/3425101065 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3c017630 con 0x7fca4c0ffda0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.624+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c1a2e00 con 0x7fca4c0ffda0 2024-09-17T00:15:20.623 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.625+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c04f2e0 con 0x7fca4c0ffda0 2024-09-17T00:15:20.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.625+0000 7fca48ff9700 1 -- 172.21.15.42:0/3425101065 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fca3c00f5d0 con 0x7fca4c0ffda0 2024-09-17T00:15:20.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.626+0000 7fca48ff9700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca3406e680 0x7fca34070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:20.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.626+0000 7fca48ff9700 1 -- 172.21.15.42:0/3425101065 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca3c08fb00 con 0x7fca4c0ffda0 2024-09-17T00:15:20.624 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.626+0000 7fca4affd700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca3406e680 0x7fca34070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:20.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.626+0000 7fca4affd700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca3406e680 0x7fca34070b40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fca40005950 tx=0x7fca400058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:20.628 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.630+0000 7fca48ff9700 1 -- 172.21.15.42:0/3425101065 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fca3c058ca0 con 0x7fca4c0ffda0 2024-09-17T00:15:20.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.936+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fca4c1a2ff0 con 0x7fca4c0ffda0 2024-09-17T00:15:20.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.937+0000 7fca48ff9700 1 -- 172.21.15.42:0/3425101065 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fca3c05c360 con 0x7fca4c0ffda0 2024-09-17T00:15:20.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca3406e680 msgr2=0x7fca34070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:20.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca3406e680 0x7fca34070b40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fca40005950 tx=0x7fca400058e0 comp rx=0 tx=0).stop 2024-09-17T00:15:20.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 msgr2=0x7fca4c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:20.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c101840 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006010 tx=0x7fca3c00be20 comp rx=0 tx=0).stop 2024-09-17T00:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 shutdown_connections 2024-09-17T00:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fca3406e680 0x7fca34070b40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca4c0ffda0 0x7fca4c101840 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 --2- 172.21.15.42:0/3425101065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca4c100750 0x7fca4c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 >> 172.21.15.42:0/3425101065 conn(0x7fca4c0fb650 msgr2=0x7fca4c0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 shutdown_connections 2024-09-17T00:15:20.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:20.939+0000 7fca521b3700 1 -- 172.21.15.42:0/3425101065 wait complete. 2024-09-17T00:15:20.947 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph version 19.1.1-330-gc2ddf972 (c2ddf9721ef254645201cdbabd9c8f49c30a586f) squid (stable)" 2024-09-17T00:15:21.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:21.267 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:21.591 DEBUG:teuthology.parallel:result is None 2024-09-17T00:15:21.591 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-17T00:15:21.610 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi042.front.sepia.ceph.com 2024-09-17T00:15:21.610 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-17T00:15:21.823 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:21.944 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:21 smithi042 ceph-mon[111056]: pgmap v714: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:21.944 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:21 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3425101065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:15:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:21 smithi043 ceph-mon[81510]: pgmap v714: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:21 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3425101065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-17T00:15:22.974 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:22 smithi042 ceph-mon[111056]: pgmap v715: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:23.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:22 smithi043 ceph-mon[81510]: pgmap v715: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 -- 172.21.15.42:0/2371442672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9650105bd0 msgr2=0x7f9650105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 --2- 172.21.15.42:0/2371442672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9650105bd0 0x7f9650105fb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9640009910 tx=0x7f9640009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 -- 172.21.15.42:0/2371442672 shutdown_connections 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 --2- 172.21.15.42:0/2371442672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96500ffc20 0x7f9650100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 --2- 172.21.15.42:0/2371442672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9650105bd0 0x7f9650105fb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 -- 172.21.15.42:0/2371442672 >> 172.21.15.42:0/2371442672 conn(0x7f9650075670 msgr2=0x7f9650075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 -- 172.21.15.42:0/2371442672 shutdown_connections 2024-09-17T00:15:23.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.547+0000 7f965524e700 1 -- 172.21.15.42:0/2371442672 wait complete. 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.548+0000 7f965524e700 1 Processor -- start 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.548+0000 7f965524e700 1 -- start start 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.548+0000 7f965524e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96500ffc20 0x7f965019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.548+0000 7f965524e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650105bd0 0x7f965019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.548+0000 7f965524e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501967f0 con 0x7f96500ffc20 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.548+0000 7f965524e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650196960 con 0x7f9650105bd0 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96500ffc20 0x7f965019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96500ffc20 0x7f965019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48678/0 (socket says 172.21.15.42:48678) 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964ed9d700 1 -- 172.21.15.42:0/3575852322 learned_addr learned my addr 172.21.15.42:0/3575852322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964e59c700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650105bd0 0x7f965019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:23.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964ed9d700 1 -- 172.21.15.42:0/3575852322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650105bd0 msgr2=0x7f965019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:23.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964ed9d700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650105bd0 0x7f965019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:23.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964ed9d700 1 -- 172.21.15.42:0/3575852322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400095f0 con 0x7f96500ffc20 2024-09-17T00:15:23.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f964ed9d700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96500ffc20 0x7f965019c800 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9640006010 tx=0x7f964000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:23.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f9637fff700 1 -- 172.21.15.42:0/3575852322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f964001d070 con 0x7f96500ffc20 2024-09-17T00:15:23.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f9637fff700 1 -- 172.21.15.42:0/3575852322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9640004fb0 con 0x7f96500ffc20 2024-09-17T00:15:23.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.549+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9650196be0 con 0x7f96500ffc20 2024-09-17T00:15:23.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.550+0000 7f9637fff700 1 -- 172.21.15.42:0/3575852322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9640022ea0 con 0x7f96500ffc20 2024-09-17T00:15:23.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.550+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96501970d0 con 0x7f96500ffc20 2024-09-17T00:15:23.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.551+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965004f2e0 con 0x7f96500ffc20 2024-09-17T00:15:23.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.551+0000 7f9637fff700 1 -- 172.21.15.42:0/3575852322 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9640022470 con 0x7f96500ffc20 2024-09-17T00:15:23.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.552+0000 7f9637fff700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f963806e680 0x7f9638070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:23.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.552+0000 7f9637fff700 1 -- 172.21.15.42:0/3575852322 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f964008fb40 con 0x7f96500ffc20 2024-09-17T00:15:23.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.552+0000 7f964e59c700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f963806e680 0x7f9638070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:23.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.552+0000 7f964e59c700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f963806e680 0x7f9638070b40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9644006fd0 tx=0x7f9644008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:23.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.556+0000 7f9637fff700 1 -- 172.21.15.42:0/3575852322 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9640058f30 con 0x7f96500ffc20 2024-09-17T00:15:23.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.813+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9650197400 con 0x7f96500ffc20 2024-09-17T00:15:23.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.814+0000 7f9637fff700 1 -- 172.21.15.42:0/3575852322 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1990 (secure 0 0 0) 0x7f9640027070 con 0x7f96500ffc20 2024-09-17T00:15:23.811 INFO:teuthology.orchestra.run.smithi042.stdout:e36 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:btime 2024-09-17T00:13:32:923776+0000 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 36 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-09-16T23:46:07.055413+0000 2024-09-17T00:15:23.812 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-09-17T00:13:32.052875+0000 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 60 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 78 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 2 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1 2024-09-17T00:15:23.813 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=34388,1=34420} 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data enabled 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 1 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 34388 members: 34420,34388 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.ctckas{0:34388} state up:active seq 243 join_fscid=1 addr [v2:172.21.15.42:6826/2046825391,v1:172.21.15.42:6827/2046825391] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi042.anvvuy{1:34420} state up:active seq 239 join_fscid=1 addr [v2:172.21.15.42:6828/4053198993,v1:172.21.15.42:6829/4053198993] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:23.814 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-09-17T00:15:23.815 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:23.815 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.pzcdzg{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6824/2567826967,v1:172.21.15.43:6825/2567826967] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:15:23.815 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.cephfs.smithi043.riwwhn{-1:44349} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.43:6826/3065998819,v1:172.21.15.43:6827/3065998819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f963806e680 msgr2=0x7f9638070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f963806e680 0x7f9638070b40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9644006fd0 tx=0x7f9644008040 comp rx=0 tx=0).stop 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96500ffc20 msgr2=0x7f965019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96500ffc20 0x7f965019c800 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9640006010 tx=0x7f964000be20 comp rx=0 tx=0).stop 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 shutdown_connections 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f963806e680 0x7f9638070b40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96500ffc20 0x7f965019c800 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 --2- 172.21.15.42:0/3575852322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650105bd0 0x7f965019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 >> 172.21.15.42:0/3575852322 conn(0x7f9650075670 msgr2=0x7f96500686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 shutdown_connections 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:23.816+0000 7f965524e700 1 -- 172.21.15.42:0/3575852322 wait complete. 2024-09-17T00:15:23.816 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-09-17T00:15:24.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:23 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3575852322' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:15:24.178 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:23 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3575852322' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-17T00:15:24.496 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-09-17T00:15:24.509 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-17T00:15:24.723 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:25.120 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:24 smithi042 ceph-mon[111056]: pgmap v716: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:25.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:24 smithi043 ceph-mon[81510]: pgmap v716: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 -- 172.21.15.42:0/2330896234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb40ffb40 msgr2=0x7f9eb40fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2330896234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb40ffb40 0x7f9eb40fffc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009910 tx=0x7f9ea0009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 -- 172.21.15.42:0/2330896234 shutdown_connections 2024-09-17T00:15:26.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2330896234 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb40ffb40 0x7f9eb40fffc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2330896234 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb4105bf0 0x7f9eb4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 -- 172.21.15.42:0/2330896234 >> 172.21.15.42:0/2330896234 conn(0x7f9eb40fb610 msgr2=0x7f9eb40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 -- 172.21.15.42:0/2330896234 shutdown_connections 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.492+0000 7f9ebad41700 1 -- 172.21.15.42:0/2330896234 wait complete. 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9ebad41700 1 Processor -- start 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9ebad41700 1 -- start start 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9ebad41700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb40ffb40 0x7f9eb41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9ebad41700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb4105bf0 0x7f9eb4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9ebad41700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4102230 con 0x7f9eb4105bf0 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9ebad41700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4102370 con 0x7f9eb40ffb40 2024-09-17T00:15:26.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9eb3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb4105bf0 0x7f9eb4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9eb3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb4105bf0 0x7f9eb4101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48692/0 (socket says 172.21.15.42:48692) 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.493+0000 7f9eb3fff700 1 -- 172.21.15.42:0/2772617392 learned_addr learned my addr 172.21.15.42:0/2772617392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb8add700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb40ffb40 0x7f9eb41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb3fff700 1 -- 172.21.15.42:0/2772617392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb40ffb40 msgr2=0x7f9eb41017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb3fff700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb40ffb40 0x7f9eb41017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb3fff700 1 -- 172.21.15.42:0/2772617392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00095f0 con 0x7f9eb4105bf0 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb8add700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb40ffb40 0x7f9eb41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb3fff700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb4105bf0 0x7f9eb4101cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006010 tx=0x7f9ea000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb1ffb700 1 -- 172.21.15.42:0/2772617392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea001d070 con 0x7f9eb4105bf0 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb1ffb700 1 -- 172.21.15.42:0/2772617392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ea000f460 con 0x7f9eb4105bf0 2024-09-17T00:15:26.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb41025f0 con 0x7f9eb4105bf0 2024-09-17T00:15:26.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9eb1ffb700 1 -- 172.21.15.42:0/2772617392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea0017630 con 0x7f9eb4105bf0 2024-09-17T00:15:26.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.494+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb41a2d10 con 0x7f9eb4105bf0 2024-09-17T00:15:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.495+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb40679c0 con 0x7f9eb4105bf0 2024-09-17T00:15:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.496+0000 7f9eb1ffb700 1 -- 172.21.15.42:0/2772617392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f9ea000f5d0 con 0x7f9eb4105bf0 2024-09-17T00:15:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.496+0000 7f9eb1ffb700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea406e680 0x7f9ea4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.496+0000 7f9eb1ffb700 1 -- 172.21.15.42:0/2772617392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9ea008fae0 con 0x7f9eb4105bf0 2024-09-17T00:15:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.496+0000 7f9eb8add700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea406e680 0x7f9ea4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:26.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.497+0000 7f9eb8add700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea406e680 0x7f9ea4070b40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005fd0 tx=0x7f9ea8005ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:26.498 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.501+0000 7f9eb1ffb700 1 -- 172.21.15.42:0/2772617392 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9ea0058c80 con 0x7f9eb4105bf0 2024-09-17T00:15:26.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.755+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb404f2e0 con 0x7f9eb4105bf0 2024-09-17T00:15:26.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.755+0000 7f9eb1ffb700 1 -- 172.21.15.42:0/2772617392 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5274 (secure 0 0 0) 0x7f9ea005c340 con 0x7f9eb4105bf0 2024-09-17T00:15:26.753 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:26.753 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":36,"btime":"2024-09-17T00:13:32:923776+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-17T00:13:32.052875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":0,"incarnation":31,"state":"up:active","state_seq":243,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34420":{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":1,"incarnation":31,"state":"up:active","state_seq":239,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34388,"qdb_cluster":[34420,34388]},"id":1}]} 2024-09-17T00:15:26.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea406e680 msgr2=0x7f9ea4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea406e680 0x7f9ea4070b40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8005fd0 tx=0x7f9ea8005ee0 comp rx=0 tx=0).stop 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb4105bf0 msgr2=0x7f9eb4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb4105bf0 0x7f9eb4101cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0006010 tx=0x7f9ea000be20 comp rx=0 tx=0).stop 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 shutdown_connections 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f9ea406e680 0x7f9ea4070b40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9eb40ffb40 0x7f9eb41017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 --2- 172.21.15.42:0/2772617392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9eb4105bf0 0x7f9eb4101cf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 >> 172.21.15.42:0/2772617392 conn(0x7f9eb40fb610 msgr2=0x7f9eb4068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 shutdown_connections 2024-09-17T00:15:26.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:26.758+0000 7f9ebad41700 1 -- 172.21.15.42:0/2772617392 wait complete. 2024-09-17T00:15:26.757 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 36 2024-09-17T00:15:27.003 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:27 smithi042 ceph-mon[111056]: pgmap v717: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:27.003 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:27 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2772617392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-17T00:15:27.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:27 smithi043 ceph-mon[81510]: pgmap v717: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:27.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:27 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2772617392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-17T00:15:27.413 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 2, 'flags': 18} 2024-09-17T00:15:27.413 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 16 2024-09-17T00:15:27.647 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:29.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:29 smithi042 ceph-mon[111056]: pgmap v718: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.285+0000 7f93e03e8700 1 -- 172.21.15.42:0/1501092504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 msgr2=0x7f93d810acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.285+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1501092504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d810acb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009910 tx=0x7f93c8009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 -- 172.21.15.42:0/1501092504 shutdown_connections 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1501092504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d810acb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1501092504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d8101ea0 0x7f93d8102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 -- 172.21.15.42:0/1501092504 >> 172.21.15.42:0/1501092504 conn(0x7f93d80fb650 msgr2=0x7f93d80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 -- 172.21.15.42:0/1501092504 shutdown_connections 2024-09-17T00:15:29.283 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 -- 172.21.15.42:0/1501092504 wait complete. 2024-09-17T00:15:29.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 Processor -- start 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 -- start start 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d8101ea0 0x7f93d80ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d8100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d8100580 con 0x7f93d81027c0 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.286+0000 7f93e03e8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81006c0 con 0x7f93d8101ea0 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93dd983700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d8100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93dd983700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d8100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48702/0 (socket says 172.21.15.42:48702) 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93dd983700 1 -- 172.21.15.42:0/1327700155 learned_addr learned my addr 172.21.15.42:0/1327700155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93de184700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d8101ea0 0x7f93d80ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93dd983700 1 -- 172.21.15.42:0/1327700155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d8101ea0 msgr2=0x7f93d80ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:29.286 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93dd983700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d8101ea0 0x7f93d80ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93dd983700 1 -- 172.21.15.42:0/1327700155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80095f0 con 0x7f93d81027c0 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93de184700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d8101ea0 0x7f93d80ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93dd983700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d8100040 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f93c80092e0 tx=0x7f93c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93cf7fe700 1 -- 172.21.15.42:0/1327700155 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c801d070 con 0x7f93d81027c0 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93cf7fe700 1 -- 172.21.15.42:0/1327700155 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93c8005dc0 con 0x7f93d81027c0 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93cf7fe700 1 -- 172.21.15.42:0/1327700155 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c8017400 con 0x7f93d81027c0 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d8100940 con 0x7f93d81027c0 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.287+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d81a2d40 con 0x7f93d81027c0 2024-09-17T00:15:29.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.288+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d8108340 con 0x7f93d81027c0 2024-09-17T00:15:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.289+0000 7f93cf7fe700 1 -- 172.21.15.42:0/1327700155 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f93c800f460 con 0x7f93d81027c0 2024-09-17T00:15:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.289+0000 7f93cf7fe700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93c406e680 0x7f93c4070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.290+0000 7f93cf7fe700 1 -- 172.21.15.42:0/1327700155 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f93c808fb30 con 0x7f93d81027c0 2024-09-17T00:15:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.290+0000 7f93de184700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93c406e680 0x7f93c4070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:29.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.290+0000 7f93de184700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93c406e680 0x7f93c4070b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005d90 tx=0x7f93d4005d00 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.294+0000 7f93cf7fe700 1 -- 172.21.15.42:0/1327700155 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f93c8058cd0 con 0x7f93d81027c0 2024-09-17T00:15:29.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:29 smithi043 ceph-mon[81510]: pgmap v718: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:29.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.548+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f93d802cc70 con 0x7f93d81027c0 2024-09-17T00:15:29.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.549+0000 7f93cf7fe700 1 -- 172.21.15.42:0/1327700155 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v36) v1 ==== 107+0+4915 (secure 0 0 0) 0x7f93c805c390 con 0x7f93d81027c0 2024-09-17T00:15:29.546 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:29.546 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":16,"btime":"1970-01-01T00:00:00:000000+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3253599157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3253599157},{"type":"v1","addr":"172.21.15.42:6827","nonce":3253599157}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:48:04.660323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi042.anvvuy","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.42:6829/1702969563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1702969563},{"type":"v1","addr":"172.21.15.42:6829","nonce":1702969563}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi043.pzcdzg","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6825/3963651257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3963651257},{"type":"v1","addr":"172.21.15.43:6825","nonce":3963651257}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:29.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93c406e680 msgr2=0x7f93c4070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:29.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93c406e680 0x7f93c4070b40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f93d4005d90 tx=0x7f93d4005d00 comp rx=0 tx=0).stop 2024-09-17T00:15:29.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 msgr2=0x7f93d8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:29.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d8100040 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f93c80092e0 tx=0x7f93c800be20 comp rx=0 tx=0).stop 2024-09-17T00:15:29.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 shutdown_connections 2024-09-17T00:15:29.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f93c406e680 0x7f93c4070b40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:29.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93d8101ea0 0x7f93d80ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:29.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 --2- 172.21.15.42:0/1327700155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93d81027c0 0x7f93d8100040 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:29.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 >> 172.21.15.42:0/1327700155 conn(0x7f93d80fb650 msgr2=0x7f93d8105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:29.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 shutdown_connections 2024-09-17T00:15:29.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:29.551+0000 7f93e03e8700 1 -- 172.21.15.42:0/1327700155 wait complete. 2024-09-17T00:15:29.549 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 16 2024-09-17T00:15:30.137 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:30 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1327700155' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-09-17T00:15:30.161 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 17 2024-09-17T00:15:30.375 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:30.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:30 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1327700155' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-09-17T00:15:31.187 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:31 smithi042 ceph-mon[111056]: pgmap v719: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:31.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:31 smithi043 ceph-mon[81510]: pgmap v719: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:32.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 -- 172.21.15.42:0/1455294178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc101e60 msgr2=0x7f10cc102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:32.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1455294178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc101e60 0x7f10cc102240 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009910 tx=0x7f10bc009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:32.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 -- 172.21.15.42:0/1455294178 shutdown_connections 2024-09-17T00:15:32.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1455294178 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc102780 0x7f10cc10ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:32.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1455294178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc101e60 0x7f10cc102240 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:32.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 -- 172.21.15.42:0/1455294178 >> 172.21.15.42:0/1455294178 conn(0x7f10cc0fb650 msgr2=0x7f10cc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:32.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 -- 172.21.15.42:0/1455294178 shutdown_connections 2024-09-17T00:15:32.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.068+0000 7f10d11ef700 1 -- 172.21.15.42:0/1455294178 wait complete. 2024-09-17T00:15:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10d11ef700 1 Processor -- start 2024-09-17T00:15:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10d11ef700 1 -- start start 2024-09-17T00:15:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10d11ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc101e60 0x7f10cc19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10d11ef700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc102780 0x7f10cc19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10d11ef700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc19d220 con 0x7f10cc102780 2024-09-17T00:15:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10d11ef700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10cc196720 con 0x7f10cc101e60 2024-09-17T00:15:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc102780 0x7f10cc19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10c3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc102780 0x7f10cc19cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48720/0 (socket says 172.21.15.42:48720) 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10c3fff700 1 -- 172.21.15.42:0/1599951334 learned_addr learned my addr 172.21.15.42:0/1599951334 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10cad9d700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc101e60 0x7f10cc19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10c3fff700 1 -- 172.21.15.42:0/1599951334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc101e60 msgr2=0x7f10cc19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10c3fff700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc101e60 0x7f10cc19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.069+0000 7f10c3fff700 1 -- 172.21.15.42:0/1599951334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc0095f0 con 0x7f10cc102780 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.070+0000 7f10c3fff700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc102780 0x7f10cc19cb90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f10b400d900 tx=0x7f10b400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.070+0000 7f10c8d99700 1 -- 172.21.15.42:0/1599951334 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b4009900 con 0x7f10cc102780 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.070+0000 7f10c8d99700 1 -- 172.21.15.42:0/1599951334 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10b4010460 con 0x7f10cc102780 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.070+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10cc196a00 con 0x7f10cc102780 2024-09-17T00:15:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.070+0000 7f10c8d99700 1 -- 172.21.15.42:0/1599951334 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b400f5f0 con 0x7f10cc102780 2024-09-17T00:15:32.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.070+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10cc196f50 con 0x7f10cc102780 2024-09-17T00:15:32.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.071+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10cc108310 con 0x7f10cc102780 2024-09-17T00:15:32.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.071+0000 7f10c8d99700 1 -- 172.21.15.42:0/1599951334 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f10b40105d0 con 0x7f10cc102780 2024-09-17T00:15:32.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.072+0000 7f10c8d99700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10b806e680 0x7f10b8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:32.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.072+0000 7f10c8d99700 1 -- 172.21.15.42:0/1599951334 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f10b408e120 con 0x7f10cc102780 2024-09-17T00:15:32.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.072+0000 7f10cad9d700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10b806e680 0x7f10b8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:32.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.072+0000 7f10cad9d700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10b806e680 0x7f10b8070b40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10bc006010 tx=0x7f10bc01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:32.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.076+0000 7f10c8d99700 1 -- 172.21.15.42:0/1599951334 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f10b405ae70 con 0x7f10cc102780 2024-09-17T00:15:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.331+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f10cc04f2e0 con 0x7f10cc102780 2024-09-17T00:15:32.329 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.332+0000 7f10c8d99700 1 -- 172.21.15.42:0/1599951334 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v36) v1 ==== 107+0+4916 (secure 0 0 0) 0x7f10b405aa00 con 0x7f10cc102780 2024-09-17T00:15:32.330 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:32.330 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"btime":"1970-01-01T00:00:00:000000+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3253599157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3253599157},{"type":"v1","addr":"172.21.15.42:6827","nonce":3253599157}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:48:49.729605+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi042.anvvuy","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.42:6829/1702969563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1702969563},{"type":"v1","addr":"172.21.15.42:6829","nonce":1702969563}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi043.pzcdzg","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6825/3963651257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3963651257},{"type":"v1","addr":"172.21.15.43:6825","nonce":3963651257}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:32.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10b806e680 msgr2=0x7f10b8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:32.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10b806e680 0x7f10b8070b40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10bc006010 tx=0x7f10bc01a040 comp rx=0 tx=0).stop 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc102780 msgr2=0x7f10cc19cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc102780 0x7f10cc19cb90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f10b400d900 tx=0x7f10b400dc10 comp rx=0 tx=0).stop 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 shutdown_connections 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10b806e680 0x7f10b8070b40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10cc101e60 0x7f10cc19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 --2- 172.21.15.42:0/1599951334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10cc102780 0x7f10cc19cb90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 >> 172.21.15.42:0/1599951334 conn(0x7f10cc0fb650 msgr2=0x7f10cc105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 shutdown_connections 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:32.334+0000 7f10d11ef700 1 -- 172.21.15.42:0/1599951334 wait complete. 2024-09-17T00:15:32.332 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2024-09-17T00:15:32.925 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 18 2024-09-17T00:15:33.134 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:33.256 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:33 smithi042 ceph-mon[111056]: pgmap v720: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:33.256 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:33 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1599951334' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-09-17T00:15:33.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:33 smithi043 ceph-mon[81510]: pgmap v720: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:33.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:33 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1599951334' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-09-17T00:15:34.260 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:34 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:15:34.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:34 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:15:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 -- 172.21.15.42:0/2299623528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60480ffda0 msgr2=0x7f6048100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 --2- 172.21.15.42:0/2299623528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60480ffda0 0x7f6048100180 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6044009910 tx=0x7f6044009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 -- 172.21.15.42:0/2299623528 shutdown_connections 2024-09-17T00:15:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 --2- 172.21.15.42:0/2299623528 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6048100750 0x7f60481047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 --2- 172.21.15.42:0/2299623528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60480ffda0 0x7f6048100180 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 -- 172.21.15.42:0/2299623528 >> 172.21.15.42:0/2299623528 conn(0x7f60480fb650 msgr2=0x7f60480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:34.859 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 -- 172.21.15.42:0/2299623528 shutdown_connections 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.848+0000 7f6050428700 1 -- 172.21.15.42:0/2299623528 wait complete. 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f6050428700 1 Processor -- start 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f6050428700 1 -- start start 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f6050428700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480ffda0 0x7f60481954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f6050428700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6048100750 0x7f6048195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f6050428700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6048195f60 con 0x7f6048100750 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f6050428700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60481960d0 con 0x7f60480ffda0 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f604d9c3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6048100750 0x7f6048195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f604d9c3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6048100750 0x7f6048195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57868/0 (socket says 172.21.15.42:57868) 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f604d9c3700 1 -- 172.21.15.42:0/2626897578 learned_addr learned my addr 172.21.15.42:0/2626897578 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f604e1c4700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480ffda0 0x7f60481954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:34.860 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f604d9c3700 1 -- 172.21.15.42:0/2626897578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480ffda0 msgr2=0x7f60481954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f604d9c3700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480ffda0 0x7f60481954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.849+0000 7f604d9c3700 1 -- 172.21.15.42:0/2626897578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440095f0 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.850+0000 7f604d9c3700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6048100750 0x7f6048195a20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f603800d8d0 tx=0x7f603800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.850+0000 7f603f7fe700 1 -- 172.21.15.42:0/2626897578 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60380041f0 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.850+0000 7f603f7fe700 1 -- 172.21.15.42:0/2626897578 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6038007620 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.850+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60481a2b50 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.850+0000 7f603f7fe700 1 -- 172.21.15.42:0/2626897578 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6038003c50 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.850+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60481a2eb0 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.851+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6048108140 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.852+0000 7f603f7fe700 1 -- 172.21.15.42:0/2626897578 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f6038004350 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.852+0000 7f603f7fe700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f603406e680 0x7f6034070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.852+0000 7f603f7fe700 1 -- 172.21.15.42:0/2626897578 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f603808e170 con 0x7f6048100750 2024-09-17T00:15:34.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.852+0000 7f604e1c4700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f603406e680 0x7f6034070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.852+0000 7f604e1c4700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f603406e680 0x7f6034070b40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6044006010 tx=0x7f604401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:34.862 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:34.857+0000 7f603f7fe700 1 -- 172.21.15.42:0/2626897578 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f603805ae40 con 0x7f6048100750 2024-09-17T00:15:35.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.110+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f6048107b20 con 0x7f6048100750 2024-09-17T00:15:35.108 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:35 smithi042 ceph-mon[111056]: pgmap v721: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:35.108 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:15:35.108 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:15:35.108 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:15:35.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.111+0000 7f603f7fe700 1 -- 172.21.15.42:0/2626897578 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v36) v1 ==== 107+0+4930 (secure 0 0 0) 0x7f6038003e80 con 0x7f6048100750 2024-09-17T00:15:35.109 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:35.109 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":18,"btime":"2024-09-16T23:52:59:633980+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3253599157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3253599157},{"type":"v1","addr":"172.21.15.42:6827","nonce":3253599157}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:52:59.633974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi042.anvvuy","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.42:6829/1702969563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1702969563},{"type":"v1","addr":"172.21.15.42:6829","nonce":1702969563}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi043.pzcdzg","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6825/3963651257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3963651257},{"type":"v1","addr":"172.21.15.43:6825","nonce":3963651257}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14478,"qdb_cluster":[14478,24269]},"id":1}]} 2024-09-17T00:15:35.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f603406e680 msgr2=0x7f6034070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:35.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f603406e680 0x7f6034070b40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6044006010 tx=0x7f604401a040 comp rx=0 tx=0).stop 2024-09-17T00:15:35.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6048100750 msgr2=0x7f6048195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6048100750 0x7f6048195a20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f603800d8d0 tx=0x7f603800dc90 comp rx=0 tx=0).stop 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 shutdown_connections 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f603406e680 0x7f6034070b40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60480ffda0 0x7f60481954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 --2- 172.21.15.42:0/2626897578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6048100750 0x7f6048195a20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 >> 172.21.15.42:0/2626897578 conn(0x7f60480fb650 msgr2=0x7f60480fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 shutdown_connections 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:35.113+0000 7f6050428700 1 -- 172.21.15.42:0/2626897578 wait complete. 2024-09-17T00:15:35.111 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 18 2024-09-17T00:15:35.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:35 smithi043 ceph-mon[81510]: pgmap v721: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:35.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:15:35.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:15:35.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:15:35.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 19 2024-09-17T00:15:35.995 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:36.284 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:36 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2626897578' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-09-17T00:15:36.284 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:36.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:36 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2626897578' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-09-17T00:15:36.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:37.375 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:37 smithi042 ceph-mon[111056]: pgmap v722: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:37.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:37 smithi043 ceph-mon[81510]: pgmap v722: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 -- 172.21.15.42:0/87144941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 msgr2=0x7f694c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 --2- 172.21.15.42:0/87144941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c100080 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f693c009910 tx=0x7f693c009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 -- 172.21.15.42:0/87144941 shutdown_connections 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 --2- 172.21.15.42:0/87144941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c100080 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 --2- 172.21.15.42:0/87144941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694c105bd0 0x7f694c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 -- 172.21.15.42:0/87144941 >> 172.21.15.42:0/87144941 conn(0x7f694c075670 msgr2=0x7f694c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 -- 172.21.15.42:0/87144941 shutdown_connections 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.762+0000 7f6953d94700 1 -- 172.21.15.42:0/87144941 wait complete. 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6953d94700 1 Processor -- start 2024-09-17T00:15:37.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6953d94700 1 -- start start 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6953d94700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6953d94700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694c105bd0 0x7f694c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6953d94700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c198360 con 0x7f694c0ffc20 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6953d94700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694c1984d0 con 0x7f694c105bd0 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6951b30700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6951b30700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57888/0 (socket says 172.21.15.42:57888) 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.763+0000 7f6951b30700 1 -- 172.21.15.42:0/1381900684 learned_addr learned my addr 172.21.15.42:0/1381900684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.764+0000 7f695132f700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694c105bd0 0x7f694c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.764+0000 7f6951b30700 1 -- 172.21.15.42:0/1381900684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694c105bd0 msgr2=0x7f694c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.764+0000 7f6951b30700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694c105bd0 0x7f694c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.764+0000 7f6951b30700 1 -- 172.21.15.42:0/1381900684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c0095f0 con 0x7f694c0ffc20 2024-09-17T00:15:37.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.764+0000 7f6951b30700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c197740 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f694800c8a0 tx=0x7f694800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:37.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.765+0000 7f6942ffd700 1 -- 172.21.15.42:0/1381900684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69480078e0 con 0x7f694c0ffc20 2024-09-17T00:15:37.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.765+0000 7f6942ffd700 1 -- 172.21.15.42:0/1381900684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f694800f450 con 0x7f694c0ffc20 2024-09-17T00:15:37.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.765+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c1a07b0 con 0x7f694c0ffc20 2024-09-17T00:15:37.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.765+0000 7f6942ffd700 1 -- 172.21.15.42:0/1381900684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694800e5e0 con 0x7f694c0ffc20 2024-09-17T00:15:37.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.765+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694c1a0d00 con 0x7f694c0ffc20 2024-09-17T00:15:37.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.766+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694c1080d0 con 0x7f694c0ffc20 2024-09-17T00:15:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.767+0000 7f6942ffd700 1 -- 172.21.15.42:0/1381900684 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f694800f5c0 con 0x7f694c0ffc20 2024-09-17T00:15:37.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.768+0000 7f6942ffd700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f693806e680 0x7f6938070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.768+0000 7f6942ffd700 1 -- 172.21.15.42:0/1381900684 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f694808e7e0 con 0x7f694c0ffc20 2024-09-17T00:15:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.768+0000 7f695132f700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f693806e680 0x7f6938070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.769+0000 7f695132f700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f693806e680 0x7f6938070b40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f694c0691d0 tx=0x7f693c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:37.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:37.772+0000 7f6942ffd700 1 -- 172.21.15.42:0/1381900684 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f69480579e0 con 0x7f694c0ffc20 2024-09-17T00:15:38.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.057+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f694c04f2e0 con 0x7f694c0ffc20 2024-09-17T00:15:38.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.057+0000 7f6942ffd700 1 -- 172.21.15.42:0/1381900684 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v36) v1 ==== 107+0+4929 (secure 0 0 0) 0x7f694805b0a0 con 0x7f694c0ffc20 2024-09-17T00:15:38.055 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:38.055 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":19,"btime":"2024-09-16T23:53:20:175453+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3253599157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3253599157},{"type":"v1","addr":"172.21.15.42:6827","nonce":3253599157}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:53:19.510146+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14478,"mds_1":24269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi042.anvvuy","rank":0,"incarnation":11,"state":"up:active","state_seq":6,"addr":"172.21.15.42:6829/1702969563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1702969563},{"type":"v1","addr":"172.21.15.42:6829","nonce":1702969563}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24269":{"gid":24269,"name":"cephfs.smithi043.pzcdzg","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6825/3963651257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3963651257},{"type":"v1","addr":"172.21.15.43:6825","nonce":3963651257}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14478,"qdb_cluster":[24269,14478]},"id":1}]} 2024-09-17T00:15:38.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f693806e680 msgr2=0x7f6938070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:38.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f693806e680 0x7f6938070b40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f694c0691d0 tx=0x7f693c01a040 comp rx=0 tx=0).stop 2024-09-17T00:15:38.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 msgr2=0x7f694c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:38.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c197740 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f694800c8a0 tx=0x7f694800cbb0 comp rx=0 tx=0).stop 2024-09-17T00:15:38.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 shutdown_connections 2024-09-17T00:15:38.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f693806e680 0x7f6938070b40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:38.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f694c0ffc20 0x7f694c197740 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:38.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 --2- 172.21.15.42:0/1381900684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f694c105bd0 0x7f694c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:38.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 >> 172.21.15.42:0/1381900684 conn(0x7f694c075670 msgr2=0x7f694c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:38.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 shutdown_connections 2024-09-17T00:15:38.058 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:38.060+0000 7f6953d94700 1 -- 172.21.15.42:0/1381900684 wait complete. 2024-09-17T00:15:38.058 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 19 2024-09-17T00:15:38.699 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 20 2024-09-17T00:15:38.912 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:39.287 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:39 smithi042 ceph-mon[111056]: pgmap v723: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:39.287 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:39 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1381900684' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-09-17T00:15:39.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:39 smithi043 ceph-mon[81510]: pgmap v723: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:39.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:39 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1381900684' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 -- 172.21.15.42:0/205252427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3180ffda0 msgr2=0x7fb318100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/205252427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3180ffda0 0x7fb318100180 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb308009910 tx=0x7fb308009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 -- 172.21.15.42:0/205252427 shutdown_connections 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/205252427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb318100750 0x7fb3181047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/205252427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3180ffda0 0x7fb318100180 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 -- 172.21.15.42:0/205252427 >> 172.21.15.42:0/205252427 conn(0x7fb3180fb650 msgr2=0x7fb3180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 -- 172.21.15.42:0/205252427 shutdown_connections 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.585+0000 7fb31f0b0700 1 -- 172.21.15.42:0/205252427 wait complete. 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31f0b0700 1 Processor -- start 2024-09-17T00:15:40.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31f0b0700 1 -- start start 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31f0b0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3180ffda0 0x7fb318101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31f0b0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb318100750 0x7fb318101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31f0b0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3181022c0 con 0x7fb318100750 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31f0b0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318102400 con 0x7fb3180ffda0 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31ce4c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3180ffda0 0x7fb318101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb317fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb318100750 0x7fb318101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31ce4c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3180ffda0 0x7fb318101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.42:58878/0 (socket says 172.21.15.42:58878) 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.586+0000 7fb31ce4c700 1 -- 172.21.15.42:0/662132584 learned_addr learned my addr 172.21.15.42:0/662132584 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb317fff700 1 -- 172.21.15.42:0/662132584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3180ffda0 msgr2=0x7fb318101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb317fff700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3180ffda0 0x7fb318101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb317fff700 1 -- 172.21.15.42:0/662132584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3080095f0 con 0x7fb318100750 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb317fff700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb318100750 0x7fb318101d80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c8c0 tx=0x7fb30c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb315ffb700 1 -- 172.21.15.42:0/662132584 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb30c004410 con 0x7fb318100750 2024-09-17T00:15:40.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb315ffb700 1 -- 172.21.15.42:0/662132584 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb30c004570 con 0x7fb318100750 2024-09-17T00:15:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3181a2ae0 con 0x7fb318100750 2024-09-17T00:15:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb315ffb700 1 -- 172.21.15.42:0/662132584 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb30c007ce0 con 0x7fb318100750 2024-09-17T00:15:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.587+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3181a2e40 con 0x7fb318100750 2024-09-17T00:15:40.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.588+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31804f2e0 con 0x7fb318100750 2024-09-17T00:15:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.589+0000 7fb315ffb700 1 -- 172.21.15.42:0/662132584 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb30c003680 con 0x7fb318100750 2024-09-17T00:15:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.589+0000 7fb315ffb700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb30006e680 0x7fb300070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.589+0000 7fb315ffb700 1 -- 172.21.15.42:0/662132584 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb30c08e360 con 0x7fb318100750 2024-09-17T00:15:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.589+0000 7fb31ce4c700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb30006e680 0x7fb300070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:40.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.590+0000 7fb31ce4c700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb30006e680 0x7fb300070b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb308009fd0 tx=0x7fb30801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:40.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.594+0000 7fb315ffb700 1 -- 172.21.15.42:0/662132584 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb30c057500 con 0x7fb318100750 2024-09-17T00:15:40.845 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.848+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fb3181a3090 con 0x7fb318100750 2024-09-17T00:15:40.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.849+0000 7fb315ffb700 1 -- 172.21.15.42:0/662132584 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v36) v1 ==== 107+0+3335 (secure 0 0 0) 0x7fb30c014d20 con 0x7fb318100750 2024-09-17T00:15:40.846 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:40.846 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":20,"btime":"2024-09-16T23:57:12:183431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3253599157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3253599157},{"type":"v1","addr":"172.21.15.42:6827","nonce":3253599157}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:40.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb30006e680 msgr2=0x7fb300070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:40.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb30006e680 0x7fb300070b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb308009fd0 tx=0x7fb30801a040 comp rx=0 tx=0).stop 2024-09-17T00:15:40.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb318100750 msgr2=0x7fb318101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:40.848 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb318100750 0x7fb318101d80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c8c0 tx=0x7fb30c00cbd0 comp rx=0 tx=0).stop 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 shutdown_connections 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb30006e680 0x7fb300070b40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3180ffda0 0x7fb318101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 --2- 172.21.15.42:0/662132584 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb318100750 0x7fb318101d80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 >> 172.21.15.42:0/662132584 conn(0x7fb3180fb650 msgr2=0x7fb318103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 shutdown_connections 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:40.851+0000 7fb31f0b0700 1 -- 172.21.15.42:0/662132584 wait complete. 2024-09-17T00:15:40.849 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 20 2024-09-17T00:15:41.198 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:41 smithi042 ceph-mon[111056]: pgmap v724: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:41.199 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:41 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/662132584' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-09-17T00:15:41.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:41 smithi043 ceph-mon[81510]: pgmap v724: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:41.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:41 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/662132584' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-09-17T00:15:41.488 DEBUG:tasks.fs:max_mds not reduced in epoch 20 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:15:41.489 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 21 2024-09-17T00:15:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:43.219 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:43 smithi042 ceph-mon[111056]: pgmap v725: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:43.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:43 smithi043 ceph-mon[81510]: pgmap v725: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.493+0000 7ff551368700 1 -- 172.21.15.42:0/4186119858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 msgr2=0x7ff54c1047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.493+0000 7ff551368700 1 --2- 172.21.15.42:0/4186119858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c1047b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff534009910 tx=0x7ff534009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.494+0000 7ff551368700 1 -- 172.21.15.42:0/4186119858 shutdown_connections 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.494+0000 7ff551368700 1 --2- 172.21.15.42:0/4186119858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c1047b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.494+0000 7ff551368700 1 --2- 172.21.15.42:0/4186119858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54c100780 0x7ff54c100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.494+0000 7ff551368700 1 -- 172.21.15.42:0/4186119858 >> 172.21.15.42:0/4186119858 conn(0x7ff54c074e30 msgr2=0x7ff54c075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.494+0000 7ff551368700 1 -- 172.21.15.42:0/4186119858 shutdown_connections 2024-09-17T00:15:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.494+0000 7ff551368700 1 -- 172.21.15.42:0/4186119858 wait complete. 2024-09-17T00:15:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.494+0000 7ff551368700 1 Processor -- start 2024-09-17T00:15:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff551368700 1 -- start start 2024-09-17T00:15:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff551368700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54c100780 0x7ff54c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff551368700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff551368700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19f470 con 0x7ff54c101130 2024-09-17T00:15:43.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff551368700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c198920 con 0x7ff54c100780 2024-09-17T00:15:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff5427fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff5427fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42346/0 (socket says 172.21.15.42:42346) 2024-09-17T00:15:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff5427fc700 1 -- 172.21.15.42:0/2160957317 learned_addr learned my addr 172.21.15.42:0/2160957317 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:43.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff54affd700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54c100780 0x7ff54c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff5427fc700 1 -- 172.21.15.42:0/2160957317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54c100780 msgr2=0x7ff54c19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff5427fc700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54c100780 0x7ff54c19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff5427fc700 1 -- 172.21.15.42:0/2160957317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340095f0 con 0x7ff54c101130 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.495+0000 7ff5427fc700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c19ede0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff5340092e0 tx=0x7ff53400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.496+0000 7ff548ff9700 1 -- 172.21.15.42:0/2160957317 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53401d070 con 0x7ff54c101130 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.496+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c198ba0 con 0x7ff54c101130 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.496+0000 7ff548ff9700 1 -- 172.21.15.42:0/2160957317 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff534005dc0 con 0x7ff54c101130 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.496+0000 7ff548ff9700 1 -- 172.21.15.42:0/2160957317 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff534017400 con 0x7ff54c101130 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.496+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c199090 con 0x7ff54c101130 2024-09-17T00:15:43.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.497+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c04f2e0 con 0x7ff54c101130 2024-09-17T00:15:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.498+0000 7ff548ff9700 1 -- 172.21.15.42:0/2160957317 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7ff53400f6f0 con 0x7ff54c101130 2024-09-17T00:15:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.498+0000 7ff548ff9700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff53806e680 0x7ff538070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.498+0000 7ff548ff9700 1 -- 172.21.15.42:0/2160957317 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff53408fa30 con 0x7ff54c101130 2024-09-17T00:15:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.498+0000 7ff54affd700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff53806e680 0x7ff538070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:43.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.499+0000 7ff54affd700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff53806e680 0x7ff538070b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005950 tx=0x7ff53c00b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:43.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.503+0000 7ff548ff9700 1 -- 172.21.15.42:0/2160957317 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff53405c700 con 0x7ff54c101130 2024-09-17T00:15:43.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.760+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7ff54c199e30 con 0x7ff54c101130 2024-09-17T00:15:43.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.761+0000 7ff548ff9700 1 -- 172.21.15.42:0/2160957317 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v36) v1 ==== 107+0+4893 (secure 0 0 0) 0x7ff53405c290 con 0x7ff54c101130 2024-09-17T00:15:43.758 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:43.759 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":21,"btime":"2024-09-16T23:57:13:185189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14490,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/3253599157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":3253599157},{"type":"v1","addr":"172.21.15.42:6827","nonce":3253599157}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4052502543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4052502543},{"type":"v1","addr":"172.21.15.42:6829","nonce":4052502543}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44307,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/3469372237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3469372237},{"type":"v1","addr":"172.21.15.43:6825","nonce":3469372237}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:43.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff53806e680 msgr2=0x7ff538070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:43.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff53806e680 0x7ff538070b40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff53c005950 tx=0x7ff53c00b410 comp rx=0 tx=0).stop 2024-09-17T00:15:43.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 msgr2=0x7ff54c19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:43.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c19ede0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff5340092e0 tx=0x7ff53400bd70 comp rx=0 tx=0).stop 2024-09-17T00:15:43.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 shutdown_connections 2024-09-17T00:15:43.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7ff53806e680 0x7ff538070b40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:43.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff54c100780 0x7ff54c19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:43.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 --2- 172.21.15.42:0/2160957317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff54c101130 0x7ff54c19ede0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:43.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.763+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 >> 172.21.15.42:0/2160957317 conn(0x7ff54c074e30 msgr2=0x7ff54c103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:43.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.764+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 shutdown_connections 2024-09-17T00:15:43.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:43.764+0000 7ff551368700 1 -- 172.21.15.42:0/2160957317 wait complete. 2024-09-17T00:15:43.762 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 21 2024-09-17T00:15:44.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:44 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2160957317' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-09-17T00:15:44.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:44 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2160957317' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-09-17T00:15:44.453 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:15:44.454 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 22 2024-09-17T00:15:44.670 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:45.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:45 smithi043 ceph-mon[81510]: pgmap v726: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:45.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:45 smithi042 ceph-mon[111056]: pgmap v726: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.434+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/2162264852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 msgr2=0x7f1bcc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.434+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/2162264852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc101280 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8009910 tx=0x7f1bc8009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.434+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/2162264852 shutdown_connections 2024-09-17T00:15:46.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.434+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/2162264852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc101850 0x7f1bcc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.434+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/2162264852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc101280 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.434+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/2162264852 >> 172.21.15.42:0/2162264852 conn(0x7f1bcc0fc750 msgr2=0x7f1bcc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.435+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/2162264852 shutdown_connections 2024-09-17T00:15:46.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.435+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/2162264852 wait complete. 2024-09-17T00:15:46.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd3f8c700 1 Processor -- start 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd3f8c700 1 -- start start 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd3f8c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd3f8c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc101850 0x7f1bcc196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd3f8c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc197560 con 0x7f1bcc100ea0 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd3f8c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc19b370 con 0x7f1bcc101850 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd1d28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd1d28700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42368/0 (socket says 172.21.15.42:42368) 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd1d28700 1 -- 172.21.15.42:0/3458313293 learned_addr learned my addr 172.21.15.42:0/3458313293 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd1d28700 1 -- 172.21.15.42:0/3458313293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc101850 msgr2=0x7f1bcc196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd1527700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc101850 0x7f1bcc196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd1d28700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc101850 0x7f1bcc196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.436+0000 7f1bd1d28700 1 -- 172.21.15.42:0/3458313293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc80095f0 con 0x7f1bcc100ea0 2024-09-17T00:15:46.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.437+0000 7f1bd1d28700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc196990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8006010 tx=0x7f1bc800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.437+0000 7f1bc2ffd700 1 -- 172.21.15.42:0/3458313293 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc801d070 con 0x7f1bcc100ea0 2024-09-17T00:15:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.437+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc19b5f0 con 0x7f1bcc100ea0 2024-09-17T00:15:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.437+0000 7f1bc2ffd700 1 -- 172.21.15.42:0/3458313293 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bc8004fb0 con 0x7f1bcc100ea0 2024-09-17T00:15:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.437+0000 7f1bc2ffd700 1 -- 172.21.15.42:0/3458313293 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc800f690 con 0x7f1bcc100ea0 2024-09-17T00:15:46.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.437+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc19bae0 con 0x7f1bcc100ea0 2024-09-17T00:15:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.438+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc109240 con 0x7f1bcc100ea0 2024-09-17T00:15:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.438+0000 7f1bc2ffd700 1 -- 172.21.15.42:0/3458313293 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f1bc80228a0 con 0x7f1bcc100ea0 2024-09-17T00:15:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.439+0000 7f1bc2ffd700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1bb806e680 0x7f1bb8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.439+0000 7f1bc2ffd700 1 -- 172.21.15.42:0/3458313293 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1bc808ff70 con 0x7f1bcc100ea0 2024-09-17T00:15:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.439+0000 7f1bd1527700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1bb806e680 0x7f1bb8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.439+0000 7f1bd1527700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1bb806e680 0x7f1bb8070b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006fd0 tx=0x7f1bbc008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:46.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.443+0000 7f1bc2ffd700 1 -- 172.21.15.42:0/3458313293 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1bc805cba0 con 0x7f1bcc100ea0 2024-09-17T00:15:46.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.702+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f1bcc197cf0 con 0x7f1bcc100ea0 2024-09-17T00:15:46.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.703+0000 7f1bc2ffd700 1 -- 172.21.15.42:0/3458313293 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v36) v1 ==== 107+0+4114 (secure 0 0 0) 0x7f1bc805c730 con 0x7f1bcc100ea0 2024-09-17T00:15:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:46.700 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":22,"btime":"2024-09-16T23:57:16:194795+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4052502543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4052502543},{"type":"v1","addr":"172.21.15.42:6829","nonce":4052502543}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44307,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/3469372237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3469372237},{"type":"v1","addr":"172.21.15.43:6825","nonce":3469372237}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:46.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1bb806e680 msgr2=0x7f1bb8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:46.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1bb806e680 0x7f1bb8070b40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc006fd0 tx=0x7f1bbc008040 comp rx=0 tx=0).stop 2024-09-17T00:15:46.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 msgr2=0x7f1bcc196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:46.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc196990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8006010 tx=0x7f1bc800be20 comp rx=0 tx=0).stop 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 shutdown_connections 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f1bb806e680 0x7f1bb8070b40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bcc100ea0 0x7f1bcc196990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 --2- 172.21.15.42:0/3458313293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bcc101850 0x7f1bcc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 >> 172.21.15.42:0/3458313293 conn(0x7f1bcc0fc750 msgr2=0x7f1bcc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 shutdown_connections 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:46.705+0000 7f1bd3f8c700 1 -- 172.21.15.42:0/3458313293 wait complete. 2024-09-17T00:15:46.703 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 22 2024-09-17T00:15:47.363 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:47 smithi042 ceph-mon[111056]: pgmap v727: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:47.363 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:47 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3458313293' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-09-17T00:15:47.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:47 smithi043 ceph-mon[81510]: pgmap v727: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:47.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:47 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3458313293' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-09-17T00:15:47.389 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:15:47.389 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 23 2024-09-17T00:15:47.603 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:49.325 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:49 smithi042 ceph-mon[111056]: pgmap v728: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:49.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 -- 172.21.15.42:0/2088536330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 msgr2=0x7f10bc0ffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 --2- 172.21.15.42:0/2088536330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc0ffd70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009910 tx=0x7f10b8009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 -- 172.21.15.42:0/2088536330 shutdown_connections 2024-09-17T00:15:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 --2- 172.21.15.42:0/2088536330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10bc100340 0x7f10bc10d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 --2- 172.21.15.42:0/2088536330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc0ffd70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 -- 172.21.15.42:0/2088536330 >> 172.21.15.42:0/2088536330 conn(0x7f10bc0fb650 msgr2=0x7f10bc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 -- 172.21.15.42:0/2088536330 shutdown_connections 2024-09-17T00:15:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.354+0000 7f10c437b700 1 -- 172.21.15.42:0/2088536330 wait complete. 2024-09-17T00:15:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c437b700 1 Processor -- start 2024-09-17T00:15:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c437b700 1 -- start start 2024-09-17T00:15:49.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c437b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c437b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10bc100340 0x7f10bc19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c437b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc19f430 con 0x7f10bc0ff990 2024-09-17T00:15:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c437b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc1988e0 con 0x7f10bc100340 2024-09-17T00:15:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c2117700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c2117700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42392/0 (socket says 172.21.15.42:42392) 2024-09-17T00:15:49.357 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c2117700 1 -- 172.21.15.42:0/2221510847 learned_addr learned my addr 172.21.15.42:0/2221510847 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c2117700 1 -- 172.21.15.42:0/2221510847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10bc100340 msgr2=0x7f10bc19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c2117700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10bc100340 0x7f10bc19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.355+0000 7f10c2117700 1 -- 172.21.15.42:0/2221510847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80095f0 con 0x7f10bc0ff990 2024-09-17T00:15:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.356+0000 7f10c2117700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc19e860 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f10b80092e0 tx=0x7f10b80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.356+0000 7f10af7fe700 1 -- 172.21.15.42:0/2221510847 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b801d070 con 0x7f10bc0ff990 2024-09-17T00:15:49.358 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.356+0000 7f10af7fe700 1 -- 172.21.15.42:0/2221510847 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10b800f460 con 0x7f10bc0ff990 2024-09-17T00:15:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.356+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc198b60 con 0x7f10bc0ff990 2024-09-17T00:15:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.356+0000 7f10af7fe700 1 -- 172.21.15.42:0/2221510847 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10b8021640 con 0x7f10bc0ff990 2024-09-17T00:15:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.356+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc1990b0 con 0x7f10bc0ff990 2024-09-17T00:15:49.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.357+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc10a770 con 0x7f10bc0ff990 2024-09-17T00:15:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.358+0000 7f10af7fe700 1 -- 172.21.15.42:0/2221510847 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f10b800f5d0 con 0x7f10bc0ff990 2024-09-17T00:15:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.358+0000 7f10af7fe700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10a806e680 0x7f10a8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.358+0000 7f10af7fe700 1 -- 172.21.15.42:0/2221510847 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f10b808f8a0 con 0x7f10bc0ff990 2024-09-17T00:15:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.358+0000 7f10c1916700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10a806e680 0x7f10a8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:49.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.359+0000 7f10c1916700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10a806e680 0x7f10a8070b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f10b0005950 tx=0x7f10b00058e0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:49.362 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.363+0000 7f10af7fe700 1 -- 172.21.15.42:0/2221510847 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f10b805c5f0 con 0x7f10bc0ff990 2024-09-17T00:15:49.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:49 smithi043 ceph-mon[81510]: pgmap v728: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:49.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.612+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f10bc1993e0 con 0x7f10bc0ff990 2024-09-17T00:15:49.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.613+0000 7f10af7fe700 1 -- 172.21.15.42:0/2221510847 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v36) v1 ==== 107+0+4961 (secure 0 0 0) 0x7f10b805c180 con 0x7f10bc0ff990 2024-09-17T00:15:49.611 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:49.611 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"btime":"2024-09-16T23:57:21:628868+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34380,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4052502543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4052502543},{"type":"v1","addr":"172.21.15.42:6829","nonce":4052502543}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44307,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/3469372237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3469372237},{"type":"v1","addr":"172.21.15.43:6825","nonce":3469372237}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10a806e680 msgr2=0x7f10a8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10a806e680 0x7f10a8070b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f10b0005950 tx=0x7f10b00058e0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 msgr2=0x7f10bc19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc19e860 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f10b80092e0 tx=0x7f10b80056a0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 shutdown_connections 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f10a806e680 0x7f10a8070b40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10bc0ff990 0x7f10bc19e860 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 --2- 172.21.15.42:0/2221510847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10bc100340 0x7f10bc19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:49.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 >> 172.21.15.42:0/2221510847 conn(0x7f10bc0fb650 msgr2=0x7f10bc0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.615+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 shutdown_connections 2024-09-17T00:15:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:49.616+0000 7f10c437b700 1 -- 172.21.15.42:0/2221510847 wait complete. 2024-09-17T00:15:49.614 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-09-17T00:15:50.212 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:50 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2221510847' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-09-17T00:15:50.236 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:15:50.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 24 2024-09-17T00:15:50.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:50 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2221510847' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-09-17T00:15:50.449 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:51.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:51 smithi042 ceph-mon[111056]: pgmap v729: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:51.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:51.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:51 smithi043 ceph-mon[81510]: pgmap v729: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:51.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:15:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.211+0000 7f7ecb885700 1 -- 172.21.15.42:0/1528554712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 msgr2=0x7f7ec40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.211+0000 7f7ecb885700 1 --2- 172.21.15.42:0/1528554712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec40689e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009910 tx=0x7f7eb4009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 -- 172.21.15.42:0/1528554712 shutdown_connections 2024-09-17T00:15:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 --2- 172.21.15.42:0/1528554712 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec40689e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 --2- 172.21.15.42:0/1528554712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4106cd0 0x7f7ec41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 -- 172.21.15.42:0/1528554712 >> 172.21.15.42:0/1528554712 conn(0x7f7ec4075670 msgr2=0x7f7ec4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:52.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 -- 172.21.15.42:0/1528554712 shutdown_connections 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 -- 172.21.15.42:0/1528554712 wait complete. 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 Processor -- start 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.212+0000 7f7ecb885700 1 -- start start 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ecb885700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ecb885700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4106cd0 0x7f7ec419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ecb885700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419f400 con 0x7f7ec4068580 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ecb885700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41988b0 con 0x7f7ec4106cd0 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec9621700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec9621700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42424/0 (socket says 172.21.15.42:42424) 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec9621700 1 -- 172.21.15.42:0/782240181 learned_addr learned my addr 172.21.15.42:0/782240181 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec8e20700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4106cd0 0x7f7ec419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:52.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec9621700 1 -- 172.21.15.42:0/782240181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4106cd0 msgr2=0x7f7ec419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec9621700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4106cd0 0x7f7ec419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec9621700 1 -- 172.21.15.42:0/782240181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb40095f0 con 0x7f7ec4068580 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7ec9621700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec419e830 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000d900 tx=0x7f7ec000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.213+0000 7f7eba7fc700 1 -- 172.21.15.42:0/782240181 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec00041f0 con 0x7f7ec4068580 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.214+0000 7f7eba7fc700 1 -- 172.21.15.42:0/782240181 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ec0004cf0 con 0x7f7ec4068580 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.214+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec4198b90 con 0x7f7ec4068580 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.214+0000 7f7eba7fc700 1 -- 172.21.15.42:0/782240181 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec000b750 con 0x7f7ec4068580 2024-09-17T00:15:52.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.214+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec41990e0 con 0x7f7ec4068580 2024-09-17T00:15:52.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.215+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec404f2e0 con 0x7f7ec4068580 2024-09-17T00:15:52.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.215+0000 7f7eba7fc700 1 -- 172.21.15.42:0/782240181 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f7ec0004350 con 0x7f7ec4068580 2024-09-17T00:15:52.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.216+0000 7f7eba7fc700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7eb006e680 0x7f7eb0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:52.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.216+0000 7f7eba7fc700 1 -- 172.21.15.42:0/782240181 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7ec008e4f0 con 0x7f7ec4068580 2024-09-17T00:15:52.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.216+0000 7f7ec8e20700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7eb006e680 0x7f7eb0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:52.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.216+0000 7f7ec8e20700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7eb006e680 0x7f7eb0070b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4005b40 tx=0x7f7eb401a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:52.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.220+0000 7f7eba7fc700 1 -- 172.21.15.42:0/782240181 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7ec005b240 con 0x7f7ec4068580 2024-09-17T00:15:52.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.475+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f7ec4199dc0 con 0x7f7ec4068580 2024-09-17T00:15:52.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.476+0000 7f7eba7fc700 1 -- 172.21.15.42:0/782240181 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v36) v1 ==== 107+0+4182 (secure 0 0 0) 0x7f7ec005add0 con 0x7f7ec4068580 2024-09-17T00:15:52.473 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:52.473 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":24,"btime":"2024-09-16T23:57:34:468457+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44307,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/3469372237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3469372237},{"type":"v1","addr":"172.21.15.43:6825","nonce":3469372237}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:52.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7eb006e680 msgr2=0x7f7eb0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:52.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7eb006e680 0x7f7eb0070b40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4005b40 tx=0x7f7eb401a040 comp rx=0 tx=0).stop 2024-09-17T00:15:52.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 msgr2=0x7f7ec419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:52.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec419e830 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000d900 tx=0x7f7ec000dcc0 comp rx=0 tx=0).stop 2024-09-17T00:15:52.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 shutdown_connections 2024-09-17T00:15:52.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f7eb006e680 0x7f7eb0070b40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:52.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ec4068580 0x7f7ec419e830 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:52.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 --2- 172.21.15.42:0/782240181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ec4106cd0 0x7f7ec419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:52.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 >> 172.21.15.42:0/782240181 conn(0x7f7ec4075670 msgr2=0x7f7ec40fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:52.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 shutdown_connections 2024-09-17T00:15:52.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:52.478+0000 7f7ecb885700 1 -- 172.21.15.42:0/782240181 wait complete. 2024-09-17T00:15:52.476 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 24 2024-09-17T00:15:53.179 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:53 smithi042 ceph-mon[111056]: pgmap v730: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:53.179 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:53 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/782240181' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-09-17T00:15:53.200 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:15:53.201 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 25 2024-09-17T00:15:53.320 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:53 smithi043 ceph-mon[81510]: pgmap v730: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:53.320 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:53 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/782240181' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-09-17T00:15:53.406 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:55.158 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:55 smithi042 ceph-mon[111056]: pgmap v731: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 -- 172.21.15.42:0/3299127051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 msgr2=0x7f60701058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 --2- 172.21.15.42:0/3299127051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60701058a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6058009910 tx=0x7f6058009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 -- 172.21.15.42:0/3299127051 shutdown_connections 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 --2- 172.21.15.42:0/3299127051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60701058a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 --2- 172.21.15.42:0/3299127051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6070100ea0 0x7f6070101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 -- 172.21.15.42:0/3299127051 >> 172.21.15.42:0/3299127051 conn(0x7f60700fc750 msgr2=0x7f60700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 -- 172.21.15.42:0/3299127051 shutdown_connections 2024-09-17T00:15:55.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.160+0000 7f60758e7700 1 -- 172.21.15.42:0/3299127051 wait complete. 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f60758e7700 1 Processor -- start 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f60758e7700 1 -- start start 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f60758e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6070100ea0 0x7f6070073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f60758e7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60700735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f60758e7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070073b10 con 0x7f6070101850 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f60758e7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6070073c50 con 0x7f6070100ea0 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f606e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60700735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f606e7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60700735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46112/0 (socket says 172.21.15.42:46112) 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f606e7fc700 1 -- 172.21.15.42:0/504068105 learned_addr learned my addr 172.21.15.42:0/504068105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.161+0000 7f606effd700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6070100ea0 0x7f6070073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f606e7fc700 1 -- 172.21.15.42:0/504068105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6070100ea0 msgr2=0x7f6070073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f606e7fc700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6070100ea0 0x7f6070073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:55.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f606e7fc700 1 -- 172.21.15.42:0/504068105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580095f0 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f606e7fc700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60700735d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f60580059d0 tx=0x7f605800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f60748e5700 1 -- 172.21.15.42:0/504068105 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605801d070 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f60748e5700 1 -- 172.21.15.42:0/504068105 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f605800b610 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f60748e5700 1 -- 172.21.15.42:0/504068105 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6058022470 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6070073ed0 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.162+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701a2e30 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.163+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60701092a0 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.164+0000 7f60748e5700 1 -- 172.21.15.42:0/504068105 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f605800fc70 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.164+0000 7f60748e5700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f605c06e680 0x7f605c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.164+0000 7f60748e5700 1 -- 172.21.15.42:0/504068105 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f605808fa20 con 0x7f6070101850 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.164+0000 7f606effd700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f605c06e680 0x7f605c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.165+0000 7f606effd700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f605c06e680 0x7f605c070b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6060009380 tx=0x7f606000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:55.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.169+0000 7f60748e5700 1 -- 172.21.15.42:0/504068105 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6058058c40 con 0x7f6070101850 2024-09-17T00:15:55.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:55 smithi043 ceph-mon[81510]: pgmap v731: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:55.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.423+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f607004f2e0 con 0x7f6070101850 2024-09-17T00:15:55.421 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.424+0000 7f60748e5700 1 -- 172.21.15.42:0/504068105 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v36) v1 ==== 107+0+5029 (secure 0 0 0) 0x7f605805c300 con 0x7f6070101850 2024-09-17T00:15:55.421 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:55.422 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":25,"btime":"2024-09-16T23:57:38:482006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44307,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/3469372237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3469372237},{"type":"v1","addr":"172.21.15.43:6825","nonce":3469372237}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:55.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f605c06e680 msgr2=0x7f605c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:55.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f605c06e680 0x7f605c070b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6060009380 tx=0x7f606000b040 comp rx=0 tx=0).stop 2024-09-17T00:15:55.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 msgr2=0x7f60700735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:55.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60700735d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f60580059d0 tx=0x7f605800f740 comp rx=0 tx=0).stop 2024-09-17T00:15:55.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 shutdown_connections 2024-09-17T00:15:55.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f605c06e680 0x7f605c070b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:55.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6070100ea0 0x7f6070073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:55.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 --2- 172.21.15.42:0/504068105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6070101850 0x7f60700735d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:55.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.426+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 >> 172.21.15.42:0/504068105 conn(0x7f60700fc750 msgr2=0x7f60700fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:55.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.427+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 shutdown_connections 2024-09-17T00:15:55.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:55.427+0000 7f60758e7700 1 -- 172.21.15.42:0/504068105 wait complete. 2024-09-17T00:15:55.425 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 25 2024-09-17T00:15:56.043 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:15:56.043 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 26 2024-09-17T00:15:56.252 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:56.374 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:56 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/504068105' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-09-17T00:15:56.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:56 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/504068105' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-09-17T00:15:57.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:57 smithi043 ceph-mon[81510]: pgmap v732: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:57.418 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:57 smithi042 ceph-mon[111056]: pgmap v732: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 -- 172.21.15.42:0/2839899207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c100750 msgr2=0x7f175c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 --2- 172.21.15.42:0/2839899207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c100750 0x7f175c1047a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f174c009910 tx=0x7f174c009c20 comp rx=0 tx=0).stop 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 -- 172.21.15.42:0/2839899207 shutdown_connections 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 --2- 172.21.15.42:0/2839899207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c100750 0x7f175c1047a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 --2- 172.21.15.42:0/2839899207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f175c0ffda0 0x7f175c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 -- 172.21.15.42:0/2839899207 >> 172.21.15.42:0/2839899207 conn(0x7f175c0fb650 msgr2=0x7f175c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 -- 172.21.15.42:0/2839899207 shutdown_connections 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.974+0000 7f176378f700 1 -- 172.21.15.42:0/2839899207 wait complete. 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176378f700 1 Processor -- start 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176378f700 1 -- start start 2024-09-17T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176378f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c0ffda0 0x7f175c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:57.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176378f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f175c100750 0x7f175c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:57.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176378f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c19d240 con 0x7f175c0ffda0 2024-09-17T00:15:57.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176378f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1966f0 con 0x7f175c100750 2024-09-17T00:15:57.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176152b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c0ffda0 0x7f175c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:57.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176152b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c0ffda0 0x7f175c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46128/0 (socket says 172.21.15.42:46128) 2024-09-17T00:15:57.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f176152b700 1 -- 172.21.15.42:0/3807536684 learned_addr learned my addr 172.21.15.42:0/3807536684 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:15:57.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.975+0000 7f1760d2a700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f175c100750 0x7f175c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:57.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f176152b700 1 -- 172.21.15.42:0/3807536684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f175c100750 msgr2=0x7f175c19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:57.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f176152b700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f175c100750 0x7f175c19cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:57.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f176152b700 1 -- 172.21.15.42:0/3807536684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c0095f0 con 0x7f175c0ffda0 2024-09-17T00:15:57.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f176152b700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c0ffda0 0x7f175c19c670 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f175800d8d0 tx=0x7f175800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:57.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f17527fc700 1 -- 172.21.15.42:0/3807536684 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1758009940 con 0x7f175c0ffda0 2024-09-17T00:15:57.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f17527fc700 1 -- 172.21.15.42:0/3807536684 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1758010460 con 0x7f175c0ffda0 2024-09-17T00:15:57.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c1969d0 con 0x7f175c0ffda0 2024-09-17T00:15:57.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f17527fc700 1 -- 172.21.15.42:0/3807536684 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f175800f5f0 con 0x7f175c0ffda0 2024-09-17T00:15:57.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.976+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c196f20 con 0x7f175c0ffda0 2024-09-17T00:15:57.978 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.977+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c1091b0 con 0x7f175c0ffda0 2024-09-17T00:15:57.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.978+0000 7f17527fc700 1 -- 172.21.15.42:0/3807536684 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f17580105d0 con 0x7f175c0ffda0 2024-09-17T00:15:57.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.978+0000 7f17527fc700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f174806e680 0x7f1748070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:15:57.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.978+0000 7f17527fc700 1 -- 172.21.15.42:0/3807536684 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f175808e680 con 0x7f175c0ffda0 2024-09-17T00:15:57.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.978+0000 7f1760d2a700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f174806e680 0x7f1748070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:15:57.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.979+0000 7f1760d2a700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f174806e680 0x7f1748070b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f175c197b60 tx=0x7f174c01a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:15:57.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:57.983+0000 7f17527fc700 1 -- 172.21.15.42:0/3807536684 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f175805b350 con 0x7f175c0ffda0 2024-09-17T00:15:58.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.264+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f175c04f2e0 con 0x7f175c0ffda0 2024-09-17T00:15:58.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.265+0000 7f17527fc700 1 -- 172.21.15.42:0/3807536684 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v36) v1 ==== 107+0+4250 (secure 0 0 0) 0x7f175805aee0 con 0x7f175c0ffda0 2024-09-17T00:15:58.262 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:15:58.262 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":26,"btime":"2024-09-16T23:57:51:395431+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:15:58.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f174806e680 msgr2=0x7f1748070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:58.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f174806e680 0x7f1748070b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f175c197b60 tx=0x7f174c01a040 comp rx=0 tx=0).stop 2024-09-17T00:15:58.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c0ffda0 msgr2=0x7f175c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:15:58.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c0ffda0 0x7f175c19c670 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f175800d8d0 tx=0x7f175800dc90 comp rx=0 tx=0).stop 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 shutdown_connections 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f174806e680 0x7f1748070b40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f175c0ffda0 0x7f175c19c670 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 --2- 172.21.15.42:0/3807536684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f175c100750 0x7f175c19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 >> 172.21.15.42:0/3807536684 conn(0x7f175c0fb650 msgr2=0x7f175c069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 shutdown_connections 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:15:58.267+0000 7f176378f700 1 -- 172.21.15.42:0/3807536684 wait complete. 2024-09-17T00:15:58.265 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 26 2024-09-17T00:15:58.910 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:15:58.910 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 27 2024-09-17T00:15:59.125 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:15:59.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:59 smithi042 ceph-mon[111056]: pgmap v733: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:59.246 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:15:59 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3807536684' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-09-17T00:15:59.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:59 smithi043 ceph-mon[81510]: pgmap v733: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:15:59.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:15:59 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3807536684' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 -- 172.21.15.42:0/1260627377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 msgr2=0x7f61f410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 --2- 172.21.15.42:0/1260627377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f410abf0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009910 tx=0x7f61e8009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 -- 172.21.15.42:0/1260627377 shutdown_connections 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 --2- 172.21.15.42:0/1260627377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f410abf0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 --2- 172.21.15.42:0/1260627377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f4101de0 0x7f61f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 -- 172.21.15.42:0/1260627377 >> 172.21.15.42:0/1260627377 conn(0x7f61f40fb610 msgr2=0x7f61f40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 -- 172.21.15.42:0/1260627377 shutdown_connections 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.831+0000 7f61f9619700 1 -- 172.21.15.42:0/1260627377 wait complete. 2024-09-17T00:16:00.829 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.832+0000 7f61f9619700 1 Processor -- start 2024-09-17T00:16:00.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.832+0000 7f61f9619700 1 -- start start 2024-09-17T00:16:00.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.832+0000 7f61f9619700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f4101de0 0x7f61f419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:00.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.832+0000 7f61f9619700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:00.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.832+0000 7f61f9619700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f419f2c0 con 0x7f61f4102700 2024-09-17T00:16:00.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.832+0000 7f61f9619700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f41988e0 con 0x7f61f4101de0 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46154/0 (socket says 172.21.15.42:46154) 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f27fc700 1 -- 172.21.15.42:0/1166288571 learned_addr learned my addr 172.21.15.42:0/1166288571 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f2ffd700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f4101de0 0x7f61f419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f27fc700 1 -- 172.21.15.42:0/1166288571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f4101de0 msgr2=0x7f61f419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f27fc700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f4101de0 0x7f61f419e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f27fc700 1 -- 172.21.15.42:0/1166288571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80095f0 con 0x7f61f4102700 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f2ffd700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f4101de0 0x7f61f419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f27fc700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f419ed30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f61e800bda0 tx=0x7f61e800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61dbfff700 1 -- 172.21.15.42:0/1166288571 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e801d070 con 0x7f61f4102700 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f4198b60 con 0x7f61f4102700 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61dbfff700 1 -- 172.21.15.42:0/1166288571 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61e800f460 con 0x7f61f4102700 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61dbfff700 1 -- 172.21.15.42:0/1166288571 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e8017630 con 0x7f61f4102700 2024-09-17T00:16:00.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.833+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f4199050 con 0x7f61f4102700 2024-09-17T00:16:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.834+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f410a750 con 0x7f61f4102700 2024-09-17T00:16:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.835+0000 7f61dbfff700 1 -- 172.21.15.42:0/1166288571 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f61e8003a60 con 0x7f61f4102700 2024-09-17T00:16:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.835+0000 7f61dbfff700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61dc06e680 0x7f61dc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:00.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.835+0000 7f61dbfff700 1 -- 172.21.15.42:0/1166288571 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f61e808fa40 con 0x7f61f4102700 2024-09-17T00:16:00.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.835+0000 7f61f2ffd700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61dc06e680 0x7f61dc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:00.834 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.836+0000 7f61f2ffd700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61dc06e680 0x7f61dc070b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e300 tx=0x7f61e4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:00.839 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:00.842+0000 7f61dbfff700 1 -- 172.21.15.42:0/1166288571 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f61e805c790 con 0x7f61f4102700 2024-09-17T00:16:01.097 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.100+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f61f4199df0 con 0x7f61f4102700 2024-09-17T00:16:01.098 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:01 smithi042 ceph-mon[111056]: pgmap v734: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:01.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.100+0000 7f61dbfff700 1 -- 172.21.15.42:0/1166288571 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v36) v1 ==== 107+0+5097 (secure 0 0 0) 0x7f61e805c320 con 0x7f61f4102700 2024-09-17T00:16:01.098 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:01.098 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"btime":"2024-09-16T23:57:54:403900+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24279,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.43:6827/2825700082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":2825700082},{"type":"v1","addr":"172.21.15.43:6827","nonce":2825700082}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61dc06e680 msgr2=0x7f61dc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61dc06e680 0x7f61dc070b40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e300 tx=0x7f61e4009040 comp rx=0 tx=0).stop 2024-09-17T00:16:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 msgr2=0x7f61f419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:01.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f419ed30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f61e800bda0 tx=0x7f61e800be80 comp rx=0 tx=0).stop 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 shutdown_connections 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f61dc06e680 0x7f61dc070b40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61f4101de0 0x7f61f419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 --2- 172.21.15.42:0/1166288571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61f4102700 0x7f61f419ed30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 >> 172.21.15.42:0/1166288571 conn(0x7f61f40fb610 msgr2=0x7f61f41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 shutdown_connections 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:01.103+0000 7f61f9619700 1 -- 172.21.15.42:0/1166288571 wait complete. 2024-09-17T00:16:01.101 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2024-09-17T00:16:01.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:01 smithi043 ceph-mon[81510]: pgmap v734: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:01.800 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:16:01.800 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 28 2024-09-17T00:16:02.012 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:02.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:02 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1166288571' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-09-17T00:16:02.394 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:02 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1166288571' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-09-17T00:16:03.190 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:03 smithi042 ceph-mon[111056]: pgmap v735: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:03.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:03 smithi043 ceph-mon[81510]: pgmap v735: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 -- 172.21.15.42:0/3733857099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 msgr2=0x7fd7f01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 --2- 172.21.15.42:0/3733857099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f01058a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009910 tx=0x7fd7e0009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 -- 172.21.15.42:0/3733857099 shutdown_connections 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 --2- 172.21.15.42:0/3733857099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f01058a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 --2- 172.21.15.42:0/3733857099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f0100ea0 0x7fd7f0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 -- 172.21.15.42:0/3733857099 >> 172.21.15.42:0/3733857099 conn(0x7fd7f00fc750 msgr2=0x7fd7f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 -- 172.21.15.42:0/3733857099 shutdown_connections 2024-09-17T00:16:03.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.744+0000 7fd7f7641700 1 -- 172.21.15.42:0/3733857099 wait complete. 2024-09-17T00:16:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.745+0000 7fd7f7641700 1 Processor -- start 2024-09-17T00:16:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.745+0000 7fd7f7641700 1 -- start start 2024-09-17T00:16:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.745+0000 7fd7f7641700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f0100ea0 0x7fd7f0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.745+0000 7fd7f7641700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.745+0000 7fd7f7641700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0073b10 con 0x7fd7f0101850 2024-09-17T00:16:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.745+0000 7fd7f7641700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0073c50 con 0x7fd7f0100ea0 2024-09-17T00:16:03.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.745+0000 7fd7f4bdc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f4bdc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60654/0 (socket says 172.21.15.42:60654) 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f4bdc700 1 -- 172.21.15.42:0/1737091874 learned_addr learned my addr 172.21.15.42:0/1737091874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f53dd700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f0100ea0 0x7fd7f0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f4bdc700 1 -- 172.21.15.42:0/1737091874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f0100ea0 msgr2=0x7fd7f0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f4bdc700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f0100ea0 0x7fd7f0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f4bdc700 1 -- 172.21.15.42:0/1737091874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e00095f0 con 0x7fd7f0101850 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f4bdc700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f00735d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000c00 tx=0x7fd7e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7e67fc700 1 -- 172.21.15.42:0/1737091874 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e001d070 con 0x7fd7f0101850 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7e67fc700 1 -- 172.21.15.42:0/1737091874 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7e0004fb0 con 0x7fd7f0101850 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7e67fc700 1 -- 172.21.15.42:0/1737091874 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e000f460 con 0x7fd7f0101850 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f0073ed0 con 0x7fd7f0101850 2024-09-17T00:16:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.746+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f01a2e30 con 0x7fd7f0101850 2024-09-17T00:16:03.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.747+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f01092a0 con 0x7fd7f0101850 2024-09-17T00:16:03.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.748+0000 7fd7e67fc700 1 -- 172.21.15.42:0/1737091874 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd7e00228a0 con 0x7fd7f0101850 2024-09-17T00:16:03.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.748+0000 7fd7e67fc700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7dc06e680 0x7fd7dc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:03.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.748+0000 7fd7e67fc700 1 -- 172.21.15.42:0/1737091874 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd7e008fb90 con 0x7fd7f0101850 2024-09-17T00:16:03.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.748+0000 7fd7f53dd700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7dc06e680 0x7fd7dc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:03.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.749+0000 7fd7f53dd700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7dc06e680 0x7fd7dc070b40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0060b0 tx=0x7fd7ec007b80 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:03.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:03.753+0000 7fd7e67fc700 1 -- 172.21.15.42:0/1737091874 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd7e0058f80 con 0x7fd7f0101850 2024-09-17T00:16:04.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.009+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fd7f00745f0 con 0x7fd7f0101850 2024-09-17T00:16:04.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.010+0000 7fd7e67fc700 1 -- 172.21.15.42:0/1737091874 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v36) v1 ==== 107+0+4318 (secure 0 0 0) 0x7fd7e0027020 con 0x7fd7f0101850 2024-09-17T00:16:04.007 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:04.007 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":28,"btime":"2024-09-16T23:58:04:819536+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7dc06e680 msgr2=0x7fd7dc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7dc06e680 0x7fd7dc070b40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0060b0 tx=0x7fd7ec007b80 comp rx=0 tx=0).stop 2024-09-17T00:16:04.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 msgr2=0x7fd7f00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f00735d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000c00 tx=0x7fd7e000be20 comp rx=0 tx=0).stop 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 shutdown_connections 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd7dc06e680 0x7fd7dc070b40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7f0100ea0 0x7fd7f0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 --2- 172.21.15.42:0/1737091874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7f0101850 0x7fd7f00735d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 >> 172.21.15.42:0/1737091874 conn(0x7fd7f00fc750 msgr2=0x7fd7f00fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 shutdown_connections 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:04.012+0000 7fd7f7641700 1 -- 172.21.15.42:0/1737091874 wait complete. 2024-09-17T00:16:04.010 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 28 2024-09-17T00:16:04.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:04 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1737091874' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-09-17T00:16:04.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:04 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1737091874' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-09-17T00:16:04.647 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:16:04.647 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 29 2024-09-17T00:16:04.860 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:05.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:05 smithi042 ceph-mon[111056]: pgmap v736: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:05.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:05 smithi043 ceph-mon[81510]: pgmap v736: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:06.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:06.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 -- 172.21.15.42:0/1976802391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 msgr2=0x7f83dc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 --2- 172.21.15.42:0/1976802391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc1058a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009910 tx=0x7f83cc009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 -- 172.21.15.42:0/1976802391 shutdown_connections 2024-09-17T00:16:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 --2- 172.21.15.42:0/1976802391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc1058a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 --2- 172.21.15.42:0/1976802391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83dc100ea0 0x7f83dc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 -- 172.21.15.42:0/1976802391 >> 172.21.15.42:0/1976802391 conn(0x7f83dc0fc750 msgr2=0x7f83dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:06.515 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 -- 172.21.15.42:0/1976802391 shutdown_connections 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 -- 172.21.15.42:0/1976802391 wait complete. 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 Processor -- start 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.516+0000 7f83e4568700 1 -- start start 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e4568700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83dc100ea0 0x7f83dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e4568700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e4568700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc073b10 con 0x7f83dc101850 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e4568700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc073c50 con 0x7f83dc100ea0 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e1b03700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e1b03700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60686/0 (socket says 172.21.15.42:60686) 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e1b03700 1 -- 172.21.15.42:0/2171994663 learned_addr learned my addr 172.21.15.42:0/2171994663 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:06.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e2304700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83dc100ea0 0x7f83dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:06.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e1b03700 1 -- 172.21.15.42:0/2171994663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83dc100ea0 msgr2=0x7f83dc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:06.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e1b03700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83dc100ea0 0x7f83dc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e1b03700 1 -- 172.21.15.42:0/2171994663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0095f0 con 0x7f83dc101850 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83e1b03700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc0735d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0059d0 tx=0x7f83cc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.517+0000 7f83d37fe700 1 -- 172.21.15.42:0/2171994663 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83cc01d070 con 0x7f83dc101850 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.518+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc073ed0 con 0x7f83dc101850 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.518+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc1a2db0 con 0x7f83dc101850 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.518+0000 7f83d37fe700 1 -- 172.21.15.42:0/2171994663 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83cc00b610 con 0x7f83dc101850 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.518+0000 7f83d37fe700 1 -- 172.21.15.42:0/2171994663 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83cc022470 con 0x7f83dc101850 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.518+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c0005320 con 0x7f83dc101850 2024-09-17T00:16:06.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.519+0000 7f83d37fe700 1 -- 172.21.15.42:0/2171994663 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f83cc00fb70 con 0x7f83dc101850 2024-09-17T00:16:06.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.520+0000 7f83d37fe700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f83c806e680 0x7f83c8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:06.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.520+0000 7f83e2304700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f83c806e680 0x7f83c8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:06.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.520+0000 7f83d37fe700 1 -- 172.21.15.42:0/2171994663 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f83cc08fd40 con 0x7f83dc101850 2024-09-17T00:16:06.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.521+0000 7f83e2304700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f83c806e680 0x7f83c8070b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009bf0 tx=0x7f83d8009260 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:06.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.524+0000 7f83d37fe700 1 -- 172.21.15.42:0/2171994663 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f83cc058f60 con 0x7f83dc101850 2024-09-17T00:16:06.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.777+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f83c0005190 con 0x7f83dc101850 2024-09-17T00:16:06.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.777+0000 7f83d37fe700 1 -- 172.21.15.42:0/2171994663 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v36) v1 ==== 107+0+5165 (secure 0 0 0) 0x7f83cc05c620 con 0x7f83dc101850 2024-09-17T00:16:06.776 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:06.776 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":29,"btime":"2024-09-16T23:58:07:827497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":20,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-16T23:57:12.183428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f83c806e680 msgr2=0x7f83c8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f83c806e680 0x7f83c8070b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009bf0 tx=0x7f83d8009260 comp rx=0 tx=0).stop 2024-09-17T00:16:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 msgr2=0x7f83dc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc0735d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f83cc0059d0 tx=0x7f83cc00f740 comp rx=0 tx=0).stop 2024-09-17T00:16:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 shutdown_connections 2024-09-17T00:16:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f83c806e680 0x7f83c8070b40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:06.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83dc100ea0 0x7f83dc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 --2- 172.21.15.42:0/2171994663 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83dc101850 0x7f83dc0735d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 >> 172.21.15.42:0/2171994663 conn(0x7f83dc0fc750 msgr2=0x7f83dc105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 shutdown_connections 2024-09-17T00:16:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:06.780+0000 7f83e4568700 1 -- 172.21.15.42:0/2171994663 wait complete. 2024-09-17T00:16:06.779 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 29 2024-09-17T00:16:07.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:07 smithi043 ceph-mon[81510]: pgmap v737: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:07.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:07 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2171994663' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-09-17T00:16:07.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:07 smithi042 ceph-mon[111056]: pgmap v737: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:07.429 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:07 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2171994663' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-09-17T00:16:07.455 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-17T00:16:07.455 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 30 2024-09-17T00:16:07.668 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:09.340 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:09 smithi042 ceph-mon[111056]: pgmap v738: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:09.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 -- 172.21.15.42:0/3156795529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 msgr2=0x7f2d440ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/3156795529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d440ff980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009910 tx=0x7f2d34009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 -- 172.21.15.42:0/3156795529 shutdown_connections 2024-09-17T00:16:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/3156795529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d440fff50 0x7f2d44105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/3156795529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d440ff980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 -- 172.21.15.42:0/3156795529 >> 172.21.15.42:0/3156795529 conn(0x7f2d44074e30 msgr2=0x7f2d44075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 -- 172.21.15.42:0/3156795529 shutdown_connections 2024-09-17T00:16:09.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.374+0000 7f2d4acb8700 1 -- 172.21.15.42:0/3156795529 wait complete. 2024-09-17T00:16:09.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d4acb8700 1 Processor -- start 2024-09-17T00:16:09.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d4acb8700 1 -- start start 2024-09-17T00:16:09.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d4acb8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d44196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d4acb8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d440fff50 0x7f2d44196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d4acb8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d44197520 con 0x7f2d440ff5a0 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d4acb8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4419b330 con 0x7f2d440fff50 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d48a54700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d44196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d48a54700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d44196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60694/0 (socket says 172.21.15.42:60694) 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d48a54700 1 -- 172.21.15.42:0/2858400955 learned_addr learned my addr 172.21.15.42:0/2858400955 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.375+0000 7f2d43fff700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d440fff50 0x7f2d44196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d48a54700 1 -- 172.21.15.42:0/2858400955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d440fff50 msgr2=0x7f2d44196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d48a54700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d440fff50 0x7f2d44196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d48a54700 1 -- 172.21.15.42:0/2858400955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d340095f0 con 0x7f2d440ff5a0 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d48a54700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d44196950 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006010 tx=0x7f2d3400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d41ffb700 1 -- 172.21.15.42:0/2858400955 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d3401d070 con 0x7f2d440ff5a0 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4419b5b0 con 0x7f2d440ff5a0 2024-09-17T00:16:09.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d41ffb700 1 -- 172.21.15.42:0/2858400955 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d34005dc0 con 0x7f2d440ff5a0 2024-09-17T00:16:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d41ffb700 1 -- 172.21.15.42:0/2858400955 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d34017400 con 0x7f2d440ff5a0 2024-09-17T00:16:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.376+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4419baa0 con 0x7f2d440ff5a0 2024-09-17T00:16:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.377+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d44109270 con 0x7f2d440ff5a0 2024-09-17T00:16:09.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.378+0000 7f2d41ffb700 1 -- 172.21.15.42:0/2858400955 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f2d34003a60 con 0x7f2d440ff5a0 2024-09-17T00:16:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.378+0000 7f2d41ffb700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2d2c06e680 0x7f2d2c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.378+0000 7f2d41ffb700 1 -- 172.21.15.42:0/2858400955 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2d3408fb80 con 0x7f2d440ff5a0 2024-09-17T00:16:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.379+0000 7f2d43fff700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2d2c06e680 0x7f2d2c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:09.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.379+0000 7f2d43fff700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2d2c06e680 0x7f2d2c070b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d44197f70 tx=0x7f2d3800b3c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:09.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:09 smithi043 ceph-mon[81510]: pgmap v738: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:09.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.383+0000 7f2d41ffb700 1 -- 172.21.15.42:0/2858400955 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2d3405c7b0 con 0x7f2d440ff5a0 2024-09-17T00:16:09.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.640+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f2d4404f2e0 con 0x7f2d440ff5a0 2024-09-17T00:16:09.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.641+0000 7f2d41ffb700 1 -- 172.21.15.42:0/2858400955 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v36) v1 ==== 107+0+5164 (secure 0 0 0) 0x7f2d3405c340 con 0x7f2d440ff5a0 2024-09-17T00:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:09.639 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":30,"btime":"2024-09-17T00:13:09:000373+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-17T00:13:08.003220+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:09.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2d2c06e680 msgr2=0x7f2d2c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:09.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2d2c06e680 0x7f2d2c070b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d44197f70 tx=0x7f2d3800b3c0 comp rx=0 tx=0).stop 2024-09-17T00:16:09.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 msgr2=0x7f2d44196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:09.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d44196950 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2d34006010 tx=0x7f2d3400bd70 comp rx=0 tx=0).stop 2024-09-17T00:16:09.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 shutdown_connections 2024-09-17T00:16:09.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f2d2c06e680 0x7f2d2c070b40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:09.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d440ff5a0 0x7f2d44196950 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:09.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 --2- 172.21.15.42:0/2858400955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d440fff50 0x7f2d44196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:09.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 >> 172.21.15.42:0/2858400955 conn(0x7f2d44074e30 msgr2=0x7f2d440fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:09.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 shutdown_connections 2024-09-17T00:16:09.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:09.644+0000 7f2d4acb8700 1 -- 172.21.15.42:0/2858400955 wait complete. 2024-09-17T00:16:09.643 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 30 2024-09-17T00:16:10.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:10 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2858400955' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-09-17T00:16:10.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 31 2024-09-17T00:16:10.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:10 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2858400955' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-09-17T00:16:10.554 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:11.361 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:11 smithi042 ceph-mon[111056]: pgmap v739: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:11.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:11 smithi043 ceph-mon[81510]: pgmap v739: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 -- 172.21.15.42:0/953628863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138101e00 msgr2=0x7fb1381021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 --2- 172.21.15.42:0/953628863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138101e00 0x7fb1381021e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb128009910 tx=0x7fb128009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 -- 172.21.15.42:0/953628863 shutdown_connections 2024-09-17T00:16:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 --2- 172.21.15.42:0/953628863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb138102720 0x7fb13810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 --2- 172.21.15.42:0/953628863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138101e00 0x7fb1381021e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 -- 172.21.15.42:0/953628863 >> 172.21.15.42:0/953628863 conn(0x7fb1380fb650 msgr2=0x7fb1380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:12.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 -- 172.21.15.42:0/953628863 shutdown_connections 2024-09-17T00:16:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.188+0000 7fb13eae0700 1 -- 172.21.15.42:0/953628863 wait complete. 2024-09-17T00:16:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb13eae0700 1 Processor -- start 2024-09-17T00:16:12.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb13eae0700 1 -- start start 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb13eae0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb138101e00 0x7fb13819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb13eae0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138102720 0x7fb13819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb13eae0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13819f450 con 0x7fb138102720 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb13eae0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb138198900 con 0x7fb138101e00 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb137fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138102720 0x7fb13819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb137fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138102720 0x7fb13819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60712/0 (socket says 172.21.15.42:60712) 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb137fff700 1 -- 172.21.15.42:0/3132596715 learned_addr learned my addr 172.21.15.42:0/3132596715 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:12.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.189+0000 7fb13c87c700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb138101e00 0x7fb13819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb137fff700 1 -- 172.21.15.42:0/3132596715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb138101e00 msgr2=0x7fb13819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb137fff700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb138101e00 0x7fb13819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb137fff700 1 -- 172.21.15.42:0/3132596715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1280095f0 con 0x7fb138102720 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb137fff700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138102720 0x7fb13819edc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00d6d0 tx=0x7fb12c00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb135ffb700 1 -- 172.21.15.42:0/3132596715 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12c00f860 con 0x7fb138102720 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb135ffb700 1 -- 172.21.15.42:0/3132596715 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb12c00fe80 con 0x7fb138102720 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb138198be0 con 0x7fb138102720 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb135ffb700 1 -- 172.21.15.42:0/3132596715 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb12c010ea0 con 0x7fb138102720 2024-09-17T00:16:12.190 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.190+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb138199130 con 0x7fb138102720 2024-09-17T00:16:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.191+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13810a770 con 0x7fb138102720 2024-09-17T00:16:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.192+0000 7fb135ffb700 1 -- 172.21.15.42:0/3132596715 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb12c010460 con 0x7fb138102720 2024-09-17T00:16:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.192+0000 7fb135ffb700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb12006e680 0x7fb120070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.192+0000 7fb135ffb700 1 -- 172.21.15.42:0/3132596715 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb12c08e2d0 con 0x7fb138102720 2024-09-17T00:16:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.192+0000 7fb13c87c700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb12006e680 0x7fb120070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:12.191 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.193+0000 7fb13c87c700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb12006e680 0x7fb120070b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b5c0 tx=0x7fb12801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:12.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.197+0000 7fb135ffb700 1 -- 172.21.15.42:0/3132596715 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb12c05b020 con 0x7fb138102720 2024-09-17T00:16:12.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.453+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fb138199400 con 0x7fb138102720 2024-09-17T00:16:12.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.454+0000 7fb135ffb700 1 -- 172.21.15.42:0/3132596715 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v36) v1 ==== 107+0+5255 (secure 0 0 0) 0x7fb12c05abb0 con 0x7fb138102720 2024-09-17T00:16:12.452 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:12.452 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":31,"btime":"2024-09-17T00:13:09:004900+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-17T00:13:09.004886+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34420":{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":1,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:12.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb12006e680 msgr2=0x7fb120070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb12006e680 0x7fb120070b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb12800b5c0 tx=0x7fb12801a040 comp rx=0 tx=0).stop 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138102720 msgr2=0x7fb13819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138102720 0x7fb13819edc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00d6d0 tx=0x7fb12c00d9e0 comp rx=0 tx=0).stop 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 shutdown_connections 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb12006e680 0x7fb120070b40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb138101e00 0x7fb13819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 --2- 172.21.15.42:0/3132596715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb138102720 0x7fb13819edc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 >> 172.21.15.42:0/3132596715 conn(0x7fb1380fb650 msgr2=0x7fb138100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 shutdown_connections 2024-09-17T00:16:12.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:12.456+0000 7fb13eae0700 1 -- 172.21.15.42:0/3132596715 wait complete. 2024-09-17T00:16:12.456 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 31 2024-09-17T00:16:13.171 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 32 2024-09-17T00:16:13.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:13 smithi043 ceph-mon[81510]: pgmap v740: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:13.380 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:13 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3132596715' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-09-17T00:16:13.389 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:13.471 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:13 smithi042 ceph-mon[111056]: pgmap v740: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:13.471 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:13 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3132596715' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-09-17T00:16:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 -- 172.21.15.42:0/567692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 msgr2=0x7fd6341058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 --2- 172.21.15.42:0/567692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd6341058a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd628009910 tx=0x7fd628009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:15.037 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 -- 172.21.15.42:0/567692 shutdown_connections 2024-09-17T00:16:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 --2- 172.21.15.42:0/567692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd6341058a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 --2- 172.21.15.42:0/567692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd634100ea0 0x7fd634101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 -- 172.21.15.42:0/567692 >> 172.21.15.42:0/567692 conn(0x7fd6340fc750 msgr2=0x7fd6340feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 -- 172.21.15.42:0/567692 shutdown_connections 2024-09-17T00:16:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.038+0000 7fd63a92d700 1 -- 172.21.15.42:0/567692 wait complete. 2024-09-17T00:16:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.039+0000 7fd63a92d700 1 Processor -- start 2024-09-17T00:16:15.038 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.039+0000 7fd63a92d700 1 -- start start 2024-09-17T00:16:15.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.039+0000 7fd63a92d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd634100ea0 0x7fd63419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:15.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.039+0000 7fd63a92d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd63419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.039+0000 7fd63a92d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63419d2d0 con 0x7fd634101850 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.039+0000 7fd63a92d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd634196780 con 0x7fd634100ea0 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6337fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd63419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6337fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd63419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44972/0 (socket says 172.21.15.42:44972) 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6337fe700 1 -- 172.21.15.42:0/2735246395 learned_addr learned my addr 172.21.15.42:0/2735246395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd633fff700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd634100ea0 0x7fd63419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6337fe700 1 -- 172.21.15.42:0/2735246395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd634100ea0 msgr2=0x7fd63419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:15.040 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6337fe700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd634100ea0 0x7fd63419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:15.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6337fe700 1 -- 172.21.15.42:0/2735246395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280095f0 con 0x7fd634101850 2024-09-17T00:16:15.041 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd633fff700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd634100ea0 0x7fd63419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:16:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6337fe700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd63419cc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd62800bda0 tx=0x7fd62800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6317fa700 1 -- 172.21.15.42:0/2735246395 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62801d070 con 0x7fd634101850 2024-09-17T00:16:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd634196a00 con 0x7fd634101850 2024-09-17T00:16:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6317fa700 1 -- 172.21.15.42:0/2735246395 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd628004fb0 con 0x7fd634101850 2024-09-17T00:16:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd6317fa700 1 -- 172.21.15.42:0/2735246395 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62800f480 con 0x7fd634101850 2024-09-17T00:16:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.040+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd634196ef0 con 0x7fd634101850 2024-09-17T00:16:15.042 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.041+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6341092a0 con 0x7fd634101850 2024-09-17T00:16:15.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.042+0000 7fd6317fa700 1 -- 172.21.15.42:0/2735246395 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fd628022870 con 0x7fd634101850 2024-09-17T00:16:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.042+0000 7fd6317fa700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd61c06e680 0x7fd61c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.043+0000 7fd6317fa700 1 -- 172.21.15.42:0/2735246395 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd62808fba0 con 0x7fd634101850 2024-09-17T00:16:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.043+0000 7fd633fff700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd61c06e680 0x7fd61c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:15.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.043+0000 7fd633fff700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd61c06e680 0x7fd61c070b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd624005950 tx=0x7fd624005f50 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:15.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.047+0000 7fd6317fa700 1 -- 172.21.15.42:0/2735246395 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd62805c920 con 0x7fd634101850 2024-09-17T00:16:15.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:15 smithi042 ceph-mon[111056]: pgmap v741: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:15.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.306+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fd63404f2e0 con 0x7fd634101850 2024-09-17T00:16:15.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.306+0000 7fd6317fa700 1 -- 172.21.15.42:0/2735246395 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v36) v1 ==== 107+0+5261 (secure 0 0 0) 0x7fd62805c4b0 con 0x7fd634101850 2024-09-17T00:16:15.305 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:15.305 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":32,"btime":"2024-09-17T00:13:14:737363+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-17T00:13:14.667097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":0,"incarnation":31,"state":"up:resolve","state_seq":240,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34420":{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":1,"incarnation":31,"state":"up:resolve","state_seq":236,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd61c06e680 msgr2=0x7fd61c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd61c06e680 0x7fd61c070b40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd624005950 tx=0x7fd624005f50 comp rx=0 tx=0).stop 2024-09-17T00:16:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 msgr2=0x7fd63419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:15.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd63419cc40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd62800bda0 tx=0x7fd62800be80 comp rx=0 tx=0).stop 2024-09-17T00:16:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 shutdown_connections 2024-09-17T00:16:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fd61c06e680 0x7fd61c070b40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd634100ea0 0x7fd63419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 --2- 172.21.15.42:0/2735246395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd634101850 0x7fd63419cc40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:15.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 >> 172.21.15.42:0/2735246395 conn(0x7fd6340fc750 msgr2=0x7fd6340fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 shutdown_connections 2024-09-17T00:16:15.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:15.309+0000 7fd63a92d700 1 -- 172.21.15.42:0/2735246395 wait complete. 2024-09-17T00:16:15.310 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 32 2024-09-17T00:16:15.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:15 smithi043 ceph-mon[81510]: pgmap v741: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:15.898 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 33 2024-09-17T00:16:16.112 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:16.234 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:16 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2735246395' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-09-17T00:16:16.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:16 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2735246395' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-09-17T00:16:17.241 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:17 smithi042 ceph-mon[111056]: pgmap v742: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:17.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:17 smithi043 ceph-mon[81510]: pgmap v742: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 -- 172.21.15.42:0/874569482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 msgr2=0x7fb0c00735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 --2- 172.21.15.42:0/874569482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c00735a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009910 tx=0x7fb0bc009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 -- 172.21.15.42:0/874569482 shutdown_connections 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 --2- 172.21.15.42:0/874569482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c0073ae0 0x7fb0c010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 --2- 172.21.15.42:0/874569482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c00735a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 -- 172.21.15.42:0/874569482 >> 172.21.15.42:0/874569482 conn(0x7fb0c00fc750 msgr2=0x7fb0c00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 -- 172.21.15.42:0/874569482 shutdown_connections 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.796+0000 7fb0c7869700 1 -- 172.21.15.42:0/874569482 wait complete. 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c7869700 1 Processor -- start 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c7869700 1 -- start start 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c7869700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c7869700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c0073ae0 0x7fb0c019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c7869700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c019f400 con 0x7fb0c00731c0 2024-09-17T00:16:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c7869700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c0198900 con 0x7fb0c0073ae0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c5605700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c5605700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44986/0 (socket says 172.21.15.42:44986) 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.797+0000 7fb0c5605700 1 -- 172.21.15.42:0/1540481660 learned_addr learned my addr 172.21.15.42:0/1540481660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0c4e04700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c0073ae0 0x7fb0c019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0c5605700 1 -- 172.21.15.42:0/1540481660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c0073ae0 msgr2=0x7fb0c019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0c5605700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c0073ae0 0x7fb0c019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0c5605700 1 -- 172.21.15.42:0/1540481660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0095f0 con 0x7fb0c00731c0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0c5605700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c019e830 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc006010 tx=0x7fb0bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0b67fc700 1 -- 172.21.15.42:0/1540481660 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0bc01d070 con 0x7fb0c00731c0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c0198b80 con 0x7fb0c00731c0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0b67fc700 1 -- 172.21.15.42:0/1540481660 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0bc004fb0 con 0x7fb0c00731c0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0b67fc700 1 -- 172.21.15.42:0/1540481660 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0bc00f480 con 0x7fb0c00731c0 2024-09-17T00:16:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.798+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c0199070 con 0x7fb0c00731c0 2024-09-17T00:16:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.799+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c010a640 con 0x7fb0c00731c0 2024-09-17T00:16:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.800+0000 7fb0b67fc700 1 -- 172.21.15.42:0/1540481660 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7fb0bc022890 con 0x7fb0c00731c0 2024-09-17T00:16:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.800+0000 7fb0b67fc700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0ac06e680 0x7fb0ac070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.800+0000 7fb0b67fc700 1 -- 172.21.15.42:0/1540481660 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb0bc08fc30 con 0x7fb0c00731c0 2024-09-17T00:16:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.800+0000 7fb0c4e04700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0ac06e680 0x7fb0ac070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.801+0000 7fb0c4e04700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0ac06e680 0x7fb0ac070b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005fd0 tx=0x7fb0b000a560 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:17.805+0000 7fb0b67fc700 1 -- 172.21.15.42:0/1540481660 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb0bc05cb50 con 0x7fb0c00731c0 2024-09-17T00:16:18.061 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.063+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fb0c004f2e0 con 0x7fb0c00731c0 2024-09-17T00:16:18.062 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.064+0000 7fb0b67fc700 1 -- 172.21.15.42:0/1540481660 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v36) v1 ==== 107+0+5265 (secure 0 0 0) 0x7fb0bc027020 con 0x7fb0c00731c0 2024-09-17T00:16:18.062 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:18.062 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":33,"btime":"2024-09-17T00:13:15:742159+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-17T00:13:14.771021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":241,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34420":{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":1,"incarnation":31,"state":"up:reconnect","state_seq":237,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:18.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0ac06e680 msgr2=0x7fb0ac070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0ac06e680 0x7fb0ac070b40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0005fd0 tx=0x7fb0b000a560 comp rx=0 tx=0).stop 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 msgr2=0x7fb0c019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c019e830 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc006010 tx=0x7fb0bc00be20 comp rx=0 tx=0).stop 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 shutdown_connections 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7fb0ac06e680 0x7fb0ac070b40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c00731c0 0x7fb0c019e830 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 --2- 172.21.15.42:0/1540481660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0c0073ae0 0x7fb0c019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 >> 172.21.15.42:0/1540481660 conn(0x7fb0c00fc750 msgr2=0x7fb0c0107780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 shutdown_connections 2024-09-17T00:16:18.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:18.066+0000 7fb0c7869700 1 -- 172.21.15.42:0/1540481660 wait complete. 2024-09-17T00:16:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 33 2024-09-17T00:16:18.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:18 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/1540481660' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-09-17T00:16:18.431 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:18 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/1540481660' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-09-17T00:16:18.770 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 34 2024-09-17T00:16:18.978 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:19.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:19 smithi043 ceph-mon[81510]: pgmap v743: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:19.427 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:19 smithi042 ceph-mon[111056]: pgmap v743: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:20.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 -- 172.21.15.42:0/233238723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 msgr2=0x7f6144101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:20.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 --2- 172.21.15.42:0/233238723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f6144101280 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6134009910 tx=0x7f6134009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:20.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 -- 172.21.15.42:0/233238723 shutdown_connections 2024-09-17T00:16:20.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 --2- 172.21.15.42:0/233238723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144101850 0x7f61441058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:20.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 --2- 172.21.15.42:0/233238723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f6144101280 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 -- 172.21.15.42:0/233238723 >> 172.21.15.42:0/233238723 conn(0x7f61440fc750 msgr2=0x7f61440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 -- 172.21.15.42:0/233238723 shutdown_connections 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.641+0000 7f614a150700 1 -- 172.21.15.42:0/233238723 wait complete. 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f614a150700 1 Processor -- start 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f614a150700 1 -- start start 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f614a150700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f614419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f614a150700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144101850 0x7f614419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f614a150700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614419d2c0 con 0x7f6144100ea0 2024-09-17T00:16:20.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f614a150700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6144196770 con 0x7f6144101850 2024-09-17T00:16:20.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f61437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f614419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:20.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f61437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f614419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:44996/0 (socket says 172.21.15.42:44996) 2024-09-17T00:16:20.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f61437fe700 1 -- 172.21.15.42:0/3474422215 learned_addr learned my addr 172.21.15.42:0/3474422215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f6142ffd700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144101850 0x7f614419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f61437fe700 1 -- 172.21.15.42:0/3474422215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144101850 msgr2=0x7f614419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f61437fe700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144101850 0x7f614419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.642+0000 7f61437fe700 1 -- 172.21.15.42:0/3474422215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61340095f0 con 0x7f6144100ea0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.643+0000 7f61437fe700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f614419c6f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f61340092e0 tx=0x7f613400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.643+0000 7f6140ff9700 1 -- 172.21.15.42:0/3474422215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f613401d070 con 0x7f6144100ea0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.643+0000 7f6140ff9700 1 -- 172.21.15.42:0/3474422215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6134005e00 con 0x7f6144100ea0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.643+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61441969f0 con 0x7f6144100ea0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.643+0000 7f6140ff9700 1 -- 172.21.15.42:0/3474422215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6134017400 con 0x7f6144100ea0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.643+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6144196ee0 con 0x7f6144100ea0 2024-09-17T00:16:20.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.644+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6144109240 con 0x7f6144100ea0 2024-09-17T00:16:20.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.644+0000 7f6140ff9700 1 -- 172.21.15.42:0/3474422215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f613400f6f0 con 0x7f6144100ea0 2024-09-17T00:16:20.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.645+0000 7f6140ff9700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f612c06e680 0x7f612c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:20.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.645+0000 7f6142ffd700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f612c06e680 0x7f612c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:20.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.645+0000 7f6140ff9700 1 -- 172.21.15.42:0/3474422215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f613408fa60 con 0x7f6144100ea0 2024-09-17T00:16:20.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.646+0000 7f6142ffd700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f612c06e680 0x7f612c070b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6144197be0 tx=0x7f613800b410 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:20.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.650+0000 7f6140ff9700 1 -- 172.21.15.42:0/3474422215 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f613405c7b0 con 0x7f6144100ea0 2024-09-17T00:16:20.912 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.914+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f614404f2e0 con 0x7f6144100ea0 2024-09-17T00:16:20.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.915+0000 7f6140ff9700 1 -- 172.21.15.42:0/3474422215 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v36) v1 ==== 107+0+5259 (secure 0 0 0) 0x7f613405c340 con 0x7f6144100ea0 2024-09-17T00:16:20.913 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:20.913 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":34,"btime":"2024-09-17T00:13:16:743609+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-17T00:13:15.747406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":242,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34420":{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":1,"incarnation":31,"state":"up:rejoin","state_seq":238,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-17T00:16:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f612c06e680 msgr2=0x7f612c070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f612c06e680 0x7f612c070b40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6144197be0 tx=0x7f613800b410 comp rx=0 tx=0).stop 2024-09-17T00:16:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 msgr2=0x7f614419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f614419c6f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f61340092e0 tx=0x7f613400bd70 comp rx=0 tx=0).stop 2024-09-17T00:16:20.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 shutdown_connections 2024-09-17T00:16:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f612c06e680 0x7f612c070b40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6144100ea0 0x7f614419c6f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 --2- 172.21.15.42:0/3474422215 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6144101850 0x7f614419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 >> 172.21.15.42:0/3474422215 conn(0x7f61440fc750 msgr2=0x7f61440fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 shutdown_connections 2024-09-17T00:16:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:20.917+0000 7f614a150700 1 -- 172.21.15.42:0/3474422215 wait complete. 2024-09-17T00:16:20.916 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 34 2024-09-17T00:16:21.238 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:21 smithi042 ceph-mon[111056]: pgmap v744: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:21.239 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:21.239 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:21 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3474422215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-09-17T00:16:21.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:21 smithi043 ceph-mon[81510]: pgmap v744: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:21.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:21.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:21 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3474422215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-09-17T00:16:21.588 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 35 2024-09-17T00:16:21.798 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:23.177 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:23 smithi042 ceph-mon[111056]: pgmap v745: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:23.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:23 smithi043 ceph-mon[81510]: pgmap v745: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 -- 172.21.15.42:0/3455385839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 msgr2=0x7f21e00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/3455385839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00689e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009910 tx=0x7f21c8009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 -- 172.21.15.42:0/3455385839 shutdown_connections 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/3455385839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00689e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/3455385839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21e0106cd0 0x7f21e01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 -- 172.21.15.42:0/3455385839 >> 172.21.15.42:0/3455385839 conn(0x7f21e0075670 msgr2=0x7f21e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 -- 172.21.15.42:0/3455385839 shutdown_connections 2024-09-17T00:16:23.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.415+0000 7f21e4bc4700 1 -- 172.21.15.42:0/3455385839 wait complete. 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21e4bc4700 1 Processor -- start 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21e4bc4700 1 -- start start 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21e4bc4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21e4bc4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21e0106cd0 0x7f21e0100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21e4bc4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0100770 con 0x7f21e0068580 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21e4bc4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e01008b0 con 0x7f21e0106cd0 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21de59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21de59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40024/0 (socket says 172.21.15.42:40024) 2024-09-17T00:16:23.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.416+0000 7f21de59c700 1 -- 172.21.15.42:0/2401931133 learned_addr learned my addr 172.21.15.42:0/2401931133 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21d5d9b700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21e0106cd0 0x7f21e0100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21de59c700 1 -- 172.21.15.42:0/2401931133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21e0106cd0 msgr2=0x7f21e0100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21de59c700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21e0106cd0 0x7f21e0100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21de59c700 1 -- 172.21.15.42:0/2401931133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c80095f0 con 0x7f21e0068580 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21de59c700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00ffcf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f21d000d8d0 tx=0x7f21d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21d7fff700 1 -- 172.21.15.42:0/2401931133 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d0009940 con 0x7f21e0068580 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21d7fff700 1 -- 172.21.15.42:0/2401931133 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21d0010460 con 0x7f21e0068580 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e01a2a30 con 0x7f21e0068580 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21d7fff700 1 -- 172.21.15.42:0/2401931133 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d000f5f0 con 0x7f21e0068580 2024-09-17T00:16:23.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.417+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e01a2e40 con 0x7f21e0068580 2024-09-17T00:16:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.418+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e01091d0 con 0x7f21e0068580 2024-09-17T00:16:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.419+0000 7f21d7fff700 1 -- 172.21.15.42:0/2401931133 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f21d00105d0 con 0x7f21e0068580 2024-09-17T00:16:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.419+0000 7f21d7fff700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21cc06e680 0x7f21cc070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.419+0000 7f21d7fff700 1 -- 172.21.15.42:0/2401931133 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f21d008e570 con 0x7f21e0068580 2024-09-17T00:16:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.419+0000 7f21d5d9b700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21cc06e680 0x7f21cc070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:23.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.420+0000 7f21d5d9b700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21cc06e680 0x7f21cc070b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21c8005b40 tx=0x7f21c801a040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:23.422 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.424+0000 7f21d7fff700 1 -- 172.21.15.42:0/2401931133 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f21d0057790 con 0x7f21e0068580 2024-09-17T00:16:23.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.694+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f21e004f2e0 con 0x7f21e0068580 2024-09-17T00:16:23.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.694+0000 7f21d7fff700 1 -- 172.21.15.42:0/2401931133 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v36) v1 ==== 107+0+5274 (secure 0 0 0) 0x7f21d005ae50 con 0x7f21e0068580 2024-09-17T00:16:23.693 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-09-17T00:16:23.693 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":35,"btime":"2024-09-17T00:13:17:752665+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi043.pzcdzg","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6825/2567826967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2567826967},{"type":"v1","addr":"172.21.15.43:6825","nonce":2567826967}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44349,"name":"cephfs.smithi043.riwwhn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6827/3065998819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3065998819},{"type":"v1","addr":"172.21.15.43:6827","nonce":3065998819}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T23:46:07.055413+0000","modified":"2024-09-17T00:13:17.752661+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34388,"mds_1":34420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34388":{"gid":34388,"name":"cephfs.smithi042.ctckas","rank":0,"incarnation":31,"state":"up:active","state_seq":243,"addr":"172.21.15.42:6827/2046825391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":2046825391},{"type":"v1","addr":"172.21.15.42:6827","nonce":2046825391}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34420":{"gid":34420,"name":"cephfs.smithi042.anvvuy","rank":1,"incarnation":31,"state":"up:active","state_seq":239,"addr":"172.21.15.42:6829/4053198993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":4053198993},{"type":"v1","addr":"172.21.15.42:6829","nonce":4053198993}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34388,"qdb_cluster":[34388,34420]},"id":1}]} 2024-09-17T00:16:23.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21cc06e680 msgr2=0x7f21cc070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:23.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21cc06e680 0x7f21cc070b40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21c8005b40 tx=0x7f21c801a040 comp rx=0 tx=0).stop 2024-09-17T00:16:23.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 msgr2=0x7f21e00ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:23.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00ffcf0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f21d000d8d0 tx=0x7f21d000dc90 comp rx=0 tx=0).stop 2024-09-17T00:16:23.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 shutdown_connections 2024-09-17T00:16:23.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f21cc06e680 0x7f21cc070b40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:23.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21e0068580 0x7f21e00ffcf0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:23.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 --2- 172.21.15.42:0/2401931133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21e0106cd0 0x7f21e0100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:23.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 >> 172.21.15.42:0/2401931133 conn(0x7f21e0075670 msgr2=0x7f21e00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:23.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 shutdown_connections 2024-09-17T00:16:23.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:23.697+0000 7f21e4bc4700 1 -- 172.21.15.42:0/2401931133 wait complete. 2024-09-17T00:16:23.696 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 35 2024-09-17T00:16:24.318 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:24 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2401931133' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-09-17T00:16:24.386 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-17T00:16:24.396 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-17T00:16:24.397 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-17T00:16:24.397 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T00:16:24.416 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-17T00:16:24.416 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T00:16:24.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-17T00:16:24.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:24 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2401931133' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-09-17T00:16:24.724 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:25 smithi042 ceph-mon[111056]: pgmap v746: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:25.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:25 smithi043 ceph-mon[81510]: pgmap v746: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:26.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 -- 172.21.15.42:0/3934873814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c068580 msgr2=0x7f719c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:26.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 --2- 172.21.15.42:0/3934873814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c068580 0x7f719c0689e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f718c009910 tx=0x7f718c009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:26.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 -- 172.21.15.42:0/3934873814 shutdown_connections 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 --2- 172.21.15.42:0/3934873814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c068580 0x7f719c0689e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 --2- 172.21.15.42:0/3934873814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f719c106cd0 0x7f719c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 -- 172.21.15.42:0/3934873814 >> 172.21.15.42:0/3934873814 conn(0x7f719c075670 msgr2=0x7f719c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 -- 172.21.15.42:0/3934873814 shutdown_connections 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.466+0000 7f71a4773700 1 -- 172.21.15.42:0/3934873814 wait complete. 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.467+0000 7f71a4773700 1 Processor -- start 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.467+0000 7f71a4773700 1 -- start start 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a4773700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f719c068580 0x7f719c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a4773700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c106cd0 0x7f719c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a4773700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c100770 con 0x7f719c106cd0 2024-09-17T00:16:26.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a4773700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1008b0 con 0x7f719c068580 2024-09-17T00:16:26.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a1d0e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c106cd0 0x7f719c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:26.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a1d0e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c106cd0 0x7f719c100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40054/0 (socket says 172.21.15.42:40054) 2024-09-17T00:16:26.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a1d0e700 1 -- 172.21.15.42:0/2055153415 learned_addr learned my addr 172.21.15.42:0/2055153415 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:26.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a250f700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f719c068580 0x7f719c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:26.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a1d0e700 1 -- 172.21.15.42:0/2055153415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f719c068580 msgr2=0x7f719c0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:26.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a1d0e700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f719c068580 0x7f719c0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:26.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a1d0e700 1 -- 172.21.15.42:0/2055153415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c0095f0 con 0x7f719c106cd0 2024-09-17T00:16:26.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a250f700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f719c068580 0x7f719c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a1d0e700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c106cd0 0x7f719c100230 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f718c00b5c0 tx=0x7f718c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71937fe700 1 -- 172.21.15.42:0/2055153415 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f718c01d070 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71937fe700 1 -- 172.21.15.42:0/2055153415 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f718c00f460 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c100b30 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.468+0000 7f71937fe700 1 -- 172.21.15.42:0/2055153415 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f718c017610 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.469+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c1a2cb0 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.470+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c1091d0 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.470+0000 7f71937fe700 1 -- 172.21.15.42:0/2055153415 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f718c00f5d0 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.471+0000 7f71937fe700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f718806e680 0x7f7188070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.471+0000 7f71937fe700 1 -- 172.21.15.42:0/2055153415 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f718c08fc70 con 0x7f719c106cd0 2024-09-17T00:16:26.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.471+0000 7f71a250f700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f718806e680 0x7f7188070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:26.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.471+0000 7f71a250f700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f718806e680 0x7f7188070b40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f719800e300 tx=0x7f7198009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:26.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.475+0000 7f71937fe700 1 -- 172.21.15.42:0/2055153415 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f718c096080 con 0x7f719c106cd0 2024-09-17T00:16:26.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.699+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f719c04f2e0 con 0x7f719c106cd0 2024-09-17T00:16:26.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.699+0000 7f71937fe700 1 -- 172.21.15.42:0/2055153415 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v84) v1 ==== 81+0+2039 (secure 0 0 0) 0x7f718c058df0 con 0x7f719c106cd0 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2643542629 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/2310231514 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1894696933 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6829/36745564 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1983510372 2024-09-17T23:40:02.177376+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6828/36745564 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/585424460 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6824/3963651257 2024-09-17T23:57:12.183213+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1281900709 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6829/1702969563 2024-09-17T23:57:12.183200+0000 2024-09-17T00:16:26.697 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/3310822508 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/218133032 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2643542629 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/2685566758 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/2894556761 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3534532679 2024-09-17T23:40:02.177376+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1732693245 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/911608840 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/241787870 2024-09-17T23:41:09.064350+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3697585678 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2316861404 2024-09-17T23:41:09.064350+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/1866101450 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6825/3963651257 2024-09-17T23:57:12.183213+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3709834403 2024-09-17T23:40:02.177376+0000 2024-09-17T00:16:26.698 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/391799329 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4154448068 2024-09-17T23:41:09.064350+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/1562188999 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6826/2136666874 2024-09-17T23:46:23.642810+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6828/1702969563 2024-09-17T23:57:12.183200+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3842592412 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6827/2136666874 2024-09-17T23:46:23.642810+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2700108058 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:26.699 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3705571149 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f718806e680 msgr2=0x7f7188070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f718806e680 0x7f7188070b40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f719800e300 tx=0x7f7198009040 comp rx=0 tx=0).stop 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c106cd0 msgr2=0x7f719c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c106cd0 0x7f719c100230 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f718c00b5c0 tx=0x7f718c00be00 comp rx=0 tx=0).stop 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 shutdown_connections 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f718806e680 0x7f7188070b40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f719c068580 0x7f719c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 --2- 172.21.15.42:0/2055153415 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f719c106cd0 0x7f719c100230 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 >> 172.21.15.42:0/2055153415 conn(0x7f719c075670 msgr2=0x7f719c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:26.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 shutdown_connections 2024-09-17T00:16:26.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:26.701+0000 7f71a4773700 1 -- 172.21.15.42:0/2055153415 wait complete. 2024-09-17T00:16:26.701 INFO:teuthology.orchestra.run.smithi042.stderr:listed 35 entries 2024-09-17T00:16:27.436 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:27 smithi042 ceph-mon[111056]: pgmap v747: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:27.436 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:27 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/2055153415' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T00:16:27.460 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-09-17T00:16:27.460 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T00:16:27.480 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-17T00:16:27.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:27 smithi043 ceph-mon[81510]: pgmap v747: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:27.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:27 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/2055153415' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T00:16:27.728 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/mon.smithi042/config 2024-09-17T00:16:29.334 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:29 smithi042 ceph-mon[111056]: pgmap v748: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.580+0000 7f20f8d49700 1 -- 172.21.15.42:0/3782880609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 msgr2=0x7f20f4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.580+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3782880609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f4100180 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009910 tx=0x7f20e4009c20 comp rx=0 tx=0).stop 2024-09-17T00:16:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.580+0000 7f20f8d49700 1 -- 172.21.15.42:0/3782880609 shutdown_connections 2024-09-17T00:16:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.580+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3782880609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f4100750 0x7f20f41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.580+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3782880609 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f4100180 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.580+0000 7f20f8d49700 1 -- 172.21.15.42:0/3782880609 >> 172.21.15.42:0/3782880609 conn(0x7f20f40fb650 msgr2=0x7f20f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.581+0000 7f20f8d49700 1 -- 172.21.15.42:0/3782880609 shutdown_connections 2024-09-17T00:16:29.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.581+0000 7f20f8d49700 1 -- 172.21.15.42:0/3782880609 wait complete. 2024-09-17T00:16:29.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.581+0000 7f20f8d49700 1 Processor -- start 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.581+0000 7f20f8d49700 1 -- start start 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f8d49700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f8d49700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f4100750 0x7f20f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f8d49700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f419d2b0 con 0x7f20f40ffda0 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f8d49700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f4196760 con 0x7f20f4100750 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40076/0 (socket says 172.21.15.42:40076) 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f259c700 1 -- 172.21.15.42:0/3838892514 learned_addr learned my addr 172.21.15.42:0/3838892514 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f1d9b700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f4100750 0x7f20f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f259c700 1 -- 172.21.15.42:0/3838892514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f4100750 msgr2=0x7f20f419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f259c700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f4100750 0x7f20f419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.582+0000 7f20f259c700 1 -- 172.21.15.42:0/3838892514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40095f0 con 0x7f20f40ffda0 2024-09-17T00:16:29.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.583+0000 7f20f259c700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f419c6e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b5c0 tx=0x7f20e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.583+0000 7f20eb7fe700 1 -- 172.21.15.42:0/3838892514 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e401d070 con 0x7f20f40ffda0 2024-09-17T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.583+0000 7f20eb7fe700 1 -- 172.21.15.42:0/3838892514 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20e4005e00 con 0x7f20f40ffda0 2024-09-17T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.583+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f41969e0 con 0x7f20f40ffda0 2024-09-17T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.583+0000 7f20eb7fe700 1 -- 172.21.15.42:0/3838892514 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e4017400 con 0x7f20f40ffda0 2024-09-17T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.583+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f4196ed0 con 0x7f20f40ffda0 2024-09-17T00:16:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.584+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f404f2e0 con 0x7f20f40ffda0 2024-09-17T00:16:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.584+0000 7f20eb7fe700 1 -- 172.21.15.42:0/3838892514 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 38) v1 ==== 92551+0+0 (secure 0 0 0) 0x7f20e4003a60 con 0x7f20f40ffda0 2024-09-17T00:16:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.585+0000 7f20eb7fe700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f20e006e680 0x7f20e0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T00:16:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.585+0000 7f20eb7fe700 1 -- 172.21.15.42:0/3838892514 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f20e408fa60 con 0x7f20f40ffda0 2024-09-17T00:16:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.585+0000 7f20f1d9b700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f20e006e680 0x7f20e0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T00:16:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.585+0000 7f20f1d9b700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f20e006e680 0x7f20e0070b40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f20f4197bd0 tx=0x7f20dc0073d0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T00:16:29.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.589+0000 7f20eb7fe700 1 -- 172.21.15.42:0/3838892514 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f20e405c7b0 con 0x7f20f40ffda0 2024-09-17T00:16:29.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:29 smithi043 ceph-mon[81510]: pgmap v748: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.812+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20f4197140 con 0x7f20f40ffda0 2024-09-17T00:16:29.810 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.813+0000 7f20eb7fe700 1 -- 172.21.15.42:0/3838892514 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v84) v1 ==== 81+0+2039 (secure 0 0 0) 0x7f20e405c340 con 0x7f20f40ffda0 2024-09-17T00:16:29.810 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2643542629 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:29.810 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/2310231514 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1894696933 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6829/36745564 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1983510372 2024-09-17T23:40:02.177376+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6828/36745564 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/585424460 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6824/3963651257 2024-09-17T23:57:12.183213+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1281900709 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6829/1702969563 2024-09-17T23:57:12.183200+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/3310822508 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/218133032 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2643542629 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:29.811 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/2685566758 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/2894556761 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3534532679 2024-09-17T23:40:02.177376+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1732693245 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/911608840 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/241787870 2024-09-17T23:41:09.064350+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3697585678 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2 2024-09-17T23:39:29.178241+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2316861404 2024-09-17T23:41:09.064350+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/1866101450 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:6825/3963651257 2024-09-17T23:57:12.183213+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3709834403 2024-09-17T23:40:02.177376+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/391799329 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4154448068 2024-09-17T23:41:09.064350+0000 2024-09-17T00:16:29.812 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.43:0/1562188999 2024-09-17T23:49:54.568952+0000 2024-09-17T00:16:29.813 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6826/2136666874 2024-09-17T23:46:23.642810+0000 2024-09-17T00:16:29.813 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6828/1702969563 2024-09-17T23:57:12.183200+0000 2024-09-17T00:16:29.813 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3842592412 2024-09-17T23:48:47.629446+0000 2024-09-17T00:16:29.813 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6827/2136666874 2024-09-17T23:46:23.642810+0000 2024-09-17T00:16:29.813 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2700108058 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:29.813 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3705571149 2024-09-17T23:52:50.564332+0000 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f20e006e680 msgr2=0x7f20e0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f20e006e680 0x7f20e0070b40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f20f4197bd0 tx=0x7f20dc0073d0 comp rx=0 tx=0).stop 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 msgr2=0x7f20f419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f419c6e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b5c0 tx=0x7f20e400bd70 comp rx=0 tx=0).stop 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 shutdown_connections 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:6800/970153898,v1:172.21.15.42:6801/970153898] conn(0x7f20e006e680 0x7f20e0070b40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20f40ffda0 0x7f20f419c6e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 --2- 172.21.15.42:0/3838892514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f4100750 0x7f20f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 >> 172.21.15.42:0/3838892514 conn(0x7f20f40fb650 msgr2=0x7f20f4103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T00:16:29.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 shutdown_connections 2024-09-17T00:16:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-09-17T00:16:29.815+0000 7f20f8d49700 1 -- 172.21.15.42:0/3838892514 wait complete. 2024-09-17T00:16:29.815 INFO:teuthology.orchestra.run.smithi042.stderr:listed 35 entries 2024-09-17T00:16:30.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:30 smithi042 ceph-mon[111056]: from='client.? 172.21.15.42:0/3838892514' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T00:16:30.600 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2024-09-17T00:16:30.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T00:16:30.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T00:16:30.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:30 smithi043 ceph-mon[81510]: from='client.? 172.21.15.42:0/3838892514' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-17T00:16:30.631 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T00:16:31.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:31 smithi042 ceph-mon[111056]: pgmap v749: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:31.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:31 smithi043 ceph-mon[81510]: pgmap v749: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:33 smithi042 ceph-mon[111056]: pgmap v750: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:33.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:33 smithi043 ceph-mon[81510]: pgmap v750: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:35.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:35 smithi042 ceph-mon[111056]: pgmap v751: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:35.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:16:35.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:16:35.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:16:35.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:16:35.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:35 smithi043 ceph-mon[81510]: pgmap v751: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:16:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:16:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:16:35.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:16:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:36.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:37.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:37 smithi042 ceph-mon[111056]: pgmap v752: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:37.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:37 smithi043 ceph-mon[81510]: pgmap v752: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:39.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:39 smithi042 ceph-mon[111056]: pgmap v753: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:39.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:39 smithi043 ceph-mon[81510]: pgmap v753: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:41.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:41 smithi042 ceph-mon[111056]: pgmap v754: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:41.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:41 smithi043 ceph-mon[81510]: pgmap v754: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:43.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:43 smithi042 ceph-mon[111056]: pgmap v755: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:43.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:43 smithi043 ceph-mon[81510]: pgmap v755: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:45.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:45 smithi042 ceph-mon[111056]: pgmap v756: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:45.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:45 smithi043 ceph-mon[81510]: pgmap v756: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:47.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:47 smithi042 ceph-mon[111056]: pgmap v757: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:47.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:47 smithi043 ceph-mon[81510]: pgmap v757: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:49.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:49 smithi042 ceph-mon[111056]: pgmap v758: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:50.027 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:49 smithi043 ceph-mon[81510]: pgmap v758: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:51.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:51 smithi042 ceph-mon[111056]: pgmap v759: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:51.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:51.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:51 smithi043 ceph-mon[81510]: pgmap v759: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:51.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:16:53.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:53 smithi042 ceph-mon[111056]: pgmap v760: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:53.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:53 smithi043 ceph-mon[81510]: pgmap v760: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:55.440 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:55 smithi042 ceph-mon[111056]: pgmap v761: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:55.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:55 smithi043 ceph-mon[81510]: pgmap v761: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:57.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:57 smithi042 ceph-mon[111056]: pgmap v762: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:57.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:57 smithi043 ceph-mon[81510]: pgmap v762: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:59.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:16:59 smithi042 ceph-mon[111056]: pgmap v763: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:16:59.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:16:59 smithi043 ceph-mon[81510]: pgmap v763: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:01.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:01 smithi042 ceph-mon[111056]: pgmap v764: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:01.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:01 smithi043 ceph-mon[81510]: pgmap v764: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:03.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:03 smithi042 ceph-mon[111056]: pgmap v765: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:03.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:03 smithi043 ceph-mon[81510]: pgmap v765: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:05.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:05 smithi042 ceph-mon[111056]: pgmap v766: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:05.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:05 smithi043 ceph-mon[81510]: pgmap v766: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:06.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:06.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:07.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:07 smithi042 ceph-mon[111056]: pgmap v767: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:07.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:07 smithi043 ceph-mon[81510]: pgmap v767: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:09.495 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:09 smithi043 ceph-mon[81510]: pgmap v768: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:09.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:09 smithi042 ceph-mon[111056]: pgmap v768: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:11.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:11 smithi042 ceph-mon[111056]: pgmap v769: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:11.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:11 smithi043 ceph-mon[81510]: pgmap v769: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:13.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:13 smithi042 ceph-mon[111056]: pgmap v770: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:13.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:13 smithi043 ceph-mon[81510]: pgmap v770: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:15.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:15 smithi042 ceph-mon[111056]: pgmap v771: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:15.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:15 smithi043 ceph-mon[81510]: pgmap v771: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:17.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:17 smithi042 ceph-mon[111056]: pgmap v772: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:17.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:17 smithi043 ceph-mon[81510]: pgmap v772: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:19.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:19 smithi042 ceph-mon[111056]: pgmap v773: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:19.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:19 smithi043 ceph-mon[81510]: pgmap v773: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:21.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:21 smithi042 ceph-mon[111056]: pgmap v774: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:21.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:21.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:21 smithi043 ceph-mon[81510]: pgmap v774: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:21.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:23.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:23 smithi042 ceph-mon[111056]: pgmap v775: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:23.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:23 smithi043 ceph-mon[81510]: pgmap v775: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:25 smithi042 ceph-mon[111056]: pgmap v776: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:25.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:25 smithi043 ceph-mon[81510]: pgmap v776: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:27.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:27 smithi042 ceph-mon[111056]: pgmap v777: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:27.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:27 smithi043 ceph-mon[81510]: pgmap v777: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:29.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:29 smithi042 ceph-mon[111056]: pgmap v778: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:29.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:29 smithi043 ceph-mon[81510]: pgmap v778: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:31.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:31 smithi042 ceph-mon[111056]: pgmap v779: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:31.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:31 smithi043 ceph-mon[81510]: pgmap v779: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:33.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:33 smithi042 ceph-mon[111056]: pgmap v780: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:33.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:33 smithi043 ceph-mon[81510]: pgmap v780: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:35 smithi042 ceph-mon[111056]: pgmap v781: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:35.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:17:35.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:35 smithi043 ceph-mon[81510]: pgmap v781: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:35.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:17:36.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:17:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:17:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:17:36.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:36.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:17:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:17:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:17:36.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:37.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:37 smithi043 ceph-mon[81510]: pgmap v782: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:37 smithi042 ceph-mon[111056]: pgmap v782: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:39.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:39 smithi043 ceph-mon[81510]: pgmap v783: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:39 smithi042 ceph-mon[111056]: pgmap v783: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:41.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:41 smithi043 ceph-mon[81510]: pgmap v784: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:42.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:41 smithi042 ceph-mon[111056]: pgmap v784: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:43.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:43 smithi043 ceph-mon[81510]: pgmap v785: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:44.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:43 smithi042 ceph-mon[111056]: pgmap v785: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:45.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:45 smithi043 ceph-mon[81510]: pgmap v786: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:46.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:45 smithi042 ceph-mon[111056]: pgmap v786: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:47.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:47 smithi043 ceph-mon[81510]: pgmap v787: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:48.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:47 smithi042 ceph-mon[111056]: pgmap v787: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:49.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:49 smithi043 ceph-mon[81510]: pgmap v788: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:50.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:49 smithi042 ceph-mon[111056]: pgmap v788: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:51.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:51 smithi043 ceph-mon[81510]: pgmap v789: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:51.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:52.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:51 smithi042 ceph-mon[111056]: pgmap v789: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:52.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:17:53.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:53 smithi043 ceph-mon[81510]: pgmap v790: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:54.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:53 smithi042 ceph-mon[111056]: pgmap v790: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:55.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:55 smithi042 ceph-mon[111056]: pgmap v791: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:55.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:55 smithi043 ceph-mon[81510]: pgmap v791: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:57.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:57 smithi043 ceph-mon[81510]: pgmap v792: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:58.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:57 smithi042 ceph-mon[111056]: pgmap v792: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:17:59.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:17:59 smithi043 ceph-mon[81510]: pgmap v793: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:00.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:17:59 smithi042 ceph-mon[111056]: pgmap v793: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:01.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:01 smithi043 ceph-mon[81510]: pgmap v794: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:01 smithi042 ceph-mon[111056]: pgmap v794: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:03.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:03 smithi043 ceph-mon[81510]: pgmap v795: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:03 smithi042 ceph-mon[111056]: pgmap v795: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:05.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:05 smithi043 ceph-mon[81510]: pgmap v796: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:06.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:05 smithi042 ceph-mon[111056]: pgmap v796: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:06.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:07.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:07.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:07 smithi043 ceph-mon[81510]: pgmap v797: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:08.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:07 smithi042 ceph-mon[111056]: pgmap v797: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:09.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:09 smithi043 ceph-mon[81510]: pgmap v798: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:09 smithi042 ceph-mon[111056]: pgmap v798: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:12.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:11 smithi042 ceph-mon[111056]: pgmap v799: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:12.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:11 smithi043 ceph-mon[81510]: pgmap v799: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:14.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:13 smithi042 ceph-mon[111056]: pgmap v800: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:14.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:13 smithi043 ceph-mon[81510]: pgmap v800: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:15.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:15 smithi043 ceph-mon[81510]: pgmap v801: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:16.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:15 smithi042 ceph-mon[111056]: pgmap v801: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:18.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:17 smithi042 ceph-mon[111056]: pgmap v802: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:18.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:17 smithi043 ceph-mon[81510]: pgmap v802: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:20.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:19 smithi042 ceph-mon[111056]: pgmap v803: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:20.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:19 smithi043 ceph-mon[81510]: pgmap v803: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:21 smithi042 ceph-mon[111056]: pgmap v804: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:22.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:21 smithi043 ceph-mon[81510]: pgmap v804: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:22.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:24.024 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:23 smithi042 ceph-mon[111056]: pgmap v805: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:24.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:23 smithi043 ceph-mon[81510]: pgmap v805: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:26.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:25 smithi042 ceph-mon[111056]: pgmap v806: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:26.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:25 smithi043 ceph-mon[81510]: pgmap v806: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:28.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:27 smithi042 ceph-mon[111056]: pgmap v807: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:27 smithi043 ceph-mon[81510]: pgmap v807: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:30.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:29 smithi042 ceph-mon[111056]: pgmap v808: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:30.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:29 smithi043 ceph-mon[81510]: pgmap v808: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:31 smithi042 ceph-mon[111056]: pgmap v809: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:32.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:31 smithi043 ceph-mon[81510]: pgmap v809: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:34.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:33 smithi042 ceph-mon[111056]: pgmap v810: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:34.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:33 smithi043 ceph-mon[81510]: pgmap v810: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:36.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:35 smithi042 ceph-mon[111056]: pgmap v811: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:36.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:36.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:35 smithi043 ceph-mon[81510]: pgmap v811: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:36.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:18:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:18:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:18:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:18:37.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:18:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:18:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:18:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:18:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:37 smithi042 ceph-mon[111056]: pgmap v812: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:38.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:37 smithi043 ceph-mon[81510]: pgmap v812: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:39 smithi042 ceph-mon[111056]: pgmap v813: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:40.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:39 smithi043 ceph-mon[81510]: pgmap v813: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:42.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:41 smithi042 ceph-mon[111056]: pgmap v814: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:42.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:41 smithi043 ceph-mon[81510]: pgmap v814: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:44.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:43 smithi042 ceph-mon[111056]: pgmap v815: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:44.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:43 smithi043 ceph-mon[81510]: pgmap v815: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:45.987 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:45 smithi043 ceph-mon[81510]: pgmap v816: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:46.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:45 smithi042 ceph-mon[111056]: pgmap v816: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:48.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:47 smithi042 ceph-mon[111056]: pgmap v817: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:48.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:47 smithi043 ceph-mon[81510]: pgmap v817: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:50.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:49 smithi042 ceph-mon[111056]: pgmap v818: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:50.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:49 smithi043 ceph-mon[81510]: pgmap v818: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:51.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:51.065 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:18:51.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:51 smithi042 ceph-mon[111056]: pgmap v819: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:52.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:51 smithi043 ceph-mon[81510]: pgmap v819: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:54.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:53 smithi042 ceph-mon[111056]: pgmap v820: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:54.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:53 smithi043 ceph-mon[81510]: pgmap v820: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:56.025 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:55 smithi042 ceph-mon[111056]: pgmap v821: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:56.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:55 smithi043 ceph-mon[81510]: pgmap v821: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:58.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:57 smithi042 ceph-mon[111056]: pgmap v822: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:18:58.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:57 smithi043 ceph-mon[81510]: pgmap v822: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:00.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:18:59 smithi042 ceph-mon[111056]: pgmap v823: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:18:59 smithi043 ceph-mon[81510]: pgmap v823: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:02.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:01 smithi042 ceph-mon[111056]: pgmap v824: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:02.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:01 smithi043 ceph-mon[81510]: pgmap v824: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:04.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:03 smithi042 ceph-mon[111056]: pgmap v825: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:04.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:03 smithi043 ceph-mon[81510]: pgmap v825: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:06.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:05 smithi042 ceph-mon[111056]: pgmap v826: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:06.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:06.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:05 smithi043 ceph-mon[81510]: pgmap v826: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:08.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:07 smithi042 ceph-mon[111056]: pgmap v827: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:08.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:07 smithi043 ceph-mon[81510]: pgmap v827: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:10.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:09 smithi042 ceph-mon[111056]: pgmap v828: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:10.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:09 smithi043 ceph-mon[81510]: pgmap v828: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:12.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:11 smithi042 ceph-mon[111056]: pgmap v829: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:12.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:11 smithi043 ceph-mon[81510]: pgmap v829: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:14.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:13 smithi042 ceph-mon[111056]: pgmap v830: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:14.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:13 smithi043 ceph-mon[81510]: pgmap v830: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:16.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:15 smithi042 ceph-mon[111056]: pgmap v831: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:15 smithi043 ceph-mon[81510]: pgmap v831: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:18.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:17 smithi042 ceph-mon[111056]: pgmap v832: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:18.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:17 smithi043 ceph-mon[81510]: pgmap v832: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:20.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:19 smithi042 ceph-mon[111056]: pgmap v833: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:20.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:19 smithi043 ceph-mon[81510]: pgmap v833: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:21.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:21.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:21.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:21 smithi043 ceph-mon[81510]: pgmap v834: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:22.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:21 smithi042 ceph-mon[111056]: pgmap v834: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:24.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:23 smithi042 ceph-mon[111056]: pgmap v835: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:24.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:23 smithi043 ceph-mon[81510]: pgmap v835: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:26.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:25 smithi042 ceph-mon[111056]: pgmap v836: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:26.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:25 smithi043 ceph-mon[81510]: pgmap v836: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:28.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:27 smithi042 ceph-mon[111056]: pgmap v837: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:28.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:27 smithi043 ceph-mon[81510]: pgmap v837: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:30.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:29 smithi042 ceph-mon[111056]: pgmap v838: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:30.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:29 smithi043 ceph-mon[81510]: pgmap v838: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:32.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:31 smithi042 ceph-mon[111056]: pgmap v839: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:32.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:31 smithi043 ceph-mon[81510]: pgmap v839: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:34.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:33 smithi042 ceph-mon[111056]: pgmap v840: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:34.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:33 smithi043 ceph-mon[81510]: pgmap v840: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:36.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:35 smithi042 ceph-mon[111056]: pgmap v841: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:36.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:35 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:36.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:35 smithi043 ceph-mon[81510]: pgmap v841: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:36.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:35 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:19:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:19:37.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:19:37.023 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:19:37.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:19:37.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:19:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:19:37.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:19:38.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:37 smithi042 ceph-mon[111056]: pgmap v842: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:38.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:37 smithi043 ceph-mon[81510]: pgmap v842: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:40.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:39 smithi042 ceph-mon[111056]: pgmap v843: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:40.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:39 smithi043 ceph-mon[81510]: pgmap v843: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:42.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:41 smithi043 ceph-mon[81510]: pgmap v844: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:42.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:41 smithi042 ceph-mon[111056]: pgmap v844: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:44.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:43 smithi043 ceph-mon[81510]: pgmap v845: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:44.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:43 smithi042 ceph-mon[111056]: pgmap v845: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:46.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:45 smithi043 ceph-mon[81510]: pgmap v846: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:46.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:45 smithi042 ceph-mon[111056]: pgmap v846: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:48.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:47 smithi042 ceph-mon[111056]: pgmap v847: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:48.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:47 smithi043 ceph-mon[81510]: pgmap v847: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:50.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:49 smithi043 ceph-mon[81510]: pgmap v848: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:50.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:49 smithi042 ceph-mon[111056]: pgmap v848: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:51.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:50 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:51.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:50 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:19:52.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:51 smithi043 ceph-mon[81510]: pgmap v849: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:52.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:51 smithi042 ceph-mon[111056]: pgmap v849: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:54.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:53 smithi043 ceph-mon[81510]: pgmap v850: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:54.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:53 smithi042 ceph-mon[111056]: pgmap v850: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:56.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:55 smithi043 ceph-mon[81510]: pgmap v851: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:56.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:55 smithi042 ceph-mon[111056]: pgmap v851: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:58.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:57 smithi043 ceph-mon[81510]: pgmap v852: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:19:58.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:57 smithi042 ceph-mon[111056]: pgmap v852: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:00.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:19:59 smithi043 ceph-mon[81510]: pgmap v853: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:00.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:19:59 smithi042 ceph-mon[111056]: pgmap v853: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:01.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:00 smithi043 ceph-mon[81510]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-17T00:20:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:00 smithi043 ceph-mon[81510]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:20:01.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:00 smithi043 ceph-mon[81510]: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:20:01.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:00 smithi042 ceph-mon[111056]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-17T00:20:01.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:00 smithi042 ceph-mon[111056]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-17T00:20:01.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:00 smithi042 ceph-mon[111056]: fs cephfs has deprecated feature inline_data enabled. 2024-09-17T00:20:02.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:01 smithi043 ceph-mon[81510]: pgmap v854: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:02.273 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:01 smithi042 ceph-mon[111056]: pgmap v854: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:04.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:03 smithi043 ceph-mon[81510]: pgmap v855: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:04.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:03 smithi042 ceph-mon[111056]: pgmap v855: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:06.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:05 smithi043 ceph-mon[81510]: pgmap v856: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:06.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:05 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:06.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:05 smithi042 ceph-mon[111056]: pgmap v856: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:06.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:05 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:08.128 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:07 smithi043 ceph-mon[81510]: pgmap v857: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:08.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:07 smithi042 ceph-mon[111056]: pgmap v857: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:10.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:09 smithi043 ceph-mon[81510]: pgmap v858: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:10.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:09 smithi042 ceph-mon[111056]: pgmap v858: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:12.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:11 smithi043 ceph-mon[81510]: pgmap v859: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:12.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:11 smithi042 ceph-mon[111056]: pgmap v859: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:14.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:13 smithi043 ceph-mon[81510]: pgmap v860: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:14.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:13 smithi042 ceph-mon[111056]: pgmap v860: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:16.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:15 smithi043 ceph-mon[81510]: pgmap v861: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:16.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:15 smithi042 ceph-mon[111056]: pgmap v861: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:18.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:17 smithi043 ceph-mon[81510]: pgmap v862: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:18.179 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:17 smithi042 ceph-mon[111056]: pgmap v862: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:20.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:19 smithi043 ceph-mon[81510]: pgmap v863: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:20.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:19 smithi042 ceph-mon[111056]: pgmap v863: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:21.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:20 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:21.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:20 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:22.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:21 smithi043 ceph-mon[81510]: pgmap v864: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:22.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:21 smithi042 ceph-mon[111056]: pgmap v864: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:23.127 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:22 smithi043 ceph-mon[81510]: pgmap v865: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:23.272 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:22 smithi042 ceph-mon[111056]: pgmap v865: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:25.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:25 smithi043 ceph-mon[81510]: pgmap v866: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:25.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:25 smithi042 ceph-mon[111056]: pgmap v866: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:27.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:27 smithi043 ceph-mon[81510]: pgmap v867: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:27.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:27 smithi042 ceph-mon[111056]: pgmap v867: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:29.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:29 smithi043 ceph-mon[81510]: pgmap v868: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:29.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:29 smithi042 ceph-mon[111056]: pgmap v868: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:31.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:31 smithi043 ceph-mon[81510]: pgmap v869: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:31.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:31 smithi042 ceph-mon[111056]: pgmap v869: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:33.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:33 smithi043 ceph-mon[81510]: pgmap v870: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:33.442 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:33 smithi042 ceph-mon[111056]: pgmap v870: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:35.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:35 smithi043 ceph-mon[81510]: pgmap v871: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:35.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:35 smithi042 ceph-mon[111056]: pgmap v871: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:36.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:36 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:36.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:36 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:37.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:37 smithi043 ceph-mon[81510]: pgmap v872: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:37.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:37 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:20:37.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:37 smithi042 ceph-mon[111056]: pgmap v872: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:37.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:37 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-17T00:20:38.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:20:38.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:20:38.378 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:38 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:20:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-17T00:20:38.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-17T00:20:38.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:38 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' 2024-09-17T00:20:39.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:39 smithi042 ceph-mon[111056]: pgmap v873: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:39.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:39 smithi043 ceph-mon[81510]: pgmap v873: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:41.524 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:41 smithi042 ceph-mon[111056]: pgmap v874: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:41.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:41 smithi043 ceph-mon[81510]: pgmap v874: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:43.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:43 smithi042 ceph-mon[111056]: pgmap v875: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:43.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:43 smithi043 ceph-mon[81510]: pgmap v875: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:45.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:45 smithi042 ceph-mon[111056]: pgmap v876: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:45.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:45 smithi043 ceph-mon[81510]: pgmap v876: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:47.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:47 smithi042 ceph-mon[111056]: pgmap v877: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:47.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:47 smithi043 ceph-mon[81510]: pgmap v877: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:49.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:49 smithi042 ceph-mon[111056]: pgmap v878: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:49.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:49 smithi043 ceph-mon[81510]: pgmap v878: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:51.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:51 smithi042 ceph-mon[111056]: pgmap v879: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:51.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:51 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:51.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:51 smithi043 ceph-mon[81510]: pgmap v879: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:51.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:51 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:20:53.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:53 smithi042 ceph-mon[111056]: pgmap v880: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:53.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:53 smithi043 ceph-mon[81510]: pgmap v880: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:55.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:55 smithi042 ceph-mon[111056]: pgmap v881: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:55.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:55 smithi043 ceph-mon[81510]: pgmap v881: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:57.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:57 smithi042 ceph-mon[111056]: pgmap v882: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:57.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:57 smithi043 ceph-mon[81510]: pgmap v882: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:59.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:20:59 smithi042 ceph-mon[111056]: pgmap v883: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:20:59.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:20:59 smithi043 ceph-mon[81510]: pgmap v883: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:01.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:01 smithi042 ceph-mon[111056]: pgmap v884: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:01.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:01 smithi043 ceph-mon[81510]: pgmap v884: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:03.377 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:03 smithi043 ceph-mon[81510]: pgmap v885: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:03.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:03 smithi042 ceph-mon[111056]: pgmap v885: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:05.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:05 smithi043 ceph-mon[81510]: pgmap v886: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:05.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:05 smithi042 ceph-mon[111056]: pgmap v886: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:06.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:06 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:21:06.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:06 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:21:07.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:07 smithi043 ceph-mon[81510]: pgmap v887: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:07.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:07 smithi042 ceph-mon[111056]: pgmap v887: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:09.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:09 smithi042 ceph-mon[111056]: pgmap v888: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:09.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:09 smithi043 ceph-mon[81510]: pgmap v888: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:11.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:11 smithi043 ceph-mon[81510]: pgmap v889: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:11.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:11 smithi042 ceph-mon[111056]: pgmap v889: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:13.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:13 smithi043 ceph-mon[81510]: pgmap v890: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:13.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:13 smithi042 ceph-mon[111056]: pgmap v890: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:15.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:15 smithi043 ceph-mon[81510]: pgmap v891: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:15.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:15 smithi042 ceph-mon[111056]: pgmap v891: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:17.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:17 smithi043 ceph-mon[81510]: pgmap v892: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:17.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:17 smithi042 ceph-mon[111056]: pgmap v892: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:19.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:19 smithi043 ceph-mon[81510]: pgmap v893: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:19.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:19 smithi042 ceph-mon[111056]: pgmap v893: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:21.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:21 smithi043 ceph-mon[81510]: pgmap v894: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:21.628 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:21 smithi043 ceph-mon[81510]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:21:21.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:21 smithi042 ceph-mon[111056]: pgmap v894: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:21.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:21 smithi042 ceph-mon[111056]: from='mgr.34102 172.21.15.42:0/4215583150' entity='mgr.smithi042.zrkumz' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-17T00:21:23.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:23 smithi043 ceph-mon[81510]: pgmap v895: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:23.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:23 smithi042 ceph-mon[111056]: pgmap v895: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:25.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:25 smithi043 ceph-mon[81510]: pgmap v896: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:25.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:25 smithi042 ceph-mon[111056]: pgmap v896: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:27.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:27 smithi043 ceph-mon[81510]: pgmap v897: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:27.772 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:27 smithi042 ceph-mon[111056]: pgmap v897: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:29.627 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:29 smithi043 ceph-mon[81510]: pgmap v898: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:29.705 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-09-17T00:21:29.706 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-17T00:21:29.708 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-17T00:21:29.728 INFO:tasks.cephadm:Teardown begin 2024-09-17T00:21:29.729 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/cephadm.py", line 2236, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-17T00:21:29.731 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T00:21:29.756 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:29 smithi042 ceph-mon[111056]: pgmap v898: 65 pgs: 65 active+clean; 319 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-17T00:21:29.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T00:21:29.797 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-17T00:21:29.797 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T00:21:29.819 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-17T00:21:29.856 INFO:tasks.cephadm:Stopping all daemons... 2024-09-17T00:21:29.856 INFO:tasks.cephadm.mon.smithi042:Stopping mon.smithi042... 2024-09-17T00:21:29.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042 2024-09-17T00:21:30.022 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:29 smithi042 systemd[1]: Stopping Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:21:30.522 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:30 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:21:30.193+0000 7fa27b879640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi042 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:21:30.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:30 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042[111052]: 2024-09-17T00:21:30.193+0000 7fa27b879640 -1 mon.smithi042@0(leader) e3 *** Got Signal Terminated *** 2024-09-17T00:21:30.523 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:30 smithi042 podman[163982]: 2024-09-17 00:21:30.224062399 +0000 UTC m=+0.161862878 container died 7ace5d09425f790362c026295ed17ada57fa3cb56be7dbe2da6f3f77cbc73e61 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T00:21:30.916 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:30 smithi042 podman[163982]: 2024-09-17 00:21:30.7350276 +0000 UTC m=+0.672828072 container cleanup 7ace5d09425f790362c026295ed17ada57fa3cb56be7dbe2da6f3f77cbc73e61 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True) 2024-09-17T00:21:30.917 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:30 smithi042 bash[163982]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042 2024-09-17T00:21:31.157 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service' 2024-09-17T00:21:31.181 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:30 smithi042 podman[163994]: 2024-09-17 00:21:30.920080156 +0000 UTC m=+0.693407293 container remove 7ace5d09425f790362c026295ed17ada57fa3cb56be7dbe2da6f3f77cbc73e61 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi042, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-09-17T00:21:31.181 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:31 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service: Deactivated successfully. 2024-09-17T00:21:31.181 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:31 smithi042 systemd[1]: Stopped Ceph mon.smithi042 for c2e6f2e2-7484-11ef-bceb-c7b262605968. 2024-09-17T00:21:31.181 INFO:journalctl@ceph.mon.smithi042.smithi042.stdout:Sep 17 00:21:31 smithi042 systemd[1]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi042.service: Consumed 20.374s CPU time. 2024-09-17T00:21:31.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:21:31.213 INFO:tasks.cephadm.mon.smithi042:Stopped mon.smithi042 2024-09-17T00:21:31.213 INFO:tasks.cephadm.mon.smithi043:Stopping mon.smithi043... 2024-09-17T00:21:31.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi043 2024-09-17T00:21:31.527 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:31 smithi043 systemd[1]: Stopping Ceph mon.smithi043 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:21:31.877 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:31 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043[81506]: 2024-09-17T00:21:31.529+0000 7f0bb8a35640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi043 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:21:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:31 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043[81506]: 2024-09-17T00:21:31.529+0000 7f0bb8a35640 -1 mon.smithi043@1(peon) e3 *** Got Signal Terminated *** 2024-09-17T00:21:31.878 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:31 smithi043 podman[119074]: 2024-09-17 00:21:31.601385375 +0000 UTC m=+0.182469401 container died a860b22006ae274b3a4a87469809c1722e0306795e61b6af0bd95b5e1648ed7d (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-17T00:21:32.370 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:32 smithi043 podman[119074]: 2024-09-17 00:21:32.071861757 +0000 UTC m=+0.652945824 container cleanup a860b22006ae274b3a4a87469809c1722e0306795e61b6af0bd95b5e1648ed7d (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-17T00:21:32.370 INFO:journalctl@ceph.mon.smithi043.smithi043.stdout:Sep 17 00:21:32 smithi043 bash[119074]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-mon-smithi043 2024-09-17T00:21:32.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@mon.smithi043.service' 2024-09-17T00:21:32.552 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:21:32.553 INFO:tasks.cephadm.mon.smithi043:Stopped mon.smithi043 2024-09-17T00:21:32.553 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-17T00:21:32.553 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.0 2024-09-17T00:21:32.861 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:32 smithi042 systemd[1]: Stopping Ceph osd.0 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:21:33.272 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:32 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[122579]: 2024-09-17T00:21:32.864+0000 7fdfaac47640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:21:33.273 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:32 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[122579]: 2024-09-17T00:21:32.864+0000 7fdfaac47640 -1 osd.0 84 *** Got signal Terminated *** 2024-09-17T00:21:33.273 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:32 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0[122579]: 2024-09-17T00:21:32.864+0000 7fdfaac47640 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T00:21:38.607 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:38 smithi042 podman[164116]: 2024-09-17 00:21:38.32299218 +0000 UTC m=+5.571349937 container died e3e56c422ec57412a91435be900a0202932c04ea9a9351672ca10b276ea3591c (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-17T00:21:39.022 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:38 smithi042 podman[164116]: 2024-09-17 00:21:38.709184371 +0000 UTC m=+5.957542127 container cleanup e3e56c422ec57412a91435be900a0202932c04ea9a9351672ca10b276ea3591c (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-17T00:21:39.022 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:38 smithi042 bash[164116]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0 2024-09-17T00:21:39.490 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:39 smithi042 podman[164131]: 2024-09-17 00:21:39.102377971 +0000 UTC m=+0.773808860 container remove e3e56c422ec57412a91435be900a0202932c04ea9a9351672ca10b276ea3591c (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, GIT_CLEAN=True) 2024-09-17T00:21:39.770 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:39 smithi042 podman[164221]: 2024-09-17 00:21:39.397166209 +0000 UTC m=+0.016754485 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-17T00:21:39.770 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:39 smithi042 podman[164221]: 2024-09-17 00:21:39.503601843 +0000 UTC m=+0.123190123 container create 2dc3a06eaa5a8a9b685dcbcca672f70abc1b61401be1c879750911088a676b0f (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-17T00:21:39.770 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:39 smithi042 podman[164221]: 2024-09-17 00:21:39.773316759 +0000 UTC m=+0.392905042 container init 2dc3a06eaa5a8a9b685dcbcca672f70abc1b61401be1c879750911088a676b0f (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-17T00:21:40.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:39 smithi042 podman[164221]: 2024-09-17 00:21:39.777732617 +0000 UTC m=+0.397320893 container start 2dc3a06eaa5a8a9b685dcbcca672f70abc1b61401be1c879750911088a676b0f (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T00:21:40.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:39 smithi042 podman[164221]: 2024-09-17 00:21:39.881743638 +0000 UTC m=+0.501331928 container attach 2dc3a06eaa5a8a9b685dcbcca672f70abc1b61401be1c879750911088a676b0f (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-17T00:21:40.023 INFO:journalctl@ceph.osd.0.smithi042.stdout:Sep 17 00:21:39 smithi042 podman[164221]: 2024-09-17 00:21:39.974214822 +0000 UTC m=+0.593803101 container died 2dc3a06eaa5a8a9b685dcbcca672f70abc1b61401be1c879750911088a676b0f (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-17T00:21:40.585 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.0.service' 2024-09-17T00:21:40.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:21:40.631 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-17T00:21:40.631 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-17T00:21:40.631 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.1 2024-09-17T00:21:40.857 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:40 smithi042 systemd[1]: Stopping Ceph osd.1 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:21:41.272 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:40 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[127990]: 2024-09-17T00:21:40.947+0000 7f476e68f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:21:41.272 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:40 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[127990]: 2024-09-17T00:21:40.947+0000 7f476e68f640 -1 osd.1 84 *** Got signal Terminated *** 2024-09-17T00:21:41.272 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:40 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1[127990]: 2024-09-17T00:21:40.947+0000 7f476e68f640 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T00:21:46.764 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:46 smithi042 podman[164326]: 2024-09-17 00:21:46.362883821 +0000 UTC m=+5.536762871 container died c378ed398a4d3689c7f970096b32b7ea86724a4c5239b40f7a91157b338e16f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T00:21:47.022 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:46 smithi042 podman[164326]: 2024-09-17 00:21:46.865355192 +0000 UTC m=+6.039234242 container cleanup c378ed398a4d3689c7f970096b32b7ea86724a4c5239b40f7a91157b338e16f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-c2ddf97) 2024-09-17T00:21:47.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:46 smithi042 bash[164326]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1 2024-09-17T00:21:47.425 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:47 smithi042 podman[164352]: 2024-09-17 00:21:47.092062404 +0000 UTC m=+0.729120205 container remove c378ed398a4d3689c7f970096b32b7ea86724a4c5239b40f7a91157b338e16f3 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T00:21:47.701 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:47 smithi042 podman[164442]: 2024-09-17 00:21:47.331640186 +0000 UTC m=+0.016928213 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-17T00:21:47.701 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:47 smithi042 podman[164442]: 2024-09-17 00:21:47.451586883 +0000 UTC m=+0.136874936 container create daab08af30c04a455bf36b9db74c22922b690859947c1a575915a4459c57c767 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T00:21:47.701 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:47 smithi042 podman[164442]: 2024-09-17 00:21:47.704502846 +0000 UTC m=+0.389790868 container init daab08af30c04a455bf36b9db74c22922b690859947c1a575915a4459c57c767 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T00:21:48.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:47 smithi042 podman[164442]: 2024-09-17 00:21:47.707922302 +0000 UTC m=+0.393210312 container start daab08af30c04a455bf36b9db74c22922b690859947c1a575915a4459c57c767 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-17T00:21:48.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:47 smithi042 podman[164442]: 2024-09-17 00:21:47.787561524 +0000 UTC m=+0.472849531 container attach daab08af30c04a455bf36b9db74c22922b690859947c1a575915a4459c57c767 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, GIT_CLEAN=True) 2024-09-17T00:21:48.023 INFO:journalctl@ceph.osd.1.smithi042.stdout:Sep 17 00:21:47 smithi042 podman[164442]: 2024-09-17 00:21:47.901947828 +0000 UTC m=+0.587235849 container died daab08af30c04a455bf36b9db74c22922b690859947c1a575915a4459c57c767 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-17T00:21:48.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.1.service' 2024-09-17T00:21:48.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:21:48.607 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-17T00:21:48.608 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-17T00:21:48.608 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.2 2024-09-17T00:21:48.951 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:48 smithi042 systemd[1]: Stopping Ceph osd.2 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:21:49.272 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:48 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[133813]: 2024-09-17T00:21:48.953+0000 7f15fd44d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:21:49.272 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:48 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[133813]: 2024-09-17T00:21:48.953+0000 7f15fd44d640 -1 osd.2 84 *** Got signal Terminated *** 2024-09-17T00:21:49.272 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:48 smithi042 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2[133813]: 2024-09-17T00:21:48.953+0000 7f15fd44d640 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T00:21:54.522 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:54 smithi042 podman[164544]: 2024-09-17 00:21:54.273275368 +0000 UTC m=+5.422147716 container died 61db938df01dc87fb13df3e944225595a7c662a190bcd9640346027d2c568ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-17T00:21:54.885 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:54 smithi042 podman[164544]: 2024-09-17 00:21:54.628998902 +0000 UTC m=+5.777871252 container cleanup 61db938df01dc87fb13df3e944225595a7c662a190bcd9640346027d2c568ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97) 2024-09-17T00:21:54.885 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:54 smithi042 bash[164544]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2 2024-09-17T00:21:55.213 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:54 smithi042 podman[164560]: 2024-09-17 00:21:54.888771775 +0000 UTC m=+0.612612808 container remove 61db938df01dc87fb13df3e944225595a7c662a190bcd9640346027d2c568ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-c2ddf97, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-17T00:21:55.489 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:55 smithi042 podman[164652]: 2024-09-17 00:21:55.120475446 +0000 UTC m=+0.016934637 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-17T00:21:55.490 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:55 smithi042 podman[164652]: 2024-09-17 00:21:55.240295576 +0000 UTC m=+0.136754808 container create 34cf32d88fae627395a2e982b13e5fba36616b90f816aaf7f8c6ba9b2c502aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97) 2024-09-17T00:21:55.490 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:55 smithi042 podman[164652]: 2024-09-17 00:21:55.493047008 +0000 UTC m=+0.389506206 container init 34cf32d88fae627395a2e982b13e5fba36616b90f816aaf7f8c6ba9b2c502aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-17T00:21:55.772 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:55 smithi042 podman[164652]: 2024-09-17 00:21:55.496453876 +0000 UTC m=+0.392913073 container start 34cf32d88fae627395a2e982b13e5fba36616b90f816aaf7f8c6ba9b2c502aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-17T00:21:55.773 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:55 smithi042 podman[164652]: 2024-09-17 00:21:55.576342279 +0000 UTC m=+0.472801493 container attach 34cf32d88fae627395a2e982b13e5fba36616b90f816aaf7f8c6ba9b2c502aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T00:21:55.773 INFO:journalctl@ceph.osd.2.smithi042.stdout:Sep 17 00:21:55 smithi042 podman[164652]: 2024-09-17 00:21:55.687659795 +0000 UTC m=+0.584118986 container died 34cf32d88fae627395a2e982b13e5fba36616b90f816aaf7f8c6ba9b2c502aa0 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-17T00:21:56.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.2.service' 2024-09-17T00:21:56.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:21:56.315 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-17T00:21:56.315 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-17T00:21:56.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.3 2024-09-17T00:21:56.627 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:21:56 smithi043 systemd[1]: Stopping Ceph osd.3 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:21:57.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:21:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[91017]: 2024-09-17T00:21:56.642+0000 7f175d740640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:21:57.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:21:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[91017]: 2024-09-17T00:21:56.642+0000 7f175d740640 -1 osd.3 84 *** Got signal Terminated *** 2024-09-17T00:21:57.128 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:21:56 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3[91017]: 2024-09-17T00:21:56.642+0000 7f175d740640 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T00:21:59.878 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:21:59 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:21:59.423+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:00.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:00 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:00.375+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:01.473 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:01 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:01.372+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:01.473 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:01 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:01.176+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:02.315 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:02 smithi043 podman[119209]: 2024-09-17 00:22:02.040386306 +0000 UTC m=+5.514788465 container died 47a2dcb81243df0477c22e3d5a394cc996ff47e734eddcab40a6c4b75a5d4421 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-17T00:22:02.358 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:02.168+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:02.627 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:02 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:02.368+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:02.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:02 smithi043 podman[119209]: 2024-09-17 00:22:02.408316889 +0000 UTC m=+5.882719021 container cleanup 47a2dcb81243df0477c22e3d5a394cc996ff47e734eddcab40a6c4b75a5d4421 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, RELEASE=squid-c2ddf97, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-17T00:22:02.628 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:02 smithi043 bash[119209]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3 2024-09-17T00:22:03.127 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:02 smithi043 podman[119229]: 2024-09-17 00:22:02.826420294 +0000 UTC m=+0.780516141 container remove 47a2dcb81243df0477c22e3d5a394cc996ff47e734eddcab40a6c4b75a5d4421 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-17T00:22:03.510 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:03 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:03.138+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:03.511 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:03 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:03.321+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:03.511 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:03 smithi043 podman[119325]: 2024-09-17 00:22:03.087361621 +0000 UTC m=+0.016404771 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-17T00:22:03.511 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:03 smithi043 podman[119325]: 2024-09-17 00:22:03.202320211 +0000 UTC m=+0.131363393 container create 7523af84c239dd3907fb832d606849690d8dd3854821120c4366d43d85a12248 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-17T00:22:03.511 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:03 smithi043 podman[119325]: 2024-09-17 00:22:03.513899307 +0000 UTC m=+0.442942449 container init 7523af84c239dd3907fb832d606849690d8dd3854821120c4366d43d85a12248 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-17T00:22:03.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:03 smithi043 podman[119325]: 2024-09-17 00:22:03.517909138 +0000 UTC m=+0.446952280 container start 7523af84c239dd3907fb832d606849690d8dd3854821120c4366d43d85a12248 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-c2ddf97, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-17T00:22:03.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:03 smithi043 podman[119325]: 2024-09-17 00:22:03.60514802 +0000 UTC m=+0.534191173 container attach 7523af84c239dd3907fb832d606849690d8dd3854821120c4366d43d85a12248 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T00:22:03.878 INFO:journalctl@ceph.osd.3.smithi043.stdout:Sep 17 00:22:03 smithi043 podman[119325]: 2024-09-17 00:22:03.72018598 +0000 UTC m=+0.649229133 container died 7523af84c239dd3907fb832d606849690d8dd3854821120c4366d43d85a12248 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-17T00:22:04.268 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:04 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:04.180+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:04.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.3.service' 2024-09-17T00:22:04.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:22:04.462 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-17T00:22:04.462 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-17T00:22:04.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.4 2024-09-17T00:22:04.525 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:04 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:04.358+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:04.817 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:04 smithi043 systemd[1]: Stopping Ceph osd.4 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:22:05.127 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:04 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:04.819+0000 7f40141c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:22:05.127 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:04 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:04.819+0000 7f40141c5640 -1 osd.4 84 *** Got signal Terminated *** 2024-09-17T00:22:05.128 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:04 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:04.819+0000 7f40141c5640 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T00:22:05.627 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:05 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:05.161+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:05.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:05 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:05.357+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:06.627 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:06 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:06.190+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:06.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:06 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:06.357+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:07.627 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:07 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:07.368+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:07.628 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:07 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:07.234+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:08.627 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:08 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:08.245+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:08.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:08 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:08.401+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:09.378 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:09 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4[96206]: 2024-09-17T00:22:09.202+0000 7f400c7c2640 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:36.319947+0000 front 2024-09-17T00:21:36.320090+0000 (oldest deadline 2024-09-17T00:22:00.419752+0000) 2024-09-17T00:22:09.877 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:09 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:09.386+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:10.493 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:10 smithi043 podman[119431]: 2024-09-17 00:22:10.209176984 +0000 UTC m=+5.510300303 container died bd894ca62c9c93ce2774f04122a7f23083267c186ae000aa58b08064f0ec0be1 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-17T00:22:10.493 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:10 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:10.388+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:10.860 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:10 smithi043 podman[119431]: 2024-09-17 00:22:10.603873773 +0000 UTC m=+5.904997079 container cleanup bd894ca62c9c93ce2774f04122a7f23083267c186ae000aa58b08064f0ec0be1 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-17T00:22:10.860 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:10 smithi043 bash[119431]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4 2024-09-17T00:22:11.127 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:10 smithi043 podman[119457]: 2024-09-17 00:22:10.863678198 +0000 UTC m=+0.644066130 container remove bd894ca62c9c93ce2774f04122a7f23083267c186ae000aa58b08064f0ec0be1 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-17T00:22:11.513 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:11 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:11.409+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:11.513 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:11 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:11.409+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-09-17T00:21:45.822065+0000 front 2024-09-17T00:21:45.822049+0000 (oldest deadline 2024-09-17T00:22:11.121905+0000) 2024-09-17T00:22:11.514 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:11 smithi043 podman[119550]: 2024-09-17 00:22:11.089474226 +0000 UTC m=+0.016741704 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-17T00:22:11.514 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:11 smithi043 podman[119550]: 2024-09-17 00:22:11.19780311 +0000 UTC m=+0.125070577 container create 2deb4041c7f0306357020c5829844ce305fc51902ff213b05627e7a45b2499f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T00:22:11.514 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:11 smithi043 podman[119550]: 2024-09-17 00:22:11.425155764 +0000 UTC m=+0.352423237 container init 2deb4041c7f0306357020c5829844ce305fc51902ff213b05627e7a45b2499f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-17T00:22:11.514 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:11 smithi043 podman[119550]: 2024-09-17 00:22:11.428601245 +0000 UTC m=+0.355868713 container start 2deb4041c7f0306357020c5829844ce305fc51902ff213b05627e7a45b2499f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-c2ddf97, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T00:22:11.514 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:11 smithi043 podman[119550]: 2024-09-17 00:22:11.516726788 +0000 UTC m=+0.443994266 container attach 2deb4041c7f0306357020c5829844ce305fc51902ff213b05627e7a45b2499f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-17T00:22:11.877 INFO:journalctl@ceph.osd.4.smithi043.stdout:Sep 17 00:22:11 smithi043 podman[119550]: 2024-09-17 00:22:11.62313497 +0000 UTC m=+0.550402445 container died 2deb4041c7f0306357020c5829844ce305fc51902ff213b05627e7a45b2499f2 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-17T00:22:12.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.4.service' 2024-09-17T00:22:12.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:22:12.286 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-17T00:22:12.286 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-17T00:22:12.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.5 2024-09-17T00:22:12.620 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:12 smithi043 systemd[1]: Stopping Ceph osd.5 for c2e6f2e2-7484-11ef-bceb-c7b262605968... 2024-09-17T00:22:12.621 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:12 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:12.388+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:12.621 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:12 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:12.388+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-09-17T00:21:45.822065+0000 front 2024-09-17T00:21:45.822049+0000 (oldest deadline 2024-09-17T00:22:11.121905+0000) 2024-09-17T00:22:12.877 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:12 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:12.623+0000 7f14b815f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-17T00:22:12.877 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:12 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:12.623+0000 7f14b815f640 -1 osd.5 84 *** Got signal Terminated *** 2024-09-17T00:22:12.877 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:12 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:12.623+0000 7f14b815f640 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-17T00:22:13.877 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:13 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:13.392+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:13.878 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:13 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:13.392+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-09-17T00:21:45.822065+0000 front 2024-09-17T00:21:45.822049+0000 (oldest deadline 2024-09-17T00:22:11.121905+0000) 2024-09-17T00:22:14.877 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:14 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:14.382+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:14.878 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:14 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:14.382+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-09-17T00:21:45.822065+0000 front 2024-09-17T00:21:45.822049+0000 (oldest deadline 2024-09-17T00:22:11.121905+0000) 2024-09-17T00:22:15.627 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:15 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:15.366+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:15.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:15 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:15.366+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-09-17T00:21:45.822065+0000 front 2024-09-17T00:21:45.822049+0000 (oldest deadline 2024-09-17T00:22:11.121905+0000) 2024-09-17T00:22:16.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:16 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:16.320+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:16.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:16 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:16.320+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-09-17T00:21:45.822065+0000 front 2024-09-17T00:21:45.822049+0000 (oldest deadline 2024-09-17T00:22:11.121905+0000) 2024-09-17T00:22:17.627 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:17 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:17.364+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-09-17T00:21:35.921071+0000 front 2024-09-17T00:21:35.920977+0000 (oldest deadline 2024-09-17T00:21:59.420840+0000) 2024-09-17T00:22:17.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:17 smithi043 ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5[101533]: 2024-09-17T00:22:17.364+0000 7f14b075c640 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.42:6814 osd.1 since back 2024-09-17T00:21:45.822065+0000 front 2024-09-17T00:21:45.822049+0000 (oldest deadline 2024-09-17T00:22:11.121905+0000) 2024-09-17T00:22:18.262 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:18 smithi043 podman[119653]: 2024-09-17 00:22:18.007128771 +0000 UTC m=+5.491702482 container died dc14c8f3f72c5ef14a0af27c8deec68367633e606ccba4681c3ba2ae47cbcc71 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-c2ddf97, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-17T00:22:18.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:18 smithi043 podman[119653]: 2024-09-17 00:22:18.356232461 +0000 UTC m=+5.840806158 container cleanup dc14c8f3f72c5ef14a0af27c8deec68367633e606ccba4681c3ba2ae47cbcc71 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-c2ddf97, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-17T00:22:18.628 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:18 smithi043 bash[119653]: ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5 2024-09-17T00:22:18.963 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:18 smithi043 podman[119672]: 2024-09-17 00:22:18.640705166 +0000 UTC m=+0.628196242 container remove dc14c8f3f72c5ef14a0af27c8deec68367633e606ccba4681c3ba2ae47cbcc71 (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-17T00:22:19.249 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:18 smithi043 podman[119756]: 2024-09-17 00:22:18.966425206 +0000 UTC m=+0.113284937 container create 383db03f10b3030ac5b87c5d1204830ebc31c30f74526b8277ab6a5c306562da (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-c2ddf97, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-17T00:22:19.249 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:18 smithi043 podman[119756]: 2024-09-17 00:22:18.869311097 +0000 UTC m=+0.016170838 image pull ebb0615149def4a3179c82c0cc47af75808300abb9f1142788e5e605a7ac0e5b quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3 2024-09-17T00:22:19.249 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:19 smithi043 podman[119756]: 2024-09-17 00:22:19.160802684 +0000 UTC m=+0.307662422 container init 383db03f10b3030ac5b87c5d1204830ebc31c30f74526b8277ab6a5c306562da (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-c2ddf97, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-17T00:22:19.249 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:19 smithi043 podman[119756]: 2024-09-17 00:22:19.164492415 +0000 UTC m=+0.311352149 container start 383db03f10b3030ac5b87c5d1204830ebc31c30f74526b8277ab6a5c306562da (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-c2ddf97) 2024-09-17T00:22:19.249 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:19 smithi043 podman[119756]: 2024-09-17 00:22:19.252348742 +0000 UTC m=+0.399208486 container attach 383db03f10b3030ac5b87c5d1204830ebc31c30f74526b8277ab6a5c306562da (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-c2ddf97, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-17T00:22:19.610 INFO:journalctl@ceph.osd.5.smithi043.stdout:Sep 17 00:22:19 smithi043 podman[119780]: 2024-09-17 00:22:19.399415227 +0000 UTC m=+0.017070265 container died 383db03f10b3030ac5b87c5d1204830ebc31c30f74526b8277ab6a5c306562da (image=quay.ceph.io/ceph-ci/ceph@sha256:475e68c989240ebfb6ca26a687a713828159881ccf6edcf69b7320f1853af3e3, name=ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-c2ddf97, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-17T00:22:19.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c2e6f2e2-7484-11ef-bceb-c7b262605968@osd.5.service' 2024-09-17T00:22:19.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-17T00:22:19.940 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-17T00:22:19.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T00:22:20.103 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-17T00:22:24.513 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi042.stderr:ceph-fuse[75722]: fuse finished with error 0 and tester_r 0 2024-09-17T00:22:43.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-17T00:22:43.623 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-17T00:23:00.759 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T00:23:00.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-17T00:23:00.824 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-17T00:23:00.825 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880/remote/smithi042/crash 2024-09-17T00:23:00.826 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash -- . 2024-09-17T00:23:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T00:23:00.865 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2024-09-17T00:23:00.867 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880/remote/smithi043/crash 2024-09-17T00:23:00.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash -- . 2024-09-17T00:23:00.892 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-17T00:23:00.892 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-09-17T00:23:00.893 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-17T00:23:00.894 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-17T00:23:00.944 INFO:teuthology.orchestra.run.smithi042.stdout:2024-09-17T00:00:00.000210+0000 mon.smithi042 (mon.0) 688 : cluster [ERR] fs cephfs is degraded 2024-09-17T00:23:00.944 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-17T00:23:00.944 INFO:tasks.cephadm:Compressing logs... 2024-09-17T00:23:00.945 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T00:23:00.987 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T00:23:01.013 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mgr.smithi043.xveyou.log 2024-09-17T00:23:01.013 INFO:teuthology.orchestra.run.smithi043.stderr:: No such file or directory 2024-09-17T00:23:01.013 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi043.riwwhn.log 2024-09-17T00:23:01.014 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mgr.smithi043.xveyou.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi043.pzcdzg.log 2024-09-17T00:23:01.014 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi043.riwwhn.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-17T00:23:01.014 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi043.pzcdzg.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-17T00:23:01.017 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-17T00:23:01.017 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi043.log 2024-09-17T00:23:01.017 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-17T00:23:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-17T00:23:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:: No such file or directory 2024-09-17T00:23:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-17T00:23:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-17T00:23:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mgr.smithi042.zrkumz.log 2024-09-17T00:23:01.019 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-17T00:23:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mgr.smithi042.zrkumz.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi042.anvvuy.log 2024-09-17T00:23:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-17T00:23:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi042.anvvuy.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-17T00:23:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi042.ctckas.log 2024-09-17T00:23:01.020 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.2.log: 86.0% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-17T00:23:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi043.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-17T00:23:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.cephadm.log: 85.9% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-17T00:23:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.log 2024-09-17T00:23:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.5.log: 94.5% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi043.log.gz 2024-09-17T00:23:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mon.smithi043.log 2024-09-17T00:23:01.022 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.log: 91.5% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-17T00:23:01.023 INFO:teuthology.orchestra.run.smithi042.stderr: 91.4% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-17T00:23:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.log 2024-09-17T00:23:01.023 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi042.ctckas.log: gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi042.log 2024-09-17T00:23:01.024 INFO:teuthology.orchestra.run.smithi043.stderr: 94.4% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi043.riwwhn.log.gz 2024-09-17T00:23:01.024 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-17T00:23:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-17T00:23:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mon.smithi043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-09-17T00:23:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-17T00:23:01.027 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.log: 89.3% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-17T00:23:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.1.log: 91.0% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-17T00:23:01.033 INFO:teuthology.orchestra.run.smithi042.stderr: 91.1% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-17T00:23:01.033 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-17T00:23:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mon.smithi042.log 2024-09-17T00:23:01.034 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-17T00:23:01.038 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mon.smithi042.log: 94.4% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi042.log.gz 2024-09-17T00:23:01.039 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-17T00:23:01.066 INFO:teuthology.orchestra.run.smithi043.stderr: 89.8% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mgr.smithi043.xveyou.log.gz 2024-09-17T00:23:01.112 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-17T00:23:01.560 INFO:teuthology.orchestra.run.smithi042.stderr: 89.3% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mgr.smithi042.zrkumz.log.gz 2024-09-17T00:23:01.710 INFO:teuthology.orchestra.run.smithi043.stderr: 92.5% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mon.smithi043.log.gz 2024-09-17T00:23:02.892 INFO:teuthology.orchestra.run.smithi042.stderr: 90.2% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mon.smithi042.log.gz 2024-09-17T00:23:05.047 INFO:teuthology.orchestra.run.smithi042.stderr: 95.0% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi042.ctckas.log.gz 2024-09-17T00:23:05.107 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-17T00:23:05.950 INFO:teuthology.orchestra.run.smithi042.stderr: 93.6% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-17T00:23:06.038 INFO:teuthology.orchestra.run.smithi042.stderr: 93.7% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-17T00:23:06.664 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-17T00:23:07.240 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-17T00:23:07.429 INFO:teuthology.orchestra.run.smithi042.stderr: 93.7% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-17T00:23:15.853 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-09-17T00:23:15.853 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-09-17T00:23:17.165 INFO:teuthology.orchestra.run.smithi042.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-17T00:23:48.213 INFO:teuthology.orchestra.run.smithi043.stderr: 93.2% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi043.pzcdzg.log.gz 2024-09-17T00:23:48.214 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-09-17T00:23:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m47.212s 2024-09-17T00:23:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:user 1m16.425s 2024-09-17T00:23:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m2.179s 2024-09-17T00:23:52.771 INFO:teuthology.orchestra.run.smithi042.stderr: 93.3% -- replaced with /var/log/ceph/c2e6f2e2-7484-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi042.anvvuy.log.gz 2024-09-17T00:23:52.773 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-09-17T00:23:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m51.772s 2024-09-17T00:23:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:user 1m27.980s 2024-09-17T00:23:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m2.542s 2024-09-17T00:23:52.774 INFO:tasks.cephadm:Archiving logs... 2024-09-17T00:23:52.774 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880/remote/smithi042/log 2024-09-17T00:23:52.775 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T00:24:04.616 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880/remote/smithi043/log 2024-09-17T00:24:04.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T00:24:15.181 INFO:tasks.cephadm:Removing cluster... 2024-09-17T00:24:15.182 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 --force 2024-09-17T00:24:15.342 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-17T00:24:15.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c2e6f2e2-7484-11ef-bceb-c7b262605968 --force 2024-09-17T00:24:15.866 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: c2e6f2e2-7484-11ef-bceb-c7b262605968 2024-09-17T00:24:16.209 INFO:tasks.cephadm:Removing cephadm ... 2024-09-17T00:24:16.209 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T00:24:16.226 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-17T00:24:16.242 INFO:tasks.cephadm:Teardown complete 2024-09-17T00:24:16.243 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T00:24:16.254 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-17T00:24:16.255 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T00:24:16.255 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T00:24:16.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T00:24:16.360 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T00:24:16.370 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T00:24:16.371 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T00:24:16.373 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T00:24:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-09-17T00:24:16.388 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-09-17T00:24:16.392 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T00:24:16.392 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-09-17T00:24:16.392 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +3442us[+3448us] +/- 39ms 2024-09-17T00:24:16.392 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -4065us[-4059us] +/- 71ms 2024-09-17T00:24:16.392 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -2272us[-2266us] +/- 74ms 2024-09-17T00:24:16.392 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T00:24:16.394 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T00:24:16.394 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-09-17T00:24:16.394 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv01.front.sepia.ceph.com 3 6 377 29 +3429us[+3440us] +/- 39ms 2024-09-17T00:24:16.394 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -4063us[-4063us] +/- 71ms 2024-09-17T00:24:16.394 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -2295us[-2295us] +/- 74ms 2024-09-17T00:24:16.394 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T00:24:16.394 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T00:24:16.404 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T00:24:16.405 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T00:24:16.415 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T00:24:16.523 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T00:24:16.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T00:24:16.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T00:24:16.634 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T00:24:16.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T00:24:16.664 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T00:24:16.674 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T00:24:16.685 INFO:teuthology.task.internal:Duration was 3108.008185 seconds 2024-09-17T00:24:16.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T00:24:16.698 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T00:24:16.698 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T00:24:16.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T00:24:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T00:24:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T00:24:17.065 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T00:24:17.065 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-09-17T00:24:17.066 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T00:24:17.129 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-09-17T00:24:17.129 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T00:24:17.152 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T00:24:17.152 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T00:24:17.174 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T00:24:17.220 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T00:24:17.220 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T00:24:17.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T00:24:18.224 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T00:24:18.237 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T00:24:18.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T00:24:18.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T00:24:18.292 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T00:24:18.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T00:24:18.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T00:24:18.329 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-09-17T00:24:18.360 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-09-17T00:24:18.397 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T00:24:18.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T00:24:18.413 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T00:24:18.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T00:24:18.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T00:24:18.466 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T00:24:18.468 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880/remote/smithi042 2024-09-17T00:24:18.468 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T00:24:18.523 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-16_21:08:04-orch-squid-distro-default-smithi/7907880/remote/smithi043 2024-09-17T00:24:18.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T00:24:18.565 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T00:24:18.565 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T00:24:18.567 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T00:24:18.616 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T00:24:18.629 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T00:24:18.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T00:24:18.665 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T00:24:18.665 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T00:24:18.667 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T00:24:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: 262169 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 17 00:24 /home/ubuntu/cephtest 2024-09-17T00:24:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: 528330 4 d--------- 2 ubuntu ubuntu 4096 Sep 16 23:46 /home/ubuntu/cephtest/mnt.0 2024-09-17T00:24:18.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T00:24:18.682 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-09-17T00:24:18.682 INFO:teuthology.orchestra.run.smithi042.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-09-17T00:24:18.682 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_e20bff7d16f5b09c58ead9548e8cf98cf87ee227/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi042 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-09-17T00:24:18.683 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T00:24:18.694 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T00:24:18.706 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-17T00:24:18.709 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3108.0081849098206 failure_reason: reached maximum tries (50) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-09-17T00:24:18.709 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T00:24:18.794 INFO:teuthology.run:FAIL